2023-10-05T19:28:28.053 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-10-05T19:28:28.053 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --owner scheduled_yuriw@teuthology --name yuriw-2023-10-05_14:30:58-rgw-wip-yuri2-testing-2023-10-04-1333-quincy-distro-default-smithi --description rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated supported-random-distro$/{ubuntu_latest}} --archive /home/teuthworker/archive/yuriw-2023-10-05_14:30:58-rgw-wip-yuri2-testing-2023-10-04-1333-quincy-distro-default-smithi/7411759 -- /home/teuthworker/archive/yuriw-2023-10-05_14:30:58-rgw-wip-yuri2-testing-2023-10-04-1333-quincy-distro-default-smithi/7411759/orig.config.yaml 2023-10-05T19:28:28.087 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-05T19:28:28.163 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-10-05_14:30:58-rgw-wip-yuri2-testing-2023-10-04-1333-quincy-distro-default-smithi/7411759 branch: wip-yuri2-testing-2023-10-04-1333-quincy description: rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated supported-random-distro$/{ubuntu_latest}} email: yweinste@redhat.com first_in_suite: false job_id: '7411759' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: yuriw-2023-10-05_14:30:58-rgw-wip-yuri2-testing-2023-10-04-1333-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 3 size: 10 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-yuri2-testing-2023-10-04-1333-quincy ceph: conf: client: debug rgw: 20 setgroup: ceph setuser: ceph mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) sha1: 487ffcb5ddb60d29c5688d1a457e812b1b697b90 wait-for-scrub: false ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 487ffcb5ddb60d29c5688d1a457e812b1b697b90 rgw: ec-data-pool: false frontend: beast thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri2-testing-2023-10-04-1333-quincy sha1: 487ffcb5ddb60d29c5688d1a457e812b1b697b90 owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - osd.0 - - mgr.x - client.0 - osd.1 - osd.2 - osd.3 seed: 880 sha1: 487ffcb5ddb60d29c5688d1a457e812b1b697b90 sleep_before_teardown: 0 subset: 111/12000 suite: rgw suite_branch: wip-yuri2-testing-2023-10-04-1333-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_487ffcb5ddb60d29c5688d1a457e812b1b697b90/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 487ffcb5ddb60d29c5688d1a457e812b1b697b90 targets: smithi008.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= smithi012.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= tasks: - install: null - ceph: conf: client: debug ms: 1 rgw gc obj min wait: 15 osd: debug ms: 1 debug objclass: 20 - rgw: client.0: null - radosgw-admin: null - radosgw-admin-rest: null teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-10-05_14:30:58 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-05T19:28:28.164 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_487ffcb5ddb60d29c5688d1a457e812b1b697b90/qa; will attempt to use it 2023-10-05T19:28:28.165 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_487ffcb5ddb60d29c5688d1a457e812b1b697b90/qa/tasks 2023-10-05T19:28:28.165 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-05T19:28:28.166 INFO:teuthology.task.internal:Checking packages... 2023-10-05T19:28:28.189 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '487ffcb5ddb60d29c5688d1a457e812b1b697b90' 2023-10-05T19:28:28.190 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-05T19:28:28.190 INFO:teuthology.packaging:ref: None 2023-10-05T19:28:28.190 INFO:teuthology.packaging:tag: None 2023-10-05T19:28:28.190 INFO:teuthology.packaging:branch: wip-yuri2-testing-2023-10-04-1333-quincy 2023-10-05T19:28:28.190 INFO:teuthology.packaging:sha1: 487ffcb5ddb60d29c5688d1a457e812b1b697b90 2023-10-05T19:28:28.190 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-yuri2-testing-2023-10-04-1333-quincy 2023-10-05T19:28:28.346 INFO:teuthology.task.internal:Found packages for ceph version 17.2.6-1278-g487ffcb5-1focal 2023-10-05T19:28:28.346 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-05T19:28:28.358 INFO:teuthology.task.internal:no buildpackages task found 2023-10-05T19:28:28.359 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-05T19:28:28.434 INFO:teuthology.task.internal:Saving configuration 2023-10-05T19:28:28.447 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-05T19:28:28.455 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-05T19:28:28.480 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi008.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_14:30:58-rgw-wip-yuri2-testing-2023-10-04-1333-quincy-distro-default-smithi/7411759', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-05 19:22:29.289770', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2023-10-05T19:28:28.501 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi012.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_14:30:58-rgw-wip-yuri2-testing-2023-10-04-1333-quincy-distro-default-smithi/7411759', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-05 19:22:29.291659', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2023-10-05T19:28:28.501 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-05T19:28:28.513 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['mon.a', 'osd.0'] 2023-10-05T19:28:28.513 INFO:teuthology.task.internal:roles: ubuntu@smithi012.front.sepia.ceph.com - ['mgr.x', 'client.0', 'osd.1', 'osd.2', 'osd.3'] 2023-10-05T19:28:28.513 INFO:teuthology.run_tasks:Running task console_log... 2023-10-05T19:28:28.593 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ff44220e040>, signals=[15]) 2023-10-05T19:28:28.593 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-05T19:28:28.605 INFO:teuthology.task.internal:Opening connections... 2023-10-05T19:28:28.605 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi008.front.sepia.ceph.com 2023-10-05T19:28:28.606 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-05T19:28:28.657 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi012.front.sepia.ceph.com 2023-10-05T19:28:28.658 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-05T19:28:28.706 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-05T19:28:28.717 DEBUG:teuthology.orchestra.run.smithi008:> uname -m 2023-10-05T19:28:28.738 INFO:teuthology.orchestra.run.smithi008.stdout:x86_64 2023-10-05T19:28:28.738 DEBUG:teuthology.orchestra.run.smithi008:> cat /etc/os-release 2023-10-05T19:28:28.785 INFO:teuthology.orchestra.run.smithi008.stdout:NAME="Ubuntu" 2023-10-05T19:28:28.785 INFO:teuthology.orchestra.run.smithi008.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-10-05T19:28:28.785 INFO:teuthology.orchestra.run.smithi008.stdout:ID=ubuntu 2023-10-05T19:28:28.785 INFO:teuthology.orchestra.run.smithi008.stdout:ID_LIKE=debian 2023-10-05T19:28:28.785 INFO:teuthology.orchestra.run.smithi008.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-10-05T19:28:28.785 INFO:teuthology.orchestra.run.smithi008.stdout:VERSION_ID="20.04" 2023-10-05T19:28:28.785 INFO:teuthology.orchestra.run.smithi008.stdout:HOME_URL="https://www.ubuntu.com/" 2023-10-05T19:28:28.785 INFO:teuthology.orchestra.run.smithi008.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-10-05T19:28:28.785 INFO:teuthology.orchestra.run.smithi008.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-10-05T19:28:28.786 INFO:teuthology.orchestra.run.smithi008.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-10-05T19:28:28.786 INFO:teuthology.orchestra.run.smithi008.stdout:VERSION_CODENAME=focal 2023-10-05T19:28:28.786 INFO:teuthology.orchestra.run.smithi008.stdout:UBUNTU_CODENAME=focal 2023-10-05T19:28:28.786 INFO:teuthology.lock.ops:Updating smithi008.front.sepia.ceph.com on lock server 2023-10-05T19:28:28.808 DEBUG:teuthology.orchestra.run.smithi012:> uname -m 2023-10-05T19:28:28.815 INFO:teuthology.orchestra.run.smithi012.stdout:x86_64 2023-10-05T19:28:28.815 DEBUG:teuthology.orchestra.run.smithi012:> cat /etc/os-release 2023-10-05T19:28:28.865 INFO:teuthology.orchestra.run.smithi012.stdout:NAME="Ubuntu" 2023-10-05T19:28:28.865 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-10-05T19:28:28.865 INFO:teuthology.orchestra.run.smithi012.stdout:ID=ubuntu 2023-10-05T19:28:28.866 INFO:teuthology.orchestra.run.smithi012.stdout:ID_LIKE=debian 2023-10-05T19:28:28.866 INFO:teuthology.orchestra.run.smithi012.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-10-05T19:28:28.866 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION_ID="20.04" 2023-10-05T19:28:28.866 INFO:teuthology.orchestra.run.smithi012.stdout:HOME_URL="https://www.ubuntu.com/" 2023-10-05T19:28:28.866 INFO:teuthology.orchestra.run.smithi012.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-10-05T19:28:28.866 INFO:teuthology.orchestra.run.smithi012.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-10-05T19:28:28.866 INFO:teuthology.orchestra.run.smithi012.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-10-05T19:28:28.866 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION_CODENAME=focal 2023-10-05T19:28:28.866 INFO:teuthology.orchestra.run.smithi012.stdout:UBUNTU_CODENAME=focal 2023-10-05T19:28:28.867 INFO:teuthology.lock.ops:Updating smithi012.front.sepia.ceph.com on lock server 2023-10-05T19:28:28.891 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-05T19:28:28.902 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-05T19:28:28.915 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-05T19:28:28.915 DEBUG:teuthology.orchestra.run.smithi008:> test '!' -e /home/ubuntu/cephtest 2023-10-05T19:28:28.918 DEBUG:teuthology.orchestra.run.smithi012:> test '!' -e /home/ubuntu/cephtest 2023-10-05T19:28:28.923 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-05T19:28:28.934 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-05T19:28:28.934 DEBUG:teuthology.orchestra.run.smithi008:> test -z $(ls -A /var/lib/ceph) 2023-10-05T19:28:28.964 DEBUG:teuthology.orchestra.run.smithi012:> test -z $(ls -A /var/lib/ceph) 2023-10-05T19:28:28.982 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-05T19:28:29.043 INFO:teuthology.run_tasks:Running task kernel... 2023-10-05T19:28:29.062 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-10-05T19:28:29.062 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mgr.x': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-10-05T19:28:29.063 DEBUG:teuthology.orchestra.run.smithi008:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-05T19:28:29.063 DEBUG:teuthology.orchestra.run.smithi012:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-05T19:28:29.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-05T19:28:29.069 DEBUG:teuthology.orchestra.run.smithi008:> uname -r 2023-10-05T19:28:29.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-05T19:28:29.070 DEBUG:teuthology.orchestra.run.smithi012:> uname -r 2023-10-05T19:28:29.076 INFO:teuthology.orchestra.run.smithi008.stdout:5.4.0-124-generic 2023-10-05T19:28:29.076 INFO:teuthology.task.kernel:Running kernel on smithi008: 5.4.0-124-generic 2023-10-05T19:28:29.076 DEBUG:teuthology.orchestra.run.smithi008:> sudo apt-get clean 2023-10-05T19:28:29.077 INFO:teuthology.orchestra.run.smithi012.stdout:5.4.0-124-generic 2023-10-05T19:28:29.077 INFO:teuthology.task.kernel:Running kernel on smithi012: 5.4.0-124-generic 2023-10-05T19:28:29.077 DEBUG:teuthology.orchestra.run.smithi012:> sudo apt-get clean 2023-10-05T19:28:29.190 DEBUG:teuthology.orchestra.run.smithi012:> sudo apt-get update 2023-10-05T19:28:29.218 DEBUG:teuthology.orchestra.run.smithi008:> sudo apt-get update 2023-10-05T19:28:29.372 INFO:teuthology.orchestra.run.smithi008.stdout:Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2023-10-05T19:28:29.490 INFO:teuthology.orchestra.run.smithi008.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2023-10-05T19:28:29.512 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-10-05T19:28:29.513 INFO:teuthology.orchestra.run.smithi012.stdout:Get:2 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2023-10-05T19:28:29.565 INFO:teuthology.orchestra.run.smithi012.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2023-10-05T19:28:29.572 INFO:teuthology.orchestra.run.smithi008.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2023-10-05T19:28:29.670 INFO:teuthology.orchestra.run.smithi008.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [659 kB] 2023-10-05T19:28:29.752 INFO:teuthology.orchestra.run.smithi008.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [2,495 kB] 2023-10-05T19:28:29.800 INFO:teuthology.orchestra.run.smithi008.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [389 kB] 2023-10-05T19:28:29.801 INFO:teuthology.orchestra.run.smithi008.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [13.2 kB] 2023-10-05T19:28:29.801 INFO:teuthology.orchestra.run.smithi008.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [33.3 kB] 2023-10-05T19:28:29.801 INFO:teuthology.orchestra.run.smithi008.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [2,257 kB] 2023-10-05T19:28:29.811 INFO:teuthology.orchestra.run.smithi008.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [316 kB] 2023-10-05T19:28:29.814 INFO:teuthology.orchestra.run.smithi008.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [572 B] 2023-10-05T19:28:29.814 INFO:teuthology.orchestra.run.smithi008.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [890 kB] 2023-10-05T19:28:29.820 INFO:teuthology.orchestra.run.smithi008.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [616 kB] 2023-10-05T19:28:29.824 INFO:teuthology.orchestra.run.smithi008.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [186 kB] 2023-10-05T19:28:29.825 INFO:teuthology.orchestra.run.smithi008.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [19.1 kB] 2023-10-05T19:28:29.825 INFO:teuthology.orchestra.run.smithi008.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,196 B] 2023-10-05T19:28:29.826 INFO:teuthology.orchestra.run.smithi008.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [23.6 kB] 2023-10-05T19:28:29.836 INFO:teuthology.orchestra.run.smithi008.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,504 B] 2023-10-05T19:28:29.836 INFO:teuthology.orchestra.run.smithi008.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [548 B] 2023-10-05T19:28:29.910 INFO:teuthology.orchestra.run.smithi012.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2023-10-05T19:28:29.916 INFO:teuthology.orchestra.run.smithi008.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2023-10-05T19:28:29.968 INFO:teuthology.orchestra.run.smithi012.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [2,495 kB] 2023-10-05T19:28:30.056 INFO:teuthology.orchestra.run.smithi008.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [893 kB] 2023-10-05T19:28:30.110 INFO:teuthology.orchestra.run.smithi012.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [893 kB] 2023-10-05T19:28:30.303 INFO:teuthology.orchestra.run.smithi008.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [2,889 kB] 2023-10-05T19:28:30.467 INFO:teuthology.orchestra.run.smithi008.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [473 kB] 2023-10-05T19:28:30.473 INFO:teuthology.orchestra.run.smithi012.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [2,889 kB] 2023-10-05T19:28:30.473 INFO:teuthology.orchestra.run.smithi012.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [659 kB] 2023-10-05T19:28:30.480 INFO:teuthology.orchestra.run.smithi008.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.1 kB] 2023-10-05T19:28:30.480 INFO:teuthology.orchestra.run.smithi008.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [2,383 kB] 2023-10-05T19:28:30.488 INFO:teuthology.orchestra.run.smithi012.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [389 kB] 2023-10-05T19:28:30.497 INFO:teuthology.orchestra.run.smithi012.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [13.2 kB] 2023-10-05T19:28:30.497 INFO:teuthology.orchestra.run.smithi012.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [33.3 kB] 2023-10-05T19:28:30.498 INFO:teuthology.orchestra.run.smithi012.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [2,257 kB] 2023-10-05T19:28:30.541 INFO:teuthology.orchestra.run.smithi008.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [34.6 kB] 2023-10-05T19:28:30.542 INFO:teuthology.orchestra.run.smithi008.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [334 kB] 2023-10-05T19:28:30.562 INFO:teuthology.orchestra.run.smithi008.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [572 B] 2023-10-05T19:28:30.572 INFO:teuthology.orchestra.run.smithi012.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [316 kB] 2023-10-05T19:28:30.580 INFO:teuthology.orchestra.run.smithi012.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [572 B] 2023-10-05T19:28:30.580 INFO:teuthology.orchestra.run.smithi012.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [890 kB] 2023-10-05T19:28:30.600 INFO:teuthology.orchestra.run.smithi012.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [616 kB] 2023-10-05T19:28:30.679 INFO:teuthology.orchestra.run.smithi012.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [473 kB] 2023-10-05T19:28:30.692 INFO:teuthology.orchestra.run.smithi012.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.1 kB] 2023-10-05T19:28:30.692 INFO:teuthology.orchestra.run.smithi012.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [2,383 kB] 2023-10-05T19:28:30.726 INFO:teuthology.orchestra.run.smithi008.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,121 kB] 2023-10-05T19:28:30.768 INFO:teuthology.orchestra.run.smithi012.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [34.6 kB] 2023-10-05T19:28:30.769 INFO:teuthology.orchestra.run.smithi012.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [334 kB] 2023-10-05T19:28:30.776 INFO:teuthology.orchestra.run.smithi012.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [572 B] 2023-10-05T19:28:30.776 INFO:teuthology.orchestra.run.smithi012.stdout:Get:23 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [186 kB] 2023-10-05T19:28:30.940 INFO:teuthology.orchestra.run.smithi012.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [749 kB] 2023-10-05T19:28:31.195 INFO:teuthology.orchestra.run.smithi012.stdout:Get:25 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [19.1 kB] 2023-10-05T19:28:31.201 INFO:teuthology.orchestra.run.smithi012.stdout:Get:26 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [23.6 kB] 2023-10-05T19:28:31.208 INFO:teuthology.orchestra.run.smithi012.stdout:Get:27 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,196 B] 2023-10-05T19:28:31.211 INFO:teuthology.orchestra.run.smithi012.stdout:Get:28 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,504 B] 2023-10-05T19:28:31.212 INFO:teuthology.orchestra.run.smithi012.stdout:Get:29 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [548 B] 2023-10-05T19:28:31.358 INFO:teuthology.orchestra.run.smithi008.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [749 kB] 2023-10-05T19:28:31.397 INFO:teuthology.orchestra.run.smithi008.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [268 kB] 2023-10-05T19:28:31.410 INFO:teuthology.orchestra.run.smithi008.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [25.6 kB] 2023-10-05T19:28:31.412 INFO:teuthology.orchestra.run.smithi008.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [25.8 kB] 2023-10-05T19:28:31.413 INFO:teuthology.orchestra.run.smithi008.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,436 B] 2023-10-05T19:28:31.414 INFO:teuthology.orchestra.run.smithi008.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,484 B] 2023-10-05T19:28:31.414 INFO:teuthology.orchestra.run.smithi008.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [620 B] 2023-10-05T19:28:31.415 INFO:teuthology.orchestra.run.smithi008.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2023-10-05T19:28:31.440 INFO:teuthology.orchestra.run.smithi008.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2023-10-05T19:28:31.442 INFO:teuthology.orchestra.run.smithi008.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2023-10-05T19:28:31.526 INFO:teuthology.orchestra.run.smithi012.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,121 kB] 2023-10-05T19:28:31.607 INFO:teuthology.orchestra.run.smithi008.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2023-10-05T19:28:31.635 INFO:teuthology.orchestra.run.smithi012.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [268 kB] 2023-10-05T19:28:31.656 INFO:teuthology.orchestra.run.smithi012.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [25.6 kB] 2023-10-05T19:28:31.657 INFO:teuthology.orchestra.run.smithi012.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [25.8 kB] 2023-10-05T19:28:31.659 INFO:teuthology.orchestra.run.smithi012.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,436 B] 2023-10-05T19:28:31.660 INFO:teuthology.orchestra.run.smithi012.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,484 B] 2023-10-05T19:28:31.660 INFO:teuthology.orchestra.run.smithi012.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [620 B] 2023-10-05T19:28:31.660 INFO:teuthology.orchestra.run.smithi012.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2023-10-05T19:28:31.663 INFO:teuthology.orchestra.run.smithi012.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2023-10-05T19:28:31.666 INFO:teuthology.orchestra.run.smithi012.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2023-10-05T19:28:31.790 INFO:teuthology.orchestra.run.smithi008.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2023-10-05T19:28:31.813 INFO:teuthology.orchestra.run.smithi008.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2023-10-05T19:28:31.829 INFO:teuthology.orchestra.run.smithi012.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2023-10-05T19:28:32.012 INFO:teuthology.orchestra.run.smithi012.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2023-10-05T19:28:32.035 INFO:teuthology.orchestra.run.smithi012.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2023-10-05T19:28:33.795 INFO:teuthology.orchestra.run.smithi008.stdout:Fetched 17.6 MB in 3s (7,013 kB/s) 2023-10-05T19:28:34.435 INFO:teuthology.orchestra.run.smithi012.stdout:Fetched 17.6 MB in 3s (5,556 kB/s) 2023-10-05T19:28:35.281 INFO:teuthology.orchestra.run.smithi008.stdout:Reading package lists... 2023-10-05T19:28:35.310 DEBUG:teuthology.orchestra.run.smithi008:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-10-05T19:28:35.392 INFO:teuthology.orchestra.run.smithi008.stdout:Reading package lists... 2023-10-05T19:28:35.600 INFO:teuthology.orchestra.run.smithi008.stdout:Building dependency tree... 2023-10-05T19:28:35.601 INFO:teuthology.orchestra.run.smithi008.stdout:Reading state information... 2023-10-05T19:28:35.801 INFO:teuthology.orchestra.run.smithi008.stdout:The following additional packages will be installed: 2023-10-05T19:28:35.801 INFO:teuthology.orchestra.run.smithi008.stdout: linux-generic linux-headers-5.4.0-164 linux-headers-5.4.0-164-generic 2023-10-05T19:28:35.801 INFO:teuthology.orchestra.run.smithi008.stdout: linux-headers-generic linux-image-5.4.0-164-generic 2023-10-05T19:28:35.802 INFO:teuthology.orchestra.run.smithi008.stdout: linux-modules-5.4.0-164-generic linux-modules-extra-5.4.0-164-generic 2023-10-05T19:28:35.802 INFO:teuthology.orchestra.run.smithi008.stdout:Suggested packages: 2023-10-05T19:28:35.802 INFO:teuthology.orchestra.run.smithi008.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2023-10-05T19:28:35.803 INFO:teuthology.orchestra.run.smithi008.stdout:Recommended packages: 2023-10-05T19:28:35.803 INFO:teuthology.orchestra.run.smithi008.stdout: thermald 2023-10-05T19:28:35.861 INFO:teuthology.orchestra.run.smithi008.stdout:The following NEW packages will be installed: 2023-10-05T19:28:35.861 INFO:teuthology.orchestra.run.smithi008.stdout: linux-headers-5.4.0-164 linux-headers-5.4.0-164-generic 2023-10-05T19:28:35.861 INFO:teuthology.orchestra.run.smithi008.stdout: linux-image-5.4.0-164-generic linux-modules-5.4.0-164-generic 2023-10-05T19:28:35.861 INFO:teuthology.orchestra.run.smithi008.stdout: linux-modules-extra-5.4.0-164-generic 2023-10-05T19:28:35.862 INFO:teuthology.orchestra.run.smithi008.stdout:The following packages will be upgraded: 2023-10-05T19:28:35.863 INFO:teuthology.orchestra.run.smithi008.stdout: linux-generic linux-headers-generic linux-image-generic 2023-10-05T19:28:35.895 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2023-10-05T19:28:35.922 DEBUG:teuthology.orchestra.run.smithi012:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-10-05T19:28:36.011 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2023-10-05T19:28:36.218 INFO:teuthology.orchestra.run.smithi012.stdout:Building dependency tree... 2023-10-05T19:28:36.219 INFO:teuthology.orchestra.run.smithi012.stdout:Reading state information... 2023-10-05T19:28:36.410 INFO:teuthology.orchestra.run.smithi012.stdout:The following additional packages will be installed: 2023-10-05T19:28:36.410 INFO:teuthology.orchestra.run.smithi012.stdout: linux-generic linux-headers-5.4.0-164 linux-headers-5.4.0-164-generic 2023-10-05T19:28:36.410 INFO:teuthology.orchestra.run.smithi012.stdout: linux-headers-generic linux-image-5.4.0-164-generic 2023-10-05T19:28:36.410 INFO:teuthology.orchestra.run.smithi012.stdout: linux-modules-5.4.0-164-generic linux-modules-extra-5.4.0-164-generic 2023-10-05T19:28:36.411 INFO:teuthology.orchestra.run.smithi012.stdout:Suggested packages: 2023-10-05T19:28:36.412 INFO:teuthology.orchestra.run.smithi012.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2023-10-05T19:28:36.412 INFO:teuthology.orchestra.run.smithi012.stdout:Recommended packages: 2023-10-05T19:28:36.412 INFO:teuthology.orchestra.run.smithi012.stdout: thermald 2023-10-05T19:28:36.415 INFO:teuthology.orchestra.run.smithi008.stdout:3 upgraded, 5 newly installed, 0 to remove and 271 not upgraded. 2023-10-05T19:28:36.415 INFO:teuthology.orchestra.run.smithi008.stdout:Need to get 77.0 MB of archives. 2023-10-05T19:28:36.415 INFO:teuthology.orchestra.run.smithi008.stdout:After this operation, 380 MB of additional disk space will be used. 2023-10-05T19:28:36.415 INFO:teuthology.orchestra.run.smithi008.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-164-generic amd64 5.4.0-164.181 [14.9 MB] 2023-10-05T19:28:36.471 INFO:teuthology.orchestra.run.smithi012.stdout:The following NEW packages will be installed: 2023-10-05T19:28:36.472 INFO:teuthology.orchestra.run.smithi012.stdout: linux-headers-5.4.0-164 linux-headers-5.4.0-164-generic 2023-10-05T19:28:36.472 INFO:teuthology.orchestra.run.smithi012.stdout: linux-image-5.4.0-164-generic linux-modules-5.4.0-164-generic 2023-10-05T19:28:36.472 INFO:teuthology.orchestra.run.smithi012.stdout: linux-modules-extra-5.4.0-164-generic 2023-10-05T19:28:36.473 INFO:teuthology.orchestra.run.smithi012.stdout:The following packages will be upgraded: 2023-10-05T19:28:36.474 INFO:teuthology.orchestra.run.smithi012.stdout: linux-generic linux-headers-generic linux-image-generic 2023-10-05T19:28:36.527 INFO:teuthology.orchestra.run.smithi012.stdout:3 upgraded, 5 newly installed, 0 to remove and 271 not upgraded. 2023-10-05T19:28:36.527 INFO:teuthology.orchestra.run.smithi012.stdout:Need to get 77.0 MB of archives. 2023-10-05T19:28:36.527 INFO:teuthology.orchestra.run.smithi012.stdout:After this operation, 380 MB of additional disk space will be used. 2023-10-05T19:28:36.527 INFO:teuthology.orchestra.run.smithi012.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-164-generic amd64 5.4.0-164.181 [14.9 MB] 2023-10-05T19:28:36.743 INFO:teuthology.orchestra.run.smithi008.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-164-generic amd64 5.4.0-164.181 [10.5 MB] 2023-10-05T19:28:36.834 INFO:teuthology.orchestra.run.smithi008.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-164-generic amd64 5.4.0-164.181 [39.2 MB] 2023-10-05T19:28:36.865 INFO:teuthology.orchestra.run.smithi012.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-164-generic amd64 5.4.0-164.181 [10.5 MB] 2023-10-05T19:28:36.960 INFO:teuthology.orchestra.run.smithi012.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-164-generic amd64 5.4.0-164.181 [39.2 MB] 2023-10-05T19:28:37.327 INFO:teuthology.orchestra.run.smithi008.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.164.161 [1,896 B] 2023-10-05T19:28:37.327 INFO:teuthology.orchestra.run.smithi008.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.164.161 [2,592 B] 2023-10-05T19:28:37.328 INFO:teuthology.orchestra.run.smithi008.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-164 all 5.4.0-164.181 [11.0 MB] 2023-10-05T19:28:37.423 INFO:teuthology.orchestra.run.smithi008.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-164-generic amd64 5.4.0-164.181 [1,360 kB] 2023-10-05T19:28:37.433 INFO:teuthology.orchestra.run.smithi008.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.164.161 [2,468 B] 2023-10-05T19:28:37.436 INFO:teuthology.orchestra.run.smithi012.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.164.161 [1,896 B] 2023-10-05T19:28:37.436 INFO:teuthology.orchestra.run.smithi012.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.164.161 [2,592 B] 2023-10-05T19:28:37.437 INFO:teuthology.orchestra.run.smithi012.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-164 all 5.4.0-164.181 [11.0 MB] 2023-10-05T19:28:37.531 INFO:teuthology.orchestra.run.smithi012.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-164-generic amd64 5.4.0-164.181 [1,360 kB] 2023-10-05T19:28:37.541 INFO:teuthology.orchestra.run.smithi012.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.164.161 [2,468 B] 2023-10-05T19:28:38.105 INFO:teuthology.orchestra.run.smithi008.stdout:Fetched 77.0 MB in 2s (49.3 MB/s) 2023-10-05T19:28:38.239 INFO:teuthology.orchestra.run.smithi012.stdout:Fetched 77.0 MB in 1s (72.6 MB/s) 2023-10-05T19:28:38.288 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package linux-modules-5.4.0-164-generic. 2023-10-05T19:28:38.432 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package linux-modules-5.4.0-164-generic. 2023-10-05T19:28:39.995 INFO:teuthology.orchestra.run.smithi008.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2023-10-05T19:28:40.000 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../0-linux-modules-5.4.0-164-generic_5.4.0-164.181_amd64.deb ... 2023-10-05T19:28:40.094 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking linux-modules-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-05T19:28:40.233 INFO:teuthology.orchestra.run.smithi012.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2023-10-05T19:28:40.238 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../0-linux-modules-5.4.0-164-generic_5.4.0-164.181_amd64.deb ... 2023-10-05T19:28:40.362 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-modules-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-05T19:28:42.095 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package linux-image-5.4.0-164-generic. 2023-10-05T19:28:42.108 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../1-linux-image-5.4.0-164-generic_5.4.0-164.181_amd64.deb ... 2023-10-05T19:28:42.206 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking linux-image-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-05T19:28:42.321 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package linux-image-5.4.0-164-generic. 2023-10-05T19:28:42.336 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../1-linux-image-5.4.0-164-generic_5.4.0-164.181_amd64.deb ... 2023-10-05T19:28:42.434 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-image-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-05T19:28:43.323 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-164-generic. 2023-10-05T19:28:43.336 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-164-generic_5.4.0-164.181_amd64.deb ... 2023-10-05T19:28:43.381 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking linux-modules-extra-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-05T19:28:43.533 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-164-generic. 2023-10-05T19:28:43.549 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-164-generic_5.4.0-164.181_amd64.deb ... 2023-10-05T19:28:43.591 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-modules-extra-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-05T19:28:48.430 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../3-linux-generic_5.4.0.164.161_amd64.deb ... 2023-10-05T19:28:48.556 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-generic (5.4.0.164.161) over (5.4.0.124.125) ... 2023-10-05T19:28:48.602 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../3-linux-generic_5.4.0.164.161_amd64.deb ... 2023-10-05T19:28:48.729 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking linux-generic (5.4.0.164.161) over (5.4.0.124.125) ... 2023-10-05T19:28:48.899 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.164.161_amd64.deb ... 2023-10-05T19:28:49.035 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-image-generic (5.4.0.164.161) over (5.4.0.124.125) ... 2023-10-05T19:28:49.041 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.164.161_amd64.deb ... 2023-10-05T19:28:49.191 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking linux-image-generic (5.4.0.164.161) over (5.4.0.124.125) ... 2023-10-05T19:28:49.388 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package linux-headers-5.4.0-164. 2023-10-05T19:28:49.408 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../5-linux-headers-5.4.0-164_5.4.0-164.181_all.deb ... 2023-10-05T19:28:49.528 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package linux-headers-5.4.0-164. 2023-10-05T19:28:49.543 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../5-linux-headers-5.4.0-164_5.4.0-164.181_all.deb ... 2023-10-05T19:28:49.586 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking linux-headers-5.4.0-164 (5.4.0-164.181) ... 2023-10-05T19:28:49.704 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-headers-5.4.0-164 (5.4.0-164.181) ... 2023-10-05T19:28:55.020 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package linux-headers-5.4.0-164-generic. 2023-10-05T19:28:55.042 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../6-linux-headers-5.4.0-164-generic_5.4.0-164.181_amd64.deb ... 2023-10-05T19:28:55.086 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking linux-headers-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-05T19:28:55.525 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package linux-headers-5.4.0-164-generic. 2023-10-05T19:28:55.548 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../6-linux-headers-5.4.0-164-generic_5.4.0-164.181_amd64.deb ... 2023-10-05T19:28:55.617 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-headers-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-05T19:28:57.950 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.164.161_amd64.deb ... 2023-10-05T19:28:58.164 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking linux-headers-generic (5.4.0.164.161) over (5.4.0.124.125) ... 2023-10-05T19:28:58.445 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.164.161_amd64.deb ... 2023-10-05T19:28:58.598 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-headers-generic (5.4.0.164.161) over (5.4.0.124.125) ... 2023-10-05T19:28:58.625 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up linux-headers-5.4.0-164 (5.4.0-164.181) ... 2023-10-05T19:28:58.761 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up linux-headers-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-05T19:28:58.887 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up linux-modules-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-05T19:28:58.942 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-headers-5.4.0-164 (5.4.0-164.181) ... 2023-10-05T19:28:59.070 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-headers-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-05T19:28:59.218 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-modules-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-05T19:29:00.156 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up linux-image-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-05T19:29:00.497 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-image-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-05T19:29:01.374 INFO:teuthology.orchestra.run.smithi008.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-164-generic 2023-10-05T19:29:01.375 INFO:teuthology.orchestra.run.smithi008.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-164-generic 2023-10-05T19:29:01.524 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up linux-headers-generic (5.4.0.164.161) ... 2023-10-05T19:29:01.651 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up linux-modules-extra-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-05T19:29:01.694 INFO:teuthology.orchestra.run.smithi012.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-164-generic 2023-10-05T19:29:01.694 INFO:teuthology.orchestra.run.smithi012.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-164-generic 2023-10-05T19:29:01.850 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-headers-generic (5.4.0.164.161) ... 2023-10-05T19:29:01.976 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-modules-extra-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-05T19:29:02.902 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up linux-image-generic (5.4.0.164.161) ... 2023-10-05T19:29:03.053 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up linux-generic (5.4.0.164.161) ... 2023-10-05T19:29:03.180 INFO:teuthology.orchestra.run.smithi008.stdout:Processing triggers for linux-image-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-05T19:29:03.202 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-image-generic (5.4.0.164.161) ... 2023-10-05T19:29:03.248 INFO:teuthology.orchestra.run.smithi008.stdout:/etc/kernel/postinst.d/initramfs-tools: 2023-10-05T19:29:03.248 INFO:teuthology.orchestra.run.smithi008.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-164-generic 2023-10-05T19:29:03.328 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-generic (5.4.0.164.161) ... 2023-10-05T19:29:03.488 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for linux-image-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-05T19:29:03.564 INFO:teuthology.orchestra.run.smithi012.stdout:/etc/kernel/postinst.d/initramfs-tools: 2023-10-05T19:29:03.564 INFO:teuthology.orchestra.run.smithi012.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-164-generic 2023-10-05T19:29:26.379 INFO:teuthology.orchestra.run.smithi012.stdout:/etc/kernel/postinst.d/zz-update-grub: 2023-10-05T19:29:26.379 INFO:teuthology.orchestra.run.smithi012.stdout:Sourcing file `/etc/default/grub' 2023-10-05T19:29:26.388 INFO:teuthology.orchestra.run.smithi012.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-05T19:29:26.400 INFO:teuthology.orchestra.run.smithi012.stdout:Generating grub configuration file ... 2023-10-05T19:29:26.401 INFO:teuthology.orchestra.run.smithi008.stdout:/etc/kernel/postinst.d/zz-update-grub: 2023-10-05T19:29:26.401 INFO:teuthology.orchestra.run.smithi008.stdout:Sourcing file `/etc/default/grub' 2023-10-05T19:29:26.420 INFO:teuthology.orchestra.run.smithi008.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-05T19:29:26.444 INFO:teuthology.orchestra.run.smithi008.stdout:Generating grub configuration file ... 2023-10-05T19:29:26.772 INFO:teuthology.orchestra.run.smithi012.stdout:Found linux image: /boot/vmlinuz-5.4.0-164-generic 2023-10-05T19:29:26.780 INFO:teuthology.orchestra.run.smithi008.stdout:Found linux image: /boot/vmlinuz-5.4.0-164-generic 2023-10-05T19:29:26.798 INFO:teuthology.orchestra.run.smithi012.stdout:Found initrd image: /boot/initrd.img-5.4.0-164-generic 2023-10-05T19:29:26.803 INFO:teuthology.orchestra.run.smithi008.stdout:Found initrd image: /boot/initrd.img-5.4.0-164-generic 2023-10-05T19:29:27.162 INFO:teuthology.orchestra.run.smithi008.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-10-05T19:29:27.174 INFO:teuthology.orchestra.run.smithi008.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-10-05T19:29:27.190 INFO:teuthology.orchestra.run.smithi012.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-10-05T19:29:27.202 INFO:teuthology.orchestra.run.smithi012.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-10-05T19:29:29.818 INFO:teuthology.orchestra.run.smithi008.stdout:done 2023-10-05T19:29:29.991 INFO:teuthology.orchestra.run.smithi012.stdout:done 2023-10-05T19:29:30.230 DEBUG:teuthology.orchestra.run.smithi008:> dpkg -s linux-image-generic 2023-10-05T19:29:30.277 INFO:teuthology.orchestra.run.smithi008.stdout:Package: linux-image-generic 2023-10-05T19:29:30.278 INFO:teuthology.orchestra.run.smithi008.stdout:Status: install ok installed 2023-10-05T19:29:30.279 INFO:teuthology.orchestra.run.smithi008.stdout:Priority: optional 2023-10-05T19:29:30.279 INFO:teuthology.orchestra.run.smithi008.stdout:Section: kernel 2023-10-05T19:29:30.279 INFO:teuthology.orchestra.run.smithi008.stdout:Installed-Size: 20 2023-10-05T19:29:30.279 INFO:teuthology.orchestra.run.smithi008.stdout:Maintainer: Ubuntu Kernel Team 2023-10-05T19:29:30.279 INFO:teuthology.orchestra.run.smithi008.stdout:Architecture: amd64 2023-10-05T19:29:30.279 INFO:teuthology.orchestra.run.smithi008.stdout:Source: linux-meta 2023-10-05T19:29:30.279 INFO:teuthology.orchestra.run.smithi008.stdout:Version: 5.4.0.164.161 2023-10-05T19:29:30.279 INFO:teuthology.orchestra.run.smithi008.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.15) 2023-10-05T19:29:30.280 INFO:teuthology.orchestra.run.smithi008.stdout:Depends: linux-image-5.4.0-164-generic, linux-modules-extra-5.4.0-164-generic, linux-firmware, intel-microcode, amd64-microcode 2023-10-05T19:29:30.280 INFO:teuthology.orchestra.run.smithi008.stdout:Recommends: thermald 2023-10-05T19:29:30.280 INFO:teuthology.orchestra.run.smithi008.stdout:Description: Generic Linux kernel image 2023-10-05T19:29:30.280 INFO:teuthology.orchestra.run.smithi008.stdout: This package will always depend on the latest generic kernel image 2023-10-05T19:29:30.280 INFO:teuthology.orchestra.run.smithi008.stdout: available. 2023-10-05T19:29:30.280 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-164-generic 2023-10-05T19:29:30.280 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-05T19:29:30.280 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-05T19:29:30.281 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-10-05T19:29:30.281 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi008.front.sepia.ceph.com, path=None, version=distro) 2023-10-05T19:29:30.281 DEBUG:teuthology.orchestra.run.smithi008:> sudo apt-get clean 2023-10-05T19:29:30.350 DEBUG:teuthology.orchestra.run.smithi008:> sudo apt-get update 2023-10-05T19:29:30.398 DEBUG:teuthology.orchestra.run.smithi012:> dpkg -s linux-image-generic 2023-10-05T19:29:30.442 INFO:teuthology.orchestra.run.smithi012.stdout:Package: linux-image-generic 2023-10-05T19:29:30.442 INFO:teuthology.orchestra.run.smithi012.stdout:Status: install ok installed 2023-10-05T19:29:30.443 INFO:teuthology.orchestra.run.smithi012.stdout:Priority: optional 2023-10-05T19:29:30.443 INFO:teuthology.orchestra.run.smithi012.stdout:Section: kernel 2023-10-05T19:29:30.443 INFO:teuthology.orchestra.run.smithi012.stdout:Installed-Size: 20 2023-10-05T19:29:30.443 INFO:teuthology.orchestra.run.smithi012.stdout:Maintainer: Ubuntu Kernel Team 2023-10-05T19:29:30.443 INFO:teuthology.orchestra.run.smithi012.stdout:Architecture: amd64 2023-10-05T19:29:30.443 INFO:teuthology.orchestra.run.smithi012.stdout:Source: linux-meta 2023-10-05T19:29:30.443 INFO:teuthology.orchestra.run.smithi012.stdout:Version: 5.4.0.164.161 2023-10-05T19:29:30.443 INFO:teuthology.orchestra.run.smithi012.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.15) 2023-10-05T19:29:30.443 INFO:teuthology.orchestra.run.smithi012.stdout:Depends: linux-image-5.4.0-164-generic, linux-modules-extra-5.4.0-164-generic, linux-firmware, intel-microcode, amd64-microcode 2023-10-05T19:29:30.443 INFO:teuthology.orchestra.run.smithi012.stdout:Recommends: thermald 2023-10-05T19:29:30.443 INFO:teuthology.orchestra.run.smithi012.stdout:Description: Generic Linux kernel image 2023-10-05T19:29:30.444 INFO:teuthology.orchestra.run.smithi012.stdout: This package will always depend on the latest generic kernel image 2023-10-05T19:29:30.444 INFO:teuthology.orchestra.run.smithi012.stdout: available. 2023-10-05T19:29:30.444 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-164-generic 2023-10-05T19:29:30.444 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-05T19:29:30.444 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-05T19:29:30.444 INFO:teuthology.task.kernel:Installing distro kernel on mgr.x... 2023-10-05T19:29:30.444 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi012.front.sepia.ceph.com, path=None, version=distro) 2023-10-05T19:29:30.444 DEBUG:teuthology.orchestra.run.smithi012:> sudo apt-get clean 2023-10-05T19:29:30.501 INFO:teuthology.orchestra.run.smithi008.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-10-05T19:29:30.510 INFO:teuthology.orchestra.run.smithi008.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-10-05T19:29:30.529 DEBUG:teuthology.orchestra.run.smithi012:> sudo apt-get update 2023-10-05T19:29:30.530 INFO:teuthology.orchestra.run.smithi008.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-10-05T19:29:30.600 INFO:teuthology.orchestra.run.smithi008.stdout:Hit:4 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-10-05T19:29:30.663 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-10-05T19:29:30.663 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2023-10-05T19:29:30.676 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-10-05T19:29:30.696 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-10-05T19:29:32.333 INFO:teuthology.orchestra.run.smithi008.stdout:Reading package lists... 2023-10-05T19:29:32.361 DEBUG:teuthology.orchestra.run.smithi008:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-10-05T19:29:32.433 INFO:teuthology.orchestra.run.smithi008.stdout:Reading package lists... 2023-10-05T19:29:32.458 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2023-10-05T19:29:32.485 DEBUG:teuthology.orchestra.run.smithi012:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-10-05T19:29:32.556 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2023-10-05T19:29:32.626 INFO:teuthology.orchestra.run.smithi008.stdout:Building dependency tree... 2023-10-05T19:29:32.626 INFO:teuthology.orchestra.run.smithi008.stdout:Reading state information... 2023-10-05T19:29:32.748 INFO:teuthology.orchestra.run.smithi012.stdout:Building dependency tree... 2023-10-05T19:29:32.748 INFO:teuthology.orchestra.run.smithi012.stdout:Reading state information... 2023-10-05T19:29:32.848 INFO:teuthology.orchestra.run.smithi008.stdout:linux-image-generic is already the newest version (5.4.0.164.161). 2023-10-05T19:29:32.848 INFO:teuthology.orchestra.run.smithi008.stdout:0 upgraded, 0 newly installed, 0 to remove and 271 not upgraded. 2023-10-05T19:29:32.850 DEBUG:teuthology.orchestra.run.smithi008:> dpkg -s linux-image-generic 2023-10-05T19:29:32.868 INFO:teuthology.orchestra.run.smithi008.stdout:Package: linux-image-generic 2023-10-05T19:29:32.869 INFO:teuthology.orchestra.run.smithi008.stdout:Status: install ok installed 2023-10-05T19:29:32.869 INFO:teuthology.orchestra.run.smithi008.stdout:Priority: optional 2023-10-05T19:29:32.869 INFO:teuthology.orchestra.run.smithi008.stdout:Section: kernel 2023-10-05T19:29:32.869 INFO:teuthology.orchestra.run.smithi008.stdout:Installed-Size: 20 2023-10-05T19:29:32.869 INFO:teuthology.orchestra.run.smithi008.stdout:Maintainer: Ubuntu Kernel Team 2023-10-05T19:29:32.869 INFO:teuthology.orchestra.run.smithi008.stdout:Architecture: amd64 2023-10-05T19:29:32.869 INFO:teuthology.orchestra.run.smithi008.stdout:Source: linux-meta 2023-10-05T19:29:32.870 INFO:teuthology.orchestra.run.smithi008.stdout:Version: 5.4.0.164.161 2023-10-05T19:29:32.870 INFO:teuthology.orchestra.run.smithi008.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.15) 2023-10-05T19:29:32.870 INFO:teuthology.orchestra.run.smithi008.stdout:Depends: linux-image-5.4.0-164-generic, linux-modules-extra-5.4.0-164-generic, linux-firmware, intel-microcode, amd64-microcode 2023-10-05T19:29:32.870 INFO:teuthology.orchestra.run.smithi008.stdout:Recommends: thermald 2023-10-05T19:29:32.870 INFO:teuthology.orchestra.run.smithi008.stdout:Description: Generic Linux kernel image 2023-10-05T19:29:32.870 INFO:teuthology.orchestra.run.smithi008.stdout: This package will always depend on the latest generic kernel image 2023-10-05T19:29:32.870 INFO:teuthology.orchestra.run.smithi008.stdout: available. 2023-10-05T19:29:32.871 DEBUG:teuthology.orchestra.run.smithi008:> mktemp 2023-10-05T19:29:32.875 INFO:teuthology.orchestra.run.smithi008.stdout:/tmp/tmp.xdr3w3bwDZ 2023-10-05T19:29:32.875 DEBUG:teuthology.orchestra.run.smithi008:> sudo cp /boot/grub/grub.cfg /tmp/tmp.xdr3w3bwDZ 2023-10-05T19:29:32.885 DEBUG:teuthology.orchestra.run.smithi008:> sudo chmod 0666 /tmp/tmp.xdr3w3bwDZ 2023-10-05T19:29:32.964 INFO:teuthology.orchestra.run.smithi012.stdout:linux-image-generic is already the newest version (5.4.0.164.161). 2023-10-05T19:29:32.964 INFO:teuthology.orchestra.run.smithi012.stdout:0 upgraded, 0 newly installed, 0 to remove and 271 not upgraded. 2023-10-05T19:29:32.966 DEBUG:teuthology.orchestra.run.smithi012:> dpkg -s linux-image-generic 2023-10-05T19:29:32.975 DEBUG:teuthology.orchestra.remote:smithi008:/tmp/tmp.xdr3w3bwDZ is 9KB 2023-10-05T19:29:32.986 INFO:teuthology.orchestra.run.smithi012.stdout:Package: linux-image-generic 2023-10-05T19:29:32.986 INFO:teuthology.orchestra.run.smithi012.stdout:Status: install ok installed 2023-10-05T19:29:32.986 INFO:teuthology.orchestra.run.smithi012.stdout:Priority: optional 2023-10-05T19:29:32.986 INFO:teuthology.orchestra.run.smithi012.stdout:Section: kernel 2023-10-05T19:29:32.986 INFO:teuthology.orchestra.run.smithi012.stdout:Installed-Size: 20 2023-10-05T19:29:32.986 INFO:teuthology.orchestra.run.smithi012.stdout:Maintainer: Ubuntu Kernel Team 2023-10-05T19:29:32.987 INFO:teuthology.orchestra.run.smithi012.stdout:Architecture: amd64 2023-10-05T19:29:32.987 INFO:teuthology.orchestra.run.smithi012.stdout:Source: linux-meta 2023-10-05T19:29:32.987 INFO:teuthology.orchestra.run.smithi012.stdout:Version: 5.4.0.164.161 2023-10-05T19:29:32.987 INFO:teuthology.orchestra.run.smithi012.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.15) 2023-10-05T19:29:32.987 INFO:teuthology.orchestra.run.smithi012.stdout:Depends: linux-image-5.4.0-164-generic, linux-modules-extra-5.4.0-164-generic, linux-firmware, intel-microcode, amd64-microcode 2023-10-05T19:29:32.987 INFO:teuthology.orchestra.run.smithi012.stdout:Recommends: thermald 2023-10-05T19:29:32.987 INFO:teuthology.orchestra.run.smithi012.stdout:Description: Generic Linux kernel image 2023-10-05T19:29:32.987 INFO:teuthology.orchestra.run.smithi012.stdout: This package will always depend on the latest generic kernel image 2023-10-05T19:29:32.987 INFO:teuthology.orchestra.run.smithi012.stdout: available. 2023-10-05T19:29:32.988 DEBUG:teuthology.orchestra.run.smithi012:> mktemp 2023-10-05T19:29:32.992 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/tmp.b3K3VxoxFv 2023-10-05T19:29:32.992 DEBUG:teuthology.orchestra.run.smithi012:> sudo cp /boot/grub/grub.cfg /tmp/tmp.b3K3VxoxFv 2023-10-05T19:29:32.993 DEBUG:teuthology.orchestra.run.smithi008:> rm -fr /tmp/tmp.xdr3w3bwDZ 2023-10-05T19:29:32.999 DEBUG:teuthology.orchestra.run.smithi008:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2023-10-05T19:29:33.010 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0666 /tmp/tmp.b3K3VxoxFv 2023-10-05T19:29:33.052 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-10-05T19:29:33.053 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/etc/grub.d/01_ceph_kernel 2023-10-05T19:29:33.053 DEBUG:teuthology.orchestra.run.smithi008:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2023-10-05T19:29:33.086 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-164-generic 2023-10-05T19:29:33.086 DEBUG:teuthology.orchestra.run.smithi008:> sudo update-grub 2023-10-05T19:29:33.097 DEBUG:teuthology.orchestra.remote:smithi012:/tmp/tmp.b3K3VxoxFv is 9KB 2023-10-05T19:29:33.111 DEBUG:teuthology.orchestra.run.smithi012:> rm -fr /tmp/tmp.b3K3VxoxFv 2023-10-05T19:29:33.117 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2023-10-05T19:29:33.130 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-05T19:29:33.130 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/grub.d/01_ceph_kernel 2023-10-05T19:29:33.130 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2023-10-05T19:29:33.157 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-164-generic 2023-10-05T19:29:33.158 DEBUG:teuthology.orchestra.run.smithi012:> sudo update-grub 2023-10-05T19:29:34.446 INFO:teuthology.orchestra.run.smithi008.stderr:Sourcing file `/etc/default/grub' 2023-10-05T19:29:34.455 INFO:teuthology.orchestra.run.smithi008.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-05T19:29:34.468 INFO:teuthology.orchestra.run.smithi008.stderr:Generating grub configuration file ... 2023-10-05T19:29:34.697 INFO:teuthology.orchestra.run.smithi008.stderr:Found linux image: /boot/vmlinuz-5.4.0-164-generic 2023-10-05T19:29:34.710 INFO:teuthology.orchestra.run.smithi008.stderr:Found initrd image: /boot/initrd.img-5.4.0-164-generic 2023-10-05T19:29:34.719 INFO:teuthology.orchestra.run.smithi012.stderr:Sourcing file `/etc/default/grub' 2023-10-05T19:29:34.729 INFO:teuthology.orchestra.run.smithi012.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-05T19:29:34.740 INFO:teuthology.orchestra.run.smithi012.stderr:Generating grub configuration file ... 2023-10-05T19:29:35.042 INFO:teuthology.orchestra.run.smithi012.stderr:Found linux image: /boot/vmlinuz-5.4.0-164-generic 2023-10-05T19:29:35.053 INFO:teuthology.orchestra.run.smithi012.stderr:Found initrd image: /boot/initrd.img-5.4.0-164-generic 2023-10-05T19:29:35.059 INFO:teuthology.orchestra.run.smithi008.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-10-05T19:29:35.071 INFO:teuthology.orchestra.run.smithi008.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-10-05T19:29:35.448 INFO:teuthology.orchestra.run.smithi012.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-10-05T19:29:35.460 INFO:teuthology.orchestra.run.smithi012.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-10-05T19:29:36.728 INFO:teuthology.orchestra.run.smithi008.stderr:done 2023-10-05T19:29:36.730 DEBUG:teuthology.orchestra.run.smithi008:> sudo shutdown -r now 2023-10-05T19:29:37.217 INFO:teuthology.orchestra.run.smithi012.stderr:done 2023-10-05T19:29:37.218 DEBUG:teuthology.orchestra.run.smithi012:> sudo shutdown -r now 2023-10-05T19:30:06.762 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-10-05T19:30:06.763 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com' 2023-10-05T19:30:06.763 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-05T19:30:07.221 INFO:teuthology.task.kernel:Checking client mgr.x for new kernel version... 2023-10-05T19:30:07.222 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-10-05T19:30:07.223 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-05T19:30:25.278 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.8 2023-10-05T19:30:25.790 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-10-05T19:30:34.286 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com' 2023-10-05T19:30:34.287 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-05T19:30:34.791 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-10-05T19:30:34.792 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-05T19:30:37.338 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.8 2023-10-05T19:30:49.349 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com' 2023-10-05T19:30:49.349 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-05T19:30:53.178 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-10-05T19:31:05.188 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-10-05T19:31:05.189 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-05T19:31:06.207 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-10-05T19:31:21.179 DEBUG:teuthology.orchestra.run.smithi008:> true 2023-10-05T19:31:21.208 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-10-05T19:31:21.209 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-05T19:31:21.410 DEBUG:teuthology.orchestra.run.smithi012:> true 2023-10-05T19:31:21.974 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-10-05T19:31:21.974 INFO:teuthology.task.kernel:Checking kernel version of mgr.x, want "5.4.0-164-generic"... 2023-10-05T19:31:21.974 DEBUG:teuthology.orchestra.run.smithi012:> uname -r 2023-10-05T19:31:21.980 INFO:teuthology.orchestra.run.smithi012.stdout:5.4.0-164-generic 2023-10-05T19:31:21.980 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-164-generic vs 5.4.0-164-generic 2023-10-05T19:31:21.981 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-05T19:31:21.981 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-10-05T19:31:22.065 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi008.front.sepia.ceph.com' 2023-10-05T19:31:22.065 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.4.0-164-generic"... 2023-10-05T19:31:22.065 DEBUG:teuthology.orchestra.run.smithi008:> uname -r 2023-10-05T19:31:22.072 INFO:teuthology.orchestra.run.smithi008.stdout:5.4.0-164-generic 2023-10-05T19:31:22.072 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-164-generic vs 5.4.0-164-generic 2023-10-05T19:31:22.072 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-05T19:31:22.073 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-10-05T19:31:22.982 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2023-10-05T19:31:22.982 DEBUG:teuthology.orchestra.run.smithi012:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-05T19:31:23.073 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-10-05T19:31:23.073 DEBUG:teuthology.orchestra.run.smithi008:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-05T19:31:23.081 INFO:teuthology.orchestra.run.smithi012.stdout:ttyS1 2023-10-05T19:31:23.101 DEBUG:teuthology.parallel:result is None 2023-10-05T19:31:23.162 INFO:teuthology.orchestra.run.smithi008.stdout:ttyS1 2023-10-05T19:31:23.186 DEBUG:teuthology.parallel:result is None 2023-10-05T19:31:23.186 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-05T19:31:23.193 INFO:teuthology.task.internal:Creating test directory... 2023-10-05T19:31:23.194 DEBUG:teuthology.orchestra.run.smithi008:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-05T19:31:23.196 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-05T19:31:23.203 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-05T19:31:23.210 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-05T19:31:23.222 INFO:teuthology.task.internal:Creating archive directory... 2023-10-05T19:31:23.222 DEBUG:teuthology.orchestra.run.smithi008:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-05T19:31:23.225 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-05T19:31:23.383 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-05T19:31:23.391 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-05T19:31:23.392 DEBUG:teuthology.orchestra.run.smithi008:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-05T19:31:23.394 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-05T19:31:23.411 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-05T19:31:23.412 INFO:teuthology.orchestra.run.smithi008.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-05T19:31:23.422 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-05T19:31:23.422 INFO:teuthology.orchestra.run.smithi008.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-05T19:31:23.424 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-05T19:31:23.432 INFO:teuthology.task.internal:Configuring sudo... 2023-10-05T19:31:23.432 DEBUG:teuthology.orchestra.run.smithi008:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-05T19:31:23.469 DEBUG:teuthology.orchestra.run.smithi012:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-05T19:31:23.485 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-05T19:31:23.493 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-05T19:31:23.494 DEBUG:teuthology.orchestra.run.smithi008:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-05T19:31:23.496 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-05T19:31:23.533 DEBUG:teuthology.orchestra.run.smithi008:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-05T19:31:23.541 DEBUG:teuthology.orchestra.run.smithi008:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-05T19:31:23.590 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-10-05T19:31:23.590 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-05T19:31:23.612 DEBUG:teuthology.orchestra.run.smithi012:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-05T19:31:23.619 DEBUG:teuthology.orchestra.run.smithi012:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-05T19:31:23.626 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-05T19:31:23.626 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-05T19:31:23.691 DEBUG:teuthology.orchestra.run.smithi008:> sudo service rsyslog restart 2023-10-05T19:31:23.694 DEBUG:teuthology.orchestra.run.smithi012:> sudo service rsyslog restart 2023-10-05T19:31:23.746 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-05T19:31:23.754 INFO:teuthology.task.internal:Starting timer... 2023-10-05T19:31:23.754 INFO:teuthology.run_tasks:Running task pcp... 2023-10-05T19:31:23.765 INFO:teuthology.run_tasks:Running task selinux... 2023-10-05T19:31:23.779 DEBUG:teuthology.task.selinux:Excluding smithi008: OS 'ubuntu' does not support SELinux 2023-10-05T19:31:23.779 DEBUG:teuthology.task.selinux:Excluding smithi012: OS 'ubuntu' does not support SELinux 2023-10-05T19:31:23.780 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-05T19:31:23.780 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2023-10-05T19:31:23.780 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-05T19:31:23.780 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-05T19:31:23.792 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-10-05T19:31:23.792 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-05T19:31:23.809 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-05T19:31:23.810 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi008.front.sepia.ceph.com,smithi012.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-05T19:37:10.784 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi008.front.sepia.ceph.com'), Remote(name='ubuntu@smithi012.front.sepia.ceph.com')] 2023-10-05T19:37:10.785 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com' 2023-10-05T19:37:10.786 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-05T19:37:10.836 DEBUG:teuthology.orchestra.run.smithi008:> true 2023-10-05T19:37:10.956 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi008.front.sepia.ceph.com' 2023-10-05T19:37:10.956 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-10-05T19:37:10.957 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-05T19:37:11.006 DEBUG:teuthology.orchestra.run.smithi012:> true 2023-10-05T19:37:11.127 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-10-05T19:37:11.128 INFO:teuthology.run_tasks:Running task clock... 2023-10-05T19:37:11.138 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-05T19:37:11.138 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-05T19:37:11.139 DEBUG:teuthology.orchestra.run.smithi008:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-05T19:37:11.141 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-05T19:37:11.141 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-05T19:37:11.174 INFO:teuthology.orchestra.run.smithi008.stdout: 5 Oct 19:37:11 ntpd[22742]: ntpd 4.2.8p12@1.3728-o (1): Starting 2023-10-05T19:37:11.175 INFO:teuthology.orchestra.run.smithi008.stdout: 5 Oct 19:37:11 ntpd[22742]: Command line: ntpd -gq 2023-10-05T19:37:11.176 INFO:teuthology.orchestra.run.smithi008.stdout: 5 Oct 19:37:11 ntpd[22742]: proto: precision = 0.099 usec (-23) 2023-10-05T19:37:11.176 INFO:teuthology.orchestra.run.smithi008.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-05T19:37:11.176 INFO:teuthology.orchestra.run.smithi008.stdout: 5 Oct 19:37:11 ntpd[22742]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-05T19:37:11.177 INFO:teuthology.orchestra.run.smithi008.stderr:restrict ::: KOD does nothing without LIMITED. 2023-10-05T19:37:11.178 INFO:teuthology.orchestra.run.smithi008.stdout: 5 Oct 19:37:11 ntpd[22742]: restrict ::: KOD does nothing without LIMITED. 2023-10-05T19:37:11.178 INFO:teuthology.orchestra.run.smithi008.stdout: 5 Oct 19:37:11 ntpd[22742]: Listen and drop on 0 v6wildcard [::]:123 2023-10-05T19:37:11.178 INFO:teuthology.orchestra.run.smithi008.stdout: 5 Oct 19:37:11 ntpd[22742]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-10-05T19:37:11.178 INFO:teuthology.orchestra.run.smithi008.stdout: 5 Oct 19:37:11 ntpd[22742]: Listen normally on 2 lo 127.0.0.1:123 2023-10-05T19:37:11.178 INFO:teuthology.orchestra.run.smithi008.stdout: 5 Oct 19:37:11 ntpd[22742]: Listen normally on 3 ens1f0 172.21.15.8:123 2023-10-05T19:37:11.178 INFO:teuthology.orchestra.run.smithi008.stdout: 5 Oct 19:37:11 ntpd[22742]: Listen normally on 4 lo [::1]:123 2023-10-05T19:37:11.178 INFO:teuthology.orchestra.run.smithi008.stdout: 5 Oct 19:37:11 ntpd[22742]: Listen normally on 5 ens1f0 [fe80::ae1f:6bff:fef8:25fe%3]:123 2023-10-05T19:37:11.178 INFO:teuthology.orchestra.run.smithi008.stdout: 5 Oct 19:37:11 ntpd[22742]: Listening on routing socket on fd #22 for interface updates 2023-10-05T19:37:11.204 INFO:teuthology.orchestra.run.smithi012.stdout: 5 Oct 19:37:11 ntpd[22758]: ntpd 4.2.8p12@1.3728-o (1): Starting 2023-10-05T19:37:11.204 INFO:teuthology.orchestra.run.smithi012.stdout: 5 Oct 19:37:11 ntpd[22758]: Command line: ntpd -gq 2023-10-05T19:37:11.204 INFO:teuthology.orchestra.run.smithi012.stdout: 5 Oct 19:37:11 ntpd[22758]: proto: precision = 0.065 usec (-24) 2023-10-05T19:37:11.205 INFO:teuthology.orchestra.run.smithi012.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-05T19:37:11.205 INFO:teuthology.orchestra.run.smithi012.stdout: 5 Oct 19:37:11 ntpd[22758]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-05T19:37:11.205 INFO:teuthology.orchestra.run.smithi012.stderr:restrict ::: KOD does nothing without LIMITED. 2023-10-05T19:37:11.206 INFO:teuthology.orchestra.run.smithi012.stdout: 5 Oct 19:37:11 ntpd[22758]: restrict ::: KOD does nothing without LIMITED. 2023-10-05T19:37:11.207 INFO:teuthology.orchestra.run.smithi012.stdout: 5 Oct 19:37:11 ntpd[22758]: Listen and drop on 0 v6wildcard [::]:123 2023-10-05T19:37:11.207 INFO:teuthology.orchestra.run.smithi012.stdout: 5 Oct 19:37:11 ntpd[22758]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-10-05T19:37:11.207 INFO:teuthology.orchestra.run.smithi012.stdout: 5 Oct 19:37:11 ntpd[22758]: Listen normally on 2 lo 127.0.0.1:123 2023-10-05T19:37:11.207 INFO:teuthology.orchestra.run.smithi012.stdout: 5 Oct 19:37:11 ntpd[22758]: Listen normally on 3 ens1f0 172.21.15.12:123 2023-10-05T19:37:11.207 INFO:teuthology.orchestra.run.smithi012.stdout: 5 Oct 19:37:11 ntpd[22758]: Listen normally on 4 lo [::1]:123 2023-10-05T19:37:11.208 INFO:teuthology.orchestra.run.smithi012.stdout: 5 Oct 19:37:11 ntpd[22758]: Listen normally on 5 ens1f0 [fe80::ae1f:6bff:fef8:25ea%3]:123 2023-10-05T19:37:11.208 INFO:teuthology.orchestra.run.smithi012.stdout: 5 Oct 19:37:11 ntpd[22758]: Listening on routing socket on fd #22 for interface updates 2023-10-05T19:37:12.176 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:12 ntpd[22742]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-05T19:37:12.176 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:12 ntpd[22742]: can't open /var/log/ntpstats/rawstats.20231005: Permission denied 2023-10-05T19:37:12.176 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:12 ntpd[22742]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-05T19:37:12.177 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:12 ntpd[22742]: can't open /var/log/ntpstats/rawstats.20231005: Permission denied 2023-10-05T19:37:12.177 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:12 ntpd[22742]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-05T19:37:12.177 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:12 ntpd[22742]: can't open /var/log/ntpstats/rawstats.20231005: Permission denied 2023-10-05T19:37:12.177 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:12 ntpd[22742]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-05T19:37:12.177 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:12 ntpd[22742]: can't open /var/log/ntpstats/peerstats.20231005: Permission denied 2023-10-05T19:37:12.177 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:12 ntpd[22742]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-05T19:37:12.177 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:12 ntpd[22742]: can't open /var/log/ntpstats/rawstats.20231005: Permission denied 2023-10-05T19:37:12.177 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:12 ntpd[22742]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-05T19:37:12.177 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:12 ntpd[22742]: can't open /var/log/ntpstats/peerstats.20231005: Permission denied 2023-10-05T19:37:12.205 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:12 ntpd[22758]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-05T19:37:12.205 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:12 ntpd[22758]: can't open /var/log/ntpstats/rawstats.20231005: Permission denied 2023-10-05T19:37:12.205 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:12 ntpd[22758]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-05T19:37:12.205 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:12 ntpd[22758]: can't open /var/log/ntpstats/rawstats.20231005: Permission denied 2023-10-05T19:37:12.205 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:12 ntpd[22758]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-05T19:37:12.206 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:12 ntpd[22758]: can't open /var/log/ntpstats/rawstats.20231005: Permission denied 2023-10-05T19:37:12.206 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:12 ntpd[22758]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-05T19:37:12.206 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:12 ntpd[22758]: can't open /var/log/ntpstats/peerstats.20231005: Permission denied 2023-10-05T19:37:12.206 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:12 ntpd[22758]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-05T19:37:12.206 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:12 ntpd[22758]: can't open /var/log/ntpstats/rawstats.20231005: Permission denied 2023-10-05T19:37:12.206 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:12 ntpd[22758]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-05T19:37:12.206 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:12 ntpd[22758]: can't open /var/log/ntpstats/peerstats.20231005: Permission denied 2023-10-05T19:37:13.204 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:13 ntpd[22758]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-05T19:37:13.204 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:13 ntpd[22758]: can't open /var/log/ntpstats/rawstats.20231005: Permission denied 2023-10-05T19:37:14.176 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:14 ntpd[22742]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-05T19:37:14.176 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:14 ntpd[22742]: can't open /var/log/ntpstats/rawstats.20231005: Permission denied 2023-10-05T19:37:14.176 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:14 ntpd[22742]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-05T19:37:14.176 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:14 ntpd[22742]: can't open /var/log/ntpstats/rawstats.20231005: Permission denied 2023-10-05T19:37:14.176 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:14 ntpd[22742]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-05T19:37:14.177 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:14 ntpd[22742]: can't open /var/log/ntpstats/rawstats.20231005: Permission denied 2023-10-05T19:37:14.177 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:14 ntpd[22742]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-05T19:37:14.177 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:14 ntpd[22742]: can't open /var/log/ntpstats/peerstats.20231005: Permission denied 2023-10-05T19:37:14.177 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:14 ntpd[22742]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-05T19:37:14.177 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:14 ntpd[22742]: can't open /var/log/ntpstats/rawstats.20231005: Permission denied 2023-10-05T19:37:14.177 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:14 ntpd[22742]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-05T19:37:14.177 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:14 ntpd[22742]: can't open /var/log/ntpstats/peerstats.20231005: Permission denied 2023-10-05T19:37:14.205 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:14 ntpd[22758]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-05T19:37:14.205 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:14 ntpd[22758]: can't open /var/log/ntpstats/rawstats.20231005: Permission denied 2023-10-05T19:37:14.205 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:14 ntpd[22758]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-05T19:37:14.206 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:14 ntpd[22758]: can't open /var/log/ntpstats/rawstats.20231005: Permission denied 2023-10-05T19:37:14.206 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:14 ntpd[22758]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-05T19:37:14.206 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:14 ntpd[22758]: can't open /var/log/ntpstats/rawstats.20231005: Permission denied 2023-10-05T19:37:14.206 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:14 ntpd[22758]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-05T19:37:14.206 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:14 ntpd[22758]: can't open /var/log/ntpstats/peerstats.20231005: Permission denied 2023-10-05T19:37:14.206 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:14 ntpd[22758]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-05T19:37:14.206 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:14 ntpd[22758]: can't open /var/log/ntpstats/rawstats.20231005: Permission denied 2023-10-05T19:37:14.206 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:14 ntpd[22758]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-05T19:37:14.206 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:14 ntpd[22758]: can't open /var/log/ntpstats/peerstats.20231005: Permission denied 2023-10-05T19:37:15.205 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:15 ntpd[22758]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-05T19:37:15.205 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:15 ntpd[22758]: can't open /var/log/ntpstats/rawstats.20231005: Permission denied 2023-10-05T19:37:15.205 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:15 ntpd[22758]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-05T19:37:15.205 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:15 ntpd[22758]: can't open /var/log/ntpstats/rawstats.20231005: Permission denied 2023-10-05T19:37:15.205 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:15 ntpd[22758]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-05T19:37:15.205 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:15 ntpd[22758]: can't open /var/log/ntpstats/peerstats.20231005: Permission denied 2023-10-05T19:37:16.177 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:16 ntpd[22742]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-05T19:37:16.177 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:16 ntpd[22742]: can't open /var/log/ntpstats/rawstats.20231005: Permission denied 2023-10-05T19:37:16.177 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:16 ntpd[22742]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-05T19:37:16.177 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:16 ntpd[22742]: can't open /var/log/ntpstats/rawstats.20231005: Permission denied 2023-10-05T19:37:16.177 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:16 ntpd[22742]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-05T19:37:16.178 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:16 ntpd[22742]: can't open /var/log/ntpstats/rawstats.20231005: Permission denied 2023-10-05T19:37:16.178 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:16 ntpd[22742]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-05T19:37:16.178 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:16 ntpd[22742]: can't open /var/log/ntpstats/rawstats.20231005: Permission denied 2023-10-05T19:37:16.178 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:16 ntpd[22742]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-05T19:37:16.178 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:16 ntpd[22742]: can't open /var/log/ntpstats/rawstats.20231005: Permission denied 2023-10-05T19:37:16.178 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:16 ntpd[22742]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-05T19:37:16.178 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:16 ntpd[22742]: can't open /var/log/ntpstats/peerstats.20231005: Permission denied 2023-10-05T19:37:16.178 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:16 ntpd[22742]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-05T19:37:16.178 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:16 ntpd[22742]: can't open /var/log/ntpstats/rawstats.20231005: Permission denied 2023-10-05T19:37:16.178 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:16 ntpd[22742]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-05T19:37:16.178 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:16 ntpd[22742]: can't open /var/log/ntpstats/peerstats.20231005: Permission denied 2023-10-05T19:37:16.179 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:16 ntpd[22742]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-05T19:37:16.179 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:16 ntpd[22742]: can't open /var/log/ntpstats/rawstats.20231005: Permission denied 2023-10-05T19:37:16.179 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:16 ntpd[22742]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-05T19:37:16.179 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:16 ntpd[22742]: can't open /var/log/ntpstats/peerstats.20231005: Permission denied 2023-10-05T19:37:16.205 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:16 ntpd[22758]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-05T19:37:16.205 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:16 ntpd[22758]: can't open /var/log/ntpstats/rawstats.20231005: Permission denied 2023-10-05T19:37:16.205 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:16 ntpd[22758]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-05T19:37:16.205 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:16 ntpd[22758]: can't open /var/log/ntpstats/rawstats.20231005: Permission denied 2023-10-05T19:37:16.206 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:16 ntpd[22758]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-05T19:37:16.206 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:16 ntpd[22758]: can't open /var/log/ntpstats/rawstats.20231005: Permission denied 2023-10-05T19:37:16.206 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:16 ntpd[22758]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-05T19:37:16.206 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:16 ntpd[22758]: can't open /var/log/ntpstats/peerstats.20231005: Permission denied 2023-10-05T19:37:16.206 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:16 ntpd[22758]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-05T19:37:16.206 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:16 ntpd[22758]: can't open /var/log/ntpstats/rawstats.20231005: Permission denied 2023-10-05T19:37:16.206 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:16 ntpd[22758]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-05T19:37:16.206 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:16 ntpd[22758]: can't open /var/log/ntpstats/peerstats.20231005: Permission denied 2023-10-05T19:37:17.205 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:17 ntpd[22758]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-05T19:37:17.205 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:17 ntpd[22758]: can't open /var/log/ntpstats/rawstats.20231005: Permission denied 2023-10-05T19:37:17.206 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:17 ntpd[22758]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-05T19:37:17.206 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:17 ntpd[22758]: can't open /var/log/ntpstats/rawstats.20231005: Permission denied 2023-10-05T19:37:17.206 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:17 ntpd[22758]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-05T19:37:17.206 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:17 ntpd[22758]: can't open /var/log/ntpstats/peerstats.20231005: Permission denied 2023-10-05T19:37:18.176 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:18 ntpd[22742]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-05T19:37:18.177 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:18 ntpd[22742]: can't open /var/log/ntpstats/rawstats.20231005: Permission denied 2023-10-05T19:37:18.177 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:18 ntpd[22742]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-05T19:37:18.177 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:18 ntpd[22742]: can't open /var/log/ntpstats/rawstats.20231005: Permission denied 2023-10-05T19:37:18.177 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:18 ntpd[22742]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-05T19:37:18.177 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:18 ntpd[22742]: can't open /var/log/ntpstats/rawstats.20231005: Permission denied 2023-10-05T19:37:18.177 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:18 ntpd[22742]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-05T19:37:18.177 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:18 ntpd[22742]: can't open /var/log/ntpstats/rawstats.20231005: Permission denied 2023-10-05T19:37:18.177 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:18 ntpd[22742]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-05T19:37:18.177 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:18 ntpd[22742]: can't open /var/log/ntpstats/peerstats.20231005: Permission denied 2023-10-05T19:37:18.178 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:18 ntpd[22742]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-05T19:37:18.178 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:18 ntpd[22742]: can't open /var/log/ntpstats/rawstats.20231005: Permission denied 2023-10-05T19:37:18.178 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:18 ntpd[22742]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-05T19:37:18.178 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:18 ntpd[22742]: can't open /var/log/ntpstats/peerstats.20231005: Permission denied 2023-10-05T19:37:18.178 INFO:teuthology.orchestra.run.smithi008.stdout: 5 Oct 19:37:18 ntpd[22742]: ntpd: time slew +0.002209 s 2023-10-05T19:37:18.178 INFO:teuthology.orchestra.run.smithi008.stdout:ntpd: time slew +0.002209s 2023-10-05T19:37:18.179 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:18 ntpd[22742]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-10-05T19:37:18.179 INFO:teuthology.orchestra.run.smithi008.stderr: 5 Oct 19:37:18 ntpd[22742]: can't open /var/log/ntpstats/loopstats.20231005: Permission denied 2023-10-05T19:37:18.205 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:18 ntpd[22758]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-05T19:37:18.205 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:18 ntpd[22758]: can't open /var/log/ntpstats/rawstats.20231005: Permission denied 2023-10-05T19:37:18.205 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:18 ntpd[22758]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-05T19:37:18.205 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:18 ntpd[22758]: can't open /var/log/ntpstats/rawstats.20231005: Permission denied 2023-10-05T19:37:18.205 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:18 ntpd[22758]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-05T19:37:18.205 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:18 ntpd[22758]: can't open /var/log/ntpstats/rawstats.20231005: Permission denied 2023-10-05T19:37:18.206 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:18 ntpd[22758]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-05T19:37:18.206 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:18 ntpd[22758]: can't open /var/log/ntpstats/peerstats.20231005: Permission denied 2023-10-05T19:37:18.206 INFO:teuthology.orchestra.run.smithi012.stdout: 5 Oct 19:37:18 ntpd[22758]: ntpd: time slew +0.001971 s 2023-10-05T19:37:18.206 INFO:teuthology.orchestra.run.smithi012.stdout:ntpd: time slew +0.001971s 2023-10-05T19:37:18.206 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:18 ntpd[22758]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-10-05T19:37:18.207 INFO:teuthology.orchestra.run.smithi012.stderr: 5 Oct 19:37:18 ntpd[22758]: can't open /var/log/ntpstats/loopstats.20231005: Permission denied 2023-10-05T19:37:18.246 INFO:teuthology.orchestra.run.smithi008.stdout: remote refid st t when poll reach delay offset jitter 2023-10-05T19:37:18.247 INFO:teuthology.orchestra.run.smithi008.stdout:============================================================================== 2023-10-05T19:37:18.247 INFO:teuthology.orchestra.run.smithi008.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-05T19:37:18.247 INFO:teuthology.orchestra.run.smithi008.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-05T19:37:18.247 INFO:teuthology.orchestra.run.smithi008.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-05T19:37:18.247 INFO:teuthology.orchestra.run.smithi008.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-05T19:37:18.297 INFO:teuthology.orchestra.run.smithi012.stdout: remote refid st t when poll reach delay offset jitter 2023-10-05T19:37:18.298 INFO:teuthology.orchestra.run.smithi012.stdout:============================================================================== 2023-10-05T19:37:18.298 INFO:teuthology.orchestra.run.smithi012.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-05T19:37:18.298 INFO:teuthology.orchestra.run.smithi012.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-05T19:37:18.298 INFO:teuthology.orchestra.run.smithi012.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-05T19:37:18.298 INFO:teuthology.orchestra.run.smithi012.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-05T19:37:18.299 INFO:teuthology.run_tasks:Running task install... 2023-10-05T19:37:18.312 DEBUG:teuthology.task.install:project ceph 2023-10-05T19:37:18.312 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '487ffcb5ddb60d29c5688d1a457e812b1b697b90'}} 2023-10-05T19:37:18.312 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '487ffcb5ddb60d29c5688d1a457e812b1b697b90'} 2023-10-05T19:37:18.312 INFO:teuthology.task.install:Using flavor: default 2023-10-05T19:37:18.318 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-05T19:37:18.318 INFO:teuthology.task.install:extra packages: [] 2023-10-05T19:37:18.319 DEBUG:teuthology.orchestra.run.smithi008:> sudo apt-key list | grep Ceph 2023-10-05T19:37:18.319 DEBUG:teuthology.orchestra.run.smithi012:> sudo apt-key list | grep Ceph 2023-10-05T19:37:18.419 INFO:teuthology.orchestra.run.smithi012.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2023-10-05T19:37:18.423 INFO:teuthology.orchestra.run.smithi008.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2023-10-05T19:37:18.481 INFO:teuthology.orchestra.run.smithi012.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-10-05T19:37:18.482 INFO:teuthology.orchestra.run.smithi012.stdout:uid [ unknown] Ceph.com (release key) 2023-10-05T19:37:18.483 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2023-10-05T19:37:18.483 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=487ffcb5ddb60d29c5688d1a457e812b1b697b90 2023-10-05T19:37:18.488 INFO:teuthology.orchestra.run.smithi008.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-10-05T19:37:18.488 INFO:teuthology.orchestra.run.smithi008.stdout:uid [ unknown] Ceph.com (release key) 2023-10-05T19:37:18.488 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2023-10-05T19:37:18.489 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=487ffcb5ddb60d29c5688d1a457e812b1b697b90 2023-10-05T19:37:18.628 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default/ 2023-10-05T19:37:18.629 INFO:teuthology.task.install.deb:Package version is 17.2.6-1278-g487ffcb5-1focal 2023-10-05T19:37:18.633 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default/ 2023-10-05T19:37:18.633 INFO:teuthology.task.install.deb:Package version is 17.2.6-1278-g487ffcb5-1focal 2023-10-05T19:37:18.739 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-05T19:37:18.739 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-10-05T19:37:18.742 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-10-05T19:37:18.742 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-10-05T19:37:18.756 DEBUG:teuthology.orchestra.run.smithi012:> sudo apt-get update 2023-10-05T19:37:18.757 DEBUG:teuthology.orchestra.run.smithi008:> sudo apt-get update 2023-10-05T19:37:18.892 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-10-05T19:37:18.892 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2023-10-05T19:37:18.910 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-10-05T19:37:18.929 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-10-05T19:37:18.951 INFO:teuthology.orchestra.run.smithi008.stdout:Ign:1 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal InRelease 2023-10-05T19:37:18.972 INFO:teuthology.orchestra.run.smithi008.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal Release [7,662 B] 2023-10-05T19:37:18.996 INFO:teuthology.orchestra.run.smithi008.stdout:Ign:3 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal Release.gpg 2023-10-05T19:37:19.011 INFO:teuthology.orchestra.run.smithi012.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal InRelease 2023-10-05T19:37:19.027 INFO:teuthology.orchestra.run.smithi008.stdout:Hit:4 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-10-05T19:37:19.027 INFO:teuthology.orchestra.run.smithi008.stdout:Hit:5 http://archive.ubuntu.com/ubuntu focal InRelease 2023-10-05T19:37:19.027 INFO:teuthology.orchestra.run.smithi008.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main i386 Packages [3,372 B] 2023-10-05T19:37:19.048 INFO:teuthology.orchestra.run.smithi012.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal Release [7,662 B] 2023-10-05T19:37:19.067 INFO:teuthology.orchestra.run.smithi008.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2023-10-05T19:37:19.086 INFO:teuthology.orchestra.run.smithi012.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal Release.gpg 2023-10-05T19:37:19.097 INFO:teuthology.orchestra.run.smithi008.stdout:Hit:8 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-10-05T19:37:19.123 INFO:teuthology.orchestra.run.smithi012.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main i386 Packages [3,372 B] 2023-10-05T19:37:19.174 INFO:teuthology.orchestra.run.smithi012.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2023-10-05T19:37:19.179 INFO:teuthology.orchestra.run.smithi008.stdout:Hit:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-10-05T19:37:19.574 INFO:teuthology.orchestra.run.smithi012.stdout:Fetched 28.4 kB in 1s (43.9 kB/s) 2023-10-05T19:37:19.713 INFO:teuthology.orchestra.run.smithi008.stdout:Fetched 28.4 kB in 1s (35.7 kB/s) 2023-10-05T19:37:20.650 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2023-10-05T19:37:20.676 DEBUG:teuthology.orchestra.run.smithi012:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.6-1278-g487ffcb5-1focal cephadm=17.2.6-1278-g487ffcb5-1focal ceph-mds=17.2.6-1278-g487ffcb5-1focal ceph-mgr=17.2.6-1278-g487ffcb5-1focal ceph-common=17.2.6-1278-g487ffcb5-1focal ceph-fuse=17.2.6-1278-g487ffcb5-1focal ceph-test=17.2.6-1278-g487ffcb5-1focal ceph-volume=17.2.6-1278-g487ffcb5-1focal radosgw=17.2.6-1278-g487ffcb5-1focal python3-rados=17.2.6-1278-g487ffcb5-1focal python3-rgw=17.2.6-1278-g487ffcb5-1focal python3-cephfs=17.2.6-1278-g487ffcb5-1focal python3-rbd=17.2.6-1278-g487ffcb5-1focal libcephfs2=17.2.6-1278-g487ffcb5-1focal libcephfs-dev=17.2.6-1278-g487ffcb5-1focal librados2=17.2.6-1278-g487ffcb5-1focal librbd1=17.2.6-1278-g487ffcb5-1focal rbd-fuse=17.2.6-1278-g487ffcb5-1focal 2023-10-05T19:37:20.750 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2023-10-05T19:37:20.785 INFO:teuthology.orchestra.run.smithi008.stdout:Reading package lists... 2023-10-05T19:37:20.812 DEBUG:teuthology.orchestra.run.smithi008:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.6-1278-g487ffcb5-1focal cephadm=17.2.6-1278-g487ffcb5-1focal ceph-mds=17.2.6-1278-g487ffcb5-1focal ceph-mgr=17.2.6-1278-g487ffcb5-1focal ceph-common=17.2.6-1278-g487ffcb5-1focal ceph-fuse=17.2.6-1278-g487ffcb5-1focal ceph-test=17.2.6-1278-g487ffcb5-1focal ceph-volume=17.2.6-1278-g487ffcb5-1focal radosgw=17.2.6-1278-g487ffcb5-1focal python3-rados=17.2.6-1278-g487ffcb5-1focal python3-rgw=17.2.6-1278-g487ffcb5-1focal python3-cephfs=17.2.6-1278-g487ffcb5-1focal python3-rbd=17.2.6-1278-g487ffcb5-1focal libcephfs2=17.2.6-1278-g487ffcb5-1focal libcephfs-dev=17.2.6-1278-g487ffcb5-1focal librados2=17.2.6-1278-g487ffcb5-1focal librbd1=17.2.6-1278-g487ffcb5-1focal rbd-fuse=17.2.6-1278-g487ffcb5-1focal 2023-10-05T19:37:20.880 INFO:teuthology.orchestra.run.smithi008.stdout:Reading package lists... 2023-10-05T19:37:20.956 INFO:teuthology.orchestra.run.smithi012.stdout:Building dependency tree... 2023-10-05T19:37:20.956 INFO:teuthology.orchestra.run.smithi012.stdout:Reading state information... 2023-10-05T19:37:21.085 INFO:teuthology.orchestra.run.smithi008.stdout:Building dependency tree... 2023-10-05T19:37:21.087 INFO:teuthology.orchestra.run.smithi008.stdout:Reading state information... 2023-10-05T19:37:21.147 INFO:teuthology.orchestra.run.smithi012.stdout:The following packages were automatically installed and are no longer required: 2023-10-05T19:37:21.149 INFO:teuthology.orchestra.run.smithi012.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-10-05T19:37:21.149 INFO:teuthology.orchestra.run.smithi012.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-05T19:37:21.149 INFO:teuthology.orchestra.run.smithi012.stdout:The following additional packages will be installed: 2023-10-05T19:37:21.149 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-10-05T19:37:21.149 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2023-10-05T19:37:21.149 INFO:teuthology.orchestra.run.smithi012.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2023-10-05T19:37:21.149 INFO:teuthology.orchestra.run.smithi012.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2023-10-05T19:37:21.150 INFO:teuthology.orchestra.run.smithi012.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2023-10-05T19:37:21.150 INFO:teuthology.orchestra.run.smithi012.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2023-10-05T19:37:21.150 INFO:teuthology.orchestra.run.smithi012.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2023-10-05T19:37:21.151 INFO:teuthology.orchestra.run.smithi012.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2023-10-05T19:37:21.151 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 2023-10-05T19:37:21.151 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2023-10-05T19:37:21.151 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2023-10-05T19:37:21.151 INFO:teuthology.orchestra.run.smithi012.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2023-10-05T19:37:21.151 INFO:teuthology.orchestra.run.smithi012.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2023-10-05T19:37:21.152 INFO:teuthology.orchestra.run.smithi012.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-10-05T19:37:21.152 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2023-10-05T19:37:21.152 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyasn1-modules python3-pyinotify python3-pytest python3-repoze.lru 2023-10-05T19:37:21.152 INFO:teuthology.orchestra.run.smithi012.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-scipy 2023-10-05T19:37:21.152 INFO:teuthology.orchestra.run.smithi012.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2023-10-05T19:37:21.153 INFO:teuthology.orchestra.run.smithi012.stdout: python3-sklearn python3-sklearn-lib python3-soupsieve python3-tempita 2023-10-05T19:37:21.153 INFO:teuthology.orchestra.run.smithi012.stdout: python3-waitress python3-wcwidth python3-webencodings python3-webob 2023-10-05T19:37:21.153 INFO:teuthology.orchestra.run.smithi012.stdout: python3-websocket python3-webtest python3-werkzeug qttranslations5-l10n 2023-10-05T19:37:21.153 INFO:teuthology.orchestra.run.smithi012.stdout: socat xmlstarlet 2023-10-05T19:37:21.153 INFO:teuthology.orchestra.run.smithi012.stdout:Suggested packages: 2023-10-05T19:37:21.154 INFO:teuthology.orchestra.run.smithi012.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2023-10-05T19:37:21.154 INFO:teuthology.orchestra.run.smithi012.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2023-10-05T19:37:21.154 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2023-10-05T19:37:21.154 INFO:teuthology.orchestra.run.smithi012.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2023-10-05T19:37:21.154 INFO:teuthology.orchestra.run.smithi012.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2023-10-05T19:37:21.154 INFO:teuthology.orchestra.run.smithi012.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2023-10-05T19:37:21.155 INFO:teuthology.orchestra.run.smithi012.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2023-10-05T19:37:21.155 INFO:teuthology.orchestra.run.smithi012.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2023-10-05T19:37:21.155 INFO:teuthology.orchestra.run.smithi012.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2023-10-05T19:37:21.155 INFO:teuthology.orchestra.run.smithi012.stdout:Recommended packages: 2023-10-05T19:37:21.155 INFO:teuthology.orchestra.run.smithi012.stdout: btrfs-tools 2023-10-05T19:37:21.269 INFO:teuthology.orchestra.run.smithi012.stdout:The following NEW packages will be installed: 2023-10-05T19:37:21.269 INFO:teuthology.orchestra.run.smithi012.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-10-05T19:37:21.269 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-10-05T19:37:21.269 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-10-05T19:37:21.269 INFO:teuthology.orchestra.run.smithi012.stdout: fonts-mathjax jq libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 2023-10-05T19:37:21.269 INFO:teuthology.orchestra.run.smithi012.stdout: libjs-mathjax libjs-sphinxdoc libjs-underscore liblbfgsb0 liblttng-ust0 2023-10-05T19:37:21.269 INFO:teuthology.orchestra.run.smithi012.stdout: liblua5.3-0 liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2023-10-05T19:37:21.269 INFO:teuthology.orchestra.run.smithi012.stdout: libqt5network5 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2023-10-05T19:37:21.270 INFO:teuthology.orchestra.run.smithi012.stdout: libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli python-natsort-doc 2023-10-05T19:37:21.270 INFO:teuthology.orchestra.run.smithi012.stdout: python-pastedeploy-tpl python3-atomicwrites python3-attr python3-bcrypt 2023-10-05T19:37:21.271 INFO:teuthology.orchestra.run.smithi012.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2023-10-05T19:37:21.271 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cryptography 2023-10-05T19:37:21.271 INFO:teuthology.orchestra.run.smithi012.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2023-10-05T19:37:21.271 INFO:teuthology.orchestra.run.smithi012.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2023-10-05T19:37:21.271 INFO:teuthology.orchestra.run.smithi012.stdout: python3-mako python3-markupsafe python3-natsort python3-oauthlib 2023-10-05T19:37:21.271 INFO:teuthology.orchestra.run.smithi012.stdout: python3-openssl python3-packaging python3-paste python3-pastedeploy 2023-10-05T19:37:21.271 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pastescript python3-pecan python3-pluggy python3-prettytable 2023-10-05T19:37:21.271 INFO:teuthology.orchestra.run.smithi012.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2023-10-05T19:37:21.271 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyinotify python3-pytest python3-rados python3-rbd 2023-10-05T19:37:21.271 INFO:teuthology.orchestra.run.smithi012.stdout: python3-repoze.lru python3-requests-oauthlib python3-rgw python3-routes 2023-10-05T19:37:21.272 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2023-10-05T19:37:21.272 INFO:teuthology.orchestra.run.smithi012.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-10-05T19:37:21.272 INFO:teuthology.orchestra.run.smithi012.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-10-05T19:37:21.272 INFO:teuthology.orchestra.run.smithi012.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-10-05T19:37:21.272 INFO:teuthology.orchestra.run.smithi012.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2023-10-05T19:37:21.272 INFO:teuthology.orchestra.run.smithi012.stdout:The following packages will be upgraded: 2023-10-05T19:37:21.272 INFO:teuthology.orchestra.run.smithi012.stdout: librados2 librbd1 2023-10-05T19:37:21.273 INFO:teuthology.orchestra.run.smithi008.stdout:The following packages were automatically installed and are no longer required: 2023-10-05T19:37:21.274 INFO:teuthology.orchestra.run.smithi008.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-10-05T19:37:21.274 INFO:teuthology.orchestra.run.smithi008.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-05T19:37:21.274 INFO:teuthology.orchestra.run.smithi008.stdout:The following additional packages will be installed: 2023-10-05T19:37:21.274 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-10-05T19:37:21.275 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2023-10-05T19:37:21.275 INFO:teuthology.orchestra.run.smithi008.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2023-10-05T19:37:21.275 INFO:teuthology.orchestra.run.smithi008.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2023-10-05T19:37:21.276 INFO:teuthology.orchestra.run.smithi008.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2023-10-05T19:37:21.276 INFO:teuthology.orchestra.run.smithi008.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2023-10-05T19:37:21.276 INFO:teuthology.orchestra.run.smithi008.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2023-10-05T19:37:21.276 INFO:teuthology.orchestra.run.smithi008.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2023-10-05T19:37:21.276 INFO:teuthology.orchestra.run.smithi008.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 2023-10-05T19:37:21.276 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2023-10-05T19:37:21.276 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2023-10-05T19:37:21.276 INFO:teuthology.orchestra.run.smithi008.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2023-10-05T19:37:21.276 INFO:teuthology.orchestra.run.smithi008.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2023-10-05T19:37:21.276 INFO:teuthology.orchestra.run.smithi008.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-10-05T19:37:21.276 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2023-10-05T19:37:21.276 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pyasn1-modules python3-pyinotify python3-pytest python3-repoze.lru 2023-10-05T19:37:21.277 INFO:teuthology.orchestra.run.smithi008.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-scipy 2023-10-05T19:37:21.277 INFO:teuthology.orchestra.run.smithi008.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2023-10-05T19:37:21.277 INFO:teuthology.orchestra.run.smithi008.stdout: python3-sklearn python3-sklearn-lib python3-soupsieve python3-tempita 2023-10-05T19:37:21.277 INFO:teuthology.orchestra.run.smithi008.stdout: python3-waitress python3-wcwidth python3-webencodings python3-webob 2023-10-05T19:37:21.277 INFO:teuthology.orchestra.run.smithi008.stdout: python3-websocket python3-webtest python3-werkzeug qttranslations5-l10n 2023-10-05T19:37:21.277 INFO:teuthology.orchestra.run.smithi008.stdout: socat xmlstarlet 2023-10-05T19:37:21.277 INFO:teuthology.orchestra.run.smithi008.stdout:Suggested packages: 2023-10-05T19:37:21.277 INFO:teuthology.orchestra.run.smithi008.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2023-10-05T19:37:21.277 INFO:teuthology.orchestra.run.smithi008.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2023-10-05T19:37:21.278 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2023-10-05T19:37:21.278 INFO:teuthology.orchestra.run.smithi008.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2023-10-05T19:37:21.278 INFO:teuthology.orchestra.run.smithi008.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2023-10-05T19:37:21.278 INFO:teuthology.orchestra.run.smithi008.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2023-10-05T19:37:21.278 INFO:teuthology.orchestra.run.smithi008.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2023-10-05T19:37:21.278 INFO:teuthology.orchestra.run.smithi008.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2023-10-05T19:37:21.278 INFO:teuthology.orchestra.run.smithi008.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2023-10-05T19:37:21.278 INFO:teuthology.orchestra.run.smithi008.stdout:Recommended packages: 2023-10-05T19:37:21.278 INFO:teuthology.orchestra.run.smithi008.stdout: btrfs-tools 2023-10-05T19:37:21.397 INFO:teuthology.orchestra.run.smithi008.stdout:The following NEW packages will be installed: 2023-10-05T19:37:21.398 INFO:teuthology.orchestra.run.smithi008.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-10-05T19:37:21.398 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-10-05T19:37:21.398 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-10-05T19:37:21.398 INFO:teuthology.orchestra.run.smithi008.stdout: fonts-mathjax jq libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 2023-10-05T19:37:21.398 INFO:teuthology.orchestra.run.smithi008.stdout: libjs-mathjax libjs-sphinxdoc libjs-underscore liblbfgsb0 liblttng-ust0 2023-10-05T19:37:21.398 INFO:teuthology.orchestra.run.smithi008.stdout: liblua5.3-0 liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2023-10-05T19:37:21.398 INFO:teuthology.orchestra.run.smithi008.stdout: libqt5network5 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2023-10-05T19:37:21.399 INFO:teuthology.orchestra.run.smithi008.stdout: libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli python-natsort-doc 2023-10-05T19:37:21.399 INFO:teuthology.orchestra.run.smithi008.stdout: python-pastedeploy-tpl python3-atomicwrites python3-attr python3-bcrypt 2023-10-05T19:37:21.399 INFO:teuthology.orchestra.run.smithi008.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2023-10-05T19:37:21.399 INFO:teuthology.orchestra.run.smithi008.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cryptography 2023-10-05T19:37:21.399 INFO:teuthology.orchestra.run.smithi008.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2023-10-05T19:37:21.399 INFO:teuthology.orchestra.run.smithi008.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2023-10-05T19:37:21.400 INFO:teuthology.orchestra.run.smithi008.stdout: python3-mako python3-markupsafe python3-natsort python3-oauthlib 2023-10-05T19:37:21.400 INFO:teuthology.orchestra.run.smithi008.stdout: python3-openssl python3-packaging python3-paste python3-pastedeploy 2023-10-05T19:37:21.400 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pastescript python3-pecan python3-pluggy python3-prettytable 2023-10-05T19:37:21.400 INFO:teuthology.orchestra.run.smithi008.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2023-10-05T19:37:21.400 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pyinotify python3-pytest python3-rados python3-rbd 2023-10-05T19:37:21.400 INFO:teuthology.orchestra.run.smithi008.stdout: python3-repoze.lru python3-requests-oauthlib python3-rgw python3-routes 2023-10-05T19:37:21.401 INFO:teuthology.orchestra.run.smithi008.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2023-10-05T19:37:21.401 INFO:teuthology.orchestra.run.smithi008.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-10-05T19:37:21.401 INFO:teuthology.orchestra.run.smithi008.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-10-05T19:37:21.401 INFO:teuthology.orchestra.run.smithi008.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-10-05T19:37:21.401 INFO:teuthology.orchestra.run.smithi008.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2023-10-05T19:37:21.401 INFO:teuthology.orchestra.run.smithi008.stdout:The following packages will be upgraded: 2023-10-05T19:37:21.401 INFO:teuthology.orchestra.run.smithi008.stdout: librados2 librbd1 2023-10-05T19:37:21.420 INFO:teuthology.orchestra.run.smithi012.stdout:2 upgraded, 108 newly installed, 0 to remove and 253 not upgraded. 2023-10-05T19:37:21.420 INFO:teuthology.orchestra.run.smithi012.stdout:Need to get 210 MB of archives. 2023-10-05T19:37:21.420 INFO:teuthology.orchestra.run.smithi012.stdout:After this operation, 980 MB of additional disk space will be used. 2023-10-05T19:37:21.420 INFO:teuthology.orchestra.run.smithi012.stdout:Get:1 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.6-1278-g487ffcb5-1focal [3,501 kB] 2023-10-05T19:37:21.463 INFO:teuthology.orchestra.run.smithi012.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2023-10-05T19:37:21.592 INFO:teuthology.orchestra.run.smithi008.stdout:2 upgraded, 108 newly installed, 0 to remove and 253 not upgraded. 2023-10-05T19:37:21.592 INFO:teuthology.orchestra.run.smithi008.stdout:Need to get 210 MB of archives. 2023-10-05T19:37:21.593 INFO:teuthology.orchestra.run.smithi008.stdout:After this operation, 980 MB of additional disk space will be used. 2023-10-05T19:37:21.593 INFO:teuthology.orchestra.run.smithi008.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2023-10-05T19:37:21.638 INFO:teuthology.orchestra.run.smithi008.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.6-1278-g487ffcb5-1focal [3,501 kB] 2023-10-05T19:37:21.646 INFO:teuthology.orchestra.run.smithi012.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.6-1278-g487ffcb5-1focal [3,555 kB] 2023-10-05T19:37:21.759 INFO:teuthology.orchestra.run.smithi012.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.6-1278-g487ffcb5-1focal [760 kB] 2023-10-05T19:37:21.786 INFO:teuthology.orchestra.run.smithi012.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.6-1278-g487ffcb5-1focal [412 kB] 2023-10-05T19:37:21.800 INFO:teuthology.orchestra.run.smithi012.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.6-1278-g487ffcb5-1focal [22.5 kB] 2023-10-05T19:37:21.800 INFO:teuthology.orchestra.run.smithi012.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.6-1278-g487ffcb5-1focal [200 kB] 2023-10-05T19:37:21.807 INFO:teuthology.orchestra.run.smithi012.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.6-1278-g487ffcb5-1focal [47.2 kB] 2023-10-05T19:37:21.809 INFO:teuthology.orchestra.run.smithi012.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.6-1278-g487ffcb5-1focal [386 kB] 2023-10-05T19:37:21.821 INFO:teuthology.orchestra.run.smithi012.stdout:Get:10 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.6-1278-g487ffcb5-1focal [5,577 kB] 2023-10-05T19:37:21.859 INFO:teuthology.orchestra.run.smithi012.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2023-10-05T19:37:21.866 INFO:teuthology.orchestra.run.smithi012.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2023-10-05T19:37:21.910 INFO:teuthology.orchestra.run.smithi012.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2023-10-05T19:37:21.923 INFO:teuthology.orchestra.run.smithi012.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2023-10-05T19:37:21.987 INFO:teuthology.orchestra.run.smithi008.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2023-10-05T19:37:21.994 INFO:teuthology.orchestra.run.smithi008.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2023-10-05T19:37:22.015 INFO:teuthology.orchestra.run.smithi012.stdout:Get:15 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.6-1278-g487ffcb5-1focal [116 kB] 2023-10-05T19:37:22.022 INFO:teuthology.orchestra.run.smithi012.stdout:Get:16 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.6-1278-g487ffcb5-1focal [479 kB] 2023-10-05T19:37:22.037 INFO:teuthology.orchestra.run.smithi012.stdout:Get:17 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.6-1278-g487ffcb5-1focal [27.7 MB] 2023-10-05T19:37:22.038 INFO:teuthology.orchestra.run.smithi008.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2023-10-05T19:37:22.050 INFO:teuthology.orchestra.run.smithi012.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2023-10-05T19:37:22.051 INFO:teuthology.orchestra.run.smithi008.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2023-10-05T19:37:22.055 INFO:teuthology.orchestra.run.smithi012.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2023-10-05T19:37:22.085 INFO:teuthology.orchestra.run.smithi012.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2023-10-05T19:37:22.178 INFO:teuthology.orchestra.run.smithi008.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2023-10-05T19:37:22.184 INFO:teuthology.orchestra.run.smithi008.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2023-10-05T19:37:22.213 INFO:teuthology.orchestra.run.smithi008.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2023-10-05T19:37:22.229 INFO:teuthology.orchestra.run.smithi012.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2023-10-05T19:37:22.237 INFO:teuthology.orchestra.run.smithi012.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2023-10-05T19:37:22.260 INFO:teuthology.orchestra.run.smithi012.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2023-10-05T19:37:22.355 INFO:teuthology.orchestra.run.smithi008.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2023-10-05T19:37:22.362 INFO:teuthology.orchestra.run.smithi008.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2023-10-05T19:37:22.386 INFO:teuthology.orchestra.run.smithi008.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2023-10-05T19:37:22.431 INFO:teuthology.orchestra.run.smithi012.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2023-10-05T19:37:22.557 INFO:teuthology.orchestra.run.smithi008.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2023-10-05T19:37:22.677 INFO:teuthology.orchestra.run.smithi012.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2023-10-05T19:37:22.804 INFO:teuthology.orchestra.run.smithi008.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2023-10-05T19:37:22.870 INFO:teuthology.orchestra.run.smithi012.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2023-10-05T19:37:22.890 INFO:teuthology.orchestra.run.smithi012.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2023-10-05T19:37:22.904 INFO:teuthology.orchestra.run.smithi012.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2023-10-05T19:37:22.955 INFO:teuthology.orchestra.run.smithi012.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2023-10-05T19:37:22.962 INFO:teuthology.orchestra.run.smithi012.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2023-10-05T19:37:22.966 INFO:teuthology.orchestra.run.smithi012.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2023-10-05T19:37:22.969 INFO:teuthology.orchestra.run.smithi012.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2023-10-05T19:37:22.973 INFO:teuthology.orchestra.run.smithi012.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2023-10-05T19:37:22.984 INFO:teuthology.orchestra.run.smithi012.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-10-05T19:37:22.998 INFO:teuthology.orchestra.run.smithi008.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2023-10-05T19:37:23.020 INFO:teuthology.orchestra.run.smithi008.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2023-10-05T19:37:23.031 INFO:teuthology.orchestra.run.smithi008.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2023-10-05T19:37:23.083 INFO:teuthology.orchestra.run.smithi008.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2023-10-05T19:37:23.084 INFO:teuthology.orchestra.run.smithi012.stdout:Get:35 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.6-1278-g487ffcb5-1focal [6,450 kB] 2023-10-05T19:37:23.090 INFO:teuthology.orchestra.run.smithi008.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2023-10-05T19:37:23.094 INFO:teuthology.orchestra.run.smithi008.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2023-10-05T19:37:23.098 INFO:teuthology.orchestra.run.smithi008.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2023-10-05T19:37:23.101 INFO:teuthology.orchestra.run.smithi008.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2023-10-05T19:37:23.112 INFO:teuthology.orchestra.run.smithi008.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-10-05T19:37:23.150 INFO:teuthology.orchestra.run.smithi012.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2023-10-05T19:37:23.234 INFO:teuthology.orchestra.run.smithi012.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2023-10-05T19:37:23.252 INFO:teuthology.orchestra.run.smithi012.stdout:Get:38 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.6-1278-g487ffcb5-1focal [221 kB] 2023-10-05T19:37:23.259 INFO:teuthology.orchestra.run.smithi012.stdout:Get:39 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.6-1278-g487ffcb5-1focal [152 kB] 2023-10-05T19:37:23.263 INFO:teuthology.orchestra.run.smithi012.stdout:Get:40 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.6-1278-g487ffcb5-1focal [1,591 kB] 2023-10-05T19:37:23.279 INFO:teuthology.orchestra.run.smithi008.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2023-10-05T19:37:23.289 INFO:teuthology.orchestra.run.smithi008.stdout:Get:25 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.6-1278-g487ffcb5-1focal [3,555 kB] 2023-10-05T19:37:23.325 INFO:teuthology.orchestra.run.smithi012.stdout:Get:41 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.6-1278-g487ffcb5-1focal [7,845 kB] 2023-10-05T19:37:23.362 INFO:teuthology.orchestra.run.smithi008.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2023-10-05T19:37:23.420 INFO:teuthology.orchestra.run.smithi012.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2023-10-05T19:37:23.454 INFO:teuthology.orchestra.run.smithi012.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2023-10-05T19:37:23.519 INFO:teuthology.orchestra.run.smithi012.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2023-10-05T19:37:23.547 INFO:teuthology.orchestra.run.smithi008.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2023-10-05T19:37:23.559 INFO:teuthology.orchestra.run.smithi012.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2023-10-05T19:37:23.566 INFO:teuthology.orchestra.run.smithi012.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2023-10-05T19:37:23.582 INFO:teuthology.orchestra.run.smithi008.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2023-10-05T19:37:23.625 INFO:teuthology.orchestra.run.smithi012.stdout:Get:47 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.6-1278-g487ffcb5-1focal [28.7 MB] 2023-10-05T19:37:23.647 INFO:teuthology.orchestra.run.smithi008.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2023-10-05T19:37:23.676 INFO:teuthology.orchestra.run.smithi012.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2023-10-05T19:37:23.677 INFO:teuthology.orchestra.run.smithi012.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2023-10-05T19:37:23.680 INFO:teuthology.orchestra.run.smithi012.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2023-10-05T19:37:23.685 INFO:teuthology.orchestra.run.smithi012.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2023-10-05T19:37:23.687 INFO:teuthology.orchestra.run.smithi008.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2023-10-05T19:37:23.693 INFO:teuthology.orchestra.run.smithi008.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2023-10-05T19:37:23.804 INFO:teuthology.orchestra.run.smithi008.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2023-10-05T19:37:23.805 INFO:teuthology.orchestra.run.smithi008.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2023-10-05T19:37:23.808 INFO:teuthology.orchestra.run.smithi008.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2023-10-05T19:37:23.813 INFO:teuthology.orchestra.run.smithi008.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2023-10-05T19:37:23.861 INFO:teuthology.orchestra.run.smithi012.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.1 [183 kB] 2023-10-05T19:37:23.989 INFO:teuthology.orchestra.run.smithi008.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.1 [183 kB] 2023-10-05T19:37:24.046 INFO:teuthology.orchestra.run.smithi008.stdout:Get:37 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.6-1278-g487ffcb5-1focal [760 kB] 2023-10-05T19:37:24.186 INFO:teuthology.orchestra.run.smithi008.stdout:Get:38 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.6-1278-g487ffcb5-1focal [412 kB] 2023-10-05T19:37:24.261 INFO:teuthology.orchestra.run.smithi008.stdout:Get:39 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.6-1278-g487ffcb5-1focal [22.5 kB] 2023-10-05T19:37:24.263 INFO:teuthology.orchestra.run.smithi008.stdout:Get:40 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.6-1278-g487ffcb5-1focal [200 kB] 2023-10-05T19:37:24.275 INFO:teuthology.orchestra.run.smithi012.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2023-10-05T19:37:24.303 INFO:teuthology.orchestra.run.smithi008.stdout:Get:41 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.6-1278-g487ffcb5-1focal [47.2 kB] 2023-10-05T19:37:24.307 INFO:teuthology.orchestra.run.smithi012.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-10-05T19:37:24.311 INFO:teuthology.orchestra.run.smithi012.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2023-10-05T19:37:24.315 INFO:teuthology.orchestra.run.smithi008.stdout:Get:42 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.6-1278-g487ffcb5-1focal [386 kB] 2023-10-05T19:37:24.360 INFO:teuthology.orchestra.run.smithi012.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2023-10-05T19:37:24.372 INFO:teuthology.orchestra.run.smithi008.stdout:Get:43 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.6-1278-g487ffcb5-1focal [5,577 kB] 2023-10-05T19:37:24.392 INFO:teuthology.orchestra.run.smithi012.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2023-10-05T19:37:24.406 INFO:teuthology.orchestra.run.smithi008.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2023-10-05T19:37:24.419 INFO:teuthology.orchestra.run.smithi012.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2023-10-05T19:37:24.421 INFO:teuthology.orchestra.run.smithi012.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2023-10-05T19:37:24.429 INFO:teuthology.orchestra.run.smithi012.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2023-10-05T19:37:24.437 INFO:teuthology.orchestra.run.smithi008.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-10-05T19:37:24.442 INFO:teuthology.orchestra.run.smithi008.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2023-10-05T19:37:24.490 INFO:teuthology.orchestra.run.smithi008.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2023-10-05T19:37:24.523 INFO:teuthology.orchestra.run.smithi008.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2023-10-05T19:37:24.550 INFO:teuthology.orchestra.run.smithi008.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2023-10-05T19:37:24.553 INFO:teuthology.orchestra.run.smithi008.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2023-10-05T19:37:24.561 INFO:teuthology.orchestra.run.smithi008.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2023-10-05T19:37:24.911 INFO:teuthology.orchestra.run.smithi012.stdout:Get:61 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.6-1278-g487ffcb5-1focal [3,928 B] 2023-10-05T19:37:24.985 INFO:teuthology.orchestra.run.smithi012.stdout:Get:62 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.6-1278-g487ffcb5-1focal [869 kB] 2023-10-05T19:37:24.986 INFO:teuthology.orchestra.run.smithi012.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2023-10-05T19:37:24.998 INFO:teuthology.orchestra.run.smithi012.stdout:Get:64 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.6-1278-g487ffcb5-1focal [2,406 kB] 2023-10-05T19:37:25.007 INFO:teuthology.orchestra.run.smithi012.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2023-10-05T19:37:25.015 INFO:teuthology.orchestra.run.smithi012.stdout:Get:66 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.6-1278-g487ffcb5-1focal [79.1 kB] 2023-10-05T19:37:25.016 INFO:teuthology.orchestra.run.smithi012.stdout:Get:67 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.6-1278-g487ffcb5-1focal [117 kB] 2023-10-05T19:37:25.017 INFO:teuthology.orchestra.run.smithi012.stdout:Get:68 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.6-1278-g487ffcb5-1focal [1,760 kB] 2023-10-05T19:37:25.056 INFO:teuthology.orchestra.run.smithi012.stdout:Get:69 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.6-1278-g487ffcb5-1focal [7,879 kB] 2023-10-05T19:37:25.108 INFO:teuthology.orchestra.run.smithi008.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2023-10-05T19:37:25.370 INFO:teuthology.orchestra.run.smithi008.stdout:Get:53 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.6-1278-g487ffcb5-1focal [116 kB] 2023-10-05T19:37:25.371 INFO:teuthology.orchestra.run.smithi012.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2023-10-05T19:37:25.371 INFO:teuthology.orchestra.run.smithi008.stdout:Get:54 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.6-1278-g487ffcb5-1focal [479 kB] 2023-10-05T19:37:25.372 INFO:teuthology.orchestra.run.smithi008.stdout:Get:55 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.6-1278-g487ffcb5-1focal [27.7 MB] 2023-10-05T19:37:25.372 INFO:teuthology.orchestra.run.smithi008.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2023-10-05T19:37:25.372 INFO:teuthology.orchestra.run.smithi008.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2023-10-05T19:37:25.416 INFO:teuthology.orchestra.run.smithi012.stdout:Get:71 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.6-1278-g487ffcb5-1focal [4,044 B] 2023-10-05T19:37:25.416 INFO:teuthology.orchestra.run.smithi012.stdout:Get:72 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.6-1278-g487ffcb5-1focal [56.3 MB] 2023-10-05T19:37:25.858 INFO:teuthology.orchestra.run.smithi012.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2023-10-05T19:37:25.859 INFO:teuthology.orchestra.run.smithi012.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2023-10-05T19:37:25.860 INFO:teuthology.orchestra.run.smithi012.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2023-10-05T19:37:25.862 INFO:teuthology.orchestra.run.smithi012.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2023-10-05T19:37:25.862 INFO:teuthology.orchestra.run.smithi012.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2023-10-05T19:37:25.864 INFO:teuthology.orchestra.run.smithi012.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2023-10-05T19:37:25.864 INFO:teuthology.orchestra.run.smithi012.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2023-10-05T19:37:25.867 INFO:teuthology.orchestra.run.smithi012.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2023-10-05T19:37:25.868 INFO:teuthology.orchestra.run.smithi012.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2023-10-05T19:37:25.940 INFO:teuthology.orchestra.run.smithi012.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2023-10-05T19:37:25.984 INFO:teuthology.orchestra.run.smithi008.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2023-10-05T19:37:25.985 INFO:teuthology.orchestra.run.smithi008.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2023-10-05T19:37:25.986 INFO:teuthology.orchestra.run.smithi008.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2023-10-05T19:37:25.987 INFO:teuthology.orchestra.run.smithi008.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2023-10-05T19:37:25.989 INFO:teuthology.orchestra.run.smithi008.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2023-10-05T19:37:25.989 INFO:teuthology.orchestra.run.smithi008.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2023-10-05T19:37:25.990 INFO:teuthology.orchestra.run.smithi008.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2023-10-05T19:37:25.992 INFO:teuthology.orchestra.run.smithi008.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2023-10-05T19:37:25.993 INFO:teuthology.orchestra.run.smithi008.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2023-10-05T19:37:26.066 INFO:teuthology.orchestra.run.smithi008.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2023-10-05T19:37:26.112 INFO:teuthology.orchestra.run.smithi012.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2023-10-05T19:37:26.238 INFO:teuthology.orchestra.run.smithi008.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2023-10-05T19:37:26.492 INFO:teuthology.orchestra.run.smithi012.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2023-10-05T19:37:26.544 INFO:teuthology.orchestra.run.smithi012.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2023-10-05T19:37:26.580 INFO:teuthology.orchestra.run.smithi012.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2023-10-05T19:37:26.617 INFO:teuthology.orchestra.run.smithi008.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2023-10-05T19:37:26.668 INFO:teuthology.orchestra.run.smithi008.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2023-10-05T19:37:26.680 INFO:teuthology.orchestra.run.smithi012.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2023-10-05T19:37:26.704 INFO:teuthology.orchestra.run.smithi008.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2023-10-05T19:37:26.727 INFO:teuthology.orchestra.run.smithi012.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2023-10-05T19:37:26.803 INFO:teuthology.orchestra.run.smithi008.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2023-10-05T19:37:26.850 INFO:teuthology.orchestra.run.smithi008.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2023-10-05T19:37:26.881 INFO:teuthology.orchestra.run.smithi012.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2023-10-05T19:37:26.890 INFO:teuthology.orchestra.run.smithi012.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2023-10-05T19:37:27.004 INFO:teuthology.orchestra.run.smithi008.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2023-10-05T19:37:27.014 INFO:teuthology.orchestra.run.smithi008.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2023-10-05T19:37:27.038 INFO:teuthology.orchestra.run.smithi012.stdout:Get:91 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2023-10-05T19:37:27.040 INFO:teuthology.orchestra.run.smithi012.stdout:Get:92 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2023-10-05T19:37:27.041 INFO:teuthology.orchestra.run.smithi012.stdout:Get:93 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2023-10-05T19:37:27.161 INFO:teuthology.orchestra.run.smithi008.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2023-10-05T19:37:27.163 INFO:teuthology.orchestra.run.smithi008.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2023-10-05T19:37:27.164 INFO:teuthology.orchestra.run.smithi008.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2023-10-05T19:37:27.205 INFO:teuthology.orchestra.run.smithi012.stdout:Get:94 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2023-10-05T19:37:27.289 INFO:teuthology.orchestra.run.smithi012.stdout:Get:95 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2023-10-05T19:37:27.327 INFO:teuthology.orchestra.run.smithi008.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2023-10-05T19:37:27.411 INFO:teuthology.orchestra.run.smithi008.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2023-10-05T19:37:27.470 INFO:teuthology.orchestra.run.smithi012.stdout:Get:96 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2023-10-05T19:37:27.593 INFO:teuthology.orchestra.run.smithi008.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2023-10-05T19:37:27.801 INFO:teuthology.orchestra.run.smithi012.stdout:Get:97 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2023-10-05T19:37:27.802 INFO:teuthology.orchestra.run.smithi012.stdout:Get:98 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-10-05T19:37:27.809 INFO:teuthology.orchestra.run.smithi012.stdout:Get:99 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2023-10-05T19:37:27.810 INFO:teuthology.orchestra.run.smithi012.stdout:Get:100 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2023-10-05T19:37:27.818 INFO:teuthology.orchestra.run.smithi012.stdout:Get:101 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2023-10-05T19:37:27.822 INFO:teuthology.orchestra.run.smithi012.stdout:Get:102 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2023-10-05T19:37:27.823 INFO:teuthology.orchestra.run.smithi012.stdout:Get:103 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2023-10-05T19:37:27.825 INFO:teuthology.orchestra.run.smithi012.stdout:Get:104 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2023-10-05T19:37:27.922 INFO:teuthology.orchestra.run.smithi008.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2023-10-05T19:37:27.924 INFO:teuthology.orchestra.run.smithi008.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-10-05T19:37:27.929 INFO:teuthology.orchestra.run.smithi008.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2023-10-05T19:37:27.930 INFO:teuthology.orchestra.run.smithi008.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2023-10-05T19:37:27.940 INFO:teuthology.orchestra.run.smithi008.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2023-10-05T19:37:27.944 INFO:teuthology.orchestra.run.smithi008.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2023-10-05T19:37:27.946 INFO:teuthology.orchestra.run.smithi008.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2023-10-05T19:37:27.947 INFO:teuthology.orchestra.run.smithi008.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2023-10-05T19:37:27.999 INFO:teuthology.orchestra.run.smithi012.stdout:Get:105 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2023-10-05T19:37:28.122 INFO:teuthology.orchestra.run.smithi008.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2023-10-05T19:37:28.256 INFO:teuthology.orchestra.run.smithi012.stdout:Get:106 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2023-10-05T19:37:28.292 INFO:teuthology.orchestra.run.smithi012.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.6-1278-g487ffcb5-1focal [229 kB] 2023-10-05T19:37:28.297 INFO:teuthology.orchestra.run.smithi012.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.6-1278-g487ffcb5-1focal [21.9 kB] 2023-10-05T19:37:28.297 INFO:teuthology.orchestra.run.smithi012.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.6-1278-g487ffcb5-1focal [17.1 MB] 2023-10-05T19:37:28.312 INFO:teuthology.orchestra.run.smithi008.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.6-1278-g487ffcb5-1focal [6,450 kB] 2023-10-05T19:37:28.393 INFO:teuthology.orchestra.run.smithi008.stdout:Get:92 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2023-10-05T19:37:28.811 INFO:teuthology.orchestra.run.smithi008.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.6-1278-g487ffcb5-1focal [221 kB] 2023-10-05T19:37:28.833 INFO:teuthology.orchestra.run.smithi008.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.6-1278-g487ffcb5-1focal [152 kB] 2023-10-05T19:37:28.852 INFO:teuthology.orchestra.run.smithi008.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.6-1278-g487ffcb5-1focal [1,591 kB] 2023-10-05T19:37:29.003 INFO:teuthology.orchestra.run.smithi008.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.6-1278-g487ffcb5-1focal [7,845 kB] 2023-10-05T19:37:29.052 INFO:teuthology.orchestra.run.smithi012.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.6-1278-g487ffcb5-1focal [79.5 kB] 2023-10-05T19:37:29.355 INFO:teuthology.orchestra.run.smithi008.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.6-1278-g487ffcb5-1focal [28.7 MB] 2023-10-05T19:37:29.698 INFO:teuthology.orchestra.run.smithi012.stdout:Fetched 210 MB in 8s (26.9 MB/s) 2023-10-05T19:37:29.882 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2023-10-05T19:37:29.939 INFO:teuthology.orchestra.run.smithi012.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141736 files and directories currently installed.) 2023-10-05T19:37:29.944 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../000-liblttng-ust0_2.11.0-1_amd64.deb ... 2023-10-05T19:37:30.031 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2023-10-05T19:37:30.410 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../001-librbd1_17.2.6-1278-g487ffcb5-1focal_amd64.deb ... 2023-10-05T19:37:30.560 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking librbd1 (17.2.6-1278-g487ffcb5-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-10-05T19:37:30.680 INFO:teuthology.orchestra.run.smithi008.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.6-1278-g487ffcb5-1focal [3,928 B] 2023-10-05T19:37:30.753 INFO:teuthology.orchestra.run.smithi008.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.6-1278-g487ffcb5-1focal [869 kB] 2023-10-05T19:37:30.767 INFO:teuthology.orchestra.run.smithi008.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.6-1278-g487ffcb5-1focal [2,406 kB] 2023-10-05T19:37:30.807 INFO:teuthology.orchestra.run.smithi008.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.6-1278-g487ffcb5-1focal [79.1 kB] 2023-10-05T19:37:30.809 INFO:teuthology.orchestra.run.smithi008.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.6-1278-g487ffcb5-1focal [117 kB] 2023-10-05T19:37:30.812 INFO:teuthology.orchestra.run.smithi008.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.6-1278-g487ffcb5-1focal [1,760 kB] 2023-10-05T19:37:30.883 INFO:teuthology.orchestra.run.smithi008.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.6-1278-g487ffcb5-1focal [7,879 kB] 2023-10-05T19:37:31.211 INFO:teuthology.orchestra.run.smithi008.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.6-1278-g487ffcb5-1focal [4,044 B] 2023-10-05T19:37:31.212 INFO:teuthology.orchestra.run.smithi008.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.6-1278-g487ffcb5-1focal [56.3 MB] 2023-10-05T19:37:31.241 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../002-librados2_17.2.6-1278-g487ffcb5-1focal_amd64.deb ... 2023-10-05T19:37:31.365 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking librados2 (17.2.6-1278-g487ffcb5-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-10-05T19:37:32.077 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libcephfs2. 2023-10-05T19:37:32.094 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../003-libcephfs2_17.2.6-1278-g487ffcb5-1focal_amd64.deb ... 2023-10-05T19:37:32.135 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libcephfs2 (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:37:32.488 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-rados. 2023-10-05T19:37:32.504 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../004-python3-rados_17.2.6-1278-g487ffcb5-1focal_amd64.deb ... 2023-10-05T19:37:32.538 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-rados (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:37:32.899 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-10-05T19:37:32.915 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../005-python3-ceph-argparse_17.2.6-1278-g487ffcb5-1focal_all.deb ... 2023-10-05T19:37:32.957 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-ceph-argparse (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:37:33.243 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-cephfs. 2023-10-05T19:37:33.262 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../006-python3-cephfs_17.2.6-1278-g487ffcb5-1focal_amd64.deb ... 2023-10-05T19:37:33.301 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-cephfs (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:37:33.645 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-ceph-common. 2023-10-05T19:37:33.665 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../007-python3-ceph-common_17.2.6-1278-g487ffcb5-1focal_all.deb ... 2023-10-05T19:37:33.704 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-ceph-common (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:37:33.853 INFO:teuthology.orchestra.run.smithi008.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.6-1278-g487ffcb5-1focal [229 kB] 2023-10-05T19:37:33.857 INFO:teuthology.orchestra.run.smithi008.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.6-1278-g487ffcb5-1focal [21.9 kB] 2023-10-05T19:37:33.858 INFO:teuthology.orchestra.run.smithi008.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.6-1278-g487ffcb5-1focal [17.1 MB] 2023-10-05T19:37:34.032 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-prettytable. 2023-10-05T19:37:34.052 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../008-python3-prettytable_0.7.2-5_all.deb ... 2023-10-05T19:37:34.224 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2023-10-05T19:37:34.509 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-rbd. 2023-10-05T19:37:34.529 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../009-python3-rbd_17.2.6-1278-g487ffcb5-1focal_amd64.deb ... 2023-10-05T19:37:34.532 INFO:teuthology.orchestra.run.smithi008.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-10-04-1333-quincy/487ffcb5ddb60d29c5688d1a457e812b1b697b90/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.6-1278-g487ffcb5-1focal [79.5 kB] 2023-10-05T19:37:34.567 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-rbd (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:37:34.914 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2023-10-05T19:37:34.930 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../010-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2023-10-05T19:37:34.958 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-10-05T19:37:35.182 INFO:teuthology.orchestra.run.smithi008.stdout:Fetched 210 MB in 13s (16.0 MB/s) 2023-10-05T19:37:35.317 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package librabbitmq4:amd64. 2023-10-05T19:37:35.334 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../011-librabbitmq4_0.10.0-1_amd64.deb ... 2023-10-05T19:37:35.353 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2023-10-05T19:37:35.375 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2023-10-05T19:37:35.413 INFO:teuthology.orchestra.run.smithi008.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141736 files and directories currently installed.) 2023-10-05T19:37:35.417 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../000-liblttng-ust0_2.11.0-1_amd64.deb ... 2023-10-05T19:37:35.502 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2023-10-05T19:37:35.701 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-10-05T19:37:35.720 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../012-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2023-10-05T19:37:35.759 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-10-05T19:37:35.951 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../001-librbd1_17.2.6-1278-g487ffcb5-1focal_amd64.deb ... 2023-10-05T19:37:36.073 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking librbd1 (17.2.6-1278-g487ffcb5-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-10-05T19:37:36.163 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-10-05T19:37:36.181 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../013-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2023-10-05T19:37:36.246 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-10-05T19:37:36.516 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-10-05T19:37:36.534 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../014-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2023-10-05T19:37:36.599 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-10-05T19:37:36.772 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../002-librados2_17.2.6-1278-g487ffcb5-1focal_amd64.deb ... 2023-10-05T19:37:36.869 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking librados2 (17.2.6-1278-g487ffcb5-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-10-05T19:37:37.086 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-10-05T19:37:37.103 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../015-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-10-05T19:37:37.155 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-05T19:37:37.548 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package libcephfs2. 2023-10-05T19:37:37.565 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../003-libcephfs2_17.2.6-1278-g487ffcb5-1focal_amd64.deb ... 2023-10-05T19:37:37.606 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking libcephfs2 (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:37:37.650 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-10-05T19:37:37.666 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../016-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-10-05T19:37:37.706 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-05T19:37:37.984 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-rados. 2023-10-05T19:37:38.001 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../004-python3-rados_17.2.6-1278-g487ffcb5-1focal_amd64.deb ... 2023-10-05T19:37:38.042 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-rados (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:37:38.068 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-10-05T19:37:38.083 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../017-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-10-05T19:37:38.126 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-05T19:37:38.337 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-10-05T19:37:38.353 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../005-python3-ceph-argparse_17.2.6-1278-g487ffcb5-1focal_all.deb ... 2023-10-05T19:37:38.395 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-ceph-argparse (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:37:38.647 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-cephfs. 2023-10-05T19:37:38.663 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../006-python3-cephfs_17.2.6-1278-g487ffcb5-1focal_amd64.deb ... 2023-10-05T19:37:38.673 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libthrift-0.13.0. 2023-10-05T19:37:38.687 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../018-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2023-10-05T19:37:38.706 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-cephfs (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:37:38.763 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2023-10-05T19:37:39.095 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package librgw2. 2023-10-05T19:37:39.113 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../019-librgw2_17.2.6-1278-g487ffcb5-1focal_amd64.deb ... 2023-10-05T19:37:39.142 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-ceph-common. 2023-10-05T19:37:39.149 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking librgw2 (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:37:39.160 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../007-python3-ceph-common_17.2.6-1278-g487ffcb5-1focal_all.deb ... 2023-10-05T19:37:39.342 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-ceph-common (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:37:39.654 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-prettytable. 2023-10-05T19:37:39.671 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../008-python3-prettytable_0.7.2-5_all.deb ... 2023-10-05T19:37:39.712 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2023-10-05T19:37:39.927 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-rgw. 2023-10-05T19:37:39.943 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../020-python3-rgw_17.2.6-1278-g487ffcb5-1focal_amd64.deb ... 2023-10-05T19:37:39.948 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-rbd. 2023-10-05T19:37:39.965 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../009-python3-rbd_17.2.6-1278-g487ffcb5-1focal_amd64.deb ... 2023-10-05T19:37:39.985 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-rgw (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:37:39.998 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-rbd (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:37:40.354 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package liboath0. 2023-10-05T19:37:40.359 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2023-10-05T19:37:40.373 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../021-liboath0_2.6.1-1.3_amd64.deb ... 2023-10-05T19:37:40.377 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../010-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2023-10-05T19:37:40.413 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2023-10-05T19:37:40.417 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-10-05T19:37:40.745 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package librabbitmq4:amd64. 2023-10-05T19:37:40.761 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../011-librabbitmq4_0.10.0-1_amd64.deb ... 2023-10-05T19:37:40.774 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libradosstriper1. 2023-10-05T19:37:40.790 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../022-libradosstriper1_17.2.6-1278-g487ffcb5-1focal_amd64.deb ... 2023-10-05T19:37:40.803 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2023-10-05T19:37:40.857 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libradosstriper1 (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:37:41.140 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-10-05T19:37:41.157 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../012-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2023-10-05T19:37:41.198 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-10-05T19:37:41.335 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-common. 2023-10-05T19:37:41.351 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../023-ceph-common_17.2.6-1278-g487ffcb5-1focal_amd64.deb ... 2023-10-05T19:37:41.385 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-common (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:37:41.552 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-10-05T19:37:41.568 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../013-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2023-10-05T19:37:41.619 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-10-05T19:37:41.956 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-10-05T19:37:41.973 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../014-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2023-10-05T19:37:42.014 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-10-05T19:37:42.484 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-10-05T19:37:42.501 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../015-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-10-05T19:37:42.552 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-05T19:37:43.054 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-10-05T19:37:43.071 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../016-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-10-05T19:37:43.105 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-05T19:37:43.441 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-10-05T19:37:43.460 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../017-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-10-05T19:37:43.499 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-05T19:37:43.860 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package libthrift-0.13.0. 2023-10-05T19:37:43.880 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../018-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2023-10-05T19:37:43.918 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2023-10-05T19:37:44.102 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-base. 2023-10-05T19:37:44.121 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../024-ceph-base_17.2.6-1278-g487ffcb5-1focal_amd64.deb ... 2023-10-05T19:37:44.170 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-base (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:37:44.288 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package librgw2. 2023-10-05T19:37:44.305 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../019-librgw2_17.2.6-1278-g487ffcb5-1focal_amd64.deb ... 2023-10-05T19:37:44.346 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking librgw2 (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:37:45.107 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-rgw. 2023-10-05T19:37:45.126 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../020-python3-rgw_17.2.6-1278-g487ffcb5-1focal_amd64.deb ... 2023-10-05T19:37:45.157 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-rgw (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:37:45.207 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-cryptography. 2023-10-05T19:37:45.223 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../025-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2023-10-05T19:37:45.264 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2023-10-05T19:37:45.527 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package liboath0. 2023-10-05T19:37:45.543 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../021-liboath0_2.6.1-1.3_amd64.deb ... 2023-10-05T19:37:45.585 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2023-10-05T19:37:45.651 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-openssl. 2023-10-05T19:37:45.667 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../026-python3-openssl_19.0.0-1build1_all.deb ... 2023-10-05T19:37:45.709 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2023-10-05T19:37:45.905 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package libradosstriper1. 2023-10-05T19:37:45.922 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../022-libradosstriper1_17.2.6-1278-g487ffcb5-1focal_amd64.deb ... 2023-10-05T19:37:45.963 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking libradosstriper1 (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:37:45.987 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-10-05T19:37:46.004 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../027-ceph-mgr-modules-core_17.2.6-1278-g487ffcb5-1focal_all.deb ... 2023-10-05T19:37:46.045 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr-modules-core (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:37:46.349 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package ceph-common. 2023-10-05T19:37:46.366 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../023-ceph-common_17.2.6-1278-g487ffcb5-1focal_amd64.deb ... 2023-10-05T19:37:46.408 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking ceph-common (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:37:46.442 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-bcrypt. 2023-10-05T19:37:46.458 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../028-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2023-10-05T19:37:46.500 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2023-10-05T19:37:46.836 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-cherrypy3. 2023-10-05T19:37:46.854 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../029-python3-cherrypy3_8.9.1-6_all.deb ... 2023-10-05T19:37:46.927 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2023-10-05T19:37:47.307 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-natsort. 2023-10-05T19:37:47.323 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../030-python3-natsort_7.0.1-1_all.deb ... 2023-10-05T19:37:47.364 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-natsort (7.0.1-1) ... 2023-10-05T19:37:47.718 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-jwt. 2023-10-05T19:37:47.734 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../031-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2023-10-05T19:37:47.776 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2023-10-05T19:37:48.120 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-logutils. 2023-10-05T19:37:48.136 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../032-python3-logutils_0.3.3-7_all.deb ... 2023-10-05T19:37:48.179 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-logutils (0.3.3-7) ... 2023-10-05T19:37:48.507 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-markupsafe. 2023-10-05T19:37:48.523 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../033-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2023-10-05T19:37:48.565 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2023-10-05T19:37:48.901 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-mako. 2023-10-05T19:37:48.918 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../034-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2023-10-05T19:37:48.959 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-10-05T19:37:49.109 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package ceph-base. 2023-10-05T19:37:49.126 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../024-ceph-base_17.2.6-1278-g487ffcb5-1focal_amd64.deb ... 2023-10-05T19:37:49.182 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking ceph-base (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:37:49.312 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-simplegeneric. 2023-10-05T19:37:49.331 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../035-python3-simplegeneric_0.8.1-3_all.deb ... 2023-10-05T19:37:49.370 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-10-05T19:37:49.690 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-singledispatch. 2023-10-05T19:37:49.709 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../036-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2023-10-05T19:37:49.773 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2023-10-05T19:37:50.076 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-webob. 2023-10-05T19:37:50.091 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../037-python3-webob_1%3a1.8.5-2_all.deb ... 2023-10-05T19:37:50.151 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2023-10-05T19:37:50.167 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-cryptography. 2023-10-05T19:37:50.185 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../025-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2023-10-05T19:37:50.217 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2023-10-05T19:37:50.512 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-soupsieve. 2023-10-05T19:37:50.530 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../038-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2023-10-05T19:37:50.570 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2023-10-05T19:37:50.606 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-openssl. 2023-10-05T19:37:50.622 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../026-python3-openssl_19.0.0-1build1_all.deb ... 2023-10-05T19:37:50.664 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2023-10-05T19:37:50.898 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-bs4. 2023-10-05T19:37:50.915 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../039-python3-bs4_4.8.2-1_all.deb ... 2023-10-05T19:37:50.925 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-10-05T19:37:50.944 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../027-ceph-mgr-modules-core_17.2.6-1278-g487ffcb5-1focal_all.deb ... 2023-10-05T19:37:50.956 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2023-10-05T19:37:50.986 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking ceph-mgr-modules-core (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:37:51.334 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-waitress. 2023-10-05T19:37:51.351 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../040-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2023-10-05T19:37:51.363 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-bcrypt. 2023-10-05T19:37:51.379 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../028-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2023-10-05T19:37:51.396 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2023-10-05T19:37:51.413 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2023-10-05T19:37:51.707 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-cherrypy3. 2023-10-05T19:37:51.723 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../029-python3-cherrypy3_8.9.1-6_all.deb ... 2023-10-05T19:37:51.737 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-tempita. 2023-10-05T19:37:51.755 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../041-python3-tempita_0.5.2-6_all.deb ... 2023-10-05T19:37:51.765 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2023-10-05T19:37:51.787 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-tempita (0.5.2-6) ... 2023-10-05T19:37:52.102 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-natsort. 2023-10-05T19:37:52.118 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../030-python3-natsort_7.0.1-1_all.deb ... 2023-10-05T19:37:52.132 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-paste. 2023-10-05T19:37:52.148 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../042-python3-paste_3.4.0+dfsg1-1_all.deb ... 2023-10-05T19:37:52.160 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-natsort (7.0.1-1) ... 2023-10-05T19:37:52.190 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2023-10-05T19:37:52.488 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-jwt. 2023-10-05T19:37:52.506 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../031-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2023-10-05T19:37:52.543 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-10-05T19:37:52.546 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2023-10-05T19:37:52.559 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../043-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2023-10-05T19:37:52.626 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2023-10-05T19:37:52.883 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-logutils. 2023-10-05T19:37:52.901 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../032-python3-logutils_0.3.3-7_all.deb ... 2023-10-05T19:37:52.941 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-logutils (0.3.3-7) ... 2023-10-05T19:37:52.953 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pastedeploy. 2023-10-05T19:37:52.973 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../044-python3-pastedeploy_2.1.0-1_all.deb ... 2023-10-05T19:37:53.020 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2023-10-05T19:37:53.253 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-markupsafe. 2023-10-05T19:37:53.269 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../033-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2023-10-05T19:37:53.311 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2023-10-05T19:37:53.365 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-webtest. 2023-10-05T19:37:53.382 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../045-python3-webtest_2.0.32-2_all.deb ... 2023-10-05T19:37:53.423 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-webtest (2.0.32-2) ... 2023-10-05T19:37:53.631 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-mako. 2023-10-05T19:37:53.647 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../034-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2023-10-05T19:37:53.689 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-10-05T19:37:53.768 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pecan. 2023-10-05T19:37:53.786 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../046-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2023-10-05T19:37:53.826 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2023-10-05T19:37:54.000 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-simplegeneric. 2023-10-05T19:37:54.020 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../035-python3-simplegeneric_0.8.1-3_all.deb ... 2023-10-05T19:37:54.058 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-10-05T19:37:54.189 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-werkzeug. 2023-10-05T19:37:54.209 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../047-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.1_all.deb ... 2023-10-05T19:37:54.247 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-10-05T19:37:54.386 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-singledispatch. 2023-10-05T19:37:54.402 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../036-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2023-10-05T19:37:54.444 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2023-10-05T19:37:54.617 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-10-05T19:37:54.634 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../048-libsqlite3-mod-ceph_17.2.6-1278-g487ffcb5-1focal_amd64.deb ... 2023-10-05T19:37:54.675 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libsqlite3-mod-ceph (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:37:55.756 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-webob. 2023-10-05T19:37:55.773 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../037-python3-webob_1%3a1.8.5-2_all.deb ... 2023-10-05T19:37:55.795 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr. 2023-10-05T19:37:55.811 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../049-ceph-mgr_17.2.6-1278-g487ffcb5-1focal_amd64.deb ... 2023-10-05T19:37:55.814 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2023-10-05T19:37:55.853 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:37:56.134 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-soupsieve. 2023-10-05T19:37:56.150 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../038-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2023-10-05T19:37:56.192 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2023-10-05T19:37:56.373 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mon. 2023-10-05T19:37:56.390 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../050-ceph-mon_17.2.6-1278-g487ffcb5-1focal_amd64.deb ... 2023-10-05T19:37:56.431 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mon (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:37:56.487 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-bs4. 2023-10-05T19:37:56.502 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../039-python3-bs4_4.8.2-1_all.deb ... 2023-10-05T19:37:56.545 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2023-10-05T19:37:56.906 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-waitress. 2023-10-05T19:37:56.925 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../040-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2023-10-05T19:37:56.968 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2023-10-05T19:37:57.284 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-tempita. 2023-10-05T19:37:57.304 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../041-python3-tempita_0.5.2-6_all.deb ... 2023-10-05T19:37:57.342 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-tempita (0.5.2-6) ... 2023-10-05T19:37:57.593 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-osd. 2023-10-05T19:37:57.610 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../051-ceph-osd_17.2.6-1278-g487ffcb5-1focal_amd64.deb ... 2023-10-05T19:37:57.628 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-paste. 2023-10-05T19:37:57.648 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../042-python3-paste_3.4.0+dfsg1-1_all.deb ... 2023-10-05T19:37:57.676 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-osd (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:37:57.687 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2023-10-05T19:37:57.982 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-10-05T19:37:57.998 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../043-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2023-10-05T19:37:58.040 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2023-10-05T19:37:58.359 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-pastedeploy. 2023-10-05T19:37:58.378 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../044-python3-pastedeploy_2.1.0-1_all.deb ... 2023-10-05T19:37:58.417 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2023-10-05T19:37:58.720 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-webtest. 2023-10-05T19:37:58.738 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../045-python3-webtest_2.0.32-2_all.deb ... 2023-10-05T19:37:58.779 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-webtest (2.0.32-2) ... 2023-10-05T19:37:59.098 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-pecan. 2023-10-05T19:37:59.114 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../046-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2023-10-05T19:37:59.156 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2023-10-05T19:37:59.536 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-werkzeug. 2023-10-05T19:37:59.554 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../047-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.1_all.deb ... 2023-10-05T19:37:59.594 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-10-05T19:37:59.939 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-10-05T19:37:59.959 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../048-libsqlite3-mod-ceph_17.2.6-1278-g487ffcb5-1focal_amd64.deb ... 2023-10-05T19:37:59.998 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking libsqlite3-mod-ceph (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:38:00.351 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package ceph-mgr. 2023-10-05T19:38:00.367 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../049-ceph-mgr_17.2.6-1278-g487ffcb5-1focal_amd64.deb ... 2023-10-05T19:38:00.409 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking ceph-mgr (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:38:00.554 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph. 2023-10-05T19:38:00.571 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../052-ceph_17.2.6-1278-g487ffcb5-1focal_amd64.deb ... 2023-10-05T19:38:00.654 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:38:00.946 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package ceph-mon. 2023-10-05T19:38:00.962 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../050-ceph-mon_17.2.6-1278-g487ffcb5-1focal_amd64.deb ... 2023-10-05T19:38:00.998 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-fuse. 2023-10-05T19:38:01.003 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking ceph-mon (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:38:01.015 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../053-ceph-fuse_17.2.6-1278-g487ffcb5-1focal_amd64.deb ... 2023-10-05T19:38:01.090 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-fuse (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:38:01.535 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mds. 2023-10-05T19:38:01.552 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../054-ceph-mds_17.2.6-1278-g487ffcb5-1focal_amd64.deb ... 2023-10-05T19:38:01.584 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mds (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:38:02.112 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package cephadm. 2023-10-05T19:38:02.131 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../055-cephadm_17.2.6-1278-g487ffcb5-1focal_amd64.deb ... 2023-10-05T19:38:02.166 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package ceph-osd. 2023-10-05T19:38:02.179 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking cephadm (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:38:02.182 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../051-ceph-osd_17.2.6-1278-g487ffcb5-1focal_amd64.deb ... 2023-10-05T19:38:02.224 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking ceph-osd (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:38:02.448 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-jinja2. 2023-10-05T19:38:02.467 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../056-python3-jinja2_2.10.1-2_all.deb ... 2023-10-05T19:38:02.523 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2023-10-05T19:38:02.792 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-10-05T19:38:02.810 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../057-ceph-mgr-cephadm_17.2.6-1278-g487ffcb5-1focal_all.deb ... 2023-10-05T19:38:02.867 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr-cephadm (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:38:03.204 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-repoze.lru. 2023-10-05T19:38:03.222 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../058-python3-repoze.lru_0.7-2_all.deb ... 2023-10-05T19:38:03.254 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-10-05T19:38:03.481 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libjs-underscore. 2023-10-05T19:38:03.499 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../059-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2023-10-05T19:38:03.573 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-10-05T19:38:03.842 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libjs-sphinxdoc. 2023-10-05T19:38:03.859 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../060-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2023-10-05T19:38:03.933 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-10-05T19:38:04.303 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-routes. 2023-10-05T19:38:04.322 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../061-python3-routes_2.4.1-2_all.deb ... 2023-10-05T19:38:04.361 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-routes (2.4.1-2) ... 2023-10-05T19:38:04.697 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-10-05T19:38:04.717 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../062-ceph-mgr-dashboard_17.2.6-1278-g487ffcb5-1focal_all.deb ... 2023-10-05T19:38:04.755 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr-dashboard (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:38:05.102 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package ceph. 2023-10-05T19:38:05.118 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../052-ceph_17.2.6-1278-g487ffcb5-1focal_amd64.deb ... 2023-10-05T19:38:05.160 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking ceph (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:38:05.267 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-decorator. 2023-10-05T19:38:05.284 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../063-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2023-10-05T19:38:05.350 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2023-10-05T19:38:05.479 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package ceph-fuse. 2023-10-05T19:38:05.495 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../053-ceph-fuse_17.2.6-1278-g487ffcb5-1focal_amd64.deb ... 2023-10-05T19:38:05.537 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking ceph-fuse (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:38:05.653 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2023-10-05T19:38:05.669 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../064-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2023-10-05T19:38:05.736 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-10-05T19:38:05.991 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package ceph-mds. 2023-10-05T19:38:06.009 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../054-ceph-mds_17.2.6-1278-g487ffcb5-1focal_amd64.deb ... 2023-10-05T19:38:06.049 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking ceph-mds (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:38:06.089 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-scipy. 2023-10-05T19:38:06.105 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../065-python3-scipy_1.3.3-3build1_amd64.deb ... 2023-10-05T19:38:06.147 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2023-10-05T19:38:06.577 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package cephadm. 2023-10-05T19:38:06.596 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../055-cephadm_17.2.6-1278-g487ffcb5-1focal_amd64.deb ... 2023-10-05T19:38:06.635 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking cephadm (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:38:06.971 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-jinja2. 2023-10-05T19:38:06.987 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../056-python3-jinja2_2.10.1-2_all.deb ... 2023-10-05T19:38:07.037 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2023-10-05T19:38:07.349 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-10-05T19:38:07.369 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../057-ceph-mgr-cephadm_17.2.6-1278-g487ffcb5-1focal_all.deb ... 2023-10-05T19:38:07.407 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking ceph-mgr-cephadm (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:38:07.473 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-sklearn-lib. 2023-10-05T19:38:07.493 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../066-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2023-10-05T19:38:07.530 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-10-05T19:38:07.743 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-repoze.lru. 2023-10-05T19:38:07.761 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../058-python3-repoze.lru_0.7-2_all.deb ... 2023-10-05T19:38:07.801 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-10-05T19:38:08.037 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package libjs-underscore. 2023-10-05T19:38:08.054 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../059-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2023-10-05T19:38:08.101 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-joblib. 2023-10-05T19:38:08.116 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-10-05T19:38:08.116 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../067-python3-joblib_0.14.0-3_all.deb ... 2023-10-05T19:38:08.159 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-joblib (0.14.0-3) ... 2023-10-05T19:38:08.390 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package libjs-sphinxdoc. 2023-10-05T19:38:08.407 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../060-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2023-10-05T19:38:08.448 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-10-05T19:38:08.562 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-sklearn. 2023-10-05T19:38:08.582 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../068-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2023-10-05T19:38:08.620 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-10-05T19:38:08.767 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-routes. 2023-10-05T19:38:08.783 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../061-python3-routes_2.4.1-2_all.deb ... 2023-10-05T19:38:08.825 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-routes (2.4.1-2) ... 2023-10-05T19:38:09.128 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-10-05T19:38:09.148 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../062-ceph-mgr-dashboard_17.2.6-1278-g487ffcb5-1focal_all.deb ... 2023-10-05T19:38:09.186 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking ceph-mgr-dashboard (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:38:09.362 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-10-05T19:38:09.382 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../069-ceph-mgr-diskprediction-local_17.2.6-1278-g487ffcb5-1focal_all.deb ... 2023-10-05T19:38:09.420 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:38:09.690 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-decorator. 2023-10-05T19:38:09.706 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../063-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2023-10-05T19:38:09.748 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2023-10-05T19:38:10.017 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2023-10-05T19:38:10.036 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../064-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2023-10-05T19:38:10.076 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-10-05T19:38:10.395 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-scipy. 2023-10-05T19:38:10.412 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../065-python3-scipy_1.3.3-3build1_amd64.deb ... 2023-10-05T19:38:10.461 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2023-10-05T19:38:10.556 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-cachetools. 2023-10-05T19:38:10.575 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../070-python3-cachetools_4.0.0-1_all.deb ... 2023-10-05T19:38:10.614 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2023-10-05T19:38:11.425 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pyasn1. 2023-10-05T19:38:11.445 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../071-python3-pyasn1_0.4.2-3build1_all.deb ... 2023-10-05T19:38:11.508 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2023-10-05T19:38:11.812 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-sklearn-lib. 2023-10-05T19:38:11.830 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../066-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2023-10-05T19:38:11.862 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-10-05T19:38:11.881 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../072-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2023-10-05T19:38:11.895 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-10-05T19:38:11.911 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-10-05T19:38:12.256 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-rsa. 2023-10-05T19:38:12.272 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../073-python3-rsa_4.0-3_all.deb ... 2023-10-05T19:38:12.305 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-rsa (4.0-3) ... 2023-10-05T19:38:12.399 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-joblib. 2023-10-05T19:38:12.418 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../067-python3-joblib_0.14.0-3_all.deb ... 2023-10-05T19:38:12.457 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-joblib (0.14.0-3) ... 2023-10-05T19:38:12.625 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-google-auth. 2023-10-05T19:38:12.643 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../074-python3-google-auth_1.5.1-2_all.deb ... 2023-10-05T19:38:12.683 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2023-10-05T19:38:12.819 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-sklearn. 2023-10-05T19:38:12.838 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../068-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2023-10-05T19:38:12.877 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-10-05T19:38:13.020 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-blinker. 2023-10-05T19:38:13.036 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../075-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2023-10-05T19:38:13.078 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-10-05T19:38:13.397 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-oauthlib. 2023-10-05T19:38:13.417 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../076-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2023-10-05T19:38:13.455 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2023-10-05T19:38:13.568 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-10-05T19:38:13.589 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../069-ceph-mgr-diskprediction-local_17.2.6-1278-g487ffcb5-1focal_all.deb ... 2023-10-05T19:38:13.618 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:38:13.817 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-10-05T19:38:13.833 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../077-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2023-10-05T19:38:13.875 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-10-05T19:38:14.220 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-websocket. 2023-10-05T19:38:14.236 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../078-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2023-10-05T19:38:14.301 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2023-10-05T19:38:14.614 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-kubernetes. 2023-10-05T19:38:14.631 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../079-python3-kubernetes_7.0.0~a1-2_all.deb ... 2023-10-05T19:38:14.689 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2023-10-05T19:38:14.738 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-cachetools. 2023-10-05T19:38:14.757 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../070-python3-cachetools_4.0.0-1_all.deb ... 2023-10-05T19:38:14.796 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2023-10-05T19:38:15.099 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-pyasn1. 2023-10-05T19:38:15.115 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../071-python3-pyasn1_0.4.2-3build1_all.deb ... 2023-10-05T19:38:15.157 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2023-10-05T19:38:15.235 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-10-05T19:38:15.254 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../080-ceph-mgr-k8sevents_17.2.6-1278-g487ffcb5-1focal_all.deb ... 2023-10-05T19:38:15.293 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr-k8sevents (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:38:15.477 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-10-05T19:38:15.497 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../072-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2023-10-05T19:38:15.535 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-10-05T19:38:15.748 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libonig5:amd64. 2023-10-05T19:38:15.767 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../081-libonig5_6.9.4-1_amd64.deb ... 2023-10-05T19:38:15.806 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2023-10-05T19:38:15.871 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-rsa. 2023-10-05T19:38:15.887 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../073-python3-rsa_4.0-3_all.deb ... 2023-10-05T19:38:15.929 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-rsa (4.0-3) ... 2023-10-05T19:38:16.168 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libjq1:amd64. 2023-10-05T19:38:16.186 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../082-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-10-05T19:38:16.226 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-10-05T19:38:16.240 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-google-auth. 2023-10-05T19:38:16.258 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../074-python3-google-auth_1.5.1-2_all.deb ... 2023-10-05T19:38:16.299 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2023-10-05T19:38:16.512 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package jq. 2023-10-05T19:38:16.531 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../083-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-10-05T19:38:16.570 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2023-10-05T19:38:16.594 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-blinker. 2023-10-05T19:38:16.613 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../075-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2023-10-05T19:38:16.643 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-10-05T19:38:16.833 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package socat. 2023-10-05T19:38:16.848 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../084-socat_1.7.3.3-2_amd64.deb ... 2023-10-05T19:38:16.889 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking socat (1.7.3.3-2) ... 2023-10-05T19:38:16.979 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-oauthlib. 2023-10-05T19:38:16.998 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../076-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2023-10-05T19:38:17.037 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2023-10-05T19:38:17.210 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package xmlstarlet. 2023-10-05T19:38:17.227 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../085-xmlstarlet_1.6.1-2build1_amd64.deb ... 2023-10-05T19:38:17.268 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2023-10-05T19:38:17.358 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-10-05T19:38:17.375 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../077-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2023-10-05T19:38:17.416 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-10-05T19:38:17.656 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-test. 2023-10-05T19:38:17.675 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../086-ceph-test_17.2.6-1278-g487ffcb5-1focal_amd64.deb ... 2023-10-05T19:38:17.714 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-test (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:38:17.735 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-websocket. 2023-10-05T19:38:17.754 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../078-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2023-10-05T19:38:17.793 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2023-10-05T19:38:18.105 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-kubernetes. 2023-10-05T19:38:18.122 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../079-python3-kubernetes_7.0.0~a1-2_all.deb ... 2023-10-05T19:38:18.163 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2023-10-05T19:38:18.768 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-10-05T19:38:18.789 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../080-ceph-mgr-k8sevents_17.2.6-1278-g487ffcb5-1focal_all.deb ... 2023-10-05T19:38:18.826 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking ceph-mgr-k8sevents (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:38:19.214 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package libonig5:amd64. 2023-10-05T19:38:19.231 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../081-libonig5_6.9.4-1_amd64.deb ... 2023-10-05T19:38:19.272 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2023-10-05T19:38:19.618 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package libjq1:amd64. 2023-10-05T19:38:19.634 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../082-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-10-05T19:38:19.667 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-10-05T19:38:19.895 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package jq. 2023-10-05T19:38:19.913 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../083-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-10-05T19:38:19.953 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2023-10-05T19:38:20.214 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package socat. 2023-10-05T19:38:20.233 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../084-socat_1.7.3.3-2_amd64.deb ... 2023-10-05T19:38:20.272 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking socat (1.7.3.3-2) ... 2023-10-05T19:38:20.593 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package xmlstarlet. 2023-10-05T19:38:20.610 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../085-xmlstarlet_1.6.1-2build1_amd64.deb ... 2023-10-05T19:38:20.650 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2023-10-05T19:38:21.022 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package ceph-test. 2023-10-05T19:38:21.039 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../086-ceph-test_17.2.6-1278-g487ffcb5-1focal_amd64.deb ... 2023-10-05T19:38:21.080 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking ceph-test (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:38:22.882 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-volume. 2023-10-05T19:38:22.899 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../087-ceph-volume_17.2.6-1278-g487ffcb5-1focal_all.deb ... 2023-10-05T19:38:22.964 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-volume (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:38:23.328 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package fonts-mathjax. 2023-10-05T19:38:23.345 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../088-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2023-10-05T19:38:23.386 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2023-10-05T19:38:23.915 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libcephfs-dev. 2023-10-05T19:38:23.935 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../089-libcephfs-dev_17.2.6-1278-g487ffcb5-1focal_amd64.deb ... 2023-10-05T19:38:24.006 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libcephfs-dev (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:38:24.426 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package nvme-cli. 2023-10-05T19:38:24.443 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../090-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2023-10-05T19:38:24.501 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2023-10-05T19:38:24.841 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libjs-mathjax. 2023-10-05T19:38:24.861 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../091-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2023-10-05T19:38:24.899 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2023-10-05T19:38:26.143 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package ceph-volume. 2023-10-05T19:38:26.161 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../087-ceph-volume_17.2.6-1278-g487ffcb5-1focal_all.deb ... 2023-10-05T19:38:26.200 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking ceph-volume (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:38:26.588 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python-natsort-doc. 2023-10-05T19:38:26.608 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../092-python-natsort-doc_7.0.1-1_all.deb ... 2023-10-05T19:38:26.663 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2023-10-05T19:38:26.781 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package fonts-mathjax. 2023-10-05T19:38:26.797 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../088-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2023-10-05T19:38:26.839 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2023-10-05T19:38:27.024 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-atomicwrites. 2023-10-05T19:38:27.041 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../093-python3-atomicwrites_1.1.5-2build1_all.deb ... 2023-10-05T19:38:27.074 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2023-10-05T19:38:27.358 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package libcephfs-dev. 2023-10-05T19:38:27.375 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../089-libcephfs-dev_17.2.6-1278-g487ffcb5-1focal_amd64.deb ... 2023-10-05T19:38:27.410 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-attr. 2023-10-05T19:38:27.424 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking libcephfs-dev (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:38:27.428 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../094-python3-attr_19.3.0-2_all.deb ... 2023-10-05T19:38:27.468 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-attr (19.3.0-2) ... 2023-10-05T19:38:27.795 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package nvme-cli. 2023-10-05T19:38:27.812 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-webencodings. 2023-10-05T19:38:27.813 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../090-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2023-10-05T19:38:27.829 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../095-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2023-10-05T19:38:27.870 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2023-10-05T19:38:27.871 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2023-10-05T19:38:28.190 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-html5lib. 2023-10-05T19:38:28.208 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package libjs-mathjax. 2023-10-05T19:38:28.211 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../096-python3-html5lib_1.0.1-2_all.deb ... 2023-10-05T19:38:28.226 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../091-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2023-10-05T19:38:28.248 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2023-10-05T19:38:28.266 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2023-10-05T19:38:28.618 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-lxml:amd64. 2023-10-05T19:38:28.636 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../097-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2023-10-05T19:38:28.676 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-10-05T19:38:29.154 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-packaging. 2023-10-05T19:38:29.172 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../098-python3-packaging_20.3-1_all.deb ... 2023-10-05T19:38:29.212 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-packaging (20.3-1) ... 2023-10-05T19:38:29.531 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pastescript. 2023-10-05T19:38:29.551 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../099-python3-pastescript_2.0.2-4_all.deb ... 2023-10-05T19:38:29.589 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-10-05T19:38:29.866 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python-natsort-doc. 2023-10-05T19:38:29.887 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../092-python-natsort-doc_7.0.1-1_all.deb ... 2023-10-05T19:38:29.918 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pluggy. 2023-10-05T19:38:29.923 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2023-10-05T19:38:29.938 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../100-python3-pluggy_0.13.0-2_all.deb ... 2023-10-05T19:38:29.976 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2023-10-05T19:38:30.260 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-atomicwrites. 2023-10-05T19:38:30.277 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../093-python3-atomicwrites_1.1.5-2build1_all.deb ... 2023-10-05T19:38:30.312 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-psutil. 2023-10-05T19:38:30.318 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2023-10-05T19:38:30.329 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../101-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2023-10-05T19:38:30.370 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2023-10-05T19:38:30.612 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-attr. 2023-10-05T19:38:30.633 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../094-python3-attr_19.3.0-2_all.deb ... 2023-10-05T19:38:30.670 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-attr (19.3.0-2) ... 2023-10-05T19:38:30.731 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-py. 2023-10-05T19:38:30.749 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../102-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2023-10-05T19:38:30.789 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2023-10-05T19:38:31.032 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-webencodings. 2023-10-05T19:38:31.049 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../095-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2023-10-05T19:38:31.090 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2023-10-05T19:38:31.143 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pyinotify. 2023-10-05T19:38:31.161 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../103-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2023-10-05T19:38:31.224 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-10-05T19:38:31.392 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-html5lib. 2023-10-05T19:38:31.409 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../096-python3-html5lib_1.0.1-2_all.deb ... 2023-10-05T19:38:31.450 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2023-10-05T19:38:31.671 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-wcwidth. 2023-10-05T19:38:31.691 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../104-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2023-10-05T19:38:31.729 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2023-10-05T19:38:31.787 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-lxml:amd64. 2023-10-05T19:38:31.804 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../097-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2023-10-05T19:38:31.837 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-10-05T19:38:32.074 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pytest. 2023-10-05T19:38:32.092 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../105-python3-pytest_4.6.9-1_all.deb ... 2023-10-05T19:38:32.132 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pytest (4.6.9-1) ... 2023-10-05T19:38:32.315 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-packaging. 2023-10-05T19:38:32.332 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../098-python3-packaging_20.3-1_all.deb ... 2023-10-05T19:38:32.373 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-packaging (20.3-1) ... 2023-10-05T19:38:32.503 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-simplejson. 2023-10-05T19:38:32.522 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../106-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2023-10-05T19:38:32.560 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2023-10-05T19:38:32.667 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-pastescript. 2023-10-05T19:38:32.684 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../099-python3-pastescript_2.0.2-4_all.deb ... 2023-10-05T19:38:32.725 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-10-05T19:38:32.863 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-10-05T19:38:32.883 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../107-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2023-10-05T19:38:32.921 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-10-05T19:38:33.046 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-pluggy. 2023-10-05T19:38:33.063 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../100-python3-pluggy_0.13.0-2_all.deb ... 2023-10-05T19:38:33.104 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2023-10-05T19:38:33.417 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-psutil. 2023-10-05T19:38:33.432 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../101-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2023-10-05T19:38:33.493 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2023-10-05T19:38:33.687 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package radosgw. 2023-10-05T19:38:33.707 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../108-radosgw_17.2.6-1278-g487ffcb5-1focal_amd64.deb ... 2023-10-05T19:38:33.758 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking radosgw (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:38:33.801 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-py. 2023-10-05T19:38:33.822 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../102-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2023-10-05T19:38:33.859 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2023-10-05T19:38:34.171 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-pyinotify. 2023-10-05T19:38:34.188 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../103-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2023-10-05T19:38:34.229 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-10-05T19:38:34.557 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-wcwidth. 2023-10-05T19:38:34.574 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../104-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2023-10-05T19:38:34.615 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2023-10-05T19:38:34.943 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-pytest. 2023-10-05T19:38:34.960 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../105-python3-pytest_4.6.9-1_all.deb ... 2023-10-05T19:38:35.001 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-pytest (4.6.9-1) ... 2023-10-05T19:38:35.338 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package python3-simplejson. 2023-10-05T19:38:35.355 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../106-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2023-10-05T19:38:35.396 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2023-10-05T19:38:35.532 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package rbd-fuse. 2023-10-05T19:38:35.549 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../109-rbd-fuse_17.2.6-1278-g487ffcb5-1focal_amd64.deb ... 2023-10-05T19:38:35.589 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking rbd-fuse (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:38:35.674 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-10-05T19:38:35.693 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../107-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2023-10-05T19:38:35.732 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-10-05T19:38:35.914 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-attr (19.3.0-2) ... 2023-10-05T19:38:36.203 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2023-10-05T19:38:36.329 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-10-05T19:38:36.445 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package radosgw. 2023-10-05T19:38:36.454 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2023-10-05T19:38:36.465 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../108-radosgw_17.2.6-1278-g487ffcb5-1focal_amd64.deb ... 2023-10-05T19:38:36.503 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking radosgw (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:38:36.673 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up cephadm (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:38:37.310 INFO:teuthology.orchestra.run.smithi012.stdout:Adding system user cephadm....done 2023-10-05T19:38:37.463 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2023-10-05T19:38:37.622 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2023-10-05T19:38:37.983 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-10-05T19:38:38.281 INFO:teuthology.orchestra.run.smithi008.stdout:Selecting previously unselected package rbd-fuse. 2023-10-05T19:38:38.287 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2023-10-05T19:38:38.299 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing to unpack .../109-rbd-fuse_17.2.6-1278-g487ffcb5-1focal_amd64.deb ... 2023-10-05T19:38:38.347 INFO:teuthology.orchestra.run.smithi008.stdout:Unpacking rbd-fuse (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:38:38.413 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up liboath0 (2.6.1-1.3) ... 2023-10-05T19:38:38.539 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2023-10-05T19:38:38.679 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-attr (19.3.0-2) ... 2023-10-05T19:38:38.891 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-joblib (0.14.0-3) ... 2023-10-05T19:38:38.987 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2023-10-05T19:38:39.113 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-10-05T19:38:39.239 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2023-10-05T19:38:39.440 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up cephadm (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:38:39.460 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2023-10-05T19:38:39.754 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-cachetools (4.0.0-1) ... 2023-10-05T19:38:40.045 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2023-10-05T19:38:40.131 INFO:teuthology.orchestra.run.smithi008.stdout:Adding system user cephadm....done 2023-10-05T19:38:40.292 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2023-10-05T19:38:40.332 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-10-05T19:38:40.418 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2023-10-05T19:38:40.675 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-ceph-argparse (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:38:40.770 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-10-05T19:38:41.034 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-10-05T19:38:41.088 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2023-10-05T19:38:41.177 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2023-10-05T19:38:41.214 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up liboath0 (2.6.1-1.3) ... 2023-10-05T19:38:41.339 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2023-10-05T19:38:41.478 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2023-10-05T19:38:41.666 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-joblib (0.14.0-3) ... 2023-10-05T19:38:41.812 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-10-05T19:38:41.938 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2023-10-05T19:38:42.152 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2023-10-05T19:38:42.454 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-cachetools (4.0.0-1) ... 2023-10-05T19:38:42.564 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-natsort (7.0.1-1) ... 2023-10-05T19:38:42.730 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2023-10-05T19:38:42.874 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2023-10-05T19:38:43.014 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-10-05T19:38:43.192 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2023-10-05T19:38:43.315 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-ceph-argparse (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:38:43.543 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2023-10-05T19:38:43.628 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-10-05T19:38:43.754 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2023-10-05T19:38:43.869 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2023-10-05T19:38:44.030 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2023-10-05T19:38:44.262 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-tempita (0.5.2-6) ... 2023-10-05T19:38:44.314 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-10-05T19:38:44.441 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2023-10-05T19:38:44.596 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-packaging (20.3-1) ... 2023-10-05T19:38:44.800 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-natsort (7.0.1-1) ... 2023-10-05T19:38:44.939 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2023-10-05T19:38:45.098 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-10-05T19:38:45.111 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2023-10-05T19:38:45.224 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2023-10-05T19:38:45.394 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2023-10-05T19:38:45.509 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2023-10-05T19:38:45.729 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2023-10-05T19:38:45.894 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-10-05T19:38:46.007 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2023-10-05T19:38:46.262 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2023-10-05T19:38:46.375 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-tempita (0.5.2-6) ... 2023-10-05T19:38:46.639 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up socat (1.7.3.3-2) ... 2023-10-05T19:38:46.659 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-packaging (20.3-1) ... 2023-10-05T19:38:46.799 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-ceph-common (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:38:46.935 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2023-10-05T19:38:47.061 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-10-05T19:38:47.133 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-05T19:38:47.195 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2023-10-05T19:38:47.270 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-html5lib (1.0.1-2) ... 2023-10-05T19:38:47.488 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2023-10-05T19:38:47.622 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-10-05T19:38:47.764 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2023-10-05T19:38:47.898 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-10-05T19:38:47.924 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pluggy (0.13.0-2) ... 2023-10-05T19:38:48.233 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-10-05T19:38:48.258 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2023-10-05T19:38:48.568 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-05T19:38:48.643 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up socat (1.7.3.3-2) ... 2023-10-05T19:38:48.694 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-10-05T19:38:48.811 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-ceph-common (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:38:48.823 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2023-10-05T19:38:49.120 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-05T19:38:49.183 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2023-10-05T19:38:49.255 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-html5lib (1.0.1-2) ... 2023-10-05T19:38:49.484 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-logutils (0.3.3-7) ... 2023-10-05T19:38:49.614 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-10-05T19:38:49.740 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2023-10-05T19:38:49.769 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2023-10-05T19:38:49.891 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-pluggy (0.13.0-2) ... 2023-10-05T19:38:49.894 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-10-05T19:38:50.176 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-10-05T19:38:50.212 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-prettytable (0.7.2-5) ... 2023-10-05T19:38:50.485 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-05T19:38:50.538 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2023-10-05T19:38:50.611 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-10-05T19:38:50.737 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2023-10-05T19:38:50.850 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2023-10-05T19:38:51.064 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2023-10-05T19:38:51.226 INFO:teuthology.orchestra.run.smithi012.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2023-10-05T19:38:51.374 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-logutils (0.3.3-7) ... 2023-10-05T19:38:51.486 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2023-10-05T19:38:51.612 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-10-05T19:38:51.675 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2023-10-05T19:38:51.792 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-10-05T19:38:51.971 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2023-10-05T19:38:52.071 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-prettytable (0.7.2-5) ... 2023-10-05T19:38:52.365 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-10-05T19:38:52.381 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2023-10-05T19:38:52.516 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-10-05T19:38:52.642 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-webob (1:1.8.5-2) ... 2023-10-05T19:38:52.682 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2023-10-05T19:38:52.984 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-10-05T19:38:53.123 INFO:teuthology.orchestra.run.smithi008.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2023-10-05T19:38:53.244 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up librados2 (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:38:53.368 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2023-10-05T19:38:53.370 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2023-10-05T19:38:53.494 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-10-05T19:38:53.837 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2023-10-05T19:38:54.208 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-10-05T19:38:54.334 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-10-05T19:38:54.443 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-webob (1:1.8.5-2) ... 2023-10-05T19:38:54.769 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-10-05T19:38:54.919 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libsqlite3-mod-ceph (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:38:55.045 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-10-05T19:38:55.079 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up librados2 (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:38:55.171 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pytest (4.6.9-1) ... 2023-10-05T19:38:55.205 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2023-10-05T19:38:55.606 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2023-10-05T19:38:55.929 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2023-10-05T19:38:56.242 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-05T19:38:56.385 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-bs4 (4.8.2-1) ... 2023-10-05T19:38:56.723 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up libsqlite3-mod-ceph (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:38:56.753 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libcephfs2 (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:38:56.849 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-10-05T19:38:56.879 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2023-10-05T19:38:56.975 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-pytest (4.6.9-1) ... 2023-10-05T19:38:57.239 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libradosstriper1 (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:38:58.311 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2023-10-05T19:38:58.311 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2023-10-05T19:38:58.311 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-10-05T19:38:58.312 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-10-05T19:38:58.396 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-05T19:38:58.522 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-bs4 (4.8.2-1) ... 2023-10-05T19:38:58.830 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up librbd1 (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:38:59.645 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up libcephfs2 (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:38:59.764 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2023-10-05T19:38:59.764 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up libradosstriper1 (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:38:59.764 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-10-05T19:38:59.765 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-10-05T19:38:59.765 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2023-10-05T19:38:59.765 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2023-10-05T19:38:59.765 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-rsa (4.0-3) ... 2023-10-05T19:38:59.766 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2023-10-05T19:38:59.786 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-10-05T19:38:59.837 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-webtest (2.0.32-2) ... 2023-10-05T19:39:00.149 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr-modules-core (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:00.281 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-fuse (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:00.513 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-05T19:39:01.326 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-05T19:39:01.328 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libcephfs-dev (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:01.331 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-10-05T19:39:01.331 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up librbd1 (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:01.332 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-10-05T19:39:01.332 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2023-10-05T19:39:01.332 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2023-10-05T19:39:01.421 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-rsa (4.0-3) ... 2023-10-05T19:39:01.504 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-10-05T19:39:01.714 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2023-10-05T19:39:01.839 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2023-10-05T19:39:01.840 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-webtest (2.0.32-2) ... 2023-10-05T19:39:02.141 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up ceph-mgr-modules-core (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:02.233 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-rados (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:02.267 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up ceph-fuse (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:02.376 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up librgw2 (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:02.500 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-05T19:39:02.500 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-05T19:39:02.502 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-rbd (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:02.653 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up rbd-fuse (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:02.779 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-routes (2.4.1-2) ... 2023-10-05T19:39:02.960 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up libcephfs-dev (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:03.097 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-google-auth (1.5.1-2) ... 2023-10-05T19:39:03.163 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-10-05T19:39:03.407 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-rgw (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:03.448 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-10-05T19:39:03.533 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-cephfs (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:03.684 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2023-10-05T19:39:03.775 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2023-10-05T19:39:04.127 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-rados (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:04.253 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up librgw2 (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:04.363 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-rbd (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:04.480 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up rbd-fuse (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:04.615 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-routes (2.4.1-2) ... 2023-10-05T19:39:04.620 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-common (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:04.917 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-google-auth (1.5.1-2) ... 2023-10-05T19:39:05.201 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-rgw (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:05.207 INFO:teuthology.orchestra.run.smithi012.stdout:Adding group ceph....done 2023-10-05T19:39:05.327 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-cephfs (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:05.453 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2023-10-05T19:39:05.628 INFO:teuthology.orchestra.run.smithi012.stdout:Adding system user ceph....done 2023-10-05T19:39:05.838 INFO:teuthology.orchestra.run.smithi012.stdout:Setting system user ceph properties....done 2023-10-05T19:39:05.870 INFO:teuthology.orchestra.run.smithi012.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-10-05T19:39:06.022 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-10-05T19:39:06.347 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-10-05T19:39:06.407 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up ceph-common (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:06.818 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-test (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:06.951 INFO:teuthology.orchestra.run.smithi008.stdout:Adding group ceph....done 2023-10-05T19:39:07.018 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up radosgw (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:07.302 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-05T19:39:07.302 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-05T19:39:07.328 INFO:teuthology.orchestra.run.smithi008.stdout:Adding system user ceph....done 2023-10-05T19:39:07.529 INFO:teuthology.orchestra.run.smithi008.stdout:Setting system user ceph properties....done 2023-10-05T19:39:07.554 INFO:teuthology.orchestra.run.smithi008.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-10-05T19:39:07.705 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-10-05T19:39:08.027 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-10-05T19:39:08.063 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-base (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:08.449 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-10-05T19:39:08.475 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up ceph-test (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:08.601 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up radosgw (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:08.896 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-05T19:39:08.896 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-05T19:39:08.923 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mds (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:09.152 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-05T19:39:09.187 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-05T19:39:09.545 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up ceph-base (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:09.592 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:09.824 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-05T19:39:09.838 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-05T19:39:09.934 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-10-05T19:39:10.268 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-osd (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:10.397 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up ceph-mds (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:10.640 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-05T19:39:10.640 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-05T19:39:10.777 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-05T19:39:10.811 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-05T19:39:11.124 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up ceph-mgr (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:11.238 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr-k8sevents (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:11.344 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-05T19:39:11.345 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-05T19:39:11.364 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr-diskprediction-local (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:11.541 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mon (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:11.772 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-05T19:39:11.772 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-05T19:39:11.792 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up ceph-osd (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:12.267 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr-cephadm (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:12.269 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-05T19:39:12.269 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-05T19:39:12.401 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:12.502 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr-dashboard (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:12.662 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-volume (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:12.718 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up ceph-mgr-k8sevents (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:12.789 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-10-05T19:39:12.844 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up ceph-mgr-diskprediction-local (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:12.982 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-10-05T19:39:12.988 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up ceph-mon (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:13.223 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-05T19:39:13.223 INFO:teuthology.orchestra.run.smithi008.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-05T19:39:13.493 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-10-05T19:39:13.668 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up ceph-mgr-cephadm (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:13.827 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up ceph (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:13.953 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up ceph-mgr-dashboard (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:14.113 INFO:teuthology.orchestra.run.smithi008.stdout:Setting up ceph-volume (17.2.6-1278-g487ffcb5-1focal) ... 2023-10-05T19:39:14.240 INFO:teuthology.orchestra.run.smithi008.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-10-05T19:39:14.400 INFO:teuthology.orchestra.run.smithi008.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-10-05T19:39:14.685 INFO:teuthology.orchestra.run.smithi008.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-10-05T19:39:16.397 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2023-10-05T19:39:17.072 INFO:teuthology.orchestra.run.smithi012.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-05T19:39:17.074 DEBUG:teuthology.parallel:result is None 2023-10-05T19:39:17.553 INFO:teuthology.orchestra.run.smithi008.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2023-10-05T19:39:18.177 INFO:teuthology.orchestra.run.smithi008.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-05T19:39:18.179 DEBUG:teuthology.parallel:result is None 2023-10-05T19:39:18.180 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=487ffcb5ddb60d29c5688d1a457e812b1b697b90 2023-10-05T19:39:18.404 DEBUG:teuthology.orchestra.run.smithi008:> dpkg-query -W -f '${Version}' ceph 2023-10-05T19:39:18.428 INFO:teuthology.orchestra.run.smithi008.stdout:17.2.6-1278-g487ffcb5-1focal 2023-10-05T19:39:18.429 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1278-g487ffcb5-1focal 2023-10-05T19:39:18.429 INFO:teuthology.task.install:The correct ceph version 17.2.6-1278-g487ffcb5-1focal is installed. 2023-10-05T19:39:18.429 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=487ffcb5ddb60d29c5688d1a457e812b1b697b90 2023-10-05T19:39:18.571 DEBUG:teuthology.orchestra.run.smithi012:> dpkg-query -W -f '${Version}' ceph 2023-10-05T19:39:18.594 INFO:teuthology.orchestra.run.smithi012.stdout:17.2.6-1278-g487ffcb5-1focal 2023-10-05T19:39:18.595 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1278-g487ffcb5-1focal 2023-10-05T19:39:18.595 INFO:teuthology.task.install:The correct ceph version 17.2.6-1278-g487ffcb5-1focal is installed. 2023-10-05T19:39:18.595 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-05T19:39:18.596 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-10-05T19:39:18.596 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-05T19:39:18.612 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-05T19:39:18.613 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-05T19:39:18.652 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-05T19:39:18.652 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-10-05T19:39:18.653 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/usr/bin/daemon-helper 2023-10-05T19:39:18.671 DEBUG:teuthology.orchestra.run.smithi008:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-05T19:39:18.682 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-05T19:39:18.682 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/daemon-helper 2023-10-05T19:39:18.696 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-05T19:39:18.751 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-05T19:39:18.752 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-10-05T19:39:18.752 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-05T19:39:18.766 DEBUG:teuthology.orchestra.run.smithi008:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-05T19:39:18.827 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-05T19:39:18.827 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-05T19:39:18.843 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-05T19:39:18.859 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-05T19:39:18.859 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-10-05T19:39:18.859 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/usr/bin/stdin-killer 2023-10-05T19:39:18.875 DEBUG:teuthology.orchestra.run.smithi008:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-05T19:39:18.886 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-05T19:39:18.886 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/stdin-killer 2023-10-05T19:39:18.902 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-05T19:39:18.960 INFO:teuthology.run_tasks:Running task ceph... 2023-10-05T19:39:19.033 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-05T19:39:19.033 DEBUG:teuthology.orchestra.run.smithi008:> sudo chmod 777 /var/log/ceph 2023-10-05T19:39:19.036 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 777 /var/log/ceph 2023-10-05T19:39:19.051 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-05T19:39:19.051 DEBUG:teuthology.orchestra.run.smithi008:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-05T19:39:19.054 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-05T19:39:19.063 INFO:tasks.ceph:Creating extra log directories... 2023-10-05T19:39:19.063 DEBUG:teuthology.orchestra.run.smithi008:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-05T19:39:19.065 DEBUG:teuthology.orchestra.run.smithi012:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-05T19:39:19.129 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-05T19:39:19.130 INFO:tasks.ceph:config {'conf': {'client': {'debug ms': 1, 'rgw gc obj min wait': 15, 'debug rgw': 20, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'osd': {'debug ms': 1, 'debug objclass': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-05T19:39:19.130 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-10-05_14:30:58-rgw-wip-yuri2-testing-2023-10-04-1333-quincy-distro-default-smithi/7411759', 'branch': 'wip-yuri2-testing-2023-10-04-1333-quincy', 'description': 'rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated supported-random-distro$/{ubuntu_latest}}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7411759', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'yuriw-2023-10-05_14:30:58-rgw-wip-yuri2-testing-2023-10-04-1333-quincy-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 3, 'size': 10}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'wip-yuri2-testing-2023-10-04-1333-quincy'}, 'ceph': {'conf': {'client': {'debug rgw': 20, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'sha1': '487ffcb5ddb60d29c5688d1a457e812b1b697b90', 'wait-for-scrub': False}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '487ffcb5ddb60d29c5688d1a457e812b1b697b90'}}, 'rgw': {'ec-data-pool': False, 'frontend': 'beast'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri2-testing-2023-10-04-1333-quincy', 'sha1': '487ffcb5ddb60d29c5688d1a457e812b1b697b90'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'osd.0'], ['mgr.x', 'client.0', 'osd.1', 'osd.2', 'osd.3']], 'seed': 880, 'sha1': '487ffcb5ddb60d29c5688d1a457e812b1b697b90', 'sleep_before_teardown': 0, 'subset': '111/12000', 'suite': 'rgw', 'suite_branch': 'wip-yuri2-testing-2023-10-04-1333-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_487ffcb5ddb60d29c5688d1a457e812b1b697b90/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '487ffcb5ddb60d29c5688d1a457e812b1b697b90', 'targets': {'smithi008.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM=', 'smithi012.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'client': {'debug ms': 1, 'rgw gc obj min wait': 15, 'debug rgw': 20, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'osd': {'debug ms': 1, 'debug objclass': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'sha1': '487ffcb5ddb60d29c5688d1a457e812b1b697b90', 'wait-for-scrub': False, 'cluster': 'ceph'}}, {'rgw': {'client.0': None}}, {'radosgw-admin': None}, {'radosgw-admin-rest': None}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '54e62bcbac4e53d9685e08328b790d3b20d71cae', 'timestamp': '2023-10-05_14:30:58', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-10-05T19:39:19.130 DEBUG:teuthology.orchestra.run.smithi008:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-05T19:39:19.133 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-05T19:39:19.140 DEBUG:teuthology.orchestra.run.smithi008:> sudo install -d -m0777 -- /var/run/ceph 2023-10-05T19:39:19.142 DEBUG:teuthology.orchestra.run.smithi012:> sudo install -d -m0777 -- /var/run/ceph 2023-10-05T19:39:19.158 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-10-05T19:39:19.158 DEBUG:teuthology.orchestra.run.smithi008:> dd if=/scratch_devs of=/dev/stdout 2023-10-05T19:39:19.164 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-05T19:39:19.164 DEBUG:teuthology.orchestra.run.smithi008:> stat /dev/vg_nvme/lv_1 2023-10-05T19:39:19.214 INFO:teuthology.orchestra.run.smithi008.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-05T19:39:19.214 INFO:teuthology.orchestra.run.smithi008.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-05T19:39:19.215 INFO:teuthology.orchestra.run.smithi008.stdout:Device: 6h/6d Inode: 360 Links: 1 2023-10-05T19:39:19.215 INFO:teuthology.orchestra.run.smithi008.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-05T19:39:19.215 INFO:teuthology.orchestra.run.smithi008.stdout:Access: 2023-10-05 19:35:33.358666169 +0000 2023-10-05T19:39:19.215 INFO:teuthology.orchestra.run.smithi008.stdout:Modify: 2023-10-05 19:35:33.094672257 +0000 2023-10-05T19:39:19.215 INFO:teuthology.orchestra.run.smithi008.stdout:Change: 2023-10-05 19:35:33.094672257 +0000 2023-10-05T19:39:19.215 INFO:teuthology.orchestra.run.smithi008.stdout: Birth: - 2023-10-05T19:39:19.215 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-05T19:39:19.228 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records in 2023-10-05T19:39:19.229 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records out 2023-10-05T19:39:19.229 INFO:teuthology.orchestra.run.smithi008.stderr:512 bytes copied, 0.000301076 s, 1.7 MB/s 2023-10-05T19:39:19.230 DEBUG:teuthology.orchestra.run.smithi008:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-05T19:39:19.239 DEBUG:teuthology.orchestra.run.smithi008:> stat /dev/vg_nvme/lv_2 2023-10-05T19:39:19.244 INFO:teuthology.orchestra.run.smithi008.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-05T19:39:19.245 INFO:teuthology.orchestra.run.smithi008.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-05T19:39:19.245 INFO:teuthology.orchestra.run.smithi008.stdout:Device: 6h/6d Inode: 367 Links: 1 2023-10-05T19:39:19.245 INFO:teuthology.orchestra.run.smithi008.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-05T19:39:19.245 INFO:teuthology.orchestra.run.smithi008.stdout:Access: 2023-10-05 19:35:33.834655192 +0000 2023-10-05T19:39:19.245 INFO:teuthology.orchestra.run.smithi008.stdout:Modify: 2023-10-05 19:35:33.574661187 +0000 2023-10-05T19:39:19.245 INFO:teuthology.orchestra.run.smithi008.stdout:Change: 2023-10-05 19:35:33.574661187 +0000 2023-10-05T19:39:19.245 INFO:teuthology.orchestra.run.smithi008.stdout: Birth: - 2023-10-05T19:39:19.246 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-05T19:39:19.306 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records in 2023-10-05T19:39:19.306 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records out 2023-10-05T19:39:19.306 INFO:teuthology.orchestra.run.smithi008.stderr:512 bytes copied, 0.000312086 s, 1.6 MB/s 2023-10-05T19:39:19.307 DEBUG:teuthology.orchestra.run.smithi008:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-05T19:39:19.315 DEBUG:teuthology.orchestra.run.smithi008:> stat /dev/vg_nvme/lv_3 2023-10-05T19:39:19.322 INFO:teuthology.orchestra.run.smithi008.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-05T19:39:19.322 INFO:teuthology.orchestra.run.smithi008.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-05T19:39:19.322 INFO:teuthology.orchestra.run.smithi008.stdout:Device: 6h/6d Inode: 374 Links: 1 2023-10-05T19:39:19.322 INFO:teuthology.orchestra.run.smithi008.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-05T19:39:19.322 INFO:teuthology.orchestra.run.smithi008.stdout:Access: 2023-10-05 19:35:34.326643845 +0000 2023-10-05T19:39:19.322 INFO:teuthology.orchestra.run.smithi008.stdout:Modify: 2023-10-05 19:35:34.062649934 +0000 2023-10-05T19:39:19.322 INFO:teuthology.orchestra.run.smithi008.stdout:Change: 2023-10-05 19:35:34.062649934 +0000 2023-10-05T19:39:19.323 INFO:teuthology.orchestra.run.smithi008.stdout: Birth: - 2023-10-05T19:39:19.323 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-05T19:39:19.376 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records in 2023-10-05T19:39:19.376 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records out 2023-10-05T19:39:19.376 INFO:teuthology.orchestra.run.smithi008.stderr:512 bytes copied, 0.000269164 s, 1.9 MB/s 2023-10-05T19:39:19.377 DEBUG:teuthology.orchestra.run.smithi008:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-05T19:39:19.385 DEBUG:teuthology.orchestra.run.smithi008:> stat /dev/vg_nvme/lv_4 2023-10-05T19:39:19.392 INFO:teuthology.orchestra.run.smithi008.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-05T19:39:19.392 INFO:teuthology.orchestra.run.smithi008.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-05T19:39:19.392 INFO:teuthology.orchestra.run.smithi008.stdout:Device: 6h/6d Inode: 378 Links: 1 2023-10-05T19:39:19.392 INFO:teuthology.orchestra.run.smithi008.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-05T19:39:19.392 INFO:teuthology.orchestra.run.smithi008.stdout:Access: 2023-10-05 19:35:34.786633237 +0000 2023-10-05T19:39:19.392 INFO:teuthology.orchestra.run.smithi008.stdout:Modify: 2023-10-05 19:35:34.518639418 +0000 2023-10-05T19:39:19.392 INFO:teuthology.orchestra.run.smithi008.stdout:Change: 2023-10-05 19:35:34.518639418 +0000 2023-10-05T19:39:19.392 INFO:teuthology.orchestra.run.smithi008.stdout: Birth: - 2023-10-05T19:39:19.393 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-05T19:39:19.405 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records in 2023-10-05T19:39:19.405 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records out 2023-10-05T19:39:19.405 INFO:teuthology.orchestra.run.smithi008.stderr:512 bytes copied, 0.00030884 s, 1.7 MB/s 2023-10-05T19:39:19.407 DEBUG:teuthology.orchestra.run.smithi008:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-05T19:39:19.460 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1'} 2023-10-05T19:39:19.461 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-05T19:39:19.461 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/scratch_devs of=/dev/stdout 2023-10-05T19:39:19.468 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-05T19:39:19.468 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_1 2023-10-05T19:39:19.476 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-05T19:39:19.476 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-05T19:39:19.476 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 6h/6d Inode: 358 Links: 1 2023-10-05T19:39:19.476 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-05T19:39:19.476 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2023-10-05 19:35:36.263761302 +0000 2023-10-05T19:39:19.476 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2023-10-05 19:35:35.983767760 +0000 2023-10-05T19:39:19.476 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2023-10-05 19:35:35.983767760 +0000 2023-10-05T19:39:19.476 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: - 2023-10-05T19:39:19.477 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-05T19:39:19.488 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2023-10-05T19:39:19.488 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2023-10-05T19:39:19.488 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000417461 s, 1.2 MB/s 2023-10-05T19:39:19.489 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-05T19:39:19.541 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_2 2023-10-05T19:39:19.549 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-05T19:39:19.549 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-05T19:39:19.549 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 6h/6d Inode: 366 Links: 1 2023-10-05T19:39:19.549 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-05T19:39:19.549 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2023-10-05 19:35:36.767749681 +0000 2023-10-05T19:39:19.550 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2023-10-05 19:35:36.503755769 +0000 2023-10-05T19:39:19.550 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2023-10-05 19:35:36.503755769 +0000 2023-10-05T19:39:19.550 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: - 2023-10-05T19:39:19.550 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-05T19:39:19.563 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2023-10-05T19:39:19.564 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2023-10-05T19:39:19.564 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000390871 s, 1.3 MB/s 2023-10-05T19:39:19.565 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-05T19:39:19.573 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_3 2023-10-05T19:39:19.623 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-05T19:39:19.624 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-05T19:39:19.624 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 6h/6d Inode: 372 Links: 1 2023-10-05T19:39:19.624 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-05T19:39:19.624 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2023-10-05 19:35:37.259738334 +0000 2023-10-05T19:39:19.624 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2023-10-05 19:35:36.971744976 +0000 2023-10-05T19:39:19.624 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2023-10-05 19:35:36.971744976 +0000 2023-10-05T19:39:19.624 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: - 2023-10-05T19:39:19.624 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-05T19:39:19.639 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2023-10-05T19:39:19.639 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2023-10-05T19:39:19.639 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000397235 s, 1.3 MB/s 2023-10-05T19:39:19.640 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-05T19:39:19.648 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_4 2023-10-05T19:39:19.699 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-05T19:39:19.699 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-05T19:39:19.700 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 6h/6d Inode: 379 Links: 1 2023-10-05T19:39:19.700 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-05T19:39:19.700 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2023-10-05 19:35:37.791726065 +0000 2023-10-05T19:39:19.700 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2023-10-05 19:35:37.503732706 +0000 2023-10-05T19:39:19.700 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2023-10-05 19:35:37.503732706 +0000 2023-10-05T19:39:19.700 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: - 2023-10-05T19:39:19.700 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-05T19:39:19.715 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2023-10-05T19:39:19.715 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2023-10-05T19:39:19.716 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000416294 s, 1.2 MB/s 2023-10-05T19:39:19.717 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-05T19:39:19.725 INFO:tasks.ceph:osd dev map: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2023-10-05T19:39:19.725 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi008.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1'}, Remote(name='ubuntu@smithi012.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'}} 2023-10-05T19:39:19.726 INFO:tasks.ceph:Generating config... 2023-10-05T19:39:19.727 INFO:tasks.ceph:[client] debug ms = 1 2023-10-05T19:39:19.727 INFO:tasks.ceph:[client] rgw gc obj min wait = 15 2023-10-05T19:39:19.727 INFO:tasks.ceph:[client] debug rgw = 20 2023-10-05T19:39:19.727 INFO:tasks.ceph:[client] setgroup = ceph 2023-10-05T19:39:19.727 INFO:tasks.ceph:[client] setuser = ceph 2023-10-05T19:39:19.727 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-05T19:39:19.727 INFO:tasks.ceph:[osd] debug objclass = 20 2023-10-05T19:39:19.728 INFO:tasks.ceph:[osd] bdev async discard = True 2023-10-05T19:39:19.728 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-10-05T19:39:19.728 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-10-05T19:39:19.728 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-05T19:39:19.728 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-05T19:39:19.728 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-10-05T19:39:19.728 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-10-05T19:39:19.728 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-05T19:39:19.728 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-10-05T19:39:19.728 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-05T19:39:19.729 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-05T19:39:19.729 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-05T19:39:19.729 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-10-05T19:39:19.729 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-10-05T19:39:19.729 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-05T19:39:19.729 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-05T19:39:19.729 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-05T19:39:19.729 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-05T19:39:19.729 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-05T19:39:19.730 INFO:tasks.ceph:Setting up mon.a... 2023-10-05T19:39:19.730 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-05T19:39:19.764 INFO:teuthology.orchestra.run.smithi008.stdout:creating /etc/ceph/ceph.keyring 2023-10-05T19:39:19.766 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-05T19:39:19.801 DEBUG:teuthology.orchestra.run.smithi008:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-05T19:39:19.811 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.8')] 2023-10-05T19:39:19.812 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.8'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug ms': 1, 'debug objclass': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug ms': 1, 'rgw gc obj min wait': 15, 'debug rgw': 20, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'mon.a': {}} 2023-10-05T19:39:19.813 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-10-05T19:39:19.813 DEBUG:teuthology.orchestra.run.smithi008:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-05T19:39:19.862 DEBUG:teuthology.orchestra.run.smithi008:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.8 --print /home/ubuntu/cephtest/ceph.monmap 2023-10-05T19:39:19.894 INFO:teuthology.orchestra.run.smithi008.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:39:19.894 INFO:teuthology.orchestra.run.smithi008.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:39:19.895 INFO:teuthology.orchestra.run.smithi008.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-05T19:39:19.895 INFO:teuthology.orchestra.run.smithi008.stdout:monmaptool: generated fsid dac6c186-0634-45d9-839d-04f1e484462d 2023-10-05T19:39:19.895 INFO:teuthology.orchestra.run.smithi008.stdout:setting min_mon_release = octopus 2023-10-05T19:39:19.895 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 0 2023-10-05T19:39:19.895 INFO:teuthology.orchestra.run.smithi008.stdout:fsid dac6c186-0634-45d9-839d-04f1e484462d 2023-10-05T19:39:19.895 INFO:teuthology.orchestra.run.smithi008.stdout:last_changed 2023-10-05T19:39:19.892868+0000 2023-10-05T19:39:19.895 INFO:teuthology.orchestra.run.smithi008.stdout:created 2023-10-05T19:39:19.892868+0000 2023-10-05T19:39:19.896 INFO:teuthology.orchestra.run.smithi008.stdout:min_mon_release 15 (octopus) 2023-10-05T19:39:19.896 INFO:teuthology.orchestra.run.smithi008.stdout:election_strategy: 1 2023-10-05T19:39:19.896 INFO:teuthology.orchestra.run.smithi008.stdout:0: [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] mon.a 2023-10-05T19:39:19.896 INFO:teuthology.orchestra.run.smithi008.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2023-10-05T19:39:19.897 DEBUG:teuthology.orchestra.run.smithi008:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-05T19:39:19.903 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID dac6c186-0634-45d9-839d-04f1e484462d... 2023-10-05T19:39:19.904 DEBUG:teuthology.orchestra.run.smithi008:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-05T19:39:19.906 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-05T19:39:19.937 INFO:teuthology.orchestra.run.smithi008.stdout:[global] 2023-10-05T19:39:19.938 INFO:teuthology.orchestra.run.smithi008.stdout: chdir = "" 2023-10-05T19:39:19.938 INFO:teuthology.orchestra.run.smithi008.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-05T19:39:19.938 INFO:teuthology.orchestra.run.smithi008.stdout: auth supported = cephx 2023-10-05T19:39:19.938 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-05T19:39:19.938 INFO:teuthology.orchestra.run.smithi008.stdout: filestore xattr use omap = true 2023-10-05T19:39:19.938 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-05T19:39:19.938 INFO:teuthology.orchestra.run.smithi008.stdout: mon clock drift allowed = 1.000 2023-10-05T19:39:19.938 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-05T19:39:19.938 INFO:teuthology.orchestra.run.smithi008.stdout: osd crush chooseleaf type = 0 2023-10-05T19:39:19.939 INFO:teuthology.orchestra.run.smithi008.stdout: auth debug = true 2023-10-05T19:39:19.939 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-05T19:39:19.939 INFO:teuthology.orchestra.run.smithi008.stdout: ms die on old message = true 2023-10-05T19:39:19.939 INFO:teuthology.orchestra.run.smithi008.stdout: ms die on bug = true 2023-10-05T19:39:19.939 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-05T19:39:19.939 INFO:teuthology.orchestra.run.smithi008.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-05T19:39:19.939 INFO:teuthology.orchestra.run.smithi008.stdout: mon pg warn max object skew = 0 2023-10-05T19:39:19.939 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-05T19:39:19.939 INFO:teuthology.orchestra.run.smithi008.stdout: # disable pg_autoscaler by default for new pools 2023-10-05T19:39:19.939 INFO:teuthology.orchestra.run.smithi008.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-05T19:39:19.939 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-05T19:39:19.940 INFO:teuthology.orchestra.run.smithi008.stdout: osd pool default size = 2 2023-10-05T19:39:19.940 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-05T19:39:19.940 INFO:teuthology.orchestra.run.smithi008.stdout: mon osd allow primary affinity = true 2023-10-05T19:39:19.940 INFO:teuthology.orchestra.run.smithi008.stdout: mon osd allow pg remap = true 2023-10-05T19:39:19.940 INFO:teuthology.orchestra.run.smithi008.stdout: mon warn on legacy crush tunables = false 2023-10-05T19:39:19.940 INFO:teuthology.orchestra.run.smithi008.stdout: mon warn on crush straw calc version zero = false 2023-10-05T19:39:19.940 INFO:teuthology.orchestra.run.smithi008.stdout: mon warn on no sortbitwise = false 2023-10-05T19:39:19.940 INFO:teuthology.orchestra.run.smithi008.stdout: mon warn on osd down out interval zero = false 2023-10-05T19:39:19.940 INFO:teuthology.orchestra.run.smithi008.stdout: mon warn on too few osds = false 2023-10-05T19:39:19.940 INFO:teuthology.orchestra.run.smithi008.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-05T19:39:19.940 INFO:teuthology.orchestra.run.smithi008.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-05T19:39:19.940 INFO:teuthology.orchestra.run.smithi008.stdout: mon_allow_pool_size_one = true 2023-10-05T19:39:19.940 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-05T19:39:19.941 INFO:teuthology.orchestra.run.smithi008.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-05T19:39:19.941 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-05T19:39:19.941 INFO:teuthology.orchestra.run.smithi008.stdout: osd default data pool replay window = 5 2023-10-05T19:39:19.941 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-05T19:39:19.941 INFO:teuthology.orchestra.run.smithi008.stdout: mon allow pool delete = true 2023-10-05T19:39:19.941 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-05T19:39:19.941 INFO:teuthology.orchestra.run.smithi008.stdout: mon cluster log file level = debug 2023-10-05T19:39:19.941 INFO:teuthology.orchestra.run.smithi008.stdout: debug asserts on shutdown = true 2023-10-05T19:39:19.941 INFO:teuthology.orchestra.run.smithi008.stdout: mon health detail to clog = false 2023-10-05T19:39:19.941 INFO:teuthology.orchestra.run.smithi008.stdout: mon host = 172.21.15.8 2023-10-05T19:39:19.941 INFO:teuthology.orchestra.run.smithi008.stdout: fsid = dac6c186-0634-45d9-839d-04f1e484462d 2023-10-05T19:39:19.941 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-05T19:39:19.942 INFO:teuthology.orchestra.run.smithi008.stdout:[osd] 2023-10-05T19:39:19.942 INFO:teuthology.orchestra.run.smithi008.stdout: osd journal size = 100 2023-10-05T19:39:19.942 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-05T19:39:19.942 INFO:teuthology.orchestra.run.smithi008.stdout: osd scrub load threshold = 5.0 2023-10-05T19:39:19.942 INFO:teuthology.orchestra.run.smithi008.stdout: osd scrub max interval = 600 2023-10-05T19:39:19.942 INFO:teuthology.orchestra.run.smithi008.stdout: osd mclock profile = high_recovery_ops 2023-10-05T19:39:19.942 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-05T19:39:19.942 INFO:teuthology.orchestra.run.smithi008.stdout: osd recover clone overlap = true 2023-10-05T19:39:19.942 INFO:teuthology.orchestra.run.smithi008.stdout: osd recovery max chunk = 1048576 2023-10-05T19:39:19.942 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-05T19:39:19.942 INFO:teuthology.orchestra.run.smithi008.stdout: osd debug shutdown = true 2023-10-05T19:39:19.943 INFO:teuthology.orchestra.run.smithi008.stdout: osd debug op order = true 2023-10-05T19:39:19.943 INFO:teuthology.orchestra.run.smithi008.stdout: osd debug verify stray on activate = true 2023-10-05T19:39:19.943 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-05T19:39:19.943 INFO:teuthology.orchestra.run.smithi008.stdout: osd open classes on start = true 2023-10-05T19:39:19.943 INFO:teuthology.orchestra.run.smithi008.stdout: osd debug pg log writeout = true 2023-10-05T19:39:19.943 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-05T19:39:19.943 INFO:teuthology.orchestra.run.smithi008.stdout: osd deep scrub update digest min age = 30 2023-10-05T19:39:19.943 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-05T19:39:19.943 INFO:teuthology.orchestra.run.smithi008.stdout: osd map max advance = 10 2023-10-05T19:39:19.943 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-05T19:39:19.944 INFO:teuthology.orchestra.run.smithi008.stdout: journal zero on create = true 2023-10-05T19:39:19.944 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-05T19:39:19.944 INFO:teuthology.orchestra.run.smithi008.stdout: filestore ondisk finisher threads = 3 2023-10-05T19:39:19.944 INFO:teuthology.orchestra.run.smithi008.stdout: filestore apply finisher threads = 3 2023-10-05T19:39:19.944 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-05T19:39:19.944 INFO:teuthology.orchestra.run.smithi008.stdout: bdev debug aio = true 2023-10-05T19:39:19.944 INFO:teuthology.orchestra.run.smithi008.stdout: osd debug misdirected ops = true 2023-10-05T19:39:19.944 INFO:teuthology.orchestra.run.smithi008.stdout: debug ms = 1 2023-10-05T19:39:19.944 INFO:teuthology.orchestra.run.smithi008.stdout: debug objclass = 20 2023-10-05T19:39:19.944 INFO:teuthology.orchestra.run.smithi008.stdout: bdev async discard = True 2023-10-05T19:39:19.944 INFO:teuthology.orchestra.run.smithi008.stdout: bdev enable discard = True 2023-10-05T19:39:19.945 INFO:teuthology.orchestra.run.smithi008.stdout: bluestore allocator = bitmap 2023-10-05T19:39:19.945 INFO:teuthology.orchestra.run.smithi008.stdout: bluestore block size = 96636764160 2023-10-05T19:39:19.945 INFO:teuthology.orchestra.run.smithi008.stdout: bluestore fsck on mount = True 2023-10-05T19:39:19.945 INFO:teuthology.orchestra.run.smithi008.stdout: debug bluefs = 1/20 2023-10-05T19:39:19.945 INFO:teuthology.orchestra.run.smithi008.stdout: debug bluestore = 1/20 2023-10-05T19:39:19.945 INFO:teuthology.orchestra.run.smithi008.stdout: debug osd = 20 2023-10-05T19:39:19.945 INFO:teuthology.orchestra.run.smithi008.stdout: debug rocksdb = 4/10 2023-10-05T19:39:19.945 INFO:teuthology.orchestra.run.smithi008.stdout: mon osd backfillfull_ratio = 0.85 2023-10-05T19:39:19.945 INFO:teuthology.orchestra.run.smithi008.stdout: mon osd full ratio = 0.9 2023-10-05T19:39:19.945 INFO:teuthology.orchestra.run.smithi008.stdout: mon osd nearfull ratio = 0.8 2023-10-05T19:39:19.945 INFO:teuthology.orchestra.run.smithi008.stdout: osd failsafe full ratio = 0.95 2023-10-05T19:39:19.945 INFO:teuthology.orchestra.run.smithi008.stdout: osd objectstore = bluestore 2023-10-05T19:39:19.946 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-05T19:39:19.946 INFO:teuthology.orchestra.run.smithi008.stdout:[mgr] 2023-10-05T19:39:19.946 INFO:teuthology.orchestra.run.smithi008.stdout: debug ms = 1 2023-10-05T19:39:19.946 INFO:teuthology.orchestra.run.smithi008.stdout: debug mgr = 20 2023-10-05T19:39:19.946 INFO:teuthology.orchestra.run.smithi008.stdout: debug mon = 20 2023-10-05T19:39:19.946 INFO:teuthology.orchestra.run.smithi008.stdout: debug auth = 20 2023-10-05T19:39:19.946 INFO:teuthology.orchestra.run.smithi008.stdout: mon reweight min pgs per osd = 4 2023-10-05T19:39:19.946 INFO:teuthology.orchestra.run.smithi008.stdout: mon reweight min bytes per osd = 10 2023-10-05T19:39:19.946 INFO:teuthology.orchestra.run.smithi008.stdout: mgr/telemetry/nag = false 2023-10-05T19:39:19.946 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-05T19:39:19.946 INFO:teuthology.orchestra.run.smithi008.stdout:[mon] 2023-10-05T19:39:19.947 INFO:teuthology.orchestra.run.smithi008.stdout: debug ms = 1 2023-10-05T19:39:19.947 INFO:teuthology.orchestra.run.smithi008.stdout: debug mon = 20 2023-10-05T19:39:19.947 INFO:teuthology.orchestra.run.smithi008.stdout: debug paxos = 20 2023-10-05T19:39:19.947 INFO:teuthology.orchestra.run.smithi008.stdout: debug auth = 20 2023-10-05T19:39:19.947 INFO:teuthology.orchestra.run.smithi008.stdout: mon data avail warn = 5 2023-10-05T19:39:19.947 INFO:teuthology.orchestra.run.smithi008.stdout: mon mgr mkfs grace = 240 2023-10-05T19:39:19.947 INFO:teuthology.orchestra.run.smithi008.stdout: mon reweight min pgs per osd = 4 2023-10-05T19:39:19.947 INFO:teuthology.orchestra.run.smithi008.stdout: mon osd reporter subtree level = osd 2023-10-05T19:39:19.947 INFO:teuthology.orchestra.run.smithi008.stdout: mon osd prime pg temp = true 2023-10-05T19:39:19.947 INFO:teuthology.orchestra.run.smithi008.stdout: mon reweight min bytes per osd = 10 2023-10-05T19:39:19.947 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-05T19:39:19.947 INFO:teuthology.orchestra.run.smithi008.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-05T19:39:19.947 INFO:teuthology.orchestra.run.smithi008.stdout: auth mon ticket ttl = 660 # 11m 2023-10-05T19:39:19.948 INFO:teuthology.orchestra.run.smithi008.stdout: auth service ticket ttl = 240 # 4m 2023-10-05T19:39:19.948 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-05T19:39:19.948 INFO:teuthology.orchestra.run.smithi008.stdout: # don't complain about insecure global_id in the test suite 2023-10-05T19:39:19.948 INFO:teuthology.orchestra.run.smithi008.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-05T19:39:19.948 INFO:teuthology.orchestra.run.smithi008.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-05T19:39:19.948 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-05T19:39:19.948 INFO:teuthology.orchestra.run.smithi008.stdout: # 1m isn't quite enough 2023-10-05T19:39:19.948 INFO:teuthology.orchestra.run.smithi008.stdout: mon_down_mkfs_grace = 2m 2023-10-05T19:39:19.948 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-05T19:39:19.949 INFO:teuthology.orchestra.run.smithi008.stdout: mon_warn_on_filestore_osds = false 2023-10-05T19:39:19.949 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-05T19:39:19.949 INFO:teuthology.orchestra.run.smithi008.stdout:[client] 2023-10-05T19:39:19.949 INFO:teuthology.orchestra.run.smithi008.stdout: rgw cache enabled = true 2023-10-05T19:39:19.949 INFO:teuthology.orchestra.run.smithi008.stdout: rgw enable ops log = true 2023-10-05T19:39:19.949 INFO:teuthology.orchestra.run.smithi008.stdout: rgw enable usage log = true 2023-10-05T19:39:19.949 INFO:teuthology.orchestra.run.smithi008.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-05T19:39:19.949 INFO:teuthology.orchestra.run.smithi008.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-05T19:39:19.949 INFO:teuthology.orchestra.run.smithi008.stdout: debug ms = 1 2023-10-05T19:39:19.949 INFO:teuthology.orchestra.run.smithi008.stdout: rgw gc obj min wait = 15 2023-10-05T19:39:19.949 INFO:teuthology.orchestra.run.smithi008.stdout: debug rgw = 20 2023-10-05T19:39:19.949 INFO:teuthology.orchestra.run.smithi008.stdout: setgroup = ceph 2023-10-05T19:39:19.950 INFO:teuthology.orchestra.run.smithi008.stdout: setuser = ceph 2023-10-05T19:39:19.950 INFO:teuthology.orchestra.run.smithi008.stdout:[mon.a] 2023-10-05T19:39:19.950 INFO:teuthology.orchestra.run.smithi012.stdout:[global] 2023-10-05T19:39:19.950 INFO:teuthology.orchestra.run.smithi012.stdout: chdir = "" 2023-10-05T19:39:19.950 INFO:teuthology.orchestra.run.smithi012.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-05T19:39:19.951 INFO:teuthology.orchestra.run.smithi012.stdout: auth supported = cephx 2023-10-05T19:39:19.951 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:39:19.951 INFO:teuthology.orchestra.run.smithi012.stdout: filestore xattr use omap = true 2023-10-05T19:39:19.951 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:39:19.951 INFO:teuthology.orchestra.run.smithi012.stdout: mon clock drift allowed = 1.000 2023-10-05T19:39:19.951 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:39:19.951 INFO:teuthology.orchestra.run.smithi012.stdout: osd crush chooseleaf type = 0 2023-10-05T19:39:19.951 INFO:teuthology.orchestra.run.smithi012.stdout: auth debug = true 2023-10-05T19:39:19.951 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:39:19.951 INFO:teuthology.orchestra.run.smithi012.stdout: ms die on old message = true 2023-10-05T19:39:19.951 INFO:teuthology.orchestra.run.smithi012.stdout: ms die on bug = true 2023-10-05T19:39:19.951 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:39:19.952 INFO:teuthology.orchestra.run.smithi012.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-05T19:39:19.952 INFO:teuthology.orchestra.run.smithi012.stdout: mon pg warn max object skew = 0 2023-10-05T19:39:19.952 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:39:19.952 INFO:teuthology.orchestra.run.smithi012.stdout: # disable pg_autoscaler by default for new pools 2023-10-05T19:39:19.952 INFO:teuthology.orchestra.run.smithi012.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-05T19:39:19.952 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:39:19.952 INFO:teuthology.orchestra.run.smithi012.stdout: osd pool default size = 2 2023-10-05T19:39:19.952 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:39:19.952 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd allow primary affinity = true 2023-10-05T19:39:19.952 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd allow pg remap = true 2023-10-05T19:39:19.952 INFO:teuthology.orchestra.run.smithi012.stdout: mon warn on legacy crush tunables = false 2023-10-05T19:39:19.952 INFO:teuthology.orchestra.run.smithi012.stdout: mon warn on crush straw calc version zero = false 2023-10-05T19:39:19.953 INFO:teuthology.orchestra.run.smithi012.stdout: mon warn on no sortbitwise = false 2023-10-05T19:39:19.953 INFO:teuthology.orchestra.run.smithi012.stdout: mon warn on osd down out interval zero = false 2023-10-05T19:39:19.953 INFO:teuthology.orchestra.run.smithi012.stdout: mon warn on too few osds = false 2023-10-05T19:39:19.953 INFO:teuthology.orchestra.run.smithi012.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-05T19:39:19.953 INFO:teuthology.orchestra.run.smithi012.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-05T19:39:19.953 INFO:teuthology.orchestra.run.smithi012.stdout: mon_allow_pool_size_one = true 2023-10-05T19:39:19.953 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:39:19.953 INFO:teuthology.orchestra.run.smithi012.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-05T19:39:19.953 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:39:19.953 INFO:teuthology.orchestra.run.smithi012.stdout: osd default data pool replay window = 5 2023-10-05T19:39:19.953 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:39:19.953 INFO:teuthology.orchestra.run.smithi012.stdout: mon allow pool delete = true 2023-10-05T19:39:19.954 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:39:19.954 INFO:teuthology.orchestra.run.smithi012.stdout: mon cluster log file level = debug 2023-10-05T19:39:19.954 INFO:teuthology.orchestra.run.smithi012.stdout: debug asserts on shutdown = true 2023-10-05T19:39:19.954 INFO:teuthology.orchestra.run.smithi012.stdout: mon health detail to clog = false 2023-10-05T19:39:19.954 INFO:teuthology.orchestra.run.smithi012.stdout: mon host = 172.21.15.8 2023-10-05T19:39:19.954 INFO:teuthology.orchestra.run.smithi012.stdout: fsid = dac6c186-0634-45d9-839d-04f1e484462d 2023-10-05T19:39:19.954 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:39:19.954 INFO:teuthology.orchestra.run.smithi012.stdout:[osd] 2023-10-05T19:39:19.954 INFO:teuthology.orchestra.run.smithi012.stdout: osd journal size = 100 2023-10-05T19:39:19.954 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:39:19.955 INFO:teuthology.orchestra.run.smithi012.stdout: osd scrub load threshold = 5.0 2023-10-05T19:39:19.955 INFO:teuthology.orchestra.run.smithi012.stdout: osd scrub max interval = 600 2023-10-05T19:39:19.955 INFO:teuthology.orchestra.run.smithi012.stdout: osd mclock profile = high_recovery_ops 2023-10-05T19:39:19.955 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:39:19.955 INFO:teuthology.orchestra.run.smithi012.stdout: osd recover clone overlap = true 2023-10-05T19:39:19.955 INFO:teuthology.orchestra.run.smithi012.stdout: osd recovery max chunk = 1048576 2023-10-05T19:39:19.955 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:39:19.955 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug shutdown = true 2023-10-05T19:39:19.955 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug op order = true 2023-10-05T19:39:19.955 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug verify stray on activate = true 2023-10-05T19:39:19.955 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:39:19.955 INFO:teuthology.orchestra.run.smithi012.stdout: osd open classes on start = true 2023-10-05T19:39:19.956 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug pg log writeout = true 2023-10-05T19:39:19.956 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:39:19.956 INFO:teuthology.orchestra.run.smithi012.stdout: osd deep scrub update digest min age = 30 2023-10-05T19:39:19.956 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:39:19.956 INFO:teuthology.orchestra.run.smithi012.stdout: osd map max advance = 10 2023-10-05T19:39:19.956 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:39:19.956 INFO:teuthology.orchestra.run.smithi012.stdout: journal zero on create = true 2023-10-05T19:39:19.956 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:39:19.956 INFO:teuthology.orchestra.run.smithi012.stdout: filestore ondisk finisher threads = 3 2023-10-05T19:39:19.956 INFO:teuthology.orchestra.run.smithi012.stdout: filestore apply finisher threads = 3 2023-10-05T19:39:19.956 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:39:19.956 INFO:teuthology.orchestra.run.smithi012.stdout: bdev debug aio = true 2023-10-05T19:39:19.957 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug misdirected ops = true 2023-10-05T19:39:19.957 INFO:teuthology.orchestra.run.smithi012.stdout: debug ms = 1 2023-10-05T19:39:19.957 INFO:teuthology.orchestra.run.smithi012.stdout: debug objclass = 20 2023-10-05T19:39:19.957 INFO:teuthology.orchestra.run.smithi012.stdout: bdev async discard = True 2023-10-05T19:39:19.957 INFO:teuthology.orchestra.run.smithi012.stdout: bdev enable discard = True 2023-10-05T19:39:19.957 INFO:teuthology.orchestra.run.smithi012.stdout: bluestore allocator = bitmap 2023-10-05T19:39:19.957 INFO:teuthology.orchestra.run.smithi012.stdout: bluestore block size = 96636764160 2023-10-05T19:39:19.957 INFO:teuthology.orchestra.run.smithi012.stdout: bluestore fsck on mount = True 2023-10-05T19:39:19.957 INFO:teuthology.orchestra.run.smithi012.stdout: debug bluefs = 1/20 2023-10-05T19:39:19.957 INFO:teuthology.orchestra.run.smithi012.stdout: debug bluestore = 1/20 2023-10-05T19:39:19.957 INFO:teuthology.orchestra.run.smithi012.stdout: debug osd = 20 2023-10-05T19:39:19.957 INFO:teuthology.orchestra.run.smithi012.stdout: debug rocksdb = 4/10 2023-10-05T19:39:19.958 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd backfillfull_ratio = 0.85 2023-10-05T19:39:19.958 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd full ratio = 0.9 2023-10-05T19:39:19.958 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd nearfull ratio = 0.8 2023-10-05T19:39:19.958 INFO:teuthology.orchestra.run.smithi012.stdout: osd failsafe full ratio = 0.95 2023-10-05T19:39:19.958 INFO:teuthology.orchestra.run.smithi012.stdout: osd objectstore = bluestore 2023-10-05T19:39:19.958 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:39:19.958 INFO:teuthology.orchestra.run.smithi012.stdout:[mgr] 2023-10-05T19:39:19.958 INFO:teuthology.orchestra.run.smithi012.stdout: debug ms = 1 2023-10-05T19:39:19.958 INFO:teuthology.orchestra.run.smithi012.stdout: debug mgr = 20 2023-10-05T19:39:19.958 INFO:teuthology.orchestra.run.smithi012.stdout: debug mon = 20 2023-10-05T19:39:19.958 INFO:teuthology.orchestra.run.smithi012.stdout: debug auth = 20 2023-10-05T19:39:19.959 INFO:teuthology.orchestra.run.smithi012.stdout: mon reweight min pgs per osd = 4 2023-10-05T19:39:19.959 INFO:teuthology.orchestra.run.smithi012.stdout: mon reweight min bytes per osd = 10 2023-10-05T19:39:19.959 INFO:teuthology.orchestra.run.smithi012.stdout: mgr/telemetry/nag = false 2023-10-05T19:39:19.959 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:39:19.959 INFO:teuthology.orchestra.run.smithi012.stdout:[mon] 2023-10-05T19:39:19.959 INFO:teuthology.orchestra.run.smithi012.stdout: debug ms = 1 2023-10-05T19:39:19.959 INFO:teuthology.orchestra.run.smithi012.stdout: debug mon = 20 2023-10-05T19:39:19.959 INFO:teuthology.orchestra.run.smithi012.stdout: debug paxos = 20 2023-10-05T19:39:19.959 INFO:teuthology.orchestra.run.smithi012.stdout: debug auth = 20 2023-10-05T19:39:19.959 INFO:teuthology.orchestra.run.smithi012.stdout: mon data avail warn = 5 2023-10-05T19:39:19.959 INFO:teuthology.orchestra.run.smithi012.stdout: mon mgr mkfs grace = 240 2023-10-05T19:39:19.959 INFO:teuthology.orchestra.run.smithi012.stdout: mon reweight min pgs per osd = 4 2023-10-05T19:39:19.959 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd reporter subtree level = osd 2023-10-05T19:39:19.960 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd prime pg temp = true 2023-10-05T19:39:19.960 INFO:teuthology.orchestra.run.smithi012.stdout: mon reweight min bytes per osd = 10 2023-10-05T19:39:19.960 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:39:19.960 INFO:teuthology.orchestra.run.smithi012.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-05T19:39:19.960 INFO:teuthology.orchestra.run.smithi012.stdout: auth mon ticket ttl = 660 # 11m 2023-10-05T19:39:19.960 INFO:teuthology.orchestra.run.smithi012.stdout: auth service ticket ttl = 240 # 4m 2023-10-05T19:39:19.960 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:39:19.960 INFO:teuthology.orchestra.run.smithi012.stdout: # don't complain about insecure global_id in the test suite 2023-10-05T19:39:19.960 INFO:teuthology.orchestra.run.smithi012.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-05T19:39:19.961 INFO:teuthology.orchestra.run.smithi012.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-05T19:39:19.961 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:39:19.961 INFO:teuthology.orchestra.run.smithi012.stdout: # 1m isn't quite enough 2023-10-05T19:39:19.961 INFO:teuthology.orchestra.run.smithi012.stdout: mon_down_mkfs_grace = 2m 2023-10-05T19:39:19.961 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:39:19.961 INFO:teuthology.orchestra.run.smithi012.stdout: mon_warn_on_filestore_osds = false 2023-10-05T19:39:19.961 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:39:19.961 INFO:teuthology.orchestra.run.smithi012.stdout:[client] 2023-10-05T19:39:19.961 INFO:teuthology.orchestra.run.smithi012.stdout: rgw cache enabled = true 2023-10-05T19:39:19.961 INFO:teuthology.orchestra.run.smithi012.stdout: rgw enable ops log = true 2023-10-05T19:39:19.961 INFO:teuthology.orchestra.run.smithi012.stdout: rgw enable usage log = true 2023-10-05T19:39:19.962 INFO:teuthology.orchestra.run.smithi012.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-05T19:39:19.962 INFO:teuthology.orchestra.run.smithi012.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-05T19:39:19.962 INFO:teuthology.orchestra.run.smithi012.stdout: debug ms = 1 2023-10-05T19:39:19.962 INFO:teuthology.orchestra.run.smithi012.stdout: rgw gc obj min wait = 15 2023-10-05T19:39:19.962 INFO:teuthology.orchestra.run.smithi012.stdout: debug rgw = 20 2023-10-05T19:39:19.962 INFO:teuthology.orchestra.run.smithi012.stdout: setgroup = ceph 2023-10-05T19:39:19.962 INFO:teuthology.orchestra.run.smithi012.stdout: setuser = ceph 2023-10-05T19:39:19.962 INFO:teuthology.orchestra.run.smithi012.stdout:[mon.a] 2023-10-05T19:39:19.963 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-05T19:39:19.964 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-05T19:39:20.054 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-05T19:39:20.054 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-10-05T19:39:20.054 DEBUG:teuthology.orchestra.run.smithi008:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-05T19:39:20.060 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-10-05T19:39:20.061 DEBUG:teuthology.orchestra.run.smithi008:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-05T19:39:20.067 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi008.front.sepia.ceph.com 2023-10-05T19:39:20.067 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-10-05T19:39:20.067 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-05T19:39:20.067 DEBUG:teuthology.orchestra.run.smithi008:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-05T19:39:20.089 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-10-05T19:39:20.089 DEBUG:teuthology.orchestra.run.smithi008:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-05T19:39:20.095 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi012.front.sepia.ceph.com 2023-10-05T19:39:20.095 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-05T19:39:20.095 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-05T19:39:20.095 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-05T19:39:20.121 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-05T19:39:20.121 DEBUG:teuthology.orchestra.run.smithi012:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-05T19:39:20.128 INFO:tasks.ceph:Setting up mon nodes... 2023-10-05T19:39:20.129 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-05T19:39:20.129 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-05T19:39:20.170 INFO:teuthology.orchestra.run.smithi012.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-05T19:39:20.172 INFO:tasks.ceph:Setting up mds nodes... 2023-10-05T19:39:20.172 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-05T19:39:20.173 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-05T19:39:20.253 INFO:teuthology.orchestra.run.smithi012.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-05T19:39:20.267 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-05T19:39:20.267 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi008.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1'}, Remote(name='ubuntu@smithi012.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'}} 2023-10-05T19:39:20.267 DEBUG:teuthology.orchestra.run.smithi008:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-05T19:39:20.284 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1'} 2023-10-05T19:39:20.284 INFO:tasks.ceph:role: osd.0 2023-10-05T19:39:20.284 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi008.front.sepia.ceph.com 2023-10-05T19:39:20.284 DEBUG:teuthology.orchestra.run.smithi008:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-05T19:39:20.323 INFO:teuthology.orchestra.run.smithi008.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-05T19:39:20.324 INFO:teuthology.orchestra.run.smithi008.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-05T19:39:20.324 INFO:teuthology.orchestra.run.smithi008.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-05T19:39:20.324 INFO:teuthology.orchestra.run.smithi008.stdout: = reflink=1 2023-10-05T19:39:20.324 INFO:teuthology.orchestra.run.smithi008.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-05T19:39:20.324 INFO:teuthology.orchestra.run.smithi008.stdout: = sunit=0 swidth=0 blks 2023-10-05T19:39:20.324 INFO:teuthology.orchestra.run.smithi008.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-05T19:39:20.324 INFO:teuthology.orchestra.run.smithi008.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-05T19:39:20.324 INFO:teuthology.orchestra.run.smithi008.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-05T19:39:20.324 INFO:teuthology.orchestra.run.smithi008.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-05T19:39:20.325 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi008.front.sepia.ceph.com -o noatime 2023-10-05T19:39:20.325 DEBUG:teuthology.orchestra.run.smithi008:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-05T19:39:20.346 DEBUG:teuthology.orchestra.run.smithi008:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-05T19:39:20.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-05T19:39:20.353 INFO:teuthology.orchestra.run.smithi008.stderr:sudo: /sbin/restorecon: command not found 2023-10-05T19:39:20.353 DEBUG:teuthology.orchestra.run.smithi008:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-05T19:39:20.423 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:20.415+0000 7f8849ca63c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-05T19:39:20.423 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:20.415+0000 7f8849ca63c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-05T19:39:20.423 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:20.415+0000 7f8849ca63c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-05T19:39:20.423 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:20.415+0000 7f8849ca63c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-05T19:39:20.423 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:20.415+0000 7f8849ca63c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-05T19:39:20.423 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:20.415+0000 7f8849ca63c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-10-05T19:39:20.978 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:20.971+0000 7f8849ca63c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-05T19:39:21.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:21.755+0000 7f8849ca63c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-05T19:39:22.542 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:22.535+0000 7f8849ca63c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-05T19:39:23.413 DEBUG:teuthology.orchestra.run.smithi008:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-05T19:39:23.429 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-05T19:39:23.445 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2023-10-05T19:39:23.445 INFO:tasks.ceph:role: osd.1 2023-10-05T19:39:23.445 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi012.front.sepia.ceph.com 2023-10-05T19:39:23.446 DEBUG:teuthology.orchestra.run.smithi012:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-05T19:39:23.968 INFO:teuthology.orchestra.run.smithi012.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-05T19:39:23.968 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-05T19:39:23.968 INFO:teuthology.orchestra.run.smithi012.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-05T19:39:23.969 INFO:teuthology.orchestra.run.smithi012.stdout: = reflink=1 2023-10-05T19:39:23.969 INFO:teuthology.orchestra.run.smithi012.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-05T19:39:23.969 INFO:teuthology.orchestra.run.smithi012.stdout: = sunit=0 swidth=0 blks 2023-10-05T19:39:23.969 INFO:teuthology.orchestra.run.smithi012.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-05T19:39:23.969 INFO:teuthology.orchestra.run.smithi012.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-05T19:39:23.969 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-05T19:39:23.969 INFO:teuthology.orchestra.run.smithi012.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-05T19:39:23.970 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi012.front.sepia.ceph.com -o noatime 2023-10-05T19:39:23.970 DEBUG:teuthology.orchestra.run.smithi012:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-1 2023-10-05T19:39:24.067 DEBUG:teuthology.orchestra.run.smithi012:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-05T19:39:24.079 INFO:teuthology.orchestra.run.smithi012.stderr:sudo: /sbin/restorecon: command not found 2023-10-05T19:39:24.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-05T19:39:24.080 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-05T19:39:24.092 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2023-10-05T19:39:24.092 INFO:tasks.ceph:role: osd.2 2023-10-05T19:39:24.092 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi012.front.sepia.ceph.com 2023-10-05T19:39:24.092 DEBUG:teuthology.orchestra.run.smithi012:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-05T19:39:24.650 INFO:teuthology.orchestra.run.smithi012.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-05T19:39:24.650 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-05T19:39:24.651 INFO:teuthology.orchestra.run.smithi012.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-05T19:39:24.651 INFO:teuthology.orchestra.run.smithi012.stdout: = reflink=1 2023-10-05T19:39:24.651 INFO:teuthology.orchestra.run.smithi012.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-05T19:39:24.651 INFO:teuthology.orchestra.run.smithi012.stdout: = sunit=0 swidth=0 blks 2023-10-05T19:39:24.651 INFO:teuthology.orchestra.run.smithi012.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-05T19:39:24.651 INFO:teuthology.orchestra.run.smithi012.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-05T19:39:24.651 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-05T19:39:24.651 INFO:teuthology.orchestra.run.smithi012.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-05T19:39:24.652 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi012.front.sepia.ceph.com -o noatime 2023-10-05T19:39:24.652 DEBUG:teuthology.orchestra.run.smithi012:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-2 2023-10-05T19:39:24.684 DEBUG:teuthology.orchestra.run.smithi012:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-05T19:39:24.693 INFO:teuthology.orchestra.run.smithi012.stderr:sudo: /sbin/restorecon: command not found 2023-10-05T19:39:24.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-05T19:39:24.694 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-05T19:39:24.752 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2023-10-05T19:39:24.752 INFO:tasks.ceph:role: osd.3 2023-10-05T19:39:24.752 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi012.front.sepia.ceph.com 2023-10-05T19:39:24.752 DEBUG:teuthology.orchestra.run.smithi012:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-05T19:39:25.247 INFO:teuthology.orchestra.run.smithi012.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-05T19:39:25.247 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-05T19:39:25.247 INFO:teuthology.orchestra.run.smithi012.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-05T19:39:25.247 INFO:teuthology.orchestra.run.smithi012.stdout: = reflink=1 2023-10-05T19:39:25.248 INFO:teuthology.orchestra.run.smithi012.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-05T19:39:25.248 INFO:teuthology.orchestra.run.smithi012.stdout: = sunit=0 swidth=0 blks 2023-10-05T19:39:25.248 INFO:teuthology.orchestra.run.smithi012.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-05T19:39:25.248 INFO:teuthology.orchestra.run.smithi012.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-05T19:39:25.248 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-05T19:39:25.248 INFO:teuthology.orchestra.run.smithi012.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-05T19:39:25.249 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi012.front.sepia.ceph.com -o noatime 2023-10-05T19:39:25.249 DEBUG:teuthology.orchestra.run.smithi012:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-3 2023-10-05T19:39:25.298 DEBUG:teuthology.orchestra.run.smithi012:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-05T19:39:25.309 INFO:teuthology.orchestra.run.smithi012.stderr:sudo: /sbin/restorecon: command not found 2023-10-05T19:39:25.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-05T19:39:25.310 DEBUG:teuthology.orchestra.run.smithi012:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-05T19:39:25.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:25.368+0000 7fe4940163c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-05T19:39:25.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:25.368+0000 7fe4940163c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-05T19:39:25.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:25.368+0000 7fe4940163c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-05T19:39:25.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:25.368+0000 7fe4940163c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-05T19:39:25.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:25.368+0000 7fe4940163c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-05T19:39:25.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:25.372+0000 7fe4940163c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-10-05T19:39:25.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:25.936+0000 7fe4940163c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-05T19:39:26.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:26.728+0000 7fe4940163c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-05T19:39:27.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:27.508+0000 7fe4940163c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-05T19:39:28.378 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-05T19:39:28.434 DEBUG:teuthology.orchestra.run.smithi012:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-05T19:39:28.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:28.468+0000 7f5b09dc93c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-05T19:39:28.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:28.468+0000 7f5b09dc93c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-05T19:39:28.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:28.468+0000 7f5b09dc93c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-05T19:39:28.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:28.468+0000 7f5b09dc93c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-05T19:39:28.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:28.468+0000 7f5b09dc93c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-05T19:39:28.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:28.468+0000 7f5b09dc93c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-10-05T19:39:29.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:29.044+0000 7f5b09dc93c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-05T19:39:29.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:29.828+0000 7f5b09dc93c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-05T19:39:30.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:30.604+0000 7f5b09dc93c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-05T19:39:31.474 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-05T19:39:31.488 DEBUG:teuthology.orchestra.run.smithi012:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-05T19:39:31.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:31.564+0000 7f39eec8c3c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-05T19:39:31.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:31.564+0000 7f39eec8c3c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-05T19:39:31.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:31.564+0000 7f39eec8c3c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-05T19:39:31.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:31.564+0000 7f39eec8c3c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-05T19:39:31.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:31.564+0000 7f39eec8c3c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-05T19:39:31.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:31.564+0000 7f39eec8c3c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-10-05T19:39:32.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:32.120+0000 7f39eec8c3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-05T19:39:32.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:32.908+0000 7f39eec8c3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-05T19:39:33.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:33.696+0000 7f39eec8c3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-05T19:39:34.577 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-05T19:39:34.593 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-05T19:39:34.594 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-10-05T19:39:34.594 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-05T19:39:34.610 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-05T19:39:34.611 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-05T19:39:34.626 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-05T19:39:34.626 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-05T19:39:34.685 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-05T19:39:34.685 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-05T19:39:34.696 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-05T19:39:34.696 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-05T19:39:34.708 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-05T19:39:34.708 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-05T19:39:34.755 INFO:tasks.ceph:Adding keys to all mons... 2023-10-05T19:39:34.755 DEBUG:teuthology.orchestra.run.smithi008:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-05T19:39:34.769 INFO:teuthology.orchestra.run.smithi008.stdout:[osd.0] 2023-10-05T19:39:34.770 INFO:teuthology.orchestra.run.smithi008.stdout: key = AQDoEB9lH5gPGRAACDBxvsJkPByhFZeB0NhlVw== 2023-10-05T19:39:34.770 INFO:teuthology.orchestra.run.smithi008.stdout:[mgr.x] 2023-10-05T19:39:34.770 INFO:teuthology.orchestra.run.smithi008.stdout: key = AQDoEB9lKgz+CRAAJFaLbuncnwEahNXhRiG9gw== 2023-10-05T19:39:34.770 INFO:teuthology.orchestra.run.smithi008.stdout:[osd.1] 2023-10-05T19:39:34.770 INFO:teuthology.orchestra.run.smithi008.stdout: key = AQDtEB9l0/NAFhAAvFmdRhi+zlsVCkvNOVOw0w== 2023-10-05T19:39:34.770 INFO:teuthology.orchestra.run.smithi008.stdout:[osd.2] 2023-10-05T19:39:34.770 INFO:teuthology.orchestra.run.smithi008.stdout: key = AQDwEB9l3ComHBAAot54+Hzthcn0iE3XFt5j7g== 2023-10-05T19:39:34.770 INFO:teuthology.orchestra.run.smithi008.stdout:[osd.3] 2023-10-05T19:39:34.770 INFO:teuthology.orchestra.run.smithi008.stdout: key = AQDzEB9lyGnMIRAAURA0V6UVkHURwKVsvrKbLw== 2023-10-05T19:39:34.770 INFO:teuthology.orchestra.run.smithi008.stdout:[client.0] 2023-10-05T19:39:34.771 INFO:teuthology.orchestra.run.smithi008.stdout: key = AQDoEB9lnQn6DhAAk4YwzYeQjoFmZr34IQ2Nvw== 2023-10-05T19:39:34.771 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-05T19:39:34.812 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-05T19:39:34.900 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-05T19:39:35.030 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-05T19:39:35.069 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-05T19:39:35.156 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-05T19:39:35.206 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-05T19:39:35.206 DEBUG:teuthology.orchestra.run.smithi008:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-05T19:39:35.216 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-05T19:39:35.355 DEBUG:teuthology.orchestra.run.smithi008:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-05T19:39:35.370 DEBUG:teuthology.orchestra.run.smithi008:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-05T19:39:35.454 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-05T19:39:35.454 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-05T19:39:35.454 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-05T19:39:35.456 INFO:tasks.ceph.mon.a:Started 2023-10-05T19:39:35.457 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-05T19:39:35.457 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-05T19:39:35.457 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-05T19:39:35.459 INFO:tasks.ceph.mgr.x:Started 2023-10-05T19:39:35.459 INFO:tasks.ceph:Setting crush tunables to default 2023-10-05T19:39:35.459 DEBUG:teuthology.orchestra.run.smithi008:> sudo ceph --cluster ceph osd crush tunables default 2023-10-05T19:39:35.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.555+0000 7f380ed48700 1 Processor -- start 2023-10-05T19:39:35.561 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.555+0000 7f380ed48700 1 -- start start 2023-10-05T19:39:35.562 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.555+0000 7f380ed48700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f3808126bd0 0x7f380812b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:35.562 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.555+0000 7f380ed48700 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f380806e5e0 con 0x7f38081260f0 2023-10-05T19:39:35.562 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.555+0000 7f380ed48700 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f380806e810 con 0x7f3808126bd0 2023-10-05T19:39:35.562 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.555+0000 7f380cae4700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f38081260f0 0x7f38081264c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:60168/0 (socket says 172.21.15.8:60168) 2023-10-05T19:39:35.562 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.555+0000 7f380cae4700 1 -- 172.21.15.8:0/3056241761 learned_addr learned my addr 172.21.15.8:0/3056241761 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-05T19:39:35.562 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.555+0000 7f3807fff700 1 --2- 172.21.15.8:0/3056241761 >> v2:172.21.15.8:3300/0 conn(0x7f3808126bd0 0x7f380812b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:35.563 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.555+0000 7f3807fff700 1 -- 172.21.15.8:0/3056241761 >> v2:172.21.15.8:3300/0 conn(0x7f3808126bd0 msgr2=0x7f380812b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-05T19:39:35.563 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.555+0000 7f3807fff700 1 -- 172.21.15.8:0/3056241761 >> v2:172.21.15.8:3300/0 conn(0x7f3808126bd0 msgr2=0x7f380812b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-05T19:39:35.563 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.555+0000 7f3807fff700 1 --2- 172.21.15.8:0/3056241761 >> v2:172.21.15.8:3300/0 conn(0x7f3808126bd0 0x7f380812b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-05T19:39:35.563 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.555+0000 7f3807fff700 1 --2- 172.21.15.8:0/3056241761 >> v2:172.21.15.8:3300/0 conn(0x7f3808126bd0 0x7f380812b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-05T19:39:35.595 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.587+0000 7f38077fe700 1 -- 172.21.15.8:0/3056241761 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 852168274 0 0) 0x7f380806e5e0 con 0x7f38081260f0 2023-10-05T19:39:35.595 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.587+0000 7f38077fe700 1 -- 172.21.15.8:0/3056241761 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37f0003570 con 0x7f38081260f0 2023-10-05T19:39:35.595 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.587+0000 7f38077fe700 1 -- 172.21.15.8:0/3056241761 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1679690814 0 0) 0x7f37f0003570 con 0x7f38081260f0 2023-10-05T19:39:35.595 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.587+0000 7f38077fe700 1 -- 172.21.15.8:0/3056241761 >> v2:172.21.15.8:3300/0 conn(0x7f3808126bd0 msgr2=0x7f380812b670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-05T19:39:35.595 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.587+0000 7f38077fe700 1 --2- 172.21.15.8:0/3056241761 >> v2:172.21.15.8:3300/0 conn(0x7f3808126bd0 0x7f380812b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:35.596 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.587+0000 7f38077fe700 1 -- 172.21.15.8:0/3056241761 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f380806d420 con 0x7f38081260f0 2023-10-05T19:39:35.596 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.587+0000 7f38077fe700 1 -- 172.21.15.8:0/3056241761 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (unknown 1630025266 0 0) 0x7f3800001930 con 0x7f38081260f0 2023-10-05T19:39:35.596 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.587+0000 7f38077fe700 1 -- 172.21.15.8:0/3056241761 >> v1:172.21.15.8:6789/0 conn(0x7f38081260f0 legacy=0x7f38081264c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:35.596 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.587+0000 7f38077fe700 1 --2- 172.21.15.8:0/3056241761 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f37f0003a60 0x7f37f0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:35.596 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.587+0000 7f38077fe700 1 -- 172.21.15.8:0/3056241761 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f380806e810 con 0x7f37f0003a60 2023-10-05T19:39:35.596 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.587+0000 7f380cae4700 1 --2- 172.21.15.8:0/3056241761 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f37f0003a60 0x7f37f0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:35.597 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.591+0000 7f380cae4700 1 -- 172.21.15.8:0/3056241761 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f0003570 con 0x7f37f0003a60 2023-10-05T19:39:35.597 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.591+0000 7f380cae4700 1 --2- 172.21.15.8:0/3056241761 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f37f0003a60 0x7f37f0003e50 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f380000a850 tx=0x7f380806c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8206d68315c15bc9 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:35.597 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.591+0000 7f38077fe700 1 -- 172.21.15.8:0/3056241761 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f38000070f0 con 0x7f37f0003a60 2023-10-05T19:39:35.597 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.591+0000 7f38077fe700 1 -- 172.21.15.8:0/3056241761 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3800007250 con 0x7f37f0003a60 2023-10-05T19:39:35.597 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.591+0000 7f38077fe700 1 -- 172.21.15.8:0/3056241761 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3800007400 con 0x7f37f0003a60 2023-10-05T19:39:35.598 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.591+0000 7f380ed48700 1 -- 172.21.15.8:0/3056241761 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f37f0003a60 msgr2=0x7f37f0003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:35.598 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.591+0000 7f380ed48700 1 --2- 172.21.15.8:0/3056241761 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f37f0003a60 0x7f37f0003e50 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f380000a850 tx=0x7f380806c000 comp rx=0 tx=0).stop 2023-10-05T19:39:35.598 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.591+0000 7f380ed48700 1 -- 172.21.15.8:0/3056241761 shutdown_connections 2023-10-05T19:39:35.598 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.591+0000 7f380ed48700 1 --2- 172.21.15.8:0/3056241761 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f37f0003a60 0x7f37f0003e50 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:35.598 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.591+0000 7f380ed48700 1 --2- 172.21.15.8:0/3056241761 >> v2:172.21.15.8:3300/0 conn(0x7f3808126bd0 0x7f380812b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:35.598 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.591+0000 7f380ed48700 1 -- 172.21.15.8:0/3056241761 >> 172.21.15.8:0/3056241761 conn(0x7f3808098f60 msgr2=0x7f38080949f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:35.599 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.591+0000 7f380ed48700 1 -- 172.21.15.8:0/3056241761 shutdown_connections 2023-10-05T19:39:35.599 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.591+0000 7f380ed48700 1 -- 172.21.15.8:0/3056241761 wait complete. 2023-10-05T19:39:35.599 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.591+0000 7f380ed48700 1 Processor -- start 2023-10-05T19:39:35.600 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.591+0000 7f380ed48700 1 -- start start 2023-10-05T19:39:35.601 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.595+0000 7f380ed48700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f37f0003a60 0x7f3808112540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:35.601 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.595+0000 7f380ed48700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38081148b0 con 0x7f37f0003a60 2023-10-05T19:39:35.601 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.595+0000 7f380cae4700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f37f0003a60 0x7f3808112540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:35.601 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.595+0000 7f380cae4700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f37f0003a60 0x7f3808112540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41028/0 (socket says 172.21.15.8:41028) 2023-10-05T19:39:35.601 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.595+0000 7f380cae4700 1 -- 172.21.15.8:0/460983559 learned_addr learned my addr 172.21.15.8:0/460983559 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-05T19:39:35.601 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.595+0000 7f380cae4700 1 -- 172.21.15.8:0/460983559 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3808114680 con 0x7f37f0003a60 2023-10-05T19:39:35.602 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.595+0000 7f380cae4700 1 --2- 172.21.15.8:0/460983559 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f37f0003a60 0x7f3808112540 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3808068f40 tx=0x7f3800016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:35.602 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.595+0000 7f3804ff9700 1 -- 172.21.15.8:0/460983559 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f38000073c0 con 0x7f37f0003a60 2023-10-05T19:39:35.602 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.595+0000 7f3804ff9700 1 -- 172.21.15.8:0/460983559 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3800001550 con 0x7f37f0003a60 2023-10-05T19:39:35.603 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.595+0000 7f3804ff9700 1 -- 172.21.15.8:0/460983559 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3800007070 con 0x7f37f0003a60 2023-10-05T19:39:35.603 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.595+0000 7f380ed48700 1 -- 172.21.15.8:0/460983559 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3808113ec0 con 0x7f37f0003a60 2023-10-05T19:39:35.603 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.595+0000 7f380ed48700 1 -- 172.21.15.8:0/460983559 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3808115030 con 0x7f37f0003a60 2023-10-05T19:39:35.603 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.595+0000 7f3804ff9700 1 -- 172.21.15.8:0/460983559 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f3800024020 con 0x7f37f0003a60 2023-10-05T19:39:35.604 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.595+0000 7f3804ff9700 1 -- 172.21.15.8:0/460983559 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f380001b5d0 con 0x7f37f0003a60 2023-10-05T19:39:35.604 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.599+0000 7f380ed48700 1 -- 172.21.15.8:0/460983559 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3808097dc0 con 0x7f37f0003a60 2023-10-05T19:39:35.609 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.603+0000 7f3804ff9700 1 -- 172.21.15.8:0/460983559 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71325 (secure 0 0 0) 0x7f38000466b0 con 0x7f37f0003a60 2023-10-05T19:39:35.680 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.675+0000 7f380ed48700 1 -- 172.21.15.8:0/460983559 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f380805ecb0 con 0x7f37f0003a60 2023-10-05T19:39:35.685 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.679+0000 7f3804ff9700 1 -- 172.21.15.8:0/460983559 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f3800035070 con 0x7f37f0003a60 2023-10-05T19:39:35.689 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.679+0000 7f380ed48700 1 -- 172.21.15.8:0/460983559 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f37f0003a60 msgr2=0x7f3808112540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:35.690 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.679+0000 7f380ed48700 1 --2- 172.21.15.8:0/460983559 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f37f0003a60 0x7f3808112540 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3808068f40 tx=0x7f3800016000 comp rx=0 tx=0).stop 2023-10-05T19:39:35.690 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.679+0000 7f380ed48700 1 -- 172.21.15.8:0/460983559 shutdown_connections 2023-10-05T19:39:35.690 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.679+0000 7f380ed48700 1 --2- 172.21.15.8:0/460983559 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f37f0003a60 0x7f3808112540 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:35.690 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.679+0000 7f380ed48700 1 -- 172.21.15.8:0/460983559 >> 172.21.15.8:0/460983559 conn(0x7f3808098f60 msgr2=0x7f3808120e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:35.690 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.679+0000 7f380ed48700 1 -- 172.21.15.8:0/460983559 shutdown_connections 2023-10-05T19:39:35.690 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:35.679+0000 7f380ed48700 1 -- 172.21.15.8:0/460983559 wait complete. 2023-10-05T19:39:35.690 INFO:teuthology.orchestra.run.smithi008.stderr:adjusted tunables profile to default 2023-10-05T19:39:35.702 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-05T19:39:35.702 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-10-05T19:39:35.702 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-05T19:39:35.713 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-05T19:39:35.713 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-05T19:39:35.731 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-05T19:39:35.731 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-05T19:39:35.788 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-10-05T19:39:35.788 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-05T19:39:35.790 INFO:tasks.ceph.mgr.x.smithi012.stderr:2023-10-05T19:39:35.784+0000 7fde2aeb7000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-05T19:39:35.797 DEBUG:teuthology.orchestra.run.smithi012:> sudo ceph --cluster ceph osd new c23c6d99-5a1e-499d-8ef8-e1a58dcd9e74 0 2023-10-05T19:39:35.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.892+0000 7f8b06000700 1 Processor -- start 2023-10-05T19:39:35.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.892+0000 7f8b06000700 1 -- start start 2023-10-05T19:39:35.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.892+0000 7f8b06000700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f8b0010f8d0 0x7f8b0010fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:35.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.892+0000 7f8b06000700 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b00110270 con 0x7f8b000678b0 2023-10-05T19:39:35.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.892+0000 7f8b06000700 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b0004c3e0 con 0x7f8b0010f8d0 2023-10-05T19:39:35.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.892+0000 7f8b04ffe700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f8b0010f8d0 0x7f8b0010fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:35.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.896+0000 7f8b04ffe700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f8b0010f8d0 0x7f8b0010fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37450/0 (socket says 172.21.15.12:37450) 2023-10-05T19:39:35.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.896+0000 7f8b04ffe700 1 -- 172.21.15.12:0/332555723 learned_addr learned my addr 172.21.15.12:0/332555723 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:39:35.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.896+0000 7f8afffff700 1 --1- 172.21.15.12:0/332555723 >> v1:172.21.15.8:6789/0 conn(0x7f8b000678b0 0x7f8b00067d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:54852/0 (socket says 172.21.15.12:54852) 2023-10-05T19:39:35.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.896+0000 7f8b04ffe700 1 -- 172.21.15.12:0/332555723 >> v1:172.21.15.8:6789/0 conn(0x7f8b000678b0 legacy=0x7f8b00067d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:35.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.896+0000 7f8b04ffe700 1 -- 172.21.15.12:0/332555723 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b00068440 con 0x7f8b0010f8d0 2023-10-05T19:39:35.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.896+0000 7f8b04ffe700 1 --2- 172.21.15.12:0/332555723 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8b0010f8d0 0x7f8b0010fca0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8af4009930 tx=0x7f8af4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe8deef136507d8e server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:35.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.896+0000 7f8aff7fe700 1 -- 172.21.15.12:0/332555723 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8af400b860 con 0x7f8b0010f8d0 2023-10-05T19:39:35.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.896+0000 7f8aff7fe700 1 -- 172.21.15.12:0/332555723 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af400b9c0 con 0x7f8b0010f8d0 2023-10-05T19:39:35.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.896+0000 7f8aff7fe700 1 -- 172.21.15.12:0/332555723 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8af400bb50 con 0x7f8b0010f8d0 2023-10-05T19:39:35.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.896+0000 7f8b06000700 1 -- 172.21.15.12:0/332555723 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8b0010f8d0 msgr2=0x7f8b0010fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:35.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.896+0000 7f8b06000700 1 --2- 172.21.15.12:0/332555723 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8b0010f8d0 0x7f8b0010fca0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8af4009930 tx=0x7f8af4009250 comp rx=0 tx=0).stop 2023-10-05T19:39:35.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.896+0000 7f8b06000700 1 -- 172.21.15.12:0/332555723 shutdown_connections 2023-10-05T19:39:35.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.896+0000 7f8b06000700 1 --2- 172.21.15.12:0/332555723 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8b0010f8d0 0x7f8b0010fca0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:35.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.896+0000 7f8b06000700 1 -- 172.21.15.12:0/332555723 >> 172.21.15.12:0/332555723 conn(0x7f8b0006afc0 msgr2=0x7f8b0006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:35.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.896+0000 7f8b06000700 1 -- 172.21.15.12:0/332555723 shutdown_connections 2023-10-05T19:39:35.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.896+0000 7f8b06000700 1 -- 172.21.15.12:0/332555723 wait complete. 2023-10-05T19:39:35.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.896+0000 7f8b06000700 1 Processor -- start 2023-10-05T19:39:35.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.896+0000 7f8b06000700 1 -- start start 2023-10-05T19:39:35.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.896+0000 7f8b06000700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8b000678b0 0x7f8b00109e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:35.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.900+0000 7f8b06000700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0006b390 con 0x7f8b000678b0 2023-10-05T19:39:35.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.900+0000 7f8b04ffe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8b000678b0 0x7f8b00109e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:35.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.900+0000 7f8b04ffe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8b000678b0 0x7f8b00109e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37454/0 (socket says 172.21.15.12:37454) 2023-10-05T19:39:35.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.900+0000 7f8b04ffe700 1 -- 172.21.15.12:0/3107818183 learned_addr learned my addr 172.21.15.12:0/3107818183 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:39:35.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.900+0000 7f8b04ffe700 1 -- 172.21.15.12:0/3107818183 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af40094f0 con 0x7f8b000678b0 2023-10-05T19:39:35.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.900+0000 7f8b04ffe700 1 --2- 172.21.15.12:0/3107818183 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8b000678b0 0x7f8b00109e10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8b000645d0 tx=0x7f8af4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:35.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.900+0000 7f8afcff9700 1 -- 172.21.15.12:0/3107818183 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8af400bd50 con 0x7f8b000678b0 2023-10-05T19:39:35.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.900+0000 7f8afcff9700 1 -- 172.21.15.12:0/3107818183 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af4020070 con 0x7f8b000678b0 2023-10-05T19:39:35.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.900+0000 7f8b06000700 1 -- 172.21.15.12:0/3107818183 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b0010a350 con 0x7f8b000678b0 2023-10-05T19:39:35.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.900+0000 7f8afcff9700 1 -- 172.21.15.12:0/3107818183 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8af401b450 con 0x7f8b000678b0 2023-10-05T19:39:35.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.900+0000 7f8b06000700 1 -- 172.21.15.12:0/3107818183 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b0010a6e0 con 0x7f8b000678b0 2023-10-05T19:39:35.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.900+0000 7f8afcff9700 1 -- 172.21.15.12:0/3107818183 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f8af4022030 con 0x7f8b000678b0 2023-10-05T19:39:35.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.900+0000 7f8afcff9700 1 -- 172.21.15.12:0/3107818183 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7f8af401c7a0 con 0x7f8b000678b0 2023-10-05T19:39:35.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.900+0000 7f8b06000700 1 -- 172.21.15.12:0/3107818183 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b0005ecb0 con 0x7f8b000678b0 2023-10-05T19:39:35.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.908+0000 7f8afcff9700 1 -- 172.21.15.12:0/3107818183 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71325 (secure 0 0 0) 0x7f8af401ba80 con 0x7f8b000678b0 2023-10-05T19:39:35.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.984+0000 7f8b06000700 1 -- 172.21.15.12:0/3107818183 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c23c6d99-5a1e-499d-8ef8-e1a58dcd9e74", "id": 0} v 0) v1 -- 0x7f8b0010f8d0 con 0x7f8b000678b0 2023-10-05T19:39:35.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.992+0000 7f8afcff9700 1 -- 172.21.15.12:0/3107818183 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c23c6d99-5a1e-499d-8ef8-e1a58dcd9e74", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8af401bc20 con 0x7f8b000678b0 2023-10-05T19:39:35.998 INFO:teuthology.orchestra.run.smithi012.stdout:0 2023-10-05T19:39:35.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.992+0000 7f8b06000700 1 -- 172.21.15.12:0/3107818183 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8b000678b0 msgr2=0x7f8b00109e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:36.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.992+0000 7f8b06000700 1 --2- 172.21.15.12:0/3107818183 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8b000678b0 0x7f8b00109e10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8b000645d0 tx=0x7f8af4016000 comp rx=0 tx=0).stop 2023-10-05T19:39:36.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.992+0000 7f8b06000700 1 -- 172.21.15.12:0/3107818183 shutdown_connections 2023-10-05T19:39:36.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.992+0000 7f8b06000700 1 --2- 172.21.15.12:0/3107818183 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8b000678b0 0x7f8b00109e10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:36.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.992+0000 7f8b06000700 1 -- 172.21.15.12:0/3107818183 >> 172.21.15.12:0/3107818183 conn(0x7f8b0006afc0 msgr2=0x7f8b001125b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:36.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.992+0000 7f8b06000700 1 -- 172.21.15.12:0/3107818183 shutdown_connections 2023-10-05T19:39:36.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:35.992+0000 7f8b06000700 1 -- 172.21.15.12:0/3107818183 wait complete. 2023-10-05T19:39:36.012 DEBUG:teuthology.orchestra.run.smithi012:> sudo ceph --cluster ceph osd new e578b376-27a7-47e2-b262-9a2a703d2504 1 2023-10-05T19:39:36.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.108+0000 7fbb98c71700 1 Processor -- start 2023-10-05T19:39:36.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.108+0000 7fbb98c71700 1 -- start start 2023-10-05T19:39:36.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.108+0000 7fbb98c71700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fbb9406ea90 0x7fbb9406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:36.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.108+0000 7fbb98c71700 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb9410ed20 con 0x7fbb9406f480 2023-10-05T19:39:36.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.108+0000 7fbb98c71700 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb9404c3e0 con 0x7fbb9406ea90 2023-10-05T19:39:36.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.108+0000 7fbb937fe700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fbb9406ea90 0x7fbb9406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:36.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.108+0000 7fbb92ffd700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7fbb9406f480 0x7fbb9410e550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:54856/0 (socket says 172.21.15.12:54856) 2023-10-05T19:39:36.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.108+0000 7fbb937fe700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fbb9406ea90 0x7fbb9406ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37458/0 (socket says 172.21.15.12:37458) 2023-10-05T19:39:36.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.108+0000 7fbb92ffd700 1 -- 172.21.15.12:0/2439912516 learned_addr learned my addr 172.21.15.12:0/2439912516 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:39:36.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.108+0000 7fbb937fe700 1 -- 172.21.15.12:0/2439912516 >> v1:172.21.15.8:6789/0 conn(0x7fbb9406f480 legacy=0x7fbb9410e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:36.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.108+0000 7fbb937fe700 1 -- 172.21.15.12:0/2439912516 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb9410ef00 con 0x7fbb9406ea90 2023-10-05T19:39:36.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.108+0000 7fbb937fe700 1 --2- 172.21.15.12:0/2439912516 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbb9406ea90 0x7fbb9406ee80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbb84009860 tx=0x7fbb840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84aa39d1f4cd8054 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:36.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.108+0000 7fbb927fc700 1 -- 172.21.15.12:0/2439912516 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbb8400b680 con 0x7fbb9406ea90 2023-10-05T19:39:36.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.108+0000 7fbb927fc700 1 -- 172.21.15.12:0/2439912516 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb8400b7e0 con 0x7fbb9406ea90 2023-10-05T19:39:36.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.108+0000 7fbb927fc700 1 -- 172.21.15.12:0/2439912516 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbb8400b990 con 0x7fbb9406ea90 2023-10-05T19:39:36.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.112+0000 7fbb98c71700 1 -- 172.21.15.12:0/2439912516 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbb9406ea90 msgr2=0x7fbb9406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:36.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.112+0000 7fbb98c71700 1 --2- 172.21.15.12:0/2439912516 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbb9406ea90 0x7fbb9406ee80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbb84009860 tx=0x7fbb840091a0 comp rx=0 tx=0).stop 2023-10-05T19:39:36.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.112+0000 7fbb98c71700 1 -- 172.21.15.12:0/2439912516 shutdown_connections 2023-10-05T19:39:36.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.112+0000 7fbb98c71700 1 --2- 172.21.15.12:0/2439912516 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbb9406ea90 0x7fbb9406ee80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:36.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.112+0000 7fbb98c71700 1 -- 172.21.15.12:0/2439912516 >> 172.21.15.12:0/2439912516 conn(0x7fbb9406a740 msgr2=0x7fbb9406cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:36.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.112+0000 7fbb98c71700 1 -- 172.21.15.12:0/2439912516 shutdown_connections 2023-10-05T19:39:36.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.112+0000 7fbb98c71700 1 -- 172.21.15.12:0/2439912516 wait complete. 2023-10-05T19:39:36.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.112+0000 7fbb98c71700 1 Processor -- start 2023-10-05T19:39:36.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.112+0000 7fbb98c71700 1 -- start start 2023-10-05T19:39:36.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.112+0000 7fbb98c71700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbb9406ea90 0x7fbb94125ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:36.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.112+0000 7fbb98c71700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9406c310 con 0x7fbb9406ea90 2023-10-05T19:39:36.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.112+0000 7fbb937fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbb9406ea90 0x7fbb94125ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:36.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.112+0000 7fbb937fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbb9406ea90 0x7fbb94125ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37462/0 (socket says 172.21.15.12:37462) 2023-10-05T19:39:36.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.112+0000 7fbb937fe700 1 -- 172.21.15.12:0/608786458 learned_addr learned my addr 172.21.15.12:0/608786458 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:39:36.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.112+0000 7fbb937fe700 1 -- 172.21.15.12:0/608786458 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb84009420 con 0x7fbb9406ea90 2023-10-05T19:39:36.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.112+0000 7fbb937fe700 1 --2- 172.21.15.12:0/608786458 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbb9406ea90 0x7fbb94125ce0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbb84000c00 tx=0x7fbb84016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:36.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.112+0000 7fbb7b7fe700 1 -- 172.21.15.12:0/608786458 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbb84011dd0 con 0x7fbb9406ea90 2023-10-05T19:39:36.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.112+0000 7fbb7b7fe700 1 -- 172.21.15.12:0/608786458 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb84020070 con 0x7fbb9406ea90 2023-10-05T19:39:36.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.112+0000 7fbb98c71700 1 -- 172.21.15.12:0/608786458 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb94126310 con 0x7fbb9406ea90 2023-10-05T19:39:36.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.112+0000 7fbb7b7fe700 1 -- 172.21.15.12:0/608786458 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbb8401c5d0 con 0x7fbb9406ea90 2023-10-05T19:39:36.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.112+0000 7fbb98c71700 1 -- 172.21.15.12:0/608786458 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb94126780 con 0x7fbb9406ea90 2023-10-05T19:39:36.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.116+0000 7fbb7b7fe700 1 -- 172.21.15.12:0/608786458 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fbb84022030 con 0x7fbb9406ea90 2023-10-05T19:39:36.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.116+0000 7fbb7b7fe700 1 -- 172.21.15.12:0/608786458 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7fbb8402e800 con 0x7fbb9406ea90 2023-10-05T19:39:36.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.116+0000 7fbb98c71700 1 -- 172.21.15.12:0/608786458 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb9405ecb0 con 0x7fbb9406ea90 2023-10-05T19:39:36.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.120+0000 7fbb7b7fe700 1 -- 172.21.15.12:0/608786458 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71325 (secure 0 0 0) 0x7fbb8401cb40 con 0x7fbb9406ea90 2023-10-05T19:39:36.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.200+0000 7fbb98c71700 1 -- 172.21.15.12:0/608786458 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e578b376-27a7-47e2-b262-9a2a703d2504", "id": 1} v 0) v1 -- 0x7fbb9406f510 con 0x7fbb9406ea90 2023-10-05T19:39:36.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.204+0000 7fbb7b7fe700 1 -- 172.21.15.12:0/608786458 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e578b376-27a7-47e2-b262-9a2a703d2504", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbb8401cce0 con 0x7fbb9406ea90 2023-10-05T19:39:36.211 INFO:teuthology.orchestra.run.smithi012.stdout:1 2023-10-05T19:39:36.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.204+0000 7fbb98c71700 1 -- 172.21.15.12:0/608786458 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbb9406ea90 msgr2=0x7fbb94125ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:36.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.204+0000 7fbb98c71700 1 --2- 172.21.15.12:0/608786458 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbb9406ea90 0x7fbb94125ce0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbb84000c00 tx=0x7fbb84016000 comp rx=0 tx=0).stop 2023-10-05T19:39:36.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.204+0000 7fbb98c71700 1 -- 172.21.15.12:0/608786458 shutdown_connections 2023-10-05T19:39:36.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.204+0000 7fbb98c71700 1 --2- 172.21.15.12:0/608786458 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbb9406ea90 0x7fbb94125ce0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:36.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.204+0000 7fbb98c71700 1 -- 172.21.15.12:0/608786458 >> 172.21.15.12:0/608786458 conn(0x7fbb9406a740 msgr2=0x7fbb9406cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:36.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.204+0000 7fbb98c71700 1 -- 172.21.15.12:0/608786458 shutdown_connections 2023-10-05T19:39:36.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.204+0000 7fbb98c71700 1 -- 172.21.15.12:0/608786458 wait complete. 2023-10-05T19:39:36.225 DEBUG:teuthology.orchestra.run.smithi012:> sudo ceph --cluster ceph osd new f687bf28-3a5f-4cd8-a242-9b95067cc77a 2 2023-10-05T19:39:36.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.316+0000 7ff6abfff700 1 Processor -- start 2023-10-05T19:39:36.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.316+0000 7ff6abfff700 1 -- start start 2023-10-05T19:39:36.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.316+0000 7ff6abfff700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7ff6ac109f70 0x7ff6ac10a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:36.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.316+0000 7ff6abfff700 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6ac114630 con 0x7ff6ac10a880 2023-10-05T19:39:36.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.316+0000 7ff6abfff700 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6ac04c3e0 con 0x7ff6ac109f70 2023-10-05T19:39:36.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.316+0000 7ff6aaffd700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7ff6ac109f70 0x7ff6ac10a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:36.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.320+0000 7ff6aaffd700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7ff6ac109f70 0x7ff6ac10a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37464/0 (socket says 172.21.15.12:37464) 2023-10-05T19:39:36.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.320+0000 7ff6aa7fc700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7ff6ac10a880 0x7ff6ac112e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:54862/0 (socket says 172.21.15.12:54862) 2023-10-05T19:39:36.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.320+0000 7ff6aaffd700 1 -- 172.21.15.12:0/3100648607 learned_addr learned my addr 172.21.15.12:0/3100648607 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:39:36.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.320+0000 7ff6aaffd700 1 -- 172.21.15.12:0/3100648607 >> v1:172.21.15.8:6789/0 conn(0x7ff6ac10a880 legacy=0x7ff6ac112e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:36.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.320+0000 7ff6a9ffb700 1 -- 172.21.15.12:0/3100648607 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1842274717 0 0) 0x7ff6ac114630 con 0x7ff6ac10a880 2023-10-05T19:39:36.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.320+0000 7ff6aaffd700 1 -- 172.21.15.12:0/3100648607 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6ac115800 con 0x7ff6ac109f70 2023-10-05T19:39:36.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.320+0000 7ff6aaffd700 1 --2- 172.21.15.12:0/3100648607 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff6ac109f70 0x7ff6ac10a340 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff69c009930 tx=0x7ff69c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89d552d6010fccc9 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:36.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.320+0000 7ff6a9ffb700 1 -- 172.21.15.12:0/3100648607 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff69c00b860 con 0x7ff6ac109f70 2023-10-05T19:39:36.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.320+0000 7ff6a9ffb700 1 -- 172.21.15.12:0/3100648607 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69c00b9c0 con 0x7ff6ac109f70 2023-10-05T19:39:36.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.320+0000 7ff6a9ffb700 1 -- 172.21.15.12:0/3100648607 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff69c00bb50 con 0x7ff6ac109f70 2023-10-05T19:39:36.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.320+0000 7ff6abfff700 1 -- 172.21.15.12:0/3100648607 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff6ac109f70 msgr2=0x7ff6ac10a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:36.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.320+0000 7ff6abfff700 1 --2- 172.21.15.12:0/3100648607 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff6ac109f70 0x7ff6ac10a340 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff69c009930 tx=0x7ff69c009250 comp rx=0 tx=0).stop 2023-10-05T19:39:36.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.320+0000 7ff6abfff700 1 -- 172.21.15.12:0/3100648607 shutdown_connections 2023-10-05T19:39:36.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.320+0000 7ff6abfff700 1 --2- 172.21.15.12:0/3100648607 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff6ac109f70 0x7ff6ac10a340 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:36.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.320+0000 7ff6abfff700 1 -- 172.21.15.12:0/3100648607 >> 172.21.15.12:0/3100648607 conn(0x7ff6ac06a740 msgr2=0x7ff6ac06cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:36.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.320+0000 7ff6abfff700 1 -- 172.21.15.12:0/3100648607 shutdown_connections 2023-10-05T19:39:36.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.320+0000 7ff6abfff700 1 -- 172.21.15.12:0/3100648607 wait complete. 2023-10-05T19:39:36.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.324+0000 7ff6abfff700 1 Processor -- start 2023-10-05T19:39:36.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.324+0000 7ff6abfff700 1 -- start start 2023-10-05T19:39:36.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.324+0000 7ff6abfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff6ac109f70 0x7ff6ac11f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:36.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.324+0000 7ff6abfff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac06cad0 con 0x7ff6ac109f70 2023-10-05T19:39:36.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.324+0000 7ff6aaffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff6ac109f70 0x7ff6ac11f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:36.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.324+0000 7ff6aaffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff6ac109f70 0x7ff6ac11f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37466/0 (socket says 172.21.15.12:37466) 2023-10-05T19:39:36.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.324+0000 7ff6aaffd700 1 -- 172.21.15.12:0/3566913189 learned_addr learned my addr 172.21.15.12:0/3566913189 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:39:36.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.324+0000 7ff6aaffd700 1 -- 172.21.15.12:0/3566913189 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff69c0094f0 con 0x7ff6ac109f70 2023-10-05T19:39:36.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.324+0000 7ff6aaffd700 1 --2- 172.21.15.12:0/3566913189 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff6ac109f70 0x7ff6ac11f480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff69c009290 tx=0x7ff69c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:36.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.324+0000 7ff692ffd700 1 -- 172.21.15.12:0/3566913189 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff69c00bd50 con 0x7ff6ac109f70 2023-10-05T19:39:36.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.324+0000 7ff692ffd700 1 -- 172.21.15.12:0/3566913189 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69c020070 con 0x7ff6ac109f70 2023-10-05T19:39:36.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.324+0000 7ff6abfff700 1 -- 172.21.15.12:0/3566913189 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6ac11fab0 con 0x7ff6ac109f70 2023-10-05T19:39:36.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.324+0000 7ff692ffd700 1 -- 172.21.15.12:0/3566913189 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff69c01b450 con 0x7ff6ac109f70 2023-10-05T19:39:36.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.324+0000 7ff6abfff700 1 -- 172.21.15.12:0/3566913189 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6ac11fea0 con 0x7ff6ac109f70 2023-10-05T19:39:36.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.324+0000 7ff692ffd700 1 -- 172.21.15.12:0/3566913189 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7ff69c022030 con 0x7ff6ac109f70 2023-10-05T19:39:36.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.324+0000 7ff692ffd700 1 -- 172.21.15.12:0/3566913189 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7ff69c01c8d0 con 0x7ff6ac109f70 2023-10-05T19:39:36.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.324+0000 7ff6abfff700 1 -- 172.21.15.12:0/3566913189 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6ac05ecb0 con 0x7ff6ac109f70 2023-10-05T19:39:36.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.332+0000 7ff692ffd700 1 -- 172.21.15.12:0/3566913189 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71325 (secure 0 0 0) 0x7ff69c01c4a0 con 0x7ff6ac109f70 2023-10-05T19:39:36.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.404+0000 7ff6abfff700 1 -- 172.21.15.12:0/3566913189 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f687bf28-3a5f-4cd8-a242-9b95067cc77a", "id": 2} v 0) v1 -- 0x7ff6ac067200 con 0x7ff6ac109f70 2023-10-05T19:39:36.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.412+0000 7ff692ffd700 1 -- 172.21.15.12:0/3566913189 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f687bf28-3a5f-4cd8-a242-9b95067cc77a", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff69c01c720 con 0x7ff6ac109f70 2023-10-05T19:39:36.418 INFO:teuthology.orchestra.run.smithi012.stdout:2 2023-10-05T19:39:36.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.412+0000 7ff6abfff700 1 -- 172.21.15.12:0/3566913189 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff6ac109f70 msgr2=0x7ff6ac11f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:36.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.412+0000 7ff6abfff700 1 --2- 172.21.15.12:0/3566913189 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff6ac109f70 0x7ff6ac11f480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff69c009290 tx=0x7ff69c016000 comp rx=0 tx=0).stop 2023-10-05T19:39:36.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.412+0000 7ff6abfff700 1 -- 172.21.15.12:0/3566913189 shutdown_connections 2023-10-05T19:39:36.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.412+0000 7ff6abfff700 1 --2- 172.21.15.12:0/3566913189 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff6ac109f70 0x7ff6ac11f480 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:36.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.412+0000 7ff6abfff700 1 -- 172.21.15.12:0/3566913189 >> 172.21.15.12:0/3566913189 conn(0x7ff6ac06a740 msgr2=0x7ff6ac10ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:36.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.412+0000 7ff6abfff700 1 -- 172.21.15.12:0/3566913189 shutdown_connections 2023-10-05T19:39:36.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.412+0000 7ff6abfff700 1 -- 172.21.15.12:0/3566913189 wait complete. 2023-10-05T19:39:36.432 DEBUG:teuthology.orchestra.run.smithi012:> sudo ceph --cluster ceph osd new db25fbc8-9f15-4a87-8310-fcc69bb92fa3 3 2023-10-05T19:39:36.440 INFO:tasks.ceph.mgr.x.smithi012.stderr:2023-10-05T19:39:36.432+0000 7fde2aeb7000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-05T19:39:36.500 INFO:tasks.ceph.mgr.x.smithi012.stderr:2023-10-05T19:39:36.492+0000 7fde2aeb7000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-05T19:39:36.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.524+0000 7f1a0ffff700 1 Processor -- start 2023-10-05T19:39:36.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.524+0000 7f1a0ffff700 1 -- start start 2023-10-05T19:39:36.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.524+0000 7f1a0ffff700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f1a1010f8d0 0x7f1a1010fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:36.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.524+0000 7f1a0ffff700 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a10110270 con 0x7f1a100678b0 2023-10-05T19:39:36.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.524+0000 7f1a0ffff700 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a1004c3e0 con 0x7f1a1010f8d0 2023-10-05T19:39:36.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.524+0000 7f1a0effd700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f1a1010f8d0 0x7f1a1010fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:36.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.524+0000 7f1a0effd700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f1a1010f8d0 0x7f1a1010fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37482/0 (socket says 172.21.15.12:37482) 2023-10-05T19:39:36.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.524+0000 7f1a0effd700 1 -- 172.21.15.12:0/1395408857 learned_addr learned my addr 172.21.15.12:0/1395408857 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:39:36.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.528+0000 7f1a0effd700 1 -- 172.21.15.12:0/1395408857 >> v1:172.21.15.8:6789/0 conn(0x7f1a100678b0 legacy=0x7f1a10067d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:36.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.528+0000 7f1a0effd700 1 -- 172.21.15.12:0/1395408857 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a10068440 con 0x7f1a1010f8d0 2023-10-05T19:39:36.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.528+0000 7f1a0effd700 1 --2- 172.21.15.12:0/1395408857 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1a1010f8d0 0x7f1a1010fca0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1a00009930 tx=0x7f1a00009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59b5a34b611238f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:36.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.528+0000 7f1a0dffb700 1 -- 172.21.15.12:0/1395408857 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a0000b860 con 0x7f1a1010f8d0 2023-10-05T19:39:36.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.528+0000 7f1a0dffb700 1 -- 172.21.15.12:0/1395408857 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a0000b9c0 con 0x7f1a1010f8d0 2023-10-05T19:39:36.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.528+0000 7f1a0dffb700 1 -- 172.21.15.12:0/1395408857 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a0000bb50 con 0x7f1a1010f8d0 2023-10-05T19:39:36.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.528+0000 7f1a0ffff700 1 -- 172.21.15.12:0/1395408857 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1a1010f8d0 msgr2=0x7f1a1010fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:36.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.528+0000 7f1a0ffff700 1 --2- 172.21.15.12:0/1395408857 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1a1010f8d0 0x7f1a1010fca0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1a00009930 tx=0x7f1a00009250 comp rx=0 tx=0).stop 2023-10-05T19:39:36.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.528+0000 7f1a0ffff700 1 -- 172.21.15.12:0/1395408857 shutdown_connections 2023-10-05T19:39:36.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.528+0000 7f1a0ffff700 1 --2- 172.21.15.12:0/1395408857 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1a1010f8d0 0x7f1a1010fca0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:36.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.528+0000 7f1a0ffff700 1 -- 172.21.15.12:0/1395408857 >> 172.21.15.12:0/1395408857 conn(0x7f1a1006afc0 msgr2=0x7f1a1006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:36.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.528+0000 7f1a0ffff700 1 -- 172.21.15.12:0/1395408857 shutdown_connections 2023-10-05T19:39:36.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.528+0000 7f1a0ffff700 1 -- 172.21.15.12:0/1395408857 wait complete. 2023-10-05T19:39:36.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.528+0000 7f1a0ffff700 1 Processor -- start 2023-10-05T19:39:36.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.528+0000 7f1a0ffff700 1 -- start start 2023-10-05T19:39:36.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.528+0000 7f1a0ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1a100678b0 0x7f1a10119b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:36.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.528+0000 7f1a0ffff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1006b390 con 0x7f1a100678b0 2023-10-05T19:39:36.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.528+0000 7f1a0effd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1a100678b0 0x7f1a10119b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:36.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.528+0000 7f1a0effd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1a100678b0 0x7f1a10119b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37486/0 (socket says 172.21.15.12:37486) 2023-10-05T19:39:36.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.528+0000 7f1a0effd700 1 -- 172.21.15.12:0/1402753962 learned_addr learned my addr 172.21.15.12:0/1402753962 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:39:36.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.528+0000 7f1a0effd700 1 -- 172.21.15.12:0/1402753962 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a000094f0 con 0x7f1a100678b0 2023-10-05T19:39:36.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.528+0000 7f1a0effd700 1 --2- 172.21.15.12:0/1402753962 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1a100678b0 0x7f1a10119b70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1a000094c0 tx=0x7f1a00016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:36.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.532+0000 7f19f6ffd700 1 -- 172.21.15.12:0/1402753962 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a0000bd40 con 0x7f1a100678b0 2023-10-05T19:39:36.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.532+0000 7f19f6ffd700 1 -- 172.21.15.12:0/1402753962 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a00020070 con 0x7f1a100678b0 2023-10-05T19:39:36.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.532+0000 7f1a0ffff700 1 -- 172.21.15.12:0/1402753962 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a1011a0b0 con 0x7f1a100678b0 2023-10-05T19:39:36.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.532+0000 7f19f6ffd700 1 -- 172.21.15.12:0/1402753962 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a0001b450 con 0x7f1a100678b0 2023-10-05T19:39:36.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.532+0000 7f1a0ffff700 1 -- 172.21.15.12:0/1402753962 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a1011a500 con 0x7f1a100678b0 2023-10-05T19:39:36.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.532+0000 7f19f6ffd700 1 -- 172.21.15.12:0/1402753962 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f1a00022030 con 0x7f1a100678b0 2023-10-05T19:39:36.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.532+0000 7f19f6ffd700 1 -- 172.21.15.12:0/1402753962 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7f1a0001c8f0 con 0x7f1a100678b0 2023-10-05T19:39:36.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.532+0000 7f1a0ffff700 1 -- 172.21.15.12:0/1402753962 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a1005ecb0 con 0x7f1a100678b0 2023-10-05T19:39:36.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.536+0000 7f19f6ffd700 1 -- 172.21.15.12:0/1402753962 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71325 (secure 0 0 0) 0x7f1a0001c450 con 0x7f1a100678b0 2023-10-05T19:39:36.618 INFO:tasks.ceph.mgr.x.smithi012.stderr:2023-10-05T19:39:36.612+0000 7fde2aeb7000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-05T19:39:36.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.616+0000 7f1a0ffff700 1 -- 172.21.15.12:0/1402753962 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd new", "uuid": "db25fbc8-9f15-4a87-8310-fcc69bb92fa3", "id": 3} v 0) v1 -- 0x7f1a1011a6f0 con 0x7f1a100678b0 2023-10-05T19:39:36.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.620+0000 7f19f6ffd700 1 -- 172.21.15.12:0/1402753962 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "db25fbc8-9f15-4a87-8310-fcc69bb92fa3", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1a0001c6b0 con 0x7f1a100678b0 2023-10-05T19:39:36.627 INFO:teuthology.orchestra.run.smithi012.stdout:3 2023-10-05T19:39:36.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.620+0000 7f1a0ffff700 1 -- 172.21.15.12:0/1402753962 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1a100678b0 msgr2=0x7f1a10119b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:36.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.620+0000 7f1a0ffff700 1 --2- 172.21.15.12:0/1402753962 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1a100678b0 0x7f1a10119b70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1a000094c0 tx=0x7f1a00016000 comp rx=0 tx=0).stop 2023-10-05T19:39:36.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.620+0000 7f1a0ffff700 1 -- 172.21.15.12:0/1402753962 shutdown_connections 2023-10-05T19:39:36.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.620+0000 7f1a0ffff700 1 --2- 172.21.15.12:0/1402753962 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1a100678b0 0x7f1a10119b70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:36.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.620+0000 7f1a0ffff700 1 -- 172.21.15.12:0/1402753962 >> 172.21.15.12:0/1402753962 conn(0x7f1a1006afc0 msgr2=0x7f1a101125b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:36.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.620+0000 7f1a0ffff700 1 -- 172.21.15.12:0/1402753962 shutdown_connections 2023-10-05T19:39:36.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:36.620+0000 7f1a0ffff700 1 -- 172.21.15.12:0/1402753962 wait complete. 2023-10-05T19:39:36.640 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-05T19:39:36.641 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-05T19:39:36.643 INFO:tasks.ceph.osd.0:Started 2023-10-05T19:39:36.644 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-05T19:39:36.644 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-05T19:39:36.646 INFO:tasks.ceph.osd.1:Started 2023-10-05T19:39:36.646 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-05T19:39:36.646 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-05T19:39:36.648 INFO:tasks.ceph.osd.2:Started 2023-10-05T19:39:36.648 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-05T19:39:36.648 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-10-05T19:39:36.650 INFO:tasks.ceph.osd.3:Started 2023-10-05T19:39:36.651 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-05T19:39:36.759 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.751+0000 7f7f360c0700 1 Processor -- start 2023-10-05T19:39:36.759 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.751+0000 7f7f360c0700 1 -- start start 2023-10-05T19:39:36.760 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.751+0000 7f7f360c0700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f7f3006cfa0 0x7f7f3006d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:36.760 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.751+0000 7f7f360c0700 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f3006b580 con 0x7f7f3006d8b0 2023-10-05T19:39:36.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.751+0000 7f7f360c0700 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f3006e370 con 0x7f7f3006cfa0 2023-10-05T19:39:36.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.751+0000 7f7f348bd700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f7f3006d8b0 0x7f7f301292c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:60174/0 (socket says 172.21.15.8:60174) 2023-10-05T19:39:36.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.751+0000 7f7f350be700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f7f3006cfa0 0x7f7f3006d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:36.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.751+0000 7f7f348bd700 1 -- 172.21.15.8:0/830915612 learned_addr learned my addr 172.21.15.8:0/830915612 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-05T19:39:36.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.755+0000 7f7f27fff700 1 -- 172.21.15.8:0/830915612 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2985619874 0 0) 0x7f7f3006b580 con 0x7f7f3006d8b0 2023-10-05T19:39:36.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.755+0000 7f7f27fff700 1 -- 172.21.15.8:0/830915612 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f1c003570 con 0x7f7f3006d8b0 2023-10-05T19:39:36.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.755+0000 7f7f27fff700 1 -- 172.21.15.8:0/830915612 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 897770498 0 0) 0x7f7f1c003570 con 0x7f7f3006d8b0 2023-10-05T19:39:36.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.755+0000 7f7f27fff700 1 -- 172.21.15.8:0/830915612 >> v2:172.21.15.8:3300/0 conn(0x7f7f3006cfa0 msgr2=0x7f7f3006d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-05T19:39:36.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.755+0000 7f7f27fff700 1 --2- 172.21.15.8:0/830915612 >> v2:172.21.15.8:3300/0 conn(0x7f7f3006cfa0 0x7f7f3006d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:36.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.755+0000 7f7f27fff700 1 -- 172.21.15.8:0/830915612 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f3006deb0 con 0x7f7f3006d8b0 2023-10-05T19:39:36.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.755+0000 7f7f27fff700 1 -- 172.21.15.8:0/830915612 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (unknown 1630025266 0 0) 0x7f7f200015b0 con 0x7f7f3006d8b0 2023-10-05T19:39:36.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.755+0000 7f7f350be700 1 --2- 172.21.15.8:0/830915612 >> v2:172.21.15.8:3300/0 conn(0x7f7f3006cfa0 0x7f7f3006d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-05T19:39:36.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.755+0000 7f7f27fff700 1 -- 172.21.15.8:0/830915612 >> v1:172.21.15.8:6789/0 conn(0x7f7f3006d8b0 legacy=0x7f7f301292c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:36.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.755+0000 7f7f27fff700 1 --2- 172.21.15.8:0/830915612 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7f1c003a90 0x7f7f1c003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:36.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.755+0000 7f7f27fff700 1 -- 172.21.15.8:0/830915612 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3006e370 con 0x7f7f1c003a90 2023-10-05T19:39:36.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.755+0000 7f7f350be700 1 --2- 172.21.15.8:0/830915612 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7f1c003a90 0x7f7f1c003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:36.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.755+0000 7f7f350be700 1 -- 172.21.15.8:0/830915612 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f1c003570 con 0x7f7f1c003a90 2023-10-05T19:39:36.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.755+0000 7f7f350be700 1 --2- 172.21.15.8:0/830915612 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7f1c003a90 0x7f7f1c003e80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7f2800e340 tx=0x7f7f2800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=982ba97ed45d6358 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:36.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.755+0000 7f7f27fff700 1 -- 172.21.15.8:0/830915612 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7f2800ec20 con 0x7f7f1c003a90 2023-10-05T19:39:36.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.755+0000 7f7f27fff700 1 -- 172.21.15.8:0/830915612 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f2800f040 con 0x7f7f1c003a90 2023-10-05T19:39:36.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.755+0000 7f7f27fff700 1 -- 172.21.15.8:0/830915612 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7f2800eda0 con 0x7f7f1c003a90 2023-10-05T19:39:36.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.755+0000 7f7f360c0700 1 -- 172.21.15.8:0/830915612 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7f1c003a90 msgr2=0x7f7f1c003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:36.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.755+0000 7f7f360c0700 1 --2- 172.21.15.8:0/830915612 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7f1c003a90 0x7f7f1c003e80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7f2800e340 tx=0x7f7f2800b040 comp rx=0 tx=0).stop 2023-10-05T19:39:36.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.755+0000 7f7f360c0700 1 -- 172.21.15.8:0/830915612 shutdown_connections 2023-10-05T19:39:36.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.755+0000 7f7f360c0700 1 --2- 172.21.15.8:0/830915612 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7f1c003a90 0x7f7f1c003e80 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:36.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.755+0000 7f7f360c0700 1 --2- 172.21.15.8:0/830915612 >> v2:172.21.15.8:3300/0 conn(0x7f7f3006cfa0 0x7f7f3006d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:36.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.755+0000 7f7f360c0700 1 -- 172.21.15.8:0/830915612 >> 172.21.15.8:0/830915612 conn(0x7f7f3009a740 msgr2=0x7f7f3009ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:36.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.755+0000 7f7f360c0700 1 -- 172.21.15.8:0/830915612 shutdown_connections 2023-10-05T19:39:36.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.755+0000 7f7f360c0700 1 -- 172.21.15.8:0/830915612 wait complete. 2023-10-05T19:39:36.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.755+0000 7f7f360c0700 1 Processor -- start 2023-10-05T19:39:36.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.755+0000 7f7f360c0700 1 -- start start 2023-10-05T19:39:36.766 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.759+0000 7f7f360c0700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7f1c003a90 0x7f7f30137f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:36.766 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.759+0000 7f7f360c0700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3011df70 con 0x7f7f1c003a90 2023-10-05T19:39:36.766 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.759+0000 7f7f350be700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7f1c003a90 0x7f7f30137f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:36.766 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.759+0000 7f7f350be700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7f1c003a90 0x7f7f30137f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41066/0 (socket says 172.21.15.8:41066) 2023-10-05T19:39:36.766 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.759+0000 7f7f350be700 1 -- 172.21.15.8:0/4078659921 learned_addr learned my addr 172.21.15.8:0/4078659921 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-05T19:39:36.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.759+0000 7f7f350be700 1 -- 172.21.15.8:0/4078659921 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f28009a20 con 0x7f7f1c003a90 2023-10-05T19:39:36.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.759+0000 7f7f350be700 1 --2- 172.21.15.8:0/4078659921 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7f1c003a90 0x7f7f30137f00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7f2800e9e0 tx=0x7f7f2801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:36.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.759+0000 7f7f257fa700 1 -- 172.21.15.8:0/4078659921 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7f28023070 con 0x7f7f1c003a90 2023-10-05T19:39:36.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.759+0000 7f7f257fa700 1 -- 172.21.15.8:0/4078659921 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f2801d630 con 0x7f7f1c003a90 2023-10-05T19:39:36.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.759+0000 7f7f360c0700 1 -- 172.21.15.8:0/4078659921 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f3011fba0 con 0x7f7f1c003a90 2023-10-05T19:39:36.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.759+0000 7f7f257fa700 1 -- 172.21.15.8:0/4078659921 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7f2801d900 con 0x7f7f1c003a90 2023-10-05T19:39:36.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.759+0000 7f7f360c0700 1 -- 172.21.15.8:0/4078659921 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f30120030 con 0x7f7f1c003a90 2023-10-05T19:39:36.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.759+0000 7f7f257fa700 1 -- 172.21.15.8:0/4078659921 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f7f28003020 con 0x7f7f1c003a90 2023-10-05T19:39:36.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.759+0000 7f7f257fa700 1 -- 172.21.15.8:0/4078659921 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f7f28028d40 con 0x7f7f1c003a90 2023-10-05T19:39:36.768 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.759+0000 7f7f360c0700 1 -- 172.21.15.8:0/4078659921 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f30120280 con 0x7f7f1c003a90 2023-10-05T19:39:36.772 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.767+0000 7f7f257fa700 1 -- 172.21.15.8:0/4078659921 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71325 (secure 0 0 0) 0x7f7f28036060 con 0x7f7f1c003a90 2023-10-05T19:39:36.839 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.831+0000 7f7f360c0700 1 -- 172.21.15.8:0/4078659921 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f3005ecb0 con 0x7f7f1c003a90 2023-10-05T19:39:36.840 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.835+0000 7f7f257fa700 1 -- 172.21.15.8:0/4078659921 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) v1 ==== 74+0+4108 (secure 0 0 0) 0x7f7f28018020 con 0x7f7f1c003a90 2023-10-05T19:39:36.840 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-05T19:39:36.840 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":6,"fsid":"dac6c186-0634-45d9-839d-04f1e484462d","created":"2023-10-05T19:39:35.575770+0000","modified":"2023-10-05T19:39:36.620941+0000","last_up_change":"0.000000","last_in_change":"2023-10-05T19:39:36.620941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"c23c6d99-5a1e-499d-8ef8-e1a58dcd9e74","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e578b376-27a7-47e2-b262-9a2a703d2504","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f687bf28-3a5f-4cd8-a242-9b95067cc77a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"db25fbc8-9f15-4a87-8310-fcc69bb92fa3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-05T19:39:36.842 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.835+0000 7f7f360c0700 1 -- 172.21.15.8:0/4078659921 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7f1c003a90 msgr2=0x7f7f30137f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:36.842 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.835+0000 7f7f360c0700 1 --2- 172.21.15.8:0/4078659921 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7f1c003a90 0x7f7f30137f00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7f2800e9e0 tx=0x7f7f2801a000 comp rx=0 tx=0).stop 2023-10-05T19:39:36.842 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.835+0000 7f7f360c0700 1 -- 172.21.15.8:0/4078659921 shutdown_connections 2023-10-05T19:39:36.843 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.835+0000 7f7f360c0700 1 --2- 172.21.15.8:0/4078659921 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7f1c003a90 0x7f7f30137f00 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:36.843 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.835+0000 7f7f360c0700 1 -- 172.21.15.8:0/4078659921 >> 172.21.15.8:0/4078659921 conn(0x7f7f3009a740 msgr2=0x7f7f300963d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:36.843 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.835+0000 7f7f360c0700 1 -- 172.21.15.8:0/4078659921 shutdown_connections 2023-10-05T19:39:36.843 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:36.835+0000 7f7f360c0700 1 -- 172.21.15.8:0/4078659921 wait complete. 2023-10-05T19:39:36.858 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-05T19:39:36.858 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-05T19:39:36.880 INFO:tasks.ceph.mgr.x.smithi012.stderr:2023-10-05T19:39:36.872+0000 7fde2aeb7000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-05T19:39:36.931 INFO:tasks.ceph.mgr.x.smithi012.stderr:2023-10-05T19:39:36.924+0000 7fde2aeb7000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-05T19:39:37.125 INFO:tasks.ceph.mgr.x.smithi012.stderr:2023-10-05T19:39:37.116+0000 7fde2aeb7000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-05T19:39:37.188 INFO:tasks.ceph.mgr.x.smithi012.stderr:2023-10-05T19:39:37.180+0000 7fde2aeb7000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-05T19:39:37.252 INFO:tasks.ceph.mgr.x.smithi012.stderr:2023-10-05T19:39:37.244+0000 7fde2aeb7000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-05T19:39:37.260 DEBUG:teuthology.orchestra.run.smithi008:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-05T19:39:37.279 INFO:tasks.ceph.osd.2.smithi012.stderr:2023-10-05T19:39:37.272+0000 7f9392b463c0 -1 Falling back to public interface 2023-10-05T19:39:37.282 INFO:tasks.ceph.osd.0.smithi008.stderr:2023-10-05T19:39:37.275+0000 7fc100e7a3c0 -1 Falling back to public interface 2023-10-05T19:39:37.283 INFO:tasks.ceph.osd.1.smithi012.stderr:2023-10-05T19:39:37.276+0000 7efe2e0ce3c0 -1 Falling back to public interface 2023-10-05T19:39:37.291 INFO:tasks.ceph.osd.3.smithi012.stderr:2023-10-05T19:39:37.284+0000 7f18889023c0 -1 Falling back to public interface 2023-10-05T19:39:37.357 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.351+0000 7f6430543700 1 Processor -- start 2023-10-05T19:39:37.357 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.351+0000 7f6430543700 1 -- start start 2023-10-05T19:39:37.359 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.351+0000 7f6430543700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f642806b6c0 0x7f642812e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:37.359 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.351+0000 7f6430543700 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f642806b120 con 0x7f6428129540 2023-10-05T19:39:37.359 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.351+0000 7f6430543700 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6428096a30 con 0x7f642806b6c0 2023-10-05T19:39:37.359 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.351+0000 7f642e2df700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f6428129540 0x7f6428129910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:60176/0 (socket says 172.21.15.8:60176) 2023-10-05T19:39:37.359 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.351+0000 7f642e2df700 1 -- 172.21.15.8:0/1668708473 learned_addr learned my addr 172.21.15.8:0/1668708473 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-05T19:39:37.359 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.351+0000 7f642dade700 1 --2- 172.21.15.8:0/1668708473 >> v2:172.21.15.8:3300/0 conn(0x7f642806b6c0 0x7f642812e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:37.359 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.351+0000 7f642d2dd700 1 -- 172.21.15.8:0/1668708473 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4022786350 0 0) 0x7f642806b120 con 0x7f6428129540 2023-10-05T19:39:37.359 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.351+0000 7f642d2dd700 1 -- 172.21.15.8:0/1668708473 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6414003570 con 0x7f6428129540 2023-10-05T19:39:37.359 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.351+0000 7f642dade700 1 -- 172.21.15.8:0/1668708473 >> v1:172.21.15.8:6789/0 conn(0x7f6428129540 legacy=0x7f6428129910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:37.360 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.351+0000 7f642dade700 1 -- 172.21.15.8:0/1668708473 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f642806bdc0 con 0x7f642806b6c0 2023-10-05T19:39:37.360 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.351+0000 7f642dade700 1 --2- 172.21.15.8:0/1668708473 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f642806b6c0 0x7f642812e770 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6418009890 tx=0x7f64180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a515bac8d6df4dfb server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:37.360 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.351+0000 7f642d2dd700 1 -- 172.21.15.8:0/1668708473 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6418011630 con 0x7f642806b6c0 2023-10-05T19:39:37.361 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.351+0000 7f642d2dd700 1 -- 172.21.15.8:0/1668708473 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6418011790 con 0x7f642806b6c0 2023-10-05T19:39:37.361 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.351+0000 7f642d2dd700 1 -- 172.21.15.8:0/1668708473 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6418011920 con 0x7f642806b6c0 2023-10-05T19:39:37.361 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.355+0000 7f6430543700 1 -- 172.21.15.8:0/1668708473 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f642806b6c0 msgr2=0x7f642812e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:37.361 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.355+0000 7f6430543700 1 --2- 172.21.15.8:0/1668708473 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f642806b6c0 0x7f642812e770 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6418009890 tx=0x7f64180091a0 comp rx=0 tx=0).stop 2023-10-05T19:39:37.361 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.355+0000 7f6430543700 1 -- 172.21.15.8:0/1668708473 shutdown_connections 2023-10-05T19:39:37.361 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.355+0000 7f6430543700 1 --2- 172.21.15.8:0/1668708473 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f642806b6c0 0x7f642812e770 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:37.361 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.355+0000 7f6430543700 1 -- 172.21.15.8:0/1668708473 >> 172.21.15.8:0/1668708473 conn(0x7f642805e590 msgr2=0x7f64280960f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:37.361 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.355+0000 7f6430543700 1 -- 172.21.15.8:0/1668708473 shutdown_connections 2023-10-05T19:39:37.361 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.355+0000 7f6430543700 1 -- 172.21.15.8:0/1668708473 wait complete. 2023-10-05T19:39:37.362 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.355+0000 7f6430543700 1 Processor -- start 2023-10-05T19:39:37.362 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.355+0000 7f6430543700 1 -- start start 2023-10-05T19:39:37.364 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.359+0000 7f6430543700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6428129540 0x7f64281385f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:37.364 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.359+0000 7f6430543700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642806dcf0 con 0x7f6428129540 2023-10-05T19:39:37.364 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.359+0000 7f642e2df700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6428129540 0x7f64281385f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:37.364 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.359+0000 7f642e2df700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6428129540 0x7f64281385f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41086/0 (socket says 172.21.15.8:41086) 2023-10-05T19:39:37.364 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.359+0000 7f642e2df700 1 -- 172.21.15.8:0/3988124044 learned_addr learned my addr 172.21.15.8:0/3988124044 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-05T19:39:37.365 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.359+0000 7f642e2df700 1 -- 172.21.15.8:0/3988124044 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6418009420 con 0x7f6428129540 2023-10-05T19:39:37.366 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.359+0000 7f642e2df700 1 --2- 172.21.15.8:0/3988124044 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6428129540 0x7f64281385f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f642000cc30 tx=0x7f6420001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:37.366 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.359+0000 7f641e7fc700 1 -- 172.21.15.8:0/3988124044 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f642000a9c0 con 0x7f6428129540 2023-10-05T19:39:37.366 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.359+0000 7f641e7fc700 1 -- 172.21.15.8:0/3988124044 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f642000ab40 con 0x7f6428129540 2023-10-05T19:39:37.366 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.359+0000 7f6430543700 1 -- 172.21.15.8:0/3988124044 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f642811c980 con 0x7f6428129540 2023-10-05T19:39:37.366 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.359+0000 7f641e7fc700 1 -- 172.21.15.8:0/3988124044 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f642000ae30 con 0x7f6428129540 2023-10-05T19:39:37.366 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.359+0000 7f6430543700 1 -- 172.21.15.8:0/3988124044 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f642811d0f0 con 0x7f6428129540 2023-10-05T19:39:37.366 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.359+0000 7f641e7fc700 1 -- 172.21.15.8:0/3988124044 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f64200070f0 con 0x7f6428129540 2023-10-05T19:39:37.366 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.359+0000 7f641e7fc700 1 -- 172.21.15.8:0/3988124044 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f642001b910 con 0x7f6428129540 2023-10-05T19:39:37.367 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.359+0000 7f6430543700 1 -- 172.21.15.8:0/3988124044 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f642811d490 con 0x7f6428129540 2023-10-05T19:39:37.372 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.367+0000 7f641e7fc700 1 -- 172.21.15.8:0/3988124044 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71325 (secure 0 0 0) 0x7f6420007530 con 0x7f6428129540 2023-10-05T19:39:37.438 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.431+0000 7f6430543700 1 -- 172.21.15.8:0/3988124044 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f642811cb10 con 0x7f6428129540 2023-10-05T19:39:37.438 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.431+0000 7f641e7fc700 1 -- 172.21.15.8:0/3988124044 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) v1 ==== 74+0+4108 (secure 0 0 0) 0x7f642002a090 con 0x7f6428129540 2023-10-05T19:39:37.439 INFO:teuthology.misc.health.smithi008.stdout: 2023-10-05T19:39:37.439 INFO:teuthology.misc.health.smithi008.stdout:{"epoch":6,"fsid":"dac6c186-0634-45d9-839d-04f1e484462d","created":"2023-10-05T19:39:35.575770+0000","modified":"2023-10-05T19:39:36.620941+0000","last_up_change":"0.000000","last_in_change":"2023-10-05T19:39:36.620941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"c23c6d99-5a1e-499d-8ef8-e1a58dcd9e74","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e578b376-27a7-47e2-b262-9a2a703d2504","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f687bf28-3a5f-4cd8-a242-9b95067cc77a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"db25fbc8-9f15-4a87-8310-fcc69bb92fa3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-05T19:39:37.440 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.431+0000 7f6430543700 1 -- 172.21.15.8:0/3988124044 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6428129540 msgr2=0x7f64281385f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:37.440 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.431+0000 7f6430543700 1 --2- 172.21.15.8:0/3988124044 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6428129540 0x7f64281385f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f642000cc30 tx=0x7f6420001b10 comp rx=0 tx=0).stop 2023-10-05T19:39:37.440 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.431+0000 7f6430543700 1 -- 172.21.15.8:0/3988124044 shutdown_connections 2023-10-05T19:39:37.440 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.431+0000 7f6430543700 1 --2- 172.21.15.8:0/3988124044 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6428129540 0x7f64281385f0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:37.440 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.431+0000 7f6430543700 1 -- 172.21.15.8:0/3988124044 >> 172.21.15.8:0/3988124044 conn(0x7f642805e590 msgr2=0x7f6428097f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:37.440 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.435+0000 7f6430543700 1 -- 172.21.15.8:0/3988124044 shutdown_connections 2023-10-05T19:39:37.440 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:37.435+0000 7f6430543700 1 -- 172.21.15.8:0/3988124044 wait complete. 2023-10-05T19:39:37.450 DEBUG:teuthology.misc:0 of 4 OSDs are up 2023-10-05T19:39:37.656 INFO:tasks.ceph.mgr.x.smithi012.stderr:2023-10-05T19:39:37.648+0000 7fde2aeb7000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-05T19:39:37.772 INFO:tasks.ceph.mgr.x.smithi012.stderr:2023-10-05T19:39:37.764+0000 7fde2aeb7000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-05T19:39:38.071 INFO:tasks.ceph.mgr.x.smithi012.stderr:2023-10-05T19:39:38.064+0000 7fde2aeb7000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-05T19:39:38.242 INFO:tasks.ceph.mgr.x.smithi012.stderr:2023-10-05T19:39:38.236+0000 7fde2aeb7000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-05T19:39:38.310 INFO:tasks.ceph.mgr.x.smithi012.stderr:2023-10-05T19:39:38.304+0000 7fde2aeb7000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-05T19:39:38.364 INFO:tasks.ceph.mgr.x.smithi012.stderr:2023-10-05T19:39:38.356+0000 7fde2aeb7000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-05T19:39:38.423 INFO:tasks.ceph.mgr.x.smithi012.stderr:2023-10-05T19:39:38.416+0000 7fde2aeb7000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-05T19:39:38.722 INFO:tasks.ceph.mgr.x.smithi012.stderr:2023-10-05T19:39:38.716+0000 7fde2aeb7000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-05T19:39:38.781 INFO:tasks.ceph.mgr.x.smithi012.stderr:2023-10-05T19:39:38.772+0000 7fde2aeb7000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-05T19:39:38.849 INFO:tasks.ceph.mgr.x.smithi012.stderr:2023-10-05T19:39:38.840+0000 7fde2aeb7000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-05T19:39:39.010 INFO:tasks.ceph.mgr.x.smithi012.stderr:2023-10-05T19:39:39.004+0000 7fde2aeb7000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-05T19:39:39.080 INFO:tasks.ceph.mgr.x.smithi012.stderr:2023-10-05T19:39:39.072+0000 7fde2aeb7000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-05T19:39:39.805 INFO:tasks.ceph.mgr.x.smithi012.stderr:2023-10-05T19:39:39.796+0000 7fde2aeb7000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-05T19:39:40.749 INFO:tasks.ceph.osd.0.smithi008.stderr:2023-10-05T19:39:40.743+0000 7fc100e7a3c0 -1 osd.0 0 log_to_monitors true 2023-10-05T19:39:40.839 INFO:tasks.ceph.mgr.x.smithi012.stderr:2023-10-05T19:39:40.832+0000 7fde13fb3700 -1 mgr.server handle_report got status from non-daemon mon.a 2023-10-05T19:39:40.944 INFO:tasks.ceph.osd.3.smithi012.stderr:2023-10-05T19:39:40.936+0000 7f18889023c0 -1 osd.3 0 log_to_monitors true 2023-10-05T19:39:41.146 INFO:tasks.ceph.osd.1.smithi012.stderr:2023-10-05T19:39:41.140+0000 7efe2e0ce3c0 -1 osd.1 0 log_to_monitors true 2023-10-05T19:39:41.182 INFO:tasks.ceph.osd.2.smithi012.stderr:2023-10-05T19:39:41.176+0000 7f9392b463c0 -1 osd.2 0 log_to_monitors true 2023-10-05T19:39:43.854 DEBUG:teuthology.orchestra.run.smithi008:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-05T19:39:43.878 INFO:tasks.ceph.osd.0.smithi008.stderr:2023-10-05T19:39:43.871+0000 7fc0fc61a700 -1 osd.0 0 waiting for initial osdmap 2023-10-05T19:39:43.886 INFO:tasks.ceph.osd.0.smithi008.stderr:2023-10-05T19:39:43.879+0000 7fc0f53be700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-05T19:39:43.949 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:43.943+0000 7fe869aeb700 1 Processor -- start 2023-10-05T19:39:43.949 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:43.943+0000 7fe869aeb700 1 -- start start 2023-10-05T19:39:43.949 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:43.943+0000 7fe869aeb700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fe86406ceb0 0x7fe86406d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:43.950 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:43.943+0000 7fe869aeb700 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe86406b7f0 con 0x7fe86406d7c0 2023-10-05T19:39:43.950 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:43.943+0000 7fe869aeb700 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe86406b5d0 con 0x7fe86406ceb0 2023-10-05T19:39:43.950 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:43.943+0000 7fe868ae9700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fe86406ceb0 0x7fe86406d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:43.950 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:43.943+0000 7fe863fff700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7fe86406d7c0 0x7fe864129540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:57542/0 (socket says 172.21.15.8:57542) 2023-10-05T19:39:43.950 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:43.943+0000 7fe863fff700 1 -- 172.21.15.8:0/2900931537 learned_addr learned my addr 172.21.15.8:0/2900931537 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-05T19:39:43.951 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:43.943+0000 7fe868ae9700 1 -- 172.21.15.8:0/2900931537 >> v1:172.21.15.8:6789/0 conn(0x7fe86406d7c0 legacy=0x7fe864129540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:43.951 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:43.943+0000 7fe868ae9700 1 -- 172.21.15.8:0/2900931537 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe86406df60 con 0x7fe86406ceb0 2023-10-05T19:39:43.951 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:43.943+0000 7fe868ae9700 1 --2- 172.21.15.8:0/2900931537 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe86406ceb0 0x7fe86406d280 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe85c009de0 tx=0x7fe85c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19456faed6393129 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:43.951 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:43.943+0000 7fe8637fe700 1 -- 172.21.15.8:0/2900931537 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe85c00e070 con 0x7fe86406ceb0 2023-10-05T19:39:43.952 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:43.943+0000 7fe8637fe700 1 -- 172.21.15.8:0/2900931537 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe85c00bdc0 con 0x7fe86406ceb0 2023-10-05T19:39:43.952 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:43.943+0000 7fe8637fe700 1 -- 172.21.15.8:0/2900931537 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe85c0125b0 con 0x7fe86406ceb0 2023-10-05T19:39:43.952 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:43.943+0000 7fe869aeb700 1 -- 172.21.15.8:0/2900931537 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe86406ceb0 msgr2=0x7fe86406d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:43.952 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:43.943+0000 7fe869aeb700 1 --2- 172.21.15.8:0/2900931537 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe86406ceb0 0x7fe86406d280 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe85c009de0 tx=0x7fe85c009780 comp rx=0 tx=0).stop 2023-10-05T19:39:43.952 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:43.943+0000 7fe869aeb700 1 -- 172.21.15.8:0/2900931537 shutdown_connections 2023-10-05T19:39:43.952 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:43.943+0000 7fe869aeb700 1 --2- 172.21.15.8:0/2900931537 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe86406ceb0 0x7fe86406d280 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:43.952 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:43.943+0000 7fe869aeb700 1 -- 172.21.15.8:0/2900931537 >> 172.21.15.8:0/2900931537 conn(0x7fe86409a6d0 msgr2=0x7fe86409aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:43.952 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:43.943+0000 7fe869aeb700 1 -- 172.21.15.8:0/2900931537 shutdown_connections 2023-10-05T19:39:43.952 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:43.943+0000 7fe869aeb700 1 -- 172.21.15.8:0/2900931537 wait complete. 2023-10-05T19:39:43.953 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:43.947+0000 7fe869aeb700 1 Processor -- start 2023-10-05T19:39:43.953 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:43.947+0000 7fe869aeb700 1 -- start start 2023-10-05T19:39:43.953 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:43.947+0000 7fe869aeb700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe86406ceb0 0x7fe864138900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:43.953 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:43.947+0000 7fe869aeb700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe864094d40 con 0x7fe86406ceb0 2023-10-05T19:39:43.953 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:43.947+0000 7fe868ae9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe86406ceb0 0x7fe864138900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:43.954 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:43.947+0000 7fe868ae9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe86406ceb0 0x7fe864138900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56694/0 (socket says 172.21.15.8:56694) 2023-10-05T19:39:43.954 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:43.947+0000 7fe868ae9700 1 -- 172.21.15.8:0/363799428 learned_addr learned my addr 172.21.15.8:0/363799428 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-05T19:39:43.955 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:43.947+0000 7fe868ae9700 1 -- 172.21.15.8:0/363799428 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe85c009a20 con 0x7fe86406ceb0 2023-10-05T19:39:43.955 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:43.947+0000 7fe868ae9700 1 --2- 172.21.15.8:0/363799428 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe86406ceb0 0x7fe864138900 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe85c0099f0 tx=0x7fe85c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:43.955 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:43.947+0000 7fe860ff9700 1 -- 172.21.15.8:0/363799428 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe85c027470 con 0x7fe86406ceb0 2023-10-05T19:39:43.955 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:43.947+0000 7fe860ff9700 1 -- 172.21.15.8:0/363799428 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe85c0275d0 con 0x7fe86406ceb0 2023-10-05T19:39:43.955 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:43.947+0000 7fe869aeb700 1 -- 172.21.15.8:0/363799428 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe86411d270 con 0x7fe86406ceb0 2023-10-05T19:39:43.955 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:43.947+0000 7fe860ff9700 1 -- 172.21.15.8:0/363799428 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe85c0278c0 con 0x7fe86406ceb0 2023-10-05T19:39:43.955 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:43.947+0000 7fe869aeb700 1 -- 172.21.15.8:0/363799428 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe86411c9d0 con 0x7fe86406ceb0 2023-10-05T19:39:43.956 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:43.947+0000 7fe860ff9700 1 -- 172.21.15.8:0/363799428 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 4) v1 ==== 40273+0+0 (secure 0 0 0) 0x7fe85c024070 con 0x7fe86406ceb0 2023-10-05T19:39:43.956 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:43.947+0000 7fe869aeb700 1 -- 172.21.15.8:0/363799428 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe86405ecb0 con 0x7fe86406ceb0 2023-10-05T19:39:43.956 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:43.951+0000 7fe860ff9700 1 --2- 172.21.15.8:0/363799428 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe844032350 0x7fe844034800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:43.956 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:43.951+0000 7fe860ff9700 1 -- 172.21.15.8:0/363799428 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1619+0+0 (secure 0 0 0) 0x7fe85c045b10 con 0x7fe86406ceb0 2023-10-05T19:39:43.957 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:43.951+0000 7fe863fff700 1 --2- 172.21.15.8:0/363799428 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe844032350 0x7fe844034800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:43.957 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:43.951+0000 7fe863fff700 1 --2- 172.21.15.8:0/363799428 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe844032350 0x7fe844034800 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe8540016a0 tx=0x7fe854007b30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:43.965 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:43.959+0000 7fe860ff9700 1 -- 172.21.15.8:0/363799428 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fe85c027a20 con 0x7fe86406ceb0 2023-10-05T19:39:44.130 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:44.123+0000 7fe869aeb700 1 -- 172.21.15.8:0/363799428 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe86404b5c0 con 0x7fe86406ceb0 2023-10-05T19:39:44.131 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:44.123+0000 7fe860ff9700 1 -- 172.21.15.8:0/363799428 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v8) v1 ==== 74+0+4108 (secure 0 0 0) 0x7fe85c01cba0 con 0x7fe86406ceb0 2023-10-05T19:39:44.131 INFO:teuthology.misc.health.smithi008.stdout: 2023-10-05T19:39:44.132 INFO:teuthology.misc.health.smithi008.stdout:{"epoch":8,"fsid":"dac6c186-0634-45d9-839d-04f1e484462d","created":"2023-10-05T19:39:35.575770+0000","modified":"2023-10-05T19:39:42.549781+0000","last_up_change":"0.000000","last_in_change":"2023-10-05T19:39:36.620941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"c23c6d99-5a1e-499d-8ef8-e1a58dcd9e74","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e578b376-27a7-47e2-b262-9a2a703d2504","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f687bf28-3a5f-4cd8-a242-9b95067cc77a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"db25fbc8-9f15-4a87-8310-fcc69bb92fa3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-05T19:39:44.134 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:44.127+0000 7fe869aeb700 1 -- 172.21.15.8:0/363799428 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe844032350 msgr2=0x7fe844034800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:44.134 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:44.127+0000 7fe869aeb700 1 --2- 172.21.15.8:0/363799428 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe844032350 0x7fe844034800 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe8540016a0 tx=0x7fe854007b30 comp rx=0 tx=0).stop 2023-10-05T19:39:44.134 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:44.127+0000 7fe869aeb700 1 -- 172.21.15.8:0/363799428 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe86406ceb0 msgr2=0x7fe864138900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:44.134 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:44.127+0000 7fe869aeb700 1 --2- 172.21.15.8:0/363799428 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe86406ceb0 0x7fe864138900 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe85c0099f0 tx=0x7fe85c018000 comp rx=0 tx=0).stop 2023-10-05T19:39:44.134 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:44.127+0000 7fe869aeb700 1 -- 172.21.15.8:0/363799428 shutdown_connections 2023-10-05T19:39:44.134 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:44.127+0000 7fe869aeb700 1 --2- 172.21.15.8:0/363799428 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe844032350 0x7fe844034800 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:44.134 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:44.127+0000 7fe869aeb700 1 --2- 172.21.15.8:0/363799428 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe86406ceb0 0x7fe864138900 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:44.134 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:44.127+0000 7fe869aeb700 1 -- 172.21.15.8:0/363799428 >> 172.21.15.8:0/363799428 conn(0x7fe86409a6d0 msgr2=0x7fe864129350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:44.135 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:44.127+0000 7fe869aeb700 1 -- 172.21.15.8:0/363799428 shutdown_connections 2023-10-05T19:39:44.135 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:44.127+0000 7fe869aeb700 1 -- 172.21.15.8:0/363799428 wait complete. 2023-10-05T19:39:44.144 DEBUG:teuthology.misc:0 of 4 OSDs are up 2023-10-05T19:39:44.496 INFO:tasks.ceph.osd.3.smithi012.stderr:2023-10-05T19:39:44.488+0000 7f18840a2700 -1 osd.3 0 waiting for initial osdmap 2023-10-05T19:39:44.505 INFO:tasks.ceph.osd.3.smithi012.stderr:2023-10-05T19:39:44.496+0000 7f187ce46700 -1 osd.3 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-05T19:39:44.547 INFO:tasks.ceph.osd.1.smithi012.stderr:2023-10-05T19:39:44.540+0000 7efe2986e700 -1 osd.1 0 waiting for initial osdmap 2023-10-05T19:39:44.552 INFO:tasks.ceph.osd.1.smithi012.stderr:2023-10-05T19:39:44.544+0000 7efe22612700 -1 osd.1 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-05T19:39:44.579 INFO:tasks.ceph.osd.2.smithi012.stderr:2023-10-05T19:39:44.572+0000 7f938e2e6700 -1 osd.2 0 waiting for initial osdmap 2023-10-05T19:39:44.586 INFO:tasks.ceph.osd.2.smithi012.stderr:2023-10-05T19:39:44.580+0000 7f938708a700 -1 osd.2 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-05T19:39:50.548 DEBUG:teuthology.orchestra.run.smithi008:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-05T19:39:50.653 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.646+0000 7fa570995700 1 Processor -- start 2023-10-05T19:39:50.653 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.646+0000 7fa570995700 1 -- start start 2023-10-05T19:39:50.653 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.646+0000 7fa570995700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fa56c090170 0x7fa56c132d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:50.653 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.646+0000 7fa570995700 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa56c091590 con 0x7fa56c08f690 2023-10-05T19:39:50.654 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.646+0000 7fa570995700 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa56c090cf0 con 0x7fa56c090170 2023-10-05T19:39:50.655 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.646+0000 7fa56a59c700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fa56c090170 0x7fa56c132d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:50.655 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.646+0000 7fa56ad9d700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7fa56c08f690 0x7fa56c08fa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:42642/0 (socket says 172.21.15.8:42642) 2023-10-05T19:39:50.655 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.646+0000 7fa56a59c700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fa56c090170 0x7fa56c132d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38450/0 (socket says 172.21.15.8:38450) 2023-10-05T19:39:50.656 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.646+0000 7fa56ad9d700 1 -- 172.21.15.8:0/4187237075 learned_addr learned my addr 172.21.15.8:0/4187237075 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-05T19:39:50.656 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.646+0000 7fa56a59c700 1 -- 172.21.15.8:0/4187237075 >> v1:172.21.15.8:6789/0 conn(0x7fa56c08f690 legacy=0x7fa56c08fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:50.656 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.646+0000 7fa56a59c700 1 -- 172.21.15.8:0/4187237075 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa56c091bb0 con 0x7fa56c090170 2023-10-05T19:39:50.656 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.646+0000 7fa56a59c700 1 --2- 172.21.15.8:0/4187237075 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa56c090170 0x7fa56c132d90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa564009de0 tx=0x7fa564009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cac170db98cee3d server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:50.656 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.646+0000 7fa569d9b700 1 -- 172.21.15.8:0/4187237075 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa56400e070 con 0x7fa56c090170 2023-10-05T19:39:50.656 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.646+0000 7fa569d9b700 1 -- 172.21.15.8:0/4187237075 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa56400bc80 con 0x7fa56c090170 2023-10-05T19:39:50.656 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.646+0000 7fa569d9b700 1 -- 172.21.15.8:0/4187237075 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa564012430 con 0x7fa56c090170 2023-10-05T19:39:50.656 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.646+0000 7fa570995700 1 -- 172.21.15.8:0/4187237075 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa56c090170 msgr2=0x7fa56c132d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:50.656 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.646+0000 7fa570995700 1 --2- 172.21.15.8:0/4187237075 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa56c090170 0x7fa56c132d90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa564009de0 tx=0x7fa564009780 comp rx=0 tx=0).stop 2023-10-05T19:39:50.657 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.646+0000 7fa570995700 1 -- 172.21.15.8:0/4187237075 shutdown_connections 2023-10-05T19:39:50.657 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.646+0000 7fa570995700 1 --2- 172.21.15.8:0/4187237075 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa56c090170 0x7fa56c132d90 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:50.657 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.646+0000 7fa570995700 1 -- 172.21.15.8:0/4187237075 >> 172.21.15.8:0/4187237075 conn(0x7fa56c094ca0 msgr2=0x7fa56c0950a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:50.657 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.646+0000 7fa570995700 1 -- 172.21.15.8:0/4187237075 shutdown_connections 2023-10-05T19:39:50.657 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.646+0000 7fa570995700 1 -- 172.21.15.8:0/4187237075 wait complete. 2023-10-05T19:39:50.659 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.650+0000 7fa570995700 1 Processor -- start 2023-10-05T19:39:50.659 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.650+0000 7fa570995700 1 -- start start 2023-10-05T19:39:50.659 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.650+0000 7fa570995700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa56c08f690 0x7fa56c13cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:50.659 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.650+0000 7fa570995700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c08c6a0 con 0x7fa56c08f690 2023-10-05T19:39:50.659 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.650+0000 7fa56ad9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa56c08f690 0x7fa56c13cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:50.659 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.650+0000 7fa56ad9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa56c08f690 0x7fa56c13cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38460/0 (socket says 172.21.15.8:38460) 2023-10-05T19:39:50.659 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.650+0000 7fa56ad9d700 1 -- 172.21.15.8:0/1778725658 learned_addr learned my addr 172.21.15.8:0/1778725658 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-05T19:39:50.660 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.650+0000 7fa56ad9d700 1 -- 172.21.15.8:0/1778725658 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa564009a20 con 0x7fa56c08f690 2023-10-05T19:39:50.660 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.650+0000 7fa56ad9d700 1 --2- 172.21.15.8:0/1778725658 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa56c08f690 0x7fa56c13cc40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa55400c6c0 tx=0x7fa554007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:50.660 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.650+0000 7fa562ffd700 1 -- 172.21.15.8:0/1778725658 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa554005b30 con 0x7fa56c08f690 2023-10-05T19:39:50.660 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.650+0000 7fa562ffd700 1 -- 172.21.15.8:0/1778725658 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa554005c90 con 0x7fa56c08f690 2023-10-05T19:39:50.660 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.650+0000 7fa570995700 1 -- 172.21.15.8:0/1778725658 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa56c08e6d0 con 0x7fa56c08f690 2023-10-05T19:39:50.660 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.650+0000 7fa562ffd700 1 -- 172.21.15.8:0/1778725658 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa554004830 con 0x7fa56c08f690 2023-10-05T19:39:50.660 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.650+0000 7fa570995700 1 -- 172.21.15.8:0/1778725658 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa56c06eda0 con 0x7fa56c08f690 2023-10-05T19:39:50.661 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.650+0000 7fa562ffd700 1 -- 172.21.15.8:0/1778725658 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fa554004990 con 0x7fa56c08f690 2023-10-05T19:39:50.661 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.650+0000 7fa570995700 1 -- 172.21.15.8:0/1778725658 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa56c08e2c0 con 0x7fa56c08f690 2023-10-05T19:39:50.661 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.654+0000 7fa562ffd700 1 --2- 172.21.15.8:0/1778725658 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fa5500323e0 0x7fa550034890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:50.661 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.654+0000 7fa562ffd700 1 -- 172.21.15.8:0/1778725658 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 3125+0+0 (secure 0 0 0) 0x7fa554042cd0 con 0x7fa56c08f690 2023-10-05T19:39:50.661 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.654+0000 7fa56a59c700 1 --2- 172.21.15.8:0/1778725658 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fa5500323e0 0x7fa550034890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:50.662 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.654+0000 7fa56a59c700 1 --2- 172.21.15.8:0/1778725658 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fa5500323e0 0x7fa550034890 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa5640097c0 tx=0x7fa564018000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:50.666 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.658+0000 7fa562ffd700 1 -- 172.21.15.8:0/1778725658 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fa5540108f0 con 0x7fa56c08f690 2023-10-05T19:39:50.834 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.826+0000 7fa570995700 1 -- 172.21.15.8:0/1778725658 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa56c05ecb0 con 0x7fa56c08f690 2023-10-05T19:39:50.835 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.826+0000 7fa562ffd700 1 -- 172.21.15.8:0/1778725658 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7411 (secure 0 0 0) 0x7fa55401b360 con 0x7fa56c08f690 2023-10-05T19:39:50.836 INFO:teuthology.misc.health.smithi008.stdout: 2023-10-05T19:39:50.836 INFO:teuthology.misc.health.smithi008.stdout:{"epoch":12,"fsid":"dac6c186-0634-45d9-839d-04f1e484462d","created":"2023-10-05T19:39:35.575770+0000","modified":"2023-10-05T19:39:47.903048+0000","last_up_change":"2023-10-05T19:39:44.877343+0000","last_in_change":"2023-10-05T19:39:36.620941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-05T19:39:45.900259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"c23c6d99-5a1e-499d-8ef8-e1a58dcd9e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6800","nonce":27563},{"type":"v1","addr":"172.21.15.8:6801","nonce":27563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":27563},{"type":"v1","addr":"172.21.15.8:6803","nonce":27563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":27563},{"type":"v1","addr":"172.21.15.8:6807","nonce":27563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":27563},{"type":"v1","addr":"172.21.15.8:6805","nonce":27563}]},"public_addr":"172.21.15.8:6801/27563","cluster_addr":"172.21.15.8:6803/27563","heartbeat_back_addr":"172.21.15.8:6807/27563","heartbeat_front_addr":"172.21.15.8:6805/27563","state":["exists","up"]},{"osd":1,"uuid":"e578b376-27a7-47e2-b262-9a2a703d2504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":29933},{"type":"v1","addr":"172.21.15.12:6809","nonce":29933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":29933},{"type":"v1","addr":"172.21.15.12:6811","nonce":29933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":29933},{"type":"v1","addr":"172.21.15.12:6815","nonce":29933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":29933},{"type":"v1","addr":"172.21.15.12:6813","nonce":29933}]},"public_addr":"172.21.15.12:6809/29933","cluster_addr":"172.21.15.12:6811/29933","heartbeat_back_addr":"172.21.15.12:6815/29933","heartbeat_front_addr":"172.21.15.12:6813/29933","state":["exists","up"]},{"osd":2,"uuid":"f687bf28-3a5f-4cd8-a242-9b95067cc77a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":29934},{"type":"v1","addr":"172.21.15.12:6801","nonce":29934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":29934},{"type":"v1","addr":"172.21.15.12:6803","nonce":29934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":29934},{"type":"v1","addr":"172.21.15.12:6807","nonce":29934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":29934},{"type":"v1","addr":"172.21.15.12:6805","nonce":29934}]},"public_addr":"172.21.15.12:6801/29934","cluster_addr":"172.21.15.12:6803/29934","heartbeat_back_addr":"172.21.15.12:6807/29934","heartbeat_front_addr":"172.21.15.12:6805/29934","state":["exists","up"]},{"osd":3,"uuid":"db25fbc8-9f15-4a87-8310-fcc69bb92fa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":29951},{"type":"v1","addr":"172.21.15.12:6817","nonce":29951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":29951},{"type":"v1","addr":"172.21.15.12:6819","nonce":29951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":29951},{"type":"v1","addr":"172.21.15.12:6823","nonce":29951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":29951},{"type":"v1","addr":"172.21.15.12:6821","nonce":29951}]},"public_addr":"172.21.15.12:6817/29951","cluster_addr":"172.21.15.12:6819/29951","heartbeat_back_addr":"172.21.15.12:6823/29951","heartbeat_front_addr":"172.21.15.12:6821/29951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-05T19:39:41.740161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-05T19:39:42.128798+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-05T19:39:42.140705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-05T19:39:41.961354+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-05T19:39:50.838 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.830+0000 7fa570995700 1 -- 172.21.15.8:0/1778725658 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fa5500323e0 msgr2=0x7fa550034890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:50.838 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.830+0000 7fa570995700 1 --2- 172.21.15.8:0/1778725658 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fa5500323e0 0x7fa550034890 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa5640097c0 tx=0x7fa564018000 comp rx=0 tx=0).stop 2023-10-05T19:39:50.838 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.830+0000 7fa570995700 1 -- 172.21.15.8:0/1778725658 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa56c08f690 msgr2=0x7fa56c13cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:50.839 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.830+0000 7fa570995700 1 --2- 172.21.15.8:0/1778725658 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa56c08f690 0x7fa56c13cc40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa55400c6c0 tx=0x7fa554007f70 comp rx=0 tx=0).stop 2023-10-05T19:39:50.839 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.830+0000 7fa570995700 1 -- 172.21.15.8:0/1778725658 shutdown_connections 2023-10-05T19:39:50.839 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.830+0000 7fa570995700 1 --2- 172.21.15.8:0/1778725658 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fa5500323e0 0x7fa550034890 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:50.839 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.830+0000 7fa570995700 1 --2- 172.21.15.8:0/1778725658 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa56c08f690 0x7fa56c13cc40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:50.839 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.830+0000 7fa570995700 1 -- 172.21.15.8:0/1778725658 >> 172.21.15.8:0/1778725658 conn(0x7fa56c094ca0 msgr2=0x7fa56c06cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:50.839 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.830+0000 7fa570995700 1 -- 172.21.15.8:0/1778725658 shutdown_connections 2023-10-05T19:39:50.839 INFO:teuthology.misc.health.smithi008.stderr:2023-10-05T19:39:50.830+0000 7fa570995700 1 -- 172.21.15.8:0/1778725658 wait complete. 2023-10-05T19:39:50.849 DEBUG:teuthology.misc:4 of 4 OSDs are up 2023-10-05T19:39:50.849 INFO:tasks.ceph:Creating RBD pool 2023-10-05T19:39:50.849 DEBUG:teuthology.orchestra.run.smithi008:> sudo ceph --cluster ceph osd pool create rbd 8 2023-10-05T19:39:50.957 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:50.950+0000 7fd225fc3700 1 Processor -- start 2023-10-05T19:39:50.957 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:50.950+0000 7fd225fc3700 1 -- start start 2023-10-05T19:39:50.959 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:50.950+0000 7fd225fc3700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fd2200693b0 0x7fd220132eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:50.959 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:50.950+0000 7fd225fc3700 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd22008e970 con 0x7fd2200688d0 2023-10-05T19:39:50.959 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:50.950+0000 7fd225fc3700 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2200697d0 con 0x7fd2200693b0 2023-10-05T19:39:50.959 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:50.950+0000 7fd224fc1700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7fd2200688d0 0x7fd220068ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:42654/0 (socket says 172.21.15.8:42654) 2023-10-05T19:39:50.959 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:50.950+0000 7fd224fc1700 1 -- 172.21.15.8:0/618091651 learned_addr learned my addr 172.21.15.8:0/618091651 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-05T19:39:50.959 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:50.950+0000 7fd21ffff700 1 --2- 172.21.15.8:0/618091651 >> v2:172.21.15.8:3300/0 conn(0x7fd2200693b0 0x7fd220132eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:50.959 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:50.950+0000 7fd21f7fe700 1 -- 172.21.15.8:0/618091651 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2336886220 0 0) 0x7fd22008e970 con 0x7fd2200688d0 2023-10-05T19:39:50.960 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:50.950+0000 7fd21f7fe700 1 -- 172.21.15.8:0/618091651 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd208003570 con 0x7fd2200688d0 2023-10-05T19:39:50.960 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:50.950+0000 7fd21ffff700 1 -- 172.21.15.8:0/618091651 >> v1:172.21.15.8:6789/0 conn(0x7fd2200688d0 legacy=0x7fd220068ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:50.960 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:50.950+0000 7fd21ffff700 1 -- 172.21.15.8:0/618091651 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2200950f0 con 0x7fd2200693b0 2023-10-05T19:39:50.960 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:50.950+0000 7fd21ffff700 1 --2- 172.21.15.8:0/618091651 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd2200693b0 0x7fd220132eb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd210009de0 tx=0x7fd210009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aec128775bb0bb6 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:50.960 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:50.950+0000 7fd21f7fe700 1 -- 172.21.15.8:0/618091651 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd21000e070 con 0x7fd2200693b0 2023-10-05T19:39:50.960 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:50.950+0000 7fd21f7fe700 1 -- 172.21.15.8:0/618091651 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd21000bca0 con 0x7fd2200693b0 2023-10-05T19:39:50.960 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:50.950+0000 7fd21f7fe700 1 -- 172.21.15.8:0/618091651 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd210012430 con 0x7fd2200693b0 2023-10-05T19:39:50.961 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:50.950+0000 7fd225fc3700 1 -- 172.21.15.8:0/618091651 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd2200693b0 msgr2=0x7fd220132eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:50.961 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:50.950+0000 7fd225fc3700 1 --2- 172.21.15.8:0/618091651 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd2200693b0 0x7fd220132eb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd210009de0 tx=0x7fd210009780 comp rx=0 tx=0).stop 2023-10-05T19:39:50.961 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:50.954+0000 7fd225fc3700 1 -- 172.21.15.8:0/618091651 shutdown_connections 2023-10-05T19:39:50.961 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:50.954+0000 7fd225fc3700 1 --2- 172.21.15.8:0/618091651 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd2200693b0 0x7fd220132eb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:50.961 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:50.954+0000 7fd225fc3700 1 -- 172.21.15.8:0/618091651 >> 172.21.15.8:0/618091651 conn(0x7fd220099460 msgr2=0x7fd220099860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:50.961 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:50.954+0000 7fd225fc3700 1 -- 172.21.15.8:0/618091651 shutdown_connections 2023-10-05T19:39:50.961 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:50.954+0000 7fd225fc3700 1 -- 172.21.15.8:0/618091651 wait complete. 2023-10-05T19:39:50.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:50.954+0000 7fd225fc3700 1 Processor -- start 2023-10-05T19:39:50.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:50.954+0000 7fd225fc3700 1 -- start start 2023-10-05T19:39:50.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:50.954+0000 7fd225fc3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd2200688d0 0x7fd22008b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:50.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:50.954+0000 7fd225fc3700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22008a520 con 0x7fd2200688d0 2023-10-05T19:39:50.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:50.954+0000 7fd224fc1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd2200688d0 0x7fd22008b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:50.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:50.954+0000 7fd224fc1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd2200688d0 0x7fd22008b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38486/0 (socket says 172.21.15.8:38486) 2023-10-05T19:39:50.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:50.954+0000 7fd224fc1700 1 -- 172.21.15.8:0/3315417997 learned_addr learned my addr 172.21.15.8:0/3315417997 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-05T19:39:50.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:50.954+0000 7fd224fc1700 1 -- 172.21.15.8:0/3315417997 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd210009a20 con 0x7fd2200688d0 2023-10-05T19:39:50.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:50.954+0000 7fd224fc1700 1 --2- 172.21.15.8:0/3315417997 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd2200688d0 0x7fd22008b480 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd21400c690 tx=0x7fd214007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:50.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:50.958+0000 7fd21cff9700 1 -- 172.21.15.8:0/3315417997 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd214007420 con 0x7fd2200688d0 2023-10-05T19:39:50.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:50.958+0000 7fd21cff9700 1 -- 172.21.15.8:0/3315417997 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd214007580 con 0x7fd2200688d0 2023-10-05T19:39:50.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:50.958+0000 7fd225fc3700 1 -- 172.21.15.8:0/3315417997 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd220092f80 con 0x7fd2200688d0 2023-10-05T19:39:50.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:50.958+0000 7fd21cff9700 1 -- 172.21.15.8:0/3315417997 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd214005930 con 0x7fd2200688d0 2023-10-05T19:39:50.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:50.958+0000 7fd225fc3700 1 -- 172.21.15.8:0/3315417997 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd22008af40 con 0x7fd2200688d0 2023-10-05T19:39:50.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:50.958+0000 7fd225fc3700 1 -- 172.21.15.8:0/3315417997 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd22005ecb0 con 0x7fd2200688d0 2023-10-05T19:39:50.967 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:50.958+0000 7fd21cff9700 1 -- 172.21.15.8:0/3315417997 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fd214005ba0 con 0x7fd2200688d0 2023-10-05T19:39:50.967 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:50.958+0000 7fd21cff9700 1 --2- 172.21.15.8:0/3315417997 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd20c032390 0x7fd20c034840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:50.967 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:50.958+0000 7fd21cff9700 1 -- 172.21.15.8:0/3315417997 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 3125+0+0 (secure 0 0 0) 0x7fd214042ee0 con 0x7fd2200688d0 2023-10-05T19:39:50.967 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:50.958+0000 7fd21ffff700 1 --2- 172.21.15.8:0/3315417997 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd20c032390 0x7fd20c034840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:50.968 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:50.958+0000 7fd21ffff700 1 --2- 172.21.15.8:0/3315417997 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd20c032390 0x7fd20c034840 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd210009480 tx=0x7fd210018000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:50.974 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:50.966+0000 7fd21cff9700 1 -- 172.21.15.8:0/3315417997 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fd21400e880 con 0x7fd2200688d0 2023-10-05T19:39:51.141 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.134+0000 7fd225fc3700 1 -- 172.21.15.8:0/3315417997 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7fd220098360 con 0x7fd2200688d0 2023-10-05T19:39:51.566 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.558+0000 7fd21cff9700 1 -- 172.21.15.8:0/3315417997 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v13) v1 ==== 109+0+0 (secure 0 0 0) 0x7fd21400e290 con 0x7fd2200688d0 2023-10-05T19:39:51.570 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.562+0000 7fd225fc3700 1 -- 172.21.15.8:0/3315417997 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd20c032390 msgr2=0x7fd20c034840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:51.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.562+0000 7fd225fc3700 1 --2- 172.21.15.8:0/3315417997 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd20c032390 0x7fd20c034840 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd210009480 tx=0x7fd210018000 comp rx=0 tx=0).stop 2023-10-05T19:39:51.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.562+0000 7fd225fc3700 1 -- 172.21.15.8:0/3315417997 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd2200688d0 msgr2=0x7fd22008b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:51.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.562+0000 7fd225fc3700 1 --2- 172.21.15.8:0/3315417997 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd2200688d0 0x7fd22008b480 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd21400c690 tx=0x7fd214007c80 comp rx=0 tx=0).stop 2023-10-05T19:39:51.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.562+0000 7fd225fc3700 1 -- 172.21.15.8:0/3315417997 shutdown_connections 2023-10-05T19:39:51.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.562+0000 7fd225fc3700 1 --2- 172.21.15.8:0/3315417997 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd20c032390 0x7fd20c034840 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:51.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.562+0000 7fd225fc3700 1 --2- 172.21.15.8:0/3315417997 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd2200688d0 0x7fd22008b480 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:51.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.562+0000 7fd225fc3700 1 -- 172.21.15.8:0/3315417997 >> 172.21.15.8:0/3315417997 conn(0x7fd220099460 msgr2=0x7fd22006a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:51.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.562+0000 7fd225fc3700 1 -- 172.21.15.8:0/3315417997 shutdown_connections 2023-10-05T19:39:51.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.562+0000 7fd225fc3700 1 -- 172.21.15.8:0/3315417997 wait complete. 2023-10-05T19:39:51.572 INFO:teuthology.orchestra.run.smithi008.stderr:pool 'rbd' created 2023-10-05T19:39:51.584 DEBUG:teuthology.orchestra.run.smithi008:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-10-05T19:39:51.707 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.698+0000 7fa9de8aa700 1 Processor -- start 2023-10-05T19:39:51.707 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.698+0000 7fa9de8aa700 1 -- start start 2023-10-05T19:39:51.708 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.702+0000 7fa9de8aa700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fa9d8098030 0x7fa9d8093b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:51.709 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.702+0000 7fa9de8aa700 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9d8095f10 con 0x7fa9d8094050 2023-10-05T19:39:51.709 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.702+0000 7fa9de8aa700 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9d80953a0 con 0x7fa9d8098030 2023-10-05T19:39:51.709 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.702+0000 7fa9dd0a7700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7fa9d8094050 0x7fa9d8129780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:42658/0 (socket says 172.21.15.8:42658) 2023-10-05T19:39:51.709 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.702+0000 7fa9dd0a7700 1 -- 172.21.15.8:0/677392227 learned_addr learned my addr 172.21.15.8:0/677392227 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-05T19:39:51.709 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.702+0000 7fa9dd8a8700 1 --2- 172.21.15.8:0/677392227 >> v2:172.21.15.8:3300/0 conn(0x7fa9d8098030 0x7fa9d8093b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:51.710 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.702+0000 7fa9dc8a6700 1 -- 172.21.15.8:0/677392227 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2844124216 0 0) 0x7fa9d8095f10 con 0x7fa9d8094050 2023-10-05T19:39:51.710 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.702+0000 7fa9dc8a6700 1 -- 172.21.15.8:0/677392227 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9c4003570 con 0x7fa9d8094050 2023-10-05T19:39:51.710 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.702+0000 7fa9dd8a8700 1 -- 172.21.15.8:0/677392227 >> v1:172.21.15.8:6789/0 conn(0x7fa9d8094050 legacy=0x7fa9d8129780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:51.710 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.702+0000 7fa9dd8a8700 1 -- 172.21.15.8:0/677392227 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d8098ee0 con 0x7fa9d8098030 2023-10-05T19:39:51.710 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.702+0000 7fa9dd8a8700 1 --2- 172.21.15.8:0/677392227 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa9d8098030 0x7fa9d8093b10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4009850 tx=0x7fa9d4009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc7d534b1ddd8b74 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:51.710 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.702+0000 7fa9dc8a6700 1 -- 172.21.15.8:0/677392227 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9d4005c10 con 0x7fa9d8098030 2023-10-05T19:39:51.711 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.702+0000 7fa9dc8a6700 1 -- 172.21.15.8:0/677392227 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d4005d70 con 0x7fa9d8098030 2023-10-05T19:39:51.711 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.702+0000 7fa9dc8a6700 1 -- 172.21.15.8:0/677392227 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9d4006170 con 0x7fa9d8098030 2023-10-05T19:39:51.711 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.702+0000 7fa9de8aa700 1 -- 172.21.15.8:0/677392227 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa9d8098030 msgr2=0x7fa9d8093b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:51.711 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.702+0000 7fa9de8aa700 1 --2- 172.21.15.8:0/677392227 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa9d8098030 0x7fa9d8093b10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4009850 tx=0x7fa9d4009120 comp rx=0 tx=0).stop 2023-10-05T19:39:51.712 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.702+0000 7fa9de8aa700 1 -- 172.21.15.8:0/677392227 shutdown_connections 2023-10-05T19:39:51.712 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.702+0000 7fa9de8aa700 1 --2- 172.21.15.8:0/677392227 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa9d8098030 0x7fa9d8093b10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:51.712 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.702+0000 7fa9de8aa700 1 -- 172.21.15.8:0/677392227 >> 172.21.15.8:0/677392227 conn(0x7fa9d8099120 msgr2=0x7fa9d8099520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:51.712 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.702+0000 7fa9de8aa700 1 -- 172.21.15.8:0/677392227 shutdown_connections 2023-10-05T19:39:51.712 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.702+0000 7fa9de8aa700 1 -- 172.21.15.8:0/677392227 wait complete. 2023-10-05T19:39:51.713 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.706+0000 7fa9de8aa700 1 Processor -- start 2023-10-05T19:39:51.713 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.706+0000 7fa9de8aa700 1 -- start start 2023-10-05T19:39:51.714 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.706+0000 7fa9de8aa700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa9d8094050 0x7fa9d808ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:51.714 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.706+0000 7fa9de8aa700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d80925c0 con 0x7fa9d8094050 2023-10-05T19:39:51.714 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.706+0000 7fa9dd8a8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa9d8094050 0x7fa9d808ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:51.714 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.706+0000 7fa9dd8a8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa9d8094050 0x7fa9d808ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38502/0 (socket says 172.21.15.8:38502) 2023-10-05T19:39:51.714 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.706+0000 7fa9dd8a8700 1 -- 172.21.15.8:0/765087391 learned_addr learned my addr 172.21.15.8:0/765087391 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-05T19:39:51.715 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.706+0000 7fa9dd8a8700 1 -- 172.21.15.8:0/765087391 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d40093c0 con 0x7fa9d8094050 2023-10-05T19:39:51.715 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.706+0000 7fa9dd8a8700 1 --2- 172.21.15.8:0/765087391 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa9d8094050 0x7fa9d808ea90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4003080 tx=0x7fa9d4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:51.716 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.706+0000 7fa9cdffb700 1 -- 172.21.15.8:0/765087391 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9d4011a00 con 0x7fa9d8094050 2023-10-05T19:39:51.716 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.706+0000 7fa9cdffb700 1 -- 172.21.15.8:0/765087391 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d4011b80 con 0x7fa9d8094050 2023-10-05T19:39:51.716 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.706+0000 7fa9de8aa700 1 -- 172.21.15.8:0/765087391 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9d806ed10 con 0x7fa9d8094050 2023-10-05T19:39:51.716 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.706+0000 7fa9cdffb700 1 -- 172.21.15.8:0/765087391 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9d4011e70 con 0x7fa9d8094050 2023-10-05T19:39:51.716 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.706+0000 7fa9de8aa700 1 -- 172.21.15.8:0/765087391 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9d8090d50 con 0x7fa9d8094050 2023-10-05T19:39:51.717 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.710+0000 7fa9c37fe700 1 -- 172.21.15.8:0/765087391 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9d805ecb0 con 0x7fa9d8094050 2023-10-05T19:39:51.717 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.710+0000 7fa9cdffb700 1 -- 172.21.15.8:0/765087391 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fa9d4023070 con 0x7fa9d8094050 2023-10-05T19:39:51.718 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.710+0000 7fa9cdffb700 1 --2- 172.21.15.8:0/765087391 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fa9bc0323e0 0x7fa9bc034890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:51.718 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.710+0000 7fa9cdffb700 1 -- 172.21.15.8:0/765087391 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 3467+0+0 (secure 0 0 0) 0x7fa9d40457a0 con 0x7fa9d8094050 2023-10-05T19:39:51.718 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.710+0000 7fa9dd0a7700 1 --2- 172.21.15.8:0/765087391 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fa9bc0323e0 0x7fa9bc034890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:51.719 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.710+0000 7fa9dd0a7700 1 --2- 172.21.15.8:0/765087391 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fa9bc0323e0 0x7fa9bc034890 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa9c80058d0 tx=0x7fa9c80079d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:51.726 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.718+0000 7fa9cdffb700 1 -- 172.21.15.8:0/765087391 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fa9d401d430 con 0x7fa9d8094050 2023-10-05T19:39:51.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:51.882+0000 7fa9c37fe700 1 -- 172.21.15.8:0/765087391 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa9d8095c00 con 0x7fa9d8094050 2023-10-05T19:39:52.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.558+0000 7fa9cdffb700 1 -- 172.21.15.8:0/765087391 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v14) v1 ==== 173+0+0 (secure 0 0 0) 0x7fa9d402e3f0 con 0x7fa9d8094050 2023-10-05T19:39:52.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.566+0000 7fa9c37fe700 1 -- 172.21.15.8:0/765087391 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fa9bc0323e0 msgr2=0x7fa9bc034890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:52.574 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.566+0000 7fa9c37fe700 1 --2- 172.21.15.8:0/765087391 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fa9bc0323e0 0x7fa9bc034890 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa9c80058d0 tx=0x7fa9c80079d0 comp rx=0 tx=0).stop 2023-10-05T19:39:52.574 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.566+0000 7fa9c37fe700 1 -- 172.21.15.8:0/765087391 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa9d8094050 msgr2=0x7fa9d808ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:52.574 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.566+0000 7fa9c37fe700 1 --2- 172.21.15.8:0/765087391 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa9d8094050 0x7fa9d808ea90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4003080 tx=0x7fa9d4015000 comp rx=0 tx=0).stop 2023-10-05T19:39:52.574 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.566+0000 7fa9c37fe700 1 -- 172.21.15.8:0/765087391 shutdown_connections 2023-10-05T19:39:52.574 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.566+0000 7fa9c37fe700 1 --2- 172.21.15.8:0/765087391 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fa9bc0323e0 0x7fa9bc034890 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:52.574 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.566+0000 7fa9c37fe700 1 --2- 172.21.15.8:0/765087391 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa9d8094050 0x7fa9d808ea90 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4003080 tx=0x7fa9d4015000 comp rx=0 tx=0).stop 2023-10-05T19:39:52.574 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.566+0000 7fa9c37fe700 1 -- 172.21.15.8:0/765087391 >> 172.21.15.8:0/765087391 conn(0x7fa9d8099120 msgr2=0x7fa9d806c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:52.575 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.566+0000 7fa9c37fe700 1 -- 172.21.15.8:0/765087391 shutdown_connections 2023-10-05T19:39:52.575 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.566+0000 7fa9c37fe700 1 -- 172.21.15.8:0/765087391 wait complete. 2023-10-05T19:39:52.575 INFO:teuthology.orchestra.run.smithi008.stderr:enabled application 'rbd' on pool 'rbd' 2023-10-05T19:39:52.588 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-05T19:39:52.589 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-05T19:39:52.589 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-05T19:39:52.589 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-05T19:39:52.589 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-05T19:39:52.695 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.686+0000 7f5cddd46700 1 Processor -- start 2023-10-05T19:39:52.695 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.686+0000 7f5cddd46700 1 -- start start 2023-10-05T19:39:52.695 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.686+0000 7f5cddd46700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f5cd8095ff0 0x7f5cd80963c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:52.695 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.686+0000 7f5cddd46700 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cd8092760 con 0x7f5cd80905f0 2023-10-05T19:39:52.695 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.686+0000 7f5cddd46700 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cd80918a0 con 0x7f5cd8095ff0 2023-10-05T19:39:52.696 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.686+0000 7f5cdcd44700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f5cd8095ff0 0x7f5cd80963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:52.696 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.686+0000 7f5cd7fff700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f5cd80905f0 0x7f5cd80909c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:42668/0 (socket says 172.21.15.8:42668) 2023-10-05T19:39:52.696 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.686+0000 7f5cd7fff700 1 -- 172.21.15.8:0/2241644234 learned_addr learned my addr 172.21.15.8:0/2241644234 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-05T19:39:52.697 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.686+0000 7f5cdcd44700 1 -- 172.21.15.8:0/2241644234 >> v1:172.21.15.8:6789/0 conn(0x7f5cd80905f0 legacy=0x7f5cd80909c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:52.697 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.686+0000 7f5cdcd44700 1 -- 172.21.15.8:0/2241644234 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd8093ba0 con 0x7f5cd8095ff0 2023-10-05T19:39:52.697 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.686+0000 7f5cdcd44700 1 --2- 172.21.15.8:0/2241644234 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5cd8095ff0 0x7f5cd80963c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8009860 tx=0x7f5cc80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd18c9ec58813305 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:52.697 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.690+0000 7f5cd77fe700 1 -- 172.21.15.8:0/2241644234 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5cc800b680 con 0x7f5cd8095ff0 2023-10-05T19:39:52.697 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.690+0000 7f5cd77fe700 1 -- 172.21.15.8:0/2241644234 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cc800b7e0 con 0x7f5cd8095ff0 2023-10-05T19:39:52.697 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.690+0000 7f5cd77fe700 1 -- 172.21.15.8:0/2241644234 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5cc800b990 con 0x7f5cd8095ff0 2023-10-05T19:39:52.698 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.690+0000 7f5cddd46700 1 -- 172.21.15.8:0/2241644234 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5cd8095ff0 msgr2=0x7f5cd80963c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:52.698 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.690+0000 7f5cddd46700 1 --2- 172.21.15.8:0/2241644234 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5cd8095ff0 0x7f5cd80963c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8009860 tx=0x7f5cc80091a0 comp rx=0 tx=0).stop 2023-10-05T19:39:52.698 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.690+0000 7f5cddd46700 1 -- 172.21.15.8:0/2241644234 shutdown_connections 2023-10-05T19:39:52.698 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.690+0000 7f5cddd46700 1 --2- 172.21.15.8:0/2241644234 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5cd8095ff0 0x7f5cd80963c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:52.699 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.690+0000 7f5cddd46700 1 -- 172.21.15.8:0/2241644234 >> 172.21.15.8:0/2241644234 conn(0x7f5cd809c5d0 msgr2=0x7f5cd809c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:52.699 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.690+0000 7f5cddd46700 1 -- 172.21.15.8:0/2241644234 shutdown_connections 2023-10-05T19:39:52.699 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.690+0000 7f5cddd46700 1 -- 172.21.15.8:0/2241644234 wait complete. 2023-10-05T19:39:52.699 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.690+0000 7f5cddd46700 1 Processor -- start 2023-10-05T19:39:52.699 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.690+0000 7f5cddd46700 1 -- start start 2023-10-05T19:39:52.700 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.690+0000 7f5cddd46700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5cd80905f0 0x7f5cd808c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:52.700 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.690+0000 7f5cddd46700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd8095b80 con 0x7f5cd80905f0 2023-10-05T19:39:52.701 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.694+0000 7f5cdcd44700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5cd80905f0 0x7f5cd808c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:52.701 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.694+0000 7f5cdcd44700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5cd80905f0 0x7f5cd808c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38524/0 (socket says 172.21.15.8:38524) 2023-10-05T19:39:52.701 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.694+0000 7f5cdcd44700 1 -- 172.21.15.8:0/310475728 learned_addr learned my addr 172.21.15.8:0/310475728 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-05T19:39:52.701 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.694+0000 7f5cdcd44700 1 -- 172.21.15.8:0/310475728 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc8009420 con 0x7f5cd80905f0 2023-10-05T19:39:52.702 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.694+0000 7f5cdcd44700 1 --2- 172.21.15.8:0/310475728 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5cd80905f0 0x7f5cd808c060 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8000c00 tx=0x7f5cc8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:52.702 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.694+0000 7f5cd4ff9700 1 -- 172.21.15.8:0/310475728 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5cc8024470 con 0x7f5cd80905f0 2023-10-05T19:39:52.702 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.694+0000 7f5cd4ff9700 1 -- 172.21.15.8:0/310475728 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cc80245d0 con 0x7f5cd80905f0 2023-10-05T19:39:52.702 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.694+0000 7f5cddd46700 1 -- 172.21.15.8:0/310475728 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cd808f720 con 0x7f5cd80905f0 2023-10-05T19:39:52.702 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.694+0000 7f5cd4ff9700 1 -- 172.21.15.8:0/310475728 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5cc8024740 con 0x7f5cd80905f0 2023-10-05T19:39:52.702 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.694+0000 7f5cddd46700 1 -- 172.21.15.8:0/310475728 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cd808f9c0 con 0x7f5cd80905f0 2023-10-05T19:39:52.703 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.694+0000 7f5cd4ff9700 1 -- 172.21.15.8:0/310475728 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f5cc8021070 con 0x7f5cd80905f0 2023-10-05T19:39:52.703 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.694+0000 7f5cddd46700 1 -- 172.21.15.8:0/310475728 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cd805ecb0 con 0x7f5cd80905f0 2023-10-05T19:39:52.704 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.694+0000 7f5cd4ff9700 1 --2- 172.21.15.8:0/310475728 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5cb80323e0 0x7f5cb8034890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:52.704 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.694+0000 7f5cd4ff9700 1 -- 172.21.15.8:0/310475728 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 3478+0+0 (secure 0 0 0) 0x7f5cc8045740 con 0x7f5cd80905f0 2023-10-05T19:39:52.704 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.694+0000 7f5cd7fff700 1 --2- 172.21.15.8:0/310475728 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5cb80323e0 0x7f5cb8034890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:52.704 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.698+0000 7f5cd7fff700 1 --2- 172.21.15.8:0/310475728 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5cb80323e0 0x7f5cb8034890 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5cc00010e0 tx=0x7f5cc00079b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:52.711 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.702+0000 7f5cd4ff9700 1 -- 172.21.15.8:0/310475728 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f5cc801a460 con 0x7f5cd80905f0 2023-10-05T19:39:52.912 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.902+0000 7f5cddd46700 1 -- 172.21.15.8:0/310475728 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f5cd8091b10 con 0x7f5cd80905f0 2023-10-05T19:39:52.915 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.906+0000 7f5cd4ff9700 1 -- 172.21.15.8:0/310475728 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+77912 (secure 0 0 0) 0x7f5cc802c3a0 con 0x7f5cd80905f0 2023-10-05T19:39:52.916 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-05T19:39:52.918 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.910+0000 7f5cddd46700 1 -- 172.21.15.8:0/310475728 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5cb80323e0 msgr2=0x7f5cb8034890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:52.918 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.910+0000 7f5cddd46700 1 --2- 172.21.15.8:0/310475728 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5cb80323e0 0x7f5cb8034890 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5cc00010e0 tx=0x7f5cc00079b0 comp rx=0 tx=0).stop 2023-10-05T19:39:52.918 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.910+0000 7f5cddd46700 1 -- 172.21.15.8:0/310475728 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5cd80905f0 msgr2=0x7f5cd808c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:52.918 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.910+0000 7f5cddd46700 1 --2- 172.21.15.8:0/310475728 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5cd80905f0 0x7f5cd808c060 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8000c00 tx=0x7f5cc8016000 comp rx=0 tx=0).stop 2023-10-05T19:39:52.918 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.910+0000 7f5cddd46700 1 -- 172.21.15.8:0/310475728 shutdown_connections 2023-10-05T19:39:52.918 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.910+0000 7f5cddd46700 1 --2- 172.21.15.8:0/310475728 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5cb80323e0 0x7f5cb8034890 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:52.918 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.910+0000 7f5cddd46700 1 --2- 172.21.15.8:0/310475728 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5cd80905f0 0x7f5cd808c060 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:52.918 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.910+0000 7f5cddd46700 1 -- 172.21.15.8:0/310475728 >> 172.21.15.8:0/310475728 conn(0x7f5cd809c5d0 msgr2=0x7f5cd8094950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:52.918 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.910+0000 7f5cddd46700 1 -- 172.21.15.8:0/310475728 shutdown_connections 2023-10-05T19:39:52.918 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:52.910+0000 7f5cddd46700 1 -- 172.21.15.8:0/310475728 wait complete. 2023-10-05T19:39:52.929 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":5,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":29788},{"type":"v1","addr":"172.21.15.12:6825","nonce":29788}]},"active_addr":"172.21.15.12:6825/29788","active_change":"2023-10-05T19:39:39.821488+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":167396985}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":3817691286}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":2899130987}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":3561211571}]}]} 2023-10-05T19:39:52.931 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-05T19:39:52.931 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-05T19:39:52.931 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-05T19:39:53.039 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.030+0000 7fc3e359e700 1 Processor -- start 2023-10-05T19:39:53.040 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.030+0000 7fc3e359e700 1 -- start start 2023-10-05T19:39:53.040 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.030+0000 7fc3e359e700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fc3e4093bb0 0x7fc3e412db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:53.040 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.030+0000 7fc3e359e700 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3e4091240 con 0x7fc3e412e0b0 2023-10-05T19:39:53.040 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.030+0000 7fc3e359e700 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3e4091800 con 0x7fc3e4093bb0 2023-10-05T19:39:53.040 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.030+0000 7fc3e259c700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fc3e4093bb0 0x7fc3e412db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:53.040 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.030+0000 7fc3e259c700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fc3e4093bb0 0x7fc3e412db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38530/0 (socket says 172.21.15.8:38530) 2023-10-05T19:39:53.040 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.030+0000 7fc3e259c700 1 -- 172.21.15.8:0/3655461270 learned_addr learned my addr 172.21.15.8:0/3655461270 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-05T19:39:53.041 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.030+0000 7fc3e259c700 1 -- 172.21.15.8:0/3655461270 >> v1:172.21.15.8:6789/0 conn(0x7fc3e412e0b0 legacy=0x7fc3e408f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:53.041 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.030+0000 7fc3e259c700 1 -- 172.21.15.8:0/3655461270 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e40947e0 con 0x7fc3e4093bb0 2023-10-05T19:39:53.041 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.034+0000 7fc3e259c700 1 --2- 172.21.15.8:0/3655461270 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc3e4093bb0 0x7fc3e412db70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc009ea0 tx=0x7fc3dc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d0ed8d75f771e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:53.041 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.034+0000 7fc3e159a700 1 -- 172.21.15.8:0/3655461270 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3dc00e070 con 0x7fc3e4093bb0 2023-10-05T19:39:53.041 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.034+0000 7fc3e159a700 1 -- 172.21.15.8:0/3655461270 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3dc00ba00 con 0x7fc3e4093bb0 2023-10-05T19:39:53.041 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.034+0000 7fc3e159a700 1 -- 172.21.15.8:0/3655461270 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3dc0092e0 con 0x7fc3e4093bb0 2023-10-05T19:39:53.042 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.034+0000 7fc3e359e700 1 -- 172.21.15.8:0/3655461270 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc3e4093bb0 msgr2=0x7fc3e412db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:53.042 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.034+0000 7fc3e359e700 1 --2- 172.21.15.8:0/3655461270 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc3e4093bb0 0x7fc3e412db70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc009ea0 tx=0x7fc3dc009840 comp rx=0 tx=0).stop 2023-10-05T19:39:53.042 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.034+0000 7fc3e359e700 1 -- 172.21.15.8:0/3655461270 shutdown_connections 2023-10-05T19:39:53.043 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.034+0000 7fc3e359e700 1 --2- 172.21.15.8:0/3655461270 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc3e4093bb0 0x7fc3e412db70 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:53.043 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.034+0000 7fc3e359e700 1 -- 172.21.15.8:0/3655461270 >> 172.21.15.8:0/3655461270 conn(0x7fc3e40981b0 msgr2=0x7fc3e40985b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:53.043 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.034+0000 7fc3e359e700 1 -- 172.21.15.8:0/3655461270 shutdown_connections 2023-10-05T19:39:53.043 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.034+0000 7fc3e359e700 1 -- 172.21.15.8:0/3655461270 wait complete. 2023-10-05T19:39:53.043 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.034+0000 7fc3e359e700 1 Processor -- start 2023-10-05T19:39:53.043 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.034+0000 7fc3e359e700 1 -- start start 2023-10-05T19:39:53.044 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.034+0000 7fc3e359e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc3e4093bb0 0x7fc3e413c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:53.044 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.034+0000 7fc3e359e700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e412ab50 con 0x7fc3e4093bb0 2023-10-05T19:39:53.044 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.034+0000 7fc3e259c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc3e4093bb0 0x7fc3e413c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:53.045 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.034+0000 7fc3e259c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc3e4093bb0 0x7fc3e413c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38538/0 (socket says 172.21.15.8:38538) 2023-10-05T19:39:53.045 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.034+0000 7fc3e259c700 1 -- 172.21.15.8:0/3007718566 learned_addr learned my addr 172.21.15.8:0/3007718566 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-05T19:39:53.045 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.034+0000 7fc3e259c700 1 -- 172.21.15.8:0/3007718566 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3dc009ae0 con 0x7fc3e4093bb0 2023-10-05T19:39:53.045 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.034+0000 7fc3e259c700 1 --2- 172.21.15.8:0/3007718566 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc3e4093bb0 0x7fc3e413c7d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc009ab0 tx=0x7fc3dc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:53.045 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.038+0000 7fc3da7fc700 1 -- 172.21.15.8:0/3007718566 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3dc007170 con 0x7fc3e4093bb0 2023-10-05T19:39:53.045 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.038+0000 7fc3da7fc700 1 -- 172.21.15.8:0/3007718566 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3dc012b50 con 0x7fc3e4093bb0 2023-10-05T19:39:53.045 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.038+0000 7fc3e359e700 1 -- 172.21.15.8:0/3007718566 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3e412af90 con 0x7fc3e4093bb0 2023-10-05T19:39:53.045 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.038+0000 7fc3da7fc700 1 -- 172.21.15.8:0/3007718566 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3dc012e20 con 0x7fc3e4093bb0 2023-10-05T19:39:53.045 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.038+0000 7fc3e359e700 1 -- 172.21.15.8:0/3007718566 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3e412a3e0 con 0x7fc3e4093bb0 2023-10-05T19:39:53.046 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.038+0000 7fc3da7fc700 1 -- 172.21.15.8:0/3007718566 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fc3dc028460 con 0x7fc3e4093bb0 2023-10-05T19:39:53.046 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.038+0000 7fc3e359e700 1 -- 172.21.15.8:0/3007718566 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3e412b180 con 0x7fc3e4093bb0 2023-10-05T19:39:53.046 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.038+0000 7fc3da7fc700 1 --2- 172.21.15.8:0/3007718566 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fc3c4032390 0x7fc3c4034840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:53.046 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.038+0000 7fc3da7fc700 1 -- 172.21.15.8:0/3007718566 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 3478+0+0 (secure 0 0 0) 0x7fc3dc045520 con 0x7fc3e4093bb0 2023-10-05T19:39:53.046 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.038+0000 7fc3e1d9b700 1 --2- 172.21.15.8:0/3007718566 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fc3c4032390 0x7fc3c4034840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:53.047 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.038+0000 7fc3e1d9b700 1 --2- 172.21.15.8:0/3007718566 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fc3c4032390 0x7fc3c4034840 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc001730 tx=0x7fc3cc007b40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:53.055 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.046+0000 7fc3da7fc700 1 -- 172.21.15.8:0/3007718566 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fc3dc026950 con 0x7fc3e4093bb0 2023-10-05T19:39:53.217 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.210+0000 7fc3e359e700 1 -- 172.21.15.8:0/3007718566 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3e405ecb0 con 0x7fc3e4093bb0 2023-10-05T19:39:53.218 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.210+0000 7fc3da7fc700 1 -- 172.21.15.8:0/3007718566 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+8943 (secure 0 0 0) 0x7fc3dc011390 con 0x7fc3e4093bb0 2023-10-05T19:39:53.219 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-05T19:39:53.219 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":14,"fsid":"dac6c186-0634-45d9-839d-04f1e484462d","created":"2023-10-05T19:39:35.575770+0000","modified":"2023-10-05T19:39:52.559370+0000","last_up_change":"2023-10-05T19:39:44.877343+0000","last_in_change":"2023-10-05T19:39:36.620941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-05T19:39:45.900259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-05T19:39:51.178391+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c23c6d99-5a1e-499d-8ef8-e1a58dcd9e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6800","nonce":27563},{"type":"v1","addr":"172.21.15.8:6801","nonce":27563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":27563},{"type":"v1","addr":"172.21.15.8:6803","nonce":27563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":27563},{"type":"v1","addr":"172.21.15.8:6807","nonce":27563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":27563},{"type":"v1","addr":"172.21.15.8:6805","nonce":27563}]},"public_addr":"172.21.15.8:6801/27563","cluster_addr":"172.21.15.8:6803/27563","heartbeat_back_addr":"172.21.15.8:6807/27563","heartbeat_front_addr":"172.21.15.8:6805/27563","state":["exists","up"]},{"osd":1,"uuid":"e578b376-27a7-47e2-b262-9a2a703d2504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":29933},{"type":"v1","addr":"172.21.15.12:6809","nonce":29933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":29933},{"type":"v1","addr":"172.21.15.12:6811","nonce":29933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":29933},{"type":"v1","addr":"172.21.15.12:6815","nonce":29933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":29933},{"type":"v1","addr":"172.21.15.12:6813","nonce":29933}]},"public_addr":"172.21.15.12:6809/29933","cluster_addr":"172.21.15.12:6811/29933","heartbeat_back_addr":"172.21.15.12:6815/29933","heartbeat_front_addr":"172.21.15.12:6813/29933","state":["exists","up"]},{"osd":2,"uuid":"f687bf28-3a5f-4cd8-a242-9b95067cc77a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":29934},{"type":"v1","addr":"172.21.15.12:6801","nonce":29934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":29934},{"type":"v1","addr":"172.21.15.12:6803","nonce":29934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":29934},{"type":"v1","addr":"172.21.15.12:6807","nonce":29934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":29934},{"type":"v1","addr":"172.21.15.12:6805","nonce":29934}]},"public_addr":"172.21.15.12:6801/29934","cluster_addr":"172.21.15.12:6803/29934","heartbeat_back_addr":"172.21.15.12:6807/29934","heartbeat_front_addr":"172.21.15.12:6805/29934","state":["exists","up"]},{"osd":3,"uuid":"db25fbc8-9f15-4a87-8310-fcc69bb92fa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":29951},{"type":"v1","addr":"172.21.15.12:6817","nonce":29951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":29951},{"type":"v1","addr":"172.21.15.12:6819","nonce":29951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":29951},{"type":"v1","addr":"172.21.15.12:6823","nonce":29951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":29951},{"type":"v1","addr":"172.21.15.12:6821","nonce":29951}]},"public_addr":"172.21.15.12:6817/29951","cluster_addr":"172.21.15.12:6819/29951","heartbeat_back_addr":"172.21.15.12:6823/29951","heartbeat_front_addr":"172.21.15.12:6821/29951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-05T19:39:41.740161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-05T19:39:42.128798+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-05T19:39:42.140705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-05T19:39:41.961354+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-05T19:39:53.221 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.214+0000 7fc3e359e700 1 -- 172.21.15.8:0/3007718566 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fc3c4032390 msgr2=0x7fc3c4034840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:53.221 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.214+0000 7fc3e359e700 1 --2- 172.21.15.8:0/3007718566 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fc3c4032390 0x7fc3c4034840 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc001730 tx=0x7fc3cc007b40 comp rx=0 tx=0).stop 2023-10-05T19:39:53.221 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.214+0000 7fc3e359e700 1 -- 172.21.15.8:0/3007718566 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc3e4093bb0 msgr2=0x7fc3e413c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:53.221 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.214+0000 7fc3e359e700 1 --2- 172.21.15.8:0/3007718566 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc3e4093bb0 0x7fc3e413c7d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc009ab0 tx=0x7fc3dc017000 comp rx=0 tx=0).stop 2023-10-05T19:39:53.221 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.214+0000 7fc3e359e700 1 -- 172.21.15.8:0/3007718566 shutdown_connections 2023-10-05T19:39:53.222 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.214+0000 7fc3e359e700 1 --2- 172.21.15.8:0/3007718566 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fc3c4032390 0x7fc3c4034840 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:53.222 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.214+0000 7fc3e359e700 1 --2- 172.21.15.8:0/3007718566 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc3e4093bb0 0x7fc3e413c7d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:53.222 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.214+0000 7fc3e359e700 1 -- 172.21.15.8:0/3007718566 >> 172.21.15.8:0/3007718566 conn(0x7fc3e40981b0 msgr2=0x7fc3e406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:53.222 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.214+0000 7fc3e359e700 1 -- 172.21.15.8:0/3007718566 shutdown_connections 2023-10-05T19:39:53.222 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.214+0000 7fc3e359e700 1 -- 172.21.15.8:0/3007718566 wait complete. 2023-10-05T19:39:53.232 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-05T19:39:53.233 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-05T19:39:53.336 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.326+0000 7f690f59e700 1 Processor -- start 2023-10-05T19:39:53.336 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.326+0000 7f690f59e700 1 -- start start 2023-10-05T19:39:53.336 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.326+0000 7f690f59e700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f6910125f70 0x7f6910126340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:53.337 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.326+0000 7f690f59e700 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f691006d0d0 con 0x7f6910126880 2023-10-05T19:39:53.337 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.326+0000 7f690f59e700 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f691006c760 con 0x7f6910125f70 2023-10-05T19:39:53.337 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.330+0000 7f690e59c700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f6910125f70 0x7f6910126340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:53.337 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.330+0000 7f690dd9b700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f6910126880 0x7f6910122bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:42694/0 (socket says 172.21.15.8:42694) 2023-10-05T19:39:53.337 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.330+0000 7f690dd9b700 1 -- 172.21.15.8:0/2210259871 learned_addr learned my addr 172.21.15.8:0/2210259871 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-05T19:39:53.338 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.330+0000 7f690d59a700 1 -- 172.21.15.8:0/2210259871 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 572261292 0 0) 0x7f691006d0d0 con 0x7f6910126880 2023-10-05T19:39:53.338 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.330+0000 7f690d59a700 1 -- 172.21.15.8:0/2210259871 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68fc003570 con 0x7f6910126880 2023-10-05T19:39:53.338 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.330+0000 7f690e59c700 1 -- 172.21.15.8:0/2210259871 >> v1:172.21.15.8:6789/0 conn(0x7f6910126880 legacy=0x7f6910122bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:53.338 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.330+0000 7f690e59c700 1 -- 172.21.15.8:0/2210259871 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6910126cc0 con 0x7f6910125f70 2023-10-05T19:39:53.338 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.330+0000 7f690e59c700 1 --2- 172.21.15.8:0/2210259871 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6910125f70 0x7f6910126340 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6900009860 tx=0x7f69000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ac0e8ac32d446ee server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:53.339 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.330+0000 7f690d59a700 1 -- 172.21.15.8:0/2210259871 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6900011630 con 0x7f6910125f70 2023-10-05T19:39:53.339 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.330+0000 7f690d59a700 1 -- 172.21.15.8:0/2210259871 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6900011790 con 0x7f6910125f70 2023-10-05T19:39:53.339 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.330+0000 7f690d59a700 1 -- 172.21.15.8:0/2210259871 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6900011920 con 0x7f6910125f70 2023-10-05T19:39:53.340 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.330+0000 7f690f59e700 1 -- 172.21.15.8:0/2210259871 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6910125f70 msgr2=0x7f6910126340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:53.340 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.330+0000 7f690f59e700 1 --2- 172.21.15.8:0/2210259871 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6910125f70 0x7f6910126340 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6900009860 tx=0x7f69000091a0 comp rx=0 tx=0).stop 2023-10-05T19:39:53.340 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.330+0000 7f690f59e700 1 -- 172.21.15.8:0/2210259871 shutdown_connections 2023-10-05T19:39:53.340 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.330+0000 7f690f59e700 1 --2- 172.21.15.8:0/2210259871 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6910125f70 0x7f6910126340 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:53.340 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.330+0000 7f690f59e700 1 -- 172.21.15.8:0/2210259871 >> 172.21.15.8:0/2210259871 conn(0x7f69100999a0 msgr2=0x7f6910099da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:53.340 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.330+0000 7f690f59e700 1 -- 172.21.15.8:0/2210259871 shutdown_connections 2023-10-05T19:39:53.340 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.330+0000 7f690f59e700 1 -- 172.21.15.8:0/2210259871 wait complete. 2023-10-05T19:39:53.341 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.334+0000 7f690f59e700 1 Processor -- start 2023-10-05T19:39:53.341 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.334+0000 7f690f59e700 1 -- start start 2023-10-05T19:39:53.341 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.334+0000 7f690f59e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6910125f70 0x7f6910138050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:53.341 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.334+0000 7f690f59e700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f691011d1c0 con 0x7f6910125f70 2023-10-05T19:39:53.341 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.334+0000 7f690e59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6910125f70 0x7f6910138050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:53.342 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.334+0000 7f690e59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6910125f70 0x7f6910138050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38554/0 (socket says 172.21.15.8:38554) 2023-10-05T19:39:53.342 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.334+0000 7f690e59c700 1 -- 172.21.15.8:0/919285457 learned_addr learned my addr 172.21.15.8:0/919285457 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-05T19:39:53.343 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.334+0000 7f690e59c700 1 -- 172.21.15.8:0/919285457 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6900009420 con 0x7f6910125f70 2023-10-05T19:39:53.343 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.334+0000 7f690e59c700 1 --2- 172.21.15.8:0/919285457 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6910125f70 0x7f6910138050 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6900000c00 tx=0x7f6900016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:53.343 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.334+0000 7f69067fc700 1 -- 172.21.15.8:0/919285457 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6900011900 con 0x7f6910125f70 2023-10-05T19:39:53.343 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.334+0000 7f69067fc700 1 -- 172.21.15.8:0/919285457 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f690001f070 con 0x7f6910125f70 2023-10-05T19:39:53.343 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.334+0000 7f690f59e700 1 -- 172.21.15.8:0/919285457 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f691011ce40 con 0x7f6910125f70 2023-10-05T19:39:53.343 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.334+0000 7f69067fc700 1 -- 172.21.15.8:0/919285457 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6900026480 con 0x7f6910125f70 2023-10-05T19:39:53.343 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.334+0000 7f690f59e700 1 -- 172.21.15.8:0/919285457 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6910117940 con 0x7f6910125f70 2023-10-05T19:39:53.344 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.338+0000 7f69067fc700 1 -- 172.21.15.8:0/919285457 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f69000265e0 con 0x7f6910125f70 2023-10-05T19:39:53.345 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.338+0000 7f69067fc700 1 --2- 172.21.15.8:0/919285457 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f68f00323e0 0x7f68f0034890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:53.345 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.338+0000 7f690f59e700 1 -- 172.21.15.8:0/919285457 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f691005ecb0 con 0x7f6910125f70 2023-10-05T19:39:53.345 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.338+0000 7f69067fc700 1 -- 172.21.15.8:0/919285457 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 3478+0+0 (secure 0 0 0) 0x7f6900021070 con 0x7f6910125f70 2023-10-05T19:39:53.345 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.338+0000 7f690dd9b700 1 --2- 172.21.15.8:0/919285457 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f68f00323e0 0x7f68f0034890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:53.346 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.338+0000 7f690dd9b700 1 --2- 172.21.15.8:0/919285457 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f68f00323e0 0x7f68f0034890 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f68f80058d0 tx=0x7f68f80079d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:53.352 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.346+0000 7f69067fc700 1 -- 172.21.15.8:0/919285457 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f6900025c60 con 0x7f6910125f70 2023-10-05T19:39:53.516 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.510+0000 7f690f59e700 1 -- 172.21.15.8:0/919285457 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f691004b5c0 con 0x7f6910125f70 2023-10-05T19:39:53.518 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.510+0000 7f69067fc700 1 -- 172.21.15.8:0/919285457 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+8943 (secure 0 0 0) 0x7f6900005a00 con 0x7f6910125f70 2023-10-05T19:39:53.518 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-05T19:39:53.519 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":14,"fsid":"dac6c186-0634-45d9-839d-04f1e484462d","created":"2023-10-05T19:39:35.575770+0000","modified":"2023-10-05T19:39:52.559370+0000","last_up_change":"2023-10-05T19:39:44.877343+0000","last_in_change":"2023-10-05T19:39:36.620941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-05T19:39:45.900259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-05T19:39:51.178391+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c23c6d99-5a1e-499d-8ef8-e1a58dcd9e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6800","nonce":27563},{"type":"v1","addr":"172.21.15.8:6801","nonce":27563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":27563},{"type":"v1","addr":"172.21.15.8:6803","nonce":27563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":27563},{"type":"v1","addr":"172.21.15.8:6807","nonce":27563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":27563},{"type":"v1","addr":"172.21.15.8:6805","nonce":27563}]},"public_addr":"172.21.15.8:6801/27563","cluster_addr":"172.21.15.8:6803/27563","heartbeat_back_addr":"172.21.15.8:6807/27563","heartbeat_front_addr":"172.21.15.8:6805/27563","state":["exists","up"]},{"osd":1,"uuid":"e578b376-27a7-47e2-b262-9a2a703d2504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":29933},{"type":"v1","addr":"172.21.15.12:6809","nonce":29933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":29933},{"type":"v1","addr":"172.21.15.12:6811","nonce":29933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":29933},{"type":"v1","addr":"172.21.15.12:6815","nonce":29933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":29933},{"type":"v1","addr":"172.21.15.12:6813","nonce":29933}]},"public_addr":"172.21.15.12:6809/29933","cluster_addr":"172.21.15.12:6811/29933","heartbeat_back_addr":"172.21.15.12:6815/29933","heartbeat_front_addr":"172.21.15.12:6813/29933","state":["exists","up"]},{"osd":2,"uuid":"f687bf28-3a5f-4cd8-a242-9b95067cc77a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":29934},{"type":"v1","addr":"172.21.15.12:6801","nonce":29934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":29934},{"type":"v1","addr":"172.21.15.12:6803","nonce":29934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":29934},{"type":"v1","addr":"172.21.15.12:6807","nonce":29934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":29934},{"type":"v1","addr":"172.21.15.12:6805","nonce":29934}]},"public_addr":"172.21.15.12:6801/29934","cluster_addr":"172.21.15.12:6803/29934","heartbeat_back_addr":"172.21.15.12:6807/29934","heartbeat_front_addr":"172.21.15.12:6805/29934","state":["exists","up"]},{"osd":3,"uuid":"db25fbc8-9f15-4a87-8310-fcc69bb92fa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":29951},{"type":"v1","addr":"172.21.15.12:6817","nonce":29951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":29951},{"type":"v1","addr":"172.21.15.12:6819","nonce":29951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":29951},{"type":"v1","addr":"172.21.15.12:6823","nonce":29951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":29951},{"type":"v1","addr":"172.21.15.12:6821","nonce":29951}]},"public_addr":"172.21.15.12:6817/29951","cluster_addr":"172.21.15.12:6819/29951","heartbeat_back_addr":"172.21.15.12:6823/29951","heartbeat_front_addr":"172.21.15.12:6821/29951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-05T19:39:41.740161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-05T19:39:42.128798+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-05T19:39:42.140705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-05T19:39:41.961354+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-05T19:39:53.520 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.510+0000 7f690f59e700 1 -- 172.21.15.8:0/919285457 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f68f00323e0 msgr2=0x7f68f0034890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:53.520 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.510+0000 7f690f59e700 1 --2- 172.21.15.8:0/919285457 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f68f00323e0 0x7f68f0034890 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f68f80058d0 tx=0x7f68f80079d0 comp rx=0 tx=0).stop 2023-10-05T19:39:53.520 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.514+0000 7f690f59e700 1 -- 172.21.15.8:0/919285457 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6910125f70 msgr2=0x7f6910138050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:53.521 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.514+0000 7f690f59e700 1 --2- 172.21.15.8:0/919285457 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6910125f70 0x7f6910138050 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6900000c00 tx=0x7f6900016000 comp rx=0 tx=0).stop 2023-10-05T19:39:53.521 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.514+0000 7f690f59e700 1 -- 172.21.15.8:0/919285457 shutdown_connections 2023-10-05T19:39:53.521 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.514+0000 7f690f59e700 1 --2- 172.21.15.8:0/919285457 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f68f00323e0 0x7f68f0034890 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:53.521 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.514+0000 7f690f59e700 1 --2- 172.21.15.8:0/919285457 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6910125f70 0x7f6910138050 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:53.521 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.514+0000 7f690f59e700 1 -- 172.21.15.8:0/919285457 >> 172.21.15.8:0/919285457 conn(0x7f69100999a0 msgr2=0x7f691010fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:53.521 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.514+0000 7f690f59e700 1 -- 172.21.15.8:0/919285457 shutdown_connections 2023-10-05T19:39:53.521 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.514+0000 7f690f59e700 1 -- 172.21.15.8:0/919285457 wait complete. 2023-10-05T19:39:53.532 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-05T19:39:53.533 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-05T19:39:53.533 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-05T19:39:53.534 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-05T19:39:53.634 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.626+0000 7f8797462700 1 Processor -- start 2023-10-05T19:39:53.634 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.626+0000 7f8797462700 1 -- start start 2023-10-05T19:39:53.635 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.626+0000 7f8797462700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f8790095ff0 0x7f87900963c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:53.635 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.626+0000 7f58e2e85700 1 Processor -- start 2023-10-05T19:39:53.636 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.626+0000 7f8797462700 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8790092760 con 0x7f87900905f0 2023-10-05T19:39:53.636 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.626+0000 7f8797462700 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87900918a0 con 0x7f8790095ff0 2023-10-05T19:39:53.636 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.626+0000 7f8796460700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f8790095ff0 0x7f87900963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:53.636 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.626+0000 7f8795c5f700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f87900905f0 0x7f87900909c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:42702/0 (socket says 172.21.15.8:42702) 2023-10-05T19:39:53.636 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.626+0000 7f8795c5f700 1 -- 172.21.15.8:0/3817606259 learned_addr learned my addr 172.21.15.8:0/3817606259 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-05T19:39:53.637 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.626+0000 7f58e2e85700 1 -- start start 2023-10-05T19:39:53.637 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.630+0000 7f58e2e85700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f58dc06a760 0x7f58dc06ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:53.637 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.630+0000 7f8796460700 1 -- 172.21.15.8:0/3817606259 >> v1:172.21.15.8:6789/0 conn(0x7f87900905f0 legacy=0x7f87900909c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:53.637 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.630+0000 7f8796460700 1 -- 172.21.15.8:0/3817606259 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8790093ba0 con 0x7f8790095ff0 2023-10-05T19:39:53.637 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.630+0000 7f58e2e85700 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58dc096b40 con 0x7f58dc06b070 2023-10-05T19:39:53.638 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.630+0000 7f58e2e85700 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58dc06ed00 con 0x7f58dc06a760 2023-10-05T19:39:53.638 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.630+0000 7f8796460700 1 --2- 172.21.15.8:0/3817606259 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8790095ff0 0x7f87900963c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8788009de0 tx=0x7f8788009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f390ee9a1201927d server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:53.638 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.630+0000 7f58e0c21700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f58dc06a760 0x7f58dc06ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:53.639 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.630+0000 7f58dbfff700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f58dc06b070 0x7f58dc11fcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:42712/0 (socket says 172.21.15.8:42712) 2023-10-05T19:39:53.639 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.630+0000 7f58dbfff700 1 -- 172.21.15.8:0/632158739 learned_addr learned my addr 172.21.15.8:0/632158739 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-05T19:39:53.639 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.630+0000 7f58e0c21700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f58dc06a760 0x7f58dc06ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38576/0 (socket says 172.21.15.8:38576) 2023-10-05T19:39:53.639 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.630+0000 7f879545e700 1 -- 172.21.15.8:0/3817606259 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f878800e070 con 0x7f8790095ff0 2023-10-05T19:39:53.639 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.630+0000 7f879545e700 1 -- 172.21.15.8:0/3817606259 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f878800bc80 con 0x7f8790095ff0 2023-10-05T19:39:53.639 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.630+0000 7f879545e700 1 -- 172.21.15.8:0/3817606259 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8788012430 con 0x7f8790095ff0 2023-10-05T19:39:53.640 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.630+0000 7f58db7fe700 1 -- 172.21.15.8:0/632158739 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2480539032 0 0) 0x7f58dc096b40 con 0x7f58dc06b070 2023-10-05T19:39:53.641 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.630+0000 7f58db7fe700 1 -- 172.21.15.8:0/632158739 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58c4003570 con 0x7f58dc06b070 2023-10-05T19:39:53.641 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.630+0000 7f58db7fe700 1 -- 172.21.15.8:0/632158739 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 790868248 0 0) 0x7f58c4003570 con 0x7f58dc06b070 2023-10-05T19:39:53.641 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.630+0000 7f58db7fe700 1 -- 172.21.15.8:0/632158739 >> v2:172.21.15.8:3300/0 conn(0x7f58dc06a760 msgr2=0x7f58dc06ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-05T19:39:53.641 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.630+0000 7f58db7fe700 1 --2- 172.21.15.8:0/632158739 >> v2:172.21.15.8:3300/0 conn(0x7f58dc06a760 0x7f58dc06ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:53.641 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.630+0000 7f58db7fe700 1 -- 172.21.15.8:0/632158739 --> v1:172.21.15.8:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58dc06c990 con 0x7f58dc06b070 2023-10-05T19:39:53.641 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.630+0000 7f58db7fe700 1 -- 172.21.15.8:0/632158739 <== mon.0 v1:172.21.15.8:6789/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (unknown 1630025266 0 0) 0x7f58d00015b0 con 0x7f58dc06b070 2023-10-05T19:39:53.641 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.630+0000 7f58db7fe700 1 -- 172.21.15.8:0/632158739 >> v1:172.21.15.8:6789/0 conn(0x7f58dc06b070 legacy=0x7f58dc11fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:53.641 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.630+0000 7f58db7fe700 1 --2- 172.21.15.8:0/632158739 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f58c4003ae0 0x7f58c4003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:53.641 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.630+0000 7f58db7fe700 1 -- 172.21.15.8:0/632158739 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc06ed00 con 0x7f58c4003ae0 2023-10-05T19:39:53.642 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.630+0000 7f8797462700 1 -- 172.21.15.8:0/3817606259 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8790095ff0 msgr2=0x7f87900963c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:53.642 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.630+0000 7f8797462700 1 --2- 172.21.15.8:0/3817606259 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8790095ff0 0x7f87900963c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8788009de0 tx=0x7f8788009780 comp rx=0 tx=0).stop 2023-10-05T19:39:53.642 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.630+0000 7f58e0c21700 1 --2- 172.21.15.8:0/632158739 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f58c4003ae0 0x7f58c4003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:53.643 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.630+0000 7f8797462700 1 -- 172.21.15.8:0/3817606259 shutdown_connections 2023-10-05T19:39:53.643 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.630+0000 7f8797462700 1 --2- 172.21.15.8:0/3817606259 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8790095ff0 0x7f87900963c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:53.643 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.630+0000 7f8797462700 1 -- 172.21.15.8:0/3817606259 >> 172.21.15.8:0/3817606259 conn(0x7f879009c5d0 msgr2=0x7f879009c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:53.643 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.630+0000 7f8797462700 1 -- 172.21.15.8:0/3817606259 shutdown_connections 2023-10-05T19:39:53.643 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.630+0000 7f8797462700 1 -- 172.21.15.8:0/3817606259 wait complete. 2023-10-05T19:39:53.644 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.630+0000 7f58e0c21700 1 -- 172.21.15.8:0/632158739 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c4003570 con 0x7f58c4003ae0 2023-10-05T19:39:53.644 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.630+0000 7f58e0c21700 1 --2- 172.21.15.8:0/632158739 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f58c4003ae0 0x7f58c4003ed0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f58cc00e140 tx=0x7f58cc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2626a91c4fb6e086 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:53.644 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.630+0000 7f58db7fe700 1 -- 172.21.15.8:0/632158739 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58cc00ea20 con 0x7f58c4003ae0 2023-10-05T19:39:53.644 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.630+0000 7f58db7fe700 1 -- 172.21.15.8:0/632158739 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58cc00f040 con 0x7f58c4003ae0 2023-10-05T19:39:53.644 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.630+0000 7f58db7fe700 1 -- 172.21.15.8:0/632158739 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58cc00eba0 con 0x7f58c4003ae0 2023-10-05T19:39:53.644 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.630+0000 7f8797462700 1 Processor -- start 2023-10-05T19:39:53.644 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.630+0000 7f8797462700 1 -- start start 2023-10-05T19:39:53.645 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.630+0000 7f58e2e85700 1 -- 172.21.15.8:0/632158739 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f58c4003ae0 msgr2=0x7f58c4003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:53.645 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.630+0000 7f58e2e85700 1 --2- 172.21.15.8:0/632158739 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f58c4003ae0 0x7f58c4003ed0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f58cc00e140 tx=0x7f58cc00b040 comp rx=0 tx=0).stop 2023-10-05T19:39:53.645 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.630+0000 7f8797462700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87900905f0 0x7f879006b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:53.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.630+0000 7f8797462700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879006cbd0 con 0x7f87900905f0 2023-10-05T19:39:53.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.634+0000 7f8796460700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87900905f0 0x7f879006b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:53.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.634+0000 7f8796460700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87900905f0 0x7f879006b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38586/0 (socket says 172.21.15.8:38586) 2023-10-05T19:39:53.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.634+0000 7f8796460700 1 -- 172.21.15.8:0/693742833 learned_addr learned my addr 172.21.15.8:0/693742833 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-05T19:39:53.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.634+0000 7f58e2e85700 1 -- 172.21.15.8:0/632158739 shutdown_connections 2023-10-05T19:39:53.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.634+0000 7f58e2e85700 1 --2- 172.21.15.8:0/632158739 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f58c4003ae0 0x7f58c4003ed0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:53.647 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.634+0000 7f58e2e85700 1 --2- 172.21.15.8:0/632158739 >> v2:172.21.15.8:3300/0 conn(0x7f58dc06a760 0x7f58dc06ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:53.647 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.634+0000 7f58e2e85700 1 -- 172.21.15.8:0/632158739 >> 172.21.15.8:0/632158739 conn(0x7f58dc09a0a0 msgr2=0x7f58dc0949f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:53.648 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.634+0000 7f8796460700 1 -- 172.21.15.8:0/693742833 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8788009a20 con 0x7f87900905f0 2023-10-05T19:39:53.648 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.634+0000 7f8796460700 1 --2- 172.21.15.8:0/693742833 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87900905f0 0x7f879006b1a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f878800ba40 tx=0x7f8788018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:53.648 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.634+0000 7f87867fc700 1 -- 172.21.15.8:0/693742833 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8788027470 con 0x7f87900905f0 2023-10-05T19:39:53.649 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.634+0000 7f87867fc700 1 -- 172.21.15.8:0/693742833 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87880275d0 con 0x7f87900905f0 2023-10-05T19:39:53.649 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.634+0000 7f87867fc700 1 -- 172.21.15.8:0/693742833 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f87880278a0 con 0x7f87900905f0 2023-10-05T19:39:53.649 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.634+0000 7f8797462700 1 -- 172.21.15.8:0/693742833 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8790095c10 con 0x7f87900905f0 2023-10-05T19:39:53.649 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.634+0000 7f8797462700 1 -- 172.21.15.8:0/693742833 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f879006eb90 con 0x7f87900905f0 2023-10-05T19:39:53.649 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.634+0000 7f8797462700 1 -- 172.21.15.8:0/693742833 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f879005ecb0 con 0x7f87900905f0 2023-10-05T19:39:53.649 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.634+0000 7f87867fc700 1 -- 172.21.15.8:0/693742833 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f8788024070 con 0x7f87900905f0 2023-10-05T19:39:53.649 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.634+0000 7f87867fc700 1 --2- 172.21.15.8:0/693742833 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f87740323d0 0x7f8774034880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:53.649 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.634+0000 7f87867fc700 1 -- 172.21.15.8:0/693742833 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3478+0+0 (secure 0 0 0) 0x7f87880461c0 con 0x7f87900905f0 2023-10-05T19:39:53.649 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.634+0000 7f87867fc700 1 --2- 172.21.15.8:0/693742833 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f8774037ae0 0x7f8774039f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:53.649 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.634+0000 7f87867fc700 1 -- 172.21.15.8:0/693742833 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f877403a5e0 con 0x7f8774037ae0 2023-10-05T19:39:53.649 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.634+0000 7f87867fc700 1 -- 172.21.15.8:0/693742833 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87880487d0 con 0x7f87900905f0 2023-10-05T19:39:53.649 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.634+0000 7f8795c5f700 1 --2- 172.21.15.8:0/693742833 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f87740323d0 0x7f8774034880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:53.650 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.634+0000 7f8795c5f700 1 --2- 172.21.15.8:0/693742833 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f87740323d0 0x7f8774034880 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8780007fd0 tx=0x7f8780007970 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:53.650 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.638+0000 7f58e2e85700 1 -- 172.21.15.8:0/632158739 shutdown_connections 2023-10-05T19:39:53.650 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.638+0000 7f58e2e85700 1 -- 172.21.15.8:0/632158739 wait complete. 2023-10-05T19:39:53.650 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.638+0000 7f8796c61700 1 --2- 172.21.15.8:0/693742833 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f8774037ae0 0x7f8774039f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:53.651 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.638+0000 7f8796c61700 1 --2- 172.21.15.8:0/693742833 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f8774037ae0 0x7f8774039f10 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:53.651 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.638+0000 7f58e2e85700 1 Processor -- start 2023-10-05T19:39:53.651 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.638+0000 7f58e2e85700 1 -- start start 2023-10-05T19:39:53.651 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.638+0000 7f87867fc700 1 -- 172.21.15.8:0/693742833 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f877403a5e0 con 0x7f8774037ae0 2023-10-05T19:39:53.654 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.638+0000 7f58e2e85700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f58c4003ae0 0x7f58dc12c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:53.654 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.638+0000 7f58e2e85700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc1234f0 con 0x7f58c4003ae0 2023-10-05T19:39:53.654 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.638+0000 7f58e0c21700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f58c4003ae0 0x7f58dc12c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:53.654 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.638+0000 7f58e0c21700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f58c4003ae0 0x7f58dc12c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38590/0 (socket says 172.21.15.8:38590) 2023-10-05T19:39:53.654 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.638+0000 7f58e0c21700 1 -- 172.21.15.8:0/2371783366 learned_addr learned my addr 172.21.15.8:0/2371783366 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-05T19:39:53.654 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.638+0000 7f58e0c21700 1 -- 172.21.15.8:0/2371783366 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58dc124c70 con 0x7f58c4003ae0 2023-10-05T19:39:53.654 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.638+0000 7f58e0c21700 1 --2- 172.21.15.8:0/2371783366 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f58c4003ae0 0x7f58dc12c980 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f58cc00e7e0 tx=0x7f58cc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:53.654 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.638+0000 7f58d8ff9700 1 -- 172.21.15.8:0/2371783366 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58cc023070 con 0x7f58c4003ae0 2023-10-05T19:39:53.654 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.638+0000 7f58d8ff9700 1 -- 172.21.15.8:0/2371783366 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58cc00ee70 con 0x7f58c4003ae0 2023-10-05T19:39:53.654 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.638+0000 7f58e2e85700 1 -- 172.21.15.8:0/2371783366 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58dc124830 con 0x7f58c4003ae0 2023-10-05T19:39:53.655 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.638+0000 7f58d8ff9700 1 -- 172.21.15.8:0/2371783366 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58cc01d7a0 con 0x7f58c4003ae0 2023-10-05T19:39:53.655 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.638+0000 7f58e2e85700 1 -- 172.21.15.8:0/2371783366 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58dc121180 con 0x7f58c4003ae0 2023-10-05T19:39:53.655 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.642+0000 7f58e2e85700 1 -- 172.21.15.8:0/2371783366 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f58dc124a80 con 0x7f58c4003ae0 2023-10-05T19:39:53.655 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.642+0000 7f58d8ff9700 1 -- 172.21.15.8:0/2371783366 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f58cc003020 con 0x7f58c4003ae0 2023-10-05T19:39:53.655 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.642+0000 7f58d8ff9700 1 --2- 172.21.15.8:0/2371783366 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f58c80323e0 0x7f58c8034890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:53.655 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.642+0000 7f58d8ff9700 1 -- 172.21.15.8:0/2371783366 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3478+0+0 (secure 0 0 0) 0x7f58cc045590 con 0x7f58c4003ae0 2023-10-05T19:39:53.655 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.642+0000 7f58d8ff9700 1 --2- 172.21.15.8:0/2371783366 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x7f58c8037af0 0x7f58c8039f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:53.655 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.642+0000 7f58d8ff9700 1 -- 172.21.15.8:0/2371783366 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f58c803a5f0 con 0x7f58c8037af0 2023-10-05T19:39:53.655 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.642+0000 7f58d8ff9700 1 -- 172.21.15.8:0/2371783366 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58cc047ba0 con 0x7f58c4003ae0 2023-10-05T19:39:53.655 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.642+0000 7f58dbfff700 1 --2- 172.21.15.8:0/2371783366 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f58c80323e0 0x7f58c8034890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:53.655 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.642+0000 7f58e1422700 1 --2- 172.21.15.8:0/2371783366 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x7f58c8037af0 0x7f58c8039f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:53.656 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.642+0000 7f58dbfff700 1 --2- 172.21.15.8:0/2371783366 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f58c80323e0 0x7f58c8034890 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f58d0001760 tx=0x7f58d0001300 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:53.656 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.642+0000 7f58e1422700 1 --2- 172.21.15.8:0/2371783366 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x7f58c8037af0 0x7f58c8039f20 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:53.656 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.642+0000 7f58d8ff9700 1 -- 172.21.15.8:0/2371783366 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f58c803a5f0 con 0x7f58c8037af0 2023-10-05T19:39:53.662 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.654+0000 7f8797462700 1 -- 172.21.15.8:0/693742833 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8790091b10 con 0x7f8774037ae0 2023-10-05T19:39:53.663 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.654+0000 7f17d2e85700 1 Processor -- start 2023-10-05T19:39:53.663 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.654+0000 7f17d2e85700 1 -- start start 2023-10-05T19:39:53.663 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.654+0000 7f87867fc700 1 -- 172.21.15.8:0/693742833 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8790091b10 con 0x7f8774037ae0 2023-10-05T19:39:53.664 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.654+0000 7f8797462700 1 -- 172.21.15.8:0/693742833 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f8774037ae0 msgr2=0x7f8774039f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:53.664 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.654+0000 7f8797462700 1 --2- 172.21.15.8:0/693742833 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f8774037ae0 0x7f8774039f10 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:53.664 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.654+0000 7f17d2e85700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f17cc126c40 0x7f17cc12b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:53.664 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.654+0000 7f17d2e85700 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17cc06df90 con 0x7f17cc126160 2023-10-05T19:39:53.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.654+0000 7f17d2e85700 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17cc06e5e0 con 0x7f17cc126c40 2023-10-05T19:39:53.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.654+0000 7f17cbfff700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f17cc126c40 0x7f17cc12b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:53.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.654+0000 7f8797462700 1 -- 172.21.15.8:0/693742833 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f87740323d0 msgr2=0x7f8774034880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:53.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.654+0000 7f8797462700 1 --2- 172.21.15.8:0/693742833 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f87740323d0 0x7f8774034880 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8780007fd0 tx=0x7f8780007970 comp rx=0 tx=0).stop 2023-10-05T19:39:53.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.654+0000 7f8797462700 1 -- 172.21.15.8:0/693742833 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87900905f0 msgr2=0x7f879006b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:53.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.654+0000 7f8797462700 1 --2- 172.21.15.8:0/693742833 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87900905f0 0x7f879006b1a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f878800ba40 tx=0x7f8788018000 comp rx=0 tx=0).stop 2023-10-05T19:39:53.666 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.654+0000 7f17d0c21700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f17cc126160 0x7f17cc126530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:42724/0 (socket says 172.21.15.8:42724) 2023-10-05T19:39:53.666 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.658+0000 7f17d0c21700 1 -- 172.21.15.8:0/4256360672 learned_addr learned my addr 172.21.15.8:0/4256360672 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-05T19:39:53.666 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.658+0000 7f8797462700 1 -- 172.21.15.8:0/693742833 shutdown_connections 2023-10-05T19:39:53.667 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.658+0000 7f8797462700 1 --2- 172.21.15.8:0/693742833 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f8774037ae0 0x7f8774039f10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:53.667 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.658+0000 7f8797462700 1 --2- 172.21.15.8:0/693742833 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f87740323d0 0x7f8774034880 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:53.667 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.658+0000 7f8797462700 1 --2- 172.21.15.8:0/693742833 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87900905f0 0x7f879006b1a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:53.667 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.658+0000 7f8797462700 1 -- 172.21.15.8:0/693742833 >> 172.21.15.8:0/693742833 conn(0x7f879009c5d0 msgr2=0x7f87900949e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:53.667 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.658+0000 7f8797462700 1 -- 172.21.15.8:0/693742833 shutdown_connections 2023-10-05T19:39:53.667 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.658+0000 7f8797462700 1 -- 172.21.15.8:0/693742833 wait complete. 2023-10-05T19:39:53.669 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.658+0000 7f17cbfff700 1 -- 172.21.15.8:0/4256360672 >> v1:172.21.15.8:6789/0 conn(0x7f17cc126160 legacy=0x7f17cc126530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:53.669 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.658+0000 7f17cb7fe700 1 -- 172.21.15.8:0/4256360672 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2783341775 0 0) 0x7f17cc06df90 con 0x7f17cc126160 2023-10-05T19:39:53.669 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.658+0000 7f17cbfff700 1 -- 172.21.15.8:0/4256360672 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17cc06a800 con 0x7f17cc126c40 2023-10-05T19:39:53.669 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.658+0000 7f17cbfff700 1 --2- 172.21.15.8:0/4256360672 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f17cc126c40 0x7f17cc12b6c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f17b8009860 tx=0x7f17b80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0f38476a32fa00a server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:53.669 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.658+0000 7f17cb7fe700 1 -- 172.21.15.8:0/4256360672 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f17b800b680 con 0x7f17cc126c40 2023-10-05T19:39:53.669 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.658+0000 7f17cb7fe700 1 -- 172.21.15.8:0/4256360672 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b800b7e0 con 0x7f17cc126c40 2023-10-05T19:39:53.669 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.658+0000 7f17cb7fe700 1 -- 172.21.15.8:0/4256360672 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f17b800b990 con 0x7f17cc126c40 2023-10-05T19:39:53.669 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.658+0000 7f17d2e85700 1 -- 172.21.15.8:0/4256360672 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f17cc126c40 msgr2=0x7f17cc12b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:53.669 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.658+0000 7f17d2e85700 1 --2- 172.21.15.8:0/4256360672 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f17cc126c40 0x7f17cc12b6c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f17b8009860 tx=0x7f17b80091a0 comp rx=0 tx=0).stop 2023-10-05T19:39:53.669 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.658+0000 7f17d2e85700 1 -- 172.21.15.8:0/4256360672 shutdown_connections 2023-10-05T19:39:53.670 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.658+0000 7f17d2e85700 1 --2- 172.21.15.8:0/4256360672 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f17cc126c40 0x7f17cc12b6c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:53.670 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.658+0000 7f17d2e85700 1 -- 172.21.15.8:0/4256360672 >> 172.21.15.8:0/4256360672 conn(0x7f17cc098f60 msgr2=0x7f17cc0949f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:53.670 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.658+0000 7f17d2e85700 1 -- 172.21.15.8:0/4256360672 shutdown_connections 2023-10-05T19:39:53.670 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.658+0000 7f17d2e85700 1 -- 172.21.15.8:0/4256360672 wait complete. 2023-10-05T19:39:53.670 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.658+0000 7f17d2e85700 1 Processor -- start 2023-10-05T19:39:53.670 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.658+0000 7f17d2e85700 1 -- start start 2023-10-05T19:39:53.670 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.658+0000 7f17d2e85700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f17cc126160 0x7f17cc136370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:53.670 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.658+0000 7f17d2e85700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc097ec0 con 0x7f17cc126160 2023-10-05T19:39:53.670 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.658+0000 7f17d0c21700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f17cc126160 0x7f17cc136370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:53.670 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.658+0000 7f17d0c21700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f17cc126160 0x7f17cc136370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38600/0 (socket says 172.21.15.8:38600) 2023-10-05T19:39:53.670 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.658+0000 7f17d0c21700 1 -- 172.21.15.8:0/2455730059 learned_addr learned my addr 172.21.15.8:0/2455730059 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-05T19:39:53.671 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.658+0000 7f17d0c21700 1 -- 172.21.15.8:0/2455730059 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b8009420 con 0x7f17cc126160 2023-10-05T19:39:53.671 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f58e2e85700 1 -- 172.21.15.8:0/2371783366 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f58dc124e00 con 0x7f58c8037af0 2023-10-05T19:39:53.671 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f17d0c21700 1 --2- 172.21.15.8:0/2455730059 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f17cc126160 0x7f17cc136370 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f17c000c460 tx=0x7f17c0000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:53.671 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f17c8ff9700 1 -- 172.21.15.8:0/2455730059 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f17c0005b30 con 0x7f17cc126160 2023-10-05T19:39:53.672 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f17c8ff9700 1 -- 172.21.15.8:0/2455730059 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17c0005c90 con 0x7f17cc126160 2023-10-05T19:39:53.672 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f17d2e85700 1 -- 172.21.15.8:0/2455730059 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17cc116ce0 con 0x7f17cc126160 2023-10-05T19:39:53.672 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f17c8ff9700 1 -- 172.21.15.8:0/2455730059 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f17c00070d0 con 0x7f17cc126160 2023-10-05T19:39:53.672 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f17d2e85700 1 -- 172.21.15.8:0/2455730059 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17cc1174d0 con 0x7f17cc126160 2023-10-05T19:39:53.672 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f58d8ff9700 1 -- 172.21.15.8:0/2371783366 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f58d4009900 con 0x7f58c8037af0 2023-10-05T19:39:53.673 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f19ad616700 1 Processor -- start 2023-10-05T19:39:53.673 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f19ad616700 1 -- start start 2023-10-05T19:39:53.673 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f58e2e85700 1 -- 172.21.15.8:0/2371783366 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x7f58c8037af0 msgr2=0x7f58c8039f20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:53.674 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f58e2e85700 1 --2- 172.21.15.8:0/2371783366 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x7f58c8037af0 0x7f58c8039f20 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:53.674 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f58e2e85700 1 -- 172.21.15.8:0/2371783366 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f58c80323e0 msgr2=0x7f58c8034890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:53.674 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f58e2e85700 1 --2- 172.21.15.8:0/2371783366 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f58c80323e0 0x7f58c8034890 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f58d0001760 tx=0x7f58d0001300 comp rx=0 tx=0).stop 2023-10-05T19:39:53.674 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f58e2e85700 1 -- 172.21.15.8:0/2371783366 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f58c4003ae0 msgr2=0x7f58dc12c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:53.674 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f58e2e85700 1 --2- 172.21.15.8:0/2371783366 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f58c4003ae0 0x7f58dc12c980 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f58cc00e7e0 tx=0x7f58cc019000 comp rx=0 tx=0).stop 2023-10-05T19:39:53.674 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f17c8ff9700 1 -- 172.21.15.8:0/2455730059 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f17c0007300 con 0x7f17cc126160 2023-10-05T19:39:53.675 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f19ad616700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f19a808ec70 0x7f19a808fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:53.675 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f19ad616700 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19a8070640 con 0x7f19a808e8a0 2023-10-05T19:39:53.675 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f19ad616700 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19a8091a80 con 0x7f19a808ec70 2023-10-05T19:39:53.675 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f19a7fff700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f19a808e8a0 0x7f19a808f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:42734/0 (socket says 172.21.15.8:42734) 2023-10-05T19:39:53.675 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f19a7fff700 1 -- 172.21.15.8:0/4216222458 learned_addr learned my addr 172.21.15.8:0/4216222458 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-05T19:39:53.675 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f17c8ff9700 1 --2- 172.21.15.8:0/2455730059 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f17b0032380 0x7f17b0034830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:53.676 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f199ffff700 1 --2- 172.21.15.8:0/4216222458 >> v2:172.21.15.8:3300/0 conn(0x7f19a808ec70 0x7f19a808fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:53.676 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f17c8ff9700 1 -- 172.21.15.8:0/2455730059 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3478+0+0 (secure 0 0 0) 0x7f17c0042fd0 con 0x7f17cc126160 2023-10-05T19:39:53.676 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f58e2e85700 1 -- 172.21.15.8:0/2371783366 shutdown_connections 2023-10-05T19:39:53.677 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f58e2e85700 1 --2- 172.21.15.8:0/2371783366 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x7f58c8037af0 0x7f58c8039f20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:53.677 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f58e2e85700 1 --2- 172.21.15.8:0/2371783366 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f58c80323e0 0x7f58c8034890 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:53.677 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f58e2e85700 1 --2- 172.21.15.8:0/2371783366 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f58c4003ae0 0x7f58dc12c980 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:53.677 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f58e2e85700 1 -- 172.21.15.8:0/2371783366 >> 172.21.15.8:0/2371783366 conn(0x7f58dc09a0a0 msgr2=0x7f58dc0949f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:53.677 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f58e2e85700 1 -- 172.21.15.8:0/2371783366 shutdown_connections 2023-10-05T19:39:53.677 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f58e2e85700 1 -- 172.21.15.8:0/2371783366 wait complete. 2023-10-05T19:39:53.677 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f19a77fe700 1 -- 172.21.15.8:0/4216222458 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 353738392 0 0) 0x7f19a8070640 con 0x7f19a808e8a0 2023-10-05T19:39:53.678 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f19a77fe700 1 -- 172.21.15.8:0/4216222458 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1994003570 con 0x7f19a808e8a0 2023-10-05T19:39:53.678 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f17cbfff700 1 --2- 172.21.15.8:0/2455730059 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f17b0032380 0x7f17b0034830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:53.678 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f199ffff700 1 -- 172.21.15.8:0/4216222458 >> v1:172.21.15.8:6789/0 conn(0x7f19a808e8a0 legacy=0x7f19a808f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:53.679 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f199ffff700 1 -- 172.21.15.8:0/4216222458 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a8094b00 con 0x7f19a808ec70 2023-10-05T19:39:53.679 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f199ffff700 1 --2- 172.21.15.8:0/4216222458 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f19a808ec70 0x7f19a808fda0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1998009860 tx=0x7f19980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a2b4775d6157d9c server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:53.679 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f19a77fe700 1 -- 172.21.15.8:0/4216222458 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1998011630 con 0x7f19a808ec70 2023-10-05T19:39:53.679 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f19a77fe700 1 -- 172.21.15.8:0/4216222458 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1998011790 con 0x7f19a808ec70 2023-10-05T19:39:53.679 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f19a77fe700 1 -- 172.21.15.8:0/4216222458 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1998011900 con 0x7f19a808ec70 2023-10-05T19:39:53.679 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f17cbfff700 1 --2- 172.21.15.8:0/2455730059 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f17b0032380 0x7f17b0034830 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f17b8000c00 tx=0x7f17b8016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:53.680 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f19ad616700 1 -- 172.21.15.8:0/4216222458 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f19a808ec70 msgr2=0x7f19a808fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:53.680 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f19ad616700 1 --2- 172.21.15.8:0/4216222458 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f19a808ec70 0x7f19a808fda0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1998009860 tx=0x7f19980091a0 comp rx=0 tx=0).stop 2023-10-05T19:39:53.680 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f19ad616700 1 -- 172.21.15.8:0/4216222458 shutdown_connections 2023-10-05T19:39:53.680 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f19ad616700 1 --2- 172.21.15.8:0/4216222458 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f19a808ec70 0x7f19a808fda0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:53.680 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.662+0000 7f19ad616700 1 -- 172.21.15.8:0/4216222458 >> 172.21.15.8:0/4216222458 conn(0x7f19a80984f0 msgr2=0x7f19a80988f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:53.680 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.666+0000 7f19ad616700 1 -- 172.21.15.8:0/4216222458 shutdown_connections 2023-10-05T19:39:53.681 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.666+0000 7f19ad616700 1 -- 172.21.15.8:0/4216222458 wait complete. 2023-10-05T19:39:53.681 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.666+0000 7f17d2e85700 1 --2- 172.21.15.8:0/2455730059 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x7f17cc126c40 0x7f17cc136dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:53.681 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.666+0000 7f17d2e85700 1 -- 172.21.15.8:0/2455730059 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f17cc117c40 con 0x7f17cc126c40 2023-10-05T19:39:53.681 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.666+0000 7f19ad616700 1 Processor -- start 2023-10-05T19:39:53.681 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.666+0000 7f19ad616700 1 -- start start 2023-10-05T19:39:53.682 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.666+0000 7f17d1422700 1 --2- 172.21.15.8:0/2455730059 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x7f17cc126c40 0x7f17cc136dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:53.682 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.666+0000 7f19ad616700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f19a808e8a0 0x7f19a8138420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:53.683 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.666+0000 7f19ad616700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a8120660 con 0x7f19a808e8a0 2023-10-05T19:39:53.683 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.666+0000 7f19a7fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f19a808e8a0 0x7f19a8138420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:53.683 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.666+0000 7f19a7fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f19a808e8a0 0x7f19a8138420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38610/0 (socket says 172.21.15.8:38610) 2023-10-05T19:39:53.683 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.666+0000 7f19a7fff700 1 -- 172.21.15.8:0/1574835219 learned_addr learned my addr 172.21.15.8:0/1574835219 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-05T19:39:53.683 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.666+0000 7f17d1422700 1 --2- 172.21.15.8:0/2455730059 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x7f17cc126c40 0x7f17cc136dc0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:53.684 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.666+0000 7f19a7fff700 1 -- 172.21.15.8:0/1574835219 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1998009420 con 0x7f19a808e8a0 2023-10-05T19:39:53.684 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.666+0000 7f19a7fff700 1 --2- 172.21.15.8:0/1574835219 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f19a808e8a0 0x7f19a8138420 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f199000c690 tx=0x7f1990007c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:53.684 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.666+0000 7f19a4ff9700 1 -- 172.21.15.8:0/1574835219 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1990007540 con 0x7f19a808e8a0 2023-10-05T19:39:53.684 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.666+0000 7f19a4ff9700 1 -- 172.21.15.8:0/1574835219 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19900076a0 con 0x7f19a808e8a0 2023-10-05T19:39:53.684 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.666+0000 7f19a4ff9700 1 -- 172.21.15.8:0/1574835219 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1990005970 con 0x7f19a808e8a0 2023-10-05T19:39:53.684 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.666+0000 7f19ad616700 1 -- 172.21.15.8:0/1574835219 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19a8135ac0 con 0x7f19a808e8a0 2023-10-05T19:39:53.685 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.666+0000 7f19ad616700 1 -- 172.21.15.8:0/1574835219 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19a8138060 con 0x7f19a808e8a0 2023-10-05T19:39:53.685 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.666+0000 7f19ad616700 1 -- 172.21.15.8:0/1574835219 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f19a805ecb0 con 0x7f19a808e8a0 2023-10-05T19:39:53.685 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.666+0000 7f17c8ff9700 1 -- 172.21.15.8:0/2455730059 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f17cc117c40 con 0x7f17cc126c40 2023-10-05T19:39:53.686 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.670+0000 7f19a4ff9700 1 -- 172.21.15.8:0/1574835219 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f1990005ad0 con 0x7f19a808e8a0 2023-10-05T19:39:53.686 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.670+0000 7f19a4ff9700 1 --2- 172.21.15.8:0/1574835219 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f19880323e0 0x7f1988034890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:53.686 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.670+0000 7f19a4ff9700 1 -- 172.21.15.8:0/1574835219 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3478+0+0 (secure 0 0 0) 0x7f1990043040 con 0x7f19a808e8a0 2023-10-05T19:39:53.686 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.670+0000 7f19a4ff9700 1 --2- 172.21.15.8:0/1574835219 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x7f1988037af0 0x7f1988039f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:53.686 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.670+0000 7f19a4ff9700 1 -- 172.21.15.8:0/1574835219 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f198803a5f0 con 0x7f1988037af0 2023-10-05T19:39:53.686 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.670+0000 7f19a4ff9700 1 -- 172.21.15.8:0/1574835219 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1990045650 con 0x7f19a808e8a0 2023-10-05T19:39:53.687 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.670+0000 7f199ffff700 1 --2- 172.21.15.8:0/1574835219 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f19880323e0 0x7f1988034890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:53.687 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.670+0000 7f19ace15700 1 --2- 172.21.15.8:0/1574835219 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x7f1988037af0 0x7f1988039f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:53.687 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.670+0000 7f19ace15700 1 --2- 172.21.15.8:0/1574835219 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x7f1988037af0 0x7f1988039f20 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:53.687 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.670+0000 7f199ffff700 1 --2- 172.21.15.8:0/1574835219 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f19880323e0 0x7f1988034890 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1998003080 tx=0x7f1998016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:53.687 INFO:teuthology.orchestra.run.smithi008.stdout:38654705668 2023-10-05T19:39:53.688 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-05T19:39:53.688 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.670+0000 7f19a4ff9700 1 -- 172.21.15.8:0/1574835219 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f198803a5f0 con 0x7f1988037af0 2023-10-05T19:39:53.689 INFO:teuthology.orchestra.run.smithi008.stdout:38654705667 2023-10-05T19:39:53.689 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-05T19:39:53.693 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.686+0000 7f17d2e85700 1 -- 172.21.15.8:0/2455730059 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f17cc067260 con 0x7f17cc126c40 2023-10-05T19:39:53.695 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.686+0000 7f17c8ff9700 1 -- 172.21.15.8:0/2455730059 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f17cc067260 con 0x7f17cc126c40 2023-10-05T19:39:53.696 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.686+0000 7f17d2e85700 1 -- 172.21.15.8:0/2455730059 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x7f17cc126c40 msgr2=0x7f17cc136dc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:53.696 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.686+0000 7f17d2e85700 1 --2- 172.21.15.8:0/2455730059 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x7f17cc126c40 0x7f17cc136dc0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:53.696 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.686+0000 7f17d2e85700 1 -- 172.21.15.8:0/2455730059 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f17b0032380 msgr2=0x7f17b0034830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:53.696 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.686+0000 7f17d2e85700 1 --2- 172.21.15.8:0/2455730059 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f17b0032380 0x7f17b0034830 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f17b8000c00 tx=0x7f17b8016000 comp rx=0 tx=0).stop 2023-10-05T19:39:53.696 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.686+0000 7f17d2e85700 1 -- 172.21.15.8:0/2455730059 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f17cc126160 msgr2=0x7f17cc136370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:53.696 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.686+0000 7f17d2e85700 1 --2- 172.21.15.8:0/2455730059 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f17cc126160 0x7f17cc136370 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f17c000c460 tx=0x7f17c0000fe0 comp rx=0 tx=0).stop 2023-10-05T19:39:53.696 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.686+0000 7f17d2e85700 1 -- 172.21.15.8:0/2455730059 shutdown_connections 2023-10-05T19:39:53.696 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.686+0000 7f17d2e85700 1 --2- 172.21.15.8:0/2455730059 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x7f17cc126c40 0x7f17cc136dc0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:53.696 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.686+0000 7f17d2e85700 1 --2- 172.21.15.8:0/2455730059 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f17b0032380 0x7f17b0034830 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:53.697 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.686+0000 7f17d2e85700 1 --2- 172.21.15.8:0/2455730059 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f17cc126160 0x7f17cc136370 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:53.697 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.686+0000 7f17d2e85700 1 -- 172.21.15.8:0/2455730059 >> 172.21.15.8:0/2455730059 conn(0x7f17cc098f60 msgr2=0x7f17cc069da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:53.697 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.686+0000 7f17d2e85700 1 -- 172.21.15.8:0/2455730059 shutdown_connections 2023-10-05T19:39:53.698 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.686+0000 7f17d2e85700 1 -- 172.21.15.8:0/2455730059 wait complete. 2023-10-05T19:39:53.698 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.690+0000 7f19ad616700 1 -- 172.21.15.8:0/1574835219 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f19a808dfe0 con 0x7f1988037af0 2023-10-05T19:39:53.698 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.690+0000 7f19a4ff9700 1 -- 172.21.15.8:0/1574835219 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f19a808dfe0 con 0x7f1988037af0 2023-10-05T19:39:53.699 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.690+0000 7f19ad616700 1 -- 172.21.15.8:0/1574835219 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x7f1988037af0 msgr2=0x7f1988039f20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:53.699 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.690+0000 7f19ad616700 1 --2- 172.21.15.8:0/1574835219 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x7f1988037af0 0x7f1988039f20 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:53.699 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.690+0000 7f19ad616700 1 -- 172.21.15.8:0/1574835219 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f19880323e0 msgr2=0x7f1988034890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:53.699 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.690+0000 7f19ad616700 1 --2- 172.21.15.8:0/1574835219 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f19880323e0 0x7f1988034890 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1998003080 tx=0x7f1998016000 comp rx=0 tx=0).stop 2023-10-05T19:39:53.699 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.690+0000 7f19ad616700 1 -- 172.21.15.8:0/1574835219 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f19a808e8a0 msgr2=0x7f19a8138420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:53.700 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.690+0000 7f19ad616700 1 --2- 172.21.15.8:0/1574835219 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f19a808e8a0 0x7f19a8138420 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f199000c690 tx=0x7f1990007c60 comp rx=0 tx=0).stop 2023-10-05T19:39:53.700 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.690+0000 7f19ad616700 1 -- 172.21.15.8:0/1574835219 shutdown_connections 2023-10-05T19:39:53.700 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.690+0000 7f19ad616700 1 --2- 172.21.15.8:0/1574835219 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x7f1988037af0 0x7f1988039f20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:53.700 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.690+0000 7f19ad616700 1 --2- 172.21.15.8:0/1574835219 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f19880323e0 0x7f1988034890 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:53.700 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.690+0000 7f19ad616700 1 --2- 172.21.15.8:0/1574835219 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f19a808e8a0 0x7f19a8138420 secure :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f199000c690 tx=0x7f1990007c60 comp rx=0 tx=0).stop 2023-10-05T19:39:53.700 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.690+0000 7f19ad616700 1 -- 172.21.15.8:0/1574835219 >> 172.21.15.8:0/1574835219 conn(0x7f19a80984f0 msgr2=0x7f19a808cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:53.700 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.690+0000 7f19ad616700 1 -- 172.21.15.8:0/1574835219 shutdown_connections 2023-10-05T19:39:53.700 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.690+0000 7f19ad616700 1 -- 172.21.15.8:0/1574835219 wait complete. 2023-10-05T19:39:53.707 INFO:teuthology.orchestra.run.smithi008.stdout:38654705668 2023-10-05T19:39:53.708 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-05T19:39:53.710 INFO:teuthology.orchestra.run.smithi008.stdout:38654705668 2023-10-05T19:39:53.711 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-05T19:39:53.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.798+0000 7f3854eec700 1 Processor -- start 2023-10-05T19:39:53.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.802+0000 7f3854eec700 1 -- start start 2023-10-05T19:39:53.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.802+0000 7f3854eec700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f385005dac0 0x7f3850123e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:53.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.802+0000 7f3854eec700 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3850095330 con 0x7f3850094e20 2023-10-05T19:39:53.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.802+0000 7f3854eec700 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f385006da80 con 0x7f385005dac0 2023-10-05T19:39:53.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.802+0000 7f384f7fe700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f385005dac0 0x7f3850123e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:53.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.802+0000 7f384effd700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f3850094e20 0x7f38501243a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:42740/0 (socket says 172.21.15.8:42740) 2023-10-05T19:39:53.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.802+0000 7f384effd700 1 -- 172.21.15.8:0/1915841869 learned_addr learned my addr 172.21.15.8:0/1915841869 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-05T19:39:53.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.802+0000 7f384f7fe700 1 -- 172.21.15.8:0/1915841869 >> v1:172.21.15.8:6789/0 conn(0x7f3850094e20 legacy=0x7f38501243a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:53.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.802+0000 7f384e7fc700 1 -- 172.21.15.8:0/1915841869 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 141095390 0 0) 0x7f3850095330 con 0x7f3850094e20 2023-10-05T19:39:53.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.802+0000 7f384f7fe700 1 -- 172.21.15.8:0/1915841869 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3850124ab0 con 0x7f385005dac0 2023-10-05T19:39:53.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.802+0000 7f384f7fe700 1 --2- 172.21.15.8:0/1915841869 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f385005dac0 0x7f3850123e60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3840009860 tx=0x7f38400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feacf79420b465cc server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:53.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.802+0000 7f384e7fc700 1 -- 172.21.15.8:0/1915841869 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f384000b680 con 0x7f385005dac0 2023-10-05T19:39:53.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.802+0000 7f384e7fc700 1 -- 172.21.15.8:0/1915841869 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f384000b7e0 con 0x7f385005dac0 2023-10-05T19:39:53.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.802+0000 7f384e7fc700 1 -- 172.21.15.8:0/1915841869 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f384000b990 con 0x7f385005dac0 2023-10-05T19:39:53.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.802+0000 7f3854eec700 1 -- 172.21.15.8:0/1915841869 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f385005dac0 msgr2=0x7f3850123e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:53.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.802+0000 7f3854eec700 1 --2- 172.21.15.8:0/1915841869 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f385005dac0 0x7f3850123e60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3840009860 tx=0x7f38400091a0 comp rx=0 tx=0).stop 2023-10-05T19:39:53.812 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.802+0000 7f3854eec700 1 -- 172.21.15.8:0/1915841869 shutdown_connections 2023-10-05T19:39:53.812 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.802+0000 7f3854eec700 1 --2- 172.21.15.8:0/1915841869 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f385005dac0 0x7f3850123e60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:53.812 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.802+0000 7f3854eec700 1 -- 172.21.15.8:0/1915841869 >> 172.21.15.8:0/1915841869 conn(0x7f3850095970 msgr2=0x7f3850095d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:53.812 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.802+0000 7f3854eec700 1 -- 172.21.15.8:0/1915841869 shutdown_connections 2023-10-05T19:39:53.812 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.802+0000 7f3854eec700 1 -- 172.21.15.8:0/1915841869 wait complete. 2023-10-05T19:39:53.812 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.802+0000 7fe1e0fbf700 1 Processor -- start 2023-10-05T19:39:53.813 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.802+0000 7fe1e0fbf700 1 -- start start 2023-10-05T19:39:53.813 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.802+0000 7f3854eec700 1 Processor -- start 2023-10-05T19:39:53.813 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.802+0000 7f3854eec700 1 -- start start 2023-10-05T19:39:53.814 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.806+0000 7fe1e0fbf700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fe1dc0881c0 0x7fe1dc088590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:53.814 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.806+0000 7fe1e0fbf700 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1dc089c80 con 0x7fe1dc088ad0 2023-10-05T19:39:53.814 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.806+0000 7fe1e0fbf700 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1dc06f6c0 con 0x7fe1dc0881c0 2023-10-05T19:39:53.814 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.806+0000 7fe1d9d9b700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7fe1dc088ad0 0x7fe1dc12b6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:42750/0 (socket says 172.21.15.8:42750) 2023-10-05T19:39:53.814 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.806+0000 7fe1d9d9b700 1 -- 172.21.15.8:0/1257498877 learned_addr learned my addr 172.21.15.8:0/1257498877 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-05T19:39:53.814 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.806+0000 7fe1da59c700 1 --2- 172.21.15.8:0/1257498877 >> v2:172.21.15.8:3300/0 conn(0x7fe1dc0881c0 0x7fe1dc088590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:53.815 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.806+0000 7f3854eec700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f385005dac0 0x7f385006b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:53.815 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.806+0000 7f3854eec700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38500971a0 con 0x7f385005dac0 2023-10-05T19:39:53.815 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.806+0000 7fe1da59c700 1 -- 172.21.15.8:0/1257498877 >> v1:172.21.15.8:6789/0 conn(0x7fe1dc088ad0 legacy=0x7fe1dc12b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:53.815 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.806+0000 7fe1da59c700 1 -- 172.21.15.8:0/1257498877 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1dc098300 con 0x7fe1dc0881c0 2023-10-05T19:39:53.816 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.806+0000 7f384f7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f385005dac0 0x7f385006b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:53.816 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.806+0000 7f384f7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f385005dac0 0x7f385006b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38642/0 (socket says 172.21.15.8:38642) 2023-10-05T19:39:53.816 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.806+0000 7f384f7fe700 1 -- 172.21.15.8:0/2158773823 learned_addr learned my addr 172.21.15.8:0/2158773823 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-05T19:39:53.816 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.806+0000 7fe1da59c700 1 --2- 172.21.15.8:0/1257498877 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe1dc0881c0 0x7fe1dc088590 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe1c4009860 tx=0x7fe1c40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78f3605900a92909 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:53.816 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.806+0000 7fe1d959a700 1 -- 172.21.15.8:0/1257498877 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1c400b680 con 0x7fe1dc0881c0 2023-10-05T19:39:53.817 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.806+0000 7f384f7fe700 1 -- 172.21.15.8:0/2158773823 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3840009420 con 0x7f385005dac0 2023-10-05T19:39:53.817 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.806+0000 7fe1d959a700 1 -- 172.21.15.8:0/1257498877 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c400b7e0 con 0x7fe1dc0881c0 2023-10-05T19:39:53.817 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.806+0000 7fe1d959a700 1 -- 172.21.15.8:0/1257498877 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1c400bad0 con 0x7fe1dc0881c0 2023-10-05T19:39:53.818 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.806+0000 7f384f7fe700 1 --2- 172.21.15.8:0/2158773823 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f385005dac0 0x7f385006b1f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f38400093f0 tx=0x7f3840016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:53.818 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.806+0000 7f38477fe700 1 -- 172.21.15.8:0/2158773823 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f384001c450 con 0x7f385005dac0 2023-10-05T19:39:53.818 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.806+0000 7f38477fe700 1 -- 172.21.15.8:0/2158773823 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f384001c5d0 con 0x7f385005dac0 2023-10-05T19:39:53.818 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.806+0000 7f3854eec700 1 -- 172.21.15.8:0/2158773823 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f385006cc20 con 0x7f385005dac0 2023-10-05T19:39:53.818 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.806+0000 7f38477fe700 1 -- 172.21.15.8:0/2158773823 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f384001c780 con 0x7f385005dac0 2023-10-05T19:39:53.818 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.806+0000 7f3854eec700 1 -- 172.21.15.8:0/2158773823 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3850087260 con 0x7f385005dac0 2023-10-05T19:39:53.819 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.806+0000 7fe1e0fbf700 1 -- 172.21.15.8:0/1257498877 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe1dc0881c0 msgr2=0x7fe1dc088590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:53.819 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.806+0000 7fe1e0fbf700 1 --2- 172.21.15.8:0/1257498877 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe1dc0881c0 0x7fe1dc088590 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe1c4009860 tx=0x7fe1c40091a0 comp rx=0 tx=0).stop 2023-10-05T19:39:53.819 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.806+0000 7fe1e0fbf700 1 -- 172.21.15.8:0/1257498877 shutdown_connections 2023-10-05T19:39:53.820 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.806+0000 7fe1e0fbf700 1 --2- 172.21.15.8:0/1257498877 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe1dc0881c0 0x7fe1dc088590 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:53.820 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.806+0000 7fe1e0fbf700 1 -- 172.21.15.8:0/1257498877 >> 172.21.15.8:0/1257498877 conn(0x7fe1dc0964a0 msgr2=0x7fe1dc0968a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:53.820 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.806+0000 7fe1e0fbf700 1 -- 172.21.15.8:0/1257498877 shutdown_connections 2023-10-05T19:39:53.820 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.806+0000 7fe1e0fbf700 1 -- 172.21.15.8:0/1257498877 wait complete. 2023-10-05T19:39:53.821 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.806+0000 7f38477fe700 1 -- 172.21.15.8:0/2158773823 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f384001c9b0 con 0x7f385005dac0 2023-10-05T19:39:53.821 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.806+0000 7f38477fe700 1 --2- 172.21.15.8:0/2158773823 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f38300323e0 0x7f3830034890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:53.821 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.806+0000 7f38477fe700 1 -- 172.21.15.8:0/2158773823 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3478+0+0 (secure 0 0 0) 0x7f38400454f0 con 0x7f385005dac0 2023-10-05T19:39:53.821 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.806+0000 7f3854eec700 1 -- 172.21.15.8:0/2158773823 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f385005ecb0 con 0x7f385005dac0 2023-10-05T19:39:53.821 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.806+0000 7f384effd700 1 --2- 172.21.15.8:0/2158773823 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f38300323e0 0x7f3830034890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:53.821 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.810+0000 7fe1e0fbf700 1 Processor -- start 2023-10-05T19:39:53.821 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.810+0000 7fe1e0fbf700 1 -- start start 2023-10-05T19:39:53.822 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.810+0000 7f384effd700 1 --2- 172.21.15.8:0/2158773823 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f38300323e0 0x7f3830034890 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f38380011e0 tx=0x7f38380079d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:53.823 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.810+0000 7fe1e0fbf700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe1dc0881c0 0x7fe1dc1363e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:53.823 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.810+0000 7fe1e0fbf700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1dc11e950 con 0x7fe1dc0881c0 2023-10-05T19:39:53.823 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.810+0000 7fe1da59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe1dc0881c0 0x7fe1dc1363e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:53.823 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.810+0000 7fe1da59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe1dc0881c0 0x7fe1dc1363e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38648/0 (socket says 172.21.15.8:38648) 2023-10-05T19:39:53.823 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.810+0000 7fe1da59c700 1 -- 172.21.15.8:0/1153205057 learned_addr learned my addr 172.21.15.8:0/1153205057 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-05T19:39:53.823 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.810+0000 7fe1da59c700 1 -- 172.21.15.8:0/1153205057 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c4009420 con 0x7fe1dc0881c0 2023-10-05T19:39:53.823 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.810+0000 7fe1da59c700 1 --2- 172.21.15.8:0/1153205057 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe1dc0881c0 0x7fe1dc1363e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe1c40091e0 tx=0x7fe1c4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:53.823 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.814+0000 7fe1d27fc700 1 -- 172.21.15.8:0/1153205057 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1c4011cf0 con 0x7fe1dc0881c0 2023-10-05T19:39:53.823 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.814+0000 7fe1d27fc700 1 -- 172.21.15.8:0/1153205057 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c401f070 con 0x7fe1dc0881c0 2023-10-05T19:39:53.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.814+0000 7f38477fe700 1 -- 172.21.15.8:0/2158773823 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f3840018350 con 0x7f385005dac0 2023-10-05T19:39:53.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.814+0000 7fe1e0fbf700 1 -- 172.21.15.8:0/1153205057 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1dc11e630 con 0x7fe1dc0881c0 2023-10-05T19:39:53.825 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.814+0000 7fe1d27fc700 1 -- 172.21.15.8:0/1153205057 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1c4011e70 con 0x7fe1dc0881c0 2023-10-05T19:39:53.825 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.814+0000 7fe1e0fbf700 1 -- 172.21.15.8:0/1153205057 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1dc11de10 con 0x7fe1dc0881c0 2023-10-05T19:39:53.825 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.814+0000 7fe1d27fc700 1 -- 172.21.15.8:0/1153205057 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fe1c4021030 con 0x7fe1dc0881c0 2023-10-05T19:39:53.825 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.814+0000 7fe1d27fc700 1 --2- 172.21.15.8:0/1153205057 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe1bc0323d0 0x7fe1bc034880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:53.825 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.814+0000 7fe1d27fc700 1 -- 172.21.15.8:0/1153205057 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3478+0+0 (secure 0 0 0) 0x7fe1c40456b0 con 0x7fe1dc0881c0 2023-10-05T19:39:53.825 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.814+0000 7fe1d9d9b700 1 --2- 172.21.15.8:0/1153205057 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe1bc0323d0 0x7fe1bc034880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:53.825 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.818+0000 7fe1d9d9b700 1 --2- 172.21.15.8:0/1153205057 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe1bc0323d0 0x7fe1bc034880 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe1d4005c50 tx=0x7fe1d4001660 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:53.828 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.822+0000 7fe1e0fbf700 1 -- 172.21.15.8:0/1153205057 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1dc05ecb0 con 0x7fe1dc0881c0 2023-10-05T19:39:53.838 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.830+0000 7fe1d27fc700 1 -- 172.21.15.8:0/1153205057 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fe1c402c980 con 0x7fe1dc0881c0 2023-10-05T19:39:53.852 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.842+0000 7f17c10a7700 1 Processor -- start 2023-10-05T19:39:53.852 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.846+0000 7f17c10a7700 1 -- start start 2023-10-05T19:39:53.853 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.846+0000 7f17c10a7700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f17bc126c40 0x7f17bc12b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:53.853 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.846+0000 7f17c10a7700 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17bc06df90 con 0x7f17bc126160 2023-10-05T19:39:53.853 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.846+0000 7f17c10a7700 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17bc06e5e0 con 0x7f17bc126c40 2023-10-05T19:39:53.853 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.846+0000 7f17ba59c700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f17bc126c40 0x7f17bc12b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:53.853 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.846+0000 7f17bad9d700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f17bc126160 0x7f17bc126530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:42756/0 (socket says 172.21.15.8:42756) 2023-10-05T19:39:53.853 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.846+0000 7f17bad9d700 1 -- 172.21.15.8:0/3408021791 learned_addr learned my addr 172.21.15.8:0/3408021791 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-05T19:39:53.854 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.846+0000 7f17ba59c700 1 -- 172.21.15.8:0/3408021791 >> v1:172.21.15.8:6789/0 conn(0x7f17bc126160 legacy=0x7f17bc126530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:53.854 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.846+0000 7f17ba59c700 1 -- 172.21.15.8:0/3408021791 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17bc06a800 con 0x7f17bc126c40 2023-10-05T19:39:53.854 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.846+0000 7f17ba59c700 1 --2- 172.21.15.8:0/3408021791 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f17bc126c40 0x7f17bc12b6c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f17b000bb10 tx=0x7f17b000b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7166f63cf7c62133 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:53.854 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.846+0000 7f17b9d9b700 1 -- 172.21.15.8:0/3408021791 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f17b000db20 con 0x7f17bc126c40 2023-10-05T19:39:53.855 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.846+0000 7f17b9d9b700 1 -- 172.21.15.8:0/3408021791 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b000dc80 con 0x7f17bc126c40 2023-10-05T19:39:53.855 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.846+0000 7f17b9d9b700 1 -- 172.21.15.8:0/3408021791 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f17b0010470 con 0x7f17bc126c40 2023-10-05T19:39:53.855 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.846+0000 7f17c10a7700 1 -- 172.21.15.8:0/3408021791 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f17bc126c40 msgr2=0x7f17bc12b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:53.856 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.846+0000 7f17c10a7700 1 --2- 172.21.15.8:0/3408021791 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f17bc126c40 0x7f17bc12b6c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f17b000bb10 tx=0x7f17b000b430 comp rx=0 tx=0).stop 2023-10-05T19:39:53.856 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.846+0000 7f17c10a7700 1 -- 172.21.15.8:0/3408021791 shutdown_connections 2023-10-05T19:39:53.856 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.846+0000 7f17c10a7700 1 --2- 172.21.15.8:0/3408021791 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f17bc126c40 0x7f17bc12b6c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:53.856 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.846+0000 7f17c10a7700 1 -- 172.21.15.8:0/3408021791 >> 172.21.15.8:0/3408021791 conn(0x7f17bc098f60 msgr2=0x7f17bc0949f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:53.856 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.846+0000 7f17c10a7700 1 -- 172.21.15.8:0/3408021791 shutdown_connections 2023-10-05T19:39:53.856 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.846+0000 7f17c10a7700 1 -- 172.21.15.8:0/3408021791 wait complete. 2023-10-05T19:39:53.856 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.850+0000 7efdac7ca700 1 Processor -- start 2023-10-05T19:39:53.857 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.850+0000 7f17c10a7700 1 Processor -- start 2023-10-05T19:39:53.857 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.850+0000 7efdac7ca700 1 -- start start 2023-10-05T19:39:53.857 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.850+0000 7f17c10a7700 1 -- start start 2023-10-05T19:39:53.858 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.850+0000 7efdac7ca700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7efda4088c60 0x7efda412c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:53.858 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.850+0000 7efdac7ca700 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efda4089f80 con 0x7efda4097fc0 2023-10-05T19:39:53.858 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.850+0000 7efdac7ca700 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efda408a4a0 con 0x7efda4088c60 2023-10-05T19:39:53.858 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.850+0000 7f17c10a7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f17bc126160 0x7f17bc1158e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:53.858 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.850+0000 7f17c10a7700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17bc1190c0 con 0x7f17bc126160 2023-10-05T19:39:53.859 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.850+0000 7efda9d65700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7efda4088c60 0x7efda412c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:53.859 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.850+0000 7f17bad9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f17bc126160 0x7f17bc1158e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:53.859 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.850+0000 7f17bad9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f17bc126160 0x7f17bc1158e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38676/0 (socket says 172.21.15.8:38676) 2023-10-05T19:39:53.859 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.850+0000 7f17bad9d700 1 -- 172.21.15.8:0/744240605 learned_addr learned my addr 172.21.15.8:0/744240605 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-05T19:39:53.860 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.850+0000 7efda9d65700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7efda4088c60 0x7efda412c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38666/0 (socket says 172.21.15.8:38666) 2023-10-05T19:39:53.860 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.850+0000 7efda9d65700 1 -- 172.21.15.8:0/126564136 learned_addr learned my addr 172.21.15.8:0/126564136 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-05T19:39:53.860 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.850+0000 7efda9d65700 1 -- 172.21.15.8:0/126564136 >> v1:172.21.15.8:6789/0 conn(0x7efda4097fc0 legacy=0x7efda4088550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:53.860 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.850+0000 7efda9d65700 1 -- 172.21.15.8:0/126564136 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda4068390 con 0x7efda4088c60 2023-10-05T19:39:53.860 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.850+0000 7f17bad9d700 1 -- 172.21.15.8:0/744240605 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b000b6d0 con 0x7f17bc126160 2023-10-05T19:39:53.861 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.850+0000 7efda9d65700 1 --2- 172.21.15.8:0/126564136 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efda4088c60 0x7efda412c8e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efd98009930 tx=0x7efd98009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce625110f7ea861a server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:53.861 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.850+0000 7f17bad9d700 1 --2- 172.21.15.8:0/744240605 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f17bc126160 0x7f17bc1158e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f17b400cc30 tx=0x7f17b4001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:53.861 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.850+0000 7efda9564700 1 -- 172.21.15.8:0/126564136 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd9800b860 con 0x7efda4088c60 2023-10-05T19:39:53.861 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.850+0000 7efda9564700 1 -- 172.21.15.8:0/126564136 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd9800b9c0 con 0x7efda4088c60 2023-10-05T19:39:53.862 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.850+0000 7f17a2ffd700 1 -- 172.21.15.8:0/744240605 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f17b4004d50 con 0x7f17bc126160 2023-10-05T19:39:53.862 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.850+0000 7efda9564700 1 -- 172.21.15.8:0/126564136 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd9800bc70 con 0x7efda4088c60 2023-10-05T19:39:53.862 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.850+0000 7f17a2ffd700 1 -- 172.21.15.8:0/744240605 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b400a960 con 0x7f17bc126160 2023-10-05T19:39:53.862 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.850+0000 7f17c10a7700 1 -- 172.21.15.8:0/744240605 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17bc06cbe0 con 0x7f17bc126160 2023-10-05T19:39:53.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.850+0000 7efdac7ca700 1 -- 172.21.15.8:0/126564136 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efda4088c60 msgr2=0x7efda412c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:53.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.850+0000 7efdac7ca700 1 --2- 172.21.15.8:0/126564136 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efda4088c60 0x7efda412c8e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efd98009930 tx=0x7efd98009250 comp rx=0 tx=0).stop 2023-10-05T19:39:53.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.850+0000 7efdac7ca700 1 -- 172.21.15.8:0/126564136 shutdown_connections 2023-10-05T19:39:53.864 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.850+0000 7efdac7ca700 1 --2- 172.21.15.8:0/126564136 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efda4088c60 0x7efda412c8e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:53.864 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.850+0000 7efdac7ca700 1 -- 172.21.15.8:0/126564136 >> 172.21.15.8:0/126564136 conn(0x7efda40964a0 msgr2=0x7efda40968a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:53.864 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.850+0000 7efdac7ca700 1 -- 172.21.15.8:0/126564136 shutdown_connections 2023-10-05T19:39:53.864 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.850+0000 7efdac7ca700 1 -- 172.21.15.8:0/126564136 wait complete. 2023-10-05T19:39:53.864 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.850+0000 7f17a2ffd700 1 -- 172.21.15.8:0/744240605 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f17b400ac10 con 0x7f17bc126160 2023-10-05T19:39:53.864 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.850+0000 7f17c10a7700 1 -- 172.21.15.8:0/744240605 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17bc117250 con 0x7f17bc126160 2023-10-05T19:39:53.865 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.850+0000 7efdac7ca700 1 Processor -- start 2023-10-05T19:39:53.865 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.850+0000 7efdac7ca700 1 -- start start 2023-10-05T19:39:53.865 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.854+0000 7f17a2ffd700 1 -- 172.21.15.8:0/744240605 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f17b40070f0 con 0x7f17bc126160 2023-10-05T19:39:53.865 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.854+0000 7f17a2ffd700 1 --2- 172.21.15.8:0/744240605 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f17a80323e0 0x7f17a8034890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:53.865 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.854+0000 7f17a2ffd700 1 -- 172.21.15.8:0/744240605 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3478+0+0 (secure 0 0 0) 0x7f17b4043910 con 0x7f17bc126160 2023-10-05T19:39:53.866 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.854+0000 7efdac7ca700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efda4088c60 0x7efda40939e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:53.866 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.854+0000 7efdac7ca700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda40956e0 con 0x7efda4088c60 2023-10-05T19:39:53.866 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.854+0000 7efdaa566700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efda4088c60 0x7efda40939e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:53.866 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.854+0000 7efdaa566700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efda4088c60 0x7efda40939e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38682/0 (socket says 172.21.15.8:38682) 2023-10-05T19:39:53.866 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.854+0000 7efdaa566700 1 -- 172.21.15.8:0/10760371 learned_addr learned my addr 172.21.15.8:0/10760371 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-05T19:39:53.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.854+0000 7f17ba59c700 1 --2- 172.21.15.8:0/744240605 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f17a80323e0 0x7f17a8034890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:53.868 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.854+0000 7efdaa566700 1 -- 172.21.15.8:0/10760371 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd980094f0 con 0x7efda4088c60 2023-10-05T19:39:53.868 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.854+0000 7efdaa566700 1 --2- 172.21.15.8:0/10760371 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efda4088c60 0x7efda40939e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efda000c3e0 tx=0x7efda000c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:53.868 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.854+0000 7efd967fc700 1 -- 172.21.15.8:0/10760371 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efda0011590 con 0x7efda4088c60 2023-10-05T19:39:53.868 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.854+0000 7efd967fc700 1 -- 172.21.15.8:0/10760371 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efda0011710 con 0x7efda4088c60 2023-10-05T19:39:53.868 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.854+0000 7efdac7ca700 1 -- 172.21.15.8:0/10760371 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efda411e8c0 con 0x7efda4088c60 2023-10-05T19:39:53.868 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.854+0000 7efd967fc700 1 -- 172.21.15.8:0/10760371 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efda0011a00 con 0x7efda4088c60 2023-10-05T19:39:53.868 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.854+0000 7efdac7ca700 1 -- 172.21.15.8:0/10760371 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efda411dff0 con 0x7efda4088c60 2023-10-05T19:39:53.868 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.854+0000 7f17ba59c700 1 --2- 172.21.15.8:0/744240605 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f17a80323e0 0x7f17a8034890 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f17b000b470 tx=0x7f17b001a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:53.869 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.854+0000 7efd967fc700 1 -- 172.21.15.8:0/10760371 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7efda0011b60 con 0x7efda4088c60 2023-10-05T19:39:53.869 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.854+0000 7efdac7ca700 1 -- 172.21.15.8:0/10760371 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efda405ecb0 con 0x7efda4088c60 2023-10-05T19:39:53.869 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.854+0000 7efd967fc700 1 --2- 172.21.15.8:0/10760371 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7efd880323e0 0x7efd88034890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:53.869 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.854+0000 7efd967fc700 1 -- 172.21.15.8:0/10760371 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3478+0+0 (secure 0 0 0) 0x7efda0042e60 con 0x7efda4088c60 2023-10-05T19:39:53.870 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.854+0000 7efda9d65700 1 --2- 172.21.15.8:0/10760371 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7efd880323e0 0x7efd88034890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:53.870 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.858+0000 7f17c10a7700 1 -- 172.21.15.8:0/744240605 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17bc118ec0 con 0x7f17bc126160 2023-10-05T19:39:53.870 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.858+0000 7efda9d65700 1 --2- 172.21.15.8:0/10760371 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7efd880323e0 0x7efd88034890 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efd98009930 tx=0x7efd98016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:53.870 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.862+0000 7efd967fc700 1 -- 172.21.15.8:0/10760371 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7efda0019b40 con 0x7efda4088c60 2023-10-05T19:39:53.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.870+0000 7f17a2ffd700 1 -- 172.21.15.8:0/744240605 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f17b4018020 con 0x7f17bc126160 2023-10-05T19:39:53.985 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.978+0000 7f3854eec700 1 -- 172.21.15.8:0/2158773823 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f385004b5c0 con 0x7f385005dac0 2023-10-05T19:39:53.985 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.978+0000 7f38477fe700 1 -- 172.21.15.8:0/2158773823 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f384002db40 con 0x7f385005dac0 2023-10-05T19:39:53.985 INFO:teuthology.orchestra.run.smithi008.stdout:38654705667 2023-10-05T19:39:53.988 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.978+0000 7f3854eec700 1 -- 172.21.15.8:0/2158773823 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f38300323e0 msgr2=0x7f3830034890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:53.988 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.978+0000 7f3854eec700 1 --2- 172.21.15.8:0/2158773823 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f38300323e0 0x7f3830034890 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f38380011e0 tx=0x7f38380079d0 comp rx=0 tx=0).stop 2023-10-05T19:39:53.988 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.982+0000 7f3854eec700 1 -- 172.21.15.8:0/2158773823 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f385005dac0 msgr2=0x7f385006b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:53.989 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.982+0000 7f3854eec700 1 --2- 172.21.15.8:0/2158773823 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f385005dac0 0x7f385006b1f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f38400093f0 tx=0x7f3840016000 comp rx=0 tx=0).stop 2023-10-05T19:39:53.989 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.982+0000 7f3854eec700 1 -- 172.21.15.8:0/2158773823 shutdown_connections 2023-10-05T19:39:53.989 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.982+0000 7f3854eec700 1 --2- 172.21.15.8:0/2158773823 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f38300323e0 0x7f3830034890 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:53.989 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.982+0000 7f3854eec700 1 --2- 172.21.15.8:0/2158773823 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f385005dac0 0x7f385006b1f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:53.989 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.982+0000 7f3854eec700 1 -- 172.21.15.8:0/2158773823 >> 172.21.15.8:0/2158773823 conn(0x7f3850095970 msgr2=0x7f3850126f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:53.989 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.982+0000 7f3854eec700 1 -- 172.21.15.8:0/2158773823 shutdown_connections 2023-10-05T19:39:53.989 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:53.982+0000 7f3854eec700 1 -- 172.21.15.8:0/2158773823 wait complete. 2023-10-05T19:39:54.002 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705667 for osd.0 2023-10-05T19:39:54.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:54.002+0000 7fe1e0fbf700 1 -- 172.21.15.8:0/1153205057 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe1dc06e970 con 0x7fe1dc0881c0 2023-10-05T19:39:54.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:54.006+0000 7fe1d27fc700 1 -- 172.21.15.8:0/1153205057 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe1c40193c0 con 0x7fe1dc0881c0 2023-10-05T19:39:54.012 INFO:teuthology.orchestra.run.smithi008.stdout:38654705666 2023-10-05T19:39:54.015 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:54.006+0000 7fe1e0fbf700 1 -- 172.21.15.8:0/1153205057 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe1bc0323d0 msgr2=0x7fe1bc034880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:54.015 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:54.006+0000 7fe1e0fbf700 1 --2- 172.21.15.8:0/1153205057 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe1bc0323d0 0x7fe1bc034880 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe1d4005c50 tx=0x7fe1d4001660 comp rx=0 tx=0).stop 2023-10-05T19:39:54.015 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:54.006+0000 7fe1e0fbf700 1 -- 172.21.15.8:0/1153205057 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe1dc0881c0 msgr2=0x7fe1dc1363e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:54.015 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:54.006+0000 7fe1e0fbf700 1 --2- 172.21.15.8:0/1153205057 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe1dc0881c0 0x7fe1dc1363e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe1c40091e0 tx=0x7fe1c4016000 comp rx=0 tx=0).stop 2023-10-05T19:39:54.015 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:54.006+0000 7fe1e0fbf700 1 -- 172.21.15.8:0/1153205057 shutdown_connections 2023-10-05T19:39:54.015 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:54.006+0000 7fe1e0fbf700 1 --2- 172.21.15.8:0/1153205057 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe1bc0323d0 0x7fe1bc034880 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:54.015 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:54.006+0000 7fe1e0fbf700 1 --2- 172.21.15.8:0/1153205057 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe1dc0881c0 0x7fe1dc1363e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:54.016 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:54.006+0000 7fe1e0fbf700 1 -- 172.21.15.8:0/1153205057 >> 172.21.15.8:0/1153205057 conn(0x7fe1dc0964a0 msgr2=0x7fe1dc0862e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:54.016 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:54.006+0000 7fe1e0fbf700 1 -- 172.21.15.8:0/1153205057 shutdown_connections 2023-10-05T19:39:54.016 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:54.006+0000 7fe1e0fbf700 1 -- 172.21.15.8:0/1153205057 wait complete. 2023-10-05T19:39:54.030 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705667 got 38654705666 for osd.1 2023-10-05T19:39:54.032 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:54.022+0000 7efdac7ca700 1 -- 172.21.15.8:0/10760371 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7efda404b5c0 con 0x7efda4088c60 2023-10-05T19:39:54.032 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:54.026+0000 7efd967fc700 1 -- 172.21.15.8:0/10760371 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7efda0017070 con 0x7efda4088c60 2023-10-05T19:39:54.032 INFO:teuthology.orchestra.run.smithi008.stdout:38654705667 2023-10-05T19:39:54.035 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:54.026+0000 7efdac7ca700 1 -- 172.21.15.8:0/10760371 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7efd880323e0 msgr2=0x7efd88034890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:54.036 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:54.026+0000 7efdac7ca700 1 --2- 172.21.15.8:0/10760371 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7efd880323e0 0x7efd88034890 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efd98009930 tx=0x7efd98016000 comp rx=0 tx=0).stop 2023-10-05T19:39:54.036 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:54.026+0000 7efdac7ca700 1 -- 172.21.15.8:0/10760371 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efda4088c60 msgr2=0x7efda40939e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:54.036 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:54.026+0000 7efdac7ca700 1 --2- 172.21.15.8:0/10760371 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efda4088c60 0x7efda40939e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efda000c3e0 tx=0x7efda000c7a0 comp rx=0 tx=0).stop 2023-10-05T19:39:54.036 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:54.026+0000 7efdac7ca700 1 -- 172.21.15.8:0/10760371 shutdown_connections 2023-10-05T19:39:54.036 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:54.026+0000 7efdac7ca700 1 --2- 172.21.15.8:0/10760371 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7efd880323e0 0x7efd88034890 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:54.036 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:54.026+0000 7efdac7ca700 1 --2- 172.21.15.8:0/10760371 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efda4088c60 0x7efda40939e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:54.036 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:54.026+0000 7efdac7ca700 1 -- 172.21.15.8:0/10760371 >> 172.21.15.8:0/10760371 conn(0x7efda40964a0 msgr2=0x7efda4086140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:54.036 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:54.026+0000 7efdac7ca700 1 -- 172.21.15.8:0/10760371 shutdown_connections 2023-10-05T19:39:54.036 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:54.026+0000 7efdac7ca700 1 -- 172.21.15.8:0/10760371 wait complete. 2023-10-05T19:39:54.043 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:54.034+0000 7f17c10a7700 1 -- 172.21.15.8:0/744240605 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f17bc05ecb0 con 0x7f17bc126160 2023-10-05T19:39:54.044 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:54.034+0000 7f17a2ffd700 1 -- 172.21.15.8:0/744240605 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f17b4025070 con 0x7f17bc126160 2023-10-05T19:39:54.044 INFO:teuthology.orchestra.run.smithi008.stdout:38654705667 2023-10-05T19:39:54.046 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:54.038+0000 7f17c10a7700 1 -- 172.21.15.8:0/744240605 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f17a80323e0 msgr2=0x7f17a8034890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:54.046 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:54.038+0000 7f17c10a7700 1 --2- 172.21.15.8:0/744240605 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f17a80323e0 0x7f17a8034890 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f17b000b470 tx=0x7f17b001a000 comp rx=0 tx=0).stop 2023-10-05T19:39:54.047 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:54.038+0000 7f17c10a7700 1 -- 172.21.15.8:0/744240605 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f17bc126160 msgr2=0x7f17bc1158e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:54.047 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:54.038+0000 7f17c10a7700 1 --2- 172.21.15.8:0/744240605 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f17bc126160 0x7f17bc1158e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f17b400cc30 tx=0x7f17b4001ad0 comp rx=0 tx=0).stop 2023-10-05T19:39:54.047 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:54.038+0000 7f17c10a7700 1 -- 172.21.15.8:0/744240605 shutdown_connections 2023-10-05T19:39:54.047 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:54.038+0000 7f17c10a7700 1 --2- 172.21.15.8:0/744240605 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f17a80323e0 0x7f17a8034890 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:54.047 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:54.038+0000 7f17c10a7700 1 --2- 172.21.15.8:0/744240605 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f17bc126160 0x7f17bc1158e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:54.047 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:54.038+0000 7f17c10a7700 1 -- 172.21.15.8:0/744240605 >> 172.21.15.8:0/744240605 conn(0x7f17bc098f60 msgr2=0x7f17bc10eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:54.047 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:54.038+0000 7f17c10a7700 1 -- 172.21.15.8:0/744240605 shutdown_connections 2023-10-05T19:39:54.047 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:54.038+0000 7f17c10a7700 1 -- 172.21.15.8:0/744240605 wait complete. 2023-10-05T19:39:54.048 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705667 for osd.3 2023-10-05T19:39:54.057 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705667 for osd.2 2023-10-05T19:39:55.003 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-05T19:39:55.031 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-05T19:39:55.049 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-05T19:39:55.058 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-05T19:39:55.118 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.110+0000 7f8aa9f35700 1 Processor -- start 2023-10-05T19:39:55.119 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.110+0000 7f8aa9f35700 1 -- start start 2023-10-05T19:39:55.120 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.110+0000 7f8aa9f35700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f8aa4091340 0x7f8aa4091710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:55.120 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.110+0000 7f8aa9f35700 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8aa4092d00 con 0x7f8aa4091c50 2023-10-05T19:39:55.120 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.110+0000 7f8aa9f35700 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aa4069080 con 0x7f8aa4091340 2023-10-05T19:39:55.120 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.110+0000 7f8aa37fe700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f8aa4091340 0x7f8aa4091710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:55.120 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.110+0000 7f8a9bfff700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f8aa4091c50 0x7f8aa408f010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:42776/0 (socket says 172.21.15.8:42776) 2023-10-05T19:39:55.120 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.110+0000 7f8a9bfff700 1 -- 172.21.15.8:0/934290890 learned_addr learned my addr 172.21.15.8:0/934290890 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-05T19:39:55.121 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.110+0000 7f8aa2ffd700 1 -- 172.21.15.8:0/934290890 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3603863423 0 0) 0x7f8aa4092d00 con 0x7f8aa4091c50 2023-10-05T19:39:55.121 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.110+0000 7f8aa2ffd700 1 -- 172.21.15.8:0/934290890 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a84003570 con 0x7f8aa4091c50 2023-10-05T19:39:55.121 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.110+0000 7f8aa37fe700 1 -- 172.21.15.8:0/934290890 >> v1:172.21.15.8:6789/0 conn(0x7f8aa4091c50 legacy=0x7f8aa408f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:55.121 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.110+0000 7f8aa37fe700 1 -- 172.21.15.8:0/934290890 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa40692c0 con 0x7f8aa4091340 2023-10-05T19:39:55.121 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.114+0000 7f8aa37fe700 1 --2- 172.21.15.8:0/934290890 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8aa4091340 0x7f8aa4091710 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8a94009860 tx=0x7f8a940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6cf9c2794ee22a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:55.122 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.114+0000 7f8aa2ffd700 1 -- 172.21.15.8:0/934290890 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a94011630 con 0x7f8aa4091340 2023-10-05T19:39:55.122 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.114+0000 7f8aa2ffd700 1 -- 172.21.15.8:0/934290890 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a94011790 con 0x7f8aa4091340 2023-10-05T19:39:55.122 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.114+0000 7f8aa2ffd700 1 -- 172.21.15.8:0/934290890 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a94011920 con 0x7f8aa4091340 2023-10-05T19:39:55.122 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.114+0000 7f8aa9f35700 1 -- 172.21.15.8:0/934290890 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8aa4091340 msgr2=0x7f8aa4091710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:55.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.114+0000 7f8aa9f35700 1 --2- 172.21.15.8:0/934290890 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8aa4091340 0x7f8aa4091710 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8a94009860 tx=0x7f8a940091a0 comp rx=0 tx=0).stop 2023-10-05T19:39:55.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.114+0000 7f8aa9f35700 1 -- 172.21.15.8:0/934290890 shutdown_connections 2023-10-05T19:39:55.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.114+0000 7f8aa9f35700 1 --2- 172.21.15.8:0/934290890 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8aa4091340 0x7f8aa4091710 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:55.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.114+0000 7f8aa9f35700 1 -- 172.21.15.8:0/934290890 >> 172.21.15.8:0/934290890 conn(0x7f8aa4096d80 msgr2=0x7f8aa4097180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:55.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.114+0000 7f8aa9f35700 1 -- 172.21.15.8:0/934290890 shutdown_connections 2023-10-05T19:39:55.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.114+0000 7f8aa9f35700 1 -- 172.21.15.8:0/934290890 wait complete. 2023-10-05T19:39:55.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.114+0000 7f8aa9f35700 1 Processor -- start 2023-10-05T19:39:55.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.114+0000 7f8aa9f35700 1 -- start start 2023-10-05T19:39:55.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.114+0000 7f8aa9f35700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8aa4091340 0x7f8aa4085f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:55.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.114+0000 7f8aa9f35700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa406ec30 con 0x7f8aa4091340 2023-10-05T19:39:55.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.114+0000 7f8aa37fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8aa4091340 0x7f8aa4085f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:55.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.114+0000 7f8aa37fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8aa4091340 0x7f8aa4085f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38706/0 (socket says 172.21.15.8:38706) 2023-10-05T19:39:55.126 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.114+0000 7f8aa37fe700 1 -- 172.21.15.8:0/920631429 learned_addr learned my addr 172.21.15.8:0/920631429 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-05T19:39:55.126 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.118+0000 7f8aa37fe700 1 -- 172.21.15.8:0/920631429 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a94009420 con 0x7f8aa4091340 2023-10-05T19:39:55.126 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.118+0000 7f8aa37fe700 1 --2- 172.21.15.8:0/920631429 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8aa4091340 0x7f8aa4085f80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8a94000c00 tx=0x7f8a94016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:55.126 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.118+0000 7f8a9b7fe700 1 -- 172.21.15.8:0/920631429 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a94020070 con 0x7f8aa4091340 2023-10-05T19:39:55.126 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.118+0000 7f8a9b7fe700 1 -- 172.21.15.8:0/920631429 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a94011900 con 0x7f8aa4091340 2023-10-05T19:39:55.126 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.118+0000 7f8aa9f35700 1 -- 172.21.15.8:0/920631429 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aa4088fc0 con 0x7f8aa4091340 2023-10-05T19:39:55.126 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.118+0000 7f8a9b7fe700 1 -- 172.21.15.8:0/920631429 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a94026420 con 0x7f8aa4091340 2023-10-05T19:39:55.126 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.118+0000 7f8aa9f35700 1 -- 172.21.15.8:0/920631429 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aa4087ed0 con 0x7f8aa4091340 2023-10-05T19:39:55.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.118+0000 7f8a9b7fe700 1 -- 172.21.15.8:0/920631429 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f8a94024070 con 0x7f8aa4091340 2023-10-05T19:39:55.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.118+0000 7f8aa9f35700 1 -- 172.21.15.8:0/920631429 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aa4089230 con 0x7f8aa4091340 2023-10-05T19:39:55.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.118+0000 7f8a9b7fe700 1 --2- 172.21.15.8:0/920631429 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f8a88032390 0x7f8a88034840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:55.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.118+0000 7f8a9b7fe700 1 -- 172.21.15.8:0/920631429 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3478+0+0 (secure 0 0 0) 0x7f8a94046b30 con 0x7f8aa4091340 2023-10-05T19:39:55.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.118+0000 7f8a9bfff700 1 --2- 172.21.15.8:0/920631429 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f8a88032390 0x7f8a88034840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:55.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.118+0000 7f8a9bfff700 1 --2- 172.21.15.8:0/920631429 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f8a88032390 0x7f8a88034840 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c0058d0 tx=0x7f8a8c0079d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:55.136 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.130+0000 7f8a9b7fe700 1 -- 172.21.15.8:0/920631429 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f8aa4089230 con 0x7f8aa4091340 2023-10-05T19:39:55.169 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.162+0000 7f9464695700 1 Processor -- start 2023-10-05T19:39:55.170 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.162+0000 7f9464695700 1 -- start start 2023-10-05T19:39:55.170 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.162+0000 7f9464695700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f945c1261b0 0x7f945c126580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:55.170 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.162+0000 7f9464695700 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f945c095350 con 0x7f945c126ac0 2023-10-05T19:39:55.170 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.162+0000 7f9464695700 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f945c094fc0 con 0x7f945c1261b0 2023-10-05T19:39:55.170 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.162+0000 7f9462431700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f945c1261b0 0x7f945c126580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:55.170 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.162+0000 7f9462431700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f945c1261b0 0x7f945c126580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38708/0 (socket says 172.21.15.8:38708) 2023-10-05T19:39:55.170 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.162+0000 7f9462431700 1 -- 172.21.15.8:0/3428850575 learned_addr learned my addr 172.21.15.8:0/3428850575 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-05T19:39:55.170 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.162+0000 7f9462431700 1 -- 172.21.15.8:0/3428850575 >> v1:172.21.15.8:6789/0 conn(0x7f945c126ac0 legacy=0x7f945c08f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:55.171 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.162+0000 7f9462431700 1 -- 172.21.15.8:0/3428850575 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f945c092dd0 con 0x7f945c1261b0 2023-10-05T19:39:55.172 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.162+0000 7f9462431700 1 --2- 172.21.15.8:0/3428850575 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f945c1261b0 0x7f945c126580 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9450009910 tx=0x7f9450009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc202704d545a49b server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:55.172 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.162+0000 7f946142f700 1 -- 172.21.15.8:0/3428850575 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9450005c10 con 0x7f945c1261b0 2023-10-05T19:39:55.172 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.162+0000 7f946142f700 1 -- 172.21.15.8:0/3428850575 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9450005d70 con 0x7f945c1261b0 2023-10-05T19:39:55.172 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.162+0000 7f946142f700 1 -- 172.21.15.8:0/3428850575 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f945000b650 con 0x7f945c1261b0 2023-10-05T19:39:55.172 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.162+0000 7f9464695700 1 -- 172.21.15.8:0/3428850575 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f945c1261b0 msgr2=0x7f945c126580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:55.172 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.162+0000 7f9464695700 1 --2- 172.21.15.8:0/3428850575 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f945c1261b0 0x7f945c126580 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9450009910 tx=0x7f9450009220 comp rx=0 tx=0).stop 2023-10-05T19:39:55.172 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.162+0000 7f9464695700 1 -- 172.21.15.8:0/3428850575 shutdown_connections 2023-10-05T19:39:55.172 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.162+0000 7f9464695700 1 --2- 172.21.15.8:0/3428850575 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f945c1261b0 0x7f945c126580 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:55.173 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.162+0000 7f9464695700 1 -- 172.21.15.8:0/3428850575 >> 172.21.15.8:0/3428850575 conn(0x7f945c095970 msgr2=0x7f945c095d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:55.173 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.162+0000 7f9464695700 1 -- 172.21.15.8:0/3428850575 shutdown_connections 2023-10-05T19:39:55.173 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.162+0000 7f9464695700 1 -- 172.21.15.8:0/3428850575 wait complete. 2023-10-05T19:39:55.174 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.166+0000 7f9464695700 1 Processor -- start 2023-10-05T19:39:55.174 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.166+0000 7f9464695700 1 -- start start 2023-10-05T19:39:55.175 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.166+0000 7f9464695700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f945c1261b0 0x7f945c08f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:55.175 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.166+0000 7f9464695700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945c125d10 con 0x7f945c1261b0 2023-10-05T19:39:55.175 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.166+0000 7f9462431700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f945c1261b0 0x7f945c08f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:55.175 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.166+0000 7f9462431700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f945c1261b0 0x7f945c08f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38714/0 (socket says 172.21.15.8:38714) 2023-10-05T19:39:55.175 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.166+0000 7f9462431700 1 -- 172.21.15.8:0/841415857 learned_addr learned my addr 172.21.15.8:0/841415857 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-05T19:39:55.175 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.166+0000 7f9462431700 1 -- 172.21.15.8:0/841415857 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9450009480 con 0x7f945c1261b0 2023-10-05T19:39:55.175 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.166+0000 7f9462431700 1 --2- 172.21.15.8:0/841415857 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f945c1261b0 0x7f945c08f3e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9450000c00 tx=0x7f9450015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:55.175 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.166+0000 7f945a7fc700 1 -- 172.21.15.8:0/841415857 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9450006b60 con 0x7f945c1261b0 2023-10-05T19:39:55.175 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.166+0000 7f945a7fc700 1 -- 172.21.15.8:0/841415857 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9450011b40 con 0x7f945c1261b0 2023-10-05T19:39:55.176 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.166+0000 7f945a7fc700 1 -- 172.21.15.8:0/841415857 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9450011cf0 con 0x7f945c1261b0 2023-10-05T19:39:55.176 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.166+0000 7f9464695700 1 -- 172.21.15.8:0/841415857 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f945c090d60 con 0x7f945c1261b0 2023-10-05T19:39:55.176 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.166+0000 7f9464695700 1 -- 172.21.15.8:0/841415857 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f945c090460 con 0x7f945c1261b0 2023-10-05T19:39:55.176 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.166+0000 7f3adffff700 1 Processor -- start 2023-10-05T19:39:55.176 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.166+0000 7f3adffff700 1 -- start start 2023-10-05T19:39:55.177 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.166+0000 7f945a7fc700 1 -- 172.21.15.8:0/841415857 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f945001d430 con 0x7f945c1261b0 2023-10-05T19:39:55.177 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.166+0000 7f945a7fc700 1 --2- 172.21.15.8:0/841415857 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f94440323b0 0x7f9444034860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:55.177 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.166+0000 7f9464695700 1 -- 172.21.15.8:0/841415857 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f945c05ecb0 con 0x7f945c1261b0 2023-10-05T19:39:55.177 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.166+0000 7f945a7fc700 1 -- 172.21.15.8:0/841415857 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3478+0+0 (secure 0 0 0) 0x7f9450045970 con 0x7f945c1261b0 2023-10-05T19:39:55.178 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.170+0000 7f3adffff700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f3ae008d300 0x7f3ae008d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:55.178 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.170+0000 7f3adffff700 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ae008f8a0 con 0x7f3ae008dc10 2023-10-05T19:39:55.178 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.170+0000 7f3adffff700 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ae0098f10 con 0x7f3ae008d300 2023-10-05T19:39:55.178 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.170+0000 7f3adeffd700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f3ae008d300 0x7f3ae008d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:55.178 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.170+0000 7f3ade7fc700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f3ae008dc10 0x7f3ae0126490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:42784/0 (socket says 172.21.15.8:42784) 2023-10-05T19:39:55.178 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.170+0000 7f3ade7fc700 1 -- 172.21.15.8:0/2905597238 learned_addr learned my addr 172.21.15.8:0/2905597238 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-05T19:39:55.179 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.170+0000 7f9461c30700 1 --2- 172.21.15.8:0/841415857 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f94440323b0 0x7f9444034860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:55.179 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.170+0000 7f3adeffd700 1 -- 172.21.15.8:0/2905597238 >> v1:172.21.15.8:6789/0 conn(0x7f3ae008dc10 legacy=0x7f3ae0126490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:55.179 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.170+0000 7f3adeffd700 1 -- 172.21.15.8:0/2905597238 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae00986b0 con 0x7f3ae008d300 2023-10-05T19:39:55.179 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.170+0000 7f3adeffd700 1 --2- 172.21.15.8:0/2905597238 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ae008d300 0x7f3ae008d6d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0009860 tx=0x7f3ad00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ded79bb5bf0499ae server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:55.180 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.170+0000 7f9461c30700 1 --2- 172.21.15.8:0/841415857 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f94440323b0 0x7f9444034860 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9454001d90 tx=0x7f9454007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:55.181 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.174+0000 7f56cfc3a700 1 Processor -- start 2023-10-05T19:39:55.181 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.174+0000 7f56cfc3a700 1 -- start start 2023-10-05T19:39:55.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.174+0000 7f56cfc3a700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f56c8126bd0 0x7f56c812b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:55.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.174+0000 7f56cfc3a700 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56c806e5e0 con 0x7f56c81260f0 2023-10-05T19:39:55.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.174+0000 7f56cfc3a700 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56c806e810 con 0x7f56c8126bd0 2023-10-05T19:39:55.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.174+0000 7f56cd1d5700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f56c8126bd0 0x7f56c812b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:55.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.174+0000 7f56cd9d6700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f56c81260f0 0x7f56c81264c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:42800/0 (socket says 172.21.15.8:42800) 2023-10-05T19:39:55.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.174+0000 7f56cd9d6700 1 -- 172.21.15.8:0/1810611354 learned_addr learned my addr 172.21.15.8:0/1810611354 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-05T19:39:55.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.174+0000 7f56cd1d5700 1 -- 172.21.15.8:0/1810611354 >> v1:172.21.15.8:6789/0 conn(0x7f56c81260f0 legacy=0x7f56c81264c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:55.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.174+0000 7f56cd1d5700 1 -- 172.21.15.8:0/1810611354 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c806d420 con 0x7f56c8126bd0 2023-10-05T19:39:55.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.174+0000 7f56cd1d5700 1 --2- 172.21.15.8:0/1810611354 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f56c8126bd0 0x7f56c812b670 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f56c4009860 tx=0x7f56c40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f54aa9e99c8b67bd server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:55.184 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.178+0000 7f56cc9d4700 1 -- 172.21.15.8:0/1810611354 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f56c400b680 con 0x7f56c8126bd0 2023-10-05T19:39:55.185 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.178+0000 7f3addffb700 1 -- 172.21.15.8:0/2905597238 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3ad000b680 con 0x7f3ae008d300 2023-10-05T19:39:55.185 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.178+0000 7f3addffb700 1 -- 172.21.15.8:0/2905597238 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad000b7e0 con 0x7f3ae008d300 2023-10-05T19:39:55.185 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.178+0000 7f3addffb700 1 -- 172.21.15.8:0/2905597238 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3ad000b990 con 0x7f3ae008d300 2023-10-05T19:39:55.186 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.178+0000 7f945a7fc700 1 -- 172.21.15.8:0/841415857 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f945001c9b0 con 0x7f945c1261b0 2023-10-05T19:39:55.186 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.178+0000 7f56cc9d4700 1 -- 172.21.15.8:0/1810611354 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c400b7e0 con 0x7f56c8126bd0 2023-10-05T19:39:55.187 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.178+0000 7f56cc9d4700 1 -- 172.21.15.8:0/1810611354 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f56c400b990 con 0x7f56c8126bd0 2023-10-05T19:39:55.187 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.178+0000 7f56cfc3a700 1 -- 172.21.15.8:0/1810611354 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f56c8126bd0 msgr2=0x7f56c812b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:55.187 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.178+0000 7f56cfc3a700 1 --2- 172.21.15.8:0/1810611354 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f56c8126bd0 0x7f56c812b670 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f56c4009860 tx=0x7f56c40091a0 comp rx=0 tx=0).stop 2023-10-05T19:39:55.187 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.178+0000 7f56cfc3a700 1 -- 172.21.15.8:0/1810611354 shutdown_connections 2023-10-05T19:39:55.187 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.178+0000 7f56cfc3a700 1 --2- 172.21.15.8:0/1810611354 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f56c8126bd0 0x7f56c812b670 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:55.187 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.178+0000 7f56cfc3a700 1 -- 172.21.15.8:0/1810611354 >> 172.21.15.8:0/1810611354 conn(0x7f56c8098f60 msgr2=0x7f56c80949f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:55.187 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.178+0000 7f56cfc3a700 1 -- 172.21.15.8:0/1810611354 shutdown_connections 2023-10-05T19:39:55.187 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.178+0000 7f3adffff700 1 -- 172.21.15.8:0/2905597238 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ae008d300 msgr2=0x7f3ae008d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:55.188 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.178+0000 7f3adffff700 1 --2- 172.21.15.8:0/2905597238 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ae008d300 0x7f3ae008d6d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0009860 tx=0x7f3ad00091a0 comp rx=0 tx=0).stop 2023-10-05T19:39:55.188 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.178+0000 7f56cfc3a700 1 -- 172.21.15.8:0/1810611354 wait complete. 2023-10-05T19:39:55.188 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.178+0000 7f56cfc3a700 1 Processor -- start 2023-10-05T19:39:55.188 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.178+0000 7f56cfc3a700 1 -- start start 2023-10-05T19:39:55.189 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.178+0000 7f3adffff700 1 -- 172.21.15.8:0/2905597238 shutdown_connections 2023-10-05T19:39:55.189 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.178+0000 7f3adffff700 1 --2- 172.21.15.8:0/2905597238 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ae008d300 0x7f3ae008d6d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:55.189 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.178+0000 7f3adffff700 1 -- 172.21.15.8:0/2905597238 >> 172.21.15.8:0/2905597238 conn(0x7f3ae00994a0 msgr2=0x7f3ae00998a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:55.189 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.178+0000 7f3adffff700 1 -- 172.21.15.8:0/2905597238 shutdown_connections 2023-10-05T19:39:55.189 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.178+0000 7f3adffff700 1 -- 172.21.15.8:0/2905597238 wait complete. 2023-10-05T19:39:55.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.178+0000 7f56cfc3a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f56c81260f0 0x7f56c81364b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:55.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.178+0000 7f56cfc3a700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c8117dd0 con 0x7f56c81260f0 2023-10-05T19:39:55.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.178+0000 7f56cd9d6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f56c81260f0 0x7f56c81364b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:55.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.178+0000 7f56cd9d6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f56c81260f0 0x7f56c81364b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38746/0 (socket says 172.21.15.8:38746) 2023-10-05T19:39:55.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.178+0000 7f56cd9d6700 1 -- 172.21.15.8:0/2667903542 learned_addr learned my addr 172.21.15.8:0/2667903542 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-05T19:39:55.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.178+0000 7f3adffff700 1 Processor -- start 2023-10-05T19:39:55.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.178+0000 7f3adffff700 1 -- start start 2023-10-05T19:39:55.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.178+0000 7f56cd9d6700 1 -- 172.21.15.8:0/2667903542 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c4009420 con 0x7f56c81260f0 2023-10-05T19:39:55.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.178+0000 7f3adffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ae008d300 0x7f3ae008b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:55.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.182+0000 7f3adffff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae006c1b0 con 0x7f3ae008d300 2023-10-05T19:39:55.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.182+0000 7f3adeffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ae008d300 0x7f3ae008b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:55.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.182+0000 7f3adeffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ae008d300 0x7f3ae008b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38758/0 (socket says 172.21.15.8:38758) 2023-10-05T19:39:55.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.182+0000 7f3adeffd700 1 -- 172.21.15.8:0/1930053915 learned_addr learned my addr 172.21.15.8:0/1930053915 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-05T19:39:55.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.182+0000 7f56cd9d6700 1 --2- 172.21.15.8:0/2667903542 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f56c81260f0 0x7f56c81364b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f56c000cb80 tx=0x7f56c0001a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:55.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.182+0000 7f3adeffd700 1 -- 172.21.15.8:0/1930053915 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad0009420 con 0x7f3ae008d300 2023-10-05T19:39:55.194 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.182+0000 7f56bdffb700 1 -- 172.21.15.8:0/2667903542 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f56c000a910 con 0x7f56c81260f0 2023-10-05T19:39:55.194 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.182+0000 7f56bdffb700 1 -- 172.21.15.8:0/2667903542 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c000aa70 con 0x7f56c81260f0 2023-10-05T19:39:55.194 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.182+0000 7f56bdffb700 1 -- 172.21.15.8:0/2667903542 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f56c000ad20 con 0x7f56c81260f0 2023-10-05T19:39:55.194 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.182+0000 7f56cfc3a700 1 -- 172.21.15.8:0/2667903542 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56c8116450 con 0x7f56c81260f0 2023-10-05T19:39:55.194 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.182+0000 7f3adeffd700 1 --2- 172.21.15.8:0/1930053915 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ae008d300 0x7f3ae008b3e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0000c00 tx=0x7f3ad0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:55.195 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.182+0000 7f56cfc3a700 1 -- 172.21.15.8:0/2667903542 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56c81369f0 con 0x7f56c81260f0 2023-10-05T19:39:55.195 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.182+0000 7f56bdffb700 1 -- 172.21.15.8:0/2667903542 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f56c00070f0 con 0x7f56c81260f0 2023-10-05T19:39:55.195 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.182+0000 7f56bdffb700 1 --2- 172.21.15.8:0/2667903542 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f56ac032430 0x7f56ac0348e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:55.195 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.182+0000 7f56bdffb700 1 -- 172.21.15.8:0/2667903542 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3478+0+0 (secure 0 0 0) 0x7f56c0043df0 con 0x7f56c81260f0 2023-10-05T19:39:55.196 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.182+0000 7f56cd1d5700 1 --2- 172.21.15.8:0/2667903542 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f56ac032430 0x7f56ac0348e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:55.196 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.182+0000 7f56cd1d5700 1 --2- 172.21.15.8:0/2667903542 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f56ac032430 0x7f56ac0348e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f56c40030c0 tx=0x7f56c4016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:55.196 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.186+0000 7f3ac6ffd700 1 -- 172.21.15.8:0/1930053915 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3ad001a460 con 0x7f3ae008d300 2023-10-05T19:39:55.196 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.186+0000 7f3ac6ffd700 1 -- 172.21.15.8:0/1930053915 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad001a5c0 con 0x7f3ae008d300 2023-10-05T19:39:55.197 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.186+0000 7f3adffff700 1 -- 172.21.15.8:0/1930053915 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ae008a520 con 0x7f3ae008d300 2023-10-05T19:39:55.197 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.186+0000 7f3ac6ffd700 1 -- 172.21.15.8:0/1930053915 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3ad001a770 con 0x7f3ae008d300 2023-10-05T19:39:55.197 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.186+0000 7f3adffff700 1 -- 172.21.15.8:0/1930053915 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ae008cb80 con 0x7f3ae008d300 2023-10-05T19:39:55.197 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.186+0000 7f56cfc3a700 1 -- 172.21.15.8:0/2667903542 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56c8118170 con 0x7f56c81260f0 2023-10-05T19:39:55.198 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.186+0000 7f3ac6ffd700 1 -- 172.21.15.8:0/1930053915 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f3ad001a8d0 con 0x7f3ae008d300 2023-10-05T19:39:55.198 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.186+0000 7f3ac6ffd700 1 --2- 172.21.15.8:0/1930053915 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3acc032430 0x7f3acc0348e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:55.198 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.186+0000 7f3ac4ff9700 1 -- 172.21.15.8:0/1930053915 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ab80050b0 con 0x7f3ae008d300 2023-10-05T19:39:55.198 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.186+0000 7f3ade7fc700 1 --2- 172.21.15.8:0/1930053915 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3acc032430 0x7f3acc0348e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:55.198 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.186+0000 7f3ade7fc700 1 --2- 172.21.15.8:0/1930053915 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3acc032430 0x7f3acc0348e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4001660 tx=0x7f3ad4007ae0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:55.199 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.190+0000 7f3ac6ffd700 1 -- 172.21.15.8:0/1930053915 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3478+0+0 (secure 0 0 0) 0x7f3ad0018e00 con 0x7f3ae008d300 2023-10-05T19:39:55.199 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.190+0000 7f56bdffb700 1 -- 172.21.15.8:0/2667903542 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f56c001a360 con 0x7f56c81260f0 2023-10-05T19:39:55.208 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.202+0000 7f3ac6ffd700 1 -- 172.21.15.8:0/1930053915 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f3ad006dbb0 con 0x7f3ae008d300 2023-10-05T19:39:55.306 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.298+0000 7f8aa9f35700 1 -- 172.21.15.8:0/920631429 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8aa4089230 con 0x7f8aa4091340 2023-10-05T19:39:55.306 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.298+0000 7f8a9b7fe700 1 -- 172.21.15.8:0/920631429 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8aa4089230 con 0x7f8aa4091340 2023-10-05T19:39:55.306 INFO:teuthology.orchestra.run.smithi008.stdout:38654705668 2023-10-05T19:39:55.309 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.302+0000 7f8aa9f35700 1 -- 172.21.15.8:0/920631429 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f8a88032390 msgr2=0x7f8a88034840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:55.309 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.302+0000 7f8aa9f35700 1 --2- 172.21.15.8:0/920631429 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f8a88032390 0x7f8a88034840 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c0058d0 tx=0x7f8a8c0079d0 comp rx=0 tx=0).stop 2023-10-05T19:39:55.309 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.302+0000 7f8aa9f35700 1 -- 172.21.15.8:0/920631429 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8aa4091340 msgr2=0x7f8aa4085f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:55.309 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.302+0000 7f8aa9f35700 1 --2- 172.21.15.8:0/920631429 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8aa4091340 0x7f8aa4085f80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8a94000c00 tx=0x7f8a94016000 comp rx=0 tx=0).stop 2023-10-05T19:39:55.309 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.302+0000 7f8aa9f35700 1 -- 172.21.15.8:0/920631429 shutdown_connections 2023-10-05T19:39:55.309 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.302+0000 7f8aa9f35700 1 --2- 172.21.15.8:0/920631429 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f8a88032390 0x7f8a88034840 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:55.309 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.302+0000 7f8aa9f35700 1 --2- 172.21.15.8:0/920631429 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8aa4091340 0x7f8aa4085f80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:55.310 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.302+0000 7f8aa9f35700 1 -- 172.21.15.8:0/920631429 >> 172.21.15.8:0/920631429 conn(0x7f8aa4096d80 msgr2=0x7f8aa406d910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:55.310 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.302+0000 7f8aa9f35700 1 -- 172.21.15.8:0/920631429 shutdown_connections 2023-10-05T19:39:55.310 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.302+0000 7f8aa9f35700 1 -- 172.21.15.8:0/920631429 wait complete. 2023-10-05T19:39:55.319 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705668 for osd.0 2023-10-05T19:39:55.319 DEBUG:teuthology.parallel:result is None 2023-10-05T19:39:55.350 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.342+0000 7f9464695700 1 -- 172.21.15.8:0/841415857 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f945c04b5c0 con 0x7f945c1261b0 2023-10-05T19:39:55.351 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.342+0000 7f945a7fc700 1 -- 172.21.15.8:0/841415857 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f945001c3c0 con 0x7f945c1261b0 2023-10-05T19:39:55.351 INFO:teuthology.orchestra.run.smithi008.stdout:38654705668 2023-10-05T19:39:55.353 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.346+0000 7f9464695700 1 -- 172.21.15.8:0/841415857 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f94440323b0 msgr2=0x7f9444034860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:55.353 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.346+0000 7f9464695700 1 --2- 172.21.15.8:0/841415857 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f94440323b0 0x7f9444034860 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9454001d90 tx=0x7f9454007ec0 comp rx=0 tx=0).stop 2023-10-05T19:39:55.354 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.346+0000 7f9464695700 1 -- 172.21.15.8:0/841415857 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f945c1261b0 msgr2=0x7f945c08f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:55.354 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.346+0000 7f9464695700 1 --2- 172.21.15.8:0/841415857 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f945c1261b0 0x7f945c08f3e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9450000c00 tx=0x7f9450015000 comp rx=0 tx=0).stop 2023-10-05T19:39:55.354 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.346+0000 7f9464695700 1 -- 172.21.15.8:0/841415857 shutdown_connections 2023-10-05T19:39:55.354 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.346+0000 7f9464695700 1 --2- 172.21.15.8:0/841415857 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f94440323b0 0x7f9444034860 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:55.354 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.346+0000 7f9464695700 1 --2- 172.21.15.8:0/841415857 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f945c1261b0 0x7f945c08f3e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:55.354 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.346+0000 7f9464695700 1 -- 172.21.15.8:0/841415857 >> 172.21.15.8:0/841415857 conn(0x7f945c095970 msgr2=0x7f945c1249b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:55.354 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.346+0000 7f9464695700 1 -- 172.21.15.8:0/841415857 shutdown_connections 2023-10-05T19:39:55.354 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.346+0000 7f9464695700 1 -- 172.21.15.8:0/841415857 wait complete. 2023-10-05T19:39:55.361 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.354+0000 7f56cfc3a700 1 -- 172.21.15.8:0/2667903542 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f56c805ecb0 con 0x7f56c81260f0 2023-10-05T19:39:55.361 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.354+0000 7f56bdffb700 1 -- 172.21.15.8:0/2667903542 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f56c001a540 con 0x7f56c81260f0 2023-10-05T19:39:55.361 INFO:teuthology.orchestra.run.smithi008.stdout:38654705667 2023-10-05T19:39:55.364 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.354+0000 7f56cfc3a700 1 -- 172.21.15.8:0/2667903542 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f56ac032430 msgr2=0x7f56ac0348e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:55.364 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.354+0000 7f56cfc3a700 1 --2- 172.21.15.8:0/2667903542 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f56ac032430 0x7f56ac0348e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f56c40030c0 tx=0x7f56c4016000 comp rx=0 tx=0).stop 2023-10-05T19:39:55.364 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.354+0000 7f56cfc3a700 1 -- 172.21.15.8:0/2667903542 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f56c81260f0 msgr2=0x7f56c81364b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:55.364 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.354+0000 7f56cfc3a700 1 --2- 172.21.15.8:0/2667903542 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f56c81260f0 0x7f56c81364b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f56c000cb80 tx=0x7f56c0001a90 comp rx=0 tx=0).stop 2023-10-05T19:39:55.364 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.354+0000 7f56cfc3a700 1 -- 172.21.15.8:0/2667903542 shutdown_connections 2023-10-05T19:39:55.364 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.354+0000 7f56cfc3a700 1 --2- 172.21.15.8:0/2667903542 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f56ac032430 0x7f56ac0348e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:55.365 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.354+0000 7f56cfc3a700 1 --2- 172.21.15.8:0/2667903542 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f56c81260f0 0x7f56c81364b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:55.365 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.354+0000 7f56cfc3a700 1 -- 172.21.15.8:0/2667903542 >> 172.21.15.8:0/2667903542 conn(0x7f56c8098f60 msgr2=0x7f56c8096db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:55.365 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.354+0000 7f56cfc3a700 1 -- 172.21.15.8:0/2667903542 shutdown_connections 2023-10-05T19:39:55.365 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.354+0000 7f56cfc3a700 1 -- 172.21.15.8:0/2667903542 wait complete. 2023-10-05T19:39:55.366 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705668 for osd.2 2023-10-05T19:39:55.366 DEBUG:teuthology.parallel:result is None 2023-10-05T19:39:55.372 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.362+0000 7f3ac4ff9700 1 -- 172.21.15.8:0/1930053915 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3ab8005360 con 0x7f3ae008d300 2023-10-05T19:39:55.372 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.362+0000 7f3ac6ffd700 1 -- 172.21.15.8:0/1930053915 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3ad0018bd0 con 0x7f3ae008d300 2023-10-05T19:39:55.372 INFO:teuthology.orchestra.run.smithi008.stdout:38654705668 2023-10-05T19:39:55.374 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.366+0000 7f3ac4ff9700 1 -- 172.21.15.8:0/1930053915 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3acc032430 msgr2=0x7f3acc0348e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:55.374 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.366+0000 7f3ac4ff9700 1 --2- 172.21.15.8:0/1930053915 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3acc032430 0x7f3acc0348e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4001660 tx=0x7f3ad4007ae0 comp rx=0 tx=0).stop 2023-10-05T19:39:55.375 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.366+0000 7f3ac4ff9700 1 -- 172.21.15.8:0/1930053915 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ae008d300 msgr2=0x7f3ae008b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:55.375 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.366+0000 7f3ac4ff9700 1 --2- 172.21.15.8:0/1930053915 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ae008d300 0x7f3ae008b3e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0000c00 tx=0x7f3ad0016000 comp rx=0 tx=0).stop 2023-10-05T19:39:55.375 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.366+0000 7f3ac4ff9700 1 -- 172.21.15.8:0/1930053915 shutdown_connections 2023-10-05T19:39:55.375 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.366+0000 7f3ac4ff9700 1 --2- 172.21.15.8:0/1930053915 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3acc032430 0x7f3acc0348e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:55.375 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.366+0000 7f3ac4ff9700 1 --2- 172.21.15.8:0/1930053915 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ae008d300 0x7f3ae008b3e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:55.375 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.366+0000 7f3ac4ff9700 1 -- 172.21.15.8:0/1930053915 >> 172.21.15.8:0/1930053915 conn(0x7f3ae00994a0 msgr2=0x7f3ae006e850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:55.375 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.366+0000 7f3ac4ff9700 1 -- 172.21.15.8:0/1930053915 shutdown_connections 2023-10-05T19:39:55.375 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.366+0000 7f3ac4ff9700 1 -- 172.21.15.8:0/1930053915 wait complete. 2023-10-05T19:39:55.376 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705667 got 38654705667 for osd.1 2023-10-05T19:39:55.376 DEBUG:teuthology.parallel:result is None 2023-10-05T19:39:55.385 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705668 for osd.3 2023-10-05T19:39:55.385 DEBUG:teuthology.parallel:result is None 2023-10-05T19:39:55.385 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-05T19:39:55.385 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-05T19:39:55.535 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.526+0000 7fa6ea722700 1 Processor -- start 2023-10-05T19:39:55.536 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.526+0000 7fa6ea722700 1 -- start start 2023-10-05T19:39:55.536 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.526+0000 7fa6ea722700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fa6e40907a0 0x7fa6e4090b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:55.536 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.526+0000 7fa6ea722700 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6e4091ad0 con 0x7fa6e40910b0 2023-10-05T19:39:55.536 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.526+0000 7fa6ea722700 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6e4094a40 con 0x7fa6e40907a0 2023-10-05T19:39:55.537 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.526+0000 7fa6e8f1f700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7fa6e40910b0 0x7fa6e4126370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:42816/0 (socket says 172.21.15.8:42816) 2023-10-05T19:39:55.537 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.526+0000 7fa6e8f1f700 1 -- 172.21.15.8:0/310382258 learned_addr learned my addr 172.21.15.8:0/310382258 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-05T19:39:55.537 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.526+0000 7fa6e9720700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fa6e40907a0 0x7fa6e4090b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:55.537 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.530+0000 7fa6dbfff700 1 -- 172.21.15.8:0/310382258 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2853976890 0 0) 0x7fa6e4091ad0 con 0x7fa6e40910b0 2023-10-05T19:39:55.537 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.530+0000 7fa6dbfff700 1 -- 172.21.15.8:0/310382258 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6d0003570 con 0x7fa6e40910b0 2023-10-05T19:39:55.537 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.530+0000 7fa6dbfff700 1 -- 172.21.15.8:0/310382258 <== mon.0 v1:172.21.15.8:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3542265313 0 0) 0x7fa6d0003570 con 0x7fa6e40910b0 2023-10-05T19:39:55.537 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.530+0000 7fa6e9720700 1 -- 172.21.15.8:0/310382258 >> v1:172.21.15.8:6789/0 conn(0x7fa6e40910b0 legacy=0x7fa6e4126370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:55.537 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.530+0000 7fa6e9720700 1 -- 172.21.15.8:0/310382258 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e4091d00 con 0x7fa6e40907a0 2023-10-05T19:39:55.538 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.530+0000 7fa6e9720700 1 --2- 172.21.15.8:0/310382258 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa6e40907a0 0x7fa6e4090b70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4009860 tx=0x7fa6d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81f167c317e31833 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:55.539 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.530+0000 7fa6dbfff700 1 -- 172.21.15.8:0/310382258 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6e00015b0 con 0x7fa6e40907a0 2023-10-05T19:39:55.539 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.530+0000 7fa6dbfff700 1 -- 172.21.15.8:0/310382258 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6d4011630 con 0x7fa6e40907a0 2023-10-05T19:39:55.539 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.530+0000 7fa6dbfff700 1 -- 172.21.15.8:0/310382258 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6d40117c0 con 0x7fa6e40907a0 2023-10-05T19:39:55.539 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.530+0000 7fa6ea722700 1 -- 172.21.15.8:0/310382258 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa6e40907a0 msgr2=0x7fa6e4090b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:55.539 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.530+0000 7fa6ea722700 1 --2- 172.21.15.8:0/310382258 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa6e40907a0 0x7fa6e4090b70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4009860 tx=0x7fa6d40091a0 comp rx=0 tx=0).stop 2023-10-05T19:39:55.539 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.530+0000 7fa6ea722700 1 -- 172.21.15.8:0/310382258 shutdown_connections 2023-10-05T19:39:55.539 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.530+0000 7fa6ea722700 1 --2- 172.21.15.8:0/310382258 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa6e40907a0 0x7fa6e4090b70 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:55.539 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.530+0000 7fa6ea722700 1 -- 172.21.15.8:0/310382258 >> 172.21.15.8:0/310382258 conn(0x7fa6e40981b0 msgr2=0x7fa6e40985b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:55.539 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.530+0000 7fa6ea722700 1 -- 172.21.15.8:0/310382258 shutdown_connections 2023-10-05T19:39:55.539 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.530+0000 7fa6ea722700 1 -- 172.21.15.8:0/310382258 wait complete. 2023-10-05T19:39:55.541 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.530+0000 7fa6ea722700 1 Processor -- start 2023-10-05T19:39:55.541 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.530+0000 7fa6ea722700 1 -- start start 2023-10-05T19:39:55.541 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.530+0000 7fa6ea722700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa6e40910b0 0x7fa6e408bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:55.541 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.530+0000 7fa6ea722700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e408eeb0 con 0x7fa6e40910b0 2023-10-05T19:39:55.541 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.530+0000 7fa6e9720700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa6e40910b0 0x7fa6e408bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:55.541 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.534+0000 7fa6e9720700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa6e40910b0 0x7fa6e408bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38772/0 (socket says 172.21.15.8:38772) 2023-10-05T19:39:55.541 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.534+0000 7fa6e9720700 1 -- 172.21.15.8:0/508615927 learned_addr learned my addr 172.21.15.8:0/508615927 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-05T19:39:55.542 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.534+0000 7fa6e9720700 1 -- 172.21.15.8:0/508615927 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d4009420 con 0x7fa6e40910b0 2023-10-05T19:39:55.542 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.534+0000 7fa6e9720700 1 --2- 172.21.15.8:0/508615927 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa6e40910b0 0x7fa6e408bb80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa6d40030c0 tx=0x7fa6d4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:55.542 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.534+0000 7fa6d97fa700 1 -- 172.21.15.8:0/508615927 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6d401cca0 con 0x7fa6e40910b0 2023-10-05T19:39:55.542 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.534+0000 7fa6d97fa700 1 -- 172.21.15.8:0/508615927 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6d401ce20 con 0x7fa6e40910b0 2023-10-05T19:39:55.542 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.534+0000 7fa6ea722700 1 -- 172.21.15.8:0/508615927 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6e408d3c0 con 0x7fa6e40910b0 2023-10-05T19:39:55.542 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.534+0000 7fa6d97fa700 1 -- 172.21.15.8:0/508615927 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6d401a450 con 0x7fa6e40910b0 2023-10-05T19:39:55.542 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.534+0000 7fa6ea722700 1 -- 172.21.15.8:0/508615927 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6e408e810 con 0x7fa6e40910b0 2023-10-05T19:39:55.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.534+0000 7fa6d97fa700 1 -- 172.21.15.8:0/508615927 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fa6d401a640 con 0x7fa6e40910b0 2023-10-05T19:39:55.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.534+0000 7fa6ea722700 1 -- 172.21.15.8:0/508615927 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6e411d3c0 con 0x7fa6e40910b0 2023-10-05T19:39:55.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.534+0000 7fa6d97fa700 1 --2- 172.21.15.8:0/508615927 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fa6c80323e0 0x7fa6c8034890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:55.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.534+0000 7fa6d97fa700 1 -- 172.21.15.8:0/508615927 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3478+0+0 (secure 0 0 0) 0x7fa6d40458d0 con 0x7fa6e40910b0 2023-10-05T19:39:55.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.534+0000 7fa6e8f1f700 1 --2- 172.21.15.8:0/508615927 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fa6c80323e0 0x7fa6c8034890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:55.544 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.534+0000 7fa6e8f1f700 1 --2- 172.21.15.8:0/508615927 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fa6c80323e0 0x7fa6c8034890 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0001380 tx=0x7fa6e0001a80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:55.549 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.538+0000 7fa6d97fa700 1 -- 172.21.15.8:0/508615927 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fa6d40243e0 con 0x7fa6e40910b0 2023-10-05T19:39:55.707 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.698+0000 7fa6ea722700 1 -- 172.21.15.8:0/508615927 --> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa6e4132960 con 0x7fa6c80323e0 2023-10-05T19:39:55.710 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.706+0000 7fa6d97fa700 1 -- 172.21.15.8:0/508615927 <== mgr.4100 v2:172.21.15.12:6824/29788 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+34632 (secure 0 0 0) 0x7fa6e00070d0 con 0x7fa6c80323e0 2023-10-05T19:39:55.711 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-05T19:39:55.712 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.706+0000 7fa6ea722700 1 -- 172.21.15.8:0/508615927 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fa6c80323e0 msgr2=0x7fa6c8034890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:55.713 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.706+0000 7fa6ea722700 1 --2- 172.21.15.8:0/508615927 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fa6c80323e0 0x7fa6c8034890 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0001380 tx=0x7fa6e0001a80 comp rx=0 tx=0).stop 2023-10-05T19:39:55.713 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.706+0000 7fa6ea722700 1 -- 172.21.15.8:0/508615927 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa6e40910b0 msgr2=0x7fa6e408bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:55.713 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.706+0000 7fa6ea722700 1 --2- 172.21.15.8:0/508615927 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa6e40910b0 0x7fa6e408bb80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa6d40030c0 tx=0x7fa6d4016000 comp rx=0 tx=0).stop 2023-10-05T19:39:55.713 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.706+0000 7fa6ea722700 1 -- 172.21.15.8:0/508615927 shutdown_connections 2023-10-05T19:39:55.713 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.706+0000 7fa6ea722700 1 --2- 172.21.15.8:0/508615927 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fa6c80323e0 0x7fa6c8034890 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:55.713 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.706+0000 7fa6ea722700 1 --2- 172.21.15.8:0/508615927 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa6e40910b0 0x7fa6e408bb80 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:55.713 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.706+0000 7fa6ea722700 1 -- 172.21.15.8:0/508615927 >> 172.21.15.8:0/508615927 conn(0x7fa6e40981b0 msgr2=0x7fa6e406d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:55.713 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.706+0000 7fa6ea722700 1 -- 172.21.15.8:0/508615927 shutdown_connections 2023-10-05T19:39:55.713 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.706+0000 7fa6ea722700 1 -- 172.21.15.8:0/508615927 wait complete. 2023-10-05T19:39:55.714 INFO:teuthology.orchestra.run.smithi008.stderr:dumped all 2023-10-05T19:39:55.725 INFO:teuthology.orchestra.run.smithi008.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2023-10-05T19:39:53.832159+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1194888,"kb_used_data":1640,"kb_used_omap":0,"kb_used_meta":1193216,"kb_avail":376292472,"statfs":{"total":386547056640,"available":385323491328,"internally_reserved":0,"allocated":1679360,"data_stored":1327976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1221853184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.272015"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-10-05T19:39:52.652830+0000","last_change":"2023-10-05T19:39:52.652830+0000","last_active":"2023-10-05T19:39:52.652830+0000","last_peered":"2023-10-05T19:39:52.652830+0000","last_clean":"2023-10-05T19:39:52.652830+0000","last_became_active":"2023-10-05T19:39:52.652468+0000","last_became_peered":"2023-10-05T19:39:52.652468+0000","last_unstale":"2023-10-05T19:39:52.652830+0000","last_undegraded":"2023-10-05T19:39:52.652830+0000","last_fullsized":"2023-10-05T19:39:52.652830+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T19:39:51.552415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T19:39:51.552415+0000","last_clean_scrub_stamp":"2023-10-05T19:39:51.552415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T00:00:58.342902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-10-05T19:39:52.574923+0000","last_change":"2023-10-05T19:39:52.574923+0000","last_active":"2023-10-05T19:39:52.574923+0000","last_peered":"2023-10-05T19:39:52.574923+0000","last_clean":"2023-10-05T19:39:52.574923+0000","last_became_active":"2023-10-05T19:39:52.574513+0000","last_became_peered":"2023-10-05T19:39:52.574513+0000","last_unstale":"2023-10-05T19:39:52.574923+0000","last_undegraded":"2023-10-05T19:39:52.574923+0000","last_fullsized":"2023-10-05T19:39:52.574923+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T19:39:51.552415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T19:39:51.552415+0000","last_clean_scrub_stamp":"2023-10-05T19:39:51.552415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T00:24:41.321264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-10-05T19:39:52.573892+0000","last_change":"2023-10-05T19:39:52.573892+0000","last_active":"2023-10-05T19:39:52.573892+0000","last_peered":"2023-10-05T19:39:52.573892+0000","last_clean":"2023-10-05T19:39:52.573892+0000","last_became_active":"2023-10-05T19:39:52.573320+0000","last_became_peered":"2023-10-05T19:39:52.573320+0000","last_unstale":"2023-10-05T19:39:52.573892+0000","last_undegraded":"2023-10-05T19:39:52.573892+0000","last_fullsized":"2023-10-05T19:39:52.573892+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T19:39:51.552415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T19:39:51.552415+0000","last_clean_scrub_stamp":"2023-10-05T19:39:51.552415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T00:29:10.852230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-10-05T19:39:52.574310+0000","last_change":"2023-10-05T19:39:52.574310+0000","last_active":"2023-10-05T19:39:52.574310+0000","last_peered":"2023-10-05T19:39:52.574310+0000","last_clean":"2023-10-05T19:39:52.574310+0000","last_became_active":"2023-10-05T19:39:52.573961+0000","last_became_peered":"2023-10-05T19:39:52.573961+0000","last_unstale":"2023-10-05T19:39:52.574310+0000","last_undegraded":"2023-10-05T19:39:52.574310+0000","last_fullsized":"2023-10-05T19:39:52.574310+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T19:39:51.552415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T19:39:51.552415+0000","last_clean_scrub_stamp":"2023-10-05T19:39:51.552415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-06T20:58:32.361291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-10-05T19:39:52.573845+0000","last_change":"2023-10-05T19:39:52.573845+0000","last_active":"2023-10-05T19:39:52.573845+0000","last_peered":"2023-10-05T19:39:52.573845+0000","last_clean":"2023-10-05T19:39:52.573845+0000","last_became_active":"2023-10-05T19:39:52.573263+0000","last_became_peered":"2023-10-05T19:39:52.573263+0000","last_unstale":"2023-10-05T19:39:52.573845+0000","last_undegraded":"2023-10-05T19:39:52.573845+0000","last_fullsized":"2023-10-05T19:39:52.573845+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T19:39:51.552415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T19:39:51.552415+0000","last_clean_scrub_stamp":"2023-10-05T19:39:51.552415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:43.884038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-10-05T19:39:52.575798+0000","last_change":"2023-10-05T19:39:52.574176+0000","last_active":"2023-10-05T19:39:52.575798+0000","last_peered":"2023-10-05T19:39:52.575798+0000","last_clean":"2023-10-05T19:39:52.575798+0000","last_became_active":"2023-10-05T19:39:52.573645+0000","last_became_peered":"2023-10-05T19:39:52.573645+0000","last_unstale":"2023-10-05T19:39:52.575798+0000","last_undegraded":"2023-10-05T19:39:52.575798+0000","last_fullsized":"2023-10-05T19:39:52.575798+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T19:39:51.552415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T19:39:51.552415+0000","last_clean_scrub_stamp":"2023-10-05T19:39:51.552415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T01:56:30.515404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-10-05T19:39:52.574735+0000","last_change":"2023-10-05T19:39:52.574735+0000","last_active":"2023-10-05T19:39:52.574735+0000","last_peered":"2023-10-05T19:39:52.574735+0000","last_clean":"2023-10-05T19:39:52.574735+0000","last_became_active":"2023-10-05T19:39:52.573854+0000","last_became_peered":"2023-10-05T19:39:52.573854+0000","last_unstale":"2023-10-05T19:39:52.574735+0000","last_undegraded":"2023-10-05T19:39:52.574735+0000","last_fullsized":"2023-10-05T19:39:52.574735+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T19:39:51.552415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T19:39:51.552415+0000","last_clean_scrub_stamp":"2023-10-05T19:39:51.552415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:41:37.507792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-10-05T19:39:52.574737+0000","last_change":"2023-10-05T19:39:52.574737+0000","last_active":"2023-10-05T19:39:52.574737+0000","last_peered":"2023-10-05T19:39:52.574737+0000","last_clean":"2023-10-05T19:39:52.574737+0000","last_became_active":"2023-10-05T19:39:52.573871+0000","last_became_peered":"2023-10-05T19:39:52.573871+0000","last_unstale":"2023-10-05T19:39:52.574737+0000","last_undegraded":"2023-10-05T19:39:52.574737+0000","last_fullsized":"2023-10-05T19:39:52.574737+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T19:39:51.552415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T19:39:51.552415+0000","last_clean_scrub_stamp":"2023-10-05T19:39:51.552415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:00:19.032737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"11'117","reported_seq":161,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-10-05T19:39:51.563051+0000","last_change":"2023-10-05T19:39:47.249340+0000","last_active":"2023-10-05T19:39:51.563051+0000","last_peered":"2023-10-05T19:39:51.563051+0000","last_clean":"2023-10-05T19:39:51.563051+0000","last_became_active":"2023-10-05T19:39:47.246681+0000","last_became_peered":"2023-10-05T19:39:47.246681+0000","last_unstale":"2023-10-05T19:39:51.563051+0000","last_undegraded":"2023-10-05T19:39:51.563051+0000","last_fullsized":"2023-10-05T19:39:51.563051+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T19:39:45.900395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T19:39:45.900395+0000","last_clean_scrub_stamp":"2023-10-05T19:39:45.900395+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:55.846109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705668,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297408,"kb_used_data":120,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074432,"statfs":{"total":96636764160,"available":96332218368,"internally_reserved":0,"allocated":122880,"data_stored":36810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0069999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]}]},{"osd":2,"up_from":9,"seq":38654705668,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297988,"kb_used_data":700,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073852,"statfs":{"total":96636764160,"available":96331624448,"internally_reserved":0,"allocated":716800,"data_stored":627178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]}]},{"osd":1,"up_from":9,"seq":38654705667,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297988,"kb_used_data":700,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073852,"statfs":{"total":96636764160,"available":96331624448,"internally_reserved":0,"allocated":716800,"data_stored":627178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.784}]}]},{"osd":0,"up_from":9,"seq":38654705668,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301504,"kb_used_data":120,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070336,"statfs":{"total":96636764160,"available":96328024064,"internally_reserved":0,"allocated":122880,"data_stored":36810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-05T19:39:55.726 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-05T19:39:55.829 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.822+0000 7f05832a1700 1 Processor -- start 2023-10-05T19:39:55.829 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.822+0000 7f05832a1700 1 -- start start 2023-10-05T19:39:55.830 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.822+0000 7f05832a1700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f057c06cc50 0x7f057c06d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:55.830 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.822+0000 7f05832a1700 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f057c06b850 con 0x7f057c06d560 2023-10-05T19:39:55.830 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.822+0000 7f05832a1700 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f057c06a670 con 0x7f057c06cc50 2023-10-05T19:39:55.831 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.822+0000 7f058229f700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f057c06cc50 0x7f057c06d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:55.831 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.822+0000 7f0581a9e700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f057c06d560 0x7f057c129360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:42828/0 (socket says 172.21.15.8:42828) 2023-10-05T19:39:55.831 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.822+0000 7f058229f700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f057c06cc50 0x7f057c06d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38780/0 (socket says 172.21.15.8:38780) 2023-10-05T19:39:55.831 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.822+0000 7f058229f700 1 -- 172.21.15.8:0/1107635447 learned_addr learned my addr 172.21.15.8:0/1107635447 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-05T19:39:55.832 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.822+0000 7f058229f700 1 -- 172.21.15.8:0/1107635447 >> v1:172.21.15.8:6789/0 conn(0x7f057c06d560 legacy=0x7f057c129360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:55.832 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.822+0000 7f058229f700 1 -- 172.21.15.8:0/1107635447 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f057c06ab90 con 0x7f057c06cc50 2023-10-05T19:39:55.832 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.822+0000 7f058229f700 1 --2- 172.21.15.8:0/1107635447 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f057c06cc50 0x7f057c06d020 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0578009ea0 tx=0x7f0578009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77e1fd6acc02c525 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:55.832 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.822+0000 7f058129d700 1 -- 172.21.15.8:0/1107635447 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f057800e070 con 0x7f057c06cc50 2023-10-05T19:39:55.832 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.822+0000 7f058129d700 1 -- 172.21.15.8:0/1107635447 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f057800ba00 con 0x7f057c06cc50 2023-10-05T19:39:55.832 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.822+0000 7f058129d700 1 -- 172.21.15.8:0/1107635447 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f05780092e0 con 0x7f057c06cc50 2023-10-05T19:39:55.833 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.826+0000 7f05832a1700 1 -- 172.21.15.8:0/1107635447 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f057c06cc50 msgr2=0x7f057c06d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:55.833 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.826+0000 7f05832a1700 1 --2- 172.21.15.8:0/1107635447 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f057c06cc50 0x7f057c06d020 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0578009ea0 tx=0x7f0578009840 comp rx=0 tx=0).stop 2023-10-05T19:39:55.833 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.826+0000 7f05832a1700 1 -- 172.21.15.8:0/1107635447 shutdown_connections 2023-10-05T19:39:55.833 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.826+0000 7f05832a1700 1 --2- 172.21.15.8:0/1107635447 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f057c06cc50 0x7f057c06d020 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:55.833 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.826+0000 7f05832a1700 1 -- 172.21.15.8:0/1107635447 >> 172.21.15.8:0/1107635447 conn(0x7f057c09a530 msgr2=0x7f057c09a930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:55.833 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.826+0000 7f05832a1700 1 -- 172.21.15.8:0/1107635447 shutdown_connections 2023-10-05T19:39:55.833 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.826+0000 7f05832a1700 1 -- 172.21.15.8:0/1107635447 wait complete. 2023-10-05T19:39:55.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.826+0000 7f05832a1700 1 Processor -- start 2023-10-05T19:39:55.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.826+0000 7f05832a1700 1 -- start start 2023-10-05T19:39:55.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.826+0000 7f05832a1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f057c06cc50 0x7f057c1386f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:55.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.826+0000 7f05832a1700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057c11bfc0 con 0x7f057c06cc50 2023-10-05T19:39:55.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.830+0000 7f058229f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f057c06cc50 0x7f057c1386f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:55.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.830+0000 7f058229f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f057c06cc50 0x7f057c1386f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38790/0 (socket says 172.21.15.8:38790) 2023-10-05T19:39:55.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.830+0000 7f058229f700 1 -- 172.21.15.8:0/631068156 learned_addr learned my addr 172.21.15.8:0/631068156 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-05T19:39:55.837 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.830+0000 7f058229f700 1 -- 172.21.15.8:0/631068156 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0578009ae0 con 0x7f057c06cc50 2023-10-05T19:39:55.837 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.830+0000 7f058229f700 1 --2- 172.21.15.8:0/631068156 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f057c06cc50 0x7f057c1386f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f057c06a7b0 tx=0x7f0578017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:55.837 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.830+0000 7f05727fc700 1 -- 172.21.15.8:0/631068156 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f05780092e0 con 0x7f057c06cc50 2023-10-05T19:39:55.837 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.830+0000 7f05727fc700 1 -- 172.21.15.8:0/631068156 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0578012b50 con 0x7f057c06cc50 2023-10-05T19:39:55.838 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.830+0000 7f05727fc700 1 -- 172.21.15.8:0/631068156 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0578012e60 con 0x7f057c06cc50 2023-10-05T19:39:55.838 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.830+0000 7f05832a1700 1 -- 172.21.15.8:0/631068156 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f057c11bd80 con 0x7f057c06cc50 2023-10-05T19:39:55.838 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.830+0000 7f05832a1700 1 -- 172.21.15.8:0/631068156 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f057c11c760 con 0x7f057c06cc50 2023-10-05T19:39:55.838 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.830+0000 7f05727fc700 1 -- 172.21.15.8:0/631068156 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f0578028460 con 0x7f057c06cc50 2023-10-05T19:39:55.839 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.830+0000 7f05832a1700 1 -- 172.21.15.8:0/631068156 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f057c11c940 con 0x7f057c06cc50 2023-10-05T19:39:55.839 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.830+0000 7f05727fc700 1 --2- 172.21.15.8:0/631068156 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f05600323e0 0x7f0560034890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:55.839 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.830+0000 7f05727fc700 1 -- 172.21.15.8:0/631068156 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3478+0+0 (secure 0 0 0) 0x7f0578045390 con 0x7f057c06cc50 2023-10-05T19:39:55.839 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.830+0000 7f0581a9e700 1 --2- 172.21.15.8:0/631068156 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f05600323e0 0x7f0560034890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:55.840 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.830+0000 7f0581a9e700 1 --2- 172.21.15.8:0/631068156 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f05600323e0 0x7f0560034890 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f056c0016a0 tx=0x7f056c007b30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:55.848 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:55.842+0000 7f05727fc700 1 -- 172.21.15.8:0/631068156 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f0578026950 con 0x7f057c06cc50 2023-10-05T19:39:56.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.002+0000 7f05832a1700 1 -- 172.21.15.8:0/631068156 --> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f057c0975c0 con 0x7f05600323e0 2023-10-05T19:39:56.015 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.010+0000 7f05727fc700 1 -- 172.21.15.8:0/631068156 <== mgr.4100 v2:172.21.15.12:6824/29788 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+34636 (secure 0 0 0) 0x7f057c0975c0 con 0x7f05600323e0 2023-10-05T19:39:56.015 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-05T19:39:56.020 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.010+0000 7f05832a1700 1 -- 172.21.15.8:0/631068156 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f05600323e0 msgr2=0x7f0560034890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:56.020 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.010+0000 7f05832a1700 1 --2- 172.21.15.8:0/631068156 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f05600323e0 0x7f0560034890 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f056c0016a0 tx=0x7f056c007b30 comp rx=0 tx=0).stop 2023-10-05T19:39:56.020 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.010+0000 7f05832a1700 1 -- 172.21.15.8:0/631068156 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f057c06cc50 msgr2=0x7f057c1386f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:56.020 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.010+0000 7f05832a1700 1 --2- 172.21.15.8:0/631068156 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f057c06cc50 0x7f057c1386f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f057c06a7b0 tx=0x7f0578017000 comp rx=0 tx=0).stop 2023-10-05T19:39:56.020 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.010+0000 7f05832a1700 1 -- 172.21.15.8:0/631068156 shutdown_connections 2023-10-05T19:39:56.020 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.010+0000 7f05832a1700 1 --2- 172.21.15.8:0/631068156 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f05600323e0 0x7f0560034890 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:56.020 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.010+0000 7f05832a1700 1 --2- 172.21.15.8:0/631068156 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f057c06cc50 0x7f057c1386f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:56.020 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.010+0000 7f05832a1700 1 -- 172.21.15.8:0/631068156 >> 172.21.15.8:0/631068156 conn(0x7f057c09a530 msgr2=0x7f057c096320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:56.021 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.010+0000 7f05832a1700 1 -- 172.21.15.8:0/631068156 shutdown_connections 2023-10-05T19:39:56.021 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.010+0000 7f05832a1700 1 -- 172.21.15.8:0/631068156 wait complete. 2023-10-05T19:39:56.021 INFO:teuthology.orchestra.run.smithi008.stderr:dumped all 2023-10-05T19:39:56.036 INFO:teuthology.orchestra.run.smithi008.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2023-10-05T19:39:55.832823+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1194920,"kb_used_data":1672,"kb_used_omap":0,"kb_used_meta":1193216,"kb_avail":376292440,"statfs":{"total":386547056640,"available":385323458560,"internally_reserved":0,"allocated":1712128,"data_stored":1344008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1221853184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.272679"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-10-05T19:39:52.654356+0000","last_change":"2023-10-05T19:39:52.652830+0000","last_active":"2023-10-05T19:39:52.654356+0000","last_peered":"2023-10-05T19:39:52.654356+0000","last_clean":"2023-10-05T19:39:52.654356+0000","last_became_active":"2023-10-05T19:39:52.652468+0000","last_became_peered":"2023-10-05T19:39:52.652468+0000","last_unstale":"2023-10-05T19:39:52.654356+0000","last_undegraded":"2023-10-05T19:39:52.654356+0000","last_fullsized":"2023-10-05T19:39:52.654356+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T19:39:51.552415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T19:39:51.552415+0000","last_clean_scrub_stamp":"2023-10-05T19:39:51.552415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T00:00:58.342902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-10-05T19:39:52.578822+0000","last_change":"2023-10-05T19:39:52.574923+0000","last_active":"2023-10-05T19:39:52.578822+0000","last_peered":"2023-10-05T19:39:52.578822+0000","last_clean":"2023-10-05T19:39:52.578822+0000","last_became_active":"2023-10-05T19:39:52.574513+0000","last_became_peered":"2023-10-05T19:39:52.574513+0000","last_unstale":"2023-10-05T19:39:52.578822+0000","last_undegraded":"2023-10-05T19:39:52.578822+0000","last_fullsized":"2023-10-05T19:39:52.578822+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T19:39:51.552415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T19:39:51.552415+0000","last_clean_scrub_stamp":"2023-10-05T19:39:51.552415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T00:24:41.321264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-10-05T19:39:52.575560+0000","last_change":"2023-10-05T19:39:52.573892+0000","last_active":"2023-10-05T19:39:52.575560+0000","last_peered":"2023-10-05T19:39:52.575560+0000","last_clean":"2023-10-05T19:39:52.575560+0000","last_became_active":"2023-10-05T19:39:52.573320+0000","last_became_peered":"2023-10-05T19:39:52.573320+0000","last_unstale":"2023-10-05T19:39:52.575560+0000","last_undegraded":"2023-10-05T19:39:52.575560+0000","last_fullsized":"2023-10-05T19:39:52.575560+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T19:39:51.552415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T19:39:51.552415+0000","last_clean_scrub_stamp":"2023-10-05T19:39:51.552415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T00:29:10.852230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-10-05T19:39:52.578805+0000","last_change":"2023-10-05T19:39:52.574310+0000","last_active":"2023-10-05T19:39:52.578805+0000","last_peered":"2023-10-05T19:39:52.578805+0000","last_clean":"2023-10-05T19:39:52.578805+0000","last_became_active":"2023-10-05T19:39:52.573961+0000","last_became_peered":"2023-10-05T19:39:52.573961+0000","last_unstale":"2023-10-05T19:39:52.578805+0000","last_undegraded":"2023-10-05T19:39:52.578805+0000","last_fullsized":"2023-10-05T19:39:52.578805+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T19:39:51.552415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T19:39:51.552415+0000","last_clean_scrub_stamp":"2023-10-05T19:39:51.552415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-06T20:58:32.361291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-10-05T19:39:52.575533+0000","last_change":"2023-10-05T19:39:52.573845+0000","last_active":"2023-10-05T19:39:52.575533+0000","last_peered":"2023-10-05T19:39:52.575533+0000","last_clean":"2023-10-05T19:39:52.575533+0000","last_became_active":"2023-10-05T19:39:52.573263+0000","last_became_peered":"2023-10-05T19:39:52.573263+0000","last_unstale":"2023-10-05T19:39:52.575533+0000","last_undegraded":"2023-10-05T19:39:52.575533+0000","last_fullsized":"2023-10-05T19:39:52.575533+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T19:39:51.552415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T19:39:51.552415+0000","last_clean_scrub_stamp":"2023-10-05T19:39:51.552415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:43.884038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-10-05T19:39:52.575798+0000","last_change":"2023-10-05T19:39:52.574176+0000","last_active":"2023-10-05T19:39:52.575798+0000","last_peered":"2023-10-05T19:39:52.575798+0000","last_clean":"2023-10-05T19:39:52.575798+0000","last_became_active":"2023-10-05T19:39:52.573645+0000","last_became_peered":"2023-10-05T19:39:52.573645+0000","last_unstale":"2023-10-05T19:39:52.575798+0000","last_undegraded":"2023-10-05T19:39:52.575798+0000","last_fullsized":"2023-10-05T19:39:52.575798+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T19:39:51.552415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T19:39:51.552415+0000","last_clean_scrub_stamp":"2023-10-05T19:39:51.552415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T01:56:30.515404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-10-05T19:39:52.576205+0000","last_change":"2023-10-05T19:39:52.574735+0000","last_active":"2023-10-05T19:39:52.576205+0000","last_peered":"2023-10-05T19:39:52.576205+0000","last_clean":"2023-10-05T19:39:52.576205+0000","last_became_active":"2023-10-05T19:39:52.573854+0000","last_became_peered":"2023-10-05T19:39:52.573854+0000","last_unstale":"2023-10-05T19:39:52.576205+0000","last_undegraded":"2023-10-05T19:39:52.576205+0000","last_fullsized":"2023-10-05T19:39:52.576205+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T19:39:51.552415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T19:39:51.552415+0000","last_clean_scrub_stamp":"2023-10-05T19:39:51.552415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:41:37.507792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-10-05T19:39:52.576212+0000","last_change":"2023-10-05T19:39:52.574737+0000","last_active":"2023-10-05T19:39:52.576212+0000","last_peered":"2023-10-05T19:39:52.576212+0000","last_clean":"2023-10-05T19:39:52.576212+0000","last_became_active":"2023-10-05T19:39:52.573871+0000","last_became_peered":"2023-10-05T19:39:52.573871+0000","last_unstale":"2023-10-05T19:39:52.576212+0000","last_undegraded":"2023-10-05T19:39:52.576212+0000","last_fullsized":"2023-10-05T19:39:52.576212+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T19:39:51.552415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T19:39:51.552415+0000","last_clean_scrub_stamp":"2023-10-05T19:39:51.552415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:00:19.032737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"11'117","reported_seq":162,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-10-05T19:39:52.649630+0000","last_change":"2023-10-05T19:39:47.249340+0000","last_active":"2023-10-05T19:39:52.649630+0000","last_peered":"2023-10-05T19:39:52.649630+0000","last_clean":"2023-10-05T19:39:52.649630+0000","last_became_active":"2023-10-05T19:39:47.246681+0000","last_became_peered":"2023-10-05T19:39:47.246681+0000","last_unstale":"2023-10-05T19:39:52.649630+0000","last_undegraded":"2023-10-05T19:39:52.649630+0000","last_fullsized":"2023-10-05T19:39:52.649630+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-05T19:39:45.900395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-05T19:39:45.900395+0000","last_clean_scrub_stamp":"2023-10-05T19:39:45.900395+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:55.846109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705669,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297416,"kb_used_data":128,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074424,"statfs":{"total":96636764160,"available":96332210176,"internally_reserved":0,"allocated":131072,"data_stored":40818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]}]},{"osd":2,"up_from":9,"seq":38654705669,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297996,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073844,"statfs":{"total":96636764160,"available":96331616256,"internally_reserved":0,"allocated":724992,"data_stored":631186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]}]},{"osd":1,"up_from":9,"seq":38654705668,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297996,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073844,"statfs":{"total":96636764160,"available":96331616256,"internally_reserved":0,"allocated":724992,"data_stored":631186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.784}]}]},{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301512,"kb_used_data":128,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070328,"statfs":{"total":96636764160,"available":96328015872,"internally_reserved":0,"allocated":131072,"data_stored":40818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-05T19:39:56.037 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-05T19:39:56.037 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-05T19:39:56.037 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-05T19:39:56.037 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-05T19:39:56.138 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.130+0000 7f79e3fff700 1 Processor -- start 2023-10-05T19:39:56.138 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.130+0000 7f79e3fff700 1 -- start start 2023-10-05T19:39:56.138 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.130+0000 7f79e3fff700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f79e406cfa0 0x7f79e406d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:56.138 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.130+0000 7f79e3fff700 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79e406b580 con 0x7f79e406d8b0 2023-10-05T19:39:56.138 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.130+0000 7f79e3fff700 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79e406e370 con 0x7f79e406cfa0 2023-10-05T19:39:56.138 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.130+0000 7f79e2ffd700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f79e406cfa0 0x7f79e406d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:56.138 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.130+0000 7f79e27fc700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f79e406d8b0 0x7f79e41292c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:42832/0 (socket says 172.21.15.8:42832) 2023-10-05T19:39:56.139 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.130+0000 7f79e2ffd700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f79e406cfa0 0x7f79e406d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38798/0 (socket says 172.21.15.8:38798) 2023-10-05T19:39:56.139 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.130+0000 7f79e2ffd700 1 -- 172.21.15.8:0/2081916295 learned_addr learned my addr 172.21.15.8:0/2081916295 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-05T19:39:56.139 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.130+0000 7f79e2ffd700 1 -- 172.21.15.8:0/2081916295 >> v1:172.21.15.8:6789/0 conn(0x7f79e406d8b0 legacy=0x7f79e41292c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:56.139 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.130+0000 7f79e2ffd700 1 -- 172.21.15.8:0/2081916295 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e406deb0 con 0x7f79e406cfa0 2023-10-05T19:39:56.140 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.130+0000 7f79e2ffd700 1 --2- 172.21.15.8:0/2081916295 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f79e406cfa0 0x7f79e406d370 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f79dc009ea0 tx=0x7f79dc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42025973c8754031 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:56.140 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.130+0000 7f79e1ffb700 1 -- 172.21.15.8:0/2081916295 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f79dc00e070 con 0x7f79e406cfa0 2023-10-05T19:39:56.140 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.130+0000 7f79e1ffb700 1 -- 172.21.15.8:0/2081916295 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79dc00ba00 con 0x7f79e406cfa0 2023-10-05T19:39:56.140 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.130+0000 7f79e1ffb700 1 -- 172.21.15.8:0/2081916295 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f79dc0092e0 con 0x7f79e406cfa0 2023-10-05T19:39:56.141 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.130+0000 7f79e3fff700 1 -- 172.21.15.8:0/2081916295 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f79e406cfa0 msgr2=0x7f79e406d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:56.141 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.130+0000 7f79e3fff700 1 --2- 172.21.15.8:0/2081916295 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f79e406cfa0 0x7f79e406d370 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f79dc009ea0 tx=0x7f79dc009840 comp rx=0 tx=0).stop 2023-10-05T19:39:56.141 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.134+0000 7f79e3fff700 1 -- 172.21.15.8:0/2081916295 shutdown_connections 2023-10-05T19:39:56.141 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.134+0000 7f79e3fff700 1 --2- 172.21.15.8:0/2081916295 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f79e406cfa0 0x7f79e406d370 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:56.141 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.134+0000 7f79e3fff700 1 -- 172.21.15.8:0/2081916295 >> 172.21.15.8:0/2081916295 conn(0x7f79e409a740 msgr2=0x7f79e409ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:56.141 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.134+0000 7f79e3fff700 1 -- 172.21.15.8:0/2081916295 shutdown_connections 2023-10-05T19:39:56.141 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.134+0000 7f79e3fff700 1 -- 172.21.15.8:0/2081916295 wait complete. 2023-10-05T19:39:56.142 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.134+0000 7f79e3fff700 1 Processor -- start 2023-10-05T19:39:56.142 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.134+0000 7f79e3fff700 1 -- start start 2023-10-05T19:39:56.143 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.134+0000 7f79e3fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f79e406cfa0 0x7f79e412b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:56.143 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.134+0000 7f79e3fff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e4120980 con 0x7f79e406cfa0 2023-10-05T19:39:56.143 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.134+0000 7f79e2ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f79e406cfa0 0x7f79e412b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:56.143 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.134+0000 7f79e2ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f79e406cfa0 0x7f79e412b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38814/0 (socket says 172.21.15.8:38814) 2023-10-05T19:39:56.143 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.134+0000 7f79e2ffd700 1 -- 172.21.15.8:0/3339135858 learned_addr learned my addr 172.21.15.8:0/3339135858 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-05T19:39:56.144 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.134+0000 7f79e2ffd700 1 -- 172.21.15.8:0/3339135858 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79dc009ae0 con 0x7f79e406cfa0 2023-10-05T19:39:56.144 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.134+0000 7f79e2ffd700 1 --2- 172.21.15.8:0/3339135858 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f79e406cfa0 0x7f79e412b0a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f79dc009540 tx=0x7f79dc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:56.144 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.134+0000 7f79daffd700 1 -- 172.21.15.8:0/3339135858 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f79dc012bd0 con 0x7f79e406cfa0 2023-10-05T19:39:56.144 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.134+0000 7f79daffd700 1 -- 172.21.15.8:0/3339135858 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79dc012d70 con 0x7f79e406cfa0 2023-10-05T19:39:56.144 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.134+0000 7f79e3fff700 1 -- 172.21.15.8:0/3339135858 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79e4120700 con 0x7f79e406cfa0 2023-10-05T19:39:56.145 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.134+0000 7f79daffd700 1 -- 172.21.15.8:0/3339135858 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f79dc01d420 con 0x7f79e406cfa0 2023-10-05T19:39:56.145 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.138+0000 7f79e3fff700 1 -- 172.21.15.8:0/3339135858 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79e4120bb0 con 0x7f79e406cfa0 2023-10-05T19:39:56.145 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.138+0000 7f79daffd700 1 -- 172.21.15.8:0/3339135858 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f79dc01d5d0 con 0x7f79e406cfa0 2023-10-05T19:39:56.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.138+0000 7f79e3fff700 1 -- 172.21.15.8:0/3339135858 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79e4095260 con 0x7f79e406cfa0 2023-10-05T19:39:56.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.138+0000 7f79daffd700 1 --2- 172.21.15.8:0/3339135858 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f79c40323e0 0x7f79c4034890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:56.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.138+0000 7f79daffd700 1 -- 172.21.15.8:0/3339135858 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3478+0+0 (secure 0 0 0) 0x7f79dc045060 con 0x7f79e406cfa0 2023-10-05T19:39:56.147 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.138+0000 7f79e27fc700 1 --2- 172.21.15.8:0/3339135858 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f79c40323e0 0x7f79c4034890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:56.147 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.138+0000 7f79e27fc700 1 --2- 172.21.15.8:0/3339135858 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f79c40323e0 0x7f79c4034890 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f79cc0016c0 tx=0x7f79cc007af0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:56.154 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.146+0000 7f79daffd700 1 -- 172.21.15.8:0/3339135858 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f79e4095260 con 0x7f79e406cfa0 2023-10-05T19:39:56.350 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.342+0000 7f79e3fff700 1 -- 172.21.15.8:0/3339135858 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f79e4095260 con 0x7f79e406cfa0 2023-10-05T19:39:56.351 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.342+0000 7f79daffd700 1 -- 172.21.15.8:0/3339135858 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f79e4095260 con 0x7f79e406cfa0 2023-10-05T19:39:56.351 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-05T19:39:56.351 INFO:teuthology.orchestra.run.smithi008.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-05T19:39:56.353 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.346+0000 7f79e3fff700 1 -- 172.21.15.8:0/3339135858 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f79c40323e0 msgr2=0x7f79c4034890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:56.353 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.346+0000 7f79e3fff700 1 --2- 172.21.15.8:0/3339135858 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f79c40323e0 0x7f79c4034890 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f79cc0016c0 tx=0x7f79cc007af0 comp rx=0 tx=0).stop 2023-10-05T19:39:56.353 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.346+0000 7f79e3fff700 1 -- 172.21.15.8:0/3339135858 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f79e406cfa0 msgr2=0x7f79e412b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:56.354 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.346+0000 7f79e3fff700 1 --2- 172.21.15.8:0/3339135858 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f79e406cfa0 0x7f79e412b0a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f79dc009540 tx=0x7f79dc017000 comp rx=0 tx=0).stop 2023-10-05T19:39:56.354 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.346+0000 7f79e3fff700 1 -- 172.21.15.8:0/3339135858 shutdown_connections 2023-10-05T19:39:56.354 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.346+0000 7f79e3fff700 1 --2- 172.21.15.8:0/3339135858 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f79c40323e0 0x7f79c4034890 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:56.354 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.346+0000 7f79e3fff700 1 --2- 172.21.15.8:0/3339135858 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f79e406cfa0 0x7f79e412b0a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:56.354 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.346+0000 7f79e3fff700 1 -- 172.21.15.8:0/3339135858 >> 172.21.15.8:0/3339135858 conn(0x7f79e409a740 msgr2=0x7f79e40965b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:56.354 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.346+0000 7f79e3fff700 1 -- 172.21.15.8:0/3339135858 shutdown_connections 2023-10-05T19:39:56.354 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:39:56.346+0000 7f79e3fff700 1 -- 172.21.15.8:0/3339135858 wait complete. 2023-10-05T19:39:56.366 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-05T19:39:56.366 INFO:teuthology.run_tasks:Running task rgw... 2023-10-05T19:39:56.380 DEBUG:tasks.rgw:config is {'client.0': None} 2023-10-05T19:39:56.380 DEBUG:tasks.rgw:client list is dict_keys(['client.0']) 2023-10-05T19:39:56.380 INFO:tasks.rgw:Creating data pools 2023-10-05T19:39:56.380 DEBUG:tasks.rgw:Obtaining remote for client client.0 2023-10-05T19:39:56.380 DEBUG:teuthology.orchestra.run.smithi012:> sudo ceph osd pool create default.rgw.buckets.data 64 64 --cluster ceph 2023-10-05T19:39:56.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:56.491+0000 7f2f97fff700 1 Processor -- start 2023-10-05T19:39:56.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:56.491+0000 7f2f97fff700 1 -- start start 2023-10-05T19:39:56.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:56.495+0000 7f2f97fff700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f2f98113620 0x7f2f98115a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:56.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:56.495+0000 7f2f97fff700 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f98115f40 con 0x7f2f981109b0 2023-10-05T19:39:56.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:56.495+0000 7f2f97fff700 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f9804c3e0 con 0x7f2f98113620 2023-10-05T19:39:56.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:56.495+0000 7f2f8e7fc700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f2f98113620 0x7f2f98115a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:56.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:56.495+0000 7f2f96ffd700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f2f981109b0 0x7f2f98110da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:56130/0 (socket says 172.21.15.12:56130) 2023-10-05T19:39:56.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:56.495+0000 7f2f8e7fc700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f2f98113620 0x7f2f98115a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37708/0 (socket says 172.21.15.12:37708) 2023-10-05T19:39:56.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:56.495+0000 7f2f96ffd700 1 -- 172.21.15.12:0/1244082415 learned_addr learned my addr 172.21.15.12:0/1244082415 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:39:56.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:56.495+0000 7f2f8e7fc700 1 -- 172.21.15.12:0/1244082415 >> v1:172.21.15.8:6789/0 conn(0x7f2f981109b0 legacy=0x7f2f98110da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:56.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:56.495+0000 7f2f8e7fc700 1 -- 172.21.15.12:0/1244082415 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f981170e0 con 0x7f2f98113620 2023-10-05T19:39:56.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:56.495+0000 7f2f8e7fc700 1 --2- 172.21.15.12:0/1244082415 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f98113620 0x7f2f98115a00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2f80009860 tx=0x7f2f800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=297d5e9425a3c0a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:56.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:56.495+0000 7f2f967fc700 1 -- 172.21.15.12:0/1244082415 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2f8000b680 con 0x7f2f98113620 2023-10-05T19:39:56.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:56.495+0000 7f2f967fc700 1 -- 172.21.15.12:0/1244082415 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f8000b7e0 con 0x7f2f98113620 2023-10-05T19:39:56.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:56.495+0000 7f2f967fc700 1 -- 172.21.15.12:0/1244082415 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2f8000b990 con 0x7f2f98113620 2023-10-05T19:39:56.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:56.495+0000 7f2f97fff700 1 -- 172.21.15.12:0/1244082415 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f98113620 msgr2=0x7f2f98115a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:56.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:56.495+0000 7f2f97fff700 1 --2- 172.21.15.12:0/1244082415 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f98113620 0x7f2f98115a00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2f80009860 tx=0x7f2f800091a0 comp rx=0 tx=0).stop 2023-10-05T19:39:56.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:56.495+0000 7f2f97fff700 1 -- 172.21.15.12:0/1244082415 shutdown_connections 2023-10-05T19:39:56.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:56.495+0000 7f2f97fff700 1 --2- 172.21.15.12:0/1244082415 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f98113620 0x7f2f98115a00 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:56.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:56.495+0000 7f2f97fff700 1 -- 172.21.15.12:0/1244082415 >> 172.21.15.12:0/1244082415 conn(0x7f2f9806c1b0 msgr2=0x7f2f9806e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:56.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:56.499+0000 7f2f97fff700 1 -- 172.21.15.12:0/1244082415 shutdown_connections 2023-10-05T19:39:56.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:56.499+0000 7f2f97fff700 1 -- 172.21.15.12:0/1244082415 wait complete. 2023-10-05T19:39:56.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:56.499+0000 7f2f97fff700 1 Processor -- start 2023-10-05T19:39:56.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:56.499+0000 7f2f97fff700 1 -- start start 2023-10-05T19:39:56.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:56.499+0000 7f2f97fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f981109b0 0x7f2f98119ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:56.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:56.499+0000 7f2f97fff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f98112000 con 0x7f2f981109b0 2023-10-05T19:39:56.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:56.499+0000 7f2f96ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f981109b0 0x7f2f98119ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:56.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:56.499+0000 7f2f96ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f981109b0 0x7f2f98119ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37718/0 (socket says 172.21.15.12:37718) 2023-10-05T19:39:56.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:56.499+0000 7f2f96ffd700 1 -- 172.21.15.12:0/1479981760 learned_addr learned my addr 172.21.15.12:0/1479981760 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:39:56.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:56.499+0000 7f2f96ffd700 1 -- 172.21.15.12:0/1479981760 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f80009420 con 0x7f2f981109b0 2023-10-05T19:39:56.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:56.499+0000 7f2f96ffd700 1 --2- 172.21.15.12:0/1479981760 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f981109b0 0x7f2f98119ae0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2f8800c690 tx=0x7f2f88007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:56.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:56.503+0000 7f2f8f7fe700 1 -- 172.21.15.12:0/1479981760 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2f88005c50 con 0x7f2f981109b0 2023-10-05T19:39:56.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:56.503+0000 7f2f8f7fe700 1 -- 172.21.15.12:0/1479981760 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f88005db0 con 0x7f2f981109b0 2023-10-05T19:39:56.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:56.503+0000 7f2f97fff700 1 -- 172.21.15.12:0/1479981760 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f9811a020 con 0x7f2f981109b0 2023-10-05T19:39:56.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:56.503+0000 7f2f8f7fe700 1 -- 172.21.15.12:0/1479981760 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2f88004990 con 0x7f2f981109b0 2023-10-05T19:39:56.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:56.503+0000 7f2f97fff700 1 -- 172.21.15.12:0/1479981760 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f9811a470 con 0x7f2f981109b0 2023-10-05T19:39:56.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:56.503+0000 7f2f8f7fe700 1 -- 172.21.15.12:0/1479981760 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f2f88004bc0 con 0x7f2f981109b0 2023-10-05T19:39:56.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:56.503+0000 7f2f97fff700 1 -- 172.21.15.12:0/1479981760 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f981196f0 con 0x7f2f981109b0 2023-10-05T19:39:56.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:56.503+0000 7f2f8f7fe700 1 --2- 172.21.15.12:0/1479981760 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2f780323e0 0x7f2f78034890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:56.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:56.503+0000 7f2f8f7fe700 1 -- 172.21.15.12:0/1479981760 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3478+0+0 (secure 0 0 0) 0x7f2f88043380 con 0x7f2f981109b0 2023-10-05T19:39:56.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:56.503+0000 7f2f8e7fc700 1 --2- 172.21.15.12:0/1479981760 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2f780323e0 0x7f2f78034890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:56.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:56.503+0000 7f2f8e7fc700 1 --2- 172.21.15.12:0/1479981760 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2f780323e0 0x7f2f78034890 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2f9810c260 tx=0x7f2f80016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:56.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:56.515+0000 7f2f8f7fe700 1 -- 172.21.15.12:0/1479981760 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f2f880053d0 con 0x7f2f981109b0 2023-10-05T19:39:56.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:56.679+0000 7f2f97fff700 1 -- 172.21.15.12:0/1479981760 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "default.rgw.buckets.data", "pg_num": 64, "pgp_num": 64} v 0) v1 -- 0x7f2f9805ecb0 con 0x7f2f981109b0 2023-10-05T19:39:57.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.591+0000 7f2f8f7fe700 1 -- 172.21.15.12:0/1479981760 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "default.rgw.buckets.data", "pg_num": 64, "pgp_num": 64}]=0 pool 'default.rgw.buckets.data' created v16) v1 ==== 167+0+0 (secure 0 0 0) 0x7f2f88011b50 con 0x7f2f981109b0 2023-10-05T19:39:57.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.591+0000 7f2f97fff700 1 -- 172.21.15.12:0/1479981760 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2f780323e0 msgr2=0x7f2f78034890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:57.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.591+0000 7f2f97fff700 1 --2- 172.21.15.12:0/1479981760 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2f780323e0 0x7f2f78034890 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2f9810c260 tx=0x7f2f80016000 comp rx=0 tx=0).stop 2023-10-05T19:39:57.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.591+0000 7f2f97fff700 1 -- 172.21.15.12:0/1479981760 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f981109b0 msgr2=0x7f2f98119ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:57.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.591+0000 7f2f97fff700 1 --2- 172.21.15.12:0/1479981760 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f981109b0 0x7f2f98119ae0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2f8800c690 tx=0x7f2f88007f40 comp rx=0 tx=0).stop 2023-10-05T19:39:57.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.591+0000 7f2f97fff700 1 -- 172.21.15.12:0/1479981760 shutdown_connections 2023-10-05T19:39:57.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.591+0000 7f2f97fff700 1 --2- 172.21.15.12:0/1479981760 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2f780323e0 0x7f2f78034890 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:57.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.591+0000 7f2f97fff700 1 --2- 172.21.15.12:0/1479981760 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f981109b0 0x7f2f98119ae0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:57.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.591+0000 7f2f97fff700 1 -- 172.21.15.12:0/1479981760 >> 172.21.15.12:0/1479981760 conn(0x7f2f9806c1b0 msgr2=0x7f2f9806c580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:57.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.595+0000 7f2f97fff700 1 -- 172.21.15.12:0/1479981760 shutdown_connections 2023-10-05T19:39:57.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.595+0000 7f2f97fff700 1 -- 172.21.15.12:0/1479981760 wait complete. 2023-10-05T19:39:57.607 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'default.rgw.buckets.data' created 2023-10-05T19:39:57.629 DEBUG:teuthology.orchestra.run.smithi012:> sudo ceph osd pool application enable default.rgw.buckets.data rgw --cluster ceph 2023-10-05T19:39:57.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.727+0000 7fb3cfe4b700 1 Processor -- start 2023-10-05T19:39:57.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.727+0000 7fb3cfe4b700 1 -- start start 2023-10-05T19:39:57.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.727+0000 7fb3cfe4b700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fb3c8115a50 0x7fb3c8117e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:57.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.727+0000 7fb3cfe4b700 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3c8118370 con 0x7fb3c810cf90 2023-10-05T19:39:57.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.727+0000 7fb3cfe4b700 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3c804bb20 con 0x7fb3c8115a50 2023-10-05T19:39:57.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.727+0000 7fb3cd3e6700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fb3c8115a50 0x7fb3c8117e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:57.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.727+0000 7fb3cd3e6700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fb3c8115a50 0x7fb3c8117e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37724/0 (socket says 172.21.15.12:37724) 2023-10-05T19:39:57.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.727+0000 7fb3cd3e6700 1 -- 172.21.15.12:0/216772164 learned_addr learned my addr 172.21.15.12:0/216772164 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:39:57.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.731+0000 7fb3cd3e6700 1 -- 172.21.15.12:0/216772164 >> v1:172.21.15.8:6789/0 conn(0x7fb3c810cf90 legacy=0x7fb3c810d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:57.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.731+0000 7fb3cd3e6700 1 -- 172.21.15.12:0/216772164 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c8119510 con 0x7fb3c8115a50 2023-10-05T19:39:57.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.731+0000 7fb3cd3e6700 1 --2- 172.21.15.12:0/216772164 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3c8115a50 0x7fb3c8117e30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8009930 tx=0x7fb3b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2033fdd7e9ed4b1f server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:57.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.731+0000 7fb3ccbe5700 1 -- 172.21.15.12:0/216772164 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb3b800b860 con 0x7fb3c8115a50 2023-10-05T19:39:57.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.731+0000 7fb3ccbe5700 1 -- 172.21.15.12:0/216772164 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3b800b9c0 con 0x7fb3c8115a50 2023-10-05T19:39:57.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.731+0000 7fb3ccbe5700 1 -- 172.21.15.12:0/216772164 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb3b800bb50 con 0x7fb3c8115a50 2023-10-05T19:39:57.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.731+0000 7fb3cfe4b700 1 -- 172.21.15.12:0/216772164 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3c8115a50 msgr2=0x7fb3c8117e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:57.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.731+0000 7fb3cfe4b700 1 --2- 172.21.15.12:0/216772164 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3c8115a50 0x7fb3c8117e30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8009930 tx=0x7fb3b8009250 comp rx=0 tx=0).stop 2023-10-05T19:39:57.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.731+0000 7fb3cfe4b700 1 -- 172.21.15.12:0/216772164 shutdown_connections 2023-10-05T19:39:57.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.731+0000 7fb3cfe4b700 1 --2- 172.21.15.12:0/216772164 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3c8115a50 0x7fb3c8117e30 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:57.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.731+0000 7fb3cfe4b700 1 -- 172.21.15.12:0/216772164 >> 172.21.15.12:0/216772164 conn(0x7fb3c8069780 msgr2=0x7fb3c8069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:57.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.731+0000 7fb3cfe4b700 1 -- 172.21.15.12:0/216772164 shutdown_connections 2023-10-05T19:39:57.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.731+0000 7fb3cfe4b700 1 -- 172.21.15.12:0/216772164 wait complete. 2023-10-05T19:39:57.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.731+0000 7fb3cfe4b700 1 Processor -- start 2023-10-05T19:39:57.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.735+0000 7fb3cfe4b700 1 -- start start 2023-10-05T19:39:57.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.735+0000 7fb3cfe4b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3c810cf90 0x7fb3c811bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:57.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.735+0000 7fb3cfe4b700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c8118070 con 0x7fb3c810cf90 2023-10-05T19:39:57.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.735+0000 7fb3cdbe7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3c810cf90 0x7fb3c811bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:57.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.735+0000 7fb3cdbe7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3c810cf90 0x7fb3c811bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37726/0 (socket says 172.21.15.12:37726) 2023-10-05T19:39:57.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.735+0000 7fb3cdbe7700 1 -- 172.21.15.12:0/2767038371 learned_addr learned my addr 172.21.15.12:0/2767038371 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:39:57.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.735+0000 7fb3cdbe7700 1 -- 172.21.15.12:0/2767038371 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b80094f0 con 0x7fb3c810cf90 2023-10-05T19:39:57.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.735+0000 7fb3cdbe7700 1 --2- 172.21.15.12:0/2767038371 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3c810cf90 0x7fb3c811bf30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb3c400c490 tx=0x7fb3c400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:57.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.735+0000 7fb3bdffb700 1 -- 172.21.15.12:0/2767038371 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb3c4011590 con 0x7fb3c810cf90 2023-10-05T19:39:57.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.735+0000 7fb3bdffb700 1 -- 172.21.15.12:0/2767038371 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3c40116f0 con 0x7fb3c810cf90 2023-10-05T19:39:57.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.735+0000 7fb3cfe4b700 1 -- 172.21.15.12:0/2767038371 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3c811c470 con 0x7fb3c810cf90 2023-10-05T19:39:57.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.735+0000 7fb3bdffb700 1 -- 172.21.15.12:0/2767038371 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb3c40119e0 con 0x7fb3c810cf90 2023-10-05T19:39:57.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.735+0000 7fb3cfe4b700 1 -- 172.21.15.12:0/2767038371 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3c811c8c0 con 0x7fb3c810cf90 2023-10-05T19:39:57.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.735+0000 7fb3cfe4b700 1 -- 172.21.15.12:0/2767038371 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3c811bb40 con 0x7fb3c810cf90 2023-10-05T19:39:57.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.739+0000 7fb3bdffb700 1 -- 172.21.15.12:0/2767038371 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fb3c4011b40 con 0x7fb3c810cf90 2023-10-05T19:39:57.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.739+0000 7fb3bdffb700 1 --2- 172.21.15.12:0/2767038371 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fb3ac032390 0x7fb3ac034840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:57.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.739+0000 7fb3bdffb700 1 -- 172.21.15.12:0/2767038371 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3841+0+0 (secure 0 0 0) 0x7fb3c4043170 con 0x7fb3c810cf90 2023-10-05T19:39:57.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.739+0000 7fb3cd3e6700 1 --2- 172.21.15.12:0/2767038371 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fb3ac032390 0x7fb3ac034840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:57.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.739+0000 7fb3cd3e6700 1 --2- 172.21.15.12:0/2767038371 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fb3ac032390 0x7fb3ac034840 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8009930 tx=0x7fb3b8016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:57.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.747+0000 7fb3bdffb700 1 -- 172.21.15.12:0/2767038371 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fb3c400fae0 con 0x7fb3c810cf90 2023-10-05T19:39:57.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:57.907+0000 7fb3cfe4b700 1 -- 172.21.15.12:0/2767038371 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "default.rgw.buckets.data", "app": "rgw"} v 0) v1 -- 0x7fb3c805ecb0 con 0x7fb3c810cf90 2023-10-05T19:39:58.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.603+0000 7fb3bdffb700 1 -- 172.21.15.12:0/2767038371 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "default.rgw.buckets.data", "app": "rgw"}]=0 enabled application 'rgw' on pool 'default.rgw.buckets.data' v17) v1 ==== 185+0+0 (secure 0 0 0) 0x7fb3c4017070 con 0x7fb3c810cf90 2023-10-05T19:39:58.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.615+0000 7fb3cfe4b700 1 -- 172.21.15.12:0/2767038371 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fb3ac032390 msgr2=0x7fb3ac034840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:58.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.615+0000 7fb3cfe4b700 1 --2- 172.21.15.12:0/2767038371 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fb3ac032390 0x7fb3ac034840 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8009930 tx=0x7fb3b8016000 comp rx=0 tx=0).stop 2023-10-05T19:39:58.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.615+0000 7fb3cfe4b700 1 -- 172.21.15.12:0/2767038371 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3c810cf90 msgr2=0x7fb3c811bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:58.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.615+0000 7fb3cfe4b700 1 --2- 172.21.15.12:0/2767038371 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3c810cf90 0x7fb3c811bf30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb3c400c490 tx=0x7fb3c400c850 comp rx=0 tx=0).stop 2023-10-05T19:39:58.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.615+0000 7fb3cfe4b700 1 -- 172.21.15.12:0/2767038371 shutdown_connections 2023-10-05T19:39:58.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.615+0000 7fb3cfe4b700 1 --2- 172.21.15.12:0/2767038371 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fb3ac032390 0x7fb3ac034840 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:58.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.615+0000 7fb3cfe4b700 1 --2- 172.21.15.12:0/2767038371 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3c810cf90 0x7fb3c811bf30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:58.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.615+0000 7fb3cfe4b700 1 -- 172.21.15.12:0/2767038371 >> 172.21.15.12:0/2767038371 conn(0x7fb3c8069780 msgr2=0x7fb3c810a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:58.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.615+0000 7fb3cfe4b700 1 -- 172.21.15.12:0/2767038371 shutdown_connections 2023-10-05T19:39:58.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.615+0000 7fb3cfe4b700 1 -- 172.21.15.12:0/2767038371 wait complete. 2023-10-05T19:39:58.628 INFO:teuthology.orchestra.run.smithi012.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.data' 2023-10-05T19:39:58.644 DEBUG:teuthology.orchestra.run.smithi012:> sudo ceph osd pool create default.rgw.buckets.index 64 64 --cluster ceph 2023-10-05T19:39:58.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.735+0000 7f78e58eb700 1 Processor -- start 2023-10-05T19:39:58.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.739+0000 7f78e58eb700 1 -- start start 2023-10-05T19:39:58.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.739+0000 7f78e58eb700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f78e010a450 0x7f78e01160d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:58.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.739+0000 7f78e58eb700 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78e006f710 con 0x7f78e006f1c0 2023-10-05T19:39:58.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.739+0000 7f78e58eb700 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78e00631f0 con 0x7f78e010a450 2023-10-05T19:39:58.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.739+0000 7f78dffff700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f78e010a450 0x7f78e01160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:58.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.739+0000 7f78e48e9700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f78e006f1c0 0x7f78e0109d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:56138/0 (socket says 172.21.15.12:56138) 2023-10-05T19:39:58.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.739+0000 7f78dffff700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f78e010a450 0x7f78e01160d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37736/0 (socket says 172.21.15.12:37736) 2023-10-05T19:39:58.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.739+0000 7f78e48e9700 1 -- 172.21.15.12:0/1419005981 learned_addr learned my addr 172.21.15.12:0/1419005981 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:39:58.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.739+0000 7f78dffff700 1 -- 172.21.15.12:0/1419005981 >> v1:172.21.15.8:6789/0 conn(0x7f78e006f1c0 legacy=0x7f78e0109d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:58.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.739+0000 7f78df7fe700 1 -- 172.21.15.12:0/1419005981 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1912149397 0 0) 0x7f78d0001110 con 0x7f78e006f1c0 2023-10-05T19:39:58.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.739+0000 7f78dffff700 1 -- 172.21.15.12:0/1419005981 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e010a8f0 con 0x7f78e010a450 2023-10-05T19:39:58.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.739+0000 7f78dffff700 1 --2- 172.21.15.12:0/1419005981 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f78e010a450 0x7f78e01160d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f78c8009860 tx=0x7f78c80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f01700712bf4ea9 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:58.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.739+0000 7f78df7fe700 1 -- 172.21.15.12:0/1419005981 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f78c800b680 con 0x7f78e010a450 2023-10-05T19:39:58.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.739+0000 7f78df7fe700 1 -- 172.21.15.12:0/1419005981 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78c800b7e0 con 0x7f78e010a450 2023-10-05T19:39:58.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.739+0000 7f78df7fe700 1 -- 172.21.15.12:0/1419005981 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f78c800b990 con 0x7f78e010a450 2023-10-05T19:39:58.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.739+0000 7f78e58eb700 1 -- 172.21.15.12:0/1419005981 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f78e010a450 msgr2=0x7f78e01160d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:58.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.739+0000 7f78e58eb700 1 --2- 172.21.15.12:0/1419005981 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f78e010a450 0x7f78e01160d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f78c8009860 tx=0x7f78c80091a0 comp rx=0 tx=0).stop 2023-10-05T19:39:58.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.739+0000 7f78e58eb700 1 -- 172.21.15.12:0/1419005981 shutdown_connections 2023-10-05T19:39:58.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.739+0000 7f78e58eb700 1 --2- 172.21.15.12:0/1419005981 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f78e010a450 0x7f78e01160d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:58.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.739+0000 7f78e58eb700 1 -- 172.21.15.12:0/1419005981 >> 172.21.15.12:0/1419005981 conn(0x7f78e006a590 msgr2=0x7f78e006c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:58.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.739+0000 7f78e58eb700 1 -- 172.21.15.12:0/1419005981 shutdown_connections 2023-10-05T19:39:58.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.739+0000 7f78e58eb700 1 -- 172.21.15.12:0/1419005981 wait complete. 2023-10-05T19:39:58.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.743+0000 7f78e58eb700 1 Processor -- start 2023-10-05T19:39:58.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.743+0000 7f78e58eb700 1 -- start start 2023-10-05T19:39:58.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.743+0000 7f78e58eb700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f78e006f1c0 0x7f78e0119aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:58.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.743+0000 7f78e58eb700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e006dcb0 con 0x7f78e006f1c0 2023-10-05T19:39:58.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.743+0000 7f78e48e9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f78e006f1c0 0x7f78e0119aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:58.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.743+0000 7f78e48e9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f78e006f1c0 0x7f78e0119aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37748/0 (socket says 172.21.15.12:37748) 2023-10-05T19:39:58.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.743+0000 7f78e48e9700 1 -- 172.21.15.12:0/2584213601 learned_addr learned my addr 172.21.15.12:0/2584213601 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:39:58.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.743+0000 7f78e48e9700 1 -- 172.21.15.12:0/2584213601 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c8009420 con 0x7f78e006f1c0 2023-10-05T19:39:58.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.743+0000 7f78e48e9700 1 --2- 172.21.15.12:0/2584213601 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f78e006f1c0 0x7f78e0119aa0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f78d000c710 tx=0x7f78d0007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:58.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.743+0000 7f78dcff9700 1 -- 172.21.15.12:0/2584213601 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f78d0005b30 con 0x7f78e006f1c0 2023-10-05T19:39:58.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.743+0000 7f78e58eb700 1 -- 172.21.15.12:0/2584213601 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78e0119fe0 con 0x7f78e006f1c0 2023-10-05T19:39:58.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.743+0000 7f78dcff9700 1 -- 172.21.15.12:0/2584213601 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d0005c90 con 0x7f78e006f1c0 2023-10-05T19:39:58.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.743+0000 7f78dcff9700 1 -- 172.21.15.12:0/2584213601 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f78d00070d0 con 0x7f78e006f1c0 2023-10-05T19:39:58.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.743+0000 7f78e58eb700 1 -- 172.21.15.12:0/2584213601 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78e011a430 con 0x7f78e006f1c0 2023-10-05T19:39:58.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.747+0000 7f78dcff9700 1 -- 172.21.15.12:0/2584213601 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f78d0007300 con 0x7f78e006f1c0 2023-10-05T19:39:58.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.747+0000 7f78e58eb700 1 -- 172.21.15.12:0/2584213601 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78e01196b0 con 0x7f78e006f1c0 2023-10-05T19:39:58.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.747+0000 7f78dcff9700 1 --2- 172.21.15.12:0/2584213601 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f78c0032380 0x7f78c0034830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:58.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.747+0000 7f78dcff9700 1 -- 172.21.15.12:0/2584213601 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3852+0+0 (secure 0 0 0) 0x7f78d0042f30 con 0x7f78e006f1c0 2023-10-05T19:39:58.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.747+0000 7f78dffff700 1 --2- 172.21.15.12:0/2584213601 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f78c0032380 0x7f78c0034830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:58.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.747+0000 7f78dffff700 1 --2- 172.21.15.12:0/2584213601 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f78c0032380 0x7f78c0034830 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f78c8006eb0 tx=0x7f78c8016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:58.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.755+0000 7f78dcff9700 1 -- 172.21.15.12:0/2584213601 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f78d0072bb0 con 0x7f78e006f1c0 2023-10-05T19:39:58.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:58.915+0000 7f78e58eb700 1 -- 172.21.15.12:0/2584213601 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "default.rgw.buckets.index", "pg_num": 64, "pgp_num": 64} v 0) v1 -- 0x7f78e005ecb0 con 0x7f78e006f1c0 2023-10-05T19:39:59.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.611+0000 7f78dcff9700 1 -- 172.21.15.12:0/2584213601 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "default.rgw.buckets.index", "pg_num": 64, "pgp_num": 64}]=0 pool 'default.rgw.buckets.index' created v18) v1 ==== 169+0+0 (secure 0 0 0) 0x7f78d0017030 con 0x7f78e006f1c0 2023-10-05T19:39:59.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.623+0000 7f78e58eb700 1 -- 172.21.15.12:0/2584213601 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f78c0032380 msgr2=0x7f78c0034830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:59.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.623+0000 7f78e58eb700 1 --2- 172.21.15.12:0/2584213601 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f78c0032380 0x7f78c0034830 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f78c8006eb0 tx=0x7f78c8016000 comp rx=0 tx=0).stop 2023-10-05T19:39:59.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.623+0000 7f78e58eb700 1 -- 172.21.15.12:0/2584213601 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f78e006f1c0 msgr2=0x7f78e0119aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:59.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.623+0000 7f78e58eb700 1 --2- 172.21.15.12:0/2584213601 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f78e006f1c0 0x7f78e0119aa0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f78d000c710 tx=0x7f78d0007bd0 comp rx=0 tx=0).stop 2023-10-05T19:39:59.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.623+0000 7f78e58eb700 1 -- 172.21.15.12:0/2584213601 shutdown_connections 2023-10-05T19:39:59.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.623+0000 7f78e58eb700 1 --2- 172.21.15.12:0/2584213601 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f78c0032380 0x7f78c0034830 secure :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f78c8006eb0 tx=0x7f78c8016000 comp rx=0 tx=0).stop 2023-10-05T19:39:59.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.623+0000 7f78e58eb700 1 --2- 172.21.15.12:0/2584213601 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f78e006f1c0 0x7f78e0119aa0 secure :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f78d000c710 tx=0x7f78d0007bd0 comp rx=0 tx=0).stop 2023-10-05T19:39:59.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.623+0000 7f78e58eb700 1 -- 172.21.15.12:0/2584213601 >> 172.21.15.12:0/2584213601 conn(0x7f78e006a590 msgr2=0x7f78e01155e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:59.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.631+0000 7f78e58eb700 1 -- 172.21.15.12:0/2584213601 shutdown_connections 2023-10-05T19:39:59.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.631+0000 7f78e58eb700 1 -- 172.21.15.12:0/2584213601 wait complete. 2023-10-05T19:39:59.641 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'default.rgw.buckets.index' created 2023-10-05T19:39:59.652 DEBUG:teuthology.orchestra.run.smithi012:> sudo ceph osd pool application enable default.rgw.buckets.index rgw --cluster ceph 2023-10-05T19:39:59.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.755+0000 7fbc9cb5a700 1 Processor -- start 2023-10-05T19:39:59.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.755+0000 7fbc9cb5a700 1 -- start start 2023-10-05T19:39:59.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.755+0000 7fbc9cb5a700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fbc98112aa0 0x7fbc98112e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:59.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.755+0000 7fbc9cb5a700 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc98113500 con 0x7fbc9810cd60 2023-10-05T19:39:59.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.755+0000 7fbc9cb5a700 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc9804c3e0 con 0x7fbc98112aa0 2023-10-05T19:39:59.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.755+0000 7fbc977fe700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fbc98112aa0 0x7fbc98112e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:59.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.755+0000 7fbc8ffff700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7fbc9810cd60 0x7fbc9810d200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:56140/0 (socket says 172.21.15.12:56140) 2023-10-05T19:39:59.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.755+0000 7fbc8ffff700 1 -- 172.21.15.12:0/2075210495 learned_addr learned my addr 172.21.15.12:0/2075210495 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:39:59.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.759+0000 7fbc977fe700 1 -- 172.21.15.12:0/2075210495 >> v1:172.21.15.8:6789/0 conn(0x7fbc9810cd60 legacy=0x7fbc9810d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:59.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.759+0000 7fbc96ffd700 1 -- 172.21.15.12:0/2075210495 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2179626807 0 0) 0x7fbc98113500 con 0x7fbc9810cd60 2023-10-05T19:39:59.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.759+0000 7fbc977fe700 1 -- 172.21.15.12:0/2075210495 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc9810d910 con 0x7fbc98112aa0 2023-10-05T19:39:59.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.759+0000 7fbc977fe700 1 --2- 172.21.15.12:0/2075210495 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc98112aa0 0x7fbc98112e70 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbc88009860 tx=0x7fbc880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c8e17b49735ef45 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:59.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.759+0000 7fbc96ffd700 1 -- 172.21.15.12:0/2075210495 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc8800b680 con 0x7fbc98112aa0 2023-10-05T19:39:59.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.759+0000 7fbc96ffd700 1 -- 172.21.15.12:0/2075210495 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc8800b7e0 con 0x7fbc98112aa0 2023-10-05T19:39:59.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.759+0000 7fbc96ffd700 1 -- 172.21.15.12:0/2075210495 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc8800b990 con 0x7fbc98112aa0 2023-10-05T19:39:59.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.759+0000 7fbc9cb5a700 1 -- 172.21.15.12:0/2075210495 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc98112aa0 msgr2=0x7fbc98112e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:39:59.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.759+0000 7fbc9cb5a700 1 --2- 172.21.15.12:0/2075210495 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc98112aa0 0x7fbc98112e70 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbc88009860 tx=0x7fbc880091a0 comp rx=0 tx=0).stop 2023-10-05T19:39:59.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.759+0000 7fbc9cb5a700 1 -- 172.21.15.12:0/2075210495 shutdown_connections 2023-10-05T19:39:59.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.759+0000 7fbc9cb5a700 1 --2- 172.21.15.12:0/2075210495 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc98112aa0 0x7fbc98112e70 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:39:59.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.759+0000 7fbc9cb5a700 1 -- 172.21.15.12:0/2075210495 >> 172.21.15.12:0/2075210495 conn(0x7fbc980695c0 msgr2=0x7fbc980699c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:39:59.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.759+0000 7fbc9cb5a700 1 -- 172.21.15.12:0/2075210495 shutdown_connections 2023-10-05T19:39:59.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.759+0000 7fbc9cb5a700 1 -- 172.21.15.12:0/2075210495 wait complete. 2023-10-05T19:39:59.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.759+0000 7fbc9cb5a700 1 Processor -- start 2023-10-05T19:39:59.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.759+0000 7fbc9cb5a700 1 -- start start 2023-10-05T19:39:59.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.759+0000 7fbc9cb5a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc9810cd60 0x7fbc9811bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:59.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.759+0000 7fbc9cb5a700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9810c2f0 con 0x7fbc9810cd60 2023-10-05T19:39:59.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.763+0000 7fbc977fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc9810cd60 0x7fbc9811bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:59.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.763+0000 7fbc977fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc9810cd60 0x7fbc9811bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37776/0 (socket says 172.21.15.12:37776) 2023-10-05T19:39:59.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.763+0000 7fbc977fe700 1 -- 172.21.15.12:0/302658125 learned_addr learned my addr 172.21.15.12:0/302658125 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:39:59.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.763+0000 7fbc977fe700 1 -- 172.21.15.12:0/302658125 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc88009420 con 0x7fbc9810cd60 2023-10-05T19:39:59.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.763+0000 7fbc977fe700 1 --2- 172.21.15.12:0/302658125 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc9810cd60 0x7fbc9811bb90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbc880091e0 tx=0x7fbc88016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:59.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.763+0000 7fbc8f7fe700 1 -- 172.21.15.12:0/302658125 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc88011d30 con 0x7fbc9810cd60 2023-10-05T19:39:59.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.763+0000 7fbc8f7fe700 1 -- 172.21.15.12:0/302658125 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc88020070 con 0x7fbc9810cd60 2023-10-05T19:39:59.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.763+0000 7fbc9cb5a700 1 -- 172.21.15.12:0/302658125 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc9811c0d0 con 0x7fbc9810cd60 2023-10-05T19:39:59.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.763+0000 7fbc8f7fe700 1 -- 172.21.15.12:0/302658125 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc8801b5d0 con 0x7fbc9810cd60 2023-10-05T19:39:59.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.763+0000 7fbc9cb5a700 1 -- 172.21.15.12:0/302658125 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc9811c520 con 0x7fbc9810cd60 2023-10-05T19:39:59.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.763+0000 7fbc9cb5a700 1 -- 172.21.15.12:0/302658125 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc9805ecb0 con 0x7fbc9810cd60 2023-10-05T19:39:59.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.763+0000 7fbc8f7fe700 1 -- 172.21.15.12:0/302658125 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fbc88022030 con 0x7fbc9810cd60 2023-10-05T19:39:59.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.763+0000 7fbc8f7fe700 1 --2- 172.21.15.12:0/302658125 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fbc780323e0 0x7fbc78034890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:39:59.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.767+0000 7fbc8f7fe700 1 -- 172.21.15.12:0/302658125 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4216+0+0 (secure 0 0 0) 0x7fbc88045b10 con 0x7fbc9810cd60 2023-10-05T19:39:59.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.767+0000 7fbc8ffff700 1 --2- 172.21.15.12:0/302658125 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fbc780323e0 0x7fbc78034890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:39:59.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.767+0000 7fbc8ffff700 1 --2- 172.21.15.12:0/302658125 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fbc780323e0 0x7fbc78034890 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbc800011e0 tx=0x7fbc800079d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:39:59.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.775+0000 7fbc8f7fe700 1 -- 172.21.15.12:0/302658125 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fbc880249c0 con 0x7fbc9810cd60 2023-10-05T19:39:59.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:39:59.939+0000 7fbc9cb5a700 1 -- 172.21.15.12:0/302658125 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "default.rgw.buckets.index", "app": "rgw"} v 0) v1 -- 0x7fbc98112aa0 con 0x7fbc9810cd60 2023-10-05T19:40:00.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:00.619+0000 7fbc8f7fe700 1 -- 172.21.15.12:0/302658125 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "default.rgw.buckets.index", "app": "rgw"}]=0 enabled application 'rgw' on pool 'default.rgw.buckets.index' v19) v1 ==== 187+0+0 (secure 0 0 0) 0x7fbc880193f0 con 0x7fbc9810cd60 2023-10-05T19:40:00.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:00.635+0000 7fbc9cb5a700 1 -- 172.21.15.12:0/302658125 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fbc780323e0 msgr2=0x7fbc78034890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:00.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:00.635+0000 7fbc9cb5a700 1 --2- 172.21.15.12:0/302658125 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fbc780323e0 0x7fbc78034890 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbc800011e0 tx=0x7fbc800079d0 comp rx=0 tx=0).stop 2023-10-05T19:40:00.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:00.635+0000 7fbc9cb5a700 1 -- 172.21.15.12:0/302658125 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc9810cd60 msgr2=0x7fbc9811bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:00.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:00.635+0000 7fbc9cb5a700 1 --2- 172.21.15.12:0/302658125 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc9810cd60 0x7fbc9811bb90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbc880091e0 tx=0x7fbc88016000 comp rx=0 tx=0).stop 2023-10-05T19:40:00.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:00.635+0000 7fbc9cb5a700 1 -- 172.21.15.12:0/302658125 shutdown_connections 2023-10-05T19:40:00.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:00.635+0000 7fbc9cb5a700 1 --2- 172.21.15.12:0/302658125 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fbc780323e0 0x7fbc78034890 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:00.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:00.635+0000 7fbc9cb5a700 1 --2- 172.21.15.12:0/302658125 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc9810cd60 0x7fbc9811bb90 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:00.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:00.635+0000 7fbc9cb5a700 1 -- 172.21.15.12:0/302658125 >> 172.21.15.12:0/302658125 conn(0x7fbc980695c0 msgr2=0x7fbc98111460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:00.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:00.635+0000 7fbc9cb5a700 1 -- 172.21.15.12:0/302658125 shutdown_connections 2023-10-05T19:40:00.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:00.635+0000 7fbc9cb5a700 1 -- 172.21.15.12:0/302658125 wait complete. 2023-10-05T19:40:00.652 INFO:teuthology.orchestra.run.smithi012.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.index' 2023-10-05T19:40:00.667 DEBUG:tasks.rgw:Pools created 2023-10-05T19:40:00.667 INFO:tasks.rgw:Starting rgw... 2023-10-05T19:40:00.667 INFO:tasks.rgw:rgw client.0 config is {} 2023-10-05T19:40:00.668 INFO:tasks.rgw:Using beast as radosgw frontend 2023-10-05T19:40:00.668 INFO:tasks.rgw.client.0:Restarting daemon 2023-10-05T19:40:00.668 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term radosgw --rgw-frontends 'beast port=80' -n client.0 --cluster ceph -k /etc/ceph/ceph.client.0.keyring --log-file /var/log/ceph/rgw.ceph.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock --foreground | sudo tee /var/log/ceph/rgw.ceph.client.0.stdout 2>&1 2023-10-05T19:40:00.670 INFO:tasks.rgw.client.0:Started 2023-10-05T19:40:00.670 INFO:tasks.rgw:Polling client.0 until it starts accepting connections on http://smithi012.front.sepia.ceph.com:80/ 2023-10-05T19:40:00.670 DEBUG:teuthology.orchestra.run.smithi012:> curl http://smithi012.front.sepia.ceph.com:80/ 2023-10-05T19:40:00.707 INFO:teuthology.orchestra.run.smithi012.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2023-10-05T19:40:00.707 INFO:teuthology.orchestra.run.smithi012.stderr: Dload Upload Total Spent Left Speed 2023-10-05T19:40:00.709 DEBUG:teuthology.orchestra.run:got remote process result: 7 2023-10-05T19:40:00.710 INFO:teuthology.orchestra.run.smithi012.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2023-10-05T19:40:00.710 INFO:teuthology.orchestra.run.smithi012.stderr:curl: (7) Failed to connect to smithi012.front.sepia.ceph.com port 80: Connection refused 2023-10-05T19:40:01.711 DEBUG:teuthology.orchestra.run.smithi012:> curl http://smithi012.front.sepia.ceph.com:80/ 2023-10-05T19:40:01.728 INFO:teuthology.orchestra.run.smithi012.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2023-10-05T19:40:01.728 INFO:teuthology.orchestra.run.smithi012.stderr: Dload Upload Total Spent Left Speed 2023-10-05T19:40:01.728 INFO:teuthology.orchestra.run.smithi012.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2023-10-05T19:40:01.728 INFO:teuthology.orchestra.run.smithi012.stderr:curl: (7) Failed to connect to smithi012.front.sepia.ceph.com port 80: Connection refused 2023-10-05T19:40:01.729 DEBUG:teuthology.orchestra.run:got remote process result: 7 2023-10-05T19:40:03.730 DEBUG:teuthology.orchestra.run.smithi012:> curl http://smithi012.front.sepia.ceph.com:80/ 2023-10-05T19:40:03.748 INFO:teuthology.orchestra.run.smithi012.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2023-10-05T19:40:03.748 INFO:teuthology.orchestra.run.smithi012.stderr: Dload Upload Total Spent Left Speed 2023-10-05T19:40:03.749 DEBUG:teuthology.orchestra.run:got remote process result: 7 2023-10-05T19:40:03.750 INFO:teuthology.orchestra.run.smithi012.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2023-10-05T19:40:03.750 INFO:teuthology.orchestra.run.smithi012.stderr:curl: (7) Failed to connect to smithi012.front.sepia.ceph.com port 80: Connection refused 2023-10-05T19:40:07.750 DEBUG:teuthology.orchestra.run.smithi012:> curl http://smithi012.front.sepia.ceph.com:80/ 2023-10-05T19:40:07.768 INFO:teuthology.orchestra.run.smithi012.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2023-10-05T19:40:07.769 INFO:teuthology.orchestra.run.smithi012.stderr: Dload Upload Total Spent Left Speed 2023-10-05T19:40:07.769 INFO:teuthology.orchestra.run.smithi012.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2023-10-05T19:40:07.769 INFO:teuthology.orchestra.run.smithi012.stderr:curl: (7) Failed to connect to smithi012.front.sepia.ceph.com port 80: Connection refused 2023-10-05T19:40:07.769 DEBUG:teuthology.orchestra.run:got remote process result: 7 2023-10-05T19:40:15.770 DEBUG:teuthology.orchestra.run.smithi012:> curl http://smithi012.front.sepia.ceph.com:80/ 2023-10-05T19:40:15.786 INFO:teuthology.orchestra.run.smithi012.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2023-10-05T19:40:15.786 INFO:teuthology.orchestra.run.smithi012.stderr: Dload Upload Total Spent Left Speed 2023-10-05T19:40:15.822 INFO:teuthology.orchestra.run.smithi012.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 214 0 214 0 0 6114 0 --:--:-- --:--:-- --:--:-- 6114 2023-10-05T19:40:15.823 INFO:teuthology.orchestra.run.smithi012.stdout:anonymous 2023-10-05T19:40:15.824 INFO:teuthology.run_tasks:Running task radosgw-admin... 2023-10-05T19:40:15.912 DEBUG:tasks.radosgw_admin:config is: {'client.0': None} 2023-10-05T19:40:15.912 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'list'] 2023-10-05T19:40:15.913 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'list'] 2023-10-05T19:40:15.913 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket list 2023-10-05T19:40:15.952 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:15.952 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:15.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.943+0000 7f7f97d62e40 1 Processor -- start 2023-10-05T19:40:15.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.943+0000 7f7f97d62e40 1 -- start start 2023-10-05T19:40:15.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.947+0000 7f7f97d62e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x562b6567bfe0 0x562b6567c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:15.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.947+0000 7f7f97d62e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x562b655c6c00 con 0x562b6567c8f0 2023-10-05T19:40:15.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.947+0000 7f7f97d62e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x562b65583ee0 con 0x562b6567bfe0 2023-10-05T19:40:15.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.947+0000 7f7f95897700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x562b6567bfe0 0x562b6567c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:15.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.947+0000 7f7f95096700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x562b6567c8f0 0x562b6567d390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:47110/0 (socket says 172.21.15.12:47110) 2023-10-05T19:40:15.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.947+0000 7f7f95096700 1 -- 172.21.15.12:0/2460421931 learned_addr learned my addr 172.21.15.12:0/2460421931 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:15.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.947+0000 7f7f94895700 1 -- 172.21.15.12:0/2460421931 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 199472997 0 0) 0x562b655c6c00 con 0x562b6567c8f0 2023-10-05T19:40:15.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.947+0000 7f7f94895700 1 -- 172.21.15.12:0/2460421931 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f78001e10 con 0x562b6567c8f0 2023-10-05T19:40:15.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.947+0000 7f7f95897700 1 -- 172.21.15.12:0/2460421931 >> v1:172.21.15.8:6789/0 conn(0x562b6567c8f0 legacy=0x562b6567d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:15.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.947+0000 7f7f95897700 1 -- 172.21.15.12:0/2460421931 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b6567daa0 con 0x562b6567bfe0 2023-10-05T19:40:15.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.947+0000 7f7f95897700 1 --2- 172.21.15.12:0/2460421931 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562b6567bfe0 0x562b6567c3b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7f8800a5d0 tx=0x7f7f88005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54b1bf7f70644c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:15.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.947+0000 7f7f94895700 1 -- 172.21.15.12:0/2460421931 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7f88007610 con 0x562b6567bfe0 2023-10-05T19:40:15.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.947+0000 7f7f94895700 1 -- 172.21.15.12:0/2460421931 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f88007770 con 0x562b6567bfe0 2023-10-05T19:40:15.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.947+0000 7f7f94895700 1 -- 172.21.15.12:0/2460421931 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7f88007a40 con 0x562b6567bfe0 2023-10-05T19:40:15.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.947+0000 7f7f97d62e40 1 -- 172.21.15.12:0/2460421931 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562b6567bfe0 msgr2=0x562b6567c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:15.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.947+0000 7f7f97d62e40 1 --2- 172.21.15.12:0/2460421931 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562b6567bfe0 0x562b6567c3b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7f8800a5d0 tx=0x7f7f88005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:15.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.947+0000 7f7f97d62e40 1 -- 172.21.15.12:0/2460421931 shutdown_connections 2023-10-05T19:40:15.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.947+0000 7f7f97d62e40 1 --2- 172.21.15.12:0/2460421931 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562b6567bfe0 0x562b6567c3b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:15.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.947+0000 7f7f97d62e40 1 -- 172.21.15.12:0/2460421931 >> 172.21.15.12:0/2460421931 conn(0x562b655ef830 msgr2=0x562b655efc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:15.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.947+0000 7f7f97d62e40 1 -- 172.21.15.12:0/2460421931 shutdown_connections 2023-10-05T19:40:15.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.947+0000 7f7f97d62e40 1 -- 172.21.15.12:0/2460421931 wait complete. 2023-10-05T19:40:15.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.951+0000 7f7f97d62e40 1 Processor -- start 2023-10-05T19:40:15.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.951+0000 7f7f97d62e40 1 -- start start 2023-10-05T19:40:15.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.951+0000 7f7f97d62e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562b6567bfe0 0x562b6569aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:15.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.951+0000 7f7f97d62e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b65547e30 con 0x562b6567bfe0 2023-10-05T19:40:15.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.951+0000 7f7f95897700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562b6567bfe0 0x562b6569aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:15.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.951+0000 7f7f95897700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562b6567bfe0 0x562b6569aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36254/0 (socket says 172.21.15.12:36254) 2023-10-05T19:40:15.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.951+0000 7f7f95897700 1 -- 172.21.15.12:0/854204943 learned_addr learned my addr 172.21.15.12:0/854204943 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:15.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.951+0000 7f7f95897700 1 -- 172.21.15.12:0/854204943 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f88005cf0 con 0x562b6567bfe0 2023-10-05T19:40:15.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.951+0000 7f7f95897700 1 --2- 172.21.15.12:0/854204943 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562b6567bfe0 0x562b6569aee0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7f88000c00 tx=0x7f7f88014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f7b2dd2c20eaa59 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:15.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.951+0000 7f7f8f7fe700 1 -- 172.21.15.12:0/854204943 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7f880078e0 con 0x562b6567bfe0 2023-10-05T19:40:15.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.951+0000 7f7f8f7fe700 1 -- 172.21.15.12:0/854204943 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f88007a40 con 0x562b6567bfe0 2023-10-05T19:40:15.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.951+0000 7f7f8f7fe700 1 -- 172.21.15.12:0/854204943 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7f88007bd0 con 0x562b6567bfe0 2023-10-05T19:40:15.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.951+0000 7f7f97d62e40 1 -- 172.21.15.12:0/854204943 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562b6567bfe0 msgr2=0x562b6569aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:15.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.951+0000 7f7f97d62e40 1 --2- 172.21.15.12:0/854204943 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562b6567bfe0 0x562b6569aee0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7f88000c00 tx=0x7f7f88014000 comp rx=0 tx=0).stop 2023-10-05T19:40:15.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.955+0000 7f7f97d62e40 1 -- 172.21.15.12:0/854204943 shutdown_connections 2023-10-05T19:40:15.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.955+0000 7f7f97d62e40 1 --2- 172.21.15.12:0/854204943 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562b6567bfe0 0x562b6569aee0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:15.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.955+0000 7f7f97d62e40 1 -- 172.21.15.12:0/854204943 >> 172.21.15.12:0/854204943 conn(0x562b655ef830 msgr2=0x562b65696aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:15.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.955+0000 7f7f97d62e40 1 -- 172.21.15.12:0/854204943 shutdown_connections 2023-10-05T19:40:15.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.955+0000 7f7f97d62e40 1 -- 172.21.15.12:0/854204943 wait complete. 2023-10-05T19:40:15.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.955+0000 7f7f97d62e40 1 Processor -- start 2023-10-05T19:40:15.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.955+0000 7f7f97d62e40 1 -- start start 2023-10-05T19:40:15.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.955+0000 7f7f97d62e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562b6567bfe0 0x562b656a1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:15.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.955+0000 7f7f97d62e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b6551aed0 con 0x562b6567bfe0 2023-10-05T19:40:15.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.955+0000 7f7f95897700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562b6567bfe0 0x562b656a1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:15.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.955+0000 7f7f95897700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562b6567bfe0 0x562b656a1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36268/0 (socket says 172.21.15.12:36268) 2023-10-05T19:40:15.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.955+0000 7f7f95897700 1 -- 172.21.15.12:0/1283401693 learned_addr learned my addr 172.21.15.12:0/1283401693 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:15.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.955+0000 7f7f95897700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b6567daa0 con 0x562b6567bfe0 2023-10-05T19:40:15.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.955+0000 7f7f95897700 1 --2- 172.21.15.12:0/1283401693 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562b6567bfe0 0x562b656a1ba0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7f8801f040 tx=0x7f7f8801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:15.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.955+0000 7f7f8dffb700 1 -- 172.21.15.12:0/1283401693 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7f88007bb0 con 0x562b6567bfe0 2023-10-05T19:40:15.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.955+0000 7f7f8dffb700 1 -- 172.21.15.12:0/1283401693 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f88029070 con 0x562b6567bfe0 2023-10-05T19:40:15.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.955+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562b656a20e0 con 0x562b6567bfe0 2023-10-05T19:40:15.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.955+0000 7f7f8dffb700 1 -- 172.21.15.12:0/1283401693 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7f88012610 con 0x562b6567bfe0 2023-10-05T19:40:15.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.959+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562b656a2520 con 0x562b6567bfe0 2023-10-05T19:40:15.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.959+0000 7f7f8dffb700 1 -- 172.21.15.12:0/1283401693 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f7f88012770 con 0x562b6567bfe0 2023-10-05T19:40:15.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.959+0000 7f7f8dffb700 1 --2- 172.21.15.12:0/1283401693 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f7f7c030ca0 0x7f7f7c033150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:15.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.959+0000 7f7f8dffb700 1 -- 172.21.15.12:0/1283401693 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f7f8802b070 con 0x562b6567bfe0 2023-10-05T19:40:15.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.959+0000 7f7f95096700 1 --2- 172.21.15.12:0/1283401693 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f7f7c030ca0 0x7f7f7c033150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:15.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.959+0000 7f7f97d62e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:15.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.959+0000 7f7f97d62e40 1 --2- 172.21.15.12:0/1283401693 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x562b6567c8f0 0x562b656ad570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:15.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.959+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656adca0 con 0x562b6567c8f0 2023-10-05T19:40:15.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.959+0000 7f7f96098700 1 --2- 172.21.15.12:0/1283401693 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x562b6567c8f0 0x562b656ad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:15.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.959+0000 7f7f95096700 1 --2- 172.21.15.12:0/1283401693 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f7f7c030ca0 0x7f7f7c033150 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7f80004170 tx=0x7f7f80001400 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:15.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.959+0000 7f7f96098700 1 --2- 172.21.15.12:0/1283401693 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x562b6567c8f0 0x562b656ad570 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:15.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.963+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7f9000a5a0 con 0x562b6567c8f0 2023-10-05T19:40:15.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.963+0000 7f7f97d62e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:15.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.963+0000 7f7f97d62e40 20 realm 2023-10-05T19:40:15.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.963+0000 7f7f97d62e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:15.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.963+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656aaba0 con 0x562b6567c8f0 2023-10-05T19:40:15.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.963+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7f9000a5a0 con 0x562b6567c8f0 2023-10-05T19:40:15.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.963+0000 7f7f97d62e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:15.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.963+0000 7f7f97d62e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:15.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.963+0000 7f7f97d62e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:15.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.963+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656ae7d0 con 0x562b6567c8f0 2023-10-05T19:40:15.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.963+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7f9000a5a0 con 0x562b6567c8f0 2023-10-05T19:40:15.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.963+0000 7f7f97d62e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:15.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.963+0000 7f7f97d62e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:15.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.963+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656ae3b0 con 0x562b6567c8f0 2023-10-05T19:40:15.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.963+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f7f9000a5a0 con 0x562b6567c8f0 2023-10-05T19:40:15.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.963+0000 7f7f97d62e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:15.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.963+0000 7f7f97d62e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:15.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.963+0000 7f7f97d62e40 1 --2- 172.21.15.12:0/1283401693 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x562b656af760 0x562b656b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:15.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.963+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b22b0 con 0x562b656af760 2023-10-05T19:40:15.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.963+0000 7f7f95897700 1 --2- 172.21.15.12:0/1283401693 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x562b656af760 0x562b656b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:15.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.963+0000 7f7f95897700 1 --2- 172.21.15.12:0/1283401693 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x562b656af760 0x562b656b1b40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:15.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.963+0000 7f7f95897700 1 -- 172.21.15.12:0/1283401693 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv1 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f7f8801cc20 con 0x562b656af760 2023-10-05T19:40:15.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.963+0000 7f7f97d62e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:15.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.963+0000 7f7f97d62e40 20 searching for the correct realm 2023-10-05T19:40:15.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.963+0000 7f7f97d62e40 1 --2- 172.21.15.12:0/1283401693 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x562b656b40e0 0x562b656b64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:15.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.963+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b6a00 con 0x562b656b40e0 2023-10-05T19:40:15.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.963+0000 7f7f95096700 1 --2- 172.21.15.12:0/1283401693 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x562b656b40e0 0x562b656b64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:15.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.967+0000 7f7f95096700 1 --2- 172.21.15.12:0/1283401693 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x562b656b40e0 0x562b656b64c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:15.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.967+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f7f80006070 con 0x562b656b40e0 2023-10-05T19:40:15.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.967+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b6a00 con 0x562b656af760 2023-10-05T19:40:15.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.967+0000 7f7f95897700 1 -- 172.21.15.12:0/1283401693 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f8801cc20 con 0x562b656af760 2023-10-05T19:40:15.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.967+0000 7f7f95897700 1 --2- 172.21.15.12:0/1283401693 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f7f88007030 0x7f7f880113b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:15.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.967+0000 7f7f95897700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b6a00 con 0x7f7f88007030 2023-10-05T19:40:15.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.971+0000 7f7f96098700 1 --2- 172.21.15.12:0/1283401693 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f7f88007030 0x7f7f880113b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:15.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.971+0000 7f7f96098700 1 --2- 172.21.15.12:0/1283401693 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f7f88007030 0x7f7f880113b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:15.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.971+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x7f7f88007030 2023-10-05T19:40:15.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.971+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b6a00 con 0x562b6567c8f0 2023-10-05T19:40:15.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.971+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x562b6567c8f0 2023-10-05T19:40:15.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.971+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b6a00 con 0x562b6567c8f0 2023-10-05T19:40:15.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.971+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x562b6567c8f0 2023-10-05T19:40:15.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.971+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b6a00 con 0x7f7f88007030 2023-10-05T19:40:15.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.975+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x7f7f88007030 2023-10-05T19:40:15.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.975+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b6a00 con 0x7f7f88007030 2023-10-05T19:40:15.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.975+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x7f7f88007030 2023-10-05T19:40:15.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.975+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b6a00 con 0x562b656af760 2023-10-05T19:40:15.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.975+0000 7f7f95897700 1 -- 172.21.15.12:0/1283401693 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f8801cc20 con 0x562b656af760 2023-10-05T19:40:15.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.975+0000 7f7f95897700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b6a00 con 0x562b656b40e0 2023-10-05T19:40:15.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.975+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f80006070 con 0x562b656b40e0 2023-10-05T19:40:15.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.975+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b6a00 con 0x562b6567c8f0 2023-10-05T19:40:15.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.975+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x562b6567c8f0 2023-10-05T19:40:15.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.975+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b6a00 con 0x7f7f88007030 2023-10-05T19:40:15.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.979+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x7f7f88007030 2023-10-05T19:40:15.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.979+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b6a00 con 0x562b6567c8f0 2023-10-05T19:40:15.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.979+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x562b6567c8f0 2023-10-05T19:40:15.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.979+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b6a00 con 0x562b656af760 2023-10-05T19:40:15.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.979+0000 7f7f95897700 1 -- 172.21.15.12:0/1283401693 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f8801cc20 con 0x562b656af760 2023-10-05T19:40:15.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.979+0000 7f7f95897700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b6a00 con 0x562b656b40e0 2023-10-05T19:40:15.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.979+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f80006070 con 0x562b656b40e0 2023-10-05T19:40:15.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.979+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b6a00 con 0x562b656b40e0 2023-10-05T19:40:15.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.979+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f80006070 con 0x562b656b40e0 2023-10-05T19:40:15.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.979+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b6a00 con 0x7f7f88007030 2023-10-05T19:40:15.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.983+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x7f7f88007030 2023-10-05T19:40:15.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.983+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b6a00 con 0x562b656b40e0 2023-10-05T19:40:15.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.983+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f80006070 con 0x562b656b40e0 2023-10-05T19:40:15.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.983+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b6a00 con 0x562b656b40e0 2023-10-05T19:40:15.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.983+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f80006070 con 0x562b656b40e0 2023-10-05T19:40:15.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.983+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b6a00 con 0x562b6567c8f0 2023-10-05T19:40:15.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.983+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x562b6567c8f0 2023-10-05T19:40:15.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.983+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b6a00 con 0x562b6567c8f0 2023-10-05T19:40:15.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.983+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x562b6567c8f0 2023-10-05T19:40:15.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.983+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b6a00 con 0x7f7f88007030 2023-10-05T19:40:15.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.987+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x7f7f88007030 2023-10-05T19:40:15.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.987+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b6a00 con 0x562b656b40e0 2023-10-05T19:40:15.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.987+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f80006070 con 0x562b656b40e0 2023-10-05T19:40:15.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.987+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b6a00 con 0x562b656af760 2023-10-05T19:40:15.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.987+0000 7f7f95897700 1 -- 172.21.15.12:0/1283401693 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f8801cc20 con 0x562b656af760 2023-10-05T19:40:15.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.987+0000 7f7f95897700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b6a00 con 0x562b6567c8f0 2023-10-05T19:40:15.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.987+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x562b6567c8f0 2023-10-05T19:40:15.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.987+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b6a00 con 0x7f7f88007030 2023-10-05T19:40:15.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.991+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x7f7f88007030 2023-10-05T19:40:15.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.991+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b6a00 con 0x562b656b40e0 2023-10-05T19:40:15.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.991+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f80006070 con 0x562b656b40e0 2023-10-05T19:40:15.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.991+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b6a00 con 0x7f7f88007030 2023-10-05T19:40:16.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.991+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x7f7f88007030 2023-10-05T19:40:16.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.991+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b6a00 con 0x562b656af760 2023-10-05T19:40:16.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.991+0000 7f7f95897700 1 -- 172.21.15.12:0/1283401693 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f8801cc20 con 0x562b656af760 2023-10-05T19:40:16.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.991+0000 7f7f95897700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b6a00 con 0x562b656b40e0 2023-10-05T19:40:16.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.995+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f80006070 con 0x562b656b40e0 2023-10-05T19:40:16.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.995+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b6a00 con 0x7f7f88007030 2023-10-05T19:40:16.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.995+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x7f7f88007030 2023-10-05T19:40:16.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.995+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b6a00 con 0x562b6567c8f0 2023-10-05T19:40:16.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.995+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f7f9000a5a0 con 0x562b6567c8f0 2023-10-05T19:40:16.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.995+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b6a00 con 0x562b656af760 2023-10-05T19:40:16.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.995+0000 7f7f95897700 1 -- 172.21.15.12:0/1283401693 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f7f8801cc20 con 0x562b656af760 2023-10-05T19:40:16.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.995+0000 7f7f97d62e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:16.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.995+0000 7f7f97d62e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:16.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.995+0000 7f7f97d62e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:16.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.995+0000 7f7f97d62e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:16.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.995+0000 7f7f97d62e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.995+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b2e90 con 0x562b6567c8f0 2023-10-05T19:40:16.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.995+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7f9000a5a0 con 0x562b6567c8f0 2023-10-05T19:40:16.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.995+0000 7f7f97d62e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:16.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.995+0000 7f7f97d62e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.995+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b37e0 con 0x562b6567c8f0 2023-10-05T19:40:16.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.999+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f7f9000a5a0 con 0x562b6567c8f0 2023-10-05T19:40:16.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.999+0000 7f7f97d62e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:16.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.999+0000 7f7f97d62e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.999+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b2a50 con 0x562b656b40e0 2023-10-05T19:40:16.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.999+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f7f80006070 con 0x562b656b40e0 2023-10-05T19:40:16.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.999+0000 7f7f97d62e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:16.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.999+0000 7f7f97d62e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.999+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b7330 con 0x562b656b40e0 2023-10-05T19:40:16.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.999+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f7f80006070 con 0x562b656b40e0 2023-10-05T19:40:16.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.999+0000 7f7f97d62e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:16.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.999+0000 7f7f97d62e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.999+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b76e0 con 0x562b6567c8f0 2023-10-05T19:40:16.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.999+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7f9000a5a0 con 0x562b6567c8f0 2023-10-05T19:40:16.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.999+0000 7f7f97d62e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:16.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.999+0000 7f7f97d62e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.999+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b7a90 con 0x562b6567c8f0 2023-10-05T19:40:16.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.999+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f7f9000a5a0 con 0x562b6567c8f0 2023-10-05T19:40:16.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.999+0000 7f7f97d62e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:16.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:15.999+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b8aa0 con 0x562b656b40e0 2023-10-05T19:40:16.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.003+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f7f80006070 con 0x562b656b40e0 2023-10-05T19:40:16.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.003+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b8aa0 con 0x562b656af760 2023-10-05T19:40:16.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.003+0000 7f7f95897700 1 -- 172.21.15.12:0/1283401693 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f8801cc20 con 0x562b656af760 2023-10-05T19:40:16.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.003+0000 7f7f95897700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b8aa0 con 0x7f7f88007030 2023-10-05T19:40:16.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.007+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x7f7f88007030 2023-10-05T19:40:16.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.007+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b76e0 con 0x562b6567c8f0 2023-10-05T19:40:16.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.007+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x562b6567c8f0 2023-10-05T19:40:16.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.007+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b76e0 con 0x562b6567c8f0 2023-10-05T19:40:16.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.007+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x562b6567c8f0 2023-10-05T19:40:16.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.007+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b76e0 con 0x7f7f88007030 2023-10-05T19:40:16.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.007+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x7f7f88007030 2023-10-05T19:40:16.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.007+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b76e0 con 0x7f7f88007030 2023-10-05T19:40:16.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.007+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x7f7f88007030 2023-10-05T19:40:16.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.007+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b76e0 con 0x562b656af760 2023-10-05T19:40:16.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.011+0000 7f7f95897700 1 -- 172.21.15.12:0/1283401693 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f8801cc20 con 0x562b656af760 2023-10-05T19:40:16.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.011+0000 7f7f95897700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b76e0 con 0x562b656b40e0 2023-10-05T19:40:16.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.011+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f80006070 con 0x562b656b40e0 2023-10-05T19:40:16.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.011+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b76e0 con 0x562b6567c8f0 2023-10-05T19:40:16.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.011+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x562b6567c8f0 2023-10-05T19:40:16.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.011+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b76e0 con 0x7f7f88007030 2023-10-05T19:40:16.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.011+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x7f7f88007030 2023-10-05T19:40:16.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.011+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b76e0 con 0x562b6567c8f0 2023-10-05T19:40:16.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.015+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x562b6567c8f0 2023-10-05T19:40:16.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.015+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b76e0 con 0x562b656af760 2023-10-05T19:40:16.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.015+0000 7f7f95897700 1 -- 172.21.15.12:0/1283401693 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f8801cc20 con 0x562b656af760 2023-10-05T19:40:16.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.015+0000 7f7f95897700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b76e0 con 0x562b656b40e0 2023-10-05T19:40:16.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.015+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f80006070 con 0x562b656b40e0 2023-10-05T19:40:16.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.015+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b76e0 con 0x562b656b40e0 2023-10-05T19:40:16.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.015+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f80006070 con 0x562b656b40e0 2023-10-05T19:40:16.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.015+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b76e0 con 0x7f7f88007030 2023-10-05T19:40:16.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.015+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x7f7f88007030 2023-10-05T19:40:16.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.015+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b76e0 con 0x562b656b40e0 2023-10-05T19:40:16.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.019+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f80006070 con 0x562b656b40e0 2023-10-05T19:40:16.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.019+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b76e0 con 0x562b656b40e0 2023-10-05T19:40:16.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.019+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f80006070 con 0x562b656b40e0 2023-10-05T19:40:16.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.019+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b76e0 con 0x562b6567c8f0 2023-10-05T19:40:16.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.019+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x562b6567c8f0 2023-10-05T19:40:16.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.019+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b76e0 con 0x562b6567c8f0 2023-10-05T19:40:16.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.019+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x562b6567c8f0 2023-10-05T19:40:16.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.019+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b76e0 con 0x7f7f88007030 2023-10-05T19:40:16.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.019+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x7f7f88007030 2023-10-05T19:40:16.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.023+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b76e0 con 0x562b656b40e0 2023-10-05T19:40:16.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.023+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f80006070 con 0x562b656b40e0 2023-10-05T19:40:16.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.023+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b76e0 con 0x562b656af760 2023-10-05T19:40:16.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.023+0000 7f7f95897700 1 -- 172.21.15.12:0/1283401693 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f8801cc20 con 0x562b656af760 2023-10-05T19:40:16.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.023+0000 7f7f95897700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b76e0 con 0x562b6567c8f0 2023-10-05T19:40:16.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.023+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x562b6567c8f0 2023-10-05T19:40:16.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.023+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b76e0 con 0x7f7f88007030 2023-10-05T19:40:16.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.023+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x7f7f88007030 2023-10-05T19:40:16.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.023+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b76e0 con 0x562b656b40e0 2023-10-05T19:40:16.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.027+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f80006070 con 0x562b656b40e0 2023-10-05T19:40:16.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.027+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b76e0 con 0x7f7f88007030 2023-10-05T19:40:16.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.027+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x7f7f88007030 2023-10-05T19:40:16.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.027+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b76e0 con 0x562b656af760 2023-10-05T19:40:16.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.027+0000 7f7f95897700 1 -- 172.21.15.12:0/1283401693 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f8801cc20 con 0x562b656af760 2023-10-05T19:40:16.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.027+0000 7f7f95897700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b76e0 con 0x562b656b40e0 2023-10-05T19:40:16.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.027+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f80006070 con 0x562b656b40e0 2023-10-05T19:40:16.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.027+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b76e0 con 0x7f7f88007030 2023-10-05T19:40:16.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.027+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x7f7f88007030 2023-10-05T19:40:16.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.027+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b76e0 con 0x562b6567c8f0 2023-10-05T19:40:16.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.031+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f7f9000a5a0 con 0x562b6567c8f0 2023-10-05T19:40:16.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.031+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b76e0 con 0x562b656af760 2023-10-05T19:40:16.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.031+0000 7f7f95897700 1 -- 172.21.15.12:0/1283401693 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f7f8801cc20 con 0x562b656af760 2023-10-05T19:40:16.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.031+0000 7f7f97d62e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:16.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.031+0000 7f7f97d62e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:16.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.031+0000 7f7f97d62e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:16.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.031+0000 7f7f97d62e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:16.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.031+0000 7f7f97d62e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.031+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b7a90 con 0x562b6567c8f0 2023-10-05T19:40:16.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.031+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f7f9000a5a0 con 0x562b6567c8f0 2023-10-05T19:40:16.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.031+0000 7f7f97d62e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:16.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.031+0000 7f7f97d62e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.031+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b7e40 con 0x562b656af760 2023-10-05T19:40:16.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.031+0000 7f7f95897700 1 -- 172.21.15.12:0/1283401693 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv1 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f7f8801cc20 con 0x562b656af760 2023-10-05T19:40:16.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.031+0000 7f7f97d62e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:16.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.031+0000 7f7f97d62e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.031+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b86e0 con 0x562b6567c8f0 2023-10-05T19:40:16.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.035+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f7f9000a5a0 con 0x562b6567c8f0 2023-10-05T19:40:16.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.035+0000 7f7f97d62e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:16.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.035+0000 7f7f97d62e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.035+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b86e0 con 0x562b656b40e0 2023-10-05T19:40:16.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.035+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f7f80006070 con 0x562b656b40e0 2023-10-05T19:40:16.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.035+0000 7f7f97d62e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:16.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.035+0000 7f7f97d62e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.035+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b8220 con 0x7f7f88007030 2023-10-05T19:40:16.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.035+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f7f9000a5a0 con 0x7f7f88007030 2023-10-05T19:40:16.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.035+0000 7f7f97d62e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:16.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.035+0000 7f7f97d62e40 20 zone default found 2023-10-05T19:40:16.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.035+0000 7f7f97d62e40 4 Realm: () 2023-10-05T19:40:16.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.035+0000 7f7f97d62e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:16.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.035+0000 7f7f97d62e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:16.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.035+0000 7f7f97d62e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:16.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.035+0000 7f7f97d62e40 20 zonegroup default 2023-10-05T19:40:16.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.035+0000 7f7f97d62e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.035+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b8220 con 0x7f7f88007030 2023-10-05T19:40:16.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.035+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f7f9000a5a0 con 0x7f7f88007030 2023-10-05T19:40:16.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.035+0000 7f7f97d62e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:16.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.035+0000 7f7f97d62e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.035+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b8220 con 0x562b656b40e0 2023-10-05T19:40:16.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.039+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f7f80006070 con 0x562b656b40e0 2023-10-05T19:40:16.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.039+0000 7f7f97d62e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:16.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.039+0000 7f7f97d62e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.039+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656ba900 con 0x562b656b40e0 2023-10-05T19:40:16.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.039+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f7f80006070 con 0x562b656b40e0 2023-10-05T19:40:16.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.039+0000 7f7f97d62e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:16.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.039+0000 7f7f97d62e40 20 started sync module instance, tier type = 2023-10-05T19:40:16.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.039+0000 7f7f97d62e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:16.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.039+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b9fc0 con 0x562b656af760 2023-10-05T19:40:16.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.043+0000 7f7f95897700 1 -- 172.21.15.12:0/1283401693 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'3 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7f8801cc20 con 0x562b656af760 2023-10-05T19:40:16.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.043+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94744385133264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bacb0 con 0x562b656af760 2023-10-05T19:40:16.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.043+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bb5b0 con 0x562b6567c8f0 2023-10-05T19:40:16.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.043+0000 7f7f95897700 1 -- 172.21.15.12:0/1283401693 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94744385133264] v28'4 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7f8801cc20 con 0x562b656af760 2023-10-05T19:40:16.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.043+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'3 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7f9000a5a0 con 0x562b6567c8f0 2023-10-05T19:40:16.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.043+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94744385138880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bb510 con 0x562b6567c8f0 2023-10-05T19:40:16.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.043+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bc390 con 0x562b656b40e0 2023-10-05T19:40:16.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.047+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94744385138880] v28'4 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7f9000a5a0 con 0x562b6567c8f0 2023-10-05T19:40:16.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.047+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'3 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7f80006070 con 0x562b656b40e0 2023-10-05T19:40:16.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.047+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94744385141168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bc740 con 0x562b656b40e0 2023-10-05T19:40:16.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.047+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bcfe0 con 0x562b656af760 2023-10-05T19:40:16.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.051+0000 7f7f95897700 1 -- 172.21.15.12:0/1283401693 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'3 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7f8801cc20 con 0x562b656af760 2023-10-05T19:40:16.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.051+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94744385145872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bd5d0 con 0x562b656af760 2023-10-05T19:40:16.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.051+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656be190 con 0x562b6567c8f0 2023-10-05T19:40:16.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.051+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94744385141168] v28'4 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7f80006070 con 0x562b656b40e0 2023-10-05T19:40:16.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.055+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'3 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7f9000a5a0 con 0x562b6567c8f0 2023-10-05T19:40:16.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.055+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94744385142272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656be150 con 0x562b6567c8f0 2023-10-05T19:40:16.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.055+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656beb60 con 0x7f7f88007030 2023-10-05T19:40:16.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.055+0000 7f7f95897700 1 -- 172.21.15.12:0/1283401693 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94744385145872] v28'4 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7f8801cc20 con 0x562b656af760 2023-10-05T19:40:16.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.059+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'3 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7f9000a5a0 con 0x7f7f88007030 2023-10-05T19:40:16.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.059+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94744385142272] v28'4 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7f9000a5a0 con 0x562b6567c8f0 2023-10-05T19:40:16.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.059+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94744385146976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656beb60 con 0x7f7f88007030 2023-10-05T19:40:16.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.059+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bf450 con 0x7f7f88007030 2023-10-05T19:40:16.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.063+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(95 notify.5 [watch watch cookie 94744385146976] v28'4 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7f9000a5a0 con 0x7f7f88007030 2023-10-05T19:40:16.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.063+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(96 notify.6 [create] v28'3 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7f9000a5a0 con 0x7f7f88007030 2023-10-05T19:40:16.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.063+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94744385152624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656befb0 con 0x7f7f88007030 2023-10-05T19:40:16.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.063+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bf9c0 con 0x7f7f88007030 2023-10-05T19:40:16.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.067+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(97 notify.6 [watch watch cookie 94744385152624] v28'4 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7f9000a5a0 con 0x7f7f88007030 2023-10-05T19:40:16.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.067+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(98 notify.7 [create] v28'3 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7f9000a5a0 con 0x7f7f88007030 2023-10-05T19:40:16.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.067+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94744385154000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bf9c0 con 0x7f7f88007030 2023-10-05T19:40:16.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.067+0000 7f7f97d62e40 20 add_watcher() i=0 2023-10-05T19:40:16.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.067+0000 7f7f97d62e40 20 add_watcher() i=1 2023-10-05T19:40:16.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.067+0000 7f7f97d62e40 20 add_watcher() i=2 2023-10-05T19:40:16.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.067+0000 7f7f97d62e40 20 add_watcher() i=3 2023-10-05T19:40:16.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.067+0000 7f7f97d62e40 20 add_watcher() i=4 2023-10-05T19:40:16.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.067+0000 7f7f97d62e40 20 add_watcher() i=5 2023-10-05T19:40:16.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.067+0000 7f7f97d62e40 20 add_watcher() i=6 2023-10-05T19:40:16.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.067+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94744385154000] v28'4 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7f9000a5a0 con 0x7f7f88007030 2023-10-05T19:40:16.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.067+0000 7f7f97d62e40 20 add_watcher() i=7 2023-10-05T19:40:16.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.067+0000 7f7f97d62e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:16.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.067+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656c2a60 con 0x562b6567c8f0 2023-10-05T19:40:16.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.071+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f7f9000a5a0 con 0x562b6567c8f0 2023-10-05T19:40:16.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.071+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94744385156304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bfe10 con 0x562b6567c8f0 2023-10-05T19:40:16.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.071+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94744385156304] v28'24 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f7f9000a5a0 con 0x562b6567c8f0 2023-10-05T19:40:16.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.071+0000 7f7f667ec700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:16.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.071+0000 7f7f97d62e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:16.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.071+0000 7f7f97d62e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:16.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.075+0000 7f7f97d62e40 1 Processor -- start 2023-10-05T19:40:16.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.075+0000 7f7f97d62e40 1 -- start start 2023-10-05T19:40:16.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.075+0000 7f7f97d62e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562b656ccfd0 0x562b656cd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:16.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.075+0000 7f7f97d62e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b656cd8e0 con 0x562b656ccfd0 2023-10-05T19:40:16.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.075+0000 7f7f95096700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562b656ccfd0 0x562b656cd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:16.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.075+0000 7f7f95096700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562b656ccfd0 0x562b656cd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36274/0 (socket says 172.21.15.12:36274) 2023-10-05T19:40:16.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.075+0000 7f7f95096700 1 -- 172.21.15.12:0/453729342 learned_addr learned my addr 172.21.15.12:0/453729342 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:16.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.075+0000 7f7f95096700 1 -- 172.21.15.12:0/453729342 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b656cda20 con 0x562b656ccfd0 2023-10-05T19:40:16.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.075+0000 7f7f95096700 1 --2- 172.21.15.12:0/453729342 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562b656ccfd0 0x562b656cd3a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7f88001eb0 tx=0x7f7f8001e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3517769e5ab7e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:16.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.075+0000 7f7f65feb700 1 -- 172.21.15.12:0/453729342 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7f80013040 con 0x562b656ccfd0 2023-10-05T19:40:16.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.075+0000 7f7f65feb700 1 -- 172.21.15.12:0/453729342 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f80022040 con 0x562b656ccfd0 2023-10-05T19:40:16.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.075+0000 7f7f65feb700 1 -- 172.21.15.12:0/453729342 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7f8000ee90 con 0x562b656ccfd0 2023-10-05T19:40:16.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.079+0000 7f7f97d62e40 1 -- 172.21.15.12:0/453729342 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562b656ccfd0 msgr2=0x562b656cd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:16.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.079+0000 7f7f97d62e40 1 --2- 172.21.15.12:0/453729342 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562b656ccfd0 0x562b656cd3a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7f88001eb0 tx=0x7f7f8001e040 comp rx=0 tx=0).stop 2023-10-05T19:40:16.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.079+0000 7f7f97d62e40 1 -- 172.21.15.12:0/453729342 shutdown_connections 2023-10-05T19:40:16.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.079+0000 7f7f97d62e40 1 --2- 172.21.15.12:0/453729342 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562b656ccfd0 0x562b656cd3a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:16.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.079+0000 7f7f97d62e40 1 -- 172.21.15.12:0/453729342 >> 172.21.15.12:0/453729342 conn(0x562b656c8390 msgr2=0x562b656c6540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:16.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.079+0000 7f7f97d62e40 1 -- 172.21.15.12:0/453729342 shutdown_connections 2023-10-05T19:40:16.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.079+0000 7f7f97d62e40 1 -- 172.21.15.12:0/453729342 wait complete. 2023-10-05T19:40:16.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.079+0000 7f7f97d62e40 1 Processor -- start 2023-10-05T19:40:16.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.079+0000 7f7f97d62e40 1 -- start start 2023-10-05T19:40:16.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.079+0000 7f7f97d62e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562b656ccfd0 0x562b656d2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:16.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.079+0000 7f7f97d62e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b656d3b30 con 0x562b656ccfd0 2023-10-05T19:40:16.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.079+0000 7f7f95096700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562b656ccfd0 0x562b656d2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:16.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.079+0000 7f7f95096700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562b656ccfd0 0x562b656d2b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36278/0 (socket says 172.21.15.12:36278) 2023-10-05T19:40:16.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.079+0000 7f7f95096700 1 -- 172.21.15.12:0/2400067435 learned_addr learned my addr 172.21.15.12:0/2400067435 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:16.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.079+0000 7f7f95096700 1 -- 172.21.15.12:0/2400067435 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f80034040 con 0x562b656ccfd0 2023-10-05T19:40:16.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.079+0000 7f7f95096700 1 --2- 172.21.15.12:0/2400067435 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562b656ccfd0 0x562b656d2b30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7f80045000 tx=0x7f7f80040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:16.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.079+0000 7f7ed7fff700 1 -- 172.21.15.12:0/2400067435 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7f8003be90 con 0x562b656ccfd0 2023-10-05T19:40:16.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.079+0000 7f7ed7fff700 1 -- 172.21.15.12:0/2400067435 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f8002c640 con 0x562b656ccfd0 2023-10-05T19:40:16.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.079+0000 7f7f97d62e40 1 -- 172.21.15.12:0/2400067435 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562b656d3130 con 0x562b656ccfd0 2023-10-05T19:40:16.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.079+0000 7f7ed7fff700 1 -- 172.21.15.12:0/2400067435 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7f8002c7f0 con 0x562b656ccfd0 2023-10-05T19:40:16.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.079+0000 7f7f97d62e40 1 -- 172.21.15.12:0/2400067435 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562b656d3570 con 0x562b656ccfd0 2023-10-05T19:40:16.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.083+0000 7f7ed7fff700 1 -- 172.21.15.12:0/2400067435 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f7f8004d070 con 0x562b656ccfd0 2023-10-05T19:40:16.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.083+0000 7f7ed7fff700 1 --2- 172.21.15.12:0/2400067435 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f7edc030c50 0x7f7edc033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:16.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.083+0000 7f7ed7fff700 1 -- 172.21.15.12:0/2400067435 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f7f80050ad0 con 0x562b656ccfd0 2023-10-05T19:40:16.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.083+0000 7f7f97d62e40 5 note: GC not initialized 2023-10-05T19:40:16.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.083+0000 7f7f96098700 1 --2- 172.21.15.12:0/2400067435 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f7edc030c50 0x7f7edc033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:16.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.083+0000 7f7f96098700 1 --2- 172.21.15.12:0/2400067435 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f7edc030c50 0x7f7edc033100 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x562b656c2bd0 tx=0x7f7f9000adc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:16.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.087+0000 7f7ed57fa700 20 reqs_thread_entry: start 2023-10-05T19:40:16.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.087+0000 7f7eceffd700 20 reqs_thread_entry: start 2023-10-05T19:40:16.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.087+0000 7f7f97d62e40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:16.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.087+0000 7f7f97d62e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:16.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.087+0000 7f7ecd7fa700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:46 2023 2023-10-05T19:40:16.096 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:16.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.087+0000 7f7eccff9700 20 reqs_thread_entry: start 2023-10-05T19:40:16.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.087+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:102 8.0 8:00000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b6573ae50 con 0x562b656b40e0 2023-10-05T19:40:16.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.091+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(102 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f80055070 con 0x562b656b40e0 2023-10-05T19:40:16.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.091+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:103 8.10 8:08000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bc740 con 0x562b656b40e0 2023-10-05T19:40:16.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.091+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 <== osd.3 v2:172.21.15.12:6816/29951 27 ==== osd_op_reply(103 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f80055070 con 0x562b656b40e0 2023-10-05T19:40:16.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.091+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:104 8.8 8:10000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bc740 con 0x562b6567c8f0 2023-10-05T19:40:16.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.091+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(104 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x562b6567c8f0 2023-10-05T19:40:16.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.091+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:105 8.18 8:18000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b7a90 con 0x562b6567c8f0 2023-10-05T19:40:16.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.091+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(105 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x562b6567c8f0 2023-10-05T19:40:16.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.091+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:106 8.4 8:20000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b7a90 con 0x562b656af760 2023-10-05T19:40:16.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.091+0000 7f7f95897700 1 -- 172.21.15.12:0/1283401693 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(106 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f8801cc20 con 0x562b656af760 2023-10-05T19:40:16.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.091+0000 7f7f95897700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:107 8.14 8:28000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bacb0 con 0x562b656b40e0 2023-10-05T19:40:16.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.095+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 <== osd.3 v2:172.21.15.12:6816/29951 28 ==== osd_op_reply(107 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f80055070 con 0x562b656b40e0 2023-10-05T19:40:16.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.095+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:108 8.c 8:30000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bacb0 con 0x562b6567c8f0 2023-10-05T19:40:16.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.095+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(108 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x562b6567c8f0 2023-10-05T19:40:16.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.095+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:109 8.1c 8:38000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bacb0 con 0x562b656b40e0 2023-10-05T19:40:16.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.095+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 <== osd.3 v2:172.21.15.12:6816/29951 29 ==== osd_op_reply(109 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f80055070 con 0x562b656b40e0 2023-10-05T19:40:16.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.095+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:110 8.2 8:40000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bacb0 con 0x562b656af760 2023-10-05T19:40:16.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.095+0000 7f7f95897700 1 -- 172.21.15.12:0/1283401693 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(110 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f8801cc20 con 0x562b656af760 2023-10-05T19:40:16.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.095+0000 7f7f95897700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:111 8.12 8:48000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bacb0 con 0x562b656b40e0 2023-10-05T19:40:16.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.095+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 <== osd.3 v2:172.21.15.12:6816/29951 30 ==== osd_op_reply(111 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f80055070 con 0x562b656b40e0 2023-10-05T19:40:16.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.095+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:112 8.a 8:50000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bacb0 con 0x562b6567c8f0 2023-10-05T19:40:16.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.099+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(112 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x562b6567c8f0 2023-10-05T19:40:16.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.099+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:113 8.1a 8:58000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bacb0 con 0x7f7f88007030 2023-10-05T19:40:16.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.099+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(113 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x7f7f88007030 2023-10-05T19:40:16.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.099+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:114 8.6 8:60000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bacb0 con 0x7f7f88007030 2023-10-05T19:40:16.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.099+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(114 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x7f7f88007030 2023-10-05T19:40:16.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.099+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:115 8.16 8:68000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bacb0 con 0x562b656b40e0 2023-10-05T19:40:16.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.099+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 <== osd.3 v2:172.21.15.12:6816/29951 31 ==== osd_op_reply(115 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f80055070 con 0x562b656b40e0 2023-10-05T19:40:16.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.099+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:116 8.e 8:70000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bacb0 con 0x7f7f88007030 2023-10-05T19:40:16.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.103+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(116 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x7f7f88007030 2023-10-05T19:40:16.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.103+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:117 8.1e 8:78000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bacb0 con 0x7f7f88007030 2023-10-05T19:40:16.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.103+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.0 v2:172.21.15.8:6800/27563 30 ==== osd_op_reply(117 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x7f7f88007030 2023-10-05T19:40:16.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.103+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:118 8.1 8:80000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bacb0 con 0x7f7f88007030 2023-10-05T19:40:16.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.103+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.0 v2:172.21.15.8:6800/27563 31 ==== osd_op_reply(118 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x7f7f88007030 2023-10-05T19:40:16.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.103+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:119 8.11 8:88000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bacb0 con 0x562b656b40e0 2023-10-05T19:40:16.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.103+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 <== osd.3 v2:172.21.15.12:6816/29951 32 ==== osd_op_reply(119 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f80055070 con 0x562b656b40e0 2023-10-05T19:40:16.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.103+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:120 8.9 8:90000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bacb0 con 0x562b656af760 2023-10-05T19:40:16.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.103+0000 7f7f95897700 1 -- 172.21.15.12:0/1283401693 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(120 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f8801cc20 con 0x562b656af760 2023-10-05T19:40:16.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.103+0000 7f7f95897700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:121 8.19 8:98000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bacb0 con 0x562b6567c8f0 2023-10-05T19:40:16.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.107+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(121 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x562b6567c8f0 2023-10-05T19:40:16.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.107+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:122 8.5 8:a0000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bacb0 con 0x562b6567c8f0 2023-10-05T19:40:16.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.107+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== osd_op_reply(122 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x562b6567c8f0 2023-10-05T19:40:16.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.107+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:123 8.15 8:a8000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bacb0 con 0x562b656b40e0 2023-10-05T19:40:16.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.107+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 <== osd.3 v2:172.21.15.12:6816/29951 33 ==== osd_op_reply(123 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f80055070 con 0x562b656b40e0 2023-10-05T19:40:16.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.107+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:124 8.d 8:b0000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bacb0 con 0x562b656af760 2023-10-05T19:40:16.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.107+0000 7f7f95897700 1 -- 172.21.15.12:0/1283401693 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(124 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f8801cc20 con 0x562b656af760 2023-10-05T19:40:16.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.107+0000 7f7f95897700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:125 8.1d 8:b8000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bacb0 con 0x7f7f88007030 2023-10-05T19:40:16.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.111+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.0 v2:172.21.15.8:6800/27563 32 ==== osd_op_reply(125 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x7f7f88007030 2023-10-05T19:40:16.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.111+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:126 8.3 8:c0000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bacb0 con 0x562b6567c8f0 2023-10-05T19:40:16.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.111+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.1 v2:172.21.15.12:6808/29933 39 ==== osd_op_reply(126 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x562b6567c8f0 2023-10-05T19:40:16.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.111+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:127 8.13 8:c8000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bacb0 con 0x7f7f88007030 2023-10-05T19:40:16.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.111+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.0 v2:172.21.15.8:6800/27563 33 ==== osd_op_reply(127 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x7f7f88007030 2023-10-05T19:40:16.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.111+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:128 8.b 8:d0000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bacb0 con 0x562b656b40e0 2023-10-05T19:40:16.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.111+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 <== osd.3 v2:172.21.15.12:6816/29951 34 ==== osd_op_reply(128 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f80055070 con 0x562b656b40e0 2023-10-05T19:40:16.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.111+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:129 8.1b 8:d8000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bacb0 con 0x562b656af760 2023-10-05T19:40:16.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.111+0000 7f7f95897700 1 -- 172.21.15.12:0/1283401693 <== osd.2 v2:172.21.15.12:6800/29934 23 ==== osd_op_reply(129 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f8801cc20 con 0x562b656af760 2023-10-05T19:40:16.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.111+0000 7f7f95897700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:130 8.7 8:e0000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bacb0 con 0x7f7f88007030 2023-10-05T19:40:16.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.115+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.0 v2:172.21.15.8:6800/27563 34 ==== osd_op_reply(130 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x7f7f88007030 2023-10-05T19:40:16.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.115+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:131 8.17 8:e8000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bacb0 con 0x562b6567c8f0 2023-10-05T19:40:16.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.115+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.1 v2:172.21.15.12:6808/29933 40 ==== osd_op_reply(131 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f9000a5a0 con 0x562b6567c8f0 2023-10-05T19:40:16.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.115+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:132 8.f 8:f0000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bacb0 con 0x562b656b40e0 2023-10-05T19:40:16.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.115+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 <== osd.3 v2:172.21.15.12:6816/29951 35 ==== osd_op_reply(132 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f80055070 con 0x562b656b40e0 2023-10-05T19:40:16.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.115+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:133 8.1f 8:f8000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bacb0 con 0x562b656af760 2023-10-05T19:40:16.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.115+0000 7f7f95897700 1 -- 172.21.15.12:0/1283401693 <== osd.2 v2:172.21.15.12:6800/29934 24 ==== osd_op_reply(133 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7f8801cc20 con 0x562b656af760 2023-10-05T19:40:16.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.119+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:134 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94744385133264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bc740 con 0x562b656af760 2023-10-05T19:40:16.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.119+0000 7f7f95897700 1 -- 172.21.15.12:0/1283401693 <== osd.2 v2:172.21.15.12:6800/29934 25 ==== osd_op_reply(134 notify.0 [watch unwatch cookie 94744385133264] v28'5 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7f8801cc20 con 0x562b656af760 2023-10-05T19:40:16.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.119+0000 7f7f97d62e40 20 remove_watcher() i=0 2023-10-05T19:40:16.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.119+0000 7f7f97d62e40 2 removed watcher, disabling cache 2023-10-05T19:40:16.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.119+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:135 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94744385138880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656b7a90 con 0x562b6567c8f0 2023-10-05T19:40:16.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.123+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.1 v2:172.21.15.12:6808/29933 41 ==== osd_op_reply(135 notify.1 [watch unwatch cookie 94744385138880] v28'5 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7f9000a5a0 con 0x562b6567c8f0 2023-10-05T19:40:16.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.123+0000 7f7f97d62e40 20 remove_watcher() i=1 2023-10-05T19:40:16.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.123+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:136 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94744385141168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bc740 con 0x562b656b40e0 2023-10-05T19:40:16.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.127+0000 7f7f95096700 1 -- 172.21.15.12:0/1283401693 <== osd.3 v2:172.21.15.12:6816/29951 36 ==== osd_op_reply(136 notify.2 [watch unwatch cookie 94744385141168] v28'5 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7f80055070 con 0x562b656b40e0 2023-10-05T19:40:16.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.127+0000 7f7f97d62e40 20 remove_watcher() i=2 2023-10-05T19:40:16.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.127+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:137 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94744385145872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bb0c0 con 0x562b656af760 2023-10-05T19:40:16.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.131+0000 7f7f95897700 1 -- 172.21.15.12:0/1283401693 <== osd.2 v2:172.21.15.12:6800/29934 26 ==== osd_op_reply(137 notify.3 [watch unwatch cookie 94744385145872] v28'5 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7f8801cc20 con 0x562b656af760 2023-10-05T19:40:16.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.131+0000 7f7f97d62e40 20 remove_watcher() i=3 2023-10-05T19:40:16.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.131+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:138 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94744385142272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bb9b0 con 0x562b6567c8f0 2023-10-05T19:40:16.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.131+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.1 v2:172.21.15.12:6808/29933 42 ==== osd_op_reply(138 notify.4 [watch unwatch cookie 94744385142272] v28'5 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7f9000a5a0 con 0x562b6567c8f0 2023-10-05T19:40:16.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.131+0000 7f7f97d62e40 20 remove_watcher() i=4 2023-10-05T19:40:16.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.131+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:139 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94744385146976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bbea0 con 0x7f7f88007030 2023-10-05T19:40:16.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.135+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.0 v2:172.21.15.8:6800/27563 35 ==== osd_op_reply(139 notify.5 [watch unwatch cookie 94744385146976] v28'5 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7f9000a5a0 con 0x7f7f88007030 2023-10-05T19:40:16.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.135+0000 7f7f97d62e40 20 remove_watcher() i=5 2023-10-05T19:40:16.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.135+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:140 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94744385152624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bbea0 con 0x7f7f88007030 2023-10-05T19:40:16.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.139+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.0 v2:172.21.15.8:6800/27563 36 ==== osd_op_reply(140 notify.6 [watch unwatch cookie 94744385152624] v28'5 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7f9000a5a0 con 0x7f7f88007030 2023-10-05T19:40:16.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.139+0000 7f7f97d62e40 20 remove_watcher() i=6 2023-10-05T19:40:16.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.139+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:141 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94744385154000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656bbea0 con 0x7f7f88007030 2023-10-05T19:40:16.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.143+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.0 v2:172.21.15.8:6800/27563 37 ==== osd_op_reply(141 notify.7 [watch unwatch cookie 94744385154000] v28'5 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7f9000a5a0 con 0x7f7f88007030 2023-10-05T19:40:16.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.143+0000 7f7f97d62e40 20 remove_watcher() i=7 2023-10-05T19:40:16.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.143+0000 7f7f667ec700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:16.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.143+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:142 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94744385156304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562b656fbf30 con 0x562b6567c8f0 2023-10-05T19:40:16.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.147+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 <== osd.1 v2:172.21.15.12:6808/29933 43 ==== osd_op_reply(142 data_loggenerations_metadata [watch unwatch cookie 94744385156304] v28'25 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f7f9000a5a0 con 0x562b6567c8f0 2023-10-05T19:40:16.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.147+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f7f88007030 msgr2=0x7f7f880113b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:16.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.147+0000 7f7f97d62e40 1 --2- 172.21.15.12:0/1283401693 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f7f88007030 0x7f7f880113b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:16.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.147+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x562b6567c8f0 msgr2=0x562b656ad570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:16.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.147+0000 7f7f97d62e40 1 --2- 172.21.15.12:0/1283401693 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x562b6567c8f0 0x562b656ad570 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:16.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.147+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x562b656af760 msgr2=0x562b656b1b40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:16.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.147+0000 7f7f97d62e40 1 --2- 172.21.15.12:0/1283401693 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x562b656af760 0x562b656b1b40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:16.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.147+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x562b656b40e0 msgr2=0x562b656b64c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:16.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.147+0000 7f7f97d62e40 1 --2- 172.21.15.12:0/1283401693 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x562b656b40e0 0x562b656b64c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:16.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.147+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f7f7c030ca0 msgr2=0x7f7f7c033150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:16.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.147+0000 7f7f97d62e40 1 --2- 172.21.15.12:0/1283401693 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f7f7c030ca0 0x7f7f7c033150 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7f80004170 tx=0x7f7f80001400 comp rx=0 tx=0).stop 2023-10-05T19:40:16.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.147+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562b6567bfe0 msgr2=0x562b656a1ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:16.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.147+0000 7f7f97d62e40 1 --2- 172.21.15.12:0/1283401693 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562b6567bfe0 0x562b656a1ba0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7f8801f040 tx=0x7f7f8801c000 comp rx=0 tx=0).stop 2023-10-05T19:40:16.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.147+0000 7f7f96098700 1 -- 172.21.15.12:0/1283401693 reap_dead start 2023-10-05T19:40:16.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.147+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 shutdown_connections 2023-10-05T19:40:16.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.147+0000 7f7f97d62e40 1 --2- 172.21.15.12:0/1283401693 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562b6567bfe0 0x562b656a1ba0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:16.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.147+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 >> 172.21.15.12:0/1283401693 conn(0x562b655ef830 msgr2=0x562b65695520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:16.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.147+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 shutdown_connections 2023-10-05T19:40:16.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.147+0000 7f7f97d62e40 1 -- 172.21.15.12:0/1283401693 wait complete. 2023-10-05T19:40:16.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.147+0000 7f7f97d62e40 1 -- 172.21.15.12:0/2400067435 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f7edc030c50 msgr2=0x7f7edc033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:16.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.147+0000 7f7f97d62e40 1 --2- 172.21.15.12:0/2400067435 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f7edc030c50 0x7f7edc033100 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x562b656c2bd0 tx=0x7f7f9000adc0 comp rx=0 tx=0).stop 2023-10-05T19:40:16.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.147+0000 7f7f97d62e40 1 -- 172.21.15.12:0/2400067435 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562b656ccfd0 msgr2=0x562b656d2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:16.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.151+0000 7f7f97d62e40 1 --2- 172.21.15.12:0/2400067435 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562b656ccfd0 0x562b656d2b30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7f80045000 tx=0x7f7f80040000 comp rx=0 tx=0).stop 2023-10-05T19:40:16.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.151+0000 7f7f97d62e40 1 -- 172.21.15.12:0/2400067435 shutdown_connections 2023-10-05T19:40:16.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.151+0000 7f7f97d62e40 1 --2- 172.21.15.12:0/2400067435 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f7edc030c50 0x7f7edc033100 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:16.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.151+0000 7f7f97d62e40 1 --2- 172.21.15.12:0/2400067435 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562b656ccfd0 0x562b656d2b30 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:16.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.151+0000 7f7f97d62e40 1 -- 172.21.15.12:0/2400067435 >> 172.21.15.12:0/2400067435 conn(0x562b656c8390 msgr2=0x562b656c6490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:16.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.151+0000 7f7f97d62e40 1 -- 172.21.15.12:0/2400067435 shutdown_connections 2023-10-05T19:40:16.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.151+0000 7f7f97d62e40 1 -- 172.21.15.12:0/2400067435 wait complete. 2023-10-05T19:40:16.161 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2023-10-05T19:40:16.165 DEBUG:tasks.util.rgw: json result: [] 2023-10-05T19:40:16.165 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'bud'] 2023-10-05T19:40:16.165 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'bud'] 2023-10-05T19:40:16.165 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid bud 2023-10-05T19:40:16.244 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:16.245 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:16.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.235+0000 7f001af13e40 1 Processor -- start 2023-10-05T19:40:16.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.235+0000 7f001af13e40 1 -- start start 2023-10-05T19:40:16.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.235+0000 7f001af13e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x562852b765f0 0x562852b789d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:16.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.235+0000 7f001af13e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x562852abcbd0 con 0x562852b72010 2023-10-05T19:40:16.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.235+0000 7f001af13e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x562852a79ee0 con 0x562852b765f0 2023-10-05T19:40:16.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.239+0000 7f0018247700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x562852b765f0 0x562852b789d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:16.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.239+0000 7f0018247700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x562852b765f0 0x562852b789d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36284/0 (socket says 172.21.15.12:36284) 2023-10-05T19:40:16.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.239+0000 7f0018247700 1 -- 172.21.15.12:0/3042050002 learned_addr learned my addr 172.21.15.12:0/3042050002 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:16.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.239+0000 7f0018247700 1 -- 172.21.15.12:0/3042050002 >> v1:172.21.15.8:6789/0 conn(0x562852b72010 legacy=0x562852b723e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:16.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.239+0000 7f0018247700 1 -- 172.21.15.12:0/3042050002 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562852b72c20 con 0x562852b765f0 2023-10-05T19:40:16.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.239+0000 7f0018247700 1 --2- 172.21.15.12:0/3042050002 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562852b765f0 0x562852b789d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f000c00a650 tx=0x7f000c005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f92c4d55b2652e96 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:16.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.239+0000 7f0017a46700 1 -- 172.21.15.12:0/3042050002 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f000c008760 con 0x562852b765f0 2023-10-05T19:40:16.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.239+0000 7f0017a46700 1 -- 172.21.15.12:0/3042050002 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f000c0088c0 con 0x562852b765f0 2023-10-05T19:40:16.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.239+0000 7f0017a46700 1 -- 172.21.15.12:0/3042050002 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f000c008bb0 con 0x562852b765f0 2023-10-05T19:40:16.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.239+0000 7f001af13e40 1 -- 172.21.15.12:0/3042050002 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562852b765f0 msgr2=0x562852b789d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:16.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.239+0000 7f001af13e40 1 --2- 172.21.15.12:0/3042050002 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562852b765f0 0x562852b789d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f000c00a650 tx=0x7f000c005b10 comp rx=0 tx=0).stop 2023-10-05T19:40:16.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.239+0000 7f001af13e40 1 -- 172.21.15.12:0/3042050002 shutdown_connections 2023-10-05T19:40:16.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.239+0000 7f001af13e40 1 --2- 172.21.15.12:0/3042050002 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562852b765f0 0x562852b789d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:16.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.239+0000 7f001af13e40 1 -- 172.21.15.12:0/3042050002 >> 172.21.15.12:0/3042050002 conn(0x562852ae5860 msgr2=0x562852ae5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:16.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.239+0000 7f001af13e40 1 -- 172.21.15.12:0/3042050002 shutdown_connections 2023-10-05T19:40:16.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.239+0000 7f001af13e40 1 -- 172.21.15.12:0/3042050002 wait complete. 2023-10-05T19:40:16.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.239+0000 7f001af13e40 1 Processor -- start 2023-10-05T19:40:16.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.239+0000 7f001af13e40 1 -- start start 2023-10-05T19:40:16.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.239+0000 7f001af13e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562852b72010 0x562852b91190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:16.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.239+0000 7f001af13e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x562852a3de30 con 0x562852b72010 2023-10-05T19:40:16.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.243+0000 7f0018a48700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562852b72010 0x562852b91190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:16.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.243+0000 7f0018a48700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562852b72010 0x562852b91190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36294/0 (socket says 172.21.15.12:36294) 2023-10-05T19:40:16.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.243+0000 7f0018a48700 1 -- 172.21.15.12:0/2874531965 learned_addr learned my addr 172.21.15.12:0/2874531965 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:16.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.243+0000 7f0018a48700 1 -- 172.21.15.12:0/2874531965 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f000c005db0 con 0x562852b72010 2023-10-05T19:40:16.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.243+0000 7f0018a48700 1 --2- 172.21.15.12:0/2874531965 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562852b72010 0x562852b91190 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0008006fd0 tx=0x7f0008006ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23052569c1a27437 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:16.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.243+0000 7f0016a44700 1 -- 172.21.15.12:0/2874531965 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0008004130 con 0x562852b72010 2023-10-05T19:40:16.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.243+0000 7f0016a44700 1 -- 172.21.15.12:0/2874531965 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f000800e470 con 0x562852b72010 2023-10-05T19:40:16.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.243+0000 7f0016a44700 1 -- 172.21.15.12:0/2874531965 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f000800e760 con 0x562852b72010 2023-10-05T19:40:16.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.243+0000 7f001af13e40 1 -- 172.21.15.12:0/2874531965 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562852b72010 msgr2=0x562852b91190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:16.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.243+0000 7f001af13e40 1 --2- 172.21.15.12:0/2874531965 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562852b72010 0x562852b91190 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0008006fd0 tx=0x7f0008006ba0 comp rx=0 tx=0).stop 2023-10-05T19:40:16.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.243+0000 7f001af13e40 1 -- 172.21.15.12:0/2874531965 shutdown_connections 2023-10-05T19:40:16.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.243+0000 7f001af13e40 1 --2- 172.21.15.12:0/2874531965 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562852b72010 0x562852b91190 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:16.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.243+0000 7f001af13e40 1 -- 172.21.15.12:0/2874531965 >> 172.21.15.12:0/2874531965 conn(0x562852ae5860 msgr2=0x562852b8cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:16.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.243+0000 7f001af13e40 1 -- 172.21.15.12:0/2874531965 shutdown_connections 2023-10-05T19:40:16.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.243+0000 7f001af13e40 1 -- 172.21.15.12:0/2874531965 wait complete. 2023-10-05T19:40:16.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.243+0000 7f001af13e40 1 Processor -- start 2023-10-05T19:40:16.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.243+0000 7f001af13e40 1 -- start start 2023-10-05T19:40:16.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.243+0000 7f001af13e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562852b72010 0x562852b97f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:16.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.243+0000 7f001af13e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x562852a10ed0 con 0x562852b72010 2023-10-05T19:40:16.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.243+0000 7f0018a48700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562852b72010 0x562852b97f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:16.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.243+0000 7f0018a48700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562852b72010 0x562852b97f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36298/0 (socket says 172.21.15.12:36298) 2023-10-05T19:40:16.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.243+0000 7f0018a48700 1 -- 172.21.15.12:0/37997056 learned_addr learned my addr 172.21.15.12:0/37997056 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:16.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.247+0000 7f0018a48700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562852abcbd0 con 0x562852b72010 2023-10-05T19:40:16.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.247+0000 7f0018a48700 1 --2- 172.21.15.12:0/37997056 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562852b72010 0x562852b97f70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0008001350 tx=0x7f0008013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:16.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.247+0000 7f0015241700 1 -- 172.21.15.12:0/37997056 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f000801a070 con 0x562852b72010 2023-10-05T19:40:16.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.247+0000 7f0015241700 1 -- 172.21.15.12:0/37997056 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0008018070 con 0x562852b72010 2023-10-05T19:40:16.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.247+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562852b984b0 con 0x562852b72010 2023-10-05T19:40:16.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.247+0000 7f0015241700 1 -- 172.21.15.12:0/37997056 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f00080068d0 con 0x562852b72010 2023-10-05T19:40:16.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.247+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562852b98890 con 0x562852b72010 2023-10-05T19:40:16.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.247+0000 7f001af13e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.247+0000 7f0015241700 1 -- 172.21.15.12:0/37997056 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f0008013bc0 con 0x562852b72010 2023-10-05T19:40:16.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.247+0000 7f0015241700 1 --2- 172.21.15.12:0/37997056 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f0004030c50 0x7f0004033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:16.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.247+0000 7f0015241700 1 -- 172.21.15.12:0/37997056 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f00080443f0 con 0x562852b72010 2023-10-05T19:40:16.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.247+0000 7f0018247700 1 --2- 172.21.15.12:0/37997056 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f0004030c50 0x7f0004033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:16.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.251+0000 7f001af13e40 1 --2- 172.21.15.12:0/37997056 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x562852b765f0 0x562852ba3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:16.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.251+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852ba4130 con 0x562852b765f0 2023-10-05T19:40:16.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.251+0000 7f0019249700 1 --2- 172.21.15.12:0/37997056 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x562852b765f0 0x562852ba3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:16.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.251+0000 7f0018247700 1 --2- 172.21.15.12:0/37997056 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f0004030c50 0x7f0004033100 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f000c0030a0 tx=0x7f000c015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:16.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.251+0000 7f0019249700 1 --2- 172.21.15.12:0/37997056 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x562852b765f0 0x562852ba3a30 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:16.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.251+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f001000a500 con 0x562852b765f0 2023-10-05T19:40:16.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.251+0000 7f001af13e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:16.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.251+0000 7f001af13e40 20 realm 2023-10-05T19:40:16.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.251+0000 7f001af13e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.251+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852ba1050 con 0x562852b765f0 2023-10-05T19:40:16.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.251+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f001000a500 con 0x562852b765f0 2023-10-05T19:40:16.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.251+0000 7f001af13e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:16.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.251+0000 7f001af13e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:16.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.251+0000 7f001af13e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.251+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852ba4c10 con 0x562852b765f0 2023-10-05T19:40:16.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.251+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f001000a500 con 0x562852b765f0 2023-10-05T19:40:16.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.251+0000 7f001af13e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:16.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.251+0000 7f001af13e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.251+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852ba47f0 con 0x562852b765f0 2023-10-05T19:40:16.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.251+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f001000a500 con 0x562852b765f0 2023-10-05T19:40:16.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.251+0000 7f001af13e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:16.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.251+0000 7f001af13e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.251+0000 7f001af13e40 1 --2- 172.21.15.12:0/37997056 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x562852ba5cd0 0x562852ba80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:16.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.251+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852ba8750 con 0x562852ba5cd0 2023-10-05T19:40:16.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.251+0000 7f0018a48700 1 --2- 172.21.15.12:0/37997056 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x562852ba5cd0 0x562852ba80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:16.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.255+0000 7f0018a48700 1 --2- 172.21.15.12:0/37997056 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x562852ba5cd0 0x562852ba80b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:16.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.255+0000 7f0018a48700 1 -- 172.21.15.12:0/37997056 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv1 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f000801d070 con 0x562852ba5cd0 2023-10-05T19:40:16.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.255+0000 7f001af13e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:16.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.255+0000 7f001af13e40 20 searching for the correct realm 2023-10-05T19:40:16.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.255+0000 7f001af13e40 1 --2- 172.21.15.12:0/37997056 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x562852baa690 0x562852bacaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:16.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.255+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bad150 con 0x562852baa690 2023-10-05T19:40:16.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.255+0000 7f0018247700 1 --2- 172.21.15.12:0/37997056 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x562852baa690 0x562852bacaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:16.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.255+0000 7f0018247700 1 --2- 172.21.15.12:0/37997056 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x562852baa690 0x562852bacaa0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:16.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.255+0000 7f0018247700 1 -- 172.21.15.12:0/37997056 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f000c008ad0 con 0x562852baa690 2023-10-05T19:40:16.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.255+0000 7f0018247700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bad150 con 0x562852ba5cd0 2023-10-05T19:40:16.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.259+0000 7f0018a48700 1 -- 172.21.15.12:0/37997056 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f000801d070 con 0x562852ba5cd0 2023-10-05T19:40:16.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.259+0000 7f0018a48700 1 --2- 172.21.15.12:0/37997056 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f000802a070 0x7f0008012a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:16.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.259+0000 7f0018a48700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bad150 con 0x7f000802a070 2023-10-05T19:40:16.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.259+0000 7f0019249700 1 --2- 172.21.15.12:0/37997056 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f000802a070 0x7f0008012a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:16.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.259+0000 7f0019249700 1 --2- 172.21.15.12:0/37997056 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f000802a070 0x7f0008012a80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:16.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.259+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0010014040 con 0x7f000802a070 2023-10-05T19:40:16.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.259+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bad150 con 0x562852b765f0 2023-10-05T19:40:16.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.259+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0010014040 con 0x562852b765f0 2023-10-05T19:40:16.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.259+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bad150 con 0x562852b765f0 2023-10-05T19:40:16.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.263+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0010014040 con 0x562852b765f0 2023-10-05T19:40:16.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.263+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bad150 con 0x7f000802a070 2023-10-05T19:40:16.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.263+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0010014040 con 0x7f000802a070 2023-10-05T19:40:16.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.263+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bad150 con 0x7f000802a070 2023-10-05T19:40:16.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.263+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0010014040 con 0x7f000802a070 2023-10-05T19:40:16.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.263+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bad150 con 0x562852ba5cd0 2023-10-05T19:40:16.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.263+0000 7f0018a48700 1 -- 172.21.15.12:0/37997056 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f000801d070 con 0x562852ba5cd0 2023-10-05T19:40:16.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.263+0000 7f0018a48700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bad150 con 0x562852baa690 2023-10-05T19:40:16.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.263+0000 7f0018247700 1 -- 172.21.15.12:0/37997056 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f000c008ad0 con 0x562852baa690 2023-10-05T19:40:16.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.263+0000 7f0018247700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bad150 con 0x562852b765f0 2023-10-05T19:40:16.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.267+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0010014040 con 0x562852b765f0 2023-10-05T19:40:16.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.267+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bad150 con 0x7f000802a070 2023-10-05T19:40:16.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.267+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0010014040 con 0x7f000802a070 2023-10-05T19:40:16.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.267+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bad150 con 0x562852b765f0 2023-10-05T19:40:16.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.267+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0010014040 con 0x562852b765f0 2023-10-05T19:40:16.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.267+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bad150 con 0x562852ba5cd0 2023-10-05T19:40:16.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.267+0000 7f0018a48700 1 -- 172.21.15.12:0/37997056 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f000801d070 con 0x562852ba5cd0 2023-10-05T19:40:16.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.267+0000 7f0018a48700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bad150 con 0x562852baa690 2023-10-05T19:40:16.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.267+0000 7f0018247700 1 -- 172.21.15.12:0/37997056 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f000c008ad0 con 0x562852baa690 2023-10-05T19:40:16.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.267+0000 7f0018247700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bad150 con 0x562852baa690 2023-10-05T19:40:16.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.271+0000 7f0018247700 1 -- 172.21.15.12:0/37997056 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f000c008ad0 con 0x562852baa690 2023-10-05T19:40:16.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.271+0000 7f0018247700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bad150 con 0x7f000802a070 2023-10-05T19:40:16.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.271+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0010014040 con 0x7f000802a070 2023-10-05T19:40:16.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.271+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bad150 con 0x562852baa690 2023-10-05T19:40:16.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.271+0000 7f0018247700 1 -- 172.21.15.12:0/37997056 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f000c008ad0 con 0x562852baa690 2023-10-05T19:40:16.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.271+0000 7f0018247700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bad150 con 0x562852baa690 2023-10-05T19:40:16.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.271+0000 7f0018247700 1 -- 172.21.15.12:0/37997056 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f000c008ad0 con 0x562852baa690 2023-10-05T19:40:16.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.271+0000 7f0018247700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bad150 con 0x562852b765f0 2023-10-05T19:40:16.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.271+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0010014040 con 0x562852b765f0 2023-10-05T19:40:16.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.271+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bad150 con 0x562852b765f0 2023-10-05T19:40:16.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.275+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0010014040 con 0x562852b765f0 2023-10-05T19:40:16.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.275+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bad150 con 0x7f000802a070 2023-10-05T19:40:16.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.275+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0010014040 con 0x7f000802a070 2023-10-05T19:40:16.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.275+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bad150 con 0x562852baa690 2023-10-05T19:40:16.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.275+0000 7f0018247700 1 -- 172.21.15.12:0/37997056 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f000c008ad0 con 0x562852baa690 2023-10-05T19:40:16.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.275+0000 7f0018247700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bad150 con 0x562852ba5cd0 2023-10-05T19:40:16.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.275+0000 7f0018a48700 1 -- 172.21.15.12:0/37997056 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f000801d070 con 0x562852ba5cd0 2023-10-05T19:40:16.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.275+0000 7f0018a48700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bad150 con 0x562852b765f0 2023-10-05T19:40:16.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.275+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0010014040 con 0x562852b765f0 2023-10-05T19:40:16.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.275+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bad150 con 0x7f000802a070 2023-10-05T19:40:16.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.279+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0010014040 con 0x7f000802a070 2023-10-05T19:40:16.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.279+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bad150 con 0x562852baa690 2023-10-05T19:40:16.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.279+0000 7f0018247700 1 -- 172.21.15.12:0/37997056 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f000c008ad0 con 0x562852baa690 2023-10-05T19:40:16.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.279+0000 7f0018247700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bad150 con 0x7f000802a070 2023-10-05T19:40:16.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.279+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0010014040 con 0x7f000802a070 2023-10-05T19:40:16.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.279+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bad150 con 0x562852ba5cd0 2023-10-05T19:40:16.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.279+0000 7f0018a48700 1 -- 172.21.15.12:0/37997056 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f000801d070 con 0x562852ba5cd0 2023-10-05T19:40:16.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.279+0000 7f0018a48700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bad150 con 0x562852baa690 2023-10-05T19:40:16.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.283+0000 7f0018247700 1 -- 172.21.15.12:0/37997056 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f000c008ad0 con 0x562852baa690 2023-10-05T19:40:16.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.283+0000 7f0018247700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bad150 con 0x7f000802a070 2023-10-05T19:40:16.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.283+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0010014040 con 0x7f000802a070 2023-10-05T19:40:16.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.283+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bad150 con 0x562852b765f0 2023-10-05T19:40:16.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.283+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f0010014040 con 0x562852b765f0 2023-10-05T19:40:16.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.283+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bad150 con 0x562852ba5cd0 2023-10-05T19:40:16.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.283+0000 7f0018a48700 1 -- 172.21.15.12:0/37997056 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f000801d070 con 0x562852ba5cd0 2023-10-05T19:40:16.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.283+0000 7f001af13e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:16.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.283+0000 7f001af13e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:16.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.283+0000 7f001af13e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:16.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.283+0000 7f001af13e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:16.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.283+0000 7f001af13e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.283+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852ba93e0 con 0x562852b765f0 2023-10-05T19:40:16.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.283+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f0010014040 con 0x562852b765f0 2023-10-05T19:40:16.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.283+0000 7f001af13e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:16.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.283+0000 7f001af13e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.283+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852ba9010 con 0x562852b765f0 2023-10-05T19:40:16.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.287+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f0010014040 con 0x562852b765f0 2023-10-05T19:40:16.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.287+0000 7f001af13e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:16.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.287+0000 7f001af13e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.287+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bad5c0 con 0x562852baa690 2023-10-05T19:40:16.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.287+0000 7f0018247700 1 -- 172.21.15.12:0/37997056 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f000c008ad0 con 0x562852baa690 2023-10-05T19:40:16.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.287+0000 7f001af13e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:16.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.287+0000 7f001af13e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.287+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bad970 con 0x562852baa690 2023-10-05T19:40:16.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.287+0000 7f0018247700 1 -- 172.21.15.12:0/37997056 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f000c008ad0 con 0x562852baa690 2023-10-05T19:40:16.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.287+0000 7f001af13e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:16.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.287+0000 7f001af13e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.287+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852badd20 con 0x562852b765f0 2023-10-05T19:40:16.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.287+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f0010014040 con 0x562852b765f0 2023-10-05T19:40:16.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.287+0000 7f001af13e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:16.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.287+0000 7f001af13e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.287+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bae100 con 0x562852b765f0 2023-10-05T19:40:16.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.287+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f0010014040 con 0x562852b765f0 2023-10-05T19:40:16.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.287+0000 7f001af13e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:16.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.291+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bae0d0 con 0x562852baa690 2023-10-05T19:40:16.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.291+0000 7f0018247700 1 -- 172.21.15.12:0/37997056 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f000c008ad0 con 0x562852baa690 2023-10-05T19:40:16.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.291+0000 7f0018247700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bae0d0 con 0x562852ba5cd0 2023-10-05T19:40:16.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.291+0000 7f0018a48700 1 -- 172.21.15.12:0/37997056 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f000801d070 con 0x562852ba5cd0 2023-10-05T19:40:16.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.291+0000 7f0018a48700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bae0d0 con 0x7f000802a070 2023-10-05T19:40:16.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.295+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0010014040 con 0x7f000802a070 2023-10-05T19:40:16.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.295+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852badd20 con 0x562852b765f0 2023-10-05T19:40:16.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.295+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0010014040 con 0x562852b765f0 2023-10-05T19:40:16.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.295+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852badd20 con 0x562852b765f0 2023-10-05T19:40:16.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.295+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0010014040 con 0x562852b765f0 2023-10-05T19:40:16.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.295+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852badd20 con 0x7f000802a070 2023-10-05T19:40:16.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.295+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0010014040 con 0x7f000802a070 2023-10-05T19:40:16.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.295+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852badd20 con 0x7f000802a070 2023-10-05T19:40:16.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.299+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0010014040 con 0x7f000802a070 2023-10-05T19:40:16.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.299+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852badd20 con 0x562852ba5cd0 2023-10-05T19:40:16.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.299+0000 7f0018a48700 1 -- 172.21.15.12:0/37997056 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f000801d070 con 0x562852ba5cd0 2023-10-05T19:40:16.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.299+0000 7f0018a48700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852badd20 con 0x562852baa690 2023-10-05T19:40:16.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.299+0000 7f0018247700 1 -- 172.21.15.12:0/37997056 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f000c008ad0 con 0x562852baa690 2023-10-05T19:40:16.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.299+0000 7f0018247700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852badd20 con 0x562852b765f0 2023-10-05T19:40:16.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.299+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0010014040 con 0x562852b765f0 2023-10-05T19:40:16.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.299+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852badd20 con 0x7f000802a070 2023-10-05T19:40:16.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.299+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0010014040 con 0x7f000802a070 2023-10-05T19:40:16.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.299+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852badd20 con 0x562852b765f0 2023-10-05T19:40:16.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.303+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0010014040 con 0x562852b765f0 2023-10-05T19:40:16.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.303+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852badd20 con 0x562852ba5cd0 2023-10-05T19:40:16.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.303+0000 7f0018a48700 1 -- 172.21.15.12:0/37997056 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f000801d070 con 0x562852ba5cd0 2023-10-05T19:40:16.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.303+0000 7f0018a48700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852badd20 con 0x562852baa690 2023-10-05T19:40:16.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.303+0000 7f0018247700 1 -- 172.21.15.12:0/37997056 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f000c008ad0 con 0x562852baa690 2023-10-05T19:40:16.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.303+0000 7f0018247700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852badd20 con 0x562852baa690 2023-10-05T19:40:16.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.303+0000 7f0018247700 1 -- 172.21.15.12:0/37997056 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f000c008ad0 con 0x562852baa690 2023-10-05T19:40:16.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.303+0000 7f0018247700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852badd20 con 0x7f000802a070 2023-10-05T19:40:16.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.303+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0010014040 con 0x7f000802a070 2023-10-05T19:40:16.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.307+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852badd20 con 0x562852baa690 2023-10-05T19:40:16.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.307+0000 7f0018247700 1 -- 172.21.15.12:0/37997056 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f000c008ad0 con 0x562852baa690 2023-10-05T19:40:16.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.307+0000 7f0018247700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852badd20 con 0x562852baa690 2023-10-05T19:40:16.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.307+0000 7f0018247700 1 -- 172.21.15.12:0/37997056 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f000c008ad0 con 0x562852baa690 2023-10-05T19:40:16.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.307+0000 7f0018247700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852badd20 con 0x562852b765f0 2023-10-05T19:40:16.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.307+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0010014040 con 0x562852b765f0 2023-10-05T19:40:16.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.307+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852badd20 con 0x562852b765f0 2023-10-05T19:40:16.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.307+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0010014040 con 0x562852b765f0 2023-10-05T19:40:16.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.307+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852badd20 con 0x7f000802a070 2023-10-05T19:40:16.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.311+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0010014040 con 0x7f000802a070 2023-10-05T19:40:16.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.311+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852badd20 con 0x562852baa690 2023-10-05T19:40:16.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.311+0000 7f0018247700 1 -- 172.21.15.12:0/37997056 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f000c008ad0 con 0x562852baa690 2023-10-05T19:40:16.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.311+0000 7f0018247700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852badd20 con 0x562852ba5cd0 2023-10-05T19:40:16.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.311+0000 7f0018a48700 1 -- 172.21.15.12:0/37997056 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f000801d070 con 0x562852ba5cd0 2023-10-05T19:40:16.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.311+0000 7f0018a48700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852badd20 con 0x562852b765f0 2023-10-05T19:40:16.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.311+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0010014040 con 0x562852b765f0 2023-10-05T19:40:16.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.311+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852badd20 con 0x7f000802a070 2023-10-05T19:40:16.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.311+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0010014040 con 0x7f000802a070 2023-10-05T19:40:16.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.311+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852badd20 con 0x562852baa690 2023-10-05T19:40:16.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.315+0000 7f0018247700 1 -- 172.21.15.12:0/37997056 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f000c008ad0 con 0x562852baa690 2023-10-05T19:40:16.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.315+0000 7f0018247700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852badd20 con 0x7f000802a070 2023-10-05T19:40:16.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.315+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0010014040 con 0x7f000802a070 2023-10-05T19:40:16.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.315+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852badd20 con 0x562852ba5cd0 2023-10-05T19:40:16.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.315+0000 7f0018a48700 1 -- 172.21.15.12:0/37997056 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f000801d070 con 0x562852ba5cd0 2023-10-05T19:40:16.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.315+0000 7f0018a48700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852badd20 con 0x562852baa690 2023-10-05T19:40:16.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.315+0000 7f0018247700 1 -- 172.21.15.12:0/37997056 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f000c008ad0 con 0x562852baa690 2023-10-05T19:40:16.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.315+0000 7f0018247700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852badd20 con 0x7f000802a070 2023-10-05T19:40:16.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.319+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0010014040 con 0x7f000802a070 2023-10-05T19:40:16.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.319+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852badd20 con 0x562852b765f0 2023-10-05T19:40:16.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.319+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f0010014040 con 0x562852b765f0 2023-10-05T19:40:16.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.319+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852badd20 con 0x562852ba5cd0 2023-10-05T19:40:16.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.319+0000 7f0018a48700 1 -- 172.21.15.12:0/37997056 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f000801d070 con 0x562852ba5cd0 2023-10-05T19:40:16.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.319+0000 7f001af13e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:16.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.319+0000 7f001af13e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:16.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.319+0000 7f001af13e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:16.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.319+0000 7f001af13e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:16.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.319+0000 7f001af13e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.319+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bae0d0 con 0x562852b765f0 2023-10-05T19:40:16.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.319+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f0010014040 con 0x562852b765f0 2023-10-05T19:40:16.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.319+0000 7f001af13e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:16.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.319+0000 7f001af13e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.319+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bae530 con 0x562852ba5cd0 2023-10-05T19:40:16.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.319+0000 7f0018a48700 1 -- 172.21.15.12:0/37997056 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv1 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f000801d070 con 0x562852ba5cd0 2023-10-05T19:40:16.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.319+0000 7f001af13e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:16.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.319+0000 7f001af13e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.319+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852baedd0 con 0x562852b765f0 2023-10-05T19:40:16.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.323+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f0010014040 con 0x562852b765f0 2023-10-05T19:40:16.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.323+0000 7f001af13e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:16.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.323+0000 7f001af13e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.323+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852baedd0 con 0x562852baa690 2023-10-05T19:40:16.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.323+0000 7f0018247700 1 -- 172.21.15.12:0/37997056 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f000c008ad0 con 0x562852baa690 2023-10-05T19:40:16.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.323+0000 7f001af13e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:16.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.323+0000 7f001af13e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.323+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852baf670 con 0x7f000802a070 2023-10-05T19:40:16.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.323+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f0010014040 con 0x7f000802a070 2023-10-05T19:40:16.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.323+0000 7f001af13e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:16.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.323+0000 7f001af13e40 20 zone default found 2023-10-05T19:40:16.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.323+0000 7f001af13e40 4 Realm: () 2023-10-05T19:40:16.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.323+0000 7f001af13e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:16.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.323+0000 7f001af13e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:16.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.323+0000 7f001af13e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:16.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.323+0000 7f001af13e40 20 zonegroup default 2023-10-05T19:40:16.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.323+0000 7f001af13e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.323+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852baf670 con 0x7f000802a070 2023-10-05T19:40:16.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.323+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f0010014040 con 0x7f000802a070 2023-10-05T19:40:16.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.323+0000 7f001af13e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:16.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.323+0000 7f001af13e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.323+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bb0ff0 con 0x562852baa690 2023-10-05T19:40:16.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.323+0000 7f0018247700 1 -- 172.21.15.12:0/37997056 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f000c008ad0 con 0x562852baa690 2023-10-05T19:40:16.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.327+0000 7f001af13e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:16.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.327+0000 7f001af13e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.327+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bb13a0 con 0x562852baa690 2023-10-05T19:40:16.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.327+0000 7f0018247700 1 -- 172.21.15.12:0/37997056 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f000c008ad0 con 0x562852baa690 2023-10-05T19:40:16.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.327+0000 7f001af13e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:16.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.327+0000 7f001af13e40 20 started sync module instance, tier type = 2023-10-05T19:40:16.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.327+0000 7f001af13e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:16.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.327+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bb06b0 con 0x562852ba5cd0 2023-10-05T19:40:16.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.327+0000 7f0018a48700 1 -- 172.21.15.12:0/37997056 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'6 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f000801d070 con 0x562852ba5cd0 2023-10-05T19:40:16.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.327+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94731186667968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bb1750 con 0x562852ba5cd0 2023-10-05T19:40:16.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.327+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bb1ff0 con 0x562852b765f0 2023-10-05T19:40:16.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.331+0000 7f0018a48700 1 -- 172.21.15.12:0/37997056 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94731186667968] v28'7 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f000801d070 con 0x562852ba5cd0 2023-10-05T19:40:16.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.331+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'6 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0010014040 con 0x562852b765f0 2023-10-05T19:40:16.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.331+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94731186674432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bb0bd0 con 0x562852b765f0 2023-10-05T19:40:16.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.331+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bb2aa0 con 0x562852baa690 2023-10-05T19:40:16.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.335+0000 7f0018247700 1 -- 172.21.15.12:0/37997056 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'6 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f000c008ad0 con 0x562852baa690 2023-10-05T19:40:16.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.335+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94731186677168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bb3390 con 0x562852baa690 2023-10-05T19:40:16.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.335+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bb3da0 con 0x562852ba5cd0 2023-10-05T19:40:16.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.335+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94731186674432] v28'7 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0010014040 con 0x562852b765f0 2023-10-05T19:40:16.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.339+0000 7f0018247700 1 -- 172.21.15.12:0/37997056 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94731186677168] v28'7 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f000c008ad0 con 0x562852baa690 2023-10-05T19:40:16.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.339+0000 7f0018a48700 1 -- 172.21.15.12:0/37997056 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'6 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f000801d070 con 0x562852ba5cd0 2023-10-05T19:40:16.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.339+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94731186682032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bb2fc0 con 0x562852ba5cd0 2023-10-05T19:40:16.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.339+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bb4850 con 0x562852b765f0 2023-10-05T19:40:16.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.339+0000 7f0018a48700 1 -- 172.21.15.12:0/37997056 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94731186682032] v28'7 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f000801d070 con 0x562852ba5cd0 2023-10-05T19:40:16.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.343+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'6 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0010014040 con 0x562852b765f0 2023-10-05T19:40:16.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.343+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94731186675536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bb2fc0 con 0x562852b765f0 2023-10-05T19:40:16.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.343+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bb52b0 con 0x7f000802a070 2023-10-05T19:40:16.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.343+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94731186675536] v28'7 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0010014040 con 0x562852b765f0 2023-10-05T19:40:16.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.343+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'6 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0010014040 con 0x7f000802a070 2023-10-05T19:40:16.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.343+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94731186684704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bb52b0 con 0x7f000802a070 2023-10-05T19:40:16.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.343+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bb5b90 con 0x7f000802a070 2023-10-05T19:40:16.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.347+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(95 notify.5 [watch watch cookie 94731186684704] v28'7 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0010014040 con 0x7f000802a070 2023-10-05T19:40:16.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.347+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(96 notify.6 [create] v28'6 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0010014040 con 0x7f000802a070 2023-10-05T19:40:16.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.347+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94731186687424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bb5740 con 0x7f000802a070 2023-10-05T19:40:16.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.347+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bb6550 con 0x7f000802a070 2023-10-05T19:40:16.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.351+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(97 notify.6 [watch watch cookie 94731186687424] v28'7 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0010014040 con 0x7f000802a070 2023-10-05T19:40:16.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.351+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(98 notify.7 [create] v28'6 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0010014040 con 0x7f000802a070 2023-10-05T19:40:16.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.351+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94731186692128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bb5700 con 0x7f000802a070 2023-10-05T19:40:16.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.351+0000 7f001af13e40 20 add_watcher() i=0 2023-10-05T19:40:16.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.351+0000 7f001af13e40 20 add_watcher() i=1 2023-10-05T19:40:16.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.351+0000 7f001af13e40 20 add_watcher() i=2 2023-10-05T19:40:16.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.351+0000 7f001af13e40 20 add_watcher() i=3 2023-10-05T19:40:16.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.351+0000 7f001af13e40 20 add_watcher() i=4 2023-10-05T19:40:16.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.351+0000 7f001af13e40 20 add_watcher() i=5 2023-10-05T19:40:16.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.351+0000 7f001af13e40 20 add_watcher() i=6 2023-10-05T19:40:16.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.355+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94731186692128] v28'7 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0010014040 con 0x7f000802a070 2023-10-05T19:40:16.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.355+0000 7f001af13e40 20 add_watcher() i=7 2023-10-05T19:40:16.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.355+0000 7f001af13e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:16.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.355+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bb9980 con 0x562852b765f0 2023-10-05T19:40:16.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.355+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f0010014040 con 0x562852b765f0 2023-10-05T19:40:16.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.355+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94731186688992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bb6df0 con 0x562852b765f0 2023-10-05T19:40:16.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.359+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94731186688992] v28'26 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f0010014040 con 0x562852b765f0 2023-10-05T19:40:16.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.359+0000 7effea7e4700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:16.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.359+0000 7f001af13e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:16.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.359+0000 7f001af13e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:16.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.359+0000 7f001af13e40 1 Processor -- start 2023-10-05T19:40:16.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.359+0000 7f001af13e40 1 -- start start 2023-10-05T19:40:16.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.359+0000 7f001af13e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562852bc3fb0 0x562852bc4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:16.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.359+0000 7f001af13e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x562852bc48c0 con 0x562852bc3fb0 2023-10-05T19:40:16.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.359+0000 7f0018247700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562852bc3fb0 0x562852bc4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:16.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.359+0000 7f0018247700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562852bc3fb0 0x562852bc4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36308/0 (socket says 172.21.15.12:36308) 2023-10-05T19:40:16.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.359+0000 7f0018247700 1 -- 172.21.15.12:0/4168039892 learned_addr learned my addr 172.21.15.12:0/4168039892 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:16.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.359+0000 7f0018247700 1 -- 172.21.15.12:0/4168039892 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562852bc4a00 con 0x562852bc3fb0 2023-10-05T19:40:16.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.359+0000 7f0018247700 1 --2- 172.21.15.12:0/4168039892 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562852bc3fb0 0x562852bc4380 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f000c002000 tx=0x7f000c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d30635d52d749c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:16.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.359+0000 7effe9fe3700 1 -- 172.21.15.12:0/4168039892 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f000c030930 con 0x562852bc3fb0 2023-10-05T19:40:16.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.359+0000 7effe9fe3700 1 -- 172.21.15.12:0/4168039892 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f000c030a90 con 0x562852bc3fb0 2023-10-05T19:40:16.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.359+0000 7effe9fe3700 1 -- 172.21.15.12:0/4168039892 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f000c030c40 con 0x562852bc3fb0 2023-10-05T19:40:16.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.363+0000 7f001af13e40 1 -- 172.21.15.12:0/4168039892 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562852bc3fb0 msgr2=0x562852bc4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:16.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.363+0000 7f001af13e40 1 --2- 172.21.15.12:0/4168039892 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562852bc3fb0 0x562852bc4380 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f000c002000 tx=0x7f000c012000 comp rx=0 tx=0).stop 2023-10-05T19:40:16.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.363+0000 7f001af13e40 1 -- 172.21.15.12:0/4168039892 shutdown_connections 2023-10-05T19:40:16.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.363+0000 7f001af13e40 1 --2- 172.21.15.12:0/4168039892 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562852bc3fb0 0x562852bc4380 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:16.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.363+0000 7f001af13e40 1 -- 172.21.15.12:0/4168039892 >> 172.21.15.12:0/4168039892 conn(0x562852bbf370 msgr2=0x562852bbd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:16.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.363+0000 7f001af13e40 1 -- 172.21.15.12:0/4168039892 shutdown_connections 2023-10-05T19:40:16.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.363+0000 7f001af13e40 1 -- 172.21.15.12:0/4168039892 wait complete. 2023-10-05T19:40:16.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.363+0000 7f001af13e40 1 Processor -- start 2023-10-05T19:40:16.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.363+0000 7f001af13e40 1 -- start start 2023-10-05T19:40:16.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.363+0000 7f001af13e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562852bc3fb0 0x562852bc9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:16.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.363+0000 7f001af13e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x562852bca970 con 0x562852bc3fb0 2023-10-05T19:40:16.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.363+0000 7f0018247700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562852bc3fb0 0x562852bc9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:16.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.363+0000 7f0018247700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562852bc3fb0 0x562852bc9970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36318/0 (socket says 172.21.15.12:36318) 2023-10-05T19:40:16.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.363+0000 7f0018247700 1 -- 172.21.15.12:0/3208319549 learned_addr learned my addr 172.21.15.12:0/3208319549 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:16.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.363+0000 7f0018247700 1 -- 172.21.15.12:0/3208319549 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562852b72c20 con 0x562852bc3fb0 2023-10-05T19:40:16.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.363+0000 7f0018247700 1 --2- 172.21.15.12:0/3208319549 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562852bc3fb0 0x562852bc9970 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x562852bbe030 tx=0x7f000c034000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:16.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.367+0000 7eff5bfff700 1 -- 172.21.15.12:0/3208319549 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f000c0277e0 con 0x562852bc3fb0 2023-10-05T19:40:16.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.367+0000 7eff5bfff700 1 -- 172.21.15.12:0/3208319549 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f000c027960 con 0x562852bc3fb0 2023-10-05T19:40:16.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.367+0000 7f001af13e40 1 -- 172.21.15.12:0/3208319549 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562852bc9f70 con 0x562852bc3fb0 2023-10-05T19:40:16.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.367+0000 7eff5bfff700 1 -- 172.21.15.12:0/3208319549 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f000c027b10 con 0x562852bc3fb0 2023-10-05T19:40:16.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.367+0000 7f001af13e40 1 -- 172.21.15.12:0/3208319549 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562852bca3b0 con 0x562852bc3fb0 2023-10-05T19:40:16.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.367+0000 7eff5bfff700 1 -- 172.21.15.12:0/3208319549 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f000c027c70 con 0x562852bc3fb0 2023-10-05T19:40:16.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.367+0000 7eff5bfff700 1 --2- 172.21.15.12:0/3208319549 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7eff60030c50 0x7eff60033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:16.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.367+0000 7eff5bfff700 1 -- 172.21.15.12:0/3208319549 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f000c054ab0 con 0x562852bc3fb0 2023-10-05T19:40:16.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.367+0000 7f0019249700 1 --2- 172.21.15.12:0/3208319549 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7eff60030c50 0x7eff60033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:16.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.367+0000 7f001af13e40 1 --2- 172.21.15.12:0/3208319549 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x562852bd4560 0x562852bd6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:16.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.367+0000 7f001af13e40 1 -- 172.21.15.12:0/3208319549 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bd6fc0 con 0x562852bd4560 2023-10-05T19:40:16.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.367+0000 7f0018a48700 1 --2- 172.21.15.12:0/3208319549 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x562852bd4560 0x562852bd6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:16.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.367+0000 7f0019249700 1 --2- 172.21.15.12:0/3208319549 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7eff60030c50 0x7eff60033100 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x562852bb5940 tx=0x7f001000ab40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:16.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.371+0000 7f0018a48700 1 --2- 172.21.15.12:0/3208319549 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x562852bd4560 0x562852bd6970 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:16.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.371+0000 7f0018a48700 1 -- 172.21.15.12:0/3208319549 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v28'16 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f000801d070 con 0x562852bd4560 2023-10-05T19:40:16.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.371+0000 7f001af13e40 1 --2- 172.21.15.12:0/3208319549 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x562852bd7c90 0x562852bda070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:16.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.371+0000 7f001af13e40 1 -- 172.21.15.12:0/3208319549 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bda7a0 con 0x562852bd7c90 2023-10-05T19:40:16.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.371+0000 7f0018247700 1 --2- 172.21.15.12:0/3208319549 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x562852bd7c90 0x562852bda070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:16.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.375+0000 7f0018247700 1 --2- 172.21.15.12:0/3208319549 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x562852bd7c90 0x562852bda070 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:16.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.375+0000 7f0018247700 1 -- 172.21.15.12:0/3208319549 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v28'10 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f000c008ad0 con 0x562852bd7c90 2023-10-05T19:40:16.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.375+0000 7f001af13e40 1 --2- 172.21.15.12:0/3208319549 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x562852bdb470 0x562852bdd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:16.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.375+0000 7f001af13e40 1 -- 172.21.15.12:0/3208319549 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bddfb0 con 0x562852bdb470 2023-10-05T19:40:16.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.375+0000 7f0019249700 1 --2- 172.21.15.12:0/3208319549 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x562852bdb470 0x562852bdd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:16.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.375+0000 7f0019249700 1 --2- 172.21.15.12:0/3208319549 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x562852bdb470 0x562852bdd880 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:16.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.379+0000 7f0019249700 1 -- 172.21.15.12:0/3208319549 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v28'10 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f0010014040 con 0x562852bdb470 2023-10-05T19:40:16.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.379+0000 7f001af13e40 1 -- 172.21.15.12:0/3208319549 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bde600 con 0x562852bd4560 2023-10-05T19:40:16.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.383+0000 7f0018a48700 1 -- 172.21.15.12:0/3208319549 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v28'27 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f000801d070 con 0x562852bd4560 2023-10-05T19:40:16.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.383+0000 7f001af13e40 1 --2- 172.21.15.12:0/3208319549 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x562852bdf2a0 0x562852be1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:16.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.383+0000 7f001af13e40 1 -- 172.21.15.12:0/3208319549 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852be1d30 con 0x562852bdf2a0 2023-10-05T19:40:16.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.383+0000 7f0018a48700 1 --2- 172.21.15.12:0/3208319549 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x562852bdf2a0 0x562852be1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:16.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.383+0000 7f0018a48700 1 --2- 172.21.15.12:0/3208319549 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x562852bdf2a0 0x562852be1680 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:16.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.387+0000 7f0018a48700 1 -- 172.21.15.12:0/3208319549 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v28'20 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f000801d070 con 0x562852bdf2a0 2023-10-05T19:40:16.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.387+0000 7f001af13e40 1 -- 172.21.15.12:0/3208319549 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852be2350 con 0x562852bdb470 2023-10-05T19:40:16.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.387+0000 7f0019249700 1 -- 172.21.15.12:0/3208319549 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v28'13 uv8 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f0010014040 con 0x562852bdb470 2023-10-05T19:40:16.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.391+0000 7f001af13e40 1 -- 172.21.15.12:0/3208319549 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852be2970 con 0x562852bd4560 2023-10-05T19:40:16.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.391+0000 7f0018a48700 1 -- 172.21.15.12:0/3208319549 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v28'12 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f000801d070 con 0x562852bd4560 2023-10-05T19:40:16.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.391+0000 7f001af13e40 1 -- 172.21.15.12:0/3208319549 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852be2e60 con 0x562852bd7c90 2023-10-05T19:40:16.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.395+0000 7f0018247700 1 -- 172.21.15.12:0/3208319549 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v28'13 uv8 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f000c008ad0 con 0x562852bd7c90 2023-10-05T19:40:16.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.395+0000 7f001af13e40 1 -- 172.21.15.12:0/3208319549 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bcfeb0 con 0x562852bdf2a0 2023-10-05T19:40:16.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.399+0000 7f0018a48700 1 -- 172.21.15.12:0/3208319549 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v28'10 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f000801d070 con 0x562852bdf2a0 2023-10-05T19:40:16.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.399+0000 7f001af13e40 1 -- 172.21.15.12:0/3208319549 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bcfeb0 con 0x562852bdb470 2023-10-05T19:40:16.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.399+0000 7f0019249700 1 -- 172.21.15.12:0/3208319549 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v28'16 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f0010014040 con 0x562852bdb470 2023-10-05T19:40:16.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.399+0000 7f001af13e40 1 -- 172.21.15.12:0/3208319549 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852be3980 con 0x562852bdf2a0 2023-10-05T19:40:16.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.403+0000 7f0018a48700 1 -- 172.21.15.12:0/3208319549 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v28'21 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f000801d070 con 0x562852bdf2a0 2023-10-05T19:40:16.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.403+0000 7f001af13e40 1 -- 172.21.15.12:0/3208319549 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852be3980 con 0x562852bd7c90 2023-10-05T19:40:16.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.407+0000 7f0018247700 1 -- 172.21.15.12:0/3208319549 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v28'12 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f000c008ad0 con 0x562852bd7c90 2023-10-05T19:40:16.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.407+0000 7f001af13e40 1 -- 172.21.15.12:0/3208319549 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852be3fa0 con 0x562852bdf2a0 2023-10-05T19:40:16.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.407+0000 7f0018a48700 1 -- 172.21.15.12:0/3208319549 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v28'10 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f000801d070 con 0x562852bdf2a0 2023-10-05T19:40:16.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.407+0000 7f001af13e40 1 -- 172.21.15.12:0/3208319549 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852be3fa0 con 0x562852bd7c90 2023-10-05T19:40:16.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.411+0000 7f0018247700 1 -- 172.21.15.12:0/3208319549 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v28'19 uv12 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f000c008ad0 con 0x562852bd7c90 2023-10-05T19:40:16.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.411+0000 7f001af13e40 1 -- 172.21.15.12:0/3208319549 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852be45c0 con 0x562852bd4560 2023-10-05T19:40:16.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.415+0000 7f0018a48700 1 -- 172.21.15.12:0/3208319549 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v28'28 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f000801d070 con 0x562852bd4560 2023-10-05T19:40:16.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.415+0000 7f001af13e40 1 -- 172.21.15.12:0/3208319549 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852be45c0 con 0x562852bdb470 2023-10-05T19:40:16.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.415+0000 7f0019249700 1 -- 172.21.15.12:0/3208319549 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v28'14 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0010014040 con 0x562852bdb470 2023-10-05T19:40:16.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.415+0000 7f001af13e40 1 -- 172.21.15.12:0/3208319549 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852be4ab0 con 0x562852bdf2a0 2023-10-05T19:40:16.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.419+0000 7f0018a48700 1 -- 172.21.15.12:0/3208319549 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v28'22 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f000801d070 con 0x562852bdf2a0 2023-10-05T19:40:16.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.419+0000 7f001af13e40 1 -- 172.21.15.12:0/3208319549 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852be4ab0 con 0x562852bdb470 2023-10-05T19:40:16.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.423+0000 7f0019249700 1 -- 172.21.15.12:0/3208319549 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v28'10 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0010014040 con 0x562852bdb470 2023-10-05T19:40:16.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.423+0000 7f001af13e40 1 -- 172.21.15.12:0/3208319549 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852be4ab0 con 0x562852bd4560 2023-10-05T19:40:16.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.427+0000 7f0018a48700 1 -- 172.21.15.12:0/3208319549 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v28'18 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f000801d070 con 0x562852bd4560 2023-10-05T19:40:16.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.427+0000 7f001af13e40 1 -- 172.21.15.12:0/3208319549 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852be4ab0 con 0x562852bd4560 2023-10-05T19:40:16.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.427+0000 7f0018a48700 1 -- 172.21.15.12:0/3208319549 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v28'29 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f000801d070 con 0x562852bd4560 2023-10-05T19:40:16.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.427+0000 7f001af13e40 1 -- 172.21.15.12:0/3208319549 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852be4ab0 con 0x562852bdb470 2023-10-05T19:40:16.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.431+0000 7f0019249700 1 -- 172.21.15.12:0/3208319549 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v28'15 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0010014040 con 0x562852bdb470 2023-10-05T19:40:16.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.431+0000 7f001af13e40 1 -- 172.21.15.12:0/3208319549 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852be4ab0 con 0x562852bdf2a0 2023-10-05T19:40:16.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.435+0000 7f0018a48700 1 -- 172.21.15.12:0/3208319549 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v28'20 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f000801d070 con 0x562852bdf2a0 2023-10-05T19:40:16.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.435+0000 7f001af13e40 1 -- 172.21.15.12:0/3208319549 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852be4ab0 con 0x562852bdb470 2023-10-05T19:40:16.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.435+0000 7f0019249700 1 -- 172.21.15.12:0/3208319549 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v28'12 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0010014040 con 0x562852bdb470 2023-10-05T19:40:16.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.439+0000 7f001af13e40 1 -- 172.21.15.12:0/3208319549 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852be4ab0 con 0x562852bdb470 2023-10-05T19:40:16.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.439+0000 7f0019249700 1 -- 172.21.15.12:0/3208319549 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v28'17 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0010014040 con 0x562852bdb470 2023-10-05T19:40:16.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.439+0000 7f001af13e40 1 -- 172.21.15.12:0/3208319549 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852be4ab0 con 0x562852bdb470 2023-10-05T19:40:16.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.443+0000 7f0019249700 1 -- 172.21.15.12:0/3208319549 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v28'6 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0010014040 con 0x562852bdb470 2023-10-05T19:40:16.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.443+0000 7f001af13e40 1 -- 172.21.15.12:0/3208319549 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852be4ab0 con 0x562852bdb470 2023-10-05T19:40:16.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.443+0000 7f0019249700 1 -- 172.21.15.12:0/3208319549 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v28'14 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0010014040 con 0x562852bdb470 2023-10-05T19:40:16.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.443+0000 7f001af13e40 1 -- 172.21.15.12:0/3208319549 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852be4ab0 con 0x562852bd7c90 2023-10-05T19:40:16.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.447+0000 7f0018247700 1 -- 172.21.15.12:0/3208319549 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v28'12 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f000c008ad0 con 0x562852bd7c90 2023-10-05T19:40:16.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.447+0000 7f001af13e40 1 -- 172.21.15.12:0/3208319549 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bceea0 con 0x562852bdb470 2023-10-05T19:40:16.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.451+0000 7f0019249700 1 -- 172.21.15.12:0/3208319549 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v28'10 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0010014040 con 0x562852bdb470 2023-10-05T19:40:16.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.451+0000 7f001af13e40 1 -- 172.21.15.12:0/3208319549 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bceea0 con 0x562852bd7c90 2023-10-05T19:40:16.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.455+0000 7f0018247700 1 -- 172.21.15.12:0/3208319549 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v28'14 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f000c008ad0 con 0x562852bd7c90 2023-10-05T19:40:16.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.455+0000 7f001af13e40 1 -- 172.21.15.12:0/3208319549 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bceea0 con 0x562852bdb470 2023-10-05T19:40:16.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.455+0000 7f0019249700 1 -- 172.21.15.12:0/3208319549 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v28'16 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0010014040 con 0x562852bdb470 2023-10-05T19:40:16.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.455+0000 7f001af13e40 1 -- 172.21.15.12:0/3208319549 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bceea0 con 0x562852bd7c90 2023-10-05T19:40:16.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.459+0000 7f0018247700 1 -- 172.21.15.12:0/3208319549 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v28'20 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f000c008ad0 con 0x562852bd7c90 2023-10-05T19:40:16.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.459+0000 7f001af13e40 1 -- 172.21.15.12:0/3208319549 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852bceea0 con 0x562852bdf2a0 2023-10-05T19:40:16.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.463+0000 7f0018a48700 1 -- 172.21.15.12:0/3208319549 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v28'16 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f000801d070 con 0x562852bdf2a0 2023-10-05T19:40:16.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.463+0000 7eff597fa700 20 reqs_thread_entry: start 2023-10-05T19:40:16.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.467+0000 7f001af13e40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:16.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.467+0000 7eff52ffd700 20 reqs_thread_entry: start 2023-10-05T19:40:16.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.467+0000 7f001af13e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:16.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.467+0000 7eff517fa700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:46 2023 2023-10-05T19:40:16.475 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:16.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.467+0000 7eff50ff9700 20 reqs_thread_entry: start 2023-10-05T19:40:16.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.467+0000 7f001af13e40 20 get_system_obj_state: rctx=0x7ffed1c96548 obj=default.rgw.meta:users.uid:bud state=0x562852a490b0 s->prefetch_data=0 2023-10-05T19:40:16.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.467+0000 7f001af13e40 10 cache get: name=default.rgw.meta+users.uid+bud : miss 2023-10-05T19:40:16.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.467+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:102 8.4 8:267425a8:users.uid::bud:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852c43380 con 0x562852ba5cd0 2023-10-05T19:40:16.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.467+0000 7f0018a48700 1 -- 172.21.15.12:0/37997056 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(102 bud [call,getxattrs,stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x7f000801d070 con 0x562852ba5cd0 2023-10-05T19:40:16.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.467+0000 7f001af13e40 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x0 2023-10-05T19:40:16.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.467+0000 7f001af13e40 10 adding default.rgw.meta+users.uid+bud to cache LRU end 2023-10-05T19:40:16.477 INFO:teuthology.orchestra.run.smithi012.stderr:could not remove user: unable to remove user, user does not exist 2023-10-05T19:40:16.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.467+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:103 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94731186667968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852c43380 con 0x562852ba5cd0 2023-10-05T19:40:16.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.471+0000 7f0018a48700 1 -- 172.21.15.12:0/37997056 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(103 notify.0 [watch unwatch cookie 94731186667968] v28'8 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f000801d070 con 0x562852ba5cd0 2023-10-05T19:40:16.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.471+0000 7f001af13e40 20 remove_watcher() i=0 2023-10-05T19:40:16.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.471+0000 7f001af13e40 2 removed watcher, disabling cache 2023-10-05T19:40:16.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.471+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:104 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94731186674432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852c43380 con 0x562852b765f0 2023-10-05T19:40:16.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.475+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(104 notify.1 [watch unwatch cookie 94731186674432] v28'8 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0010014040 con 0x562852b765f0 2023-10-05T19:40:16.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.475+0000 7f001af13e40 20 remove_watcher() i=1 2023-10-05T19:40:16.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.475+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:105 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94731186677168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852c43380 con 0x562852baa690 2023-10-05T19:40:16.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.479+0000 7f0018247700 1 -- 172.21.15.12:0/37997056 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(105 notify.2 [watch unwatch cookie 94731186677168] v28'8 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f000c008ad0 con 0x562852baa690 2023-10-05T19:40:16.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.479+0000 7f001af13e40 20 remove_watcher() i=2 2023-10-05T19:40:16.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.479+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:106 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94731186682032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852c43380 con 0x562852ba5cd0 2023-10-05T19:40:16.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.479+0000 7f0018a48700 1 -- 172.21.15.12:0/37997056 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(106 notify.3 [watch unwatch cookie 94731186682032] v28'8 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f000801d070 con 0x562852ba5cd0 2023-10-05T19:40:16.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.479+0000 7f001af13e40 20 remove_watcher() i=3 2023-10-05T19:40:16.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.479+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:107 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94731186675536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852c43380 con 0x562852b765f0 2023-10-05T19:40:16.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.483+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(107 notify.4 [watch unwatch cookie 94731186675536] v28'8 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0010014040 con 0x562852b765f0 2023-10-05T19:40:16.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.483+0000 7f001af13e40 20 remove_watcher() i=4 2023-10-05T19:40:16.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.483+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:108 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94731186684704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852c43380 con 0x7f000802a070 2023-10-05T19:40:16.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.487+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(108 notify.5 [watch unwatch cookie 94731186684704] v28'8 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0010014040 con 0x7f000802a070 2023-10-05T19:40:16.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.487+0000 7f001af13e40 20 remove_watcher() i=5 2023-10-05T19:40:16.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.487+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:109 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94731186687424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852c43380 con 0x7f000802a070 2023-10-05T19:40:16.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.491+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(109 notify.6 [watch unwatch cookie 94731186687424] v28'8 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0010014040 con 0x7f000802a070 2023-10-05T19:40:16.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.491+0000 7f001af13e40 20 remove_watcher() i=6 2023-10-05T19:40:16.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.491+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:110 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94731186692128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852c43380 con 0x7f000802a070 2023-10-05T19:40:16.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.491+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(110 notify.7 [watch unwatch cookie 94731186692128] v28'8 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0010014040 con 0x7f000802a070 2023-10-05T19:40:16.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.491+0000 7f001af13e40 20 remove_watcher() i=7 2023-10-05T19:40:16.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.495+0000 7effea7e4700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:16.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.495+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:111 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94731186688992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562852c43380 con 0x562852b765f0 2023-10-05T19:40:16.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.499+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(111 data_loggenerations_metadata [watch unwatch cookie 94731186688992] v28'30 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f0010014040 con 0x562852b765f0 2023-10-05T19:40:16.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.499+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f000802a070 msgr2=0x7f0008012a80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:16.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.499+0000 7f001af13e40 1 --2- 172.21.15.12:0/37997056 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f000802a070 0x7f0008012a80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:16.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.499+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x562852b765f0 msgr2=0x562852ba3a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:16.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.499+0000 7f001af13e40 1 --2- 172.21.15.12:0/37997056 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x562852b765f0 0x562852ba3a30 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:16.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.499+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x562852ba5cd0 msgr2=0x562852ba80b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:16.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.499+0000 7f001af13e40 1 --2- 172.21.15.12:0/37997056 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x562852ba5cd0 0x562852ba80b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:16.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.499+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x562852baa690 msgr2=0x562852bacaa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:16.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.499+0000 7f001af13e40 1 --2- 172.21.15.12:0/37997056 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x562852baa690 0x562852bacaa0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:16.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.499+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f0004030c50 msgr2=0x7f0004033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:16.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.499+0000 7f001af13e40 1 --2- 172.21.15.12:0/37997056 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f0004030c50 0x7f0004033100 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f000c0030a0 tx=0x7f000c015000 comp rx=0 tx=0).stop 2023-10-05T19:40:16.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.499+0000 7f0019249700 1 -- 172.21.15.12:0/37997056 reap_dead start 2023-10-05T19:40:16.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.499+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562852b72010 msgr2=0x562852b97f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:16.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.499+0000 7f001af13e40 1 --2- 172.21.15.12:0/37997056 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562852b72010 0x562852b97f70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0008001350 tx=0x7f0008013000 comp rx=0 tx=0).stop 2023-10-05T19:40:16.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.499+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 shutdown_connections 2023-10-05T19:40:16.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.499+0000 7f001af13e40 1 --2- 172.21.15.12:0/37997056 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562852b72010 0x562852b97f70 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:16.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.499+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 >> 172.21.15.12:0/37997056 conn(0x562852ae5860 msgr2=0x562852b8b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:16.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.499+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 shutdown_connections 2023-10-05T19:40:16.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.499+0000 7f001af13e40 1 -- 172.21.15.12:0/37997056 wait complete. 2023-10-05T19:40:16.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.499+0000 7f001af13e40 1 -- 172.21.15.12:0/3208319549 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x562852bd7c90 msgr2=0x562852bda070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:16.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.499+0000 7f001af13e40 1 --2- 172.21.15.12:0/3208319549 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x562852bd7c90 0x562852bda070 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:16.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.499+0000 7f001af13e40 1 -- 172.21.15.12:0/3208319549 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x562852bd4560 msgr2=0x562852bd6970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:16.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.499+0000 7f001af13e40 1 --2- 172.21.15.12:0/3208319549 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x562852bd4560 0x562852bd6970 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:16.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.499+0000 7f001af13e40 1 -- 172.21.15.12:0/3208319549 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x562852bdf2a0 msgr2=0x562852be1680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:16.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.499+0000 7f001af13e40 1 --2- 172.21.15.12:0/3208319549 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x562852bdf2a0 0x562852be1680 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:16.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.499+0000 7f001af13e40 1 -- 172.21.15.12:0/3208319549 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x562852bdb470 msgr2=0x562852bdd880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:16.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.499+0000 7f001af13e40 1 --2- 172.21.15.12:0/3208319549 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x562852bdb470 0x562852bdd880 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:16.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.499+0000 7f001af13e40 1 -- 172.21.15.12:0/3208319549 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7eff60030c50 msgr2=0x7eff60033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:16.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.499+0000 7f001af13e40 1 --2- 172.21.15.12:0/3208319549 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7eff60030c50 0x7eff60033100 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x562852bb5940 tx=0x7f001000ab40 comp rx=0 tx=0).stop 2023-10-05T19:40:16.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.499+0000 7f0018a48700 1 -- 172.21.15.12:0/3208319549 reap_dead start 2023-10-05T19:40:16.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.499+0000 7f001af13e40 1 -- 172.21.15.12:0/3208319549 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562852bc3fb0 msgr2=0x562852bc9970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:16.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.499+0000 7f001af13e40 1 --2- 172.21.15.12:0/3208319549 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562852bc3fb0 0x562852bc9970 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x562852bbe030 tx=0x7f000c034000 comp rx=0 tx=0).stop 2023-10-05T19:40:16.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.499+0000 7f001af13e40 1 -- 172.21.15.12:0/3208319549 shutdown_connections 2023-10-05T19:40:16.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.499+0000 7f001af13e40 1 --2- 172.21.15.12:0/3208319549 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562852bc3fb0 0x562852bc9970 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:16.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.499+0000 7f001af13e40 1 -- 172.21.15.12:0/3208319549 >> 172.21.15.12:0/3208319549 conn(0x562852bbf370 msgr2=0x562852bbd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:16.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.499+0000 7f001af13e40 1 -- 172.21.15.12:0/3208319549 shutdown_connections 2023-10-05T19:40:16.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.499+0000 7f001af13e40 1 -- 172.21.15.12:0/3208319549 wait complete. 2023-10-05T19:40:16.516 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-05T19:40:16.517 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'bar'] 2023-10-05T19:40:16.517 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'bar'] 2023-10-05T19:40:16.517 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid bar 2023-10-05T19:40:16.558 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:16.558 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:16.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.551+0000 7f3af1ecde40 1 Processor -- start 2023-10-05T19:40:16.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.551+0000 7f3af1ecde40 1 -- start start 2023-10-05T19:40:16.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.551+0000 7f3af1ecde40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55585833e7e0 0x55585833d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:16.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.551+0000 7f3af1ecde40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55585828abd0 con 0x55585833e410 2023-10-05T19:40:16.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.551+0000 7f3af1ecde40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x555858247ee0 con 0x55585833e7e0 2023-10-05T19:40:16.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.551+0000 7f3aef201700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55585833e7e0 0x55585833d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:16.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.551+0000 7f3aefa02700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x55585833e410 0x55585833cfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:47130/0 (socket says 172.21.15.12:47130) 2023-10-05T19:40:16.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.551+0000 7f3aefa02700 1 -- 172.21.15.12:0/4134020612 learned_addr learned my addr 172.21.15.12:0/4134020612 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:16.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.551+0000 7f3aef201700 1 -- 172.21.15.12:0/4134020612 >> v1:172.21.15.8:6789/0 conn(0x55585833e410 legacy=0x55585833cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:16.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.551+0000 7f3aeea00700 1 -- 172.21.15.12:0/4134020612 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2839014048 0 0) 0x55585828abd0 con 0x55585833e410 2023-10-05T19:40:16.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.551+0000 7f3aef201700 1 -- 172.21.15.12:0/4134020612 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55585833dcd0 con 0x55585833e7e0 2023-10-05T19:40:16.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.551+0000 7f3aef201700 1 --2- 172.21.15.12:0/4134020612 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55585833e7e0 0x55585833d6d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3ad800a520 tx=0x7f3ad8005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6be34dee8097495a server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:16.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.555+0000 7f3aeea00700 1 -- 172.21.15.12:0/4134020612 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3ad800f590 con 0x55585833e7e0 2023-10-05T19:40:16.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.555+0000 7f3aeea00700 1 -- 172.21.15.12:0/4134020612 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad800f6f0 con 0x55585833e7e0 2023-10-05T19:40:16.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.555+0000 7f3aeea00700 1 -- 172.21.15.12:0/4134020612 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3ad800f880 con 0x55585833e7e0 2023-10-05T19:40:16.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.555+0000 7f3af1ecde40 1 -- 172.21.15.12:0/4134020612 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55585833e7e0 msgr2=0x55585833d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:16.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.555+0000 7f3af1ecde40 1 --2- 172.21.15.12:0/4134020612 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55585833e7e0 0x55585833d6d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3ad800a520 tx=0x7f3ad8005d40 comp rx=0 tx=0).stop 2023-10-05T19:40:16.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.555+0000 7f3af1ecde40 1 -- 172.21.15.12:0/4134020612 shutdown_connections 2023-10-05T19:40:16.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.555+0000 7f3af1ecde40 1 --2- 172.21.15.12:0/4134020612 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55585833e7e0 0x55585833d6d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:16.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.555+0000 7f3af1ecde40 1 -- 172.21.15.12:0/4134020612 >> 172.21.15.12:0/4134020612 conn(0x555858338720 msgr2=0x55585833ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:16.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.555+0000 7f3af1ecde40 1 -- 172.21.15.12:0/4134020612 shutdown_connections 2023-10-05T19:40:16.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.555+0000 7f3af1ecde40 1 -- 172.21.15.12:0/4134020612 wait complete. 2023-10-05T19:40:16.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.555+0000 7f3af1ecde40 1 Processor -- start 2023-10-05T19:40:16.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.555+0000 7f3af1ecde40 1 -- start start 2023-10-05T19:40:16.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.555+0000 7f3af1ecde40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55585833e410 0x55585833ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:16.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.555+0000 7f3af1ecde40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55585820be30 con 0x55585833e410 2023-10-05T19:40:16.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.555+0000 7f3aefa02700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55585833e410 0x55585833ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:16.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.555+0000 7f3aefa02700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55585833e410 0x55585833ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36336/0 (socket says 172.21.15.12:36336) 2023-10-05T19:40:16.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.555+0000 7f3aefa02700 1 -- 172.21.15.12:0/3009662450 learned_addr learned my addr 172.21.15.12:0/3009662450 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:16.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.555+0000 7f3aefa02700 1 -- 172.21.15.12:0/3009662450 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad800a0e0 con 0x55585833e410 2023-10-05T19:40:16.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.555+0000 7f3aefa02700 1 --2- 172.21.15.12:0/3009662450 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55585833e410 0x55585833ee60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0001dc0 tx=0x7f3ae0001790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e13b550a7b72e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:16.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.559+0000 7f3aed9fe700 1 -- 172.21.15.12:0/3009662450 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3ae000bce0 con 0x55585833e410 2023-10-05T19:40:16.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.559+0000 7f3aed9fe700 1 -- 172.21.15.12:0/3009662450 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ae000be40 con 0x55585833e410 2023-10-05T19:40:16.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.559+0000 7f3aed9fe700 1 -- 172.21.15.12:0/3009662450 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3ae000e5d0 con 0x55585833e410 2023-10-05T19:40:16.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.559+0000 7f3af1ecde40 1 -- 172.21.15.12:0/3009662450 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55585833e410 msgr2=0x55585833ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:16.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.559+0000 7f3af1ecde40 1 --2- 172.21.15.12:0/3009662450 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55585833e410 0x55585833ee60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0001dc0 tx=0x7f3ae0001790 comp rx=0 tx=0).stop 2023-10-05T19:40:16.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.559+0000 7f3af1ecde40 1 -- 172.21.15.12:0/3009662450 shutdown_connections 2023-10-05T19:40:16.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.559+0000 7f3af1ecde40 1 --2- 172.21.15.12:0/3009662450 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55585833e410 0x55585833ee60 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:16.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.559+0000 7f3af1ecde40 1 -- 172.21.15.12:0/3009662450 >> 172.21.15.12:0/3009662450 conn(0x555858338720 msgr2=0x55585835aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:16.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.559+0000 7f3af1ecde40 1 -- 172.21.15.12:0/3009662450 shutdown_connections 2023-10-05T19:40:16.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.559+0000 7f3af1ecde40 1 -- 172.21.15.12:0/3009662450 wait complete. 2023-10-05T19:40:16.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.559+0000 7f3af1ecde40 1 Processor -- start 2023-10-05T19:40:16.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.559+0000 7f3af1ecde40 1 -- start start 2023-10-05T19:40:16.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.559+0000 7f3af1ecde40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55585833e410 0x55585836a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:16.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.559+0000 7f3af1ecde40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558581deed0 con 0x55585833e410 2023-10-05T19:40:16.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.559+0000 7f3aefa02700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55585833e410 0x55585836a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:16.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.559+0000 7f3aefa02700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55585833e410 0x55585836a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36352/0 (socket says 172.21.15.12:36352) 2023-10-05T19:40:16.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.559+0000 7f3aefa02700 1 -- 172.21.15.12:0/926293803 learned_addr learned my addr 172.21.15.12:0/926293803 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:16.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.559+0000 7f3aefa02700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae0001a70 con 0x55585833e410 2023-10-05T19:40:16.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.559+0000 7f3aefa02700 1 --2- 172.21.15.12:0/926293803 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55585833e410 0x55585836a0e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3ae00017d0 tx=0x7f3ae0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:16.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.563+0000 7f3ae7fff700 1 -- 172.21.15.12:0/926293803 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3ae001f070 con 0x55585833e410 2023-10-05T19:40:16.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.563+0000 7f3ae7fff700 1 -- 172.21.15.12:0/926293803 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ae001a4d0 con 0x55585833e410 2023-10-05T19:40:16.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.563+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55585836a620 con 0x55585833e410 2023-10-05T19:40:16.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.563+0000 7f3ae7fff700 1 -- 172.21.15.12:0/926293803 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3ae001a7a0 con 0x55585833e410 2023-10-05T19:40:16.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.563+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55585836aa60 con 0x55585833e410 2023-10-05T19:40:16.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.563+0000 7f3af1ecde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.563+0000 7f3ae7fff700 1 -- 172.21.15.12:0/926293803 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f3ae001a900 con 0x55585833e410 2023-10-05T19:40:16.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.563+0000 7f3ae7fff700 1 --2- 172.21.15.12:0/926293803 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3ad0030c50 0x7f3ad0033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:16.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.563+0000 7f3ae7fff700 1 -- 172.21.15.12:0/926293803 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f3ae0045030 con 0x55585833e410 2023-10-05T19:40:16.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.563+0000 7f3aef201700 1 --2- 172.21.15.12:0/926293803 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3ad0030c50 0x7f3ad0033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:16.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.563+0000 7f3af1ecde40 1 --2- 172.21.15.12:0/926293803 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55585833e7e0 0x555858375ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:16.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.563+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555858376210 con 0x55585833e7e0 2023-10-05T19:40:16.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.563+0000 7f3af0203700 1 --2- 172.21.15.12:0/926293803 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55585833e7e0 0x555858375ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:16.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.567+0000 7f3aef201700 1 --2- 172.21.15.12:0/926293803 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3ad0030c50 0x7f3ad0033100 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8005d80 tx=0x7f3ad8014000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:16.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.567+0000 7f3af0203700 1 --2- 172.21.15.12:0/926293803 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55585833e7e0 0x555858375ba0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:16.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.567+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3ae800a500 con 0x55585833e7e0 2023-10-05T19:40:16.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.567+0000 7f3af1ecde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:16.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.567+0000 7f3af1ecde40 20 realm 2023-10-05T19:40:16.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.567+0000 7f3af1ecde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.567+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555858373200 con 0x55585833e7e0 2023-10-05T19:40:16.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.567+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3ae800a500 con 0x55585833e7e0 2023-10-05T19:40:16.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.567+0000 7f3af1ecde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:16.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.567+0000 7f3af1ecde40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:16.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.567+0000 7f3af1ecde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.567+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555858376cf0 con 0x55585833e7e0 2023-10-05T19:40:16.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.567+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3ae800a500 con 0x55585833e7e0 2023-10-05T19:40:16.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.567+0000 7f3af1ecde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:16.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.567+0000 7f3af1ecde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.567+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555858376900 con 0x55585833e7e0 2023-10-05T19:40:16.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.567+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f3ae800a500 con 0x55585833e7e0 2023-10-05T19:40:16.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.567+0000 7f3af1ecde40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:16.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.567+0000 7f3af1ecde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.567+0000 7f3af1ecde40 1 --2- 172.21.15.12:0/926293803 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x555858378090 0x55585837a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:16.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.567+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55585837a9b0 con 0x555858378090 2023-10-05T19:40:16.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.567+0000 7f3aefa02700 1 --2- 172.21.15.12:0/926293803 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x555858378090 0x55585837a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:16.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.571+0000 7f3aefa02700 1 --2- 172.21.15.12:0/926293803 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x555858378090 0x55585837a470 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:16.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.571+0000 7f3aefa02700 1 -- 172.21.15.12:0/926293803 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv1 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f3ae0014070 con 0x555858378090 2023-10-05T19:40:16.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.571+0000 7f3af1ecde40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:16.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.571+0000 7f3af1ecde40 20 searching for the correct realm 2023-10-05T19:40:16.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.571+0000 7f3af1ecde40 1 --2- 172.21.15.12:0/926293803 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55585837c830 0x55585837ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:16.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.571+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55585837f2f0 con 0x55585837c830 2023-10-05T19:40:16.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.571+0000 7f3aef201700 1 --2- 172.21.15.12:0/926293803 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55585837c830 0x55585837ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:16.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.571+0000 7f3aef201700 1 --2- 172.21.15.12:0/926293803 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55585837c830 0x55585837ec40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:16.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.575+0000 7f3aef201700 1 -- 172.21.15.12:0/926293803 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f3ad800fd60 con 0x55585837c830 2023-10-05T19:40:16.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.575+0000 7f3aef201700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55585837f2f0 con 0x555858378090 2023-10-05T19:40:16.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.575+0000 7f3aefa02700 1 -- 172.21.15.12:0/926293803 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ae0014070 con 0x555858378090 2023-10-05T19:40:16.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.575+0000 7f3aefa02700 1 --2- 172.21.15.12:0/926293803 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f3ae0016730 0x7f3ae0042260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:16.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.575+0000 7f3aefa02700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55585837f2f0 con 0x7f3ae0016730 2023-10-05T19:40:16.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.575+0000 7f3af0203700 1 --2- 172.21.15.12:0/926293803 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f3ae0016730 0x7f3ae0042260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:16.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.575+0000 7f3af0203700 1 --2- 172.21.15.12:0/926293803 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f3ae0016730 0x7f3ae0042260 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:16.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.575+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ae8014040 con 0x7f3ae0016730 2023-10-05T19:40:16.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.575+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55585837f2f0 con 0x55585833e7e0 2023-10-05T19:40:16.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.575+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ae8014040 con 0x55585833e7e0 2023-10-05T19:40:16.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.575+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55585837f2f0 con 0x55585833e7e0 2023-10-05T19:40:16.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.579+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ae8014040 con 0x55585833e7e0 2023-10-05T19:40:16.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.579+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55585837f2f0 con 0x7f3ae0016730 2023-10-05T19:40:16.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.579+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ae8014040 con 0x7f3ae0016730 2023-10-05T19:40:16.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.579+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55585837f2f0 con 0x7f3ae0016730 2023-10-05T19:40:16.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.579+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ae8014040 con 0x7f3ae0016730 2023-10-05T19:40:16.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.579+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55585837f2f0 con 0x555858378090 2023-10-05T19:40:16.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.579+0000 7f3aefa02700 1 -- 172.21.15.12:0/926293803 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ae0014070 con 0x555858378090 2023-10-05T19:40:16.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.579+0000 7f3aefa02700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55585837f2f0 con 0x55585837c830 2023-10-05T19:40:16.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.579+0000 7f3aef201700 1 -- 172.21.15.12:0/926293803 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ad800fd60 con 0x55585837c830 2023-10-05T19:40:16.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.579+0000 7f3aef201700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55585837f2f0 con 0x55585833e7e0 2023-10-05T19:40:16.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.583+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ae8014040 con 0x55585833e7e0 2023-10-05T19:40:16.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.583+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55585837f2f0 con 0x7f3ae0016730 2023-10-05T19:40:16.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.583+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ae8014040 con 0x7f3ae0016730 2023-10-05T19:40:16.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.583+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55585837f2f0 con 0x55585833e7e0 2023-10-05T19:40:16.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.583+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ae8014040 con 0x55585833e7e0 2023-10-05T19:40:16.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.583+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55585837f2f0 con 0x555858378090 2023-10-05T19:40:16.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.583+0000 7f3aefa02700 1 -- 172.21.15.12:0/926293803 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ae0014070 con 0x555858378090 2023-10-05T19:40:16.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.583+0000 7f3aefa02700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55585837f2f0 con 0x55585837c830 2023-10-05T19:40:16.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.583+0000 7f3aef201700 1 -- 172.21.15.12:0/926293803 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ad800fd60 con 0x55585837c830 2023-10-05T19:40:16.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.583+0000 7f3aef201700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55585837f2f0 con 0x55585837c830 2023-10-05T19:40:16.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.583+0000 7f3aef201700 1 -- 172.21.15.12:0/926293803 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ad800fd60 con 0x55585837c830 2023-10-05T19:40:16.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.583+0000 7f3aef201700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55585837f2f0 con 0x7f3ae0016730 2023-10-05T19:40:16.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.587+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ae8014040 con 0x7f3ae0016730 2023-10-05T19:40:16.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.587+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55585837f2f0 con 0x55585837c830 2023-10-05T19:40:16.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.587+0000 7f3aef201700 1 -- 172.21.15.12:0/926293803 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ad800fd60 con 0x55585837c830 2023-10-05T19:40:16.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.587+0000 7f3aef201700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55585837f2f0 con 0x55585837c830 2023-10-05T19:40:16.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.587+0000 7f3aef201700 1 -- 172.21.15.12:0/926293803 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ad800fd60 con 0x55585837c830 2023-10-05T19:40:16.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.587+0000 7f3aef201700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55585837f2f0 con 0x55585833e7e0 2023-10-05T19:40:16.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.587+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ae8014040 con 0x55585833e7e0 2023-10-05T19:40:16.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.587+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55585837f2f0 con 0x55585833e7e0 2023-10-05T19:40:16.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.587+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ae8014040 con 0x55585833e7e0 2023-10-05T19:40:16.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.587+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55585837f2f0 con 0x7f3ae0016730 2023-10-05T19:40:16.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.591+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ae8014040 con 0x7f3ae0016730 2023-10-05T19:40:16.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.591+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55585837f2f0 con 0x55585837c830 2023-10-05T19:40:16.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.591+0000 7f3aef201700 1 -- 172.21.15.12:0/926293803 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ad800fd60 con 0x55585837c830 2023-10-05T19:40:16.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.591+0000 7f3aef201700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55585837f2f0 con 0x555858378090 2023-10-05T19:40:16.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.591+0000 7f3aefa02700 1 -- 172.21.15.12:0/926293803 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ae0014070 con 0x555858378090 2023-10-05T19:40:16.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.591+0000 7f3aefa02700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55585837f2f0 con 0x55585833e7e0 2023-10-05T19:40:16.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.591+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ae8014040 con 0x55585833e7e0 2023-10-05T19:40:16.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.591+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55585837f2f0 con 0x7f3ae0016730 2023-10-05T19:40:16.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.595+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ae8014040 con 0x7f3ae0016730 2023-10-05T19:40:16.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.595+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55585837f2f0 con 0x55585837c830 2023-10-05T19:40:16.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.595+0000 7f3aef201700 1 -- 172.21.15.12:0/926293803 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ad800fd60 con 0x55585837c830 2023-10-05T19:40:16.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.595+0000 7f3aef201700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55585837f2f0 con 0x7f3ae0016730 2023-10-05T19:40:16.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.595+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ae8014040 con 0x7f3ae0016730 2023-10-05T19:40:16.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.595+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55585837f2f0 con 0x555858378090 2023-10-05T19:40:16.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.595+0000 7f3aefa02700 1 -- 172.21.15.12:0/926293803 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ae0014070 con 0x555858378090 2023-10-05T19:40:16.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.595+0000 7f3aefa02700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55585837f2f0 con 0x55585837c830 2023-10-05T19:40:16.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.595+0000 7f3aef201700 1 -- 172.21.15.12:0/926293803 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ad800fd60 con 0x55585837c830 2023-10-05T19:40:16.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.595+0000 7f3aef201700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55585837f2f0 con 0x7f3ae0016730 2023-10-05T19:40:16.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.599+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ae8014040 con 0x7f3ae0016730 2023-10-05T19:40:16.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.599+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55585837f2f0 con 0x55585833e7e0 2023-10-05T19:40:16.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.599+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f3ae8014040 con 0x55585833e7e0 2023-10-05T19:40:16.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.599+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55585837f2f0 con 0x555858378090 2023-10-05T19:40:16.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.599+0000 7f3aefa02700 1 -- 172.21.15.12:0/926293803 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f3ae0014070 con 0x555858378090 2023-10-05T19:40:16.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.599+0000 7f3af1ecde40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:16.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.599+0000 7f3af1ecde40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:16.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.599+0000 7f3af1ecde40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:16.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.599+0000 7f3af1ecde40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:16.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.599+0000 7f3af1ecde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.599+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55585837b5b0 con 0x55585833e7e0 2023-10-05T19:40:16.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.599+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3ae8014040 con 0x55585833e7e0 2023-10-05T19:40:16.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.599+0000 7f3af1ecde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:16.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.599+0000 7f3af1ecde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.599+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55585837b1e0 con 0x55585833e7e0 2023-10-05T19:40:16.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.599+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f3ae8014040 con 0x55585833e7e0 2023-10-05T19:40:16.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.599+0000 7f3af1ecde40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:16.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.599+0000 7f3af1ecde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.599+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55585837f760 con 0x55585837c830 2023-10-05T19:40:16.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.603+0000 7f3aef201700 1 -- 172.21.15.12:0/926293803 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f3ad800fd60 con 0x55585837c830 2023-10-05T19:40:16.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.603+0000 7f3af1ecde40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:16.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.603+0000 7f3af1ecde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.603+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55585837fb10 con 0x55585837c830 2023-10-05T19:40:16.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.603+0000 7f3aef201700 1 -- 172.21.15.12:0/926293803 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f3ad800fd60 con 0x55585837c830 2023-10-05T19:40:16.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.603+0000 7f3af1ecde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:16.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.603+0000 7f3af1ecde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.603+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55585837fec0 con 0x55585833e7e0 2023-10-05T19:40:16.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.603+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3ae8014040 con 0x55585833e7e0 2023-10-05T19:40:16.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.603+0000 7f3af1ecde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:16.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.603+0000 7f3af1ecde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.603+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558583802d0 con 0x55585833e7e0 2023-10-05T19:40:16.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.603+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f3ae8014040 con 0x55585833e7e0 2023-10-05T19:40:16.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.603+0000 7f3af1ecde40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:16.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.603+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558583802a0 con 0x55585837c830 2023-10-05T19:40:16.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.607+0000 7f3aef201700 1 -- 172.21.15.12:0/926293803 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f3ad800fd60 con 0x55585837c830 2023-10-05T19:40:16.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.607+0000 7f3aef201700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558583802a0 con 0x555858378090 2023-10-05T19:40:16.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.607+0000 7f3aefa02700 1 -- 172.21.15.12:0/926293803 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ae0014070 con 0x555858378090 2023-10-05T19:40:16.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.607+0000 7f3aefa02700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558583802a0 con 0x7f3ae0016730 2023-10-05T19:40:16.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.607+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ae8014040 con 0x7f3ae0016730 2023-10-05T19:40:16.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.607+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55585837fec0 con 0x55585833e7e0 2023-10-05T19:40:16.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.611+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ae8014040 con 0x55585833e7e0 2023-10-05T19:40:16.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.611+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55585837fec0 con 0x55585833e7e0 2023-10-05T19:40:16.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.611+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ae8014040 con 0x55585833e7e0 2023-10-05T19:40:16.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.611+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55585837fec0 con 0x7f3ae0016730 2023-10-05T19:40:16.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.611+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ae8014040 con 0x7f3ae0016730 2023-10-05T19:40:16.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.611+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55585837fec0 con 0x7f3ae0016730 2023-10-05T19:40:16.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.611+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ae8014040 con 0x7f3ae0016730 2023-10-05T19:40:16.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.611+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55585837fec0 con 0x555858378090 2023-10-05T19:40:16.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.615+0000 7f3aefa02700 1 -- 172.21.15.12:0/926293803 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ae0014070 con 0x555858378090 2023-10-05T19:40:16.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.615+0000 7f3aefa02700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55585837fec0 con 0x55585837c830 2023-10-05T19:40:16.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.615+0000 7f3aef201700 1 -- 172.21.15.12:0/926293803 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ad800fd60 con 0x55585837c830 2023-10-05T19:40:16.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.615+0000 7f3aef201700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f3ad8039040 con 0x55585833e7e0 2023-10-05T19:40:16.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.615+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ae8014040 con 0x55585833e7e0 2023-10-05T19:40:16.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.615+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f3ad8039040 con 0x7f3ae0016730 2023-10-05T19:40:16.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.615+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ae8014040 con 0x7f3ae0016730 2023-10-05T19:40:16.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.615+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f3ad8039040 con 0x55585833e7e0 2023-10-05T19:40:16.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.615+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ae8014040 con 0x55585833e7e0 2023-10-05T19:40:16.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.615+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f3ad8039040 con 0x555858378090 2023-10-05T19:40:16.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.619+0000 7f3aefa02700 1 -- 172.21.15.12:0/926293803 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ae0014070 con 0x555858378090 2023-10-05T19:40:16.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.619+0000 7f3aefa02700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f3ad8039040 con 0x55585837c830 2023-10-05T19:40:16.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.619+0000 7f3aef201700 1 -- 172.21.15.12:0/926293803 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ad800fd60 con 0x55585837c830 2023-10-05T19:40:16.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.619+0000 7f3aef201700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f3ad8039040 con 0x55585837c830 2023-10-05T19:40:16.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.619+0000 7f3aef201700 1 -- 172.21.15.12:0/926293803 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ad800fd60 con 0x55585837c830 2023-10-05T19:40:16.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.619+0000 7f3aef201700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f3ad8039040 con 0x7f3ae0016730 2023-10-05T19:40:16.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.619+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ae8014040 con 0x7f3ae0016730 2023-10-05T19:40:16.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.619+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f3ad8039040 con 0x55585837c830 2023-10-05T19:40:16.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.619+0000 7f3aef201700 1 -- 172.21.15.12:0/926293803 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ad800fd60 con 0x55585837c830 2023-10-05T19:40:16.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.619+0000 7f3aef201700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f3ad8039040 con 0x55585837c830 2023-10-05T19:40:16.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.623+0000 7f3aef201700 1 -- 172.21.15.12:0/926293803 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ad800fd60 con 0x55585837c830 2023-10-05T19:40:16.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.623+0000 7f3aef201700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f3ad8039040 con 0x55585833e7e0 2023-10-05T19:40:16.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.623+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ae8014040 con 0x55585833e7e0 2023-10-05T19:40:16.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.623+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f3ad8039040 con 0x55585833e7e0 2023-10-05T19:40:16.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.623+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ae8014040 con 0x55585833e7e0 2023-10-05T19:40:16.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.623+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f3ad8039040 con 0x7f3ae0016730 2023-10-05T19:40:16.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.623+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ae8014040 con 0x7f3ae0016730 2023-10-05T19:40:16.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.623+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f3ad8039040 con 0x55585837c830 2023-10-05T19:40:16.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.627+0000 7f3aef201700 1 -- 172.21.15.12:0/926293803 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ad800fd60 con 0x55585837c830 2023-10-05T19:40:16.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.627+0000 7f3aef201700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f3ad8039040 con 0x555858378090 2023-10-05T19:40:16.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.627+0000 7f3aefa02700 1 -- 172.21.15.12:0/926293803 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ae0014070 con 0x555858378090 2023-10-05T19:40:16.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.627+0000 7f3aefa02700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f3ad8039040 con 0x55585833e7e0 2023-10-05T19:40:16.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.627+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ae8014040 con 0x55585833e7e0 2023-10-05T19:40:16.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.627+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f3ad8039040 con 0x7f3ae0016730 2023-10-05T19:40:16.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.627+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ae8014040 con 0x7f3ae0016730 2023-10-05T19:40:16.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.627+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f3ad8039040 con 0x55585837c830 2023-10-05T19:40:16.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.631+0000 7f3aef201700 1 -- 172.21.15.12:0/926293803 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ad800fd60 con 0x55585837c830 2023-10-05T19:40:16.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.631+0000 7f3aef201700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f3ad8039040 con 0x7f3ae0016730 2023-10-05T19:40:16.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.631+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ae8014040 con 0x7f3ae0016730 2023-10-05T19:40:16.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.631+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f3ad8039040 con 0x555858378090 2023-10-05T19:40:16.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.631+0000 7f3aefa02700 1 -- 172.21.15.12:0/926293803 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ae0014070 con 0x555858378090 2023-10-05T19:40:16.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.631+0000 7f3aefa02700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f3ad8039040 con 0x55585837c830 2023-10-05T19:40:16.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.631+0000 7f3aef201700 1 -- 172.21.15.12:0/926293803 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ad800fd60 con 0x55585837c830 2023-10-05T19:40:16.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.631+0000 7f3aef201700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f3ad8039040 con 0x7f3ae0016730 2023-10-05T19:40:16.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.631+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ae8014040 con 0x7f3ae0016730 2023-10-05T19:40:16.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.631+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f3ad8039040 con 0x55585833e7e0 2023-10-05T19:40:16.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.635+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f3ae8014040 con 0x55585833e7e0 2023-10-05T19:40:16.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.635+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f3ad8039040 con 0x555858378090 2023-10-05T19:40:16.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.635+0000 7f3aefa02700 1 -- 172.21.15.12:0/926293803 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f3ae0014070 con 0x555858378090 2023-10-05T19:40:16.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.635+0000 7f3af1ecde40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:16.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.635+0000 7f3af1ecde40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:16.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.635+0000 7f3af1ecde40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:16.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.635+0000 7f3af1ecde40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:16.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.635+0000 7f3af1ecde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.635+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558583802a0 con 0x55585833e7e0 2023-10-05T19:40:16.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.635+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f3ae8014040 con 0x55585833e7e0 2023-10-05T19:40:16.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.635+0000 7f3af1ecde40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:16.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.635+0000 7f3af1ecde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.635+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555858380700 con 0x555858378090 2023-10-05T19:40:16.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.635+0000 7f3aefa02700 1 -- 172.21.15.12:0/926293803 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv1 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f3ae0014070 con 0x555858378090 2023-10-05T19:40:16.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.635+0000 7f3af1ecde40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:16.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.639+0000 7f3af1ecde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.639+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555858380ff0 con 0x55585833e7e0 2023-10-05T19:40:16.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.639+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f3ae8014040 con 0x55585833e7e0 2023-10-05T19:40:16.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.639+0000 7f3af1ecde40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:16.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.639+0000 7f3af1ecde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.639+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555858380ff0 con 0x55585837c830 2023-10-05T19:40:16.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.639+0000 7f3aef201700 1 -- 172.21.15.12:0/926293803 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f3ad800fd60 con 0x55585837c830 2023-10-05T19:40:16.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.639+0000 7f3af1ecde40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:16.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.639+0000 7f3af1ecde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.639+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555858381760 con 0x7f3ae0016730 2023-10-05T19:40:16.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.639+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f3ae8014040 con 0x7f3ae0016730 2023-10-05T19:40:16.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.639+0000 7f3af1ecde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:16.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.639+0000 7f3af1ecde40 20 zone default found 2023-10-05T19:40:16.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.639+0000 7f3af1ecde40 4 Realm: () 2023-10-05T19:40:16.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.639+0000 7f3af1ecde40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:16.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.639+0000 7f3af1ecde40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:16.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.639+0000 7f3af1ecde40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:16.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.639+0000 7f3af1ecde40 20 zonegroup default 2023-10-05T19:40:16.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.639+0000 7f3af1ecde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.639+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555858381560 con 0x7f3ae0016730 2023-10-05T19:40:16.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.639+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f3ae8014040 con 0x7f3ae0016730 2023-10-05T19:40:16.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.639+0000 7f3af1ecde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:16.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.639+0000 7f3af1ecde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.639+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555858382ee0 con 0x55585837c830 2023-10-05T19:40:16.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.643+0000 7f3aef201700 1 -- 172.21.15.12:0/926293803 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f3ad800fd60 con 0x55585837c830 2023-10-05T19:40:16.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.643+0000 7f3af1ecde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:16.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.643+0000 7f3af1ecde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.643+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555858383290 con 0x55585837c830 2023-10-05T19:40:16.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.643+0000 7f3aef201700 1 -- 172.21.15.12:0/926293803 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f3ad800fd60 con 0x55585837c830 2023-10-05T19:40:16.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.643+0000 7f3af1ecde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:16.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.643+0000 7f3af1ecde40 20 started sync module instance, tier type = 2023-10-05T19:40:16.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.643+0000 7f3af1ecde40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:16.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.643+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558583825a0 con 0x555858378090 2023-10-05T19:40:16.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.647+0000 7f3aefa02700 1 -- 172.21.15.12:0/926293803 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'9 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3ae0014070 con 0x555858378090 2023-10-05T19:40:16.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.647+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93837925558688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555858380c20 con 0x555858378090 2023-10-05T19:40:16.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.647+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555858384020 con 0x55585833e7e0 2023-10-05T19:40:16.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.647+0000 7f3aefa02700 1 -- 172.21.15.12:0/926293803 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 93837925558688] v28'10 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3ae0014070 con 0x555858378090 2023-10-05T19:40:16.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.651+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'9 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3ae8014040 con 0x55585833e7e0 2023-10-05T19:40:16.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.651+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93837925562208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558583840a0 con 0x55585833e7e0 2023-10-05T19:40:16.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.651+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555858384bb0 con 0x55585837c830 2023-10-05T19:40:16.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.651+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 93837925562208] v28'10 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3ae8014040 con 0x55585833e7e0 2023-10-05T19:40:16.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.651+0000 7f3aef201700 1 -- 172.21.15.12:0/926293803 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'9 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3ad800fd60 con 0x55585837c830 2023-10-05T19:40:16.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.651+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93837925563312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558583840c0 con 0x55585837c830 2023-10-05T19:40:16.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.651+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555858385470 con 0x555858378090 2023-10-05T19:40:16.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.655+0000 7f3aefa02700 1 -- 172.21.15.12:0/926293803 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'9 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3ae0014070 con 0x555858378090 2023-10-05T19:40:16.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.655+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 93837925570672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558583861b0 con 0x555858378090 2023-10-05T19:40:16.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.655+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555858386a90 con 0x55585833e7e0 2023-10-05T19:40:16.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.655+0000 7f3aef201700 1 -- 172.21.15.12:0/926293803 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 93837925563312] v28'10 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3ad800fd60 con 0x55585837c830 2023-10-05T19:40:16.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.659+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'9 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3ae8014040 con 0x55585833e7e0 2023-10-05T19:40:16.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.659+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 93837925566368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558583866a0 con 0x55585833e7e0 2023-10-05T19:40:16.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.659+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558583874f0 con 0x7f3ae0016730 2023-10-05T19:40:16.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.659+0000 7f3aefa02700 1 -- 172.21.15.12:0/926293803 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 93837925570672] v28'10 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3ae0014070 con 0x555858378090 2023-10-05T19:40:16.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.659+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 93837925566368] v28'10 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3ae8014040 con 0x55585833e7e0 2023-10-05T19:40:16.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.663+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'9 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3ae8014040 con 0x7f3ae0016730 2023-10-05T19:40:16.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.663+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 93837925577264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558583861b0 con 0x7f3ae0016730 2023-10-05T19:40:16.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.663+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555858387de0 con 0x7f3ae0016730 2023-10-05T19:40:16.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.663+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(95 notify.5 [watch watch cookie 93837925577264] v28'10 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3ae8014040 con 0x7f3ae0016730 2023-10-05T19:40:16.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.663+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(96 notify.6 [create] v28'9 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3ae8014040 con 0x7f3ae0016730 2023-10-05T19:40:16.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.663+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93837925571776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555858386110 con 0x7f3ae0016730 2023-10-05T19:40:16.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.663+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555858388310 con 0x7f3ae0016730 2023-10-05T19:40:16.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.667+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(98 notify.7 [create] v28'9 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3ae8014040 con 0x7f3ae0016730 2023-10-05T19:40:16.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.667+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93837925580256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555858388720 con 0x7f3ae0016730 2023-10-05T19:40:16.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.667+0000 7f3af1ecde40 20 add_watcher() i=0 2023-10-05T19:40:16.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.667+0000 7f3af1ecde40 20 add_watcher() i=1 2023-10-05T19:40:16.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.667+0000 7f3af1ecde40 20 add_watcher() i=2 2023-10-05T19:40:16.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.667+0000 7f3af1ecde40 20 add_watcher() i=3 2023-10-05T19:40:16.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.667+0000 7f3af1ecde40 20 add_watcher() i=4 2023-10-05T19:40:16.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.667+0000 7f3af1ecde40 20 add_watcher() i=5 2023-10-05T19:40:16.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.667+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(97 notify.6 [watch watch cookie 93837925571776] v28'10 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3ae8014040 con 0x7f3ae0016730 2023-10-05T19:40:16.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.667+0000 7f3af1ecde40 20 add_watcher() i=6 2023-10-05T19:40:16.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.671+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 93837925580256] v28'10 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3ae8014040 con 0x7f3ae0016730 2023-10-05T19:40:16.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.671+0000 7f3af1ecde40 20 add_watcher() i=7 2023-10-05T19:40:16.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.671+0000 7f3af1ecde40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:16.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.671+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558583878f0 con 0x55585833e7e0 2023-10-05T19:40:16.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.671+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f3ae8014040 con 0x55585833e7e0 2023-10-05T19:40:16.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.671+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93837925581360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555858388bc0 con 0x55585833e7e0 2023-10-05T19:40:16.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.671+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 93837925581360] v28'31 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f3ae8014040 con 0x55585833e7e0 2023-10-05T19:40:16.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.671+0000 7f3ac17e2700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:16.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.671+0000 7f3af1ecde40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:16.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.671+0000 7f3af1ecde40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:16.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.671+0000 7f3af1ecde40 1 Processor -- start 2023-10-05T19:40:16.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.671+0000 7f3af1ecde40 1 -- start start 2023-10-05T19:40:16.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.675+0000 7f3af1ecde40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555858395ca0 0x555858396070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:16.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.675+0000 7f3af1ecde40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558583965b0 con 0x555858395ca0 2023-10-05T19:40:16.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.675+0000 7f3aef201700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555858395ca0 0x555858396070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:16.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.675+0000 7f3aef201700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555858395ca0 0x555858396070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36368/0 (socket says 172.21.15.12:36368) 2023-10-05T19:40:16.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.675+0000 7f3aef201700 1 -- 172.21.15.12:0/4191602449 learned_addr learned my addr 172.21.15.12:0/4191602449 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:16.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.675+0000 7f3aef201700 1 -- 172.21.15.12:0/4191602449 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558583966f0 con 0x555858395ca0 2023-10-05T19:40:16.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.675+0000 7f3aef201700 1 --2- 172.21.15.12:0/4191602449 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555858395ca0 0x555858396070 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8003000 tx=0x7f3ad8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1399e5d16b5170b server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:16.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.675+0000 7f3ac0fe1700 1 -- 172.21.15.12:0/4191602449 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3ad801cd00 con 0x555858395ca0 2023-10-05T19:40:16.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.675+0000 7f3ac0fe1700 1 -- 172.21.15.12:0/4191602449 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad801ce60 con 0x555858395ca0 2023-10-05T19:40:16.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.675+0000 7f3ac0fe1700 1 -- 172.21.15.12:0/4191602449 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3ad8029420 con 0x555858395ca0 2023-10-05T19:40:16.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.675+0000 7f3af1ecde40 1 -- 172.21.15.12:0/4191602449 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555858395ca0 msgr2=0x555858396070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:16.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.675+0000 7f3af1ecde40 1 --2- 172.21.15.12:0/4191602449 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555858395ca0 0x555858396070 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8003000 tx=0x7f3ad8011000 comp rx=0 tx=0).stop 2023-10-05T19:40:16.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.675+0000 7f3af1ecde40 1 -- 172.21.15.12:0/4191602449 shutdown_connections 2023-10-05T19:40:16.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.675+0000 7f3af1ecde40 1 --2- 172.21.15.12:0/4191602449 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555858395ca0 0x555858396070 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:16.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.675+0000 7f3af1ecde40 1 -- 172.21.15.12:0/4191602449 >> 172.21.15.12:0/4191602449 conn(0x555858391060 msgr2=0x55585838f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:16.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.675+0000 7f3af1ecde40 1 -- 172.21.15.12:0/4191602449 shutdown_connections 2023-10-05T19:40:16.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.675+0000 7f3af1ecde40 1 -- 172.21.15.12:0/4191602449 wait complete. 2023-10-05T19:40:16.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.675+0000 7f3af1ecde40 1 Processor -- start 2023-10-05T19:40:16.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.675+0000 7f3af1ecde40 1 -- start start 2023-10-05T19:40:16.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.679+0000 7f3af1ecde40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555858395ca0 0x55585839b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:16.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.679+0000 7f3af1ecde40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55585839c740 con 0x555858395ca0 2023-10-05T19:40:16.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.679+0000 7f3aef201700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555858395ca0 0x55585839b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:16.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.679+0000 7f3aef201700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555858395ca0 0x55585839b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36382/0 (socket says 172.21.15.12:36382) 2023-10-05T19:40:16.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.679+0000 7f3aef201700 1 -- 172.21.15.12:0/748884875 learned_addr learned my addr 172.21.15.12:0/748884875 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:16.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.679+0000 7f3aef201700 1 -- 172.21.15.12:0/748884875 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad802e040 con 0x555858395ca0 2023-10-05T19:40:16.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.679+0000 7f3aef201700 1 --2- 172.21.15.12:0/748884875 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555858395ca0 0x55585839b740 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3ad803c040 tx=0x7f3ad8042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:16.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.679+0000 7f3a3affd700 1 -- 172.21.15.12:0/748884875 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3ad8010c90 con 0x555858395ca0 2023-10-05T19:40:16.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.679+0000 7f3a3affd700 1 -- 172.21.15.12:0/748884875 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad8010e10 con 0x555858395ca0 2023-10-05T19:40:16.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.679+0000 7f3af1ecde40 1 -- 172.21.15.12:0/748884875 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55585839bd40 con 0x555858395ca0 2023-10-05T19:40:16.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.679+0000 7f3a3affd700 1 -- 172.21.15.12:0/748884875 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3ad804f440 con 0x555858395ca0 2023-10-05T19:40:16.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.679+0000 7f3af1ecde40 1 -- 172.21.15.12:0/748884875 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55585839c1e0 con 0x555858395ca0 2023-10-05T19:40:16.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.679+0000 7f3a3affd700 1 -- 172.21.15.12:0/748884875 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f3ad804b040 con 0x555858395ca0 2023-10-05T19:40:16.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.679+0000 7f3a3affd700 1 --2- 172.21.15.12:0/748884875 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3a30030c50 0x7f3a30033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:16.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.679+0000 7f3a3affd700 1 -- 172.21.15.12:0/748884875 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f3ad8050c60 con 0x555858395ca0 2023-10-05T19:40:16.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.683+0000 7f3af0203700 1 --2- 172.21.15.12:0/748884875 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3a30030c50 0x7f3a30033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:16.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.683+0000 7f3af1ecde40 1 --2- 172.21.15.12:0/748884875 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5558583a6330 0x5558583a8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:16.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.683+0000 7f3af1ecde40 1 -- 172.21.15.12:0/748884875 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558583a8d30 con 0x5558583a6330 2023-10-05T19:40:16.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.683+0000 7f3aefa02700 1 --2- 172.21.15.12:0/748884875 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5558583a6330 0x5558583a8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:16.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.683+0000 7f3af0203700 1 --2- 172.21.15.12:0/748884875 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3a30030c50 0x7f3a30033100 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x5558583862f0 tx=0x7f3ae800ab20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:16.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.683+0000 7f3aefa02700 1 --2- 172.21.15.12:0/748884875 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5558583a6330 0x5558583a8710 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:16.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.683+0000 7f3aefa02700 1 -- 172.21.15.12:0/748884875 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v28'17 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f3ae0014070 con 0x5558583a6330 2023-10-05T19:40:16.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.687+0000 7f3af1ecde40 1 --2- 172.21.15.12:0/748884875 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x5558583a9a00 0x5558583abde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:16.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.687+0000 7f3af1ecde40 1 -- 172.21.15.12:0/748884875 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558583ac4e0 con 0x5558583a9a00 2023-10-05T19:40:16.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.687+0000 7f3aef201700 1 --2- 172.21.15.12:0/748884875 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x5558583a9a00 0x5558583abde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:16.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.687+0000 7f3aef201700 1 --2- 172.21.15.12:0/748884875 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x5558583a9a00 0x5558583abde0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:16.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.687+0000 7f3aef201700 1 -- 172.21.15.12:0/748884875 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v28'11 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f3ad800fd60 con 0x5558583a9a00 2023-10-05T19:40:16.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.687+0000 7f3af1ecde40 1 --2- 172.21.15.12:0/748884875 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5558583ad1b0 0x5558583af5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:16.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.687+0000 7f3af1ecde40 1 -- 172.21.15.12:0/748884875 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558583afcf0 con 0x5558583ad1b0 2023-10-05T19:40:16.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.691+0000 7f3af0203700 1 --2- 172.21.15.12:0/748884875 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5558583ad1b0 0x5558583af5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:16.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.691+0000 7f3af0203700 1 --2- 172.21.15.12:0/748884875 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5558583ad1b0 0x5558583af5c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:16.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.691+0000 7f3af0203700 1 -- 172.21.15.12:0/748884875 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v28'11 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f3ae8014040 con 0x5558583ad1b0 2023-10-05T19:40:16.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.691+0000 7f3af1ecde40 1 -- 172.21.15.12:0/748884875 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558583b0340 con 0x5558583a6330 2023-10-05T19:40:16.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.695+0000 7f3aefa02700 1 -- 172.21.15.12:0/748884875 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v28'32 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f3ae0014070 con 0x5558583a6330 2023-10-05T19:40:16.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.695+0000 7f3af1ecde40 1 --2- 172.21.15.12:0/748884875 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5558583b0fe0 0x5558583b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:16.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.695+0000 7f3af1ecde40 1 -- 172.21.15.12:0/748884875 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558583b3a70 con 0x5558583b0fe0 2023-10-05T19:40:16.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.695+0000 7f3aefa02700 1 --2- 172.21.15.12:0/748884875 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5558583b0fe0 0x5558583b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:16.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.695+0000 7f3aefa02700 1 --2- 172.21.15.12:0/748884875 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5558583b0fe0 0x5558583b33c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:16.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.699+0000 7f3aefa02700 1 -- 172.21.15.12:0/748884875 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v28'23 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f3ae0014070 con 0x5558583b0fe0 2023-10-05T19:40:16.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.699+0000 7f3af1ecde40 1 -- 172.21.15.12:0/748884875 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558583b4090 con 0x5558583ad1b0 2023-10-05T19:40:16.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.703+0000 7f3af0203700 1 -- 172.21.15.12:0/748884875 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v28'15 uv8 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f3ae8014040 con 0x5558583ad1b0 2023-10-05T19:40:16.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.703+0000 7f3af1ecde40 1 -- 172.21.15.12:0/748884875 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558583b46b0 con 0x5558583a6330 2023-10-05T19:40:16.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.703+0000 7f3aefa02700 1 -- 172.21.15.12:0/748884875 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v28'13 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f3ae0014070 con 0x5558583a6330 2023-10-05T19:40:16.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.703+0000 7f3af1ecde40 1 -- 172.21.15.12:0/748884875 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558583b46b0 con 0x5558583a9a00 2023-10-05T19:40:16.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.707+0000 7f3aef201700 1 -- 172.21.15.12:0/748884875 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v28'15 uv8 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f3ad800fd60 con 0x5558583a9a00 2023-10-05T19:40:16.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.707+0000 7f3af1ecde40 1 -- 172.21.15.12:0/748884875 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558583b4cd0 con 0x5558583b0fe0 2023-10-05T19:40:16.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.711+0000 7f3aefa02700 1 -- 172.21.15.12:0/748884875 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v28'11 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f3ae0014070 con 0x5558583b0fe0 2023-10-05T19:40:16.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.711+0000 7f3af1ecde40 1 -- 172.21.15.12:0/748884875 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558583b4cd0 con 0x5558583ad1b0 2023-10-05T19:40:16.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.711+0000 7f3af0203700 1 -- 172.21.15.12:0/748884875 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v28'17 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f3ae8014040 con 0x5558583ad1b0 2023-10-05T19:40:16.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.711+0000 7f3af1ecde40 1 -- 172.21.15.12:0/748884875 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558583b52f0 con 0x5558583b0fe0 2023-10-05T19:40:16.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.715+0000 7f3aefa02700 1 -- 172.21.15.12:0/748884875 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v28'24 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3ae0014070 con 0x5558583b0fe0 2023-10-05T19:40:16.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.715+0000 7f3af1ecde40 1 -- 172.21.15.12:0/748884875 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558583b52f0 con 0x5558583a9a00 2023-10-05T19:40:16.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.715+0000 7f3aef201700 1 -- 172.21.15.12:0/748884875 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v28'13 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3ad800fd60 con 0x5558583a9a00 2023-10-05T19:40:16.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.715+0000 7f3af1ecde40 1 -- 172.21.15.12:0/748884875 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558583b57e0 con 0x5558583b0fe0 2023-10-05T19:40:16.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.719+0000 7f3aefa02700 1 -- 172.21.15.12:0/748884875 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v28'11 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3ae0014070 con 0x5558583b0fe0 2023-10-05T19:40:16.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.719+0000 7f3af1ecde40 1 -- 172.21.15.12:0/748884875 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558583b57e0 con 0x5558583a9a00 2023-10-05T19:40:16.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.719+0000 7f3aef201700 1 -- 172.21.15.12:0/748884875 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v28'21 uv12 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3ad800fd60 con 0x5558583a9a00 2023-10-05T19:40:16.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.719+0000 7f3af1ecde40 1 -- 172.21.15.12:0/748884875 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558583b57e0 con 0x5558583a6330 2023-10-05T19:40:16.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.723+0000 7f3aefa02700 1 -- 172.21.15.12:0/748884875 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v28'33 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3ae0014070 con 0x5558583a6330 2023-10-05T19:40:16.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.723+0000 7f3af1ecde40 1 -- 172.21.15.12:0/748884875 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558583b57e0 con 0x5558583ad1b0 2023-10-05T19:40:16.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.727+0000 7f3af0203700 1 -- 172.21.15.12:0/748884875 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v28'17 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3ae8014040 con 0x5558583ad1b0 2023-10-05T19:40:16.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.727+0000 7f3af1ecde40 1 -- 172.21.15.12:0/748884875 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558583a1d40 con 0x5558583b0fe0 2023-10-05T19:40:16.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.727+0000 7f3aefa02700 1 -- 172.21.15.12:0/748884875 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v28'25 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3ae0014070 con 0x5558583b0fe0 2023-10-05T19:40:16.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.727+0000 7f3af1ecde40 1 -- 172.21.15.12:0/748884875 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558583a1d40 con 0x5558583ad1b0 2023-10-05T19:40:16.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.731+0000 7f3af0203700 1 -- 172.21.15.12:0/748884875 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v28'11 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3ae8014040 con 0x5558583ad1b0 2023-10-05T19:40:16.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.731+0000 7f3af1ecde40 1 -- 172.21.15.12:0/748884875 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558583a1d10 con 0x5558583a6330 2023-10-05T19:40:16.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.735+0000 7f3aefa02700 1 -- 172.21.15.12:0/748884875 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v28'19 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3ae0014070 con 0x5558583a6330 2023-10-05T19:40:16.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.735+0000 7f3af1ecde40 1 -- 172.21.15.12:0/748884875 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558583a1f80 con 0x5558583a6330 2023-10-05T19:40:16.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.735+0000 7f3aefa02700 1 -- 172.21.15.12:0/748884875 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v28'34 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3ae0014070 con 0x5558583a6330 2023-10-05T19:40:16.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.735+0000 7f3af1ecde40 1 -- 172.21.15.12:0/748884875 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558583a1f80 con 0x5558583ad1b0 2023-10-05T19:40:16.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.739+0000 7f3af0203700 1 -- 172.21.15.12:0/748884875 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v28'18 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3ae8014040 con 0x5558583ad1b0 2023-10-05T19:40:16.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.739+0000 7f3af1ecde40 1 -- 172.21.15.12:0/748884875 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558583a1f80 con 0x5558583b0fe0 2023-10-05T19:40:16.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.743+0000 7f3aefa02700 1 -- 172.21.15.12:0/748884875 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v28'21 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3ae0014070 con 0x5558583b0fe0 2023-10-05T19:40:16.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.743+0000 7f3af1ecde40 1 -- 172.21.15.12:0/748884875 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558583a1c80 con 0x5558583ad1b0 2023-10-05T19:40:16.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.743+0000 7f3af0203700 1 -- 172.21.15.12:0/748884875 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v28'13 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3ae8014040 con 0x5558583ad1b0 2023-10-05T19:40:16.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.743+0000 7f3af1ecde40 1 -- 172.21.15.12:0/748884875 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558583a1c80 con 0x5558583ad1b0 2023-10-05T19:40:16.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.747+0000 7f3af0203700 1 -- 172.21.15.12:0/748884875 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v28'18 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3ae8014040 con 0x5558583ad1b0 2023-10-05T19:40:16.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.747+0000 7f3af1ecde40 1 -- 172.21.15.12:0/748884875 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558583a1c80 con 0x5558583ad1b0 2023-10-05T19:40:16.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.747+0000 7f3af0203700 1 -- 172.21.15.12:0/748884875 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v28'7 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3ae8014040 con 0x5558583ad1b0 2023-10-05T19:40:16.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.747+0000 7f3af1ecde40 1 -- 172.21.15.12:0/748884875 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558583a1ef0 con 0x5558583ad1b0 2023-10-05T19:40:16.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.751+0000 7f3af0203700 1 -- 172.21.15.12:0/748884875 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v28'16 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3ae8014040 con 0x5558583ad1b0 2023-10-05T19:40:16.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.751+0000 7f3af1ecde40 1 -- 172.21.15.12:0/748884875 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558583a1ef0 con 0x5558583a9a00 2023-10-05T19:40:16.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.755+0000 7f3aef201700 1 -- 172.21.15.12:0/748884875 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v28'13 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3ad800fd60 con 0x5558583a9a00 2023-10-05T19:40:16.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.755+0000 7f3af1ecde40 1 -- 172.21.15.12:0/748884875 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558583a1ef0 con 0x5558583ad1b0 2023-10-05T19:40:16.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.759+0000 7f3af0203700 1 -- 172.21.15.12:0/748884875 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v28'11 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3ae8014040 con 0x5558583ad1b0 2023-10-05T19:40:16.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.759+0000 7f3af1ecde40 1 -- 172.21.15.12:0/748884875 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558583a1ef0 con 0x5558583a9a00 2023-10-05T19:40:16.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.763+0000 7f3aef201700 1 -- 172.21.15.12:0/748884875 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v28'16 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3ad800fd60 con 0x5558583a9a00 2023-10-05T19:40:16.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.763+0000 7f3af1ecde40 1 -- 172.21.15.12:0/748884875 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558583a1ef0 con 0x5558583ad1b0 2023-10-05T19:40:16.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.767+0000 7f3af0203700 1 -- 172.21.15.12:0/748884875 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v28'19 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3ae8014040 con 0x5558583ad1b0 2023-10-05T19:40:16.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.767+0000 7f3af1ecde40 1 -- 172.21.15.12:0/748884875 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558583a1ef0 con 0x5558583a9a00 2023-10-05T19:40:16.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.767+0000 7f3aef201700 1 -- 172.21.15.12:0/748884875 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v28'22 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3ad800fd60 con 0x5558583a9a00 2023-10-05T19:40:16.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.767+0000 7f3af1ecde40 1 -- 172.21.15.12:0/748884875 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558583a1ef0 con 0x5558583b0fe0 2023-10-05T19:40:16.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.771+0000 7f3aefa02700 1 -- 172.21.15.12:0/748884875 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v28'17 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3ae0014070 con 0x5558583b0fe0 2023-10-05T19:40:16.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.775+0000 7f3a2ffff700 20 reqs_thread_entry: start 2023-10-05T19:40:16.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.775+0000 7f3a2dffb700 20 reqs_thread_entry: start 2023-10-05T19:40:16.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.775+0000 7f3af1ecde40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:16.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.775+0000 7f3af1ecde40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:16.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.775+0000 7f3a1bfff700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:46 2023 2023-10-05T19:40:16.785 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:16.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.775+0000 7f3a1b7fe700 20 reqs_thread_entry: start 2023-10-05T19:40:16.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.775+0000 7f3af1ecde40 20 get_system_obj_state: rctx=0x7ffc38f103b8 obj=default.rgw.meta:users.uid:bar state=0x5558582170b0 s->prefetch_data=0 2023-10-05T19:40:16.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.775+0000 7f3af1ecde40 10 cache get: name=default.rgw.meta+users.uid+bar : miss 2023-10-05T19:40:16.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.775+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:102 8.e 8:718f6a54:users.uid::bar:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555858414830 con 0x7f3ae0016730 2023-10-05T19:40:16.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.775+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(102 bar [call,getxattrs,stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x7f3ae8014040 con 0x7f3ae0016730 2023-10-05T19:40:16.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.779+0000 7f3af1ecde40 10 cache put: name=default.rgw.meta+users.uid+bar info.flags=0x0 2023-10-05T19:40:16.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.779+0000 7f3af1ecde40 10 adding default.rgw.meta+users.uid+bar to cache LRU end 2023-10-05T19:40:16.786 INFO:teuthology.orchestra.run.smithi012.stderr:could not remove user: unable to remove user, user does not exist 2023-10-05T19:40:16.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.779+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:103 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93837925558688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555858414830 con 0x555858378090 2023-10-05T19:40:16.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.779+0000 7f3aefa02700 1 -- 172.21.15.12:0/926293803 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(103 notify.0 [watch unwatch cookie 93837925558688] v28'11 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3ae0014070 con 0x555858378090 2023-10-05T19:40:16.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.779+0000 7f3af1ecde40 20 remove_watcher() i=0 2023-10-05T19:40:16.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.779+0000 7f3af1ecde40 2 removed watcher, disabling cache 2023-10-05T19:40:16.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.779+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:104 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93837925562208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555858414830 con 0x55585833e7e0 2023-10-05T19:40:16.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.783+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(104 notify.1 [watch unwatch cookie 93837925562208] v28'11 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3ae8014040 con 0x55585833e7e0 2023-10-05T19:40:16.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.783+0000 7f3af1ecde40 20 remove_watcher() i=1 2023-10-05T19:40:16.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.783+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:105 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93837925563312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555858414830 con 0x55585837c830 2023-10-05T19:40:16.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.787+0000 7f3aef201700 1 -- 172.21.15.12:0/926293803 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(105 notify.2 [watch unwatch cookie 93837925563312] v28'11 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3ad800fd60 con 0x55585837c830 2023-10-05T19:40:16.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.787+0000 7f3af1ecde40 20 remove_watcher() i=2 2023-10-05T19:40:16.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.787+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:106 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93837925570672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555858414830 con 0x555858378090 2023-10-05T19:40:16.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.791+0000 7f3aefa02700 1 -- 172.21.15.12:0/926293803 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(106 notify.3 [watch unwatch cookie 93837925570672] v28'11 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3ae0014070 con 0x555858378090 2023-10-05T19:40:16.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.791+0000 7f3af1ecde40 20 remove_watcher() i=3 2023-10-05T19:40:16.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.791+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:107 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93837925566368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555858414830 con 0x55585833e7e0 2023-10-05T19:40:16.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.791+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(107 notify.4 [watch unwatch cookie 93837925566368] v28'11 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3ae8014040 con 0x55585833e7e0 2023-10-05T19:40:16.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.791+0000 7f3af1ecde40 20 remove_watcher() i=4 2023-10-05T19:40:16.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.791+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:108 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93837925577264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555858414830 con 0x7f3ae0016730 2023-10-05T19:40:16.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.795+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(108 notify.5 [watch unwatch cookie 93837925577264] v28'11 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3ae8014040 con 0x7f3ae0016730 2023-10-05T19:40:16.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.795+0000 7f3af1ecde40 20 remove_watcher() i=5 2023-10-05T19:40:16.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.795+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:109 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93837925571776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555858414830 con 0x7f3ae0016730 2023-10-05T19:40:16.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.799+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(109 notify.6 [watch unwatch cookie 93837925571776] v28'11 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3ae8014040 con 0x7f3ae0016730 2023-10-05T19:40:16.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.799+0000 7f3af1ecde40 20 remove_watcher() i=6 2023-10-05T19:40:16.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.799+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:110 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93837925580256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555858414830 con 0x7f3ae0016730 2023-10-05T19:40:16.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.803+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.0 v2:172.21.15.8:6800/27563 30 ==== osd_op_reply(110 notify.7 [watch unwatch cookie 93837925580256] v28'11 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3ae8014040 con 0x7f3ae0016730 2023-10-05T19:40:16.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.803+0000 7f3af1ecde40 20 remove_watcher() i=7 2023-10-05T19:40:16.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.803+0000 7f3ac17e2700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:16.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.803+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:111 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93837925581360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555858414830 con 0x55585833e7e0 2023-10-05T19:40:16.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.807+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(111 data_loggenerations_metadata [watch unwatch cookie 93837925581360] v28'35 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f3ae8014040 con 0x55585833e7e0 2023-10-05T19:40:16.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.807+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f3ae0016730 msgr2=0x7f3ae0042260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:16.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.807+0000 7f3af1ecde40 1 --2- 172.21.15.12:0/926293803 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f3ae0016730 0x7f3ae0042260 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:16.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.807+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55585833e7e0 msgr2=0x555858375ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:16.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.807+0000 7f3af1ecde40 1 --2- 172.21.15.12:0/926293803 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55585833e7e0 0x555858375ba0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:16.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.807+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x555858378090 msgr2=0x55585837a470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:16.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.807+0000 7f3af1ecde40 1 --2- 172.21.15.12:0/926293803 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x555858378090 0x55585837a470 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:16.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.807+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55585837c830 msgr2=0x55585837ec40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:16.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.807+0000 7f3af1ecde40 1 --2- 172.21.15.12:0/926293803 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55585837c830 0x55585837ec40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:16.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.807+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3ad0030c50 msgr2=0x7f3ad0033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:16.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.807+0000 7f3af1ecde40 1 --2- 172.21.15.12:0/926293803 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3ad0030c50 0x7f3ad0033100 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8005d80 tx=0x7f3ad8014000 comp rx=0 tx=0).stop 2023-10-05T19:40:16.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.807+0000 7f3af0203700 1 -- 172.21.15.12:0/926293803 reap_dead start 2023-10-05T19:40:16.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.807+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55585833e410 msgr2=0x55585836a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:16.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.807+0000 7f3af1ecde40 1 --2- 172.21.15.12:0/926293803 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55585833e410 0x55585836a0e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3ae00017d0 tx=0x7f3ae0016000 comp rx=0 tx=0).stop 2023-10-05T19:40:16.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.807+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 shutdown_connections 2023-10-05T19:40:16.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.807+0000 7f3af1ecde40 1 --2- 172.21.15.12:0/926293803 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55585833e410 0x55585836a0e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:16.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.807+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 >> 172.21.15.12:0/926293803 conn(0x555858338720 msgr2=0x555858359560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:16.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.807+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 shutdown_connections 2023-10-05T19:40:16.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.807+0000 7f3af1ecde40 1 -- 172.21.15.12:0/926293803 wait complete. 2023-10-05T19:40:16.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.807+0000 7f3af1ecde40 1 -- 172.21.15.12:0/748884875 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x5558583a9a00 msgr2=0x5558583abde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:16.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.807+0000 7f3af1ecde40 1 --2- 172.21.15.12:0/748884875 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x5558583a9a00 0x5558583abde0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:16.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.807+0000 7f3af1ecde40 1 -- 172.21.15.12:0/748884875 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5558583a6330 msgr2=0x5558583a8710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:16.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.807+0000 7f3af1ecde40 1 --2- 172.21.15.12:0/748884875 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5558583a6330 0x5558583a8710 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:16.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.807+0000 7f3af1ecde40 1 -- 172.21.15.12:0/748884875 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5558583b0fe0 msgr2=0x5558583b33c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:16.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.807+0000 7f3af1ecde40 1 --2- 172.21.15.12:0/748884875 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5558583b0fe0 0x5558583b33c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:16.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.807+0000 7f3af1ecde40 1 -- 172.21.15.12:0/748884875 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5558583ad1b0 msgr2=0x5558583af5c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:16.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.807+0000 7f3af1ecde40 1 --2- 172.21.15.12:0/748884875 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5558583ad1b0 0x5558583af5c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:16.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.807+0000 7f3af1ecde40 1 -- 172.21.15.12:0/748884875 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3a30030c50 msgr2=0x7f3a30033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:16.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.807+0000 7f3af1ecde40 1 --2- 172.21.15.12:0/748884875 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3a30030c50 0x7f3a30033100 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x5558583862f0 tx=0x7f3ae800ab20 comp rx=0 tx=0).stop 2023-10-05T19:40:16.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.807+0000 7f3af1ecde40 1 -- 172.21.15.12:0/748884875 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555858395ca0 msgr2=0x55585839b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:16.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.807+0000 7f3af1ecde40 1 --2- 172.21.15.12:0/748884875 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555858395ca0 0x55585839b740 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3ad803c040 tx=0x7f3ad8042000 comp rx=0 tx=0).stop 2023-10-05T19:40:16.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.807+0000 7f3aefa02700 1 -- 172.21.15.12:0/748884875 reap_dead start 2023-10-05T19:40:16.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.807+0000 7f3aefa02700 1 -- 172.21.15.12:0/748884875 reap_dead start 2023-10-05T19:40:16.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.811+0000 7f3af1ecde40 1 -- 172.21.15.12:0/748884875 shutdown_connections 2023-10-05T19:40:16.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.811+0000 7f3af1ecde40 1 -- 172.21.15.12:0/748884875 >> 172.21.15.12:0/748884875 conn(0x555858391060 msgr2=0x55585838f100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:16.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.811+0000 7f3af1ecde40 1 -- 172.21.15.12:0/748884875 shutdown_connections 2023-10-05T19:40:16.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.811+0000 7f3af1ecde40 1 -- 172.21.15.12:0/748884875 wait complete. 2023-10-05T19:40:16.823 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-05T19:40:16.823 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'fud'] 2023-10-05T19:40:16.824 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'fud'] 2023-10-05T19:40:16.824 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid fud 2023-10-05T19:40:16.860 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:16.860 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:16.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.851+0000 7f0004016e40 1 Processor -- start 2023-10-05T19:40:16.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.851+0000 7f0004016e40 1 -- start start 2023-10-05T19:40:16.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.851+0000 7f0004016e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55bb2c8a9410 0x55bb2c8a7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:16.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.851+0000 7f0004016e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55bb2c7f5bd0 con 0x55bb2c8a97e0 2023-10-05T19:40:16.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.851+0000 7f0004016e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x55bb2c7b2ee0 con 0x55bb2c8a9410 2023-10-05T19:40:16.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.855+0000 7f0001b4b700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55bb2c8a9410 0x55bb2c8a7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:16.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.855+0000 7f000134a700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x55bb2c8a97e0 0x55bb2c8a8500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:47144/0 (socket says 172.21.15.12:47144) 2023-10-05T19:40:16.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.855+0000 7f000134a700 1 -- 172.21.15.12:0/784003538 learned_addr learned my addr 172.21.15.12:0/784003538 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:16.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.855+0000 7f0001b4b700 1 -- 172.21.15.12:0/784003538 >> v1:172.21.15.8:6789/0 conn(0x55bb2c8a97e0 legacy=0x55bb2c8a8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:16.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.855+0000 7f0001b4b700 1 -- 172.21.15.12:0/784003538 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb2c8a8cd0 con 0x55bb2c8a9410 2023-10-05T19:40:16.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.855+0000 7f0001b4b700 1 --2- 172.21.15.12:0/784003538 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb2c8a9410 0x55bb2c8a7fc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efff400a520 tx=0x7efff4005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f9505eba34f053 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:16.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.855+0000 7f0000b49700 1 -- 172.21.15.12:0/784003538 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efff400f590 con 0x55bb2c8a9410 2023-10-05T19:40:16.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.855+0000 7f0000b49700 1 -- 172.21.15.12:0/784003538 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff400f6f0 con 0x55bb2c8a9410 2023-10-05T19:40:16.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.855+0000 7f0000b49700 1 -- 172.21.15.12:0/784003538 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efff400f880 con 0x55bb2c8a9410 2023-10-05T19:40:16.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.855+0000 7f0004016e40 1 -- 172.21.15.12:0/784003538 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb2c8a9410 msgr2=0x55bb2c8a7fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:16.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.855+0000 7f0004016e40 1 --2- 172.21.15.12:0/784003538 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb2c8a9410 0x55bb2c8a7fc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efff400a520 tx=0x7efff4005d40 comp rx=0 tx=0).stop 2023-10-05T19:40:16.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.855+0000 7f0004016e40 1 -- 172.21.15.12:0/784003538 shutdown_connections 2023-10-05T19:40:16.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.855+0000 7f0004016e40 1 --2- 172.21.15.12:0/784003538 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb2c8a9410 0x55bb2c8a7fc0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:16.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.855+0000 7f0004016e40 1 -- 172.21.15.12:0/784003538 >> 172.21.15.12:0/784003538 conn(0x55bb2c8a3720 msgr2=0x55bb2c8a5b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:16.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.855+0000 7f0004016e40 1 -- 172.21.15.12:0/784003538 shutdown_connections 2023-10-05T19:40:16.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.855+0000 7f0004016e40 1 -- 172.21.15.12:0/784003538 wait complete. 2023-10-05T19:40:16.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.855+0000 7f0004016e40 1 Processor -- start 2023-10-05T19:40:16.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.855+0000 7f0004016e40 1 -- start start 2023-10-05T19:40:16.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.855+0000 7f0004016e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb2c8a9410 0x55bb2c8ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:16.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.855+0000 7f0004016e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb2c776e30 con 0x55bb2c8a9410 2023-10-05T19:40:16.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.859+0000 7f0001b4b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb2c8a9410 0x55bb2c8ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:16.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.859+0000 7f0001b4b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb2c8a9410 0x55bb2c8ca0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36390/0 (socket says 172.21.15.12:36390) 2023-10-05T19:40:16.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.859+0000 7f0001b4b700 1 -- 172.21.15.12:0/2527439345 learned_addr learned my addr 172.21.15.12:0/2527439345 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:16.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.859+0000 7f0001b4b700 1 -- 172.21.15.12:0/2527439345 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff400a0e0 con 0x55bb2c8a9410 2023-10-05T19:40:16.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.859+0000 7f0001b4b700 1 --2- 172.21.15.12:0/2527439345 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb2c8a9410 0x55bb2c8ca0b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efff4005d80 tx=0x7efff4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=503cef9060358a2c server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:16.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.859+0000 7efffb7fe700 1 -- 172.21.15.12:0/2527439345 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efff400f860 con 0x55bb2c8a9410 2023-10-05T19:40:16.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.859+0000 7efffb7fe700 1 -- 172.21.15.12:0/2527439345 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff400f9c0 con 0x55bb2c8a9410 2023-10-05T19:40:16.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.859+0000 7efffb7fe700 1 -- 172.21.15.12:0/2527439345 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efff400fb70 con 0x55bb2c8a9410 2023-10-05T19:40:16.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.859+0000 7f0004016e40 1 -- 172.21.15.12:0/2527439345 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb2c8a9410 msgr2=0x55bb2c8ca0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:16.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.859+0000 7f0004016e40 1 --2- 172.21.15.12:0/2527439345 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb2c8a9410 0x55bb2c8ca0b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efff4005d80 tx=0x7efff4014000 comp rx=0 tx=0).stop 2023-10-05T19:40:16.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.859+0000 7f0004016e40 1 -- 172.21.15.12:0/2527439345 shutdown_connections 2023-10-05T19:40:16.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.859+0000 7f0004016e40 1 --2- 172.21.15.12:0/2527439345 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb2c8a9410 0x55bb2c8ca0b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:16.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.859+0000 7f0004016e40 1 -- 172.21.15.12:0/2527439345 >> 172.21.15.12:0/2527439345 conn(0x55bb2c8a3720 msgr2=0x55bb2c8c5c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:16.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.859+0000 7f0004016e40 1 -- 172.21.15.12:0/2527439345 shutdown_connections 2023-10-05T19:40:16.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.859+0000 7f0004016e40 1 -- 172.21.15.12:0/2527439345 wait complete. 2023-10-05T19:40:16.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.859+0000 7f0004016e40 1 Processor -- start 2023-10-05T19:40:16.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.859+0000 7f0004016e40 1 -- start start 2023-10-05T19:40:16.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.859+0000 7f0004016e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb2c8a9410 0x55bb2c8d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:16.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.859+0000 7f0004016e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb2c749ed0 con 0x55bb2c8a9410 2023-10-05T19:40:16.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.859+0000 7f0001b4b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb2c8a9410 0x55bb2c8d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:16.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.859+0000 7f0001b4b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb2c8a9410 0x55bb2c8d0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36406/0 (socket says 172.21.15.12:36406) 2023-10-05T19:40:16.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.859+0000 7f0001b4b700 1 -- 172.21.15.12:0/363495163 learned_addr learned my addr 172.21.15.12:0/363495163 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:16.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.863+0000 7f0001b4b700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb2c8a8cd0 con 0x55bb2c8a9410 2023-10-05T19:40:16.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.863+0000 7f0001b4b700 1 --2- 172.21.15.12:0/363495163 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb2c8a9410 0x55bb2c8d0d10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efff400a520 tx=0x7efff401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:16.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.863+0000 7efff9ffb700 1 -- 172.21.15.12:0/363495163 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efff402f430 con 0x55bb2c8a9410 2023-10-05T19:40:16.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.863+0000 7efff9ffb700 1 -- 172.21.15.12:0/363495163 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff402f590 con 0x55bb2c8a9410 2023-10-05T19:40:16.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.863+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bb2c8d1250 con 0x55bb2c8a9410 2023-10-05T19:40:16.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.863+0000 7efff9ffb700 1 -- 172.21.15.12:0/363495163 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efff402f700 con 0x55bb2c8a9410 2023-10-05T19:40:16.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.863+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bb2c8d1690 con 0x55bb2c8a9410 2023-10-05T19:40:16.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.863+0000 7f0004016e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.863+0000 7efff9ffb700 1 -- 172.21.15.12:0/363495163 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7efff4038450 con 0x55bb2c8a9410 2023-10-05T19:40:16.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.863+0000 7efff9ffb700 1 --2- 172.21.15.12:0/363495163 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7effe4030c50 0x7effe4033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:16.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.863+0000 7efff9ffb700 1 -- 172.21.15.12:0/363495163 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7efff402b070 con 0x55bb2c8a9410 2023-10-05T19:40:16.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.867+0000 7f000134a700 1 --2- 172.21.15.12:0/363495163 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7effe4030c50 0x7effe4033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:16.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.867+0000 7f0004016e40 1 --2- 172.21.15.12:0/363495163 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55bb2c8a97e0 0x55bb2c8dc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:16.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.867+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8dce40 con 0x55bb2c8a97e0 2023-10-05T19:40:16.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.867+0000 7f000234c700 1 --2- 172.21.15.12:0/363495163 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55bb2c8a97e0 0x55bb2c8dc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:16.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.867+0000 7f000134a700 1 --2- 172.21.15.12:0/363495163 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7effe4030c50 0x7effe4033100 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7effec001e90 tx=0x7effec0013b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:16.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.867+0000 7f000234c700 1 --2- 172.21.15.12:0/363495163 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55bb2c8a97e0 0x55bb2c8dc7d0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:16.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.867+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7efffc00a500 con 0x55bb2c8a97e0 2023-10-05T19:40:16.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.867+0000 7f0004016e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:16.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.867+0000 7f0004016e40 20 realm 2023-10-05T19:40:16.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.867+0000 7f0004016e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.867+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8d9e30 con 0x55bb2c8a97e0 2023-10-05T19:40:16.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.867+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7efffc00a500 con 0x55bb2c8a97e0 2023-10-05T19:40:16.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.867+0000 7f0004016e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:16.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.867+0000 7f0004016e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:16.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.867+0000 7f0004016e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.867+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8dd920 con 0x55bb2c8a97e0 2023-10-05T19:40:16.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.867+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7efffc00a500 con 0x55bb2c8a97e0 2023-10-05T19:40:16.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.867+0000 7f0004016e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:16.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.867+0000 7f0004016e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.867+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8dd530 con 0x55bb2c8a97e0 2023-10-05T19:40:16.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.867+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7efffc00a500 con 0x55bb2c8a97e0 2023-10-05T19:40:16.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.867+0000 7f0004016e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:16.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.867+0000 7f0004016e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.871+0000 7f0004016e40 1 --2- 172.21.15.12:0/363495163 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55bb2c8decc0 0x55bb2c8e10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:16.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.871+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e15e0 con 0x55bb2c8decc0 2023-10-05T19:40:16.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.871+0000 7f0001b4b700 1 --2- 172.21.15.12:0/363495163 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55bb2c8decc0 0x55bb2c8e10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:16.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.871+0000 7f0001b4b700 1 --2- 172.21.15.12:0/363495163 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55bb2c8decc0 0x55bb2c8e10a0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:16.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.871+0000 7f0001b4b700 1 -- 172.21.15.12:0/363495163 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv1 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7efff4035070 con 0x55bb2c8decc0 2023-10-05T19:40:16.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.871+0000 7f0004016e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:16.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.871+0000 7f0004016e40 20 searching for the correct realm 2023-10-05T19:40:16.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.871+0000 7f0004016e40 1 --2- 172.21.15.12:0/363495163 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55bb2c8e3490 0x55bb2c8e58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:16.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.871+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e5f50 con 0x55bb2c8e3490 2023-10-05T19:40:16.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.871+0000 7f000134a700 1 --2- 172.21.15.12:0/363495163 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55bb2c8e3490 0x55bb2c8e58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:16.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.871+0000 7f000134a700 1 --2- 172.21.15.12:0/363495163 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55bb2c8e3490 0x55bb2c8e58a0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:16.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.875+0000 7f000134a700 1 -- 172.21.15.12:0/363495163 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7effec006070 con 0x55bb2c8e3490 2023-10-05T19:40:16.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.875+0000 7f000134a700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e5f50 con 0x55bb2c8decc0 2023-10-05T19:40:16.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.875+0000 7f0001b4b700 1 -- 172.21.15.12:0/363495163 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efff4035070 con 0x55bb2c8decc0 2023-10-05T19:40:16.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.875+0000 7f0001b4b700 1 --2- 172.21.15.12:0/363495163 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7efff401bbe0 0x7efff40079b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:16.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.875+0000 7f0001b4b700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e5f50 con 0x7efff401bbe0 2023-10-05T19:40:16.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.875+0000 7f000234c700 1 --2- 172.21.15.12:0/363495163 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7efff401bbe0 0x7efff40079b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:16.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.875+0000 7f000234c700 1 --2- 172.21.15.12:0/363495163 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7efff401bbe0 0x7efff40079b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:16.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.879+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efffc014040 con 0x7efff401bbe0 2023-10-05T19:40:16.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.879+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e5f50 con 0x55bb2c8a97e0 2023-10-05T19:40:16.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.879+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efffc014040 con 0x55bb2c8a97e0 2023-10-05T19:40:16.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.879+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e5f50 con 0x55bb2c8a97e0 2023-10-05T19:40:16.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.879+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efffc014040 con 0x55bb2c8a97e0 2023-10-05T19:40:16.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.879+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e5f50 con 0x7efff401bbe0 2023-10-05T19:40:16.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.879+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efffc014040 con 0x7efff401bbe0 2023-10-05T19:40:16.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.879+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e5f50 con 0x7efff401bbe0 2023-10-05T19:40:16.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.879+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efffc014040 con 0x7efff401bbe0 2023-10-05T19:40:16.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.879+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e5f50 con 0x55bb2c8decc0 2023-10-05T19:40:16.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.879+0000 7f0001b4b700 1 -- 172.21.15.12:0/363495163 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efff4035070 con 0x55bb2c8decc0 2023-10-05T19:40:16.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.883+0000 7f0001b4b700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e5f50 con 0x55bb2c8e3490 2023-10-05T19:40:16.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.883+0000 7f000134a700 1 -- 172.21.15.12:0/363495163 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7effec006070 con 0x55bb2c8e3490 2023-10-05T19:40:16.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.883+0000 7f000134a700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e5f50 con 0x55bb2c8a97e0 2023-10-05T19:40:16.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.883+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efffc014040 con 0x55bb2c8a97e0 2023-10-05T19:40:16.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.883+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e5f50 con 0x7efff401bbe0 2023-10-05T19:40:16.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.883+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efffc014040 con 0x7efff401bbe0 2023-10-05T19:40:16.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.883+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e5f50 con 0x55bb2c8a97e0 2023-10-05T19:40:16.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.883+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efffc014040 con 0x55bb2c8a97e0 2023-10-05T19:40:16.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.883+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e5f50 con 0x55bb2c8decc0 2023-10-05T19:40:16.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.883+0000 7f0001b4b700 1 -- 172.21.15.12:0/363495163 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efff4035070 con 0x55bb2c8decc0 2023-10-05T19:40:16.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.883+0000 7f0001b4b700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e5f50 con 0x55bb2c8e3490 2023-10-05T19:40:16.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.887+0000 7f000134a700 1 -- 172.21.15.12:0/363495163 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7effec006070 con 0x55bb2c8e3490 2023-10-05T19:40:16.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.887+0000 7f000134a700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e5f50 con 0x55bb2c8e3490 2023-10-05T19:40:16.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.887+0000 7f000134a700 1 -- 172.21.15.12:0/363495163 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7effec006070 con 0x55bb2c8e3490 2023-10-05T19:40:16.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.887+0000 7f000134a700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e5f50 con 0x7efff401bbe0 2023-10-05T19:40:16.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.887+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efffc014040 con 0x7efff401bbe0 2023-10-05T19:40:16.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.887+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e5f50 con 0x55bb2c8e3490 2023-10-05T19:40:16.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.887+0000 7f000134a700 1 -- 172.21.15.12:0/363495163 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7effec006070 con 0x55bb2c8e3490 2023-10-05T19:40:16.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.887+0000 7f000134a700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e5f50 con 0x55bb2c8e3490 2023-10-05T19:40:16.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.891+0000 7f000134a700 1 -- 172.21.15.12:0/363495163 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7effec006070 con 0x55bb2c8e3490 2023-10-05T19:40:16.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.891+0000 7f000134a700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e5f50 con 0x55bb2c8a97e0 2023-10-05T19:40:16.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.891+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efffc014040 con 0x55bb2c8a97e0 2023-10-05T19:40:16.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.891+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e5f50 con 0x55bb2c8a97e0 2023-10-05T19:40:16.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.891+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efffc014040 con 0x55bb2c8a97e0 2023-10-05T19:40:16.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.891+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e5f50 con 0x7efff401bbe0 2023-10-05T19:40:16.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.891+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efffc014040 con 0x7efff401bbe0 2023-10-05T19:40:16.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.891+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e5f50 con 0x55bb2c8e3490 2023-10-05T19:40:16.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.891+0000 7f000134a700 1 -- 172.21.15.12:0/363495163 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7effec006070 con 0x55bb2c8e3490 2023-10-05T19:40:16.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.891+0000 7f000134a700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e5f50 con 0x55bb2c8decc0 2023-10-05T19:40:16.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.895+0000 7f0001b4b700 1 -- 172.21.15.12:0/363495163 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efff4035070 con 0x55bb2c8decc0 2023-10-05T19:40:16.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.895+0000 7f0001b4b700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e5f50 con 0x55bb2c8a97e0 2023-10-05T19:40:16.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.895+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efffc014040 con 0x55bb2c8a97e0 2023-10-05T19:40:16.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.895+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e5f50 con 0x7efff401bbe0 2023-10-05T19:40:16.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.895+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efffc014040 con 0x7efff401bbe0 2023-10-05T19:40:16.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.895+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e5f50 con 0x55bb2c8e3490 2023-10-05T19:40:16.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.895+0000 7f000134a700 1 -- 172.21.15.12:0/363495163 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7effec006070 con 0x55bb2c8e3490 2023-10-05T19:40:16.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.895+0000 7f000134a700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e5f50 con 0x7efff401bbe0 2023-10-05T19:40:16.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.899+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efffc014040 con 0x7efff401bbe0 2023-10-05T19:40:16.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.899+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e5f50 con 0x55bb2c8decc0 2023-10-05T19:40:16.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.899+0000 7f0001b4b700 1 -- 172.21.15.12:0/363495163 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efff4035070 con 0x55bb2c8decc0 2023-10-05T19:40:16.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.899+0000 7f0001b4b700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e5f50 con 0x55bb2c8e3490 2023-10-05T19:40:16.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.899+0000 7f000134a700 1 -- 172.21.15.12:0/363495163 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7effec006070 con 0x55bb2c8e3490 2023-10-05T19:40:16.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.899+0000 7f000134a700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e5f50 con 0x7efff401bbe0 2023-10-05T19:40:16.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.899+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efffc014040 con 0x7efff401bbe0 2023-10-05T19:40:16.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.899+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e5f50 con 0x55bb2c8a97e0 2023-10-05T19:40:16.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.899+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7efffc014040 con 0x55bb2c8a97e0 2023-10-05T19:40:16.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.903+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e5f50 con 0x55bb2c8decc0 2023-10-05T19:40:16.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.903+0000 7f0001b4b700 1 -- 172.21.15.12:0/363495163 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7efff4035070 con 0x55bb2c8decc0 2023-10-05T19:40:16.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.903+0000 7f0004016e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:16.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.903+0000 7f0004016e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:16.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.903+0000 7f0004016e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:16.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.903+0000 7f0004016e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:16.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.903+0000 7f0004016e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.903+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e2210 con 0x55bb2c8a97e0 2023-10-05T19:40:16.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.903+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7efffc014040 con 0x55bb2c8a97e0 2023-10-05T19:40:16.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.903+0000 7f0004016e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:16.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.903+0000 7f0004016e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.903+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e1e40 con 0x55bb2c8a97e0 2023-10-05T19:40:16.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.903+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7efffc014040 con 0x55bb2c8a97e0 2023-10-05T19:40:16.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.903+0000 7f0004016e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:16.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.903+0000 7f0004016e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.903+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e63c0 con 0x55bb2c8e3490 2023-10-05T19:40:16.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.903+0000 7f000134a700 1 -- 172.21.15.12:0/363495163 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7effec006070 con 0x55bb2c8e3490 2023-10-05T19:40:16.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.903+0000 7f0004016e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:16.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.903+0000 7f0004016e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.903+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e6770 con 0x55bb2c8e3490 2023-10-05T19:40:16.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.903+0000 7f000134a700 1 -- 172.21.15.12:0/363495163 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7effec006070 con 0x55bb2c8e3490 2023-10-05T19:40:16.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.903+0000 7f0004016e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:16.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.903+0000 7f0004016e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.903+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e6b20 con 0x55bb2c8a97e0 2023-10-05T19:40:16.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.907+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7efffc014040 con 0x55bb2c8a97e0 2023-10-05T19:40:16.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.907+0000 7f0004016e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:16.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.907+0000 7f0004016e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.907+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e2700 con 0x55bb2c8a97e0 2023-10-05T19:40:16.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.907+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7efffc014040 con 0x55bb2c8a97e0 2023-10-05T19:40:16.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.907+0000 7f0004016e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:16.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.907+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e26d0 con 0x55bb2c8e3490 2023-10-05T19:40:16.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.911+0000 7f000134a700 1 -- 172.21.15.12:0/363495163 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7effec006070 con 0x55bb2c8e3490 2023-10-05T19:40:16.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.911+0000 7f000134a700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e26d0 con 0x55bb2c8decc0 2023-10-05T19:40:16.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.911+0000 7f0001b4b700 1 -- 172.21.15.12:0/363495163 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efff4035070 con 0x55bb2c8decc0 2023-10-05T19:40:16.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.911+0000 7f0001b4b700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e26d0 con 0x7efff401bbe0 2023-10-05T19:40:16.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.911+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efffc014040 con 0x7efff401bbe0 2023-10-05T19:40:16.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.911+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e6b20 con 0x55bb2c8a97e0 2023-10-05T19:40:16.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.911+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efffc014040 con 0x55bb2c8a97e0 2023-10-05T19:40:16.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.911+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e6b20 con 0x55bb2c8a97e0 2023-10-05T19:40:16.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.911+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efffc014040 con 0x55bb2c8a97e0 2023-10-05T19:40:16.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.911+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e6b20 con 0x7efff401bbe0 2023-10-05T19:40:16.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.915+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efffc014040 con 0x7efff401bbe0 2023-10-05T19:40:16.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.915+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e6b20 con 0x7efff401bbe0 2023-10-05T19:40:16.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.915+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efffc014040 con 0x7efff401bbe0 2023-10-05T19:40:16.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.915+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e6b20 con 0x55bb2c8decc0 2023-10-05T19:40:16.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.915+0000 7f0001b4b700 1 -- 172.21.15.12:0/363495163 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efff4035070 con 0x55bb2c8decc0 2023-10-05T19:40:16.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.915+0000 7f0001b4b700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e6b20 con 0x55bb2c8e3490 2023-10-05T19:40:16.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.915+0000 7f000134a700 1 -- 172.21.15.12:0/363495163 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7effec006070 con 0x55bb2c8e3490 2023-10-05T19:40:16.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.915+0000 7f000134a700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e6b20 con 0x55bb2c8a97e0 2023-10-05T19:40:16.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.915+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efffc014040 con 0x55bb2c8a97e0 2023-10-05T19:40:16.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.919+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e6b20 con 0x7efff401bbe0 2023-10-05T19:40:16.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.919+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efffc014040 con 0x7efff401bbe0 2023-10-05T19:40:16.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.919+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e6b20 con 0x55bb2c8a97e0 2023-10-05T19:40:16.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.919+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efffc014040 con 0x55bb2c8a97e0 2023-10-05T19:40:16.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.919+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e6b20 con 0x55bb2c8decc0 2023-10-05T19:40:16.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.919+0000 7f0001b4b700 1 -- 172.21.15.12:0/363495163 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efff4035070 con 0x55bb2c8decc0 2023-10-05T19:40:16.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.919+0000 7f0001b4b700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e6b20 con 0x55bb2c8e3490 2023-10-05T19:40:16.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.919+0000 7f000134a700 1 -- 172.21.15.12:0/363495163 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7effec006070 con 0x55bb2c8e3490 2023-10-05T19:40:16.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.919+0000 7f000134a700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e6b20 con 0x55bb2c8e3490 2023-10-05T19:40:16.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.923+0000 7f000134a700 1 -- 172.21.15.12:0/363495163 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7effec006070 con 0x55bb2c8e3490 2023-10-05T19:40:16.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.923+0000 7f000134a700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e6b20 con 0x7efff401bbe0 2023-10-05T19:40:16.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.923+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efffc014040 con 0x7efff401bbe0 2023-10-05T19:40:16.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.923+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e6b20 con 0x55bb2c8e3490 2023-10-05T19:40:16.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.923+0000 7f000134a700 1 -- 172.21.15.12:0/363495163 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7effec006070 con 0x55bb2c8e3490 2023-10-05T19:40:16.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.923+0000 7f000134a700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e6b20 con 0x55bb2c8e3490 2023-10-05T19:40:16.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.923+0000 7f000134a700 1 -- 172.21.15.12:0/363495163 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7effec006070 con 0x55bb2c8e3490 2023-10-05T19:40:16.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.923+0000 7f000134a700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e6b20 con 0x55bb2c8a97e0 2023-10-05T19:40:16.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.923+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efffc014040 con 0x55bb2c8a97e0 2023-10-05T19:40:16.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.923+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e6b20 con 0x55bb2c8a97e0 2023-10-05T19:40:16.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.927+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efffc014040 con 0x55bb2c8a97e0 2023-10-05T19:40:16.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.927+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e6b20 con 0x7efff401bbe0 2023-10-05T19:40:16.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.927+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efffc014040 con 0x7efff401bbe0 2023-10-05T19:40:16.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.927+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e6b20 con 0x55bb2c8e3490 2023-10-05T19:40:16.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.927+0000 7f000134a700 1 -- 172.21.15.12:0/363495163 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7effec006070 con 0x55bb2c8e3490 2023-10-05T19:40:16.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.927+0000 7f000134a700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e6b20 con 0x55bb2c8decc0 2023-10-05T19:40:16.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.927+0000 7f0001b4b700 1 -- 172.21.15.12:0/363495163 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efff4035070 con 0x55bb2c8decc0 2023-10-05T19:40:16.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.927+0000 7f0001b4b700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e6b20 con 0x55bb2c8a97e0 2023-10-05T19:40:16.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.927+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efffc014040 con 0x55bb2c8a97e0 2023-10-05T19:40:16.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.927+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e6b20 con 0x7efff401bbe0 2023-10-05T19:40:16.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.931+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efffc014040 con 0x7efff401bbe0 2023-10-05T19:40:16.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.931+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e6b20 con 0x55bb2c8e3490 2023-10-05T19:40:16.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.931+0000 7f000134a700 1 -- 172.21.15.12:0/363495163 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7effec006070 con 0x55bb2c8e3490 2023-10-05T19:40:16.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.931+0000 7f000134a700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e6b20 con 0x7efff401bbe0 2023-10-05T19:40:16.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.931+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efffc014040 con 0x7efff401bbe0 2023-10-05T19:40:16.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.931+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e6b20 con 0x55bb2c8decc0 2023-10-05T19:40:16.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.931+0000 7f0001b4b700 1 -- 172.21.15.12:0/363495163 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efff4035070 con 0x55bb2c8decc0 2023-10-05T19:40:16.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.931+0000 7f0001b4b700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e6b20 con 0x55bb2c8e3490 2023-10-05T19:40:16.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.935+0000 7f000134a700 1 -- 172.21.15.12:0/363495163 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7effec006070 con 0x55bb2c8e3490 2023-10-05T19:40:16.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.935+0000 7f000134a700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e6b20 con 0x7efff401bbe0 2023-10-05T19:40:16.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.935+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efffc014040 con 0x7efff401bbe0 2023-10-05T19:40:16.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.935+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e6b20 con 0x55bb2c8a97e0 2023-10-05T19:40:16.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.935+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7efffc014040 con 0x55bb2c8a97e0 2023-10-05T19:40:16.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.935+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e6b20 con 0x55bb2c8decc0 2023-10-05T19:40:16.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.935+0000 7f0001b4b700 1 -- 172.21.15.12:0/363495163 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7efff4035070 con 0x55bb2c8decc0 2023-10-05T19:40:16.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.935+0000 7f0004016e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:16.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.935+0000 7f0004016e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:16.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.935+0000 7f0004016e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:16.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.935+0000 7f0004016e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:16.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.935+0000 7f0004016e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.935+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e26d0 con 0x55bb2c8a97e0 2023-10-05T19:40:16.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.939+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7efffc014040 con 0x55bb2c8a97e0 2023-10-05T19:40:16.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.939+0000 7f0004016e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:16.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.939+0000 7f0004016e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.939+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e7460 con 0x55bb2c8decc0 2023-10-05T19:40:16.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.939+0000 7f0001b4b700 1 -- 172.21.15.12:0/363495163 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv1 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7efff4035070 con 0x55bb2c8decc0 2023-10-05T19:40:16.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.939+0000 7f0004016e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:16.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.939+0000 7f0004016e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.939+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e7d00 con 0x55bb2c8a97e0 2023-10-05T19:40:16.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.939+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7efffc014040 con 0x55bb2c8a97e0 2023-10-05T19:40:16.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.939+0000 7f0004016e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:16.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.939+0000 7f0004016e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.939+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e7d00 con 0x55bb2c8e3490 2023-10-05T19:40:16.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.939+0000 7f000134a700 1 -- 172.21.15.12:0/363495163 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7effec006070 con 0x55bb2c8e3490 2023-10-05T19:40:16.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.939+0000 7f0004016e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:16.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.939+0000 7f0004016e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.939+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e7910 con 0x7efff401bbe0 2023-10-05T19:40:16.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.943+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7efffc014040 con 0x7efff401bbe0 2023-10-05T19:40:16.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.943+0000 7f0004016e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:16.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.943+0000 7f0004016e40 20 zone default found 2023-10-05T19:40:16.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.943+0000 7f0004016e40 4 Realm: () 2023-10-05T19:40:16.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.943+0000 7f0004016e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:16.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.943+0000 7f0004016e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:16.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.943+0000 7f0004016e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:16.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.943+0000 7f0004016e40 20 zonegroup default 2023-10-05T19:40:16.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.943+0000 7f0004016e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.943+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e7910 con 0x7efff401bbe0 2023-10-05T19:40:16.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.943+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7efffc014040 con 0x7efff401bbe0 2023-10-05T19:40:16.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.943+0000 7f0004016e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:16.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.943+0000 7f0004016e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.943+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e7910 con 0x55bb2c8e3490 2023-10-05T19:40:16.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.943+0000 7f000134a700 1 -- 172.21.15.12:0/363495163 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7effec006070 con 0x55bb2c8e3490 2023-10-05T19:40:16.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.943+0000 7f0004016e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:16.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.943+0000 7f0004016e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:16.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.943+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e9f20 con 0x55bb2c8e3490 2023-10-05T19:40:16.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.943+0000 7f000134a700 1 -- 172.21.15.12:0/363495163 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7effec006070 con 0x55bb2c8e3490 2023-10-05T19:40:16.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.943+0000 7f0004016e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:16.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.943+0000 7f0004016e40 20 started sync module instance, tier type = 2023-10-05T19:40:16.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.943+0000 7f0004016e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:16.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.943+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e95e0 con 0x55bb2c8decc0 2023-10-05T19:40:16.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.947+0000 7f0001b4b700 1 -- 172.21.15.12:0/363495163 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'12 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efff4035070 con 0x55bb2c8decc0 2023-10-05T19:40:16.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.947+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94262394788240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8e9210 con 0x55bb2c8decc0 2023-10-05T19:40:16.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.947+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8eacb0 con 0x55bb2c8a97e0 2023-10-05T19:40:16.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.951+0000 7f0001b4b700 1 -- 172.21.15.12:0/363495163 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94262394788240] v28'13 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efff4035070 con 0x55bb2c8decc0 2023-10-05T19:40:16.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.951+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'12 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efffc014040 con 0x55bb2c8a97e0 2023-10-05T19:40:16.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.951+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94262394790896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8ead30 con 0x55bb2c8a97e0 2023-10-05T19:40:16.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.951+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8eb8c0 con 0x55bb2c8e3490 2023-10-05T19:40:16.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.955+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94262394790896] v28'13 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efffc014040 con 0x55bb2c8a97e0 2023-10-05T19:40:16.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.955+0000 7f000134a700 1 -- 172.21.15.12:0/363495163 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'12 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7effec006070 con 0x55bb2c8e3490 2023-10-05T19:40:16.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.955+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94262394792000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8eac90 con 0x55bb2c8e3490 2023-10-05T19:40:16.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.955+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8ec6a0 con 0x55bb2c8decc0 2023-10-05T19:40:16.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.959+0000 7f0001b4b700 1 -- 172.21.15.12:0/363495163 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'12 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efff4035070 con 0x55bb2c8decc0 2023-10-05T19:40:16.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.959+0000 7f000134a700 1 -- 172.21.15.12:0/363495163 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94262394792000] v28'13 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7effec006070 con 0x55bb2c8e3490 2023-10-05T19:40:16.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.959+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94262394798800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8ecc50 con 0x55bb2c8decc0 2023-10-05T19:40:16.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.959+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8ed630 con 0x55bb2c8a97e0 2023-10-05T19:40:16.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.959+0000 7f0001b4b700 1 -- 172.21.15.12:0/363495163 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94262394798800] v28'13 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efff4035070 con 0x55bb2c8decc0 2023-10-05T19:40:16.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.959+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'12 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efffc014040 con 0x55bb2c8a97e0 2023-10-05T19:40:16.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.959+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94262394799904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8ecb70 con 0x55bb2c8a97e0 2023-10-05T19:40:16.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.959+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8edc90 con 0x7efff401bbe0 2023-10-05T19:40:16.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.963+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'12 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efffc014040 con 0x7efff401bbe0 2023-10-05T19:40:16.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.963+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94262394804128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8ee0e0 con 0x7efff401bbe0 2023-10-05T19:40:16.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.963+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8eeb60 con 0x7efff401bbe0 2023-10-05T19:40:16.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.963+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94262394799904] v28'13 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efffc014040 con 0x55bb2c8a97e0 2023-10-05T19:40:16.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.967+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(95 notify.5 [watch watch cookie 94262394804128] v28'13 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efffc014040 con 0x7efff401bbe0 2023-10-05T19:40:16.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.967+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(96 notify.6 [create] v28'12 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efffc014040 con 0x7efff401bbe0 2023-10-05T19:40:16.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.967+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94262394802496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8edbf0 con 0x7efff401bbe0 2023-10-05T19:40:16.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.967+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8ef280 con 0x7efff401bbe0 2023-10-05T19:40:16.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.967+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(98 notify.7 [create] v28'12 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efffc014040 con 0x7efff401bbe0 2023-10-05T19:40:16.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.967+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94262394809744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8ef6d0 con 0x7efff401bbe0 2023-10-05T19:40:16.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.967+0000 7f0004016e40 20 add_watcher() i=0 2023-10-05T19:40:16.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.967+0000 7f0004016e40 20 add_watcher() i=1 2023-10-05T19:40:16.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.967+0000 7f0004016e40 20 add_watcher() i=2 2023-10-05T19:40:16.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.967+0000 7f0004016e40 20 add_watcher() i=3 2023-10-05T19:40:16.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.967+0000 7f0004016e40 20 add_watcher() i=4 2023-10-05T19:40:16.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.967+0000 7f0004016e40 20 add_watcher() i=5 2023-10-05T19:40:16.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.967+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(97 notify.6 [watch watch cookie 94262394802496] v28'13 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efffc014040 con 0x7efff401bbe0 2023-10-05T19:40:16.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.971+0000 7f0004016e40 20 add_watcher() i=6 2023-10-05T19:40:16.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.971+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94262394809744] v28'13 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efffc014040 con 0x7efff401bbe0 2023-10-05T19:40:16.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.971+0000 7f0004016e40 20 add_watcher() i=7 2023-10-05T19:40:16.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.971+0000 7f0004016e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:16.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.971+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8f2210 con 0x55bb2c8a97e0 2023-10-05T19:40:16.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.971+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7efffc014040 con 0x55bb2c8a97e0 2023-10-05T19:40:16.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.971+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94262394807856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8ef710 con 0x55bb2c8a97e0 2023-10-05T19:40:16.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.975+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94262394807856] v28'36 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7efffc014040 con 0x55bb2c8a97e0 2023-10-05T19:40:16.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.975+0000 7effd2fe5700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:16.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.975+0000 7f0004016e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:16.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.975+0000 7f0004016e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:16.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.975+0000 7f0004016e40 1 Processor -- start 2023-10-05T19:40:16.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.975+0000 7f0004016e40 1 -- start start 2023-10-05T19:40:16.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.975+0000 7f0004016e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb2c8fc9c0 0x55bb2c8fcd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:16.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.975+0000 7f0004016e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb2c8fd2d0 con 0x55bb2c8fc9c0 2023-10-05T19:40:16.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.975+0000 7f000134a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb2c8fc9c0 0x55bb2c8fcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:16.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.975+0000 7f000134a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb2c8fc9c0 0x55bb2c8fcd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36414/0 (socket says 172.21.15.12:36414) 2023-10-05T19:40:16.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.975+0000 7f000134a700 1 -- 172.21.15.12:0/2574613642 learned_addr learned my addr 172.21.15.12:0/2574613642 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:16.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.975+0000 7f000134a700 1 -- 172.21.15.12:0/2574613642 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb2c8fd410 con 0x55bb2c8fc9c0 2023-10-05T19:40:16.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.975+0000 7f000134a700 1 --2- 172.21.15.12:0/2574613642 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb2c8fc9c0 0x55bb2c8fcd90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x55bb2c8da260 tx=0x7effec01e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b21ebc053731ba2 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:16.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.979+0000 7effd27e4700 1 -- 172.21.15.12:0/2574613642 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effec013040 con 0x55bb2c8fc9c0 2023-10-05T19:40:16.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.979+0000 7effd27e4700 1 -- 172.21.15.12:0/2574613642 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effec022040 con 0x55bb2c8fc9c0 2023-10-05T19:40:16.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.979+0000 7effd27e4700 1 -- 172.21.15.12:0/2574613642 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effec00ee90 con 0x55bb2c8fc9c0 2023-10-05T19:40:16.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.979+0000 7f0004016e40 1 -- 172.21.15.12:0/2574613642 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb2c8fc9c0 msgr2=0x55bb2c8fcd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:16.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.979+0000 7f0004016e40 1 --2- 172.21.15.12:0/2574613642 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb2c8fc9c0 0x55bb2c8fcd90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x55bb2c8da260 tx=0x7effec01e040 comp rx=0 tx=0).stop 2023-10-05T19:40:16.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.979+0000 7f0004016e40 1 -- 172.21.15.12:0/2574613642 shutdown_connections 2023-10-05T19:40:16.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.979+0000 7f0004016e40 1 --2- 172.21.15.12:0/2574613642 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb2c8fc9c0 0x55bb2c8fcd90 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:16.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.979+0000 7f0004016e40 1 -- 172.21.15.12:0/2574613642 >> 172.21.15.12:0/2574613642 conn(0x55bb2c8f7d80 msgr2=0x55bb2c8f5ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:16.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.979+0000 7f0004016e40 1 -- 172.21.15.12:0/2574613642 shutdown_connections 2023-10-05T19:40:16.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.979+0000 7f0004016e40 1 -- 172.21.15.12:0/2574613642 wait complete. 2023-10-05T19:40:16.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.979+0000 7f0004016e40 1 Processor -- start 2023-10-05T19:40:16.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.979+0000 7f0004016e40 1 -- start start 2023-10-05T19:40:16.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.979+0000 7f0004016e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb2c8fc9c0 0x55bb2c902510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:16.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.979+0000 7f0004016e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb2c903510 con 0x55bb2c8fc9c0 2023-10-05T19:40:16.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.979+0000 7f000134a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb2c8fc9c0 0x55bb2c902510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:16.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.979+0000 7f000134a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb2c8fc9c0 0x55bb2c902510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36418/0 (socket says 172.21.15.12:36418) 2023-10-05T19:40:16.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.979+0000 7f000134a700 1 -- 172.21.15.12:0/267042342 learned_addr learned my addr 172.21.15.12:0/267042342 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:16.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.979+0000 7f000134a700 1 -- 172.21.15.12:0/267042342 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effec034040 con 0x55bb2c8fc9c0 2023-10-05T19:40:16.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.979+0000 7f000134a700 1 --2- 172.21.15.12:0/267042342 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb2c8fc9c0 0x55bb2c902510 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x55bb2c8f6a40 tx=0x7effec03d030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:16.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.983+0000 7effd0fe1700 1 -- 172.21.15.12:0/267042342 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effec02c640 con 0x55bb2c8fc9c0 2023-10-05T19:40:16.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.983+0000 7effd0fe1700 1 -- 172.21.15.12:0/267042342 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effec02c7a0 con 0x55bb2c8fc9c0 2023-10-05T19:40:16.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.983+0000 7f0004016e40 1 -- 172.21.15.12:0/267042342 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bb2c902b10 con 0x55bb2c8fc9c0 2023-10-05T19:40:16.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.983+0000 7effd0fe1700 1 -- 172.21.15.12:0/267042342 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effec02c910 con 0x55bb2c8fc9c0 2023-10-05T19:40:16.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.983+0000 7f0004016e40 1 -- 172.21.15.12:0/267042342 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bb2c902f50 con 0x55bb2c8fc9c0 2023-10-05T19:40:16.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.983+0000 7effd0fe1700 1 -- 172.21.15.12:0/267042342 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7effec02ca70 con 0x55bb2c8fc9c0 2023-10-05T19:40:16.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.983+0000 7effd0fe1700 1 --2- 172.21.15.12:0/267042342 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7eff48030c00 0x7eff480330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:16.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.983+0000 7effd0fe1700 1 -- 172.21.15.12:0/267042342 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7effec031140 con 0x55bb2c8fc9c0 2023-10-05T19:40:16.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.983+0000 7f000234c700 1 --2- 172.21.15.12:0/267042342 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7eff48030c00 0x7eff480330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:16.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.983+0000 7f0004016e40 1 --2- 172.21.15.12:0/267042342 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55bb2c90d0d0 0x55bb2c90f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:16.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.983+0000 7f0004016e40 1 -- 172.21.15.12:0/267042342 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c90fb30 con 0x55bb2c90d0d0 2023-10-05T19:40:16.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.983+0000 7f0001b4b700 1 --2- 172.21.15.12:0/267042342 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55bb2c90d0d0 0x55bb2c90f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:16.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.983+0000 7f000234c700 1 --2- 172.21.15.12:0/267042342 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7eff48030c00 0x7eff480330b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x55bb2c8edca0 tx=0x7efffc00a9d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:16.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.983+0000 7f0001b4b700 1 --2- 172.21.15.12:0/267042342 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55bb2c90d0d0 0x55bb2c90f4e0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:16.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.987+0000 7f0001b4b700 1 -- 172.21.15.12:0/267042342 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v28'18 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7efff4035070 con 0x55bb2c90d0d0 2023-10-05T19:40:16.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.987+0000 7f0004016e40 1 --2- 172.21.15.12:0/267042342 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55bb2c910890 0x55bb2c912ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:16.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.987+0000 7f0004016e40 1 -- 172.21.15.12:0/267042342 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c913330 con 0x55bb2c910890 2023-10-05T19:40:16.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.987+0000 7f000134a700 1 --2- 172.21.15.12:0/267042342 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55bb2c910890 0x55bb2c912ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:16.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.991+0000 7f000134a700 1 --2- 172.21.15.12:0/267042342 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55bb2c910890 0x55bb2c912ca0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.991+0000 7f000134a700 1 -- 172.21.15.12:0/267042342 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v28'12 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7effec054020 con 0x55bb2c910890 2023-10-05T19:40:17.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.991+0000 7f0004016e40 1 --2- 172.21.15.12:0/267042342 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55bb2c914000 0x55bb2c9163e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.991+0000 7f0004016e40 1 -- 172.21.15.12:0/267042342 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c916b10 con 0x55bb2c914000 2023-10-05T19:40:17.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.991+0000 7f000234c700 1 --2- 172.21.15.12:0/267042342 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55bb2c914000 0x55bb2c9163e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.991+0000 7f000234c700 1 --2- 172.21.15.12:0/267042342 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55bb2c914000 0x55bb2c9163e0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.995+0000 7f000234c700 1 -- 172.21.15.12:0/267042342 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v28'12 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7efffc014040 con 0x55bb2c914000 2023-10-05T19:40:17.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.995+0000 7f0004016e40 1 -- 172.21.15.12:0/267042342 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c917160 con 0x55bb2c90d0d0 2023-10-05T19:40:17.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.999+0000 7f0001b4b700 1 -- 172.21.15.12:0/267042342 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v28'37 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7efff4035070 con 0x55bb2c90d0d0 2023-10-05T19:40:17.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.999+0000 7f0004016e40 1 --2- 172.21.15.12:0/267042342 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55bb2c917e00 0x55bb2c91a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.999+0000 7f0004016e40 1 -- 172.21.15.12:0/267042342 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c91a890 con 0x55bb2c917e00 2023-10-05T19:40:17.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.999+0000 7f0001b4b700 1 --2- 172.21.15.12:0/267042342 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55bb2c917e00 0x55bb2c91a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:16.999+0000 7f0001b4b700 1 --2- 172.21.15.12:0/267042342 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55bb2c917e00 0x55bb2c91a1e0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.003+0000 7f0001b4b700 1 -- 172.21.15.12:0/267042342 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v28'26 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7efff4035070 con 0x55bb2c917e00 2023-10-05T19:40:17.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.003+0000 7f0004016e40 1 -- 172.21.15.12:0/267042342 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c908830 con 0x55bb2c914000 2023-10-05T19:40:17.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.003+0000 7f000234c700 1 -- 172.21.15.12:0/267042342 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v28'17 uv8 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7efffc014040 con 0x55bb2c914000 2023-10-05T19:40:17.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.003+0000 7f0004016e40 1 -- 172.21.15.12:0/267042342 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c908830 con 0x55bb2c90d0d0 2023-10-05T19:40:17.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.007+0000 7f0001b4b700 1 -- 172.21.15.12:0/267042342 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v28'14 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7efff4035070 con 0x55bb2c90d0d0 2023-10-05T19:40:17.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.007+0000 7f0004016e40 1 -- 172.21.15.12:0/267042342 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c908830 con 0x55bb2c910890 2023-10-05T19:40:17.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.011+0000 7f000134a700 1 -- 172.21.15.12:0/267042342 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v28'17 uv8 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7effec054020 con 0x55bb2c910890 2023-10-05T19:40:17.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.011+0000 7f0004016e40 1 -- 172.21.15.12:0/267042342 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c91bb10 con 0x55bb2c917e00 2023-10-05T19:40:17.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.015+0000 7f0001b4b700 1 -- 172.21.15.12:0/267042342 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v28'12 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7efff4035070 con 0x55bb2c917e00 2023-10-05T19:40:17.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.015+0000 7f0004016e40 1 -- 172.21.15.12:0/267042342 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c91bb10 con 0x55bb2c914000 2023-10-05T19:40:17.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.015+0000 7f000234c700 1 -- 172.21.15.12:0/267042342 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v28'18 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7efffc014040 con 0x55bb2c914000 2023-10-05T19:40:17.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.015+0000 7f0004016e40 1 -- 172.21.15.12:0/267042342 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c91c130 con 0x55bb2c917e00 2023-10-05T19:40:17.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.019+0000 7f0001b4b700 1 -- 172.21.15.12:0/267042342 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v28'27 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7efff4035070 con 0x55bb2c917e00 2023-10-05T19:40:17.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.019+0000 7f0004016e40 1 -- 172.21.15.12:0/267042342 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c908640 con 0x55bb2c910890 2023-10-05T19:40:17.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.023+0000 7f000134a700 1 -- 172.21.15.12:0/267042342 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v28'14 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7effec054020 con 0x55bb2c910890 2023-10-05T19:40:17.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.023+0000 7f0004016e40 1 -- 172.21.15.12:0/267042342 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c91c620 con 0x55bb2c917e00 2023-10-05T19:40:17.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.023+0000 7f0001b4b700 1 -- 172.21.15.12:0/267042342 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v28'12 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7efff4035070 con 0x55bb2c917e00 2023-10-05T19:40:17.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.023+0000 7f0004016e40 1 -- 172.21.15.12:0/267042342 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c91c620 con 0x55bb2c910890 2023-10-05T19:40:17.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.027+0000 7f000134a700 1 -- 172.21.15.12:0/267042342 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v28'23 uv12 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7effec054020 con 0x55bb2c910890 2023-10-05T19:40:17.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.027+0000 7f0004016e40 1 -- 172.21.15.12:0/267042342 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c91c620 con 0x55bb2c90d0d0 2023-10-05T19:40:17.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.031+0000 7f0001b4b700 1 -- 172.21.15.12:0/267042342 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v28'38 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7efff4035070 con 0x55bb2c90d0d0 2023-10-05T19:40:17.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.031+0000 7f0004016e40 1 -- 172.21.15.12:0/267042342 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c91c620 con 0x55bb2c914000 2023-10-05T19:40:17.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.031+0000 7f000234c700 1 -- 172.21.15.12:0/267042342 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v28'20 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7efffc014040 con 0x55bb2c914000 2023-10-05T19:40:17.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.031+0000 7f0004016e40 1 -- 172.21.15.12:0/267042342 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c90bc50 con 0x55bb2c917e00 2023-10-05T19:40:17.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.035+0000 7f0001b4b700 1 -- 172.21.15.12:0/267042342 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v28'28 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7efff4035070 con 0x55bb2c917e00 2023-10-05T19:40:17.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.035+0000 7f0004016e40 1 -- 172.21.15.12:0/267042342 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c90bc50 con 0x55bb2c914000 2023-10-05T19:40:17.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.039+0000 7f000234c700 1 -- 172.21.15.12:0/267042342 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v28'12 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7efffc014040 con 0x55bb2c914000 2023-10-05T19:40:17.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.039+0000 7f0004016e40 1 -- 172.21.15.12:0/267042342 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c90bc50 con 0x55bb2c90d0d0 2023-10-05T19:40:17.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.043+0000 7f0001b4b700 1 -- 172.21.15.12:0/267042342 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v28'20 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7efff4035070 con 0x55bb2c90d0d0 2023-10-05T19:40:17.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.043+0000 7f0004016e40 1 -- 172.21.15.12:0/267042342 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c90bc80 con 0x55bb2c90d0d0 2023-10-05T19:40:17.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.043+0000 7f0001b4b700 1 -- 172.21.15.12:0/267042342 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v28'39 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7efff4035070 con 0x55bb2c90d0d0 2023-10-05T19:40:17.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.043+0000 7f0004016e40 1 -- 172.21.15.12:0/267042342 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c90bc50 con 0x55bb2c914000 2023-10-05T19:40:17.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.047+0000 7f000234c700 1 -- 172.21.15.12:0/267042342 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v28'21 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7efffc014040 con 0x55bb2c914000 2023-10-05T19:40:17.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.047+0000 7f0004016e40 1 -- 172.21.15.12:0/267042342 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c90bc50 con 0x55bb2c917e00 2023-10-05T19:40:17.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.047+0000 7f0001b4b700 1 -- 172.21.15.12:0/267042342 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v28'22 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7efff4035070 con 0x55bb2c917e00 2023-10-05T19:40:17.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.047+0000 7f0004016e40 1 -- 172.21.15.12:0/267042342 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c90bc50 con 0x55bb2c914000 2023-10-05T19:40:17.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.051+0000 7f000234c700 1 -- 172.21.15.12:0/267042342 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v28'14 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7efffc014040 con 0x55bb2c914000 2023-10-05T19:40:17.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.051+0000 7f0004016e40 1 -- 172.21.15.12:0/267042342 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c90bc50 con 0x55bb2c914000 2023-10-05T19:40:17.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.055+0000 7f000234c700 1 -- 172.21.15.12:0/267042342 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v28'19 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7efffc014040 con 0x55bb2c914000 2023-10-05T19:40:17.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.055+0000 7f0004016e40 1 -- 172.21.15.12:0/267042342 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c90bc50 con 0x55bb2c914000 2023-10-05T19:40:17.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.059+0000 7f000234c700 1 -- 172.21.15.12:0/267042342 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v28'8 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7efffc014040 con 0x55bb2c914000 2023-10-05T19:40:17.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.059+0000 7f0004016e40 1 -- 172.21.15.12:0/267042342 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c90bc50 con 0x55bb2c914000 2023-10-05T19:40:17.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.059+0000 7f000234c700 1 -- 172.21.15.12:0/267042342 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v28'18 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7efffc014040 con 0x55bb2c914000 2023-10-05T19:40:17.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.059+0000 7f0004016e40 1 -- 172.21.15.12:0/267042342 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c90bc50 con 0x55bb2c910890 2023-10-05T19:40:17.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.063+0000 7f000134a700 1 -- 172.21.15.12:0/267042342 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v28'14 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7effec054020 con 0x55bb2c910890 2023-10-05T19:40:17.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.063+0000 7f0004016e40 1 -- 172.21.15.12:0/267042342 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c90bc50 con 0x55bb2c914000 2023-10-05T19:40:17.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.063+0000 7f000234c700 1 -- 172.21.15.12:0/267042342 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v28'12 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7efffc014040 con 0x55bb2c914000 2023-10-05T19:40:17.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.063+0000 7f0004016e40 1 -- 172.21.15.12:0/267042342 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c90bc50 con 0x55bb2c910890 2023-10-05T19:40:17.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.067+0000 7f000134a700 1 -- 172.21.15.12:0/267042342 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v28'18 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7effec054020 con 0x55bb2c910890 2023-10-05T19:40:17.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.067+0000 7f0004016e40 1 -- 172.21.15.12:0/267042342 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c90bc50 con 0x55bb2c914000 2023-10-05T19:40:17.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.071+0000 7f000234c700 1 -- 172.21.15.12:0/267042342 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v28'22 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7efffc014040 con 0x55bb2c914000 2023-10-05T19:40:17.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.071+0000 7f0004016e40 1 -- 172.21.15.12:0/267042342 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c90bc50 con 0x55bb2c910890 2023-10-05T19:40:17.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.075+0000 7f000134a700 1 -- 172.21.15.12:0/267042342 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v28'24 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7effec054020 con 0x55bb2c910890 2023-10-05T19:40:17.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.075+0000 7f0004016e40 1 -- 172.21.15.12:0/267042342 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c90bc50 con 0x55bb2c917e00 2023-10-05T19:40:17.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.075+0000 7f0001b4b700 1 -- 172.21.15.12:0/267042342 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v28'18 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7efff4035070 con 0x55bb2c917e00 2023-10-05T19:40:17.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.079+0000 7eff41ffb700 20 reqs_thread_entry: start 2023-10-05T19:40:17.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.079+0000 7f0004016e40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:17.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.079+0000 7eff337fe700 20 reqs_thread_entry: start 2023-10-05T19:40:17.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.079+0000 7f0004016e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:17.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.079+0000 7eff31ffb700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:47 2023 2023-10-05T19:40:17.090 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:17.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.079+0000 7f0004016e40 20 get_system_obj_state: rctx=0x7ffc7c3c9e18 obj=default.rgw.meta:users.uid:fud state=0x55bb2c7820b0 s->prefetch_data=0 2023-10-05T19:40:17.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.079+0000 7f0004016e40 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2023-10-05T19:40:17.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.079+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:102 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c97afd0 con 0x7efff401bbe0 2023-10-05T19:40:17.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.079+0000 7eff317fa700 20 reqs_thread_entry: start 2023-10-05T19:40:17.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.083+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(102 fud [call,getxattrs,stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x7efffc014040 con 0x7efff401bbe0 2023-10-05T19:40:17.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.083+0000 7f0004016e40 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x0 2023-10-05T19:40:17.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.083+0000 7f0004016e40 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2023-10-05T19:40:17.091 INFO:teuthology.orchestra.run.smithi012.stderr:could not remove user: unable to remove user, user does not exist 2023-10-05T19:40:17.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.083+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:103 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94262394788240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c97afd0 con 0x55bb2c8decc0 2023-10-05T19:40:17.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.087+0000 7f0001b4b700 1 -- 172.21.15.12:0/363495163 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(103 notify.0 [watch unwatch cookie 94262394788240] v28'14 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efff4035070 con 0x55bb2c8decc0 2023-10-05T19:40:17.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.087+0000 7f0004016e40 20 remove_watcher() i=0 2023-10-05T19:40:17.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.087+0000 7f0004016e40 2 removed watcher, disabling cache 2023-10-05T19:40:17.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.087+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:104 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94262394790896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c97afd0 con 0x55bb2c8a97e0 2023-10-05T19:40:17.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.087+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(104 notify.1 [watch unwatch cookie 94262394790896] v28'14 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efffc014040 con 0x55bb2c8a97e0 2023-10-05T19:40:17.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.087+0000 7f0004016e40 20 remove_watcher() i=1 2023-10-05T19:40:17.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.087+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:105 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94262394792000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c97afd0 con 0x55bb2c8e3490 2023-10-05T19:40:17.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.091+0000 7f000134a700 1 -- 172.21.15.12:0/363495163 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(105 notify.2 [watch unwatch cookie 94262394792000] v28'14 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7effec054020 con 0x55bb2c8e3490 2023-10-05T19:40:17.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.091+0000 7f0004016e40 20 remove_watcher() i=2 2023-10-05T19:40:17.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.091+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:106 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94262394798800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c97afd0 con 0x55bb2c8decc0 2023-10-05T19:40:17.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.095+0000 7f0001b4b700 1 -- 172.21.15.12:0/363495163 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(106 notify.3 [watch unwatch cookie 94262394798800] v28'14 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efff4035070 con 0x55bb2c8decc0 2023-10-05T19:40:17.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.095+0000 7f0004016e40 20 remove_watcher() i=3 2023-10-05T19:40:17.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.095+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:107 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94262394799904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c97afd0 con 0x55bb2c8a97e0 2023-10-05T19:40:17.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.099+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(107 notify.4 [watch unwatch cookie 94262394799904] v28'14 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efffc014040 con 0x55bb2c8a97e0 2023-10-05T19:40:17.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.099+0000 7f0004016e40 20 remove_watcher() i=4 2023-10-05T19:40:17.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.099+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:108 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94262394804128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8ec7c0 con 0x7efff401bbe0 2023-10-05T19:40:17.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.099+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(108 notify.5 [watch unwatch cookie 94262394804128] v28'14 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efffc014040 con 0x7efff401bbe0 2023-10-05T19:40:17.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.099+0000 7f0004016e40 20 remove_watcher() i=5 2023-10-05T19:40:17.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.099+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:109 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94262394802496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c97afd0 con 0x7efff401bbe0 2023-10-05T19:40:17.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.103+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(109 notify.6 [watch unwatch cookie 94262394802496] v28'14 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efffc014040 con 0x7efff401bbe0 2023-10-05T19:40:17.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.103+0000 7f0004016e40 20 remove_watcher() i=6 2023-10-05T19:40:17.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.103+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:110 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94262394809744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c8ec7c0 con 0x7efff401bbe0 2023-10-05T19:40:17.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.107+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.0 v2:172.21.15.8:6800/27563 30 ==== osd_op_reply(110 notify.7 [watch unwatch cookie 94262394809744] v28'14 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efffc014040 con 0x7efff401bbe0 2023-10-05T19:40:17.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.107+0000 7f0004016e40 20 remove_watcher() i=7 2023-10-05T19:40:17.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.107+0000 7effd2fe5700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:17.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.107+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:111 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94262394807856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb2c97afd0 con 0x55bb2c8a97e0 2023-10-05T19:40:17.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.111+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(111 data_loggenerations_metadata [watch unwatch cookie 94262394807856] v28'40 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7efffc014040 con 0x55bb2c8a97e0 2023-10-05T19:40:17.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.111+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7efff401bbe0 msgr2=0x7efff40079b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.111+0000 7f0004016e40 1 --2- 172.21.15.12:0/363495163 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7efff401bbe0 0x7efff40079b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:17.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.111+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55bb2c8a97e0 msgr2=0x55bb2c8dc7d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.111+0000 7f0004016e40 1 --2- 172.21.15.12:0/363495163 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55bb2c8a97e0 0x55bb2c8dc7d0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:17.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.111+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55bb2c8decc0 msgr2=0x55bb2c8e10a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.111+0000 7f0004016e40 1 --2- 172.21.15.12:0/363495163 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55bb2c8decc0 0x55bb2c8e10a0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:17.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.111+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55bb2c8e3490 msgr2=0x55bb2c8e58a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.111+0000 7f0004016e40 1 --2- 172.21.15.12:0/363495163 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55bb2c8e3490 0x55bb2c8e58a0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:17.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.111+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7effe4030c50 msgr2=0x7effe4033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.111+0000 7f0004016e40 1 --2- 172.21.15.12:0/363495163 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7effe4030c50 0x7effe4033100 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7effec001e90 tx=0x7effec0013b0 comp rx=0 tx=0).stop 2023-10-05T19:40:17.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.111+0000 7f000234c700 1 -- 172.21.15.12:0/363495163 reap_dead start 2023-10-05T19:40:17.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.111+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb2c8a9410 msgr2=0x55bb2c8d0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.111+0000 7f0004016e40 1 --2- 172.21.15.12:0/363495163 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb2c8a9410 0x55bb2c8d0d10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efff400a520 tx=0x7efff401d000 comp rx=0 tx=0).stop 2023-10-05T19:40:17.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.111+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 shutdown_connections 2023-10-05T19:40:17.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.111+0000 7f0004016e40 1 --2- 172.21.15.12:0/363495163 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb2c8a9410 0x55bb2c8d0d10 secure :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efff400a520 tx=0x7efff401d000 comp rx=0 tx=0).stop 2023-10-05T19:40:17.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.111+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 >> 172.21.15.12:0/363495163 conn(0x55bb2c8a3720 msgr2=0x55bb2c8c46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:17.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.111+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 shutdown_connections 2023-10-05T19:40:17.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.111+0000 7f0004016e40 1 -- 172.21.15.12:0/363495163 wait complete. 2023-10-05T19:40:17.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.111+0000 7f0004016e40 1 -- 172.21.15.12:0/267042342 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55bb2c910890 msgr2=0x55bb2c912ca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.111+0000 7f0004016e40 1 --2- 172.21.15.12:0/267042342 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55bb2c910890 0x55bb2c912ca0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:17.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.111+0000 7f0004016e40 1 -- 172.21.15.12:0/267042342 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55bb2c90d0d0 msgr2=0x55bb2c90f4e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.111+0000 7f0004016e40 1 --2- 172.21.15.12:0/267042342 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55bb2c90d0d0 0x55bb2c90f4e0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:17.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.111+0000 7f0004016e40 1 -- 172.21.15.12:0/267042342 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55bb2c917e00 msgr2=0x55bb2c91a1e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.111+0000 7f0004016e40 1 --2- 172.21.15.12:0/267042342 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55bb2c917e00 0x55bb2c91a1e0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:17.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.111+0000 7f0004016e40 1 -- 172.21.15.12:0/267042342 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55bb2c914000 msgr2=0x55bb2c9163e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.111+0000 7f0004016e40 1 --2- 172.21.15.12:0/267042342 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55bb2c914000 0x55bb2c9163e0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:17.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.111+0000 7f0004016e40 1 -- 172.21.15.12:0/267042342 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7eff48030c00 msgr2=0x7eff480330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.111+0000 7f0004016e40 1 --2- 172.21.15.12:0/267042342 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7eff48030c00 0x7eff480330b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x55bb2c8edca0 tx=0x7efffc00a9d0 comp rx=0 tx=0).stop 2023-10-05T19:40:17.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.111+0000 7f0004016e40 1 -- 172.21.15.12:0/267042342 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb2c8fc9c0 msgr2=0x55bb2c902510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.111+0000 7f0004016e40 1 --2- 172.21.15.12:0/267042342 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb2c8fc9c0 0x55bb2c902510 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x55bb2c8f6a40 tx=0x7effec03d030 comp rx=0 tx=0).stop 2023-10-05T19:40:17.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.111+0000 7f0001b4b700 1 -- 172.21.15.12:0/267042342 reap_dead start 2023-10-05T19:40:17.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.111+0000 7f0004016e40 1 -- 172.21.15.12:0/267042342 shutdown_connections 2023-10-05T19:40:17.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.111+0000 7f0004016e40 1 -- 172.21.15.12:0/267042342 >> 172.21.15.12:0/267042342 conn(0x55bb2c8f7d80 msgr2=0x55bb2c8f5e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:17.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.115+0000 7f0004016e40 1 -- 172.21.15.12:0/267042342 shutdown_connections 2023-10-05T19:40:17.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.115+0000 7f0004016e40 1 -- 172.21.15.12:0/267042342 wait complete. 2023-10-05T19:40:17.128 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-05T19:40:17.128 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'foo'] 2023-10-05T19:40:17.128 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'foo'] 2023-10-05T19:40:17.129 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid foo 2023-10-05T19:40:17.168 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:17.168 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:17.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.159+0000 7f97777dae40 1 Processor -- start 2023-10-05T19:40:17.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.159+0000 7f97777dae40 1 -- start start 2023-10-05T19:40:17.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.159+0000 7f97777dae40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x559b51e4e010 0x559b51e4e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.159+0000 7f97777dae40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x559b51d98bd0 con 0x559b51e4e920 2023-10-05T19:40:17.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.159+0000 7f97777dae40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x559b51d55ee0 con 0x559b51e4e010 2023-10-05T19:40:17.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.163+0000 7f977530f700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x559b51e4e010 0x559b51e4e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.163+0000 7f9774b0e700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x559b51e4e920 0x559b51e4f3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:47156/0 (socket says 172.21.15.12:47156) 2023-10-05T19:40:17.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.163+0000 7f9774b0e700 1 -- 172.21.15.12:0/1067816347 learned_addr learned my addr 172.21.15.12:0/1067816347 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:17.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.163+0000 7f976ffff700 1 -- 172.21.15.12:0/1067816347 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3078538379 0 0) 0x559b51d98bd0 con 0x559b51e4e920 2023-10-05T19:40:17.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.163+0000 7f976ffff700 1 -- 172.21.15.12:0/1067816347 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9758001e10 con 0x559b51e4e920 2023-10-05T19:40:17.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.163+0000 7f977530f700 1 -- 172.21.15.12:0/1067816347 >> v1:172.21.15.8:6789/0 conn(0x559b51e4e920 legacy=0x559b51e4f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.163+0000 7f977530f700 1 -- 172.21.15.12:0/1067816347 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b51e4fad0 con 0x559b51e4e010 2023-10-05T19:40:17.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.163+0000 7f977530f700 1 --2- 172.21.15.12:0/1067816347 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559b51e4e010 0x559b51e4e3e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f976800a5d0 tx=0x7f9768005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed5ce96f288c3c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.163+0000 7f976ffff700 1 -- 172.21.15.12:0/1067816347 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9768007610 con 0x559b51e4e010 2023-10-05T19:40:17.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.163+0000 7f976ffff700 1 -- 172.21.15.12:0/1067816347 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9768007770 con 0x559b51e4e010 2023-10-05T19:40:17.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.163+0000 7f976ffff700 1 -- 172.21.15.12:0/1067816347 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9768007a40 con 0x559b51e4e010 2023-10-05T19:40:17.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.163+0000 7f97777dae40 1 -- 172.21.15.12:0/1067816347 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559b51e4e010 msgr2=0x559b51e4e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.163+0000 7f97777dae40 1 --2- 172.21.15.12:0/1067816347 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559b51e4e010 0x559b51e4e3e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f976800a5d0 tx=0x7f9768005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:17.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.163+0000 7f97777dae40 1 -- 172.21.15.12:0/1067816347 shutdown_connections 2023-10-05T19:40:17.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.163+0000 7f97777dae40 1 --2- 172.21.15.12:0/1067816347 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559b51e4e010 0x559b51e4e3e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:17.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.163+0000 7f97777dae40 1 -- 172.21.15.12:0/1067816347 >> 172.21.15.12:0/1067816347 conn(0x559b51dc1860 msgr2=0x559b51dc1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:17.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.163+0000 7f97777dae40 1 -- 172.21.15.12:0/1067816347 shutdown_connections 2023-10-05T19:40:17.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.163+0000 7f97777dae40 1 -- 172.21.15.12:0/1067816347 wait complete. 2023-10-05T19:40:17.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.163+0000 7f97777dae40 1 Processor -- start 2023-10-05T19:40:17.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.163+0000 7f97777dae40 1 -- start start 2023-10-05T19:40:17.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.163+0000 7f97777dae40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559b51e4e010 0x559b51e6cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.163+0000 7f97777dae40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b51d19e30 con 0x559b51e4e010 2023-10-05T19:40:17.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.167+0000 7f977530f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559b51e4e010 0x559b51e6cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.167+0000 7f977530f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559b51e4e010 0x559b51e6cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36438/0 (socket says 172.21.15.12:36438) 2023-10-05T19:40:17.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.167+0000 7f977530f700 1 -- 172.21.15.12:0/2977328184 learned_addr learned my addr 172.21.15.12:0/2977328184 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:17.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.167+0000 7f977530f700 1 -- 172.21.15.12:0/2977328184 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9768005cf0 con 0x559b51e4e010 2023-10-05T19:40:17.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.167+0000 7f977530f700 1 --2- 172.21.15.12:0/2977328184 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559b51e4e010 0x559b51e6cf10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9768000c00 tx=0x7f9768014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=838803bd9f3d5fbd server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.167+0000 7f976effd700 1 -- 172.21.15.12:0/2977328184 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f97680078e0 con 0x559b51e4e010 2023-10-05T19:40:17.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.167+0000 7f976effd700 1 -- 172.21.15.12:0/2977328184 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9768007a40 con 0x559b51e4e010 2023-10-05T19:40:17.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.167+0000 7f976effd700 1 -- 172.21.15.12:0/2977328184 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9768007bd0 con 0x559b51e4e010 2023-10-05T19:40:17.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.167+0000 7f97777dae40 1 -- 172.21.15.12:0/2977328184 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559b51e4e010 msgr2=0x559b51e6cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.167+0000 7f97777dae40 1 --2- 172.21.15.12:0/2977328184 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559b51e4e010 0x559b51e6cf10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9768000c00 tx=0x7f9768014000 comp rx=0 tx=0).stop 2023-10-05T19:40:17.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.167+0000 7f97777dae40 1 -- 172.21.15.12:0/2977328184 shutdown_connections 2023-10-05T19:40:17.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.167+0000 7f97777dae40 1 --2- 172.21.15.12:0/2977328184 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559b51e4e010 0x559b51e6cf10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:17.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.167+0000 7f97777dae40 1 -- 172.21.15.12:0/2977328184 >> 172.21.15.12:0/2977328184 conn(0x559b51dc1860 msgr2=0x559b51e68ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:17.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.167+0000 7f97777dae40 1 -- 172.21.15.12:0/2977328184 shutdown_connections 2023-10-05T19:40:17.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.167+0000 7f97777dae40 1 -- 172.21.15.12:0/2977328184 wait complete. 2023-10-05T19:40:17.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.171+0000 7f97777dae40 1 Processor -- start 2023-10-05T19:40:17.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.171+0000 7f97777dae40 1 -- start start 2023-10-05T19:40:17.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.171+0000 7f97777dae40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559b51e4e010 0x559b51e73c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.171+0000 7f97777dae40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b51ceced0 con 0x559b51e4e010 2023-10-05T19:40:17.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.171+0000 7f977530f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559b51e4e010 0x559b51e73c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.171+0000 7f977530f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559b51e4e010 0x559b51e73c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36442/0 (socket says 172.21.15.12:36442) 2023-10-05T19:40:17.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.171+0000 7f977530f700 1 -- 172.21.15.12:0/371941302 learned_addr learned my addr 172.21.15.12:0/371941302 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:17.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.171+0000 7f977530f700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b51e4fad0 con 0x559b51e4e010 2023-10-05T19:40:17.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.171+0000 7f977530f700 1 --2- 172.21.15.12:0/371941302 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559b51e4e010 0x559b51e73c40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f976800afd0 tx=0x7f976801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.171+0000 7f976d7fa700 1 -- 172.21.15.12:0/371941302 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f97680078e0 con 0x559b51e4e010 2023-10-05T19:40:17.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.171+0000 7f976d7fa700 1 -- 172.21.15.12:0/371941302 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9768029070 con 0x559b51e4e010 2023-10-05T19:40:17.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.171+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559b51e74180 con 0x559b51e4e010 2023-10-05T19:40:17.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.171+0000 7f976d7fa700 1 -- 172.21.15.12:0/371941302 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9768030480 con 0x559b51e4e010 2023-10-05T19:40:17.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.171+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559b51e745c0 con 0x559b51e4e010 2023-10-05T19:40:17.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.175+0000 7f97777dae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.175+0000 7f976d7fa700 1 -- 172.21.15.12:0/371941302 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f9768030600 con 0x559b51e4e010 2023-10-05T19:40:17.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.175+0000 7f976d7fa700 1 --2- 172.21.15.12:0/371941302 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f975c030ca0 0x7f975c033150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.175+0000 7f976d7fa700 1 -- 172.21.15.12:0/371941302 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f976802b070 con 0x559b51e4e010 2023-10-05T19:40:17.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.175+0000 7f9774b0e700 1 --2- 172.21.15.12:0/371941302 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f975c030ca0 0x7f975c033150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.175+0000 7f97777dae40 1 --2- 172.21.15.12:0/371941302 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x559b51e4e920 0x559b51e7f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.175+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e7fd70 con 0x559b51e4e920 2023-10-05T19:40:17.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.175+0000 7f9775b10700 1 --2- 172.21.15.12:0/371941302 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x559b51e4e920 0x559b51e7f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.175+0000 7f9774b0e700 1 --2- 172.21.15.12:0/371941302 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f975c030ca0 0x7f975c033150 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f97600044f0 tx=0x7f9760006690 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.175+0000 7f9775b10700 1 --2- 172.21.15.12:0/371941302 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x559b51e4e920 0x559b51e7f700 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.175+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f977000a500 con 0x559b51e4e920 2023-10-05T19:40:17.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.175+0000 7f97777dae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:17.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.175+0000 7f97777dae40 20 realm 2023-10-05T19:40:17.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.175+0000 7f97777dae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.175+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e7cd60 con 0x559b51e4e920 2023-10-05T19:40:17.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.179+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f977000a500 con 0x559b51e4e920 2023-10-05T19:40:17.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.179+0000 7f97777dae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:17.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.179+0000 7f97777dae40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:17.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.179+0000 7f97777dae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.179+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e80850 con 0x559b51e4e920 2023-10-05T19:40:17.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.179+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f977000a500 con 0x559b51e4e920 2023-10-05T19:40:17.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.179+0000 7f97777dae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:17.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.179+0000 7f97777dae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.179+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e80460 con 0x559b51e4e920 2023-10-05T19:40:17.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.179+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f977000a500 con 0x559b51e4e920 2023-10-05T19:40:17.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.179+0000 7f97777dae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:17.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.179+0000 7f97777dae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.179+0000 7f97777dae40 1 --2- 172.21.15.12:0/371941302 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x559b51e81bf0 0x559b51e83fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.179+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e84510 con 0x559b51e81bf0 2023-10-05T19:40:17.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.179+0000 7f977530f700 1 --2- 172.21.15.12:0/371941302 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x559b51e81bf0 0x559b51e83fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.179+0000 7f977530f700 1 --2- 172.21.15.12:0/371941302 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x559b51e81bf0 0x559b51e83fd0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.179+0000 7f977530f700 1 -- 172.21.15.12:0/371941302 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv1 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f9768025c50 con 0x559b51e81bf0 2023-10-05T19:40:17.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.179+0000 7f97777dae40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:17.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.179+0000 7f97777dae40 20 searching for the correct realm 2023-10-05T19:40:17.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.179+0000 7f97777dae40 1 --2- 172.21.15.12:0/371941302 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x559b51e86390 0x559b51e887a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.179+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e88e50 con 0x559b51e86390 2023-10-05T19:40:17.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.179+0000 7f9774b0e700 1 --2- 172.21.15.12:0/371941302 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x559b51e86390 0x559b51e887a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.183+0000 7f9774b0e700 1 --2- 172.21.15.12:0/371941302 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x559b51e86390 0x559b51e887a0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.183+0000 7f9774b0e700 1 -- 172.21.15.12:0/371941302 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f9760004520 con 0x559b51e86390 2023-10-05T19:40:17.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.183+0000 7f9774b0e700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e88e50 con 0x559b51e81bf0 2023-10-05T19:40:17.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.183+0000 7f977530f700 1 -- 172.21.15.12:0/371941302 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9768025c50 con 0x559b51e81bf0 2023-10-05T19:40:17.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.183+0000 7f977530f700 1 --2- 172.21.15.12:0/371941302 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f9768007030 0x7f976801c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.183+0000 7f977530f700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e88e50 con 0x7f9768007030 2023-10-05T19:40:17.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.183+0000 7f9775b10700 1 --2- 172.21.15.12:0/371941302 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f9768007030 0x7f976801c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.187+0000 7f9775b10700 1 --2- 172.21.15.12:0/371941302 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f9768007030 0x7f976801c3d0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.187+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9770014040 con 0x7f9768007030 2023-10-05T19:40:17.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.187+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e88e50 con 0x559b51e4e920 2023-10-05T19:40:17.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.187+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9770014040 con 0x559b51e4e920 2023-10-05T19:40:17.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.187+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e88e50 con 0x559b51e4e920 2023-10-05T19:40:17.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.187+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9770014040 con 0x559b51e4e920 2023-10-05T19:40:17.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.187+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e88e50 con 0x7f9768007030 2023-10-05T19:40:17.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.187+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9770014040 con 0x7f9768007030 2023-10-05T19:40:17.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.187+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e88e50 con 0x7f9768007030 2023-10-05T19:40:17.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.191+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9770014040 con 0x7f9768007030 2023-10-05T19:40:17.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.191+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e88e50 con 0x559b51e81bf0 2023-10-05T19:40:17.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.191+0000 7f977530f700 1 -- 172.21.15.12:0/371941302 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9768025c50 con 0x559b51e81bf0 2023-10-05T19:40:17.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.191+0000 7f977530f700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e88e50 con 0x559b51e86390 2023-10-05T19:40:17.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.191+0000 7f9774b0e700 1 -- 172.21.15.12:0/371941302 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9760004520 con 0x559b51e86390 2023-10-05T19:40:17.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.191+0000 7f9774b0e700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e88e50 con 0x559b51e4e920 2023-10-05T19:40:17.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.191+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9770014040 con 0x559b51e4e920 2023-10-05T19:40:17.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.191+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e88e50 con 0x7f9768007030 2023-10-05T19:40:17.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.191+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9770014040 con 0x7f9768007030 2023-10-05T19:40:17.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.191+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e88e50 con 0x559b51e4e920 2023-10-05T19:40:17.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.195+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9770014040 con 0x559b51e4e920 2023-10-05T19:40:17.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.195+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e88e50 con 0x559b51e81bf0 2023-10-05T19:40:17.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.195+0000 7f977530f700 1 -- 172.21.15.12:0/371941302 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9768025c50 con 0x559b51e81bf0 2023-10-05T19:40:17.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.195+0000 7f977530f700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e88e50 con 0x559b51e86390 2023-10-05T19:40:17.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.195+0000 7f9774b0e700 1 -- 172.21.15.12:0/371941302 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9760004520 con 0x559b51e86390 2023-10-05T19:40:17.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.195+0000 7f9774b0e700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e88e50 con 0x559b51e86390 2023-10-05T19:40:17.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.195+0000 7f9774b0e700 1 -- 172.21.15.12:0/371941302 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9760004520 con 0x559b51e86390 2023-10-05T19:40:17.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.195+0000 7f9774b0e700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e88e50 con 0x7f9768007030 2023-10-05T19:40:17.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.195+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9770014040 con 0x7f9768007030 2023-10-05T19:40:17.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.195+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e88e50 con 0x559b51e86390 2023-10-05T19:40:17.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.199+0000 7f9774b0e700 1 -- 172.21.15.12:0/371941302 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9760004520 con 0x559b51e86390 2023-10-05T19:40:17.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.199+0000 7f9774b0e700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e88e50 con 0x559b51e86390 2023-10-05T19:40:17.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.199+0000 7f9774b0e700 1 -- 172.21.15.12:0/371941302 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9760004520 con 0x559b51e86390 2023-10-05T19:40:17.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.199+0000 7f9774b0e700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e88e50 con 0x559b51e4e920 2023-10-05T19:40:17.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.199+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9770014040 con 0x559b51e4e920 2023-10-05T19:40:17.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.199+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e88e50 con 0x559b51e4e920 2023-10-05T19:40:17.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.199+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9770014040 con 0x559b51e4e920 2023-10-05T19:40:17.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.199+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e88e50 con 0x7f9768007030 2023-10-05T19:40:17.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.199+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9770014040 con 0x7f9768007030 2023-10-05T19:40:17.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.199+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e88e50 con 0x559b51e86390 2023-10-05T19:40:17.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.203+0000 7f9774b0e700 1 -- 172.21.15.12:0/371941302 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9760004520 con 0x559b51e86390 2023-10-05T19:40:17.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.203+0000 7f9774b0e700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e88e50 con 0x559b51e81bf0 2023-10-05T19:40:17.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.203+0000 7f977530f700 1 -- 172.21.15.12:0/371941302 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9768025c50 con 0x559b51e81bf0 2023-10-05T19:40:17.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.203+0000 7f977530f700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e88e50 con 0x559b51e4e920 2023-10-05T19:40:17.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.203+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9770014040 con 0x559b51e4e920 2023-10-05T19:40:17.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.203+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e88e50 con 0x7f9768007030 2023-10-05T19:40:17.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.203+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9770014040 con 0x7f9768007030 2023-10-05T19:40:17.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.203+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e88e50 con 0x559b51e86390 2023-10-05T19:40:17.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.203+0000 7f9774b0e700 1 -- 172.21.15.12:0/371941302 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9760004520 con 0x559b51e86390 2023-10-05T19:40:17.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.203+0000 7f9774b0e700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e88e50 con 0x7f9768007030 2023-10-05T19:40:17.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.207+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9770014040 con 0x7f9768007030 2023-10-05T19:40:17.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.207+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e88e50 con 0x559b51e81bf0 2023-10-05T19:40:17.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.207+0000 7f977530f700 1 -- 172.21.15.12:0/371941302 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9768025c50 con 0x559b51e81bf0 2023-10-05T19:40:17.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.207+0000 7f977530f700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e88e50 con 0x559b51e86390 2023-10-05T19:40:17.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.207+0000 7f9774b0e700 1 -- 172.21.15.12:0/371941302 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9760004520 con 0x559b51e86390 2023-10-05T19:40:17.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.207+0000 7f9774b0e700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e88e50 con 0x7f9768007030 2023-10-05T19:40:17.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.207+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9770014040 con 0x7f9768007030 2023-10-05T19:40:17.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.207+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e88e50 con 0x559b51e4e920 2023-10-05T19:40:17.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.211+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f9770014040 con 0x559b51e4e920 2023-10-05T19:40:17.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.211+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e88e50 con 0x559b51e81bf0 2023-10-05T19:40:17.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.211+0000 7f977530f700 1 -- 172.21.15.12:0/371941302 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f9768025c50 con 0x559b51e81bf0 2023-10-05T19:40:17.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.211+0000 7f97777dae40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:17.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.211+0000 7f97777dae40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:17.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.211+0000 7f97777dae40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:17.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.211+0000 7f97777dae40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:17.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.211+0000 7f97777dae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.211+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e85110 con 0x559b51e4e920 2023-10-05T19:40:17.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.211+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9770014040 con 0x559b51e4e920 2023-10-05T19:40:17.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.211+0000 7f97777dae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:17.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.211+0000 7f97777dae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.211+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e84d40 con 0x559b51e4e920 2023-10-05T19:40:17.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.211+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f9770014040 con 0x559b51e4e920 2023-10-05T19:40:17.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.211+0000 7f97777dae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:17.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.211+0000 7f97777dae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.211+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e892c0 con 0x559b51e86390 2023-10-05T19:40:17.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.211+0000 7f9774b0e700 1 -- 172.21.15.12:0/371941302 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f9760004520 con 0x559b51e86390 2023-10-05T19:40:17.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.211+0000 7f97777dae40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:17.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.211+0000 7f97777dae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.211+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e89670 con 0x559b51e86390 2023-10-05T19:40:17.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.215+0000 7f9774b0e700 1 -- 172.21.15.12:0/371941302 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f9760004520 con 0x559b51e86390 2023-10-05T19:40:17.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.215+0000 7f97777dae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:17.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.215+0000 7f97777dae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.215+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e89a20 con 0x559b51e4e920 2023-10-05T19:40:17.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.215+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9770014040 con 0x559b51e4e920 2023-10-05T19:40:17.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.215+0000 7f97777dae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:17.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.215+0000 7f97777dae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.215+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e855a0 con 0x559b51e4e920 2023-10-05T19:40:17.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.215+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f9770014040 con 0x559b51e4e920 2023-10-05T19:40:17.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.215+0000 7f97777dae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:17.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.215+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e85570 con 0x559b51e86390 2023-10-05T19:40:17.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.219+0000 7f9774b0e700 1 -- 172.21.15.12:0/371941302 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f9760004520 con 0x559b51e86390 2023-10-05T19:40:17.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.219+0000 7f9774b0e700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e85570 con 0x559b51e81bf0 2023-10-05T19:40:17.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.219+0000 7f977530f700 1 -- 172.21.15.12:0/371941302 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9768025c50 con 0x559b51e81bf0 2023-10-05T19:40:17.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.219+0000 7f977530f700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e85570 con 0x7f9768007030 2023-10-05T19:40:17.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.219+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9770014040 con 0x7f9768007030 2023-10-05T19:40:17.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.219+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e89a20 con 0x559b51e4e920 2023-10-05T19:40:17.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.219+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9770014040 con 0x559b51e4e920 2023-10-05T19:40:17.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.219+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e89a20 con 0x559b51e4e920 2023-10-05T19:40:17.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.223+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9770014040 con 0x559b51e4e920 2023-10-05T19:40:17.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.223+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e89a20 con 0x7f9768007030 2023-10-05T19:40:17.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.223+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9770014040 con 0x7f9768007030 2023-10-05T19:40:17.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.223+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e89a20 con 0x7f9768007030 2023-10-05T19:40:17.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.223+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9770014040 con 0x7f9768007030 2023-10-05T19:40:17.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.223+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e89a20 con 0x559b51e81bf0 2023-10-05T19:40:17.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.223+0000 7f977530f700 1 -- 172.21.15.12:0/371941302 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9768025c50 con 0x559b51e81bf0 2023-10-05T19:40:17.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.223+0000 7f977530f700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e89a20 con 0x559b51e86390 2023-10-05T19:40:17.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.227+0000 7f9774b0e700 1 -- 172.21.15.12:0/371941302 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f976002d040 con 0x559b51e86390 2023-10-05T19:40:17.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.227+0000 7f9774b0e700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e89a20 con 0x559b51e4e920 2023-10-05T19:40:17.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.227+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9770014040 con 0x559b51e4e920 2023-10-05T19:40:17.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.227+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e89a20 con 0x7f9768007030 2023-10-05T19:40:17.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.227+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9770014040 con 0x7f9768007030 2023-10-05T19:40:17.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.227+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e89a20 con 0x559b51e4e920 2023-10-05T19:40:17.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.227+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9770014040 con 0x559b51e4e920 2023-10-05T19:40:17.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.227+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e89a20 con 0x559b51e81bf0 2023-10-05T19:40:17.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.231+0000 7f977530f700 1 -- 172.21.15.12:0/371941302 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9768025c50 con 0x559b51e81bf0 2023-10-05T19:40:17.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.231+0000 7f977530f700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e89a20 con 0x559b51e86390 2023-10-05T19:40:17.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.231+0000 7f9774b0e700 1 -- 172.21.15.12:0/371941302 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f976002d040 con 0x559b51e86390 2023-10-05T19:40:17.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.231+0000 7f9774b0e700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e89a20 con 0x559b51e86390 2023-10-05T19:40:17.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.231+0000 7f9774b0e700 1 -- 172.21.15.12:0/371941302 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f976002d040 con 0x559b51e86390 2023-10-05T19:40:17.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.231+0000 7f9774b0e700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e89a20 con 0x7f9768007030 2023-10-05T19:40:17.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.231+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9770014040 con 0x7f9768007030 2023-10-05T19:40:17.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.231+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e89a20 con 0x559b51e86390 2023-10-05T19:40:17.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.231+0000 7f9774b0e700 1 -- 172.21.15.12:0/371941302 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f976002d040 con 0x559b51e86390 2023-10-05T19:40:17.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.231+0000 7f9774b0e700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e89a20 con 0x559b51e86390 2023-10-05T19:40:17.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.235+0000 7f9774b0e700 1 -- 172.21.15.12:0/371941302 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f976002d040 con 0x559b51e86390 2023-10-05T19:40:17.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.235+0000 7f9774b0e700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e89a20 con 0x559b51e4e920 2023-10-05T19:40:17.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.235+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9770014040 con 0x559b51e4e920 2023-10-05T19:40:17.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.235+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e89a20 con 0x559b51e4e920 2023-10-05T19:40:17.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.235+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9770014040 con 0x559b51e4e920 2023-10-05T19:40:17.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.235+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e89a20 con 0x7f9768007030 2023-10-05T19:40:17.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.235+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9770014040 con 0x7f9768007030 2023-10-05T19:40:17.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.235+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e89a20 con 0x559b51e86390 2023-10-05T19:40:17.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.235+0000 7f9774b0e700 1 -- 172.21.15.12:0/371941302 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f976002d040 con 0x559b51e86390 2023-10-05T19:40:17.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.235+0000 7f9774b0e700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e89a20 con 0x559b51e81bf0 2023-10-05T19:40:17.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.239+0000 7f977530f700 1 -- 172.21.15.12:0/371941302 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9768025c50 con 0x559b51e81bf0 2023-10-05T19:40:17.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.239+0000 7f977530f700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e89a20 con 0x559b51e4e920 2023-10-05T19:40:17.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.239+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9770014040 con 0x559b51e4e920 2023-10-05T19:40:17.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.239+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e89a20 con 0x7f9768007030 2023-10-05T19:40:17.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.239+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9770014040 con 0x7f9768007030 2023-10-05T19:40:17.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.239+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e89a20 con 0x559b51e86390 2023-10-05T19:40:17.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.239+0000 7f9774b0e700 1 -- 172.21.15.12:0/371941302 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f976002d040 con 0x559b51e86390 2023-10-05T19:40:17.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.239+0000 7f9774b0e700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e89a20 con 0x7f9768007030 2023-10-05T19:40:17.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.243+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9770014040 con 0x7f9768007030 2023-10-05T19:40:17.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.243+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e89a20 con 0x559b51e81bf0 2023-10-05T19:40:17.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.243+0000 7f977530f700 1 -- 172.21.15.12:0/371941302 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9768025c50 con 0x559b51e81bf0 2023-10-05T19:40:17.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.243+0000 7f977530f700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e89a20 con 0x559b51e86390 2023-10-05T19:40:17.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.243+0000 7f9774b0e700 1 -- 172.21.15.12:0/371941302 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f976002d040 con 0x559b51e86390 2023-10-05T19:40:17.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.243+0000 7f9774b0e700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e89a20 con 0x7f9768007030 2023-10-05T19:40:17.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.243+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9770014040 con 0x7f9768007030 2023-10-05T19:40:17.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.243+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e89a20 con 0x559b51e4e920 2023-10-05T19:40:17.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.243+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f9770014040 con 0x559b51e4e920 2023-10-05T19:40:17.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.247+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e89a20 con 0x559b51e81bf0 2023-10-05T19:40:17.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.247+0000 7f977530f700 1 -- 172.21.15.12:0/371941302 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f9768025c50 con 0x559b51e81bf0 2023-10-05T19:40:17.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.247+0000 7f97777dae40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:17.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.247+0000 7f97777dae40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:17.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.247+0000 7f97777dae40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:17.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.247+0000 7f97777dae40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:17.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.247+0000 7f97777dae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.247+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e85570 con 0x559b51e4e920 2023-10-05T19:40:17.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.247+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f9770014040 con 0x559b51e4e920 2023-10-05T19:40:17.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.247+0000 7f97777dae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:17.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.247+0000 7f97777dae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.247+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e8a310 con 0x559b51e81bf0 2023-10-05T19:40:17.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.247+0000 7f977530f700 1 -- 172.21.15.12:0/371941302 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv1 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f9768025c50 con 0x559b51e81bf0 2023-10-05T19:40:17.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.247+0000 7f97777dae40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:17.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.247+0000 7f97777dae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.247+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e8ac00 con 0x559b51e4e920 2023-10-05T19:40:17.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.247+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f9770014040 con 0x559b51e4e920 2023-10-05T19:40:17.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.247+0000 7f97777dae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:17.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.247+0000 7f97777dae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.247+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e8ac00 con 0x559b51e86390 2023-10-05T19:40:17.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.251+0000 7f9774b0e700 1 -- 172.21.15.12:0/371941302 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f976002d040 con 0x559b51e86390 2023-10-05T19:40:17.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.251+0000 7f97777dae40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:17.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.251+0000 7f97777dae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.251+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e8b370 con 0x7f9768007030 2023-10-05T19:40:17.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.251+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f9770014040 con 0x7f9768007030 2023-10-05T19:40:17.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.251+0000 7f97777dae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:17.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.251+0000 7f97777dae40 20 zone default found 2023-10-05T19:40:17.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.251+0000 7f97777dae40 4 Realm: () 2023-10-05T19:40:17.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.251+0000 7f97777dae40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:17.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.251+0000 7f97777dae40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:17.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.251+0000 7f97777dae40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:17.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.251+0000 7f97777dae40 20 zonegroup default 2023-10-05T19:40:17.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.251+0000 7f97777dae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.251+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e8b320 con 0x7f9768007030 2023-10-05T19:40:17.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.251+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f9770014040 con 0x7f9768007030 2023-10-05T19:40:17.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.251+0000 7f97777dae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:17.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.251+0000 7f97777dae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.251+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e8cca0 con 0x559b51e86390 2023-10-05T19:40:17.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.251+0000 7f9774b0e700 1 -- 172.21.15.12:0/371941302 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f976002d040 con 0x559b51e86390 2023-10-05T19:40:17.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.251+0000 7f97777dae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:17.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.251+0000 7f97777dae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.251+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e8d050 con 0x559b51e86390 2023-10-05T19:40:17.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.255+0000 7f9774b0e700 1 -- 172.21.15.12:0/371941302 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f976003c040 con 0x559b51e86390 2023-10-05T19:40:17.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.255+0000 7f97777dae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:17.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.255+0000 7f97777dae40 20 started sync module instance, tier type = 2023-10-05T19:40:17.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.255+0000 7f97777dae40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:17.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.255+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e8c360 con 0x559b51e81bf0 2023-10-05T19:40:17.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.255+0000 7f977530f700 1 -- 172.21.15.12:0/371941302 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'15 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9768025c50 con 0x559b51e81bf0 2023-10-05T19:40:17.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.255+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94125582493456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e8d400 con 0x559b51e81bf0 2023-10-05T19:40:17.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.255+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e8dca0 con 0x559b51e4e920 2023-10-05T19:40:17.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.259+0000 7f977530f700 1 -- 172.21.15.12:0/371941302 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94125582493456] v28'16 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9768025c50 con 0x559b51e81bf0 2023-10-05T19:40:17.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.259+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'15 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9770014040 con 0x559b51e4e920 2023-10-05T19:40:17.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.259+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94125582505904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e8c880 con 0x559b51e4e920 2023-10-05T19:40:17.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.259+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e8e860 con 0x559b51e86390 2023-10-05T19:40:17.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.263+0000 7f9774b0e700 1 -- 172.21.15.12:0/371941302 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'15 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f976003c040 con 0x559b51e86390 2023-10-05T19:40:17.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.263+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94125582508912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e8f150 con 0x559b51e86390 2023-10-05T19:40:17.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.263+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e8fb60 con 0x559b51e81bf0 2023-10-05T19:40:17.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.263+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94125582505904] v28'16 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9770014040 con 0x559b51e4e920 2023-10-05T19:40:17.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.267+0000 7f977530f700 1 -- 172.21.15.12:0/371941302 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'15 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9768025c50 con 0x559b51e81bf0 2023-10-05T19:40:17.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.267+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94125582513776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e8dd20 con 0x559b51e81bf0 2023-10-05T19:40:17.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.267+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e90780 con 0x559b51e4e920 2023-10-05T19:40:17.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.267+0000 7f9774b0e700 1 -- 172.21.15.12:0/371941302 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94125582508912] v28'16 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f976003c040 con 0x559b51e86390 2023-10-05T19:40:17.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.267+0000 7f977530f700 1 -- 172.21.15.12:0/371941302 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94125582513776] v28'16 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9768025c50 con 0x559b51e81bf0 2023-10-05T19:40:17.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.267+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'15 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9770014040 con 0x559b51e4e920 2023-10-05T19:40:17.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.271+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94125582514880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e90290 con 0x559b51e4e920 2023-10-05T19:40:17.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.271+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e91070 con 0x7f9768007030 2023-10-05T19:40:17.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.271+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'15 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9770014040 con 0x7f9768007030 2023-10-05T19:40:17.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.271+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94125582518480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e91210 con 0x7f9768007030 2023-10-05T19:40:17.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.271+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94125582514880] v28'16 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9770014040 con 0x559b51e4e920 2023-10-05T19:40:17.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.271+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e91af0 con 0x7f9768007030 2023-10-05T19:40:17.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.275+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(96 notify.6 [create] v28'15 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9770014040 con 0x7f9768007030 2023-10-05T19:40:17.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.275+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94125582511200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e91700 con 0x7f9768007030 2023-10-05T19:40:17.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.275+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e92550 con 0x7f9768007030 2023-10-05T19:40:17.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.275+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(95 notify.5 [watch watch cookie 94125582518480] v28'16 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9770014040 con 0x7f9768007030 2023-10-05T19:40:17.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.275+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(97 notify.6 [watch watch cookie 94125582511200] v28'16 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9770014040 con 0x7f9768007030 2023-10-05T19:40:17.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.279+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(98 notify.7 [create] v28'15 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9770014040 con 0x7f9768007030 2023-10-05T19:40:17.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.279+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94125582520144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e92060 con 0x7f9768007030 2023-10-05T19:40:17.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.279+0000 7f97777dae40 20 add_watcher() i=0 2023-10-05T19:40:17.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.279+0000 7f97777dae40 20 add_watcher() i=1 2023-10-05T19:40:17.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.279+0000 7f97777dae40 20 add_watcher() i=2 2023-10-05T19:40:17.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.279+0000 7f97777dae40 20 add_watcher() i=3 2023-10-05T19:40:17.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.279+0000 7f97777dae40 20 add_watcher() i=4 2023-10-05T19:40:17.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.279+0000 7f97777dae40 20 add_watcher() i=5 2023-10-05T19:40:17.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.279+0000 7f97777dae40 20 add_watcher() i=6 2023-10-05T19:40:17.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.279+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94125582520144] v28'16 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9770014040 con 0x7f9768007030 2023-10-05T19:40:17.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.279+0000 7f97777dae40 20 add_watcher() i=7 2023-10-05T19:40:17.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.279+0000 7f97777dae40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:17.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.279+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e95100 con 0x559b51e4e920 2023-10-05T19:40:17.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.279+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f9770014040 con 0x559b51e4e920 2023-10-05T19:40:17.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.279+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94125582521248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51e92550 con 0x559b51e4e920 2023-10-05T19:40:17.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.283+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94125582521248] v28'41 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f9770014040 con 0x559b51e4e920 2023-10-05T19:40:17.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.283+0000 7f9745feb700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:17.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.283+0000 7f97777dae40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:17.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.283+0000 7f97777dae40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:17.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.283+0000 7f97777dae40 1 Processor -- start 2023-10-05T19:40:17.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.283+0000 7f97777dae40 1 -- start start 2023-10-05T19:40:17.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.283+0000 7f97777dae40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559b51e9f8d0 0x559b51e9fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.283+0000 7f97777dae40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b51ea01e0 con 0x559b51e9f8d0 2023-10-05T19:40:17.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.283+0000 7f9774b0e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559b51e9f8d0 0x559b51e9fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.283+0000 7f9774b0e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559b51e9f8d0 0x559b51e9fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36448/0 (socket says 172.21.15.12:36448) 2023-10-05T19:40:17.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.283+0000 7f9774b0e700 1 -- 172.21.15.12:0/3296177508 learned_addr learned my addr 172.21.15.12:0/3296177508 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:17.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.283+0000 7f9774b0e700 1 -- 172.21.15.12:0/3296177508 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b51ea0320 con 0x559b51e9f8d0 2023-10-05T19:40:17.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.287+0000 7f9774b0e700 1 --2- 172.21.15.12:0/3296177508 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559b51e9f8d0 0x559b51e9fca0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x559b51e84920 tx=0x7f9760027040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d839d8ac1aabfa4 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.287+0000 7f97457ea700 1 -- 172.21.15.12:0/3296177508 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9760039040 con 0x559b51e9f8d0 2023-10-05T19:40:17.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.287+0000 7f97457ea700 1 -- 172.21.15.12:0/3296177508 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f976003a040 con 0x559b51e9f8d0 2023-10-05T19:40:17.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.287+0000 7f97457ea700 1 -- 172.21.15.12:0/3296177508 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f976002a080 con 0x559b51e9f8d0 2023-10-05T19:40:17.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.287+0000 7f97777dae40 1 -- 172.21.15.12:0/3296177508 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559b51e9f8d0 msgr2=0x559b51e9fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.287+0000 7f97777dae40 1 --2- 172.21.15.12:0/3296177508 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559b51e9f8d0 0x559b51e9fca0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x559b51e84920 tx=0x7f9760027040 comp rx=0 tx=0).stop 2023-10-05T19:40:17.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.287+0000 7f97777dae40 1 -- 172.21.15.12:0/3296177508 shutdown_connections 2023-10-05T19:40:17.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.287+0000 7f97777dae40 1 --2- 172.21.15.12:0/3296177508 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559b51e9f8d0 0x559b51e9fca0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:17.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.287+0000 7f97777dae40 1 -- 172.21.15.12:0/3296177508 >> 172.21.15.12:0/3296177508 conn(0x559b51e9ac90 msgr2=0x559b51e98de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:17.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.287+0000 7f97777dae40 1 -- 172.21.15.12:0/3296177508 shutdown_connections 2023-10-05T19:40:17.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.287+0000 7f97777dae40 1 -- 172.21.15.12:0/3296177508 wait complete. 2023-10-05T19:40:17.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.287+0000 7f97777dae40 1 Processor -- start 2023-10-05T19:40:17.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.287+0000 7f97777dae40 1 -- start start 2023-10-05T19:40:17.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.287+0000 7f97777dae40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559b51e9f8d0 0x559b51ea5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.287+0000 7f97777dae40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b51ea6550 con 0x559b51e9f8d0 2023-10-05T19:40:17.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.287+0000 7f9774b0e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559b51e9f8d0 0x559b51ea5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.287+0000 7f9774b0e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559b51e9f8d0 0x559b51ea5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36460/0 (socket says 172.21.15.12:36460) 2023-10-05T19:40:17.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.287+0000 7f9774b0e700 1 -- 172.21.15.12:0/2736207440 learned_addr learned my addr 172.21.15.12:0/2736207440 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:17.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.291+0000 7f9774b0e700 1 -- 172.21.15.12:0/2736207440 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9758001e10 con 0x559b51e9f8d0 2023-10-05T19:40:17.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.291+0000 7f9774b0e700 1 --2- 172.21.15.12:0/2736207440 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559b51e9f8d0 0x559b51ea5550 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f97600067a0 tx=0x7f976003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.291+0000 7f96bf7fe700 1 -- 172.21.15.12:0/2736207440 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9760039040 con 0x559b51e9f8d0 2023-10-05T19:40:17.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.291+0000 7f96bf7fe700 1 -- 172.21.15.12:0/2736207440 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f976002b040 con 0x559b51e9f8d0 2023-10-05T19:40:17.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.291+0000 7f97777dae40 1 -- 172.21.15.12:0/2736207440 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559b51ea5b50 con 0x559b51e9f8d0 2023-10-05T19:40:17.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.291+0000 7f96bf7fe700 1 -- 172.21.15.12:0/2736207440 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f976001a040 con 0x559b51e9f8d0 2023-10-05T19:40:17.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.291+0000 7f97777dae40 1 -- 172.21.15.12:0/2736207440 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559b51ea5f90 con 0x559b51e9f8d0 2023-10-05T19:40:17.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.291+0000 7f96bf7fe700 1 -- 172.21.15.12:0/2736207440 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f976004c9f0 con 0x559b51e9f8d0 2023-10-05T19:40:17.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.291+0000 7f96bf7fe700 1 --2- 172.21.15.12:0/2736207440 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f96b4030bb0 0x7f96b4033060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.291+0000 7f96bf7fe700 1 -- 172.21.15.12:0/2736207440 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f9760028040 con 0x559b51e9f8d0 2023-10-05T19:40:17.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.291+0000 7f9775b10700 1 --2- 172.21.15.12:0/2736207440 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f96b4030bb0 0x7f96b4033060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.291+0000 7f97777dae40 1 --2- 172.21.15.12:0/2736207440 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x559b51eb0170 0x559b51eb2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.291+0000 7f97777dae40 1 -- 172.21.15.12:0/2736207440 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51eb2bd0 con 0x559b51eb0170 2023-10-05T19:40:17.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.291+0000 7f977530f700 1 --2- 172.21.15.12:0/2736207440 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x559b51eb0170 0x559b51eb2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.295+0000 7f9775b10700 1 --2- 172.21.15.12:0/2736207440 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f96b4030bb0 0x7f96b4033060 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x559b51e91830 tx=0x7f977000a970 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.295+0000 7f977530f700 1 --2- 172.21.15.12:0/2736207440 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x559b51eb0170 0x559b51eb2580 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.295+0000 7f977530f700 1 -- 172.21.15.12:0/2736207440 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v28'19 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f9768025c50 con 0x559b51eb0170 2023-10-05T19:40:17.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.295+0000 7f97777dae40 1 --2- 172.21.15.12:0/2736207440 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x559b51eb38a0 0x559b51eb5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.295+0000 7f97777dae40 1 -- 172.21.15.12:0/2736207440 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51eb6380 con 0x559b51eb38a0 2023-10-05T19:40:17.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.299+0000 7f9774b0e700 1 --2- 172.21.15.12:0/2736207440 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x559b51eb38a0 0x559b51eb5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.299+0000 7f9774b0e700 1 --2- 172.21.15.12:0/2736207440 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x559b51eb38a0 0x559b51eb5c80 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.299+0000 7f9774b0e700 1 -- 172.21.15.12:0/2736207440 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v28'13 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f976003c040 con 0x559b51eb38a0 2023-10-05T19:40:17.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.299+0000 7f97777dae40 1 --2- 172.21.15.12:0/2736207440 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x559b51eb7050 0x559b51eb9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.299+0000 7f97777dae40 1 -- 172.21.15.12:0/2736207440 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51eb9b60 con 0x559b51eb7050 2023-10-05T19:40:17.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.303+0000 7f9775b10700 1 --2- 172.21.15.12:0/2736207440 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x559b51eb7050 0x559b51eb9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.303+0000 7f9775b10700 1 --2- 172.21.15.12:0/2736207440 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x559b51eb7050 0x559b51eb9430 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.303+0000 7f9775b10700 1 -- 172.21.15.12:0/2736207440 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v28'13 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f9770014040 con 0x559b51eb7050 2023-10-05T19:40:17.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.303+0000 7f97777dae40 1 -- 172.21.15.12:0/2736207440 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51eba1b0 con 0x559b51eb0170 2023-10-05T19:40:17.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.307+0000 7f977530f700 1 -- 172.21.15.12:0/2736207440 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v28'42 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f9768025c50 con 0x559b51eb0170 2023-10-05T19:40:17.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.307+0000 7f97777dae40 1 --2- 172.21.15.12:0/2736207440 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x559b51ebae50 0x559b51ebd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.307+0000 7f97777dae40 1 -- 172.21.15.12:0/2736207440 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51ebd890 con 0x559b51ebae50 2023-10-05T19:40:17.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.307+0000 7f977530f700 1 --2- 172.21.15.12:0/2736207440 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x559b51ebae50 0x559b51ebd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.307+0000 7f977530f700 1 --2- 172.21.15.12:0/2736207440 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x559b51ebae50 0x559b51ebd230 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.311+0000 7f977530f700 1 -- 172.21.15.12:0/2736207440 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v28'29 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f9768025c50 con 0x559b51ebae50 2023-10-05T19:40:17.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.311+0000 7f97777dae40 1 -- 172.21.15.12:0/2736207440 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51ebdee0 con 0x559b51eb7050 2023-10-05T19:40:17.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.315+0000 7f9775b10700 1 -- 172.21.15.12:0/2736207440 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v28'19 uv8 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f9770014040 con 0x559b51eb7050 2023-10-05T19:40:17.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.315+0000 7f97777dae40 1 -- 172.21.15.12:0/2736207440 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51ebe500 con 0x559b51eb0170 2023-10-05T19:40:17.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.315+0000 7f977530f700 1 -- 172.21.15.12:0/2736207440 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v28'15 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f9768025c50 con 0x559b51eb0170 2023-10-05T19:40:17.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.319+0000 7f97777dae40 1 -- 172.21.15.12:0/2736207440 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51ebe500 con 0x559b51eb38a0 2023-10-05T19:40:17.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.319+0000 7f9774b0e700 1 -- 172.21.15.12:0/2736207440 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v28'19 uv8 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f976003c040 con 0x559b51eb38a0 2023-10-05T19:40:17.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.319+0000 7f97777dae40 1 -- 172.21.15.12:0/2736207440 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51ebeb20 con 0x559b51ebae50 2023-10-05T19:40:17.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.323+0000 7f977530f700 1 -- 172.21.15.12:0/2736207440 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v28'13 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f9768025c50 con 0x559b51ebae50 2023-10-05T19:40:17.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.323+0000 7f97777dae40 1 -- 172.21.15.12:0/2736207440 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51ebeb20 con 0x559b51eb7050 2023-10-05T19:40:17.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.327+0000 7f9775b10700 1 -- 172.21.15.12:0/2736207440 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v28'19 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f9770014040 con 0x559b51eb7050 2023-10-05T19:40:17.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.327+0000 7f97777dae40 1 -- 172.21.15.12:0/2736207440 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51ebf140 con 0x559b51ebae50 2023-10-05T19:40:17.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.327+0000 7f977530f700 1 -- 172.21.15.12:0/2736207440 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v28'30 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9768025c50 con 0x559b51ebae50 2023-10-05T19:40:17.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.327+0000 7f97777dae40 1 -- 172.21.15.12:0/2736207440 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51ebf140 con 0x559b51eb38a0 2023-10-05T19:40:17.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.331+0000 7f9774b0e700 1 -- 172.21.15.12:0/2736207440 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v28'15 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f976003c040 con 0x559b51eb38a0 2023-10-05T19:40:17.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.331+0000 7f97777dae40 1 -- 172.21.15.12:0/2736207440 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51ebf760 con 0x559b51ebae50 2023-10-05T19:40:17.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.335+0000 7f977530f700 1 -- 172.21.15.12:0/2736207440 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v28'13 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9768025c50 con 0x559b51ebae50 2023-10-05T19:40:17.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.335+0000 7f97777dae40 1 -- 172.21.15.12:0/2736207440 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51ebf760 con 0x559b51eb38a0 2023-10-05T19:40:17.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.339+0000 7f9774b0e700 1 -- 172.21.15.12:0/2736207440 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v28'25 uv12 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f976003c040 con 0x559b51eb38a0 2023-10-05T19:40:17.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.339+0000 7f97777dae40 1 -- 172.21.15.12:0/2736207440 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51ebfc50 con 0x559b51eb0170 2023-10-05T19:40:17.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.339+0000 7f977530f700 1 -- 172.21.15.12:0/2736207440 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v28'43 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9768025c50 con 0x559b51eb0170 2023-10-05T19:40:17.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.339+0000 7f97777dae40 1 -- 172.21.15.12:0/2736207440 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51ebfc50 con 0x559b51eb7050 2023-10-05T19:40:17.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.343+0000 7f9775b10700 1 -- 172.21.15.12:0/2736207440 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v28'23 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9770014040 con 0x559b51eb7050 2023-10-05T19:40:17.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.343+0000 7f97777dae40 1 -- 172.21.15.12:0/2736207440 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51eaba90 con 0x559b51ebae50 2023-10-05T19:40:17.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.347+0000 7f977530f700 1 -- 172.21.15.12:0/2736207440 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v28'31 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9768025c50 con 0x559b51ebae50 2023-10-05T19:40:17.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.347+0000 7f97777dae40 1 -- 172.21.15.12:0/2736207440 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51eaba90 con 0x559b51eb7050 2023-10-05T19:40:17.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.347+0000 7f9775b10700 1 -- 172.21.15.12:0/2736207440 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v28'13 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9770014040 con 0x559b51eb7050 2023-10-05T19:40:17.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.347+0000 7f97777dae40 1 -- 172.21.15.12:0/2736207440 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51eaba90 con 0x559b51eb0170 2023-10-05T19:40:17.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.351+0000 7f977530f700 1 -- 172.21.15.12:0/2736207440 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v28'21 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9768025c50 con 0x559b51eb0170 2023-10-05T19:40:17.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.351+0000 7f97777dae40 1 -- 172.21.15.12:0/2736207440 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51eaba90 con 0x559b51eb0170 2023-10-05T19:40:17.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.355+0000 7f977530f700 1 -- 172.21.15.12:0/2736207440 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v28'44 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9768025c50 con 0x559b51eb0170 2023-10-05T19:40:17.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.355+0000 7f97777dae40 1 -- 172.21.15.12:0/2736207440 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51eaba90 con 0x559b51eb7050 2023-10-05T19:40:17.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.359+0000 7f9775b10700 1 -- 172.21.15.12:0/2736207440 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v28'24 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9770014040 con 0x559b51eb7050 2023-10-05T19:40:17.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.359+0000 7f97777dae40 1 -- 172.21.15.12:0/2736207440 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51eaba90 con 0x559b51ebae50 2023-10-05T19:40:17.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.359+0000 7f977530f700 1 -- 172.21.15.12:0/2736207440 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v28'23 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9768025c50 con 0x559b51ebae50 2023-10-05T19:40:17.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.359+0000 7f97777dae40 1 -- 172.21.15.12:0/2736207440 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51eaba90 con 0x559b51eb7050 2023-10-05T19:40:17.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.363+0000 7f9775b10700 1 -- 172.21.15.12:0/2736207440 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v28'15 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9770014040 con 0x559b51eb7050 2023-10-05T19:40:17.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.363+0000 7f97777dae40 1 -- 172.21.15.12:0/2736207440 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51eaba90 con 0x559b51eb7050 2023-10-05T19:40:17.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.367+0000 7f9775b10700 1 -- 172.21.15.12:0/2736207440 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v28'20 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9770014040 con 0x559b51eb7050 2023-10-05T19:40:17.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.367+0000 7f97777dae40 1 -- 172.21.15.12:0/2736207440 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51eaba90 con 0x559b51eb7050 2023-10-05T19:40:17.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.367+0000 7f9775b10700 1 -- 172.21.15.12:0/2736207440 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v28'9 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9770014040 con 0x559b51eb7050 2023-10-05T19:40:17.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.367+0000 7f97777dae40 1 -- 172.21.15.12:0/2736207440 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51eaba90 con 0x559b51eb7050 2023-10-05T19:40:17.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.371+0000 7f9775b10700 1 -- 172.21.15.12:0/2736207440 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v28'20 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9770014040 con 0x559b51eb7050 2023-10-05T19:40:17.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.371+0000 7f97777dae40 1 -- 172.21.15.12:0/2736207440 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51eaba90 con 0x559b51eb38a0 2023-10-05T19:40:17.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.375+0000 7f9774b0e700 1 -- 172.21.15.12:0/2736207440 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v28'15 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f976003c040 con 0x559b51eb38a0 2023-10-05T19:40:17.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.375+0000 7f97777dae40 1 -- 172.21.15.12:0/2736207440 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51eaba90 con 0x559b51eb7050 2023-10-05T19:40:17.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.375+0000 7f9775b10700 1 -- 172.21.15.12:0/2736207440 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v28'13 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9770014040 con 0x559b51eb7050 2023-10-05T19:40:17.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.375+0000 7f97777dae40 1 -- 172.21.15.12:0/2736207440 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51eaba90 con 0x559b51eb38a0 2023-10-05T19:40:17.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.379+0000 7f9774b0e700 1 -- 172.21.15.12:0/2736207440 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v28'20 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f976003c040 con 0x559b51eb38a0 2023-10-05T19:40:17.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.379+0000 7f97777dae40 1 -- 172.21.15.12:0/2736207440 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51eaba90 con 0x559b51eb7050 2023-10-05T19:40:17.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.383+0000 7f9775b10700 1 -- 172.21.15.12:0/2736207440 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v28'25 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9770014040 con 0x559b51eb7050 2023-10-05T19:40:17.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.383+0000 7f97777dae40 1 -- 172.21.15.12:0/2736207440 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51eaba90 con 0x559b51eb38a0 2023-10-05T19:40:17.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.387+0000 7f9774b0e700 1 -- 172.21.15.12:0/2736207440 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v28'26 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f976003c040 con 0x559b51eb38a0 2023-10-05T19:40:17.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.387+0000 7f97777dae40 1 -- 172.21.15.12:0/2736207440 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51eaba90 con 0x559b51ebae50 2023-10-05T19:40:17.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.391+0000 7f977530f700 1 -- 172.21.15.12:0/2736207440 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v28'19 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9768025c50 con 0x559b51ebae50 2023-10-05T19:40:17.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.391+0000 7f96bcff9700 20 reqs_thread_entry: start 2023-10-05T19:40:17.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.391+0000 7f97777dae40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:17.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.391+0000 7f96b27fc700 20 reqs_thread_entry: start 2023-10-05T19:40:17.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.391+0000 7f97777dae40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:17.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.391+0000 7f96b0ff9700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:47 2023 2023-10-05T19:40:17.402 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:17.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.395+0000 7f9697fff700 20 reqs_thread_entry: start 2023-10-05T19:40:17.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.395+0000 7f97777dae40 20 get_system_obj_state: rctx=0x7ffc5ec92328 obj=default.rgw.meta:users.uid:foo state=0x559b51d250b0 s->prefetch_data=0 2023-10-05T19:40:17.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.395+0000 7f97777dae40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2023-10-05T19:40:17.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.395+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51f1ea20 con 0x559b51e4e920 2023-10-05T19:40:17.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.395+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(102 foo [call,getxattrs,stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x7f9770014040 con 0x559b51e4e920 2023-10-05T19:40:17.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.395+0000 7f97777dae40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x0 2023-10-05T19:40:17.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.395+0000 7f97777dae40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:17.403 INFO:teuthology.orchestra.run.smithi012.stderr:could not remove user: unable to remove user, user does not exist 2023-10-05T19:40:17.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.395+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:103 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94125582493456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51f1ea20 con 0x559b51e81bf0 2023-10-05T19:40:17.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.399+0000 7f977530f700 1 -- 172.21.15.12:0/371941302 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(103 notify.0 [watch unwatch cookie 94125582493456] v28'17 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9768025c50 con 0x559b51e81bf0 2023-10-05T19:40:17.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.399+0000 7f97777dae40 20 remove_watcher() i=0 2023-10-05T19:40:17.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.399+0000 7f97777dae40 2 removed watcher, disabling cache 2023-10-05T19:40:17.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.399+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:104 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94125582505904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51f1ea20 con 0x559b51e4e920 2023-10-05T19:40:17.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.399+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(104 notify.1 [watch unwatch cookie 94125582505904] v28'17 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9770014040 con 0x559b51e4e920 2023-10-05T19:40:17.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.399+0000 7f97777dae40 20 remove_watcher() i=1 2023-10-05T19:40:17.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.399+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:105 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94125582508912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51f1ea20 con 0x559b51e86390 2023-10-05T19:40:17.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.403+0000 7f9774b0e700 1 -- 172.21.15.12:0/371941302 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(105 notify.2 [watch unwatch cookie 94125582508912] v28'17 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f976003c040 con 0x559b51e86390 2023-10-05T19:40:17.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.403+0000 7f97777dae40 20 remove_watcher() i=2 2023-10-05T19:40:17.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.403+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:106 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94125582513776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51f1ea20 con 0x559b51e81bf0 2023-10-05T19:40:17.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.407+0000 7f977530f700 1 -- 172.21.15.12:0/371941302 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(106 notify.3 [watch unwatch cookie 94125582513776] v28'17 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9768025c50 con 0x559b51e81bf0 2023-10-05T19:40:17.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.407+0000 7f97777dae40 20 remove_watcher() i=3 2023-10-05T19:40:17.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.407+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:107 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94125582514880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51f1ea20 con 0x559b51e4e920 2023-10-05T19:40:17.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.411+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(107 notify.4 [watch unwatch cookie 94125582514880] v28'17 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9770014040 con 0x559b51e4e920 2023-10-05T19:40:17.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.411+0000 7f97777dae40 20 remove_watcher() i=4 2023-10-05T19:40:17.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.411+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:108 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94125582518480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51f1ea20 con 0x7f9768007030 2023-10-05T19:40:17.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.411+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(108 notify.5 [watch unwatch cookie 94125582518480] v28'17 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9770014040 con 0x7f9768007030 2023-10-05T19:40:17.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.411+0000 7f97777dae40 20 remove_watcher() i=5 2023-10-05T19:40:17.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.411+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:109 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94125582511200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51f1ea20 con 0x7f9768007030 2023-10-05T19:40:17.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.415+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(109 notify.6 [watch unwatch cookie 94125582511200] v28'17 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9770014040 con 0x7f9768007030 2023-10-05T19:40:17.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.415+0000 7f97777dae40 20 remove_watcher() i=6 2023-10-05T19:40:17.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.415+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:110 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94125582520144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51f1ea20 con 0x7f9768007030 2023-10-05T19:40:17.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.419+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(110 notify.7 [watch unwatch cookie 94125582520144] v28'17 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9770014040 con 0x7f9768007030 2023-10-05T19:40:17.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.419+0000 7f97777dae40 20 remove_watcher() i=7 2023-10-05T19:40:17.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.419+0000 7f9745feb700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:17.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.419+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:111 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94125582521248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559b51eddf60 con 0x559b51e4e920 2023-10-05T19:40:17.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.423+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(111 data_loggenerations_metadata [watch unwatch cookie 94125582521248] v28'45 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f9770014040 con 0x559b51e4e920 2023-10-05T19:40:17.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.423+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f9768007030 msgr2=0x7f976801c3d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.423+0000 7f97777dae40 1 --2- 172.21.15.12:0/371941302 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f9768007030 0x7f976801c3d0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:17.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.423+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x559b51e4e920 msgr2=0x559b51e7f700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.423+0000 7f97777dae40 1 --2- 172.21.15.12:0/371941302 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x559b51e4e920 0x559b51e7f700 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:17.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.423+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x559b51e81bf0 msgr2=0x559b51e83fd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.423+0000 7f97777dae40 1 --2- 172.21.15.12:0/371941302 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x559b51e81bf0 0x559b51e83fd0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:17.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.423+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x559b51e86390 msgr2=0x559b51e887a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.423+0000 7f97777dae40 1 --2- 172.21.15.12:0/371941302 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x559b51e86390 0x559b51e887a0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:17.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.423+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f975c030ca0 msgr2=0x7f975c033150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.423+0000 7f97777dae40 1 --2- 172.21.15.12:0/371941302 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f975c030ca0 0x7f975c033150 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f97600044f0 tx=0x7f9760006690 comp rx=0 tx=0).stop 2023-10-05T19:40:17.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.423+0000 7f9775b10700 1 -- 172.21.15.12:0/371941302 reap_dead start 2023-10-05T19:40:17.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.423+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559b51e4e010 msgr2=0x559b51e73c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.423+0000 7f97777dae40 1 --2- 172.21.15.12:0/371941302 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559b51e4e010 0x559b51e73c40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f976800afd0 tx=0x7f976801c000 comp rx=0 tx=0).stop 2023-10-05T19:40:17.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.423+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 shutdown_connections 2023-10-05T19:40:17.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.423+0000 7f97777dae40 1 --2- 172.21.15.12:0/371941302 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559b51e4e010 0x559b51e73c40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:17.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.423+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 >> 172.21.15.12:0/371941302 conn(0x559b51dc1860 msgr2=0x559b51e67550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:17.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.423+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 shutdown_connections 2023-10-05T19:40:17.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.423+0000 7f97777dae40 1 -- 172.21.15.12:0/371941302 wait complete. 2023-10-05T19:40:17.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.423+0000 7f97777dae40 1 -- 172.21.15.12:0/2736207440 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x559b51eb38a0 msgr2=0x559b51eb5c80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.423+0000 7f97777dae40 1 --2- 172.21.15.12:0/2736207440 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x559b51eb38a0 0x559b51eb5c80 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:17.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.423+0000 7f97777dae40 1 -- 172.21.15.12:0/2736207440 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x559b51eb0170 msgr2=0x559b51eb2580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.423+0000 7f97777dae40 1 --2- 172.21.15.12:0/2736207440 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x559b51eb0170 0x559b51eb2580 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:17.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.423+0000 7f97777dae40 1 -- 172.21.15.12:0/2736207440 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x559b51ebae50 msgr2=0x559b51ebd230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.423+0000 7f97777dae40 1 --2- 172.21.15.12:0/2736207440 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x559b51ebae50 0x559b51ebd230 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:17.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.423+0000 7f97777dae40 1 -- 172.21.15.12:0/2736207440 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x559b51eb7050 msgr2=0x559b51eb9430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.423+0000 7f97777dae40 1 --2- 172.21.15.12:0/2736207440 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x559b51eb7050 0x559b51eb9430 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:17.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.427+0000 7f97777dae40 1 -- 172.21.15.12:0/2736207440 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f96b4030bb0 msgr2=0x7f96b4033060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.427+0000 7f97777dae40 1 --2- 172.21.15.12:0/2736207440 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f96b4030bb0 0x7f96b4033060 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x559b51e91830 tx=0x7f977000a970 comp rx=0 tx=0).stop 2023-10-05T19:40:17.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.427+0000 7f97777dae40 1 -- 172.21.15.12:0/2736207440 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559b51e9f8d0 msgr2=0x559b51ea5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.427+0000 7f97777dae40 1 --2- 172.21.15.12:0/2736207440 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559b51e9f8d0 0x559b51ea5550 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f97600067a0 tx=0x7f976003f000 comp rx=0 tx=0).stop 2023-10-05T19:40:17.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.427+0000 7f977530f700 1 -- 172.21.15.12:0/2736207440 reap_dead start 2023-10-05T19:40:17.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.427+0000 7f97777dae40 1 -- 172.21.15.12:0/2736207440 shutdown_connections 2023-10-05T19:40:17.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.427+0000 7f97777dae40 1 --2- 172.21.15.12:0/2736207440 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559b51e9f8d0 0x559b51ea5550 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:17.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.427+0000 7f97777dae40 1 -- 172.21.15.12:0/2736207440 >> 172.21.15.12:0/2736207440 conn(0x559b51e9ac90 msgr2=0x559b51e98d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:17.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.427+0000 7f97777dae40 1 -- 172.21.15.12:0/2736207440 shutdown_connections 2023-10-05T19:40:17.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.427+0000 7f97777dae40 1 -- 172.21.15.12:0/2736207440 wait complete. 2023-10-05T19:40:17.440 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-05T19:40:17.441 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'rm', '--rgw-zone', 'default', '--placement-id', 'new-placement'] 2023-10-05T19:40:17.441 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'rm', '--rgw-zone', 'default', '--placement-id', 'new-placement'] 2023-10-05T19:40:17.441 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement rm --rgw-zone default --placement-id new-placement 2023-10-05T19:40:17.525 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:17.525 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:17.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.519+0000 7f3a6b444e40 1 Processor -- start 2023-10-05T19:40:17.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.519+0000 7f3a6b444e40 1 -- start start 2023-10-05T19:40:17.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.519+0000 7f3a6b444e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x557f04cf33d0 0x557f04cf37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.519+0000 7f3a6b444e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x557f04c3dbd0 con 0x557f04cf3ce0 2023-10-05T19:40:17.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.519+0000 7f3a6b444e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x557f04bfaee0 con 0x557f04cf33d0 2023-10-05T19:40:17.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.519+0000 7f3a68778700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x557f04cf3ce0 0x557f04cf43e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:47172/0 (socket says 172.21.15.12:47172) 2023-10-05T19:40:17.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.519+0000 7f3a68778700 1 -- 172.21.15.12:0/3685117925 learned_addr learned my addr 172.21.15.12:0/3685117925 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:17.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.519+0000 7f3a68f79700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x557f04cf33d0 0x557f04cf37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.519+0000 7f3a67f77700 1 -- 172.21.15.12:0/3685117925 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3314373347 0 0) 0x557f04c3dbd0 con 0x557f04cf3ce0 2023-10-05T19:40:17.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.519+0000 7f3a67f77700 1 -- 172.21.15.12:0/3685117925 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a54001e10 con 0x557f04cf3ce0 2023-10-05T19:40:17.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.519+0000 7f3a68f79700 1 -- 172.21.15.12:0/3685117925 >> v1:172.21.15.8:6789/0 conn(0x557f04cf3ce0 legacy=0x557f04cf43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.519+0000 7f3a68f79700 1 -- 172.21.15.12:0/3685117925 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f04cf4af0 con 0x557f04cf33d0 2023-10-05T19:40:17.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.519+0000 7f3a68f79700 1 --2- 172.21.15.12:0/3685117925 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f04cf33d0 0x557f04cf37a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3a6000a5d0 tx=0x7f3a60005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a70f6b432e5b02eb server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.519+0000 7f3a67f77700 1 -- 172.21.15.12:0/3685117925 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a60007610 con 0x557f04cf33d0 2023-10-05T19:40:17.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.519+0000 7f3a67f77700 1 -- 172.21.15.12:0/3685117925 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a60007770 con 0x557f04cf33d0 2023-10-05T19:40:17.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.519+0000 7f3a67f77700 1 -- 172.21.15.12:0/3685117925 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a60007a40 con 0x557f04cf33d0 2023-10-05T19:40:17.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.519+0000 7f3a6b444e40 1 -- 172.21.15.12:0/3685117925 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f04cf33d0 msgr2=0x557f04cf37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.519+0000 7f3a6b444e40 1 --2- 172.21.15.12:0/3685117925 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f04cf33d0 0x557f04cf37a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3a6000a5d0 tx=0x7f3a60005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:17.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.519+0000 7f3a6b444e40 1 -- 172.21.15.12:0/3685117925 shutdown_connections 2023-10-05T19:40:17.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.519+0000 7f3a6b444e40 1 --2- 172.21.15.12:0/3685117925 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f04cf33d0 0x557f04cf37a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:17.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.519+0000 7f3a6b444e40 1 -- 172.21.15.12:0/3685117925 >> 172.21.15.12:0/3685117925 conn(0x557f04c66880 msgr2=0x557f04c66c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:17.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.519+0000 7f3a6b444e40 1 -- 172.21.15.12:0/3685117925 shutdown_connections 2023-10-05T19:40:17.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.519+0000 7f3a6b444e40 1 -- 172.21.15.12:0/3685117925 wait complete. 2023-10-05T19:40:17.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.523+0000 7f3a6b444e40 1 Processor -- start 2023-10-05T19:40:17.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.523+0000 7f3a6b444e40 1 -- start start 2023-10-05T19:40:17.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.523+0000 7f3a6b444e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f04cf33d0 0x557f04d11fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.523+0000 7f3a6b444e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f04bbee30 con 0x557f04cf33d0 2023-10-05T19:40:17.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.523+0000 7f3a68f79700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f04cf33d0 0x557f04d11fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.523+0000 7f3a68f79700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f04cf33d0 0x557f04d11fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36476/0 (socket says 172.21.15.12:36476) 2023-10-05T19:40:17.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.523+0000 7f3a68f79700 1 -- 172.21.15.12:0/1808210656 learned_addr learned my addr 172.21.15.12:0/1808210656 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:17.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.523+0000 7f3a68f79700 1 -- 172.21.15.12:0/1808210656 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a60005cf0 con 0x557f04cf33d0 2023-10-05T19:40:17.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.523+0000 7f3a68f79700 1 --2- 172.21.15.12:0/1808210656 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f04cf33d0 0x557f04d11fd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3a60000c00 tx=0x7f3a60014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfe3a26a33ed2a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.523+0000 7f3a66f75700 1 -- 172.21.15.12:0/1808210656 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a600078e0 con 0x557f04cf33d0 2023-10-05T19:40:17.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.523+0000 7f3a66f75700 1 -- 172.21.15.12:0/1808210656 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a60007a40 con 0x557f04cf33d0 2023-10-05T19:40:17.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.523+0000 7f3a66f75700 1 -- 172.21.15.12:0/1808210656 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a60007bd0 con 0x557f04cf33d0 2023-10-05T19:40:17.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.523+0000 7f3a6b444e40 1 -- 172.21.15.12:0/1808210656 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f04cf33d0 msgr2=0x557f04d11fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.523+0000 7f3a6b444e40 1 --2- 172.21.15.12:0/1808210656 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f04cf33d0 0x557f04d11fd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3a60000c00 tx=0x7f3a60014000 comp rx=0 tx=0).stop 2023-10-05T19:40:17.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.523+0000 7f3a6b444e40 1 -- 172.21.15.12:0/1808210656 shutdown_connections 2023-10-05T19:40:17.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.523+0000 7f3a6b444e40 1 --2- 172.21.15.12:0/1808210656 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f04cf33d0 0x557f04d11fd0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:17.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.523+0000 7f3a6b444e40 1 -- 172.21.15.12:0/1808210656 >> 172.21.15.12:0/1808210656 conn(0x557f04c66880 msgr2=0x557f04d0db10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:17.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.523+0000 7f3a6b444e40 1 -- 172.21.15.12:0/1808210656 shutdown_connections 2023-10-05T19:40:17.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.523+0000 7f3a6b444e40 1 -- 172.21.15.12:0/1808210656 wait complete. 2023-10-05T19:40:17.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.527+0000 7f3a6b444e40 1 Processor -- start 2023-10-05T19:40:17.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.527+0000 7f3a6b444e40 1 -- start start 2023-10-05T19:40:17.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.527+0000 7f3a6b444e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f04cf33d0 0x557f04d18cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.527+0000 7f3a6b444e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f04b91ed0 con 0x557f04cf33d0 2023-10-05T19:40:17.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.527+0000 7f3a68f79700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f04cf33d0 0x557f04d18cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.527+0000 7f3a68f79700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f04cf33d0 0x557f04d18cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36486/0 (socket says 172.21.15.12:36486) 2023-10-05T19:40:17.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.527+0000 7f3a68f79700 1 -- 172.21.15.12:0/1599656020 learned_addr learned my addr 172.21.15.12:0/1599656020 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:17.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.527+0000 7f3a68f79700 1 -- 172.21.15.12:0/1599656020 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f04cf4af0 con 0x557f04cf33d0 2023-10-05T19:40:17.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.527+0000 7f3a68f79700 1 --2- 172.21.15.12:0/1599656020 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f04cf33d0 0x557f04d18cd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3a6001f040 tx=0x7f3a6001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.527+0000 7f3a65772700 1 -- 172.21.15.12:0/1599656020 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a60007bb0 con 0x557f04cf33d0 2023-10-05T19:40:17.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.527+0000 7f3a65772700 1 -- 172.21.15.12:0/1599656020 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a60029070 con 0x557f04cf33d0 2023-10-05T19:40:17.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.527+0000 7f3a65772700 1 -- 172.21.15.12:0/1599656020 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a60012610 con 0x557f04cf33d0 2023-10-05T19:40:17.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.527+0000 7f3a6b444e40 1 -- 172.21.15.12:0/1599656020 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557f04d19210 con 0x557f04cf33d0 2023-10-05T19:40:17.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.527+0000 7f3a6b444e40 1 -- 172.21.15.12:0/1599656020 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557f04d19650 con 0x557f04cf33d0 2023-10-05T19:40:17.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.531+0000 7f3a6b444e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:17.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.531+0000 7f3a6b444e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:17.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.531+0000 7f3a6b444e40 1 Processor -- start 2023-10-05T19:40:17.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.531+0000 7f3a65772700 1 -- 172.21.15.12:0/1599656020 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f3a60012770 con 0x557f04cf33d0 2023-10-05T19:40:17.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.531+0000 7f3a6b444e40 1 -- start start 2023-10-05T19:40:17.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.531+0000 7f3a6b444e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f04d2a420 0x557f04d2a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.531+0000 7f3a65772700 1 --2- 172.21.15.12:0/1599656020 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3a48030ca0 0x7f3a48033150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.531+0000 7f3a6b444e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f04d2ad30 con 0x557f04d2a420 2023-10-05T19:40:17.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.531+0000 7f3a65772700 1 -- 172.21.15.12:0/1599656020 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f3a6002b070 con 0x557f04cf33d0 2023-10-05T19:40:17.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.531+0000 7f3a68f79700 1 --2- 172.21.15.12:0/1599656020 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3a48030ca0 0x7f3a48033150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.531+0000 7f3a6977a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f04d2a420 0x557f04d2a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.531+0000 7f3a6977a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f04d2a420 0x557f04d2a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36496/0 (socket says 172.21.15.12:36496) 2023-10-05T19:40:17.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.531+0000 7f3a6977a700 1 -- 172.21.15.12:0/4041808769 learned_addr learned my addr 172.21.15.12:0/4041808769 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:17.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.531+0000 7f3a68f79700 1 --2- 172.21.15.12:0/1599656020 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3a48030ca0 0x7f3a48033150 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3a600053a0 tx=0x7f3a60030f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.531+0000 7f3a6977a700 1 -- 172.21.15.12:0/4041808769 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f04d2ae70 con 0x557f04d2a420 2023-10-05T19:40:17.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.531+0000 7f3a6977a700 1 --2- 172.21.15.12:0/4041808769 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f04d2a420 0x557f04d2a7f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3a5800a320 tx=0x7f3a580059d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67767f9f8b9134fa server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.531+0000 7f3a3a7e4700 1 -- 172.21.15.12:0/4041808769 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a5800ae30 con 0x557f04d2a420 2023-10-05T19:40:17.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.531+0000 7f3a3a7e4700 1 -- 172.21.15.12:0/4041808769 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a5800c610 con 0x557f04d2a420 2023-10-05T19:40:17.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.531+0000 7f3a3a7e4700 1 -- 172.21.15.12:0/4041808769 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a5800c7c0 con 0x557f04d2a420 2023-10-05T19:40:17.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.535+0000 7f3a6b444e40 1 -- 172.21.15.12:0/4041808769 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f04d2a420 msgr2=0x557f04d2a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.535+0000 7f3a6b444e40 1 --2- 172.21.15.12:0/4041808769 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f04d2a420 0x557f04d2a7f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3a5800a320 tx=0x7f3a580059d0 comp rx=0 tx=0).stop 2023-10-05T19:40:17.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.535+0000 7f3a6b444e40 1 -- 172.21.15.12:0/4041808769 shutdown_connections 2023-10-05T19:40:17.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.535+0000 7f3a6b444e40 1 --2- 172.21.15.12:0/4041808769 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f04d2a420 0x557f04d2a7f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:17.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.535+0000 7f3a6b444e40 1 -- 172.21.15.12:0/4041808769 >> 172.21.15.12:0/4041808769 conn(0x557f04cf3ce0 msgr2=0x557f04d27150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:17.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.535+0000 7f3a6b444e40 1 -- 172.21.15.12:0/4041808769 shutdown_connections 2023-10-05T19:40:17.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.535+0000 7f3a6b444e40 1 -- 172.21.15.12:0/4041808769 wait complete. 2023-10-05T19:40:17.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.535+0000 7f3a6b444e40 1 Processor -- start 2023-10-05T19:40:17.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.535+0000 7f3a6b444e40 1 -- start start 2023-10-05T19:40:17.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.535+0000 7f3a6b444e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f04d2a420 0x557f04d315d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.535+0000 7f3a6b444e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f04d325d0 con 0x557f04d2a420 2023-10-05T19:40:17.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.535+0000 7f3a6977a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f04d2a420 0x557f04d315d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.535+0000 7f3a6977a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f04d2a420 0x557f04d315d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36506/0 (socket says 172.21.15.12:36506) 2023-10-05T19:40:17.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.535+0000 7f3a6977a700 1 -- 172.21.15.12:0/2375359693 learned_addr learned my addr 172.21.15.12:0/2375359693 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:17.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.535+0000 7f3a6977a700 1 -- 172.21.15.12:0/2375359693 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a58005c90 con 0x557f04d2a420 2023-10-05T19:40:17.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.535+0000 7f3a6977a700 1 --2- 172.21.15.12:0/2375359693 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f04d2a420 0x557f04d315d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x557f04d278e0 tx=0x7f3a58013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.535+0000 7f3a38fe1700 1 -- 172.21.15.12:0/2375359693 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a58015660 con 0x557f04d2a420 2023-10-05T19:40:17.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.535+0000 7f3a38fe1700 1 -- 172.21.15.12:0/2375359693 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a580157c0 con 0x557f04d2a420 2023-10-05T19:40:17.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.535+0000 7f3a6b444e40 1 -- 172.21.15.12:0/2375359693 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557f04d31bd0 con 0x557f04d2a420 2023-10-05T19:40:17.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.535+0000 7f3a38fe1700 1 -- 172.21.15.12:0/2375359693 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a58015ab0 con 0x557f04d2a420 2023-10-05T19:40:17.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.539+0000 7f3a6b444e40 1 -- 172.21.15.12:0/2375359693 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557f04d32090 con 0x557f04d2a420 2023-10-05T19:40:17.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.539+0000 7f3a38fe1700 1 -- 172.21.15.12:0/2375359693 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f3a58007050 con 0x557f04d2a420 2023-10-05T19:40:17.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.539+0000 7f3a38fe1700 1 --2- 172.21.15.12:0/2375359693 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f39b8030c50 0x7f39b8033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.539+0000 7f3a38fe1700 1 -- 172.21.15.12:0/2375359693 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f3a58043fc0 con 0x557f04d2a420 2023-10-05T19:40:17.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.539+0000 7f3a68778700 1 --2- 172.21.15.12:0/2375359693 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f39b8030c50 0x7f39b8033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.539+0000 7f39b27fc700 20 reqs_thread_entry: start 2023-10-05T19:40:17.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.539+0000 7f3a6b444e40 20 try_to_resolve_local_entities(): before: realm_id= realm_name= zonegroup_id= zonegroup_name= zone_id= zone_name=default 2023-10-05T19:40:17.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.539+0000 7f3a6b444e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.539+0000 7f3a68778700 1 --2- 172.21.15.12:0/2375359693 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f39b8030c50 0x7f39b8033100 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3a500044f0 tx=0x7f3a50006560 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.539+0000 7f3a6b444e40 1 --2- 172.21.15.12:0/1599656020 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x557f04d4f330 0x557f04d4f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.543+0000 7f3a6b444e40 1 -- 172.21.15.12:0/1599656020 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x557f04d4fd10 con 0x557f04d4f330 2023-10-05T19:40:17.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.543+0000 7f3a6977a700 1 --2- 172.21.15.12:0/1599656020 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x557f04d4f330 0x557f04d4f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.543+0000 7f3a6977a700 1 --2- 172.21.15.12:0/1599656020 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x557f04d4f330 0x557f04d4f7d0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.543+0000 7f3a6977a700 1 -- 172.21.15.12:0/1599656020 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f3a580207c0 con 0x557f04d4f330 2023-10-05T19:40:17.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.543+0000 7f3a6b444e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:17.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.543+0000 7f3a6b444e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.543+0000 7f3a6b444e40 1 --2- 172.21.15.12:0/1599656020 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x557f04d62c30 0x557f04d65010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.543+0000 7f3a6b444e40 1 -- 172.21.15.12:0/1599656020 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:2 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x557f04d65550 con 0x557f04d62c30 2023-10-05T19:40:17.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.543+0000 7f3a68f79700 1 --2- 172.21.15.12:0/1599656020 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x557f04d62c30 0x557f04d65010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.543+0000 7f3a68f79700 1 --2- 172.21.15.12:0/1599656020 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x557f04d62c30 0x557f04d65010 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.543+0000 7f3a68f79700 1 -- 172.21.15.12:0/1599656020 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(2 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv1 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f3a6004c040 con 0x557f04d62c30 2023-10-05T19:40:17.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.543+0000 7f3a6b444e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:17.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.543+0000 7f3a6b444e40 1 --2- 172.21.15.12:0/1599656020 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x557f04d66e20 0x557f04d69200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.543+0000 7f3a6b444e40 1 -- 172.21.15.12:0/1599656020 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:3 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557f04d69740 con 0x557f04d66e20 2023-10-05T19:40:17.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.543+0000 7f3a68778700 1 --2- 172.21.15.12:0/1599656020 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x557f04d66e20 0x557f04d69200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.547+0000 7f3a68778700 1 --2- 172.21.15.12:0/1599656020 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x557f04d66e20 0x557f04d69200 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.547+0000 7f3a68778700 1 -- 172.21.15.12:0/1599656020 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(3 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f3a50004520 con 0x557f04d66e20 2023-10-05T19:40:17.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.547+0000 7f3a68778700 1 -- 172.21.15.12:0/1599656020 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:4 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557f04d69740 con 0x557f04d62c30 2023-10-05T19:40:17.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.547+0000 7f3a68f79700 1 -- 172.21.15.12:0/1599656020 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(4 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a6004c040 con 0x557f04d62c30 2023-10-05T19:40:17.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.547+0000 7f3a68f79700 1 --2- 172.21.15.12:0/1599656020 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f3a60007030 0x7f3a6004d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.547+0000 7f3a68f79700 1 -- 172.21.15.12:0/1599656020 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:5 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557f04d69740 con 0x7f3a60007030 2023-10-05T19:40:17.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.547+0000 7f3a6977a700 1 --2- 172.21.15.12:0/1599656020 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f3a60007030 0x7f3a6004d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.551+0000 7f3a6977a700 1 --2- 172.21.15.12:0/1599656020 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f3a60007030 0x7f3a6004d640 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.551+0000 7f3a6977a700 1 -- 172.21.15.12:0/1599656020 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(5 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a580207c0 con 0x7f3a60007030 2023-10-05T19:40:17.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.551+0000 7f3a6977a700 1 -- 172.21.15.12:0/1599656020 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:6 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557f04d69740 con 0x557f04d4f330 2023-10-05T19:40:17.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.551+0000 7f3a6977a700 1 -- 172.21.15.12:0/1599656020 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(6 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a580207c0 con 0x557f04d4f330 2023-10-05T19:40:17.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.551+0000 7f3a6977a700 1 -- 172.21.15.12:0/1599656020 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:7 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557f04d69740 con 0x557f04d4f330 2023-10-05T19:40:17.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.551+0000 7f3a6977a700 1 -- 172.21.15.12:0/1599656020 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a580207c0 con 0x557f04d4f330 2023-10-05T19:40:17.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.551+0000 7f3a6977a700 1 -- 172.21.15.12:0/1599656020 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557f04d69740 con 0x7f3a60007030 2023-10-05T19:40:17.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.551+0000 7f3a6977a700 1 -- 172.21.15.12:0/1599656020 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a580207c0 con 0x7f3a60007030 2023-10-05T19:40:17.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.551+0000 7f3a6977a700 1 -- 172.21.15.12:0/1599656020 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:9 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557f04d69740 con 0x7f3a60007030 2023-10-05T19:40:17.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.555+0000 7f3a6977a700 1 -- 172.21.15.12:0/1599656020 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a580207c0 con 0x7f3a60007030 2023-10-05T19:40:17.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.555+0000 7f3a6977a700 1 -- 172.21.15.12:0/1599656020 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:10 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557f04d69740 con 0x557f04d62c30 2023-10-05T19:40:17.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.555+0000 7f3a68f79700 1 -- 172.21.15.12:0/1599656020 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a6004c040 con 0x557f04d62c30 2023-10-05T19:40:17.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.555+0000 7f3a68f79700 1 -- 172.21.15.12:0/1599656020 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:11 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557f04d69740 con 0x557f04d66e20 2023-10-05T19:40:17.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.555+0000 7f3a68778700 1 -- 172.21.15.12:0/1599656020 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a50004520 con 0x557f04d66e20 2023-10-05T19:40:17.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.555+0000 7f3a68778700 1 -- 172.21.15.12:0/1599656020 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:12 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557f04d69740 con 0x557f04d4f330 2023-10-05T19:40:17.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.555+0000 7f3a6977a700 1 -- 172.21.15.12:0/1599656020 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a580207c0 con 0x557f04d4f330 2023-10-05T19:40:17.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.555+0000 7f3a6977a700 1 -- 172.21.15.12:0/1599656020 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:13 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557f04d69740 con 0x7f3a60007030 2023-10-05T19:40:17.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.555+0000 7f3a6977a700 1 -- 172.21.15.12:0/1599656020 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a580207c0 con 0x7f3a60007030 2023-10-05T19:40:17.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.555+0000 7f3a6977a700 1 -- 172.21.15.12:0/1599656020 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:14 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557f04d69740 con 0x557f04d4f330 2023-10-05T19:40:17.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.559+0000 7f3a6977a700 1 -- 172.21.15.12:0/1599656020 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a580207c0 con 0x557f04d4f330 2023-10-05T19:40:17.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.559+0000 7f3a6977a700 1 -- 172.21.15.12:0/1599656020 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:15 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557f04d69740 con 0x557f04d62c30 2023-10-05T19:40:17.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.559+0000 7f3a68f79700 1 -- 172.21.15.12:0/1599656020 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a6004c040 con 0x557f04d62c30 2023-10-05T19:40:17.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.559+0000 7f3a68f79700 1 -- 172.21.15.12:0/1599656020 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:16 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557f04d69740 con 0x557f04d66e20 2023-10-05T19:40:17.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.559+0000 7f3a68778700 1 -- 172.21.15.12:0/1599656020 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a50004520 con 0x557f04d66e20 2023-10-05T19:40:17.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.559+0000 7f3a68778700 1 -- 172.21.15.12:0/1599656020 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:17 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557f04d69740 con 0x557f04d66e20 2023-10-05T19:40:17.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.559+0000 7f3a68778700 1 -- 172.21.15.12:0/1599656020 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a50004520 con 0x557f04d66e20 2023-10-05T19:40:17.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.559+0000 7f3a68778700 1 -- 172.21.15.12:0/1599656020 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:18 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557f04d69740 con 0x7f3a60007030 2023-10-05T19:40:17.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.559+0000 7f3a6977a700 1 -- 172.21.15.12:0/1599656020 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a580207c0 con 0x7f3a60007030 2023-10-05T19:40:17.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.559+0000 7f3a6977a700 1 -- 172.21.15.12:0/1599656020 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557f04d69740 con 0x557f04d66e20 2023-10-05T19:40:17.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.563+0000 7f3a68778700 1 -- 172.21.15.12:0/1599656020 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a50004520 con 0x557f04d66e20 2023-10-05T19:40:17.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.563+0000 7f3a68778700 1 -- 172.21.15.12:0/1599656020 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557f04d69740 con 0x557f04d66e20 2023-10-05T19:40:17.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.563+0000 7f3a68778700 1 -- 172.21.15.12:0/1599656020 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a50004520 con 0x557f04d66e20 2023-10-05T19:40:17.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.563+0000 7f3a68778700 1 -- 172.21.15.12:0/1599656020 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:21 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557f04d69740 con 0x557f04d4f330 2023-10-05T19:40:17.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.563+0000 7f3a6977a700 1 -- 172.21.15.12:0/1599656020 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a580207c0 con 0x557f04d4f330 2023-10-05T19:40:17.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.563+0000 7f3a6977a700 1 -- 172.21.15.12:0/1599656020 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:22 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557f04d69740 con 0x557f04d4f330 2023-10-05T19:40:17.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.563+0000 7f3a6977a700 1 -- 172.21.15.12:0/1599656020 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a580207c0 con 0x557f04d4f330 2023-10-05T19:40:17.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.563+0000 7f3a6977a700 1 -- 172.21.15.12:0/1599656020 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:23 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557f04d69740 con 0x7f3a60007030 2023-10-05T19:40:17.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.563+0000 7f3a6977a700 1 -- 172.21.15.12:0/1599656020 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a580207c0 con 0x7f3a60007030 2023-10-05T19:40:17.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.563+0000 7f3a6977a700 1 -- 172.21.15.12:0/1599656020 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:24 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557f04d69740 con 0x557f04d66e20 2023-10-05T19:40:17.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.567+0000 7f3a68778700 1 -- 172.21.15.12:0/1599656020 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a50004520 con 0x557f04d66e20 2023-10-05T19:40:17.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.567+0000 7f3a68778700 1 -- 172.21.15.12:0/1599656020 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:25 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557f04d69740 con 0x557f04d62c30 2023-10-05T19:40:17.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.567+0000 7f3a68f79700 1 -- 172.21.15.12:0/1599656020 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a6004c040 con 0x557f04d62c30 2023-10-05T19:40:17.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.567+0000 7f3a68f79700 1 -- 172.21.15.12:0/1599656020 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:26 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557f04d69740 con 0x557f04d4f330 2023-10-05T19:40:17.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.567+0000 7f3a6977a700 1 -- 172.21.15.12:0/1599656020 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a580207c0 con 0x557f04d4f330 2023-10-05T19:40:17.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.567+0000 7f3a6977a700 1 -- 172.21.15.12:0/1599656020 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:27 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557f04d69740 con 0x7f3a60007030 2023-10-05T19:40:17.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.567+0000 7f3a6977a700 1 -- 172.21.15.12:0/1599656020 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a58051040 con 0x7f3a60007030 2023-10-05T19:40:17.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.567+0000 7f3a6977a700 1 -- 172.21.15.12:0/1599656020 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:28 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557f04d69740 con 0x557f04d66e20 2023-10-05T19:40:17.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.567+0000 7f3a68778700 1 -- 172.21.15.12:0/1599656020 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a50004520 con 0x557f04d66e20 2023-10-05T19:40:17.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.567+0000 7f3a68778700 1 -- 172.21.15.12:0/1599656020 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:29 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557f04d69740 con 0x7f3a60007030 2023-10-05T19:40:17.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.571+0000 7f3a6977a700 1 -- 172.21.15.12:0/1599656020 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a58051040 con 0x7f3a60007030 2023-10-05T19:40:17.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.571+0000 7f3a6977a700 1 -- 172.21.15.12:0/1599656020 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:30 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557f04d69740 con 0x557f04d62c30 2023-10-05T19:40:17.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.571+0000 7f3a68f79700 1 -- 172.21.15.12:0/1599656020 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a6004c040 con 0x557f04d62c30 2023-10-05T19:40:17.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.571+0000 7f3a68f79700 1 -- 172.21.15.12:0/1599656020 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557f04d69740 con 0x557f04d66e20 2023-10-05T19:40:17.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.571+0000 7f3a68778700 1 -- 172.21.15.12:0/1599656020 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a50004520 con 0x557f04d66e20 2023-10-05T19:40:17.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.571+0000 7f3a68778700 1 -- 172.21.15.12:0/1599656020 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557f04d69740 con 0x7f3a60007030 2023-10-05T19:40:17.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.571+0000 7f3a6977a700 1 -- 172.21.15.12:0/1599656020 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a58051040 con 0x7f3a60007030 2023-10-05T19:40:17.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.571+0000 7f3a6977a700 1 -- 172.21.15.12:0/1599656020 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:33 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557f04d69740 con 0x557f04d4f330 2023-10-05T19:40:17.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.575+0000 7f3a6977a700 1 -- 172.21.15.12:0/1599656020 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(33 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f3a58051040 con 0x557f04d4f330 2023-10-05T19:40:17.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.575+0000 7f3a6977a700 1 -- 172.21.15.12:0/1599656020 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:34 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557f04d69740 con 0x557f04d62c30 2023-10-05T19:40:17.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.575+0000 7f3a68f79700 1 -- 172.21.15.12:0/1599656020 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(34 [pgnls start_epoch 28 out=107b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f3a6004c040 con 0x557f04d62c30 2023-10-05T19:40:17.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.575+0000 7f3a6b444e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:17.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.575+0000 7f3a6b444e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:17.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.575+0000 7f3a6b444e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:17.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.575+0000 7f3a6b444e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:17.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.575+0000 7f3a6b444e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.575+0000 7f3a6b444e40 1 -- 172.21.15.12:0/1599656020 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:35 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x557f04d500e0 con 0x557f04d62c30 2023-10-05T19:40:17.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.575+0000 7f3a68f79700 1 -- 172.21.15.12:0/1599656020 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(35 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv1 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f3a6004c040 con 0x557f04d62c30 2023-10-05T19:40:17.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.575+0000 7f3a6b444e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:17.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.575+0000 7f3a6b444e40 1 -- 172.21.15.12:0/1599656020 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:36 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [delete,create,writefull 0~874 in=874b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x557f04d69af0 con 0x557f04d62c30 2023-10-05T19:40:17.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.579+0000 7f3a68f79700 1 -- 172.21.15.12:0/1599656020 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(36 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [delete,create,writefull 0~874] v28'2 uv2 ondisk = 0) v8 ==== 274+0+0 (crc 0 0 0) 0x7f3a6004c040 con 0x557f04d62c30 2023-10-05T19:40:17.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.579+0000 7f3a6b444e40 1 -- 172.21.15.12:0/1599656020 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f3a60007030 msgr2=0x7f3a6004d640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.579+0000 7f3a6b444e40 1 --2- 172.21.15.12:0/1599656020 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f3a60007030 0x7f3a6004d640 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:17.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.583+0000 7f3a6b444e40 1 -- 172.21.15.12:0/1599656020 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x557f04d4f330 msgr2=0x557f04d4f7d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.583+0000 7f3a6b444e40 1 --2- 172.21.15.12:0/1599656020 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x557f04d4f330 0x557f04d4f7d0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:17.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.583+0000 7f3a6b444e40 1 -- 172.21.15.12:0/1599656020 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x557f04d62c30 msgr2=0x557f04d65010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.583+0000 7f3a6b444e40 1 --2- 172.21.15.12:0/1599656020 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x557f04d62c30 0x557f04d65010 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:17.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.583+0000 7f3a6b444e40 1 -- 172.21.15.12:0/1599656020 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x557f04d66e20 msgr2=0x557f04d69200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.583+0000 7f3a6b444e40 1 --2- 172.21.15.12:0/1599656020 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x557f04d66e20 0x557f04d69200 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:17.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.583+0000 7f3a6b444e40 1 -- 172.21.15.12:0/1599656020 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3a48030ca0 msgr2=0x7f3a48033150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.583+0000 7f3a6b444e40 1 --2- 172.21.15.12:0/1599656020 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3a48030ca0 0x7f3a48033150 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3a600053a0 tx=0x7f3a60030f90 comp rx=0 tx=0).stop 2023-10-05T19:40:17.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.583+0000 7f3a6977a700 1 -- 172.21.15.12:0/1599656020 reap_dead start 2023-10-05T19:40:17.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.583+0000 7f3a6b444e40 1 -- 172.21.15.12:0/1599656020 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f04cf33d0 msgr2=0x557f04d18cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.583+0000 7f3a6b444e40 1 --2- 172.21.15.12:0/1599656020 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f04cf33d0 0x557f04d18cd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3a6001f040 tx=0x7f3a6001c000 comp rx=0 tx=0).stop 2023-10-05T19:40:17.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.583+0000 7f3a6b444e40 1 -- 172.21.15.12:0/1599656020 shutdown_connections 2023-10-05T19:40:17.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.583+0000 7f3a6b444e40 1 --2- 172.21.15.12:0/1599656020 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f04cf33d0 0x557f04d18cd0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:17.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.583+0000 7f3a6b444e40 1 -- 172.21.15.12:0/1599656020 >> 172.21.15.12:0/1599656020 conn(0x557f04c66880 msgr2=0x557f04d0c590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:17.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.583+0000 7f3a6b444e40 1 -- 172.21.15.12:0/1599656020 shutdown_connections 2023-10-05T19:40:17.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.583+0000 7f3a6b444e40 1 -- 172.21.15.12:0/1599656020 wait complete. 2023-10-05T19:40:17.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.583+0000 7f3a6b444e40 1 -- 172.21.15.12:0/2375359693 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f39b8030c50 msgr2=0x7f39b8033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.583+0000 7f3a6b444e40 1 --2- 172.21.15.12:0/2375359693 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f39b8030c50 0x7f39b8033100 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3a500044f0 tx=0x7f3a50006560 comp rx=0 tx=0).stop 2023-10-05T19:40:17.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.583+0000 7f3a6b444e40 1 -- 172.21.15.12:0/2375359693 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f04d2a420 msgr2=0x557f04d315d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.583+0000 7f3a6b444e40 1 --2- 172.21.15.12:0/2375359693 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f04d2a420 0x557f04d315d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x557f04d278e0 tx=0x7f3a58013000 comp rx=0 tx=0).stop 2023-10-05T19:40:17.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.583+0000 7f3a6b444e40 1 -- 172.21.15.12:0/2375359693 shutdown_connections 2023-10-05T19:40:17.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.583+0000 7f3a6b444e40 1 --2- 172.21.15.12:0/2375359693 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f39b8030c50 0x7f39b8033100 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:17.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.583+0000 7f3a6b444e40 1 --2- 172.21.15.12:0/2375359693 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557f04d2a420 0x557f04d315d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:17.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.583+0000 7f3a6b444e40 1 -- 172.21.15.12:0/2375359693 >> 172.21.15.12:0/2375359693 conn(0x557f04cf3ce0 msgr2=0x557f04d26d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:17.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.583+0000 7f3a6b444e40 1 -- 172.21.15.12:0/2375359693 shutdown_connections 2023-10-05T19:40:17.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.583+0000 7f3a6b444e40 1 -- 172.21.15.12:0/2375359693 wait complete. 2023-10-05T19:40:17.594 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:40:17.595 INFO:teuthology.orchestra.run.smithi012.stdout: "id": "93ac49f4-ff49-4f99-811b-6453b28decd5", 2023-10-05T19:40:17.595 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default", 2023-10-05T19:40:17.595 INFO:teuthology.orchestra.run.smithi012.stdout: "domain_root": "default.rgw.meta:root", 2023-10-05T19:40:17.595 INFO:teuthology.orchestra.run.smithi012.stdout: "control_pool": "default.rgw.control", 2023-10-05T19:40:17.595 INFO:teuthology.orchestra.run.smithi012.stdout: "gc_pool": "default.rgw.log:gc", 2023-10-05T19:40:17.595 INFO:teuthology.orchestra.run.smithi012.stdout: "lc_pool": "default.rgw.log:lc", 2023-10-05T19:40:17.595 INFO:teuthology.orchestra.run.smithi012.stdout: "log_pool": "default.rgw.log", 2023-10-05T19:40:17.595 INFO:teuthology.orchestra.run.smithi012.stdout: "intent_log_pool": "default.rgw.log:intent", 2023-10-05T19:40:17.595 INFO:teuthology.orchestra.run.smithi012.stdout: "usage_log_pool": "default.rgw.log:usage", 2023-10-05T19:40:17.595 INFO:teuthology.orchestra.run.smithi012.stdout: "roles_pool": "default.rgw.meta:roles", 2023-10-05T19:40:17.595 INFO:teuthology.orchestra.run.smithi012.stdout: "reshard_pool": "default.rgw.log:reshard", 2023-10-05T19:40:17.595 INFO:teuthology.orchestra.run.smithi012.stdout: "user_keys_pool": "default.rgw.meta:users.keys", 2023-10-05T19:40:17.596 INFO:teuthology.orchestra.run.smithi012.stdout: "user_email_pool": "default.rgw.meta:users.email", 2023-10-05T19:40:17.596 INFO:teuthology.orchestra.run.smithi012.stdout: "user_swift_pool": "default.rgw.meta:users.swift", 2023-10-05T19:40:17.596 INFO:teuthology.orchestra.run.smithi012.stdout: "user_uid_pool": "default.rgw.meta:users.uid", 2023-10-05T19:40:17.596 INFO:teuthology.orchestra.run.smithi012.stdout: "otp_pool": "default.rgw.otp", 2023-10-05T19:40:17.596 INFO:teuthology.orchestra.run.smithi012.stdout: "system_key": { 2023-10-05T19:40:17.596 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key": "", 2023-10-05T19:40:17.596 INFO:teuthology.orchestra.run.smithi012.stdout: "secret_key": "" 2023-10-05T19:40:17.596 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:17.596 INFO:teuthology.orchestra.run.smithi012.stdout: "placement_pools": [ 2023-10-05T19:40:17.596 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:17.596 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "default-placement", 2023-10-05T19:40:17.596 INFO:teuthology.orchestra.run.smithi012.stdout: "val": { 2023-10-05T19:40:17.597 INFO:teuthology.orchestra.run.smithi012.stdout: "index_pool": "default.rgw.buckets.index", 2023-10-05T19:40:17.597 INFO:teuthology.orchestra.run.smithi012.stdout: "storage_classes": { 2023-10-05T19:40:17.597 INFO:teuthology.orchestra.run.smithi012.stdout: "STANDARD": { 2023-10-05T19:40:17.597 INFO:teuthology.orchestra.run.smithi012.stdout: "data_pool": "default.rgw.buckets.data" 2023-10-05T19:40:17.597 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:17.597 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:17.597 INFO:teuthology.orchestra.run.smithi012.stdout: "data_extra_pool": "default.rgw.buckets.non-ec", 2023-10-05T19:40:17.597 INFO:teuthology.orchestra.run.smithi012.stdout: "index_type": 0, 2023-10-05T19:40:17.597 INFO:teuthology.orchestra.run.smithi012.stdout: "inline_data": "true" 2023-10-05T19:40:17.597 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:17.597 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:17.597 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:17.598 INFO:teuthology.orchestra.run.smithi012.stdout: "realm_id": "", 2023-10-05T19:40:17.598 INFO:teuthology.orchestra.run.smithi012.stdout: "notif_pool": "default.rgw.log:notif" 2023-10-05T19:40:17.598 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:40:17.599 DEBUG:tasks.util.rgw: json result: {'id': '93ac49f4-ff49-4f99-811b-6453b28decd5', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': 'true'}}], 'realm_id': '', 'notif_pool': 'default.rgw.log:notif'} 2023-10-05T19:40:17.600 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show'] 2023-10-05T19:40:17.600 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show'] 2023-10-05T19:40:17.600 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show 2023-10-05T19:40:17.639 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:17.639 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:17.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.631+0000 7f3076a9ae40 1 Processor -- start 2023-10-05T19:40:17.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.631+0000 7f3076a9ae40 1 -- start start 2023-10-05T19:40:17.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.631+0000 7f3076a9ae40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x558b4b770bf0 0x558b4b76f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.631+0000 7f3076a9ae40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x558b4b747c00 con 0x558b4b76f710 2023-10-05T19:40:17.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.631+0000 7f3076a9ae40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x558b4b704ee0 con 0x558b4b770bf0 2023-10-05T19:40:17.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.631+0000 7f30745cf700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x558b4b770bf0 0x558b4b76f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.631+0000 7f3073dce700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x558b4b76f710 0x558b4b802f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:47186/0 (socket says 172.21.15.12:47186) 2023-10-05T19:40:17.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.631+0000 7f3073dce700 1 -- 172.21.15.12:0/2848716821 learned_addr learned my addr 172.21.15.12:0/2848716821 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:17.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.635+0000 7f30745cf700 1 -- 172.21.15.12:0/2848716821 >> v1:172.21.15.8:6789/0 conn(0x558b4b76f710 legacy=0x558b4b802f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.635+0000 7f30735cd700 1 -- 172.21.15.12:0/2848716821 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2580047601 0 0) 0x558b4b747c00 con 0x558b4b76f710 2023-10-05T19:40:17.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.635+0000 7f30745cf700 1 -- 172.21.15.12:0/2848716821 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b4b771080 con 0x558b4b770bf0 2023-10-05T19:40:17.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.635+0000 7f30745cf700 1 --2- 172.21.15.12:0/2848716821 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b4b770bf0 0x558b4b76f1d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f306400a5d0 tx=0x7f3064005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=988d653973ae1725 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.635+0000 7f30735cd700 1 -- 172.21.15.12:0/2848716821 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30640086a0 con 0x558b4b770bf0 2023-10-05T19:40:17.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.635+0000 7f30735cd700 1 -- 172.21.15.12:0/2848716821 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3064008800 con 0x558b4b770bf0 2023-10-05T19:40:17.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.635+0000 7f30735cd700 1 -- 172.21.15.12:0/2848716821 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3064008ad0 con 0x558b4b770bf0 2023-10-05T19:40:17.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.635+0000 7f3076a9ae40 1 -- 172.21.15.12:0/2848716821 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b4b770bf0 msgr2=0x558b4b76f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.635+0000 7f3076a9ae40 1 --2- 172.21.15.12:0/2848716821 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b4b770bf0 0x558b4b76f1d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f306400a5d0 tx=0x7f3064005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:17.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.635+0000 7f3076a9ae40 1 -- 172.21.15.12:0/2848716821 shutdown_connections 2023-10-05T19:40:17.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.635+0000 7f3076a9ae40 1 --2- 172.21.15.12:0/2848716821 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b4b770bf0 0x558b4b76f1d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:17.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.635+0000 7f3076a9ae40 1 -- 172.21.15.12:0/2848716821 >> 172.21.15.12:0/2848716821 conn(0x558b4b7f6820 msgr2=0x558b4b7f8c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:17.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.635+0000 7f3076a9ae40 1 -- 172.21.15.12:0/2848716821 shutdown_connections 2023-10-05T19:40:17.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.635+0000 7f3076a9ae40 1 -- 172.21.15.12:0/2848716821 wait complete. 2023-10-05T19:40:17.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.635+0000 7f3076a9ae40 1 Processor -- start 2023-10-05T19:40:17.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.635+0000 7f3076a9ae40 1 -- start start 2023-10-05T19:40:17.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.635+0000 7f3076a9ae40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b4b76f710 0x558b4b81bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.635+0000 7f3076a9ae40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b4b6c8e30 con 0x558b4b76f710 2023-10-05T19:40:17.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.635+0000 7f30745cf700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b4b76f710 0x558b4b81bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.635+0000 7f30745cf700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b4b76f710 0x558b4b81bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36514/0 (socket says 172.21.15.12:36514) 2023-10-05T19:40:17.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.635+0000 7f30745cf700 1 -- 172.21.15.12:0/3121596707 learned_addr learned my addr 172.21.15.12:0/3121596707 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:17.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.639+0000 7f30745cf700 1 -- 172.21.15.12:0/3121596707 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3064005cf0 con 0x558b4b76f710 2023-10-05T19:40:17.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.639+0000 7f30745cf700 1 --2- 172.21.15.12:0/3121596707 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b4b76f710 0x558b4b81bec0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3064000c00 tx=0x7f3064015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c38223271b9e774e server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.639+0000 7f30725cb700 1 -- 172.21.15.12:0/3121596707 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3064008970 con 0x558b4b76f710 2023-10-05T19:40:17.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.639+0000 7f30725cb700 1 -- 172.21.15.12:0/3121596707 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3064008af0 con 0x558b4b76f710 2023-10-05T19:40:17.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.639+0000 7f30725cb700 1 -- 172.21.15.12:0/3121596707 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3064008de0 con 0x558b4b76f710 2023-10-05T19:40:17.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.639+0000 7f3076a9ae40 1 -- 172.21.15.12:0/3121596707 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b4b76f710 msgr2=0x558b4b81bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.639+0000 7f3076a9ae40 1 --2- 172.21.15.12:0/3121596707 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b4b76f710 0x558b4b81bec0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3064000c00 tx=0x7f3064015000 comp rx=0 tx=0).stop 2023-10-05T19:40:17.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.639+0000 7f3076a9ae40 1 -- 172.21.15.12:0/3121596707 shutdown_connections 2023-10-05T19:40:17.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.639+0000 7f3076a9ae40 1 --2- 172.21.15.12:0/3121596707 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b4b76f710 0x558b4b81bec0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:17.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.639+0000 7f3076a9ae40 1 -- 172.21.15.12:0/3121596707 >> 172.21.15.12:0/3121596707 conn(0x558b4b7f6820 msgr2=0x558b4b817a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:17.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.639+0000 7f3076a9ae40 1 -- 172.21.15.12:0/3121596707 shutdown_connections 2023-10-05T19:40:17.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.639+0000 7f3076a9ae40 1 -- 172.21.15.12:0/3121596707 wait complete. 2023-10-05T19:40:17.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.639+0000 7f3076a9ae40 1 Processor -- start 2023-10-05T19:40:17.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.639+0000 7f3076a9ae40 1 -- start start 2023-10-05T19:40:17.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.639+0000 7f3076a9ae40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b4b76f710 0x558b4b822b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.639+0000 7f3076a9ae40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b4b69bed0 con 0x558b4b76f710 2023-10-05T19:40:17.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.643+0000 7f30745cf700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b4b76f710 0x558b4b822b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.643+0000 7f30745cf700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b4b76f710 0x558b4b822b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36524/0 (socket says 172.21.15.12:36524) 2023-10-05T19:40:17.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.643+0000 7f30745cf700 1 -- 172.21.15.12:0/572202354 learned_addr learned my addr 172.21.15.12:0/572202354 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:17.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.643+0000 7f30745cf700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b4b771080 con 0x558b4b76f710 2023-10-05T19:40:17.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.643+0000 7f30745cf700 1 --2- 172.21.15.12:0/572202354 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b4b76f710 0x558b4b822b20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3064000c00 tx=0x7f3064003050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.643+0000 7f3070dc8700 1 -- 172.21.15.12:0/572202354 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3064012b70 con 0x558b4b76f710 2023-10-05T19:40:17.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.643+0000 7f3070dc8700 1 -- 172.21.15.12:0/572202354 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f306402a070 con 0x558b4b76f710 2023-10-05T19:40:17.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.643+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558b4b823060 con 0x558b4b76f710 2023-10-05T19:40:17.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.643+0000 7f3070dc8700 1 -- 172.21.15.12:0/572202354 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3064012e50 con 0x558b4b76f710 2023-10-05T19:40:17.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.643+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558b4b823500 con 0x558b4b76f710 2023-10-05T19:40:17.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.643+0000 7f3070dc8700 1 -- 172.21.15.12:0/572202354 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f3064019070 con 0x558b4b76f710 2023-10-05T19:40:17.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.647+0000 7f3070dc8700 1 --2- 172.21.15.12:0/572202354 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3054030bf0 0x7f30540330a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.647+0000 7f3070dc8700 1 -- 172.21.15.12:0/572202354 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f306402c070 con 0x558b4b76f710 2023-10-05T19:40:17.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.647+0000 7f3076a9ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.647+0000 7f3076a9ae40 1 --2- 172.21.15.12:0/572202354 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x558b4b770bf0 0x558b4b82e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.647+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b82ec70 con 0x558b4b770bf0 2023-10-05T19:40:17.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.647+0000 7f3073dce700 1 --2- 172.21.15.12:0/572202354 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3054030bf0 0x7f30540330a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.647+0000 7f3074dd0700 1 --2- 172.21.15.12:0/572202354 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x558b4b770bf0 0x558b4b82e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.647+0000 7f3073dce700 1 --2- 172.21.15.12:0/572202354 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3054030bf0 0x7f30540330a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f305c001560 tx=0x7f305c006650 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.647+0000 7f3074dd0700 1 --2- 172.21.15.12:0/572202354 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x558b4b770bf0 0x558b4b82e5c0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.647+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f306c00a500 con 0x558b4b770bf0 2023-10-05T19:40:17.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.647+0000 7f3076a9ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:17.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.647+0000 7f3076a9ae40 20 realm 2023-10-05T19:40:17.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.647+0000 7f3076a9ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.647+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b82bc20 con 0x558b4b770bf0 2023-10-05T19:40:17.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.647+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f306c00a500 con 0x558b4b770bf0 2023-10-05T19:40:17.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.647+0000 7f3076a9ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:17.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.647+0000 7f3076a9ae40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:17.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.647+0000 7f3076a9ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.647+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b82f750 con 0x558b4b770bf0 2023-10-05T19:40:17.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.647+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f306c00a500 con 0x558b4b770bf0 2023-10-05T19:40:17.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.647+0000 7f3076a9ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:17.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.647+0000 7f3076a9ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.647+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b82f330 con 0x558b4b770bf0 2023-10-05T19:40:17.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.647+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f306c00a500 con 0x558b4b770bf0 2023-10-05T19:40:17.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.647+0000 7f3076a9ae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:17.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.647+0000 7f3076a9ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.647+0000 7f3076a9ae40 1 --2- 172.21.15.12:0/572202354 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x558b4b8306e0 0x558b4b832ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.647+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b833230 con 0x558b4b8306e0 2023-10-05T19:40:17.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.651+0000 7f30745cf700 1 --2- 172.21.15.12:0/572202354 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x558b4b8306e0 0x558b4b832ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.651+0000 7f30745cf700 1 --2- 172.21.15.12:0/572202354 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x558b4b8306e0 0x558b4b832ac0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.651+0000 7f30745cf700 1 -- 172.21.15.12:0/572202354 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f3064022020 con 0x558b4b8306e0 2023-10-05T19:40:17.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.651+0000 7f3076a9ae40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:17.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.651+0000 7f3076a9ae40 20 searching for the correct realm 2023-10-05T19:40:17.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.651+0000 7f3076a9ae40 1 --2- 172.21.15.12:0/572202354 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x558b4b835060 0x558b4b837440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.651+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b837980 con 0x558b4b835060 2023-10-05T19:40:17.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.651+0000 7f3073dce700 1 --2- 172.21.15.12:0/572202354 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x558b4b835060 0x558b4b837440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.651+0000 7f3073dce700 1 --2- 172.21.15.12:0/572202354 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x558b4b835060 0x558b4b837440 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.655+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f305c004520 con 0x558b4b835060 2023-10-05T19:40:17.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.655+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b837980 con 0x558b4b8306e0 2023-10-05T19:40:17.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.655+0000 7f30745cf700 1 -- 172.21.15.12:0/572202354 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3064022020 con 0x558b4b8306e0 2023-10-05T19:40:17.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.655+0000 7f30745cf700 1 --2- 172.21.15.12:0/572202354 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f3064007030 0x7f3064026880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.655+0000 7f30745cf700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b837980 con 0x7f3064007030 2023-10-05T19:40:17.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.655+0000 7f3074dd0700 1 --2- 172.21.15.12:0/572202354 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f3064007030 0x7f3064026880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.655+0000 7f3074dd0700 1 --2- 172.21.15.12:0/572202354 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f3064007030 0x7f3064026880 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.659+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f306c014040 con 0x7f3064007030 2023-10-05T19:40:17.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.659+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b837980 con 0x558b4b770bf0 2023-10-05T19:40:17.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.659+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f306c014040 con 0x558b4b770bf0 2023-10-05T19:40:17.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.659+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b837980 con 0x558b4b770bf0 2023-10-05T19:40:17.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.659+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f306c014040 con 0x558b4b770bf0 2023-10-05T19:40:17.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.659+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b837980 con 0x7f3064007030 2023-10-05T19:40:17.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.659+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f306c014040 con 0x7f3064007030 2023-10-05T19:40:17.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.659+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b837980 con 0x7f3064007030 2023-10-05T19:40:17.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.663+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f306c014040 con 0x7f3064007030 2023-10-05T19:40:17.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.663+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b837980 con 0x558b4b8306e0 2023-10-05T19:40:17.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.663+0000 7f30745cf700 1 -- 172.21.15.12:0/572202354 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3064022020 con 0x558b4b8306e0 2023-10-05T19:40:17.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.663+0000 7f30745cf700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b837980 con 0x558b4b835060 2023-10-05T19:40:17.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.663+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f305c004520 con 0x558b4b835060 2023-10-05T19:40:17.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.663+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b837980 con 0x558b4b770bf0 2023-10-05T19:40:17.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.663+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f306c014040 con 0x558b4b770bf0 2023-10-05T19:40:17.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.663+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b837980 con 0x7f3064007030 2023-10-05T19:40:17.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.663+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f306c014040 con 0x7f3064007030 2023-10-05T19:40:17.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.663+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b837980 con 0x558b4b770bf0 2023-10-05T19:40:17.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.667+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f306c014040 con 0x558b4b770bf0 2023-10-05T19:40:17.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.667+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b837980 con 0x558b4b8306e0 2023-10-05T19:40:17.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.667+0000 7f30745cf700 1 -- 172.21.15.12:0/572202354 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3064022020 con 0x558b4b8306e0 2023-10-05T19:40:17.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.667+0000 7f30745cf700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b837980 con 0x558b4b835060 2023-10-05T19:40:17.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.667+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f305c004520 con 0x558b4b835060 2023-10-05T19:40:17.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.667+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b837980 con 0x558b4b835060 2023-10-05T19:40:17.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.667+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f305c004520 con 0x558b4b835060 2023-10-05T19:40:17.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.667+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b837980 con 0x7f3064007030 2023-10-05T19:40:17.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.667+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f306c014040 con 0x7f3064007030 2023-10-05T19:40:17.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.667+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b837980 con 0x558b4b835060 2023-10-05T19:40:17.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.667+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f305c004520 con 0x558b4b835060 2023-10-05T19:40:17.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.671+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b837980 con 0x558b4b835060 2023-10-05T19:40:17.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.671+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f305c004520 con 0x558b4b835060 2023-10-05T19:40:17.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.671+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b837980 con 0x558b4b770bf0 2023-10-05T19:40:17.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.671+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f306c014040 con 0x558b4b770bf0 2023-10-05T19:40:17.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.671+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b837980 con 0x558b4b770bf0 2023-10-05T19:40:17.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.671+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f306c014040 con 0x558b4b770bf0 2023-10-05T19:40:17.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.671+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b837980 con 0x7f3064007030 2023-10-05T19:40:17.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.671+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f306c014040 con 0x7f3064007030 2023-10-05T19:40:17.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.671+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b837980 con 0x558b4b835060 2023-10-05T19:40:17.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.671+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f305c004520 con 0x558b4b835060 2023-10-05T19:40:17.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.671+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b837980 con 0x558b4b8306e0 2023-10-05T19:40:17.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.675+0000 7f30745cf700 1 -- 172.21.15.12:0/572202354 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3064022020 con 0x558b4b8306e0 2023-10-05T19:40:17.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.675+0000 7f30745cf700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b837980 con 0x558b4b770bf0 2023-10-05T19:40:17.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.675+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f306c014040 con 0x558b4b770bf0 2023-10-05T19:40:17.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.675+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b837980 con 0x7f3064007030 2023-10-05T19:40:17.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.675+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f306c014040 con 0x7f3064007030 2023-10-05T19:40:17.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.675+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b837980 con 0x558b4b835060 2023-10-05T19:40:17.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.675+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f305c004520 con 0x558b4b835060 2023-10-05T19:40:17.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.675+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b837980 con 0x7f3064007030 2023-10-05T19:40:17.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.679+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f306c014040 con 0x7f3064007030 2023-10-05T19:40:17.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.679+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b837980 con 0x558b4b8306e0 2023-10-05T19:40:17.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.679+0000 7f30745cf700 1 -- 172.21.15.12:0/572202354 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3064022020 con 0x558b4b8306e0 2023-10-05T19:40:17.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.679+0000 7f30745cf700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b837980 con 0x558b4b835060 2023-10-05T19:40:17.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.679+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f305c004520 con 0x558b4b835060 2023-10-05T19:40:17.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.679+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b837980 con 0x7f3064007030 2023-10-05T19:40:17.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.679+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f306c014040 con 0x7f3064007030 2023-10-05T19:40:17.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.679+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b837980 con 0x558b4b770bf0 2023-10-05T19:40:17.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.683+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f306c014040 con 0x558b4b770bf0 2023-10-05T19:40:17.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.683+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b837980 con 0x558b4b8306e0 2023-10-05T19:40:17.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.683+0000 7f30745cf700 1 -- 172.21.15.12:0/572202354 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f3064022020 con 0x558b4b8306e0 2023-10-05T19:40:17.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.683+0000 7f3076a9ae40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:17.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.683+0000 7f3076a9ae40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:17.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.683+0000 7f3076a9ae40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:17.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.683+0000 7f3076a9ae40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:17.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.683+0000 7f3076a9ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.683+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b833de0 con 0x558b4b770bf0 2023-10-05T19:40:17.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.683+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f306c014040 con 0x558b4b770bf0 2023-10-05T19:40:17.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.683+0000 7f3076a9ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:17.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.683+0000 7f3076a9ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.683+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b834730 con 0x558b4b770bf0 2023-10-05T19:40:17.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.683+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f306c014040 con 0x558b4b770bf0 2023-10-05T19:40:17.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.683+0000 7f3076a9ae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:17.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.683+0000 7f3076a9ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.683+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8339a0 con 0x558b4b835060 2023-10-05T19:40:17.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.687+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f305c004520 con 0x558b4b835060 2023-10-05T19:40:17.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.687+0000 7f3076a9ae40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:17.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.687+0000 7f3076a9ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.687+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b838220 con 0x558b4b835060 2023-10-05T19:40:17.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.687+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f305c004520 con 0x558b4b835060 2023-10-05T19:40:17.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.687+0000 7f3076a9ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:17.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.687+0000 7f3076a9ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.687+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8385d0 con 0x558b4b770bf0 2023-10-05T19:40:17.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.687+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f306c014040 con 0x558b4b770bf0 2023-10-05T19:40:17.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.687+0000 7f3076a9ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:17.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.687+0000 7f3076a9ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.687+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b838980 con 0x558b4b770bf0 2023-10-05T19:40:17.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.687+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f306c014040 con 0x558b4b770bf0 2023-10-05T19:40:17.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.687+0000 7f3076a9ae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:17.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.687+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b839990 con 0x558b4b835060 2023-10-05T19:40:17.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.691+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f305c004520 con 0x558b4b835060 2023-10-05T19:40:17.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.691+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b839990 con 0x558b4b8306e0 2023-10-05T19:40:17.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.691+0000 7f30745cf700 1 -- 172.21.15.12:0/572202354 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3064022020 con 0x558b4b8306e0 2023-10-05T19:40:17.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.691+0000 7f30745cf700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b839990 con 0x7f3064007030 2023-10-05T19:40:17.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.691+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f306c014040 con 0x7f3064007030 2023-10-05T19:40:17.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.691+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8385d0 con 0x558b4b770bf0 2023-10-05T19:40:17.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.691+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f306c014040 con 0x558b4b770bf0 2023-10-05T19:40:17.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.691+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8385d0 con 0x558b4b770bf0 2023-10-05T19:40:17.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.695+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f306c014040 con 0x558b4b770bf0 2023-10-05T19:40:17.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.695+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8385d0 con 0x7f3064007030 2023-10-05T19:40:17.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.695+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f306c014040 con 0x7f3064007030 2023-10-05T19:40:17.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.695+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8385d0 con 0x7f3064007030 2023-10-05T19:40:17.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.695+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f306c014040 con 0x7f3064007030 2023-10-05T19:40:17.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.695+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8385d0 con 0x558b4b8306e0 2023-10-05T19:40:17.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.695+0000 7f30745cf700 1 -- 172.21.15.12:0/572202354 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3064022020 con 0x558b4b8306e0 2023-10-05T19:40:17.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.695+0000 7f30745cf700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8385d0 con 0x558b4b835060 2023-10-05T19:40:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.699+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f305c02d040 con 0x558b4b835060 2023-10-05T19:40:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.699+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8385d0 con 0x558b4b770bf0 2023-10-05T19:40:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.699+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f306c014040 con 0x558b4b770bf0 2023-10-05T19:40:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.699+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8385d0 con 0x7f3064007030 2023-10-05T19:40:17.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.699+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f306c014040 con 0x7f3064007030 2023-10-05T19:40:17.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.699+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8385d0 con 0x558b4b770bf0 2023-10-05T19:40:17.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.699+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f306c014040 con 0x558b4b770bf0 2023-10-05T19:40:17.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.699+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8385d0 con 0x558b4b8306e0 2023-10-05T19:40:17.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.699+0000 7f30745cf700 1 -- 172.21.15.12:0/572202354 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3064022020 con 0x558b4b8306e0 2023-10-05T19:40:17.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.699+0000 7f30745cf700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8385d0 con 0x558b4b835060 2023-10-05T19:40:17.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.703+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f305c02d040 con 0x558b4b835060 2023-10-05T19:40:17.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.703+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8385d0 con 0x558b4b835060 2023-10-05T19:40:17.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.703+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f305c02d040 con 0x558b4b835060 2023-10-05T19:40:17.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.703+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8385d0 con 0x7f3064007030 2023-10-05T19:40:17.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.703+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f306c014040 con 0x7f3064007030 2023-10-05T19:40:17.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.703+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8385d0 con 0x558b4b835060 2023-10-05T19:40:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.703+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f305c02d040 con 0x558b4b835060 2023-10-05T19:40:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.703+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f305c031040 con 0x558b4b835060 2023-10-05T19:40:17.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.707+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f305c02d040 con 0x558b4b835060 2023-10-05T19:40:17.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.707+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f305c031040 con 0x558b4b770bf0 2023-10-05T19:40:17.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.707+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f306c014040 con 0x558b4b770bf0 2023-10-05T19:40:17.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.707+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f305c031040 con 0x558b4b770bf0 2023-10-05T19:40:17.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.707+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f306c014040 con 0x558b4b770bf0 2023-10-05T19:40:17.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.707+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f305c031040 con 0x7f3064007030 2023-10-05T19:40:17.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.707+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f306c014040 con 0x7f3064007030 2023-10-05T19:40:17.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.707+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f305c031040 con 0x558b4b835060 2023-10-05T19:40:17.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.707+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f305c02d040 con 0x558b4b835060 2023-10-05T19:40:17.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.707+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f305c031040 con 0x558b4b8306e0 2023-10-05T19:40:17.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.711+0000 7f30745cf700 1 -- 172.21.15.12:0/572202354 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3064022020 con 0x558b4b8306e0 2023-10-05T19:40:17.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.711+0000 7f30745cf700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f305c031040 con 0x558b4b770bf0 2023-10-05T19:40:17.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.711+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f306c014040 con 0x558b4b770bf0 2023-10-05T19:40:17.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.711+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f305c031040 con 0x7f3064007030 2023-10-05T19:40:17.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.711+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f306c014040 con 0x7f3064007030 2023-10-05T19:40:17.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.711+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f305c031040 con 0x558b4b835060 2023-10-05T19:40:17.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.711+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f305c02d040 con 0x558b4b835060 2023-10-05T19:40:17.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.711+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f305c031040 con 0x7f3064007030 2023-10-05T19:40:17.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.715+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f306c014040 con 0x7f3064007030 2023-10-05T19:40:17.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.715+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f305c031040 con 0x558b4b8306e0 2023-10-05T19:40:17.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.715+0000 7f30745cf700 1 -- 172.21.15.12:0/572202354 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3064022020 con 0x558b4b8306e0 2023-10-05T19:40:17.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.715+0000 7f30745cf700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f305c031040 con 0x558b4b835060 2023-10-05T19:40:17.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.715+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f305c02d040 con 0x558b4b835060 2023-10-05T19:40:17.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.715+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f305c031040 con 0x7f3064007030 2023-10-05T19:40:17.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.715+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f306c014040 con 0x7f3064007030 2023-10-05T19:40:17.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.719+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f305c031040 con 0x558b4b770bf0 2023-10-05T19:40:17.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.719+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f306c014040 con 0x558b4b770bf0 2023-10-05T19:40:17.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.719+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f305c031040 con 0x558b4b8306e0 2023-10-05T19:40:17.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.719+0000 7f30745cf700 1 -- 172.21.15.12:0/572202354 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f3064022020 con 0x558b4b8306e0 2023-10-05T19:40:17.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.719+0000 7f3076a9ae40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:17.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.719+0000 7f3076a9ae40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:17.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.719+0000 7f3076a9ae40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:17.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.719+0000 7f3076a9ae40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:17.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.719+0000 7f3076a9ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.719+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b838980 con 0x558b4b770bf0 2023-10-05T19:40:17.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.719+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f306c014040 con 0x558b4b770bf0 2023-10-05T19:40:17.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.719+0000 7f3076a9ae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:17.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.719+0000 7f3076a9ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.719+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b838d30 con 0x558b4b8306e0 2023-10-05T19:40:17.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.723+0000 7f30745cf700 1 -- 172.21.15.12:0/572202354 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f3064022020 con 0x558b4b8306e0 2023-10-05T19:40:17.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.723+0000 7f3076a9ae40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:17.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.723+0000 7f3076a9ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.723+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8395d0 con 0x558b4b770bf0 2023-10-05T19:40:17.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.723+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f306c014040 con 0x558b4b770bf0 2023-10-05T19:40:17.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.723+0000 7f3076a9ae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:17.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.723+0000 7f3076a9ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.723+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8395d0 con 0x558b4b835060 2023-10-05T19:40:17.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.723+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f305c02d040 con 0x558b4b835060 2023-10-05T19:40:17.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.723+0000 7f3076a9ae40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:17.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.723+0000 7f3076a9ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.723+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b839160 con 0x7f3064007030 2023-10-05T19:40:17.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.723+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f306c014040 con 0x7f3064007030 2023-10-05T19:40:17.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.723+0000 7f3076a9ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:17.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.723+0000 7f3076a9ae40 20 zone default found 2023-10-05T19:40:17.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.723+0000 7f3076a9ae40 4 Realm: () 2023-10-05T19:40:17.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.723+0000 7f3076a9ae40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:17.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.723+0000 7f3076a9ae40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:17.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.723+0000 7f3076a9ae40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:17.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.723+0000 7f3076a9ae40 20 zonegroup default 2023-10-05T19:40:17.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.723+0000 7f3076a9ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.723+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b839160 con 0x7f3064007030 2023-10-05T19:40:17.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.727+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f306c014040 con 0x7f3064007030 2023-10-05T19:40:17.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.727+0000 7f3076a9ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:17.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.727+0000 7f3076a9ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.727+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b839160 con 0x558b4b835060 2023-10-05T19:40:17.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.727+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f305c02d040 con 0x558b4b835060 2023-10-05T19:40:17.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.727+0000 7f3076a9ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:17.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.727+0000 7f3076a9ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.727+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b83b7f0 con 0x558b4b835060 2023-10-05T19:40:17.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.727+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f305c02d040 con 0x558b4b835060 2023-10-05T19:40:17.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.727+0000 7f3076a9ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:17.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.727+0000 7f3076a9ae40 20 started sync module instance, tier type = 2023-10-05T19:40:17.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.727+0000 7f3076a9ae40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:17.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.727+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b83aeb0 con 0x558b4b8306e0 2023-10-05T19:40:17.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.731+0000 7f30745cf700 1 -- 172.21.15.12:0/572202354 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'18 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3064022020 con 0x558b4b8306e0 2023-10-05T19:40:17.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.731+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94056755737264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b83aae0 con 0x558b4b8306e0 2023-10-05T19:40:17.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.731+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b83c580 con 0x558b4b770bf0 2023-10-05T19:40:17.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.735+0000 7f30745cf700 1 -- 172.21.15.12:0/572202354 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94056755737264] v28'19 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3064022020 con 0x558b4b8306e0 2023-10-05T19:40:17.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.735+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'18 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f306c014040 con 0x558b4b770bf0 2023-10-05T19:40:17.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.735+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94056755739552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b83c4e0 con 0x558b4b770bf0 2023-10-05T19:40:17.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.735+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b83cef0 con 0x558b4b835060 2023-10-05T19:40:17.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.735+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'18 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f305c02d040 con 0x558b4b835060 2023-10-05T19:40:17.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.735+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94056755743232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b83d790 con 0x558b4b835060 2023-10-05T19:40:17.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.739+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b83e070 con 0x558b4b8306e0 2023-10-05T19:40:17.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.739+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94056755739552] v28'19 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f306c014040 con 0x558b4b770bf0 2023-10-05T19:40:17.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.739+0000 7f30745cf700 1 -- 172.21.15.12:0/572202354 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'18 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3064022020 con 0x558b4b8306e0 2023-10-05T19:40:17.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.739+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94056755740656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b83dc80 con 0x558b4b8306e0 2023-10-05T19:40:17.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.739+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b83ead0 con 0x558b4b770bf0 2023-10-05T19:40:17.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.739+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94056755743232] v28'19 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f305c02d040 con 0x558b4b835060 2023-10-05T19:40:17.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.743+0000 7f30745cf700 1 -- 172.21.15.12:0/572202354 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94056755740656] v28'19 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3064022020 con 0x558b4b8306e0 2023-10-05T19:40:17.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.743+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'18 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f306c014040 con 0x558b4b770bf0 2023-10-05T19:40:17.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.743+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94056755746000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b83e5e0 con 0x558b4b770bf0 2023-10-05T19:40:17.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.743+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b83ee80 con 0x7f3064007030 2023-10-05T19:40:17.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.747+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'18 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f306c014040 con 0x7f3064007030 2023-10-05T19:40:17.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.747+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94056755746000] v28'19 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f306c014040 con 0x558b4b770bf0 2023-10-05T19:40:17.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.747+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94056755750928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b83f150 con 0x7f3064007030 2023-10-05T19:40:17.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.747+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b83fb00 con 0x7f3064007030 2023-10-05T19:40:17.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.747+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(96 notify.6 [create] v28'18 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f306c014040 con 0x7f3064007030 2023-10-05T19:40:17.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.747+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94056755754512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b83f150 con 0x7f3064007030 2023-10-05T19:40:17.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.747+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b840670 con 0x7f3064007030 2023-10-05T19:40:17.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.751+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(95 notify.5 [watch watch cookie 94056755750928] v28'19 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f306c014040 con 0x7f3064007030 2023-10-05T19:40:17.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.751+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(98 notify.7 [create] v28'18 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f306c014040 con 0x7f3064007030 2023-10-05T19:40:17.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.751+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94056755752032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8408a0 con 0x7f3064007030 2023-10-05T19:40:17.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.751+0000 7f3076a9ae40 20 add_watcher() i=0 2023-10-05T19:40:17.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.751+0000 7f3076a9ae40 20 add_watcher() i=1 2023-10-05T19:40:17.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.751+0000 7f3076a9ae40 20 add_watcher() i=2 2023-10-05T19:40:17.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.751+0000 7f3076a9ae40 20 add_watcher() i=3 2023-10-05T19:40:17.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.751+0000 7f3076a9ae40 20 add_watcher() i=4 2023-10-05T19:40:17.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.751+0000 7f3076a9ae40 20 add_watcher() i=5 2023-10-05T19:40:17.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.751+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(97 notify.6 [watch watch cookie 94056755754512] v28'19 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f306c014040 con 0x7f3064007030 2023-10-05T19:40:17.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.751+0000 7f3076a9ae40 20 add_watcher() i=6 2023-10-05T19:40:17.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.755+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94056755752032] v28'19 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f306c014040 con 0x7f3064007030 2023-10-05T19:40:17.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.755+0000 7f3076a9ae40 20 add_watcher() i=7 2023-10-05T19:40:17.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.755+0000 7f3076a9ae40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:17.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.755+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8432d0 con 0x558b4b770bf0 2023-10-05T19:40:17.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.755+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f306c014040 con 0x558b4b770bf0 2023-10-05T19:40:17.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.755+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94056755755616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b840790 con 0x558b4b770bf0 2023-10-05T19:40:17.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.755+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94056755755616] v28'46 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f306c014040 con 0x558b4b770bf0 2023-10-05T19:40:17.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.759+0000 7f3045fe3700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:17.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.759+0000 7f3076a9ae40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:17.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.759+0000 7f3076a9ae40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:17.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.759+0000 7f3076a9ae40 1 Processor -- start 2023-10-05T19:40:17.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.759+0000 7f3076a9ae40 1 -- start start 2023-10-05T19:40:17.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.759+0000 7f3076a9ae40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b4b84d7b0 0x558b4b84db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.759+0000 7f3076a9ae40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b4b84e0c0 con 0x558b4b84d7b0 2023-10-05T19:40:17.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.759+0000 7f3073dce700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b4b84d7b0 0x558b4b84db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.759+0000 7f3073dce700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b4b84d7b0 0x558b4b84db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36532/0 (socket says 172.21.15.12:36532) 2023-10-05T19:40:17.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.759+0000 7f3073dce700 1 -- 172.21.15.12:0/1981222838 learned_addr learned my addr 172.21.15.12:0/1981222838 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:17.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.759+0000 7f3073dce700 1 -- 172.21.15.12:0/1981222838 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b4b84e200 con 0x558b4b84d7b0 2023-10-05T19:40:17.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.759+0000 7f3073dce700 1 --2- 172.21.15.12:0/1981222838 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b4b84d7b0 0x558b4b84db80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x558b4b82fdc0 tx=0x7f305c027040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3295e7fe550d0241 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.759+0000 7f30457e2700 1 -- 172.21.15.12:0/1981222838 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f305c047070 con 0x558b4b84d7b0 2023-10-05T19:40:17.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.759+0000 7f30457e2700 1 -- 172.21.15.12:0/1981222838 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f305c039040 con 0x558b4b84d7b0 2023-10-05T19:40:17.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.759+0000 7f30457e2700 1 -- 172.21.15.12:0/1981222838 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f305c038040 con 0x558b4b84d7b0 2023-10-05T19:40:17.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.759+0000 7f3076a9ae40 1 -- 172.21.15.12:0/1981222838 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b4b84d7b0 msgr2=0x558b4b84db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.759+0000 7f3076a9ae40 1 --2- 172.21.15.12:0/1981222838 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b4b84d7b0 0x558b4b84db80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x558b4b82fdc0 tx=0x7f305c027040 comp rx=0 tx=0).stop 2023-10-05T19:40:17.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.763+0000 7f3076a9ae40 1 -- 172.21.15.12:0/1981222838 shutdown_connections 2023-10-05T19:40:17.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.763+0000 7f3076a9ae40 1 --2- 172.21.15.12:0/1981222838 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b4b84d7b0 0x558b4b84db80 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:17.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.763+0000 7f3076a9ae40 1 -- 172.21.15.12:0/1981222838 >> 172.21.15.12:0/1981222838 conn(0x558b4b848b70 msgr2=0x558b4b846d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:17.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.763+0000 7f3076a9ae40 1 -- 172.21.15.12:0/1981222838 shutdown_connections 2023-10-05T19:40:17.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.763+0000 7f3076a9ae40 1 -- 172.21.15.12:0/1981222838 wait complete. 2023-10-05T19:40:17.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.763+0000 7f3076a9ae40 1 Processor -- start 2023-10-05T19:40:17.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.763+0000 7f3076a9ae40 1 -- start start 2023-10-05T19:40:17.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.763+0000 7f3076a9ae40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b4b84d7b0 0x558b4b853280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.763+0000 7f3076a9ae40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b4b854280 con 0x558b4b84d7b0 2023-10-05T19:40:17.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.763+0000 7f3073dce700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b4b84d7b0 0x558b4b853280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.763+0000 7f3073dce700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b4b84d7b0 0x558b4b853280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36534/0 (socket says 172.21.15.12:36534) 2023-10-05T19:40:17.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.763+0000 7f3073dce700 1 -- 172.21.15.12:0/512643147 learned_addr learned my addr 172.21.15.12:0/512643147 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:17.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.763+0000 7f3073dce700 1 -- 172.21.15.12:0/512643147 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f305c004d90 con 0x558b4b84d7b0 2023-10-05T19:40:17.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.763+0000 7f3073dce700 1 --2- 172.21.15.12:0/512643147 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b4b84d7b0 0x558b4b853280 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f305c0062a0 tx=0x7f305c042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.763+0000 7f2fbf7fe700 1 -- 172.21.15.12:0/512643147 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f305c038030 con 0x558b4b84d7b0 2023-10-05T19:40:17.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.763+0000 7f2fbf7fe700 1 -- 172.21.15.12:0/512643147 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f305c01b040 con 0x558b4b84d7b0 2023-10-05T19:40:17.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.763+0000 7f3076a9ae40 1 -- 172.21.15.12:0/512643147 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558b4b853880 con 0x558b4b84d7b0 2023-10-05T19:40:17.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.763+0000 7f2fbf7fe700 1 -- 172.21.15.12:0/512643147 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f305c02a040 con 0x558b4b84d7b0 2023-10-05T19:40:17.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.763+0000 7f3076a9ae40 1 -- 172.21.15.12:0/512643147 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558b4b853d20 con 0x558b4b84d7b0 2023-10-05T19:40:17.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.767+0000 7f2fbf7fe700 1 -- 172.21.15.12:0/512643147 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f305c04a020 con 0x558b4b84d7b0 2023-10-05T19:40:17.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.767+0000 7f2fbf7fe700 1 --2- 172.21.15.12:0/512643147 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2fb4030c00 0x7f2fb40330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.767+0000 7f2fbf7fe700 1 -- 172.21.15.12:0/512643147 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f305c028040 con 0x558b4b84d7b0 2023-10-05T19:40:17.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.767+0000 7f3076a9ae40 5 note: GC not initialized 2023-10-05T19:40:17.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.767+0000 7f3074dd0700 1 --2- 172.21.15.12:0/512643147 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2fb4030c00 0x7f2fb40330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.767+0000 7f3074dd0700 1 --2- 172.21.15.12:0/512643147 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2fb4030c00 0x7f2fb40330b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x558b4b83dd30 tx=0x7f306c00ab80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.771+0000 7f3076a9ae40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:17.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.771+0000 7f2fb27fc700 20 reqs_thread_entry: start 2023-10-05T19:40:17.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.771+0000 7f2fbcff9700 20 reqs_thread_entry: start 2023-10-05T19:40:17.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.771+0000 7f3076a9ae40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:17.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.771+0000 7f2f9ffff700 20 reqs_thread_entry: start 2023-10-05T19:40:17.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.771+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 6.e 6:74cd889a:usage::usage.0:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8bc440 con 0x558b4b770bf0 2023-10-05T19:40:17.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.771+0000 7f2fb0ff9700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:47 2023 2023-10-05T19:40:17.779 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:17.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.771+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(102 usage.0 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f306c014040 con 0x558b4b770bf0 2023-10-05T19:40:17.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.771+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:103 6.3 6:c0e78a64:usage::usage.1:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8bc440 con 0x558b4b8306e0 2023-10-05T19:40:17.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.771+0000 7f30745cf700 1 -- 172.21.15.12:0/572202354 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(103 usage.1 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f3064022020 con 0x558b4b8306e0 2023-10-05T19:40:17.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.771+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:104 6.d 6:b564efa4:usage::usage.2:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8bc440 con 0x558b4b770bf0 2023-10-05T19:40:17.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.771+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(104 usage.2 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f306c014040 con 0x558b4b770bf0 2023-10-05T19:40:17.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.771+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:105 6.2 6:42d64401:usage::usage.3:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8bc440 con 0x558b4b835060 2023-10-05T19:40:17.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.775+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(105 usage.3 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f305c03b070 con 0x558b4b835060 2023-10-05T19:40:17.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.775+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:106 6.7 6:e09b6f01:usage::usage.4:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8bc440 con 0x558b4b835060 2023-10-05T19:40:17.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.775+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 <== osd.3 v2:172.21.15.12:6816/29951 27 ==== osd_op_reply(106 usage.4 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f305c03b070 con 0x558b4b835060 2023-10-05T19:40:17.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.775+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:107 6.6 6:672810f1:usage::usage.5:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8bc440 con 0x558b4b835060 2023-10-05T19:40:17.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.775+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 <== osd.3 v2:172.21.15.12:6816/29951 28 ==== osd_op_reply(107 usage.5 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f305c03b070 con 0x558b4b835060 2023-10-05T19:40:17.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.775+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:108 6.4 6:25a86931:usage::usage.6:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8bc440 con 0x558b4b770bf0 2023-10-05T19:40:17.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.775+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(108 usage.6 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f306c014040 con 0x558b4b770bf0 2023-10-05T19:40:17.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.775+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:109 6.17 6:ebbe7322:usage::usage.7:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8bc440 con 0x558b4b770bf0 2023-10-05T19:40:17.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.775+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(109 usage.7 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f306c014040 con 0x558b4b770bf0 2023-10-05T19:40:17.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.775+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:110 6.6 6:607e1922:usage::usage.8:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8bc440 con 0x558b4b835060 2023-10-05T19:40:17.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.775+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 <== osd.3 v2:172.21.15.12:6816/29951 29 ==== osd_op_reply(110 usage.8 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f305c03b070 con 0x558b4b835060 2023-10-05T19:40:17.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.775+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:111 6.b 6:d7809252:usage::usage.9:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8bc440 con 0x558b4b835060 2023-10-05T19:40:17.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.775+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 <== osd.3 v2:172.21.15.12:6816/29951 30 ==== osd_op_reply(111 usage.9 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f305c03b070 con 0x558b4b835060 2023-10-05T19:40:17.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.775+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:112 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8bc440 con 0x7f3064007030 2023-10-05T19:40:17.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.779+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(112 usage.10 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f306c014040 con 0x7f3064007030 2023-10-05T19:40:17.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.779+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:113 6.4 6:21f845ae:usage::usage.11:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8bc440 con 0x558b4b770bf0 2023-10-05T19:40:17.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.779+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(113 usage.11 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f306c014040 con 0x558b4b770bf0 2023-10-05T19:40:17.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.779+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:114 6.f 6:f0386354:usage::usage.12:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8bc440 con 0x558b4b8306e0 2023-10-05T19:40:17.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.779+0000 7f30745cf700 1 -- 172.21.15.12:0/572202354 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(114 usage.12 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3064022020 con 0x558b4b8306e0 2023-10-05T19:40:17.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.779+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:115 6.c 6:3735eec4:usage::usage.13:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8bc440 con 0x558b4b835060 2023-10-05T19:40:17.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.779+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 <== osd.3 v2:172.21.15.12:6816/29951 31 ==== osd_op_reply(115 usage.13 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f305c03b070 con 0x558b4b835060 2023-10-05T19:40:17.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.779+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:116 6.18 6:1f520d0a:usage::usage.14:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8bc440 con 0x7f3064007030 2023-10-05T19:40:17.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.779+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(116 usage.14 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f306c014040 con 0x7f3064007030 2023-10-05T19:40:17.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.779+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:117 6.0 6:029cee12:usage::usage.15:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8bc440 con 0x7f3064007030 2023-10-05T19:40:17.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.779+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(117 usage.15 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f306c014040 con 0x7f3064007030 2023-10-05T19:40:17.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.779+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:118 6.1e 6:7f3e030e:usage::usage.16:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8bc440 con 0x558b4b835060 2023-10-05T19:40:17.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.783+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 <== osd.3 v2:172.21.15.12:6816/29951 32 ==== osd_op_reply(118 usage.16 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f305c03b070 con 0x558b4b835060 2023-10-05T19:40:17.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.783+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:119 6.10 6:0fbdcb32:usage::usage.17:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8bc440 con 0x7f3064007030 2023-10-05T19:40:17.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.783+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.0 v2:172.21.15.8:6800/27563 30 ==== osd_op_reply(119 usage.17 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f306c014040 con 0x7f3064007030 2023-10-05T19:40:17.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.783+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:120 6.19 6:993b03e9:usage::usage.18:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8bc440 con 0x558b4b770bf0 2023-10-05T19:40:17.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.783+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== osd_op_reply(120 usage.18 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f306c014040 con 0x558b4b770bf0 2023-10-05T19:40:17.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.783+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:121 6.1a 6:58fcd270:usage::usage.19:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8bc440 con 0x558b4b835060 2023-10-05T19:40:17.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.783+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 <== osd.3 v2:172.21.15.12:6816/29951 33 ==== osd_op_reply(121 usage.19 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f305c03b070 con 0x558b4b835060 2023-10-05T19:40:17.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.783+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:122 6.1f 6:fcd63d49:usage::usage.20:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8bc440 con 0x7f3064007030 2023-10-05T19:40:17.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.783+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.0 v2:172.21.15.8:6800/27563 31 ==== osd_op_reply(122 usage.20 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f306c014040 con 0x7f3064007030 2023-10-05T19:40:17.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.783+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:123 6.5 6:a2c1a412:usage::usage.21:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8bc440 con 0x558b4b8306e0 2023-10-05T19:40:17.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.783+0000 7f30745cf700 1 -- 172.21.15.12:0/572202354 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(123 usage.21 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3064022020 con 0x558b4b8306e0 2023-10-05T19:40:17.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.783+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:124 6.13 6:cdb71d68:usage::usage.22:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8bc440 con 0x7f3064007030 2023-10-05T19:40:17.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.787+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.0 v2:172.21.15.8:6800/27563 32 ==== osd_op_reply(124 usage.22 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f306c014040 con 0x7f3064007030 2023-10-05T19:40:17.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.787+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:125 6.17 6:ef3bca03:usage::usage.23:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8bc440 con 0x558b4b770bf0 2023-10-05T19:40:17.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.787+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.1 v2:172.21.15.12:6808/29933 39 ==== osd_op_reply(125 usage.23 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f306c014040 con 0x558b4b770bf0 2023-10-05T19:40:17.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.787+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:126 6.e 6:7412960a:usage::usage.24:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8bc440 con 0x558b4b770bf0 2023-10-05T19:40:17.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.787+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.1 v2:172.21.15.12:6808/29933 40 ==== osd_op_reply(126 usage.24 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f306c014040 con 0x558b4b770bf0 2023-10-05T19:40:17.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.787+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:127 6.7 6:e62d1024:usage::usage.25:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8bc440 con 0x558b4b835060 2023-10-05T19:40:17.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.787+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 <== osd.3 v2:172.21.15.12:6816/29951 34 ==== osd_op_reply(127 usage.25 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f305c03b070 con 0x558b4b835060 2023-10-05T19:40:17.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.787+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:128 6.d 6:b172cd59:usage::usage.26:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8bc440 con 0x558b4b770bf0 2023-10-05T19:40:17.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.787+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.1 v2:172.21.15.12:6808/29933 41 ==== osd_op_reply(128 usage.26 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f306c014040 con 0x558b4b770bf0 2023-10-05T19:40:17.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.787+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:129 6.1 6:83912023:usage::usage.27:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8bc440 con 0x558b4b770bf0 2023-10-05T19:40:17.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.787+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.1 v2:172.21.15.12:6808/29933 42 ==== osd_op_reply(129 usage.27 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f306c014040 con 0x558b4b770bf0 2023-10-05T19:40:17.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.787+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:130 6.1e 6:7c73bdb3:usage::usage.28:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8bc440 con 0x558b4b835060 2023-10-05T19:40:17.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.791+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 <== osd.3 v2:172.21.15.12:6816/29951 35 ==== osd_op_reply(130 usage.28 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f305c03b070 con 0x558b4b835060 2023-10-05T19:40:17.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.791+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:131 6.16 6:6e59fa35:usage::usage.29:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8bc440 con 0x558b4b835060 2023-10-05T19:40:17.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.791+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 <== osd.3 v2:172.21.15.12:6816/29951 36 ==== osd_op_reply(131 usage.29 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f305c03b070 con 0x558b4b835060 2023-10-05T19:40:17.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.791+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:132 6.1c 6:38fe6885:usage::usage.30:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8bc440 con 0x558b4b835060 2023-10-05T19:40:17.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.791+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 <== osd.3 v2:172.21.15.12:6816/29951 37 ==== osd_op_reply(132 usage.30 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f305c03b070 con 0x558b4b835060 2023-10-05T19:40:17.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.791+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:133 6.1d 6:bb28dff0:usage::usage.31:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b8bc440 con 0x7f3064007030 2023-10-05T19:40:17.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.791+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.0 v2:172.21.15.8:6800/27563 33 ==== osd_op_reply(133 usage.31 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f306c014040 con 0x7f3064007030 2023-10-05T19:40:17.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.791+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:134 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94056755737264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b879f60 con 0x558b4b8306e0 2023-10-05T19:40:17.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.795+0000 7f30745cf700 1 -- 172.21.15.12:0/572202354 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(134 notify.0 [watch unwatch cookie 94056755737264] v28'20 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3064022020 con 0x558b4b8306e0 2023-10-05T19:40:17.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.795+0000 7f3076a9ae40 20 remove_watcher() i=0 2023-10-05T19:40:17.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.795+0000 7f3076a9ae40 2 removed watcher, disabling cache 2023-10-05T19:40:17.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.795+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:135 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94056755739552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b83b2b0 con 0x558b4b770bf0 2023-10-05T19:40:17.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.799+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.1 v2:172.21.15.12:6808/29933 43 ==== osd_op_reply(135 notify.1 [watch unwatch cookie 94056755739552] v28'20 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f306c014040 con 0x558b4b770bf0 2023-10-05T19:40:17.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.799+0000 7f3076a9ae40 20 remove_watcher() i=1 2023-10-05T19:40:17.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.799+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:136 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94056755743232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b83b2b0 con 0x558b4b835060 2023-10-05T19:40:17.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.803+0000 7f3073dce700 1 -- 172.21.15.12:0/572202354 <== osd.3 v2:172.21.15.12:6816/29951 38 ==== osd_op_reply(136 notify.2 [watch unwatch cookie 94056755743232] v28'20 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f305c03b070 con 0x558b4b835060 2023-10-05T19:40:17.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.803+0000 7f3076a9ae40 20 remove_watcher() i=2 2023-10-05T19:40:17.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.803+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:137 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94056755740656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b83ca00 con 0x558b4b8306e0 2023-10-05T19:40:17.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.803+0000 7f30745cf700 1 -- 172.21.15.12:0/572202354 <== osd.2 v2:172.21.15.12:6800/29934 23 ==== osd_op_reply(137 notify.3 [watch unwatch cookie 94056755740656] v28'20 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3064022020 con 0x558b4b8306e0 2023-10-05T19:40:17.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.803+0000 7f3076a9ae40 20 remove_watcher() i=3 2023-10-05T19:40:17.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.803+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:138 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94056755746000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b83c090 con 0x558b4b770bf0 2023-10-05T19:40:17.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.807+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.1 v2:172.21.15.12:6808/29933 44 ==== osd_op_reply(138 notify.4 [watch unwatch cookie 94056755746000] v28'20 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f306c014040 con 0x558b4b770bf0 2023-10-05T19:40:17.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.807+0000 7f3076a9ae40 20 remove_watcher() i=4 2023-10-05T19:40:17.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.807+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:139 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94056755750928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b83c090 con 0x7f3064007030 2023-10-05T19:40:17.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.811+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.0 v2:172.21.15.8:6800/27563 34 ==== osd_op_reply(139 notify.5 [watch unwatch cookie 94056755750928] v28'20 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f306c014040 con 0x7f3064007030 2023-10-05T19:40:17.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.811+0000 7f3076a9ae40 20 remove_watcher() i=5 2023-10-05T19:40:17.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.811+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:140 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94056755754512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b83c090 con 0x7f3064007030 2023-10-05T19:40:17.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.815+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.0 v2:172.21.15.8:6800/27563 35 ==== osd_op_reply(140 notify.6 [watch unwatch cookie 94056755754512] v28'20 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f306c014040 con 0x7f3064007030 2023-10-05T19:40:17.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.815+0000 7f3076a9ae40 20 remove_watcher() i=6 2023-10-05T19:40:17.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.815+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:141 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94056755752032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b83c090 con 0x7f3064007030 2023-10-05T19:40:17.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.815+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.0 v2:172.21.15.8:6800/27563 36 ==== osd_op_reply(141 notify.7 [watch unwatch cookie 94056755752032] v28'20 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f306c014040 con 0x7f3064007030 2023-10-05T19:40:17.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.815+0000 7f3076a9ae40 20 remove_watcher() i=7 2023-10-05T19:40:17.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.819+0000 7f3045fe3700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:17.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.819+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:142 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94056755755616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558b4b83ed00 con 0x558b4b770bf0 2023-10-05T19:40:17.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.823+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 <== osd.1 v2:172.21.15.12:6808/29933 45 ==== osd_op_reply(142 data_loggenerations_metadata [watch unwatch cookie 94056755755616] v28'47 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f306c014040 con 0x558b4b770bf0 2023-10-05T19:40:17.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.823+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f3064007030 msgr2=0x7f3064026880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.823+0000 7f3076a9ae40 1 --2- 172.21.15.12:0/572202354 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f3064007030 0x7f3064026880 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:17.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.823+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x558b4b770bf0 msgr2=0x558b4b82e5c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.823+0000 7f3076a9ae40 1 --2- 172.21.15.12:0/572202354 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x558b4b770bf0 0x558b4b82e5c0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:17.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.823+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x558b4b8306e0 msgr2=0x558b4b832ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.823+0000 7f3076a9ae40 1 --2- 172.21.15.12:0/572202354 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x558b4b8306e0 0x558b4b832ac0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:17.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.823+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x558b4b835060 msgr2=0x558b4b837440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.823+0000 7f3076a9ae40 1 --2- 172.21.15.12:0/572202354 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x558b4b835060 0x558b4b837440 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:17.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.823+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3054030bf0 msgr2=0x7f30540330a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.823+0000 7f3076a9ae40 1 --2- 172.21.15.12:0/572202354 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3054030bf0 0x7f30540330a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f305c001560 tx=0x7f305c006650 comp rx=0 tx=0).stop 2023-10-05T19:40:17.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.823+0000 7f3074dd0700 1 -- 172.21.15.12:0/572202354 reap_dead start 2023-10-05T19:40:17.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.823+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b4b76f710 msgr2=0x558b4b822b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.823+0000 7f3076a9ae40 1 --2- 172.21.15.12:0/572202354 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b4b76f710 0x558b4b822b20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3064000c00 tx=0x7f3064003050 comp rx=0 tx=0).stop 2023-10-05T19:40:17.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.823+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 shutdown_connections 2023-10-05T19:40:17.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.823+0000 7f3076a9ae40 1 --2- 172.21.15.12:0/572202354 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b4b76f710 0x558b4b822b20 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:17.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.823+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 >> 172.21.15.12:0/572202354 conn(0x558b4b7f6820 msgr2=0x558b4b816500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:17.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.823+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 shutdown_connections 2023-10-05T19:40:17.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.823+0000 7f3076a9ae40 1 -- 172.21.15.12:0/572202354 wait complete. 2023-10-05T19:40:17.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.823+0000 7f3076a9ae40 1 -- 172.21.15.12:0/512643147 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2fb4030c00 msgr2=0x7f2fb40330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.823+0000 7f3076a9ae40 1 --2- 172.21.15.12:0/512643147 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2fb4030c00 0x7f2fb40330b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x558b4b83dd30 tx=0x7f306c00ab80 comp rx=0 tx=0).stop 2023-10-05T19:40:17.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.823+0000 7f3076a9ae40 1 -- 172.21.15.12:0/512643147 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b4b84d7b0 msgr2=0x558b4b853280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.823+0000 7f3076a9ae40 1 --2- 172.21.15.12:0/512643147 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b4b84d7b0 0x558b4b853280 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f305c0062a0 tx=0x7f305c042000 comp rx=0 tx=0).stop 2023-10-05T19:40:17.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.823+0000 7f3076a9ae40 1 -- 172.21.15.12:0/512643147 shutdown_connections 2023-10-05T19:40:17.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.823+0000 7f3076a9ae40 1 --2- 172.21.15.12:0/512643147 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2fb4030c00 0x7f2fb40330b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:17.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.823+0000 7f3076a9ae40 1 --2- 172.21.15.12:0/512643147 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558b4b84d7b0 0x558b4b853280 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:17.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.823+0000 7f3076a9ae40 1 -- 172.21.15.12:0/512643147 >> 172.21.15.12:0/512643147 conn(0x558b4b848b70 msgr2=0x558b4b846c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:17.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.823+0000 7f3076a9ae40 1 -- 172.21.15.12:0/512643147 shutdown_connections 2023-10-05T19:40:17.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.823+0000 7f3076a9ae40 1 -- 172.21.15.12:0/512643147 wait complete. 2023-10-05T19:40:17.835 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:40:17.835 INFO:teuthology.orchestra.run.smithi012.stdout: "entries": [], 2023-10-05T19:40:17.836 INFO:teuthology.orchestra.run.smithi012.stdout: "summary": [] 2023-10-05T19:40:17.836 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:40:17.839 DEBUG:tasks.util.rgw: json result: {'entries': [], 'summary': []} 2023-10-05T19:40:17.839 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2023-10-05T19:40:17.839 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2023-10-05T19:40:17.839 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2023-10-05T19:40:17.880 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:17.880 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:17.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.871+0000 7f6ee7e0be40 1 Processor -- start 2023-10-05T19:40:17.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.871+0000 7f6ee7e0be40 1 -- start start 2023-10-05T19:40:17.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.871+0000 7f6ee7e0be40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x5592762e4410 0x5592762e67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.871+0000 7f6ee7e0be40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55927622cbd0 con 0x5592762dfe30 2023-10-05T19:40:17.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.871+0000 7f6ee7e0be40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x5592761e9ee0 con 0x5592762e4410 2023-10-05T19:40:17.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.875+0000 7f6ee513f700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x5592762e4410 0x5592762e67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.875+0000 7f6ee5940700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x5592762dfe30 0x5592762e0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:47198/0 (socket says 172.21.15.12:47198) 2023-10-05T19:40:17.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.875+0000 7f6ee5940700 1 -- 172.21.15.12:0/271602857 learned_addr learned my addr 172.21.15.12:0/271602857 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:17.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.875+0000 7f6ee493e700 1 -- 172.21.15.12:0/271602857 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 471368895 0 0) 0x55927622cbd0 con 0x5592762dfe30 2023-10-05T19:40:17.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.875+0000 7f6ee493e700 1 -- 172.21.15.12:0/271602857 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ed4001e10 con 0x5592762dfe30 2023-10-05T19:40:17.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.875+0000 7f6ee513f700 1 -- 172.21.15.12:0/271602857 >> v1:172.21.15.8:6789/0 conn(0x5592762dfe30 legacy=0x5592762e0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.875+0000 7f6ee513f700 1 -- 172.21.15.12:0/271602857 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592762e0a40 con 0x5592762e4410 2023-10-05T19:40:17.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.875+0000 7f6ee513f700 1 --2- 172.21.15.12:0/271602857 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592762e4410 0x5592762e67f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6ed000a5d0 tx=0x7f6ed0005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6529d8d1280d4f6d server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.875+0000 7f6ee493e700 1 -- 172.21.15.12:0/271602857 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6ed0007610 con 0x5592762e4410 2023-10-05T19:40:17.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.875+0000 7f6ee493e700 1 -- 172.21.15.12:0/271602857 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed0007770 con 0x5592762e4410 2023-10-05T19:40:17.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.875+0000 7f6ee493e700 1 -- 172.21.15.12:0/271602857 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6ed0007a40 con 0x5592762e4410 2023-10-05T19:40:17.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.875+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/271602857 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592762e4410 msgr2=0x5592762e67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.875+0000 7f6ee7e0be40 1 --2- 172.21.15.12:0/271602857 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592762e4410 0x5592762e67f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6ed000a5d0 tx=0x7f6ed0005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:17.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.875+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/271602857 shutdown_connections 2023-10-05T19:40:17.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.875+0000 7f6ee7e0be40 1 --2- 172.21.15.12:0/271602857 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592762e4410 0x5592762e67f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:17.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.875+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/271602857 >> 172.21.15.12:0/271602857 conn(0x5592762db850 msgr2=0x5592762ddc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:17.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.875+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/271602857 shutdown_connections 2023-10-05T19:40:17.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.875+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/271602857 wait complete. 2023-10-05T19:40:17.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.875+0000 7f6ee7e0be40 1 Processor -- start 2023-10-05T19:40:17.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.875+0000 7f6ee7e0be40 1 -- start start 2023-10-05T19:40:17.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.879+0000 7f6ee7e0be40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592762dfe30 0x5592762fed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.879+0000 7f6ee7e0be40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592761ade30 con 0x5592762dfe30 2023-10-05T19:40:17.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.879+0000 7f6ee5940700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592762dfe30 0x5592762fed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.879+0000 7f6ee5940700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592762dfe30 0x5592762fed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36564/0 (socket says 172.21.15.12:36564) 2023-10-05T19:40:17.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.879+0000 7f6ee5940700 1 -- 172.21.15.12:0/3638056406 learned_addr learned my addr 172.21.15.12:0/3638056406 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:17.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.879+0000 7f6ee5940700 1 -- 172.21.15.12:0/3638056406 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed0005cf0 con 0x5592762dfe30 2023-10-05T19:40:17.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.879+0000 7f6ee5940700 1 --2- 172.21.15.12:0/3638056406 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592762dfe30 0x5592762fed60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0001c70 tx=0x7f6ee0001820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f93931864eed318 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.879+0000 7f6edf7fe700 1 -- 172.21.15.12:0/3638056406 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6ee000bbf0 con 0x5592762dfe30 2023-10-05T19:40:17.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.879+0000 7f6edf7fe700 1 -- 172.21.15.12:0/3638056406 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ee000bd70 con 0x5592762dfe30 2023-10-05T19:40:17.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.879+0000 7f6edf7fe700 1 -- 172.21.15.12:0/3638056406 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6ee000e5f0 con 0x5592762dfe30 2023-10-05T19:40:17.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.879+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/3638056406 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592762dfe30 msgr2=0x5592762fed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:17.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.879+0000 7f6ee7e0be40 1 --2- 172.21.15.12:0/3638056406 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592762dfe30 0x5592762fed60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0001c70 tx=0x7f6ee0001820 comp rx=0 tx=0).stop 2023-10-05T19:40:17.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.879+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/3638056406 shutdown_connections 2023-10-05T19:40:17.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.879+0000 7f6ee7e0be40 1 --2- 172.21.15.12:0/3638056406 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592762dfe30 0x5592762fed60 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:17.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.879+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/3638056406 >> 172.21.15.12:0/3638056406 conn(0x5592762db850 msgr2=0x5592762fa920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:17.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.879+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/3638056406 shutdown_connections 2023-10-05T19:40:17.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.879+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/3638056406 wait complete. 2023-10-05T19:40:17.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.883+0000 7f6ee7e0be40 1 Processor -- start 2023-10-05T19:40:17.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.883+0000 7f6ee7e0be40 1 -- start start 2023-10-05T19:40:17.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.883+0000 7f6ee7e0be40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592762dfe30 0x559276305b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.883+0000 7f6ee7e0be40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559276180ed0 con 0x5592762dfe30 2023-10-05T19:40:17.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.883+0000 7f6ee5940700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592762dfe30 0x559276305b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.883+0000 7f6ee5940700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592762dfe30 0x559276305b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36572/0 (socket says 172.21.15.12:36572) 2023-10-05T19:40:17.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.883+0000 7f6ee5940700 1 -- 172.21.15.12:0/1090226059 learned_addr learned my addr 172.21.15.12:0/1090226059 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:17.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.883+0000 7f6ee5940700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed4001e10 con 0x5592762dfe30 2023-10-05T19:40:17.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.883+0000 7f6ee5940700 1 --2- 172.21.15.12:0/1090226059 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592762dfe30 0x559276305b40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x5592762fb440 tx=0x7f6ee0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.883+0000 7f6eddffb700 1 -- 172.21.15.12:0/1090226059 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6ee001f070 con 0x5592762dfe30 2023-10-05T19:40:17.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.883+0000 7f6eddffb700 1 -- 172.21.15.12:0/1090226059 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ee0018640 con 0x5592762dfe30 2023-10-05T19:40:17.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.883+0000 7f6eddffb700 1 -- 172.21.15.12:0/1090226059 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6ee00188f0 con 0x5592762dfe30 2023-10-05T19:40:17.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.883+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559276306080 con 0x5592762dfe30 2023-10-05T19:40:17.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.883+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5592763064c0 con 0x5592762dfe30 2023-10-05T19:40:17.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.887+0000 7f6eddffb700 1 -- 172.21.15.12:0/1090226059 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f6ee0018a50 con 0x5592762dfe30 2023-10-05T19:40:17.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.887+0000 7f6eddffb700 1 --2- 172.21.15.12:0/1090226059 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f6ec8030c40 0x7f6ec80330f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.887+0000 7f6eddffb700 1 -- 172.21.15.12:0/1090226059 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f6ee0044e40 con 0x5592762dfe30 2023-10-05T19:40:17.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.887+0000 7f6ee513f700 1 --2- 172.21.15.12:0/1090226059 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f6ec8030c40 0x7f6ec80330f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.887+0000 7f6ee7e0be40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.887+0000 7f6ee7e0be40 1 --2- 172.21.15.12:0/1090226059 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5592762e4410 0x559276311500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.887+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559276311c30 con 0x5592762e4410 2023-10-05T19:40:17.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.887+0000 7f6ee513f700 1 --2- 172.21.15.12:0/1090226059 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f6ec8030c40 0x7f6ec80330f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6ed000afd0 tx=0x7f6ed0014000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.887+0000 7f6ee6141700 1 --2- 172.21.15.12:0/1090226059 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5592762e4410 0x559276311500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.887+0000 7f6ee6141700 1 --2- 172.21.15.12:0/1090226059 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5592762e4410 0x559276311500 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.887+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6ed800a5a0 con 0x5592762e4410 2023-10-05T19:40:17.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.887+0000 7f6ee7e0be40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:17.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.887+0000 7f6ee7e0be40 20 realm 2023-10-05T19:40:17.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.887+0000 7f6ee7e0be40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.887+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927630eae0 con 0x5592762e4410 2023-10-05T19:40:17.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.891+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6ed800a5a0 con 0x5592762e4410 2023-10-05T19:40:17.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.891+0000 7f6ee7e0be40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:17.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.891+0000 7f6ee7e0be40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:17.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.891+0000 7f6ee7e0be40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.891+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559276312710 con 0x5592762e4410 2023-10-05T19:40:17.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.891+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6ed800a5a0 con 0x5592762e4410 2023-10-05T19:40:17.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.891+0000 7f6ee7e0be40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:17.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.891+0000 7f6ee7e0be40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.891+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5592763122f0 con 0x5592762e4410 2023-10-05T19:40:17.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.891+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f6ed800a5a0 con 0x5592762e4410 2023-10-05T19:40:17.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.891+0000 7f6ee7e0be40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:17.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.891+0000 7f6ee7e0be40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.891+0000 7f6ee7e0be40 1 --2- 172.21.15.12:0/1090226059 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5592763136a0 0x559276315a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.891+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5592763161f0 con 0x5592763136a0 2023-10-05T19:40:17.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.891+0000 7f6ee5940700 1 --2- 172.21.15.12:0/1090226059 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5592763136a0 0x559276315a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.891+0000 7f6ee5940700 1 --2- 172.21.15.12:0/1090226059 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5592763136a0 0x559276315a80 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.891+0000 7f6ee5940700 1 -- 172.21.15.12:0/1090226059 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f6ee0014070 con 0x5592763136a0 2023-10-05T19:40:17.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.891+0000 7f6ee7e0be40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:17.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.891+0000 7f6ee7e0be40 20 searching for the correct realm 2023-10-05T19:40:17.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.891+0000 7f6ee7e0be40 1 --2- 172.21.15.12:0/1090226059 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x559276318020 0x55927631a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.891+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631a940 con 0x559276318020 2023-10-05T19:40:17.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.891+0000 7f6ee513f700 1 --2- 172.21.15.12:0/1090226059 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x559276318020 0x55927631a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.895+0000 7f6ee513f700 1 --2- 172.21.15.12:0/1090226059 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x559276318020 0x55927631a400 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.895+0000 7f6ee513f700 1 -- 172.21.15.12:0/1090226059 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f6ed0007da0 con 0x559276318020 2023-10-05T19:40:17.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.895+0000 7f6ee513f700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631a940 con 0x5592763136a0 2023-10-05T19:40:17.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.899+0000 7f6ee5940700 1 -- 172.21.15.12:0/1090226059 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ee0014070 con 0x5592763136a0 2023-10-05T19:40:17.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.899+0000 7f6ee5940700 1 --2- 172.21.15.12:0/1090226059 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f6ee0016730 0x7f6ee0042130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:17.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.899+0000 7f6ee5940700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631a940 con 0x7f6ee0016730 2023-10-05T19:40:17.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.899+0000 7f6ee6141700 1 --2- 172.21.15.12:0/1090226059 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f6ee0016730 0x7f6ee0042130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:17.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.899+0000 7f6ee6141700 1 --2- 172.21.15.12:0/1090226059 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f6ee0016730 0x7f6ee0042130 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:17.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.899+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed800a5a0 con 0x7f6ee0016730 2023-10-05T19:40:17.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.899+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631a940 con 0x5592762e4410 2023-10-05T19:40:17.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.899+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed800a5a0 con 0x5592762e4410 2023-10-05T19:40:17.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.899+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631a940 con 0x5592762e4410 2023-10-05T19:40:17.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.899+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed800a5a0 con 0x5592762e4410 2023-10-05T19:40:17.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.899+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631a940 con 0x7f6ee0016730 2023-10-05T19:40:17.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.903+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed800a5a0 con 0x7f6ee0016730 2023-10-05T19:40:17.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.903+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631a940 con 0x7f6ee0016730 2023-10-05T19:40:17.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.903+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed800a5a0 con 0x7f6ee0016730 2023-10-05T19:40:17.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.903+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631a940 con 0x5592763136a0 2023-10-05T19:40:17.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.903+0000 7f6ee5940700 1 -- 172.21.15.12:0/1090226059 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ee0014070 con 0x5592763136a0 2023-10-05T19:40:17.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.903+0000 7f6ee5940700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631a940 con 0x559276318020 2023-10-05T19:40:17.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.903+0000 7f6ee513f700 1 -- 172.21.15.12:0/1090226059 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed001f480 con 0x559276318020 2023-10-05T19:40:17.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.903+0000 7f6ee513f700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631a940 con 0x5592762e4410 2023-10-05T19:40:17.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.903+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed800a5a0 con 0x5592762e4410 2023-10-05T19:40:17.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.903+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631a940 con 0x7f6ee0016730 2023-10-05T19:40:17.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.907+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed800a5a0 con 0x7f6ee0016730 2023-10-05T19:40:17.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.907+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631a940 con 0x5592762e4410 2023-10-05T19:40:17.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.907+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed800a5a0 con 0x5592762e4410 2023-10-05T19:40:17.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.907+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631a940 con 0x5592763136a0 2023-10-05T19:40:17.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.907+0000 7f6ee5940700 1 -- 172.21.15.12:0/1090226059 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ee0014070 con 0x5592763136a0 2023-10-05T19:40:17.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.907+0000 7f6ee5940700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631a940 con 0x559276318020 2023-10-05T19:40:17.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.907+0000 7f6ee513f700 1 -- 172.21.15.12:0/1090226059 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed001f480 con 0x559276318020 2023-10-05T19:40:17.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.907+0000 7f6ee513f700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631a940 con 0x559276318020 2023-10-05T19:40:17.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.907+0000 7f6ee513f700 1 -- 172.21.15.12:0/1090226059 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed001f480 con 0x559276318020 2023-10-05T19:40:17.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.907+0000 7f6ee513f700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631a940 con 0x7f6ee0016730 2023-10-05T19:40:17.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.911+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed800a5a0 con 0x7f6ee0016730 2023-10-05T19:40:17.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.911+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631a940 con 0x559276318020 2023-10-05T19:40:17.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.911+0000 7f6ee513f700 1 -- 172.21.15.12:0/1090226059 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed001f480 con 0x559276318020 2023-10-05T19:40:17.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.911+0000 7f6ee513f700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631a940 con 0x559276318020 2023-10-05T19:40:17.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.911+0000 7f6ee513f700 1 -- 172.21.15.12:0/1090226059 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed001f480 con 0x559276318020 2023-10-05T19:40:17.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.911+0000 7f6ee513f700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631a940 con 0x5592762e4410 2023-10-05T19:40:17.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.911+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed800a5a0 con 0x5592762e4410 2023-10-05T19:40:17.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.911+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631a940 con 0x5592762e4410 2023-10-05T19:40:17.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.915+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed800a5a0 con 0x5592762e4410 2023-10-05T19:40:17.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.915+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631a940 con 0x7f6ee0016730 2023-10-05T19:40:17.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.915+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed800a5a0 con 0x7f6ee0016730 2023-10-05T19:40:17.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.915+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631a940 con 0x559276318020 2023-10-05T19:40:17.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.915+0000 7f6ee513f700 1 -- 172.21.15.12:0/1090226059 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed001f480 con 0x559276318020 2023-10-05T19:40:17.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.915+0000 7f6ee513f700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631a940 con 0x5592763136a0 2023-10-05T19:40:17.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.915+0000 7f6ee5940700 1 -- 172.21.15.12:0/1090226059 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ee0014070 con 0x5592763136a0 2023-10-05T19:40:17.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.915+0000 7f6ee5940700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631a940 con 0x5592762e4410 2023-10-05T19:40:17.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.915+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed800a5a0 con 0x5592762e4410 2023-10-05T19:40:17.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.915+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631a940 con 0x7f6ee0016730 2023-10-05T19:40:17.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.919+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed800a5a0 con 0x7f6ee0016730 2023-10-05T19:40:17.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.919+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631a940 con 0x559276318020 2023-10-05T19:40:17.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.919+0000 7f6ee513f700 1 -- 172.21.15.12:0/1090226059 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed001f480 con 0x559276318020 2023-10-05T19:40:17.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.919+0000 7f6ee513f700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631a940 con 0x7f6ee0016730 2023-10-05T19:40:17.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.919+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed800a5a0 con 0x7f6ee0016730 2023-10-05T19:40:17.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.919+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631a940 con 0x5592763136a0 2023-10-05T19:40:17.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.919+0000 7f6ee5940700 1 -- 172.21.15.12:0/1090226059 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ee0014070 con 0x5592763136a0 2023-10-05T19:40:17.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.919+0000 7f6ee5940700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631a940 con 0x559276318020 2023-10-05T19:40:17.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.923+0000 7f6ee513f700 1 -- 172.21.15.12:0/1090226059 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed001f480 con 0x559276318020 2023-10-05T19:40:17.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.923+0000 7f6ee513f700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631a940 con 0x7f6ee0016730 2023-10-05T19:40:17.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.923+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed800a5a0 con 0x7f6ee0016730 2023-10-05T19:40:17.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.923+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631a940 con 0x5592762e4410 2023-10-05T19:40:17.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.923+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f6ed800a5a0 con 0x5592762e4410 2023-10-05T19:40:17.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.923+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631a940 con 0x5592763136a0 2023-10-05T19:40:17.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.923+0000 7f6ee5940700 1 -- 172.21.15.12:0/1090226059 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f6ee0014070 con 0x5592763136a0 2023-10-05T19:40:17.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.923+0000 7f6ee7e0be40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:17.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.923+0000 7f6ee7e0be40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:17.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.923+0000 7f6ee7e0be40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:17.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.923+0000 7f6ee7e0be40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:17.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.923+0000 7f6ee7e0be40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.923+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559276316dd0 con 0x5592762e4410 2023-10-05T19:40:17.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.923+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6ed800a5a0 con 0x5592762e4410 2023-10-05T19:40:17.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.923+0000 7f6ee7e0be40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:17.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.923+0000 7f6ee7e0be40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.923+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559276317720 con 0x5592762e4410 2023-10-05T19:40:17.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.927+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f6ed800a5a0 con 0x5592762e4410 2023-10-05T19:40:17.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.927+0000 7f6ee7e0be40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:17.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.927+0000 7f6ee7e0be40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.927+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559276316990 con 0x559276318020 2023-10-05T19:40:17.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.927+0000 7f6ee513f700 1 -- 172.21.15.12:0/1090226059 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f6ed001f480 con 0x559276318020 2023-10-05T19:40:17.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.927+0000 7f6ee7e0be40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:17.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.927+0000 7f6ee7e0be40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.927+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631b270 con 0x559276318020 2023-10-05T19:40:17.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.927+0000 7f6ee513f700 1 -- 172.21.15.12:0/1090226059 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f6ed001f480 con 0x559276318020 2023-10-05T19:40:17.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.927+0000 7f6ee7e0be40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:17.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.927+0000 7f6ee7e0be40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.927+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631b620 con 0x5592762e4410 2023-10-05T19:40:17.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.927+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6ed800a5a0 con 0x5592762e4410 2023-10-05T19:40:17.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.927+0000 7f6ee7e0be40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:17.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.927+0000 7f6ee7e0be40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.927+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631b9d0 con 0x5592762e4410 2023-10-05T19:40:17.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.927+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f6ed800a5a0 con 0x5592762e4410 2023-10-05T19:40:17.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.927+0000 7f6ee7e0be40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:17.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.927+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631c9e0 con 0x559276318020 2023-10-05T19:40:17.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.931+0000 7f6ee513f700 1 -- 172.21.15.12:0/1090226059 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f6ed001f480 con 0x559276318020 2023-10-05T19:40:17.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.931+0000 7f6ee513f700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631c9e0 con 0x5592763136a0 2023-10-05T19:40:17.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.931+0000 7f6ee5940700 1 -- 172.21.15.12:0/1090226059 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ee0014070 con 0x5592763136a0 2023-10-05T19:40:17.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.931+0000 7f6ee5940700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631c9e0 con 0x7f6ee0016730 2023-10-05T19:40:17.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.935+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed800a5a0 con 0x7f6ee0016730 2023-10-05T19:40:17.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.935+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631b620 con 0x5592762e4410 2023-10-05T19:40:17.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.935+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed800a5a0 con 0x5592762e4410 2023-10-05T19:40:17.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.935+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631b620 con 0x5592762e4410 2023-10-05T19:40:17.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.935+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed800a5a0 con 0x5592762e4410 2023-10-05T19:40:17.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.935+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631b620 con 0x7f6ee0016730 2023-10-05T19:40:17.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.935+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed800a5a0 con 0x7f6ee0016730 2023-10-05T19:40:17.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.935+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631b620 con 0x7f6ee0016730 2023-10-05T19:40:17.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.935+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed800a5a0 con 0x7f6ee0016730 2023-10-05T19:40:17.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.935+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631b620 con 0x5592763136a0 2023-10-05T19:40:17.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.939+0000 7f6ee5940700 1 -- 172.21.15.12:0/1090226059 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ee0014070 con 0x5592763136a0 2023-10-05T19:40:17.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.939+0000 7f6ee5940700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631b620 con 0x559276318020 2023-10-05T19:40:17.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.939+0000 7f6ee513f700 1 -- 172.21.15.12:0/1090226059 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed001f480 con 0x559276318020 2023-10-05T19:40:17.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.939+0000 7f6ee513f700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631b620 con 0x5592762e4410 2023-10-05T19:40:17.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.939+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed800a5a0 con 0x5592762e4410 2023-10-05T19:40:17.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.939+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631b620 con 0x7f6ee0016730 2023-10-05T19:40:17.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.939+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed800a5a0 con 0x7f6ee0016730 2023-10-05T19:40:17.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.939+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631b620 con 0x5592762e4410 2023-10-05T19:40:17.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.939+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed800a5a0 con 0x5592762e4410 2023-10-05T19:40:17.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.939+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631b620 con 0x5592763136a0 2023-10-05T19:40:17.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.943+0000 7f6ee5940700 1 -- 172.21.15.12:0/1090226059 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ee0014070 con 0x5592763136a0 2023-10-05T19:40:17.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.943+0000 7f6ee5940700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631b620 con 0x559276318020 2023-10-05T19:40:17.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.943+0000 7f6ee513f700 1 -- 172.21.15.12:0/1090226059 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed001f480 con 0x559276318020 2023-10-05T19:40:17.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.943+0000 7f6ee513f700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631b620 con 0x559276318020 2023-10-05T19:40:17.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.943+0000 7f6ee513f700 1 -- 172.21.15.12:0/1090226059 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed001f480 con 0x559276318020 2023-10-05T19:40:17.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.943+0000 7f6ee513f700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631b620 con 0x7f6ee0016730 2023-10-05T19:40:17.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.943+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed800a5a0 con 0x7f6ee0016730 2023-10-05T19:40:17.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.943+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631b620 con 0x559276318020 2023-10-05T19:40:17.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.947+0000 7f6ee513f700 1 -- 172.21.15.12:0/1090226059 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed001f480 con 0x559276318020 2023-10-05T19:40:17.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.947+0000 7f6ee513f700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631b620 con 0x559276318020 2023-10-05T19:40:17.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.947+0000 7f6ee513f700 1 -- 172.21.15.12:0/1090226059 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed001f480 con 0x559276318020 2023-10-05T19:40:17.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.947+0000 7f6ee513f700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631b620 con 0x5592762e4410 2023-10-05T19:40:17.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.947+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed800a5a0 con 0x5592762e4410 2023-10-05T19:40:17.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.947+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631b620 con 0x5592762e4410 2023-10-05T19:40:17.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.947+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed800a5a0 con 0x5592762e4410 2023-10-05T19:40:17.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.947+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631b620 con 0x7f6ee0016730 2023-10-05T19:40:17.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.947+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed800a5a0 con 0x7f6ee0016730 2023-10-05T19:40:17.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.947+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631b620 con 0x559276318020 2023-10-05T19:40:17.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.951+0000 7f6ee513f700 1 -- 172.21.15.12:0/1090226059 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed001f480 con 0x559276318020 2023-10-05T19:40:17.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.951+0000 7f6ee513f700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631b620 con 0x5592763136a0 2023-10-05T19:40:17.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.951+0000 7f6ee5940700 1 -- 172.21.15.12:0/1090226059 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ee0014070 con 0x5592763136a0 2023-10-05T19:40:17.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.951+0000 7f6ee5940700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631b620 con 0x5592762e4410 2023-10-05T19:40:17.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.951+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed800a5a0 con 0x5592762e4410 2023-10-05T19:40:17.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.951+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631b620 con 0x7f6ee0016730 2023-10-05T19:40:17.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.951+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed800a5a0 con 0x7f6ee0016730 2023-10-05T19:40:17.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.951+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631b620 con 0x559276318020 2023-10-05T19:40:17.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.955+0000 7f6ee513f700 1 -- 172.21.15.12:0/1090226059 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed001f480 con 0x559276318020 2023-10-05T19:40:17.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.955+0000 7f6ee513f700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631b620 con 0x7f6ee0016730 2023-10-05T19:40:17.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.955+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed800a5a0 con 0x7f6ee0016730 2023-10-05T19:40:17.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.955+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631b620 con 0x5592763136a0 2023-10-05T19:40:17.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.955+0000 7f6ee5940700 1 -- 172.21.15.12:0/1090226059 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ee0014070 con 0x5592763136a0 2023-10-05T19:40:17.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.955+0000 7f6ee5940700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631b620 con 0x559276318020 2023-10-05T19:40:17.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.955+0000 7f6ee513f700 1 -- 172.21.15.12:0/1090226059 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed001f480 con 0x559276318020 2023-10-05T19:40:17.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.955+0000 7f6ee513f700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631b620 con 0x7f6ee0016730 2023-10-05T19:40:17.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.955+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ed800a5a0 con 0x7f6ee0016730 2023-10-05T19:40:17.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.955+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631b620 con 0x5592762e4410 2023-10-05T19:40:17.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.959+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f6ed800a5a0 con 0x5592762e4410 2023-10-05T19:40:17.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.959+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631b620 con 0x5592763136a0 2023-10-05T19:40:17.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.959+0000 7f6ee5940700 1 -- 172.21.15.12:0/1090226059 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f6ee0014070 con 0x5592763136a0 2023-10-05T19:40:17.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.959+0000 7f6ee7e0be40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:17.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.959+0000 7f6ee7e0be40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:17.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.959+0000 7f6ee7e0be40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:17.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.959+0000 7f6ee7e0be40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:17.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.959+0000 7f6ee7e0be40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.959+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631b9d0 con 0x5592762e4410 2023-10-05T19:40:17.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.959+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f6ed800a5a0 con 0x5592762e4410 2023-10-05T19:40:17.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.959+0000 7f6ee7e0be40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:17.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.959+0000 7f6ee7e0be40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.959+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631bd80 con 0x5592763136a0 2023-10-05T19:40:17.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.959+0000 7f6ee5940700 1 -- 172.21.15.12:0/1090226059 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f6ee0014070 con 0x5592763136a0 2023-10-05T19:40:17.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.959+0000 7f6ee7e0be40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:17.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.959+0000 7f6ee7e0be40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.959+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631c620 con 0x5592762e4410 2023-10-05T19:40:17.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.963+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f6ed800a5a0 con 0x5592762e4410 2023-10-05T19:40:17.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.963+0000 7f6ee7e0be40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:17.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.963+0000 7f6ee7e0be40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.963+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631c620 con 0x559276318020 2023-10-05T19:40:17.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.963+0000 7f6ee513f700 1 -- 172.21.15.12:0/1090226059 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f6ed001f480 con 0x559276318020 2023-10-05T19:40:17.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.963+0000 7f6ee7e0be40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:17.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.963+0000 7f6ee7e0be40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.963+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631c1b0 con 0x7f6ee0016730 2023-10-05T19:40:17.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.963+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f6ed800a5a0 con 0x7f6ee0016730 2023-10-05T19:40:17.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.963+0000 7f6ee7e0be40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:17.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.963+0000 7f6ee7e0be40 20 zone default found 2023-10-05T19:40:17.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.963+0000 7f6ee7e0be40 4 Realm: () 2023-10-05T19:40:17.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.963+0000 7f6ee7e0be40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:17.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.963+0000 7f6ee7e0be40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:17.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.963+0000 7f6ee7e0be40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:17.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.963+0000 7f6ee7e0be40 20 zonegroup default 2023-10-05T19:40:17.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.963+0000 7f6ee7e0be40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.963+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631c1b0 con 0x7f6ee0016730 2023-10-05T19:40:17.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.963+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f6ed800a5a0 con 0x7f6ee0016730 2023-10-05T19:40:17.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.963+0000 7f6ee7e0be40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:17.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.963+0000 7f6ee7e0be40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.963+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631c1b0 con 0x559276318020 2023-10-05T19:40:17.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.967+0000 7f6ee513f700 1 -- 172.21.15.12:0/1090226059 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f6ed001f480 con 0x559276318020 2023-10-05T19:40:17.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.967+0000 7f6ee7e0be40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:17.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.967+0000 7f6ee7e0be40 20 rados->read ofs=0 len=0 2023-10-05T19:40:17.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.967+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631e840 con 0x559276318020 2023-10-05T19:40:17.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.967+0000 7f6ee513f700 1 -- 172.21.15.12:0/1090226059 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f6ed001f480 con 0x559276318020 2023-10-05T19:40:17.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.967+0000 7f6ee7e0be40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:17.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.967+0000 7f6ee7e0be40 20 started sync module instance, tier type = 2023-10-05T19:40:17.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.967+0000 7f6ee7e0be40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:17.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.967+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631df00 con 0x5592763136a0 2023-10-05T19:40:17.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.971+0000 7f6ee5940700 1 -- 172.21.15.12:0/1090226059 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'21 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6ee0014070 con 0x5592763136a0 2023-10-05T19:40:17.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.971+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94087536567088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631db30 con 0x5592763136a0 2023-10-05T19:40:17.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.971+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631f5d0 con 0x5592762e4410 2023-10-05T19:40:17.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.971+0000 7f6ee5940700 1 -- 172.21.15.12:0/1090226059 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94087536567088] v28'22 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6ee0014070 con 0x5592763136a0 2023-10-05T19:40:17.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.971+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'21 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6ed800a5a0 con 0x5592762e4410 2023-10-05T19:40:17.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.971+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94087536569616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631f9d0 con 0x5592762e4410 2023-10-05T19:40:17.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.971+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927631f9d0 con 0x559276318020 2023-10-05T19:40:17.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.975+0000 7f6ee513f700 1 -- 172.21.15.12:0/1090226059 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'21 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6ed001f480 con 0x559276318020 2023-10-05T19:40:17.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.975+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94087536574112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5592763208f0 con 0x559276318020 2023-10-05T19:40:17.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.975+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5592763214b0 con 0x5592763136a0 2023-10-05T19:40:17.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.975+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94087536569616] v28'22 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6ed800a5a0 con 0x5592762e4410 2023-10-05T19:40:17.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.979+0000 7f6ee5940700 1 -- 172.21.15.12:0/1090226059 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'21 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6ee0014070 con 0x5592763136a0 2023-10-05T19:40:17.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.979+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94087536570720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5592763210e0 con 0x5592763136a0 2023-10-05T19:40:17.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.979+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559276322530 con 0x5592762e4410 2023-10-05T19:40:17.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.979+0000 7f6ee513f700 1 -- 172.21.15.12:0/1090226059 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94087536574112] v28'22 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6ed001f480 con 0x559276318020 2023-10-05T19:40:17.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.979+0000 7f6ee5940700 1 -- 172.21.15.12:0/1090226059 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94087536570720] v28'22 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6ee0014070 con 0x5592763136a0 2023-10-05T19:40:17.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.979+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'21 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6ed800a5a0 con 0x5592762e4410 2023-10-05T19:40:17.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.979+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94087536580704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5592763210a0 con 0x5592762e4410 2023-10-05T19:40:17.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.979+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5592763204f0 con 0x7f6ee0016730 2023-10-05T19:40:17.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.983+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'21 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6ed800a5a0 con 0x7f6ee0016730 2023-10-05T19:40:17.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.983+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94087536584592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5592763204f0 con 0x7f6ee0016730 2023-10-05T19:40:17.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.983+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559276323860 con 0x7f6ee0016730 2023-10-05T19:40:17.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.983+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94087536580704] v28'22 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6ed800a5a0 con 0x5592762e4410 2023-10-05T19:40:17.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.987+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(96 notify.6 [create] v28'21 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6ed800a5a0 con 0x7f6ee0016730 2023-10-05T19:40:17.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.987+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94087536582656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559276323820 con 0x7f6ee0016730 2023-10-05T19:40:17.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.987+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559276324270 con 0x7f6ee0016730 2023-10-05T19:40:17.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.987+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(95 notify.5 [watch watch cookie 94087536584592] v28'22 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6ed800a5a0 con 0x7f6ee0016730 2023-10-05T19:40:17.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.987+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(97 notify.6 [watch watch cookie 94087536582656] v28'22 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6ed800a5a0 con 0x7f6ee0016730 2023-10-05T19:40:17.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.987+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(98 notify.7 [create] v28'21 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6ed800a5a0 con 0x7f6ee0016730 2023-10-05T19:40:17.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.991+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94087536585696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559276323330 con 0x7f6ee0016730 2023-10-05T19:40:17.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.991+0000 7f6ee7e0be40 20 add_watcher() i=0 2023-10-05T19:40:17.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.991+0000 7f6ee7e0be40 20 add_watcher() i=1 2023-10-05T19:40:17.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.991+0000 7f6ee7e0be40 20 add_watcher() i=2 2023-10-05T19:40:17.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.991+0000 7f6ee7e0be40 20 add_watcher() i=3 2023-10-05T19:40:17.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.991+0000 7f6ee7e0be40 20 add_watcher() i=4 2023-10-05T19:40:17.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.991+0000 7f6ee7e0be40 20 add_watcher() i=5 2023-10-05T19:40:17.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.991+0000 7f6ee7e0be40 20 add_watcher() i=6 2023-10-05T19:40:18.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.991+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94087536585696] v28'22 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6ed800a5a0 con 0x7f6ee0016730 2023-10-05T19:40:18.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.991+0000 7f6ee7e0be40 20 add_watcher() i=7 2023-10-05T19:40:18.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.991+0000 7f6ee7e0be40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:18.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.991+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559276323330 con 0x5592762e4410 2023-10-05T19:40:18.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.991+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f6ed800a5a0 con 0x5592762e4410 2023-10-05T19:40:18.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.991+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94087536590144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559276323330 con 0x5592762e4410 2023-10-05T19:40:18.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.995+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94087536590144] v28'48 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f6ed800a5a0 con 0x5592762e4410 2023-10-05T19:40:18.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.995+0000 7f6eb77de700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:18.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.995+0000 7f6ee7e0be40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:18.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.995+0000 7f6ee7e0be40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:18.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.995+0000 7f6ee7e0be40 1 Processor -- start 2023-10-05T19:40:18.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.995+0000 7f6ee7e0be40 1 -- start start 2023-10-05T19:40:18.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.995+0000 7f6ee7e0be40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592763313f0 0x5592763317c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.995+0000 7f6ee7e0be40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559276331d00 con 0x5592763313f0 2023-10-05T19:40:18.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.995+0000 7f6ee513f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592763313f0 0x5592763317c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.995+0000 7f6ee513f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592763313f0 0x5592763317c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36578/0 (socket says 172.21.15.12:36578) 2023-10-05T19:40:18.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.995+0000 7f6ee513f700 1 -- 172.21.15.12:0/139383858 learned_addr learned my addr 172.21.15.12:0/139383858 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:18.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.995+0000 7f6ee513f700 1 -- 172.21.15.12:0/139383858 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559276331e40 con 0x5592763313f0 2023-10-05T19:40:18.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.995+0000 7f6ee513f700 1 --2- 172.21.15.12:0/139383858 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592763313f0 0x5592763317c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x5592762e0eb0 tx=0x7f6ed0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7b45438c13d01a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.999+0000 7f6eb6fdd700 1 -- 172.21.15.12:0/139383858 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6ed0025040 con 0x5592763313f0 2023-10-05T19:40:18.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.999+0000 7f6eb6fdd700 1 -- 172.21.15.12:0/139383858 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed001fdc0 con 0x5592763313f0 2023-10-05T19:40:18.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.999+0000 7f6eb6fdd700 1 -- 172.21.15.12:0/139383858 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6ed001dc20 con 0x5592763313f0 2023-10-05T19:40:18.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.999+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/139383858 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592763313f0 msgr2=0x5592763317c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.999+0000 7f6ee7e0be40 1 --2- 172.21.15.12:0/139383858 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592763313f0 0x5592763317c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x5592762e0eb0 tx=0x7f6ed0011000 comp rx=0 tx=0).stop 2023-10-05T19:40:18.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.999+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/139383858 shutdown_connections 2023-10-05T19:40:18.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.999+0000 7f6ee7e0be40 1 --2- 172.21.15.12:0/139383858 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592763313f0 0x5592763317c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.999+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/139383858 >> 172.21.15.12:0/139383858 conn(0x55927632c7b0 msgr2=0x55927632a960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:18.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.999+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/139383858 shutdown_connections 2023-10-05T19:40:18.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.999+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/139383858 wait complete. 2023-10-05T19:40:18.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.999+0000 7f6ee7e0be40 1 Processor -- start 2023-10-05T19:40:18.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.999+0000 7f6ee7e0be40 1 -- start start 2023-10-05T19:40:18.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.999+0000 7f6ee7e0be40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592763313f0 0x559276336f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.999+0000 7f6ee7e0be40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559276337f10 con 0x5592763313f0 2023-10-05T19:40:18.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.999+0000 7f6ee513f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592763313f0 0x559276336f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.999+0000 7f6ee513f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592763313f0 0x559276336f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36594/0 (socket says 172.21.15.12:36594) 2023-10-05T19:40:18.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.999+0000 7f6ee513f700 1 -- 172.21.15.12:0/3237908763 learned_addr learned my addr 172.21.15.12:0/3237908763 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:18.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:17.999+0000 7f6ee513f700 1 -- 172.21.15.12:0/3237908763 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592762e0a40 con 0x5592763313f0 2023-10-05T19:40:18.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.003+0000 7f6ee513f700 1 --2- 172.21.15.12:0/3237908763 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592763313f0 0x559276336f10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x55927632b4d0 tx=0x7f6ed003c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.003+0000 7f6eb57da700 1 -- 172.21.15.12:0/3237908763 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6ed0025070 con 0x5592763313f0 2023-10-05T19:40:18.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.003+0000 7f6eb57da700 1 -- 172.21.15.12:0/3237908763 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed003cd00 con 0x5592763313f0 2023-10-05T19:40:18.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.003+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/3237908763 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559276337510 con 0x5592763313f0 2023-10-05T19:40:18.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.003+0000 7f6eb57da700 1 -- 172.21.15.12:0/3237908763 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6ed003ce90 con 0x5592763313f0 2023-10-05T19:40:18.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.003+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/3237908763 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559276337950 con 0x5592763313f0 2023-10-05T19:40:18.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.003+0000 7f6eb57da700 1 -- 172.21.15.12:0/3237908763 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f6ed0043040 con 0x5592763313f0 2023-10-05T19:40:18.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.003+0000 7f6eb57da700 1 --2- 172.21.15.12:0/3237908763 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f6e2c030c50 0x7f6e2c033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.003+0000 7f6eb57da700 1 -- 172.21.15.12:0/3237908763 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f6ed00470d0 con 0x5592763313f0 2023-10-05T19:40:18.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.003+0000 7f6ee6141700 1 --2- 172.21.15.12:0/3237908763 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f6e2c030c50 0x7f6e2c033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.003+0000 7f6ee7e0be40 5 note: GC not initialized 2023-10-05T19:40:18.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.003+0000 7f6ee6141700 1 --2- 172.21.15.12:0/3237908763 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f6e2c030c50 0x7f6e2c033100 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x559276323930 tx=0x7f6ed800ae90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.007+0000 7f6e267fc700 20 reqs_thread_entry: start 2023-10-05T19:40:18.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.007+0000 7f6ee7e0be40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:18.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.007+0000 7f6e17fff700 20 reqs_thread_entry: start 2023-10-05T19:40:18.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.007+0000 7f6ee7e0be40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:18.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.007+0000 7f6e167fc700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:48 2023 2023-10-05T19:40:18.016 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:18.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.007+0000 7f6e15ffb700 20 reqs_thread_entry: start 2023-10-05T19:40:18.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.007+0000 7f6ee7e0be40 20 get_system_obj_state: rctx=0x7fff844d5c18 obj=default.rgw.meta:users.uid:foo state=0x5592761b90b0 s->prefetch_data=0 2023-10-05T19:40:18.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.007+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927639f600 con 0x5592762e4410 2023-10-05T19:40:18.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.007+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(102 foo [call,getxattrs,stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x7f6ed800a5a0 con 0x5592762e4410 2023-10-05T19:40:18.017 INFO:teuthology.orchestra.run.smithi012.stderr:could not fetch user info: no user info saved 2023-10-05T19:40:18.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.007+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:103 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94087536567088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927639f600 con 0x5592763136a0 2023-10-05T19:40:18.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.011+0000 7f6ee5940700 1 -- 172.21.15.12:0/1090226059 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(103 notify.0 [watch unwatch cookie 94087536567088] v28'23 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6ee0014070 con 0x5592763136a0 2023-10-05T19:40:18.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.011+0000 7f6ee7e0be40 20 remove_watcher() i=0 2023-10-05T19:40:18.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.011+0000 7f6ee7e0be40 2 removed watcher, disabling cache 2023-10-05T19:40:18.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.011+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:104 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94087536569616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927639f600 con 0x5592762e4410 2023-10-05T19:40:18.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.015+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(104 notify.1 [watch unwatch cookie 94087536569616] v28'23 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6ed800a5a0 con 0x5592762e4410 2023-10-05T19:40:18.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.015+0000 7f6ee7e0be40 20 remove_watcher() i=1 2023-10-05T19:40:18.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.015+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:105 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94087536574112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927639f600 con 0x559276318020 2023-10-05T19:40:18.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.019+0000 7f6ee513f700 1 -- 172.21.15.12:0/1090226059 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(105 notify.2 [watch unwatch cookie 94087536574112] v28'23 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6ed004b020 con 0x559276318020 2023-10-05T19:40:18.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.019+0000 7f6ee7e0be40 20 remove_watcher() i=2 2023-10-05T19:40:18.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.019+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:106 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94087536570720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927639f600 con 0x5592763136a0 2023-10-05T19:40:18.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.019+0000 7f6ee5940700 1 -- 172.21.15.12:0/1090226059 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(106 notify.3 [watch unwatch cookie 94087536570720] v28'23 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6ee0014070 con 0x5592763136a0 2023-10-05T19:40:18.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.019+0000 7f6ee7e0be40 20 remove_watcher() i=3 2023-10-05T19:40:18.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.019+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:107 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94087536580704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927639f600 con 0x5592762e4410 2023-10-05T19:40:18.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.023+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(107 notify.4 [watch unwatch cookie 94087536580704] v28'23 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6ed800a5a0 con 0x5592762e4410 2023-10-05T19:40:18.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.023+0000 7f6ee7e0be40 20 remove_watcher() i=4 2023-10-05T19:40:18.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.023+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:108 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94087536584592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927639f600 con 0x7f6ee0016730 2023-10-05T19:40:18.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.027+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(108 notify.5 [watch unwatch cookie 94087536584592] v28'23 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6ed800a5a0 con 0x7f6ee0016730 2023-10-05T19:40:18.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.027+0000 7f6ee7e0be40 20 remove_watcher() i=5 2023-10-05T19:40:18.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.027+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:109 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94087536582656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927639f600 con 0x7f6ee0016730 2023-10-05T19:40:18.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.031+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(109 notify.6 [watch unwatch cookie 94087536582656] v28'23 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6ed800a5a0 con 0x7f6ee0016730 2023-10-05T19:40:18.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.031+0000 7f6ee7e0be40 20 remove_watcher() i=6 2023-10-05T19:40:18.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.031+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:110 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94087536585696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927639f600 con 0x7f6ee0016730 2023-10-05T19:40:18.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.031+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(110 notify.7 [watch unwatch cookie 94087536585696] v28'23 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6ed800a5a0 con 0x7f6ee0016730 2023-10-05T19:40:18.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.031+0000 7f6ee7e0be40 20 remove_watcher() i=7 2023-10-05T19:40:18.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.035+0000 7f6eb77de700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:18.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.035+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:111 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94087536590144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55927639f600 con 0x5592762e4410 2023-10-05T19:40:18.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.039+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(111 data_loggenerations_metadata [watch unwatch cookie 94087536590144] v28'49 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f6ed800a5a0 con 0x5592762e4410 2023-10-05T19:40:18.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.039+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f6ee0016730 msgr2=0x7f6ee0042130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.039+0000 7f6ee7e0be40 1 --2- 172.21.15.12:0/1090226059 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f6ee0016730 0x7f6ee0042130 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.039+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5592762e4410 msgr2=0x559276311500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.039+0000 7f6ee7e0be40 1 --2- 172.21.15.12:0/1090226059 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5592762e4410 0x559276311500 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.039+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5592763136a0 msgr2=0x559276315a80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.039+0000 7f6ee7e0be40 1 --2- 172.21.15.12:0/1090226059 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5592763136a0 0x559276315a80 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.039+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x559276318020 msgr2=0x55927631a400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.039+0000 7f6ee7e0be40 1 --2- 172.21.15.12:0/1090226059 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x559276318020 0x55927631a400 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.039+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f6ec8030c40 msgr2=0x7f6ec80330f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.039+0000 7f6ee7e0be40 1 --2- 172.21.15.12:0/1090226059 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f6ec8030c40 0x7f6ec80330f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6ed000afd0 tx=0x7f6ed0014000 comp rx=0 tx=0).stop 2023-10-05T19:40:18.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.039+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592762dfe30 msgr2=0x559276305b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.039+0000 7f6ee7e0be40 1 --2- 172.21.15.12:0/1090226059 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592762dfe30 0x559276305b40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x5592762fb440 tx=0x7f6ee0016000 comp rx=0 tx=0).stop 2023-10-05T19:40:18.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.039+0000 7f6ee6141700 1 -- 172.21.15.12:0/1090226059 reap_dead start 2023-10-05T19:40:18.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.039+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 shutdown_connections 2023-10-05T19:40:18.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.039+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 >> 172.21.15.12:0/1090226059 conn(0x5592762db850 msgr2=0x5592762f93a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:18.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.039+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 shutdown_connections 2023-10-05T19:40:18.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.039+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/1090226059 wait complete. 2023-10-05T19:40:18.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.039+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/3237908763 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f6e2c030c50 msgr2=0x7f6e2c033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.039+0000 7f6ee7e0be40 1 --2- 172.21.15.12:0/3237908763 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f6e2c030c50 0x7f6e2c033100 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x559276323930 tx=0x7f6ed800ae90 comp rx=0 tx=0).stop 2023-10-05T19:40:18.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.039+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/3237908763 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592763313f0 msgr2=0x559276336f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.039+0000 7f6ee7e0be40 1 --2- 172.21.15.12:0/3237908763 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592763313f0 0x559276336f10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x55927632b4d0 tx=0x7f6ed003c000 comp rx=0 tx=0).stop 2023-10-05T19:40:18.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.039+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/3237908763 shutdown_connections 2023-10-05T19:40:18.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.039+0000 7f6ee7e0be40 1 --2- 172.21.15.12:0/3237908763 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f6e2c030c50 0x7f6e2c033100 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.039+0000 7f6ee7e0be40 1 --2- 172.21.15.12:0/3237908763 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5592763313f0 0x559276336f10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.039+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/3237908763 >> 172.21.15.12:0/3237908763 conn(0x55927632c7b0 msgr2=0x55927632a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:18.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.039+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/3237908763 shutdown_connections 2023-10-05T19:40:18.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.039+0000 7f6ee7e0be40 1 -- 172.21.15.12:0/3237908763 wait complete. 2023-10-05T19:40:18.054 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-10-05T19:40:18.055 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'foo', '--display-name', 'Foo', '--email', 'foo@foo.com', '--access-key', '9te6NH5mcdcq0Tc5i8i1', '--secret', 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', '--max-buckets', '4'] 2023-10-05T19:40:18.055 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'foo', '--display-name', 'Foo', '--email', 'foo@foo.com', '--access-key', '9te6NH5mcdcq0Tc5i8i1', '--secret', 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', '--max-buckets', '4'] 2023-10-05T19:40:18.055 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid foo --display-name Foo --email foo@foo.com --access-key 9te6NH5mcdcq0Tc5i8i1 --secret Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu --max-buckets 4 2023-10-05T19:40:18.148 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:18.148 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:18.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.139+0000 7fa9eeed0e40 1 Processor -- start 2023-10-05T19:40:18.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.139+0000 7fa9eeed0e40 1 -- start start 2023-10-05T19:40:18.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.143+0000 7fa9eeed0e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55c8c3c94fb0 0x55c8c3c95380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.143+0000 7fa9eeed0e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55c8c3bdfbd0 con 0x55c8c3c958c0 2023-10-05T19:40:18.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.143+0000 7fa9eeed0e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x55c8c3b9cee0 con 0x55c8c3c94fb0 2023-10-05T19:40:18.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.143+0000 7fa9eca05700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55c8c3c94fb0 0x55c8c3c95380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.143+0000 7fa9ec204700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x55c8c3c958c0 0x55c8c3c96360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:47208/0 (socket says 172.21.15.12:47208) 2023-10-05T19:40:18.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.143+0000 7fa9ec204700 1 -- 172.21.15.12:0/570689649 learned_addr learned my addr 172.21.15.12:0/570689649 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:18.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.143+0000 7fa9eba03700 1 -- 172.21.15.12:0/570689649 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3565487314 0 0) 0x55c8c3bdfbd0 con 0x55c8c3c958c0 2023-10-05T19:40:18.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.143+0000 7fa9eba03700 1 -- 172.21.15.12:0/570689649 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9d8001e10 con 0x55c8c3c958c0 2023-10-05T19:40:18.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.143+0000 7fa9eca05700 1 -- 172.21.15.12:0/570689649 >> v1:172.21.15.8:6789/0 conn(0x55c8c3c958c0 legacy=0x55c8c3c96360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.143+0000 7fa9eca05700 1 -- 172.21.15.12:0/570689649 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8c3c96a70 con 0x55c8c3c94fb0 2023-10-05T19:40:18.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.143+0000 7fa9eca05700 1 --2- 172.21.15.12:0/570689649 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c8c3c94fb0 0x55c8c3c95380 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc00a5d0 tx=0x7fa9dc005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=724cf9a1edffcc51 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.143+0000 7fa9eba03700 1 -- 172.21.15.12:0/570689649 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9dc007610 con 0x55c8c3c94fb0 2023-10-05T19:40:18.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.143+0000 7fa9eba03700 1 -- 172.21.15.12:0/570689649 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9dc007770 con 0x55c8c3c94fb0 2023-10-05T19:40:18.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.143+0000 7fa9eba03700 1 -- 172.21.15.12:0/570689649 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9dc007a40 con 0x55c8c3c94fb0 2023-10-05T19:40:18.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.143+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/570689649 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c8c3c94fb0 msgr2=0x55c8c3c95380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.143+0000 7fa9eeed0e40 1 --2- 172.21.15.12:0/570689649 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c8c3c94fb0 0x55c8c3c95380 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc00a5d0 tx=0x7fa9dc005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:18.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.143+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/570689649 shutdown_connections 2023-10-05T19:40:18.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.143+0000 7fa9eeed0e40 1 --2- 172.21.15.12:0/570689649 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c8c3c94fb0 0x55c8c3c95380 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.143+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/570689649 >> 172.21.15.12:0/570689649 conn(0x55c8c3c08800 msgr2=0x55c8c3c08c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:18.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.143+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/570689649 shutdown_connections 2023-10-05T19:40:18.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.143+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/570689649 wait complete. 2023-10-05T19:40:18.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.143+0000 7fa9eeed0e40 1 Processor -- start 2023-10-05T19:40:18.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.143+0000 7fa9eeed0e40 1 -- start start 2023-10-05T19:40:18.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.143+0000 7fa9eeed0e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c8c3c94fb0 0x55c8c3cb3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.147+0000 7fa9eeed0e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8c3b60e30 con 0x55c8c3c94fb0 2023-10-05T19:40:18.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.147+0000 7fa9eca05700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c8c3c94fb0 0x55c8c3cb3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.147+0000 7fa9eca05700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c8c3c94fb0 0x55c8c3cb3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36618/0 (socket says 172.21.15.12:36618) 2023-10-05T19:40:18.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.147+0000 7fa9eca05700 1 -- 172.21.15.12:0/1399979057 learned_addr learned my addr 172.21.15.12:0/1399979057 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:18.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.147+0000 7fa9eca05700 1 -- 172.21.15.12:0/1399979057 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9dc005cf0 con 0x55c8c3c94fb0 2023-10-05T19:40:18.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.147+0000 7fa9eca05700 1 --2- 172.21.15.12:0/1399979057 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c8c3c94fb0 0x55c8c3cb3f50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc000c00 tx=0x7fa9dc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99b131921999bbb server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.147+0000 7fa9eaa01700 1 -- 172.21.15.12:0/1399979057 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9dc0078e0 con 0x55c8c3c94fb0 2023-10-05T19:40:18.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.147+0000 7fa9eaa01700 1 -- 172.21.15.12:0/1399979057 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9dc007a40 con 0x55c8c3c94fb0 2023-10-05T19:40:18.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.147+0000 7fa9eaa01700 1 -- 172.21.15.12:0/1399979057 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9dc007bd0 con 0x55c8c3c94fb0 2023-10-05T19:40:18.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.147+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/1399979057 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c8c3c94fb0 msgr2=0x55c8c3cb3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.147+0000 7fa9eeed0e40 1 --2- 172.21.15.12:0/1399979057 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c8c3c94fb0 0x55c8c3cb3f50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc000c00 tx=0x7fa9dc014000 comp rx=0 tx=0).stop 2023-10-05T19:40:18.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.147+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/1399979057 shutdown_connections 2023-10-05T19:40:18.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.147+0000 7fa9eeed0e40 1 --2- 172.21.15.12:0/1399979057 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c8c3c94fb0 0x55c8c3cb3f50 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.147+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/1399979057 >> 172.21.15.12:0/1399979057 conn(0x55c8c3c08800 msgr2=0x55c8c3cafb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:18.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.147+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/1399979057 shutdown_connections 2023-10-05T19:40:18.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.147+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/1399979057 wait complete. 2023-10-05T19:40:18.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.147+0000 7fa9eeed0e40 1 Processor -- start 2023-10-05T19:40:18.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.147+0000 7fa9eeed0e40 1 -- start start 2023-10-05T19:40:18.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.147+0000 7fa9eeed0e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c8c3c94fb0 0x55c8c3cbac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.147+0000 7fa9eeed0e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8c3b33ed0 con 0x55c8c3c94fb0 2023-10-05T19:40:18.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.147+0000 7fa9eca05700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c8c3c94fb0 0x55c8c3cbac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.147+0000 7fa9eca05700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c8c3c94fb0 0x55c8c3cbac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36630/0 (socket says 172.21.15.12:36630) 2023-10-05T19:40:18.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.147+0000 7fa9eca05700 1 -- 172.21.15.12:0/3501119661 learned_addr learned my addr 172.21.15.12:0/3501119661 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:18.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.151+0000 7fa9eca05700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8c3c96a70 con 0x55c8c3c94fb0 2023-10-05T19:40:18.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.151+0000 7fa9eca05700 1 --2- 172.21.15.12:0/3501119661 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c8c3c94fb0 0x55c8c3cbac10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc01f040 tx=0x7fa9dc01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.151+0000 7fa9e91fe700 1 -- 172.21.15.12:0/3501119661 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9dc007bb0 con 0x55c8c3c94fb0 2023-10-05T19:40:18.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.151+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c8c3cbb150 con 0x55c8c3c94fb0 2023-10-05T19:40:18.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.151+0000 7fa9e91fe700 1 -- 172.21.15.12:0/3501119661 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9dc029070 con 0x55c8c3c94fb0 2023-10-05T19:40:18.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.151+0000 7fa9e91fe700 1 -- 172.21.15.12:0/3501119661 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9dc012610 con 0x55c8c3c94fb0 2023-10-05T19:40:18.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.151+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c8c3cbb590 con 0x55c8c3c94fb0 2023-10-05T19:40:18.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.151+0000 7fa9eeed0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.151+0000 7fa9e91fe700 1 -- 172.21.15.12:0/3501119661 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fa9dc012770 con 0x55c8c3c94fb0 2023-10-05T19:40:18.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.155+0000 7fa9e91fe700 1 --2- 172.21.15.12:0/3501119661 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fa9cc030ca0 0x7fa9cc033150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.155+0000 7fa9e91fe700 1 -- 172.21.15.12:0/3501119661 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7fa9dc02b070 con 0x55c8c3c94fb0 2023-10-05T19:40:18.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.155+0000 7fa9ec204700 1 --2- 172.21.15.12:0/3501119661 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fa9cc030ca0 0x7fa9cc033150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.155+0000 7fa9eeed0e40 1 --2- 172.21.15.12:0/3501119661 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55c8c3c958c0 0x55c8c3cc66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.155+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cc6d40 con 0x55c8c3c958c0 2023-10-05T19:40:18.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.155+0000 7fa9ed206700 1 --2- 172.21.15.12:0/3501119661 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55c8c3c958c0 0x55c8c3cc66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.155+0000 7fa9ec204700 1 --2- 172.21.15.12:0/3501119661 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fa9cc030ca0 0x7fa9cc033150 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4004170 tx=0x7fa9d4001400 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.155+0000 7fa9ed206700 1 --2- 172.21.15.12:0/3501119661 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55c8c3c958c0 0x55c8c3cc66d0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.155+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa9e400a5a0 con 0x55c8c3c958c0 2023-10-05T19:40:18.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.155+0000 7fa9eeed0e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:18.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.155+0000 7fa9eeed0e40 20 realm 2023-10-05T19:40:18.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.155+0000 7fa9eeed0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.155+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cc3d30 con 0x55c8c3c958c0 2023-10-05T19:40:18.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.155+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa9e400a5a0 con 0x55c8c3c958c0 2023-10-05T19:40:18.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.155+0000 7fa9eeed0e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:18.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.155+0000 7fa9eeed0e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:18.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.155+0000 7fa9eeed0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.155+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cc7820 con 0x55c8c3c958c0 2023-10-05T19:40:18.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.155+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa9e400a5a0 con 0x55c8c3c958c0 2023-10-05T19:40:18.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.155+0000 7fa9eeed0e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:18.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.155+0000 7fa9eeed0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.155+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cc7430 con 0x55c8c3c958c0 2023-10-05T19:40:18.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.155+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fa9e400a5a0 con 0x55c8c3c958c0 2023-10-05T19:40:18.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.155+0000 7fa9eeed0e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:18.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.155+0000 7fa9eeed0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.155+0000 7fa9eeed0e40 1 --2- 172.21.15.12:0/3501119661 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55c8c3cc8bc0 0x55c8c3ccafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.155+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3ccb4e0 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.159+0000 7fa9eca05700 1 --2- 172.21.15.12:0/3501119661 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55c8c3cc8bc0 0x55c8c3ccafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.159+0000 7fa9eca05700 1 --2- 172.21.15.12:0/3501119661 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55c8c3cc8bc0 0x55c8c3ccafa0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.159+0000 7fa9eca05700 1 -- 172.21.15.12:0/3501119661 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fa9dc041040 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.159+0000 7fa9eeed0e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:18.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.159+0000 7fa9eeed0e40 20 searching for the correct realm 2023-10-05T19:40:18.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.159+0000 7fa9eeed0e40 1 --2- 172.21.15.12:0/3501119661 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55c8c3ccd360 0x55c8c3ccf770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.159+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3ccfe20 con 0x55c8c3ccd360 2023-10-05T19:40:18.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.159+0000 7fa9ec204700 1 --2- 172.21.15.12:0/3501119661 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55c8c3ccd360 0x55c8c3ccf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.159+0000 7fa9ec204700 1 --2- 172.21.15.12:0/3501119661 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55c8c3ccd360 0x55c8c3ccf770 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.159+0000 7fa9ec204700 1 -- 172.21.15.12:0/3501119661 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fa9d4006070 con 0x55c8c3ccd360 2023-10-05T19:40:18.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.159+0000 7fa9ec204700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3ccfe20 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.163+0000 7fa9eca05700 1 -- 172.21.15.12:0/3501119661 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9dc041040 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.163+0000 7fa9eca05700 1 --2- 172.21.15.12:0/3501119661 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fa9dc007030 0x7fa9dc046050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.163+0000 7fa9eca05700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3ccfe20 con 0x7fa9dc007030 2023-10-05T19:40:18.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.163+0000 7fa9ed206700 1 --2- 172.21.15.12:0/3501119661 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fa9dc007030 0x7fa9dc046050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.163+0000 7fa9ed206700 1 --2- 172.21.15.12:0/3501119661 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fa9dc007030 0x7fa9dc046050 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.163+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9e400a5a0 con 0x7fa9dc007030 2023-10-05T19:40:18.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.163+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3ccfe20 con 0x55c8c3c958c0 2023-10-05T19:40:18.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.163+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9e400a5a0 con 0x55c8c3c958c0 2023-10-05T19:40:18.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.163+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3ccfe20 con 0x55c8c3c958c0 2023-10-05T19:40:18.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.167+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9e400a5a0 con 0x55c8c3c958c0 2023-10-05T19:40:18.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.167+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3ccfe20 con 0x7fa9dc007030 2023-10-05T19:40:18.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.167+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9e400a5a0 con 0x7fa9dc007030 2023-10-05T19:40:18.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.167+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3ccfe20 con 0x7fa9dc007030 2023-10-05T19:40:18.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.167+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9e400a5a0 con 0x7fa9dc007030 2023-10-05T19:40:18.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.167+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3ccfe20 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.167+0000 7fa9eca05700 1 -- 172.21.15.12:0/3501119661 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9dc041040 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.167+0000 7fa9eca05700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3ccfe20 con 0x55c8c3ccd360 2023-10-05T19:40:18.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.167+0000 7fa9ec204700 1 -- 172.21.15.12:0/3501119661 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9d4006070 con 0x55c8c3ccd360 2023-10-05T19:40:18.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.167+0000 7fa9ec204700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3ccfe20 con 0x55c8c3c958c0 2023-10-05T19:40:18.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.171+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9e400a5a0 con 0x55c8c3c958c0 2023-10-05T19:40:18.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.171+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3ccfe20 con 0x7fa9dc007030 2023-10-05T19:40:18.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.171+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9e400a5a0 con 0x7fa9dc007030 2023-10-05T19:40:18.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.171+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3ccfe20 con 0x55c8c3c958c0 2023-10-05T19:40:18.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.171+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9e400a5a0 con 0x55c8c3c958c0 2023-10-05T19:40:18.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.171+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3ccfe20 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.171+0000 7fa9eca05700 1 -- 172.21.15.12:0/3501119661 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9dc041040 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.171+0000 7fa9eca05700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3ccfe20 con 0x55c8c3ccd360 2023-10-05T19:40:18.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.171+0000 7fa9ec204700 1 -- 172.21.15.12:0/3501119661 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9d4006070 con 0x55c8c3ccd360 2023-10-05T19:40:18.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.171+0000 7fa9ec204700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3ccfe20 con 0x55c8c3ccd360 2023-10-05T19:40:18.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.175+0000 7fa9ec204700 1 -- 172.21.15.12:0/3501119661 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9d4006070 con 0x55c8c3ccd360 2023-10-05T19:40:18.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.175+0000 7fa9ec204700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3ccfe20 con 0x7fa9dc007030 2023-10-05T19:40:18.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.175+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9e400a5a0 con 0x7fa9dc007030 2023-10-05T19:40:18.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.175+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3ccfe20 con 0x55c8c3ccd360 2023-10-05T19:40:18.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.175+0000 7fa9ec204700 1 -- 172.21.15.12:0/3501119661 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9d4006070 con 0x55c8c3ccd360 2023-10-05T19:40:18.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.175+0000 7fa9ec204700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3ccfe20 con 0x55c8c3ccd360 2023-10-05T19:40:18.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.175+0000 7fa9ec204700 1 -- 172.21.15.12:0/3501119661 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9d4006070 con 0x55c8c3ccd360 2023-10-05T19:40:18.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.175+0000 7fa9ec204700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3ccfe20 con 0x55c8c3c958c0 2023-10-05T19:40:18.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.175+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9e400a5a0 con 0x55c8c3c958c0 2023-10-05T19:40:18.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.175+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3ccfe20 con 0x55c8c3c958c0 2023-10-05T19:40:18.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.179+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9e400a5a0 con 0x55c8c3c958c0 2023-10-05T19:40:18.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.179+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3ccfe20 con 0x7fa9dc007030 2023-10-05T19:40:18.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.179+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9e400a5a0 con 0x7fa9dc007030 2023-10-05T19:40:18.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.179+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3ccfe20 con 0x55c8c3ccd360 2023-10-05T19:40:18.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.179+0000 7fa9ec204700 1 -- 172.21.15.12:0/3501119661 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9d4006070 con 0x55c8c3ccd360 2023-10-05T19:40:18.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.179+0000 7fa9ec204700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3ccfe20 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.179+0000 7fa9eca05700 1 -- 172.21.15.12:0/3501119661 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9dc041040 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.179+0000 7fa9eca05700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3ccfe20 con 0x55c8c3c958c0 2023-10-05T19:40:18.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.183+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9e400a5a0 con 0x55c8c3c958c0 2023-10-05T19:40:18.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.183+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3ccfe20 con 0x7fa9dc007030 2023-10-05T19:40:18.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.183+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9e400a5a0 con 0x7fa9dc007030 2023-10-05T19:40:18.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.183+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3ccfe20 con 0x55c8c3ccd360 2023-10-05T19:40:18.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.183+0000 7fa9ec204700 1 -- 172.21.15.12:0/3501119661 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9d4006070 con 0x55c8c3ccd360 2023-10-05T19:40:18.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.183+0000 7fa9ec204700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3ccfe20 con 0x7fa9dc007030 2023-10-05T19:40:18.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.183+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9e400a5a0 con 0x7fa9dc007030 2023-10-05T19:40:18.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.183+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3ccfe20 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.187+0000 7fa9eca05700 1 -- 172.21.15.12:0/3501119661 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9dc041040 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.187+0000 7fa9eca05700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3ccfe20 con 0x55c8c3ccd360 2023-10-05T19:40:18.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.187+0000 7fa9ec204700 1 -- 172.21.15.12:0/3501119661 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9d4006070 con 0x55c8c3ccd360 2023-10-05T19:40:18.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.187+0000 7fa9ec204700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3ccfe20 con 0x7fa9dc007030 2023-10-05T19:40:18.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.187+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9e400a5a0 con 0x7fa9dc007030 2023-10-05T19:40:18.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.187+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3ccfe20 con 0x55c8c3c958c0 2023-10-05T19:40:18.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.187+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fa9e400a5a0 con 0x55c8c3c958c0 2023-10-05T19:40:18.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.187+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3ccfe20 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.187+0000 7fa9eca05700 1 -- 172.21.15.12:0/3501119661 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fa9dc041040 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.191+0000 7fa9eeed0e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:18.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.191+0000 7fa9eeed0e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:18.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.191+0000 7fa9eeed0e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:18.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.191+0000 7fa9eeed0e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:18.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.191+0000 7fa9eeed0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.191+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3ccc110 con 0x55c8c3c958c0 2023-10-05T19:40:18.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.191+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa9e400a5a0 con 0x55c8c3c958c0 2023-10-05T19:40:18.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.191+0000 7fa9eeed0e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:18.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.191+0000 7fa9eeed0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.191+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3ccbd40 con 0x55c8c3c958c0 2023-10-05T19:40:18.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.191+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fa9e400a5a0 con 0x55c8c3c958c0 2023-10-05T19:40:18.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.191+0000 7fa9eeed0e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:18.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.191+0000 7fa9eeed0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.191+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd0290 con 0x55c8c3ccd360 2023-10-05T19:40:18.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.191+0000 7fa9ec204700 1 -- 172.21.15.12:0/3501119661 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fa9d4006070 con 0x55c8c3ccd360 2023-10-05T19:40:18.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.191+0000 7fa9eeed0e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:18.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.191+0000 7fa9eeed0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.191+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd0640 con 0x55c8c3ccd360 2023-10-05T19:40:18.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.191+0000 7fa9ec204700 1 -- 172.21.15.12:0/3501119661 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7fa9d4006070 con 0x55c8c3ccd360 2023-10-05T19:40:18.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.191+0000 7fa9eeed0e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:18.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.191+0000 7fa9eeed0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.191+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd09f0 con 0x55c8c3c958c0 2023-10-05T19:40:18.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.191+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa9e400a5a0 con 0x55c8c3c958c0 2023-10-05T19:40:18.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.191+0000 7fa9eeed0e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:18.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.191+0000 7fa9eeed0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.191+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3ccc600 con 0x55c8c3c958c0 2023-10-05T19:40:18.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.195+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fa9e400a5a0 con 0x55c8c3c958c0 2023-10-05T19:40:18.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.195+0000 7fa9eeed0e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:18.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.195+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3ccc5d0 con 0x55c8c3ccd360 2023-10-05T19:40:18.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.195+0000 7fa9ec204700 1 -- 172.21.15.12:0/3501119661 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fa9d4006070 con 0x55c8c3ccd360 2023-10-05T19:40:18.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.195+0000 7fa9ec204700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3ccc5d0 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.195+0000 7fa9eca05700 1 -- 172.21.15.12:0/3501119661 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9dc041040 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.195+0000 7fa9eca05700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3ccc5d0 con 0x7fa9dc007030 2023-10-05T19:40:18.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.199+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9e400a5a0 con 0x7fa9dc007030 2023-10-05T19:40:18.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.199+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd09f0 con 0x55c8c3c958c0 2023-10-05T19:40:18.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.199+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9e400a5a0 con 0x55c8c3c958c0 2023-10-05T19:40:18.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.199+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd09f0 con 0x55c8c3c958c0 2023-10-05T19:40:18.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.199+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9e400a5a0 con 0x55c8c3c958c0 2023-10-05T19:40:18.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.199+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd09f0 con 0x7fa9dc007030 2023-10-05T19:40:18.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.199+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9e400a5a0 con 0x7fa9dc007030 2023-10-05T19:40:18.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.199+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd09f0 con 0x7fa9dc007030 2023-10-05T19:40:18.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.203+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9e400a5a0 con 0x7fa9dc007030 2023-10-05T19:40:18.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.203+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd09f0 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.203+0000 7fa9eca05700 1 -- 172.21.15.12:0/3501119661 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9dc041040 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.203+0000 7fa9eca05700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd09f0 con 0x55c8c3ccd360 2023-10-05T19:40:18.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.203+0000 7fa9ec204700 1 -- 172.21.15.12:0/3501119661 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9d4006070 con 0x55c8c3ccd360 2023-10-05T19:40:18.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.203+0000 7fa9ec204700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd09f0 con 0x55c8c3c958c0 2023-10-05T19:40:18.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.203+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9e400a5a0 con 0x55c8c3c958c0 2023-10-05T19:40:18.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.203+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd09f0 con 0x7fa9dc007030 2023-10-05T19:40:18.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.203+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9e400a5a0 con 0x7fa9dc007030 2023-10-05T19:40:18.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.203+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd09f0 con 0x55c8c3c958c0 2023-10-05T19:40:18.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.207+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9e400a5a0 con 0x55c8c3c958c0 2023-10-05T19:40:18.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.207+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd09f0 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.207+0000 7fa9eca05700 1 -- 172.21.15.12:0/3501119661 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9dc041040 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.207+0000 7fa9eca05700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd09f0 con 0x55c8c3ccd360 2023-10-05T19:40:18.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.207+0000 7fa9ec204700 1 -- 172.21.15.12:0/3501119661 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9d4006070 con 0x55c8c3ccd360 2023-10-05T19:40:18.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.207+0000 7fa9ec204700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd09f0 con 0x55c8c3ccd360 2023-10-05T19:40:18.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.207+0000 7fa9ec204700 1 -- 172.21.15.12:0/3501119661 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9d4006070 con 0x55c8c3ccd360 2023-10-05T19:40:18.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.207+0000 7fa9ec204700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd09f0 con 0x7fa9dc007030 2023-10-05T19:40:18.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.211+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9e400a5a0 con 0x7fa9dc007030 2023-10-05T19:40:18.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.211+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd09f0 con 0x55c8c3ccd360 2023-10-05T19:40:18.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.211+0000 7fa9ec204700 1 -- 172.21.15.12:0/3501119661 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9d4006070 con 0x55c8c3ccd360 2023-10-05T19:40:18.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.211+0000 7fa9ec204700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd09f0 con 0x55c8c3ccd360 2023-10-05T19:40:18.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.211+0000 7fa9ec204700 1 -- 172.21.15.12:0/3501119661 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9d4006070 con 0x55c8c3ccd360 2023-10-05T19:40:18.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.211+0000 7fa9ec204700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd09f0 con 0x55c8c3c958c0 2023-10-05T19:40:18.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.211+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9e400a5a0 con 0x55c8c3c958c0 2023-10-05T19:40:18.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.211+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd09f0 con 0x55c8c3c958c0 2023-10-05T19:40:18.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.211+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9e400a5a0 con 0x55c8c3c958c0 2023-10-05T19:40:18.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.211+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd09f0 con 0x7fa9dc007030 2023-10-05T19:40:18.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.215+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9e400a5a0 con 0x7fa9dc007030 2023-10-05T19:40:18.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.215+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd09f0 con 0x55c8c3ccd360 2023-10-05T19:40:18.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.215+0000 7fa9ec204700 1 -- 172.21.15.12:0/3501119661 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9d4006070 con 0x55c8c3ccd360 2023-10-05T19:40:18.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.215+0000 7fa9ec204700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd09f0 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.215+0000 7fa9eca05700 1 -- 172.21.15.12:0/3501119661 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9dc041040 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.215+0000 7fa9eca05700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd09f0 con 0x55c8c3c958c0 2023-10-05T19:40:18.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.215+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9e400a5a0 con 0x55c8c3c958c0 2023-10-05T19:40:18.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.215+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd09f0 con 0x7fa9dc007030 2023-10-05T19:40:18.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.219+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9e400a5a0 con 0x7fa9dc007030 2023-10-05T19:40:18.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.219+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd09f0 con 0x55c8c3ccd360 2023-10-05T19:40:18.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.219+0000 7fa9ec204700 1 -- 172.21.15.12:0/3501119661 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9d4006070 con 0x55c8c3ccd360 2023-10-05T19:40:18.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.219+0000 7fa9ec204700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd09f0 con 0x7fa9dc007030 2023-10-05T19:40:18.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.219+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9e400a5a0 con 0x7fa9dc007030 2023-10-05T19:40:18.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.219+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd09f0 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.219+0000 7fa9eca05700 1 -- 172.21.15.12:0/3501119661 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9dc041040 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.219+0000 7fa9eca05700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd09f0 con 0x55c8c3ccd360 2023-10-05T19:40:18.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.219+0000 7fa9ec204700 1 -- 172.21.15.12:0/3501119661 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9d4006070 con 0x55c8c3ccd360 2023-10-05T19:40:18.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.219+0000 7fa9ec204700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd09f0 con 0x7fa9dc007030 2023-10-05T19:40:18.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.223+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa9e400a5a0 con 0x7fa9dc007030 2023-10-05T19:40:18.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.223+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd09f0 con 0x55c8c3c958c0 2023-10-05T19:40:18.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.223+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fa9e400a5a0 con 0x55c8c3c958c0 2023-10-05T19:40:18.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.223+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd09f0 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.223+0000 7fa9eca05700 1 -- 172.21.15.12:0/3501119661 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fa9dc041040 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.223+0000 7fa9eeed0e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:18.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.223+0000 7fa9eeed0e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:18.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.223+0000 7fa9eeed0e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:18.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.223+0000 7fa9eeed0e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:18.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.223+0000 7fa9eeed0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.223+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3ccc5d0 con 0x55c8c3c958c0 2023-10-05T19:40:18.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.223+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fa9e400a5a0 con 0x55c8c3c958c0 2023-10-05T19:40:18.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.223+0000 7fa9eeed0e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:18.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.223+0000 7fa9eeed0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.223+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd1330 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.227+0000 7fa9eca05700 1 -- 172.21.15.12:0/3501119661 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fa9dc041040 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.227+0000 7fa9eeed0e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:18.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.227+0000 7fa9eeed0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.227+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd1bd0 con 0x55c8c3c958c0 2023-10-05T19:40:18.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.227+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fa9e400a5a0 con 0x55c8c3c958c0 2023-10-05T19:40:18.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.227+0000 7fa9eeed0e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:18.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.227+0000 7fa9eeed0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.227+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd1bd0 con 0x55c8c3ccd360 2023-10-05T19:40:18.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.227+0000 7fa9ec204700 1 -- 172.21.15.12:0/3501119661 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fa9d4006070 con 0x55c8c3ccd360 2023-10-05T19:40:18.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.227+0000 7fa9eeed0e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:18.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.227+0000 7fa9eeed0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.227+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd17e0 con 0x7fa9dc007030 2023-10-05T19:40:18.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.227+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7fa9e400a5a0 con 0x7fa9dc007030 2023-10-05T19:40:18.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.227+0000 7fa9eeed0e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:18.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.227+0000 7fa9eeed0e40 20 zone default found 2023-10-05T19:40:18.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.227+0000 7fa9eeed0e40 4 Realm: () 2023-10-05T19:40:18.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.227+0000 7fa9eeed0e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:18.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.227+0000 7fa9eeed0e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:18.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.227+0000 7fa9eeed0e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:18.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.227+0000 7fa9eeed0e40 20 zonegroup default 2023-10-05T19:40:18.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.227+0000 7fa9eeed0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.227+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd17e0 con 0x7fa9dc007030 2023-10-05T19:40:18.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.227+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fa9e400a5a0 con 0x7fa9dc007030 2023-10-05T19:40:18.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.227+0000 7fa9eeed0e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:18.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.227+0000 7fa9eeed0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.231+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd17e0 con 0x55c8c3ccd360 2023-10-05T19:40:18.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.231+0000 7fa9ec204700 1 -- 172.21.15.12:0/3501119661 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fa9d4006070 con 0x55c8c3ccd360 2023-10-05T19:40:18.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.231+0000 7fa9eeed0e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:18.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.231+0000 7fa9eeed0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.231+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd3df0 con 0x55c8c3ccd360 2023-10-05T19:40:18.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.231+0000 7fa9ec204700 1 -- 172.21.15.12:0/3501119661 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fa9d4006070 con 0x55c8c3ccd360 2023-10-05T19:40:18.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.231+0000 7fa9eeed0e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:18.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.231+0000 7fa9eeed0e40 20 started sync module instance, tier type = 2023-10-05T19:40:18.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.231+0000 7fa9eeed0e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:18.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.231+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd34b0 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.235+0000 7fa9eca05700 1 -- 172.21.15.12:0/3501119661 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'24 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa9dc041040 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.235+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94320766826672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd30e0 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.235+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd4b80 con 0x55c8c3c958c0 2023-10-05T19:40:18.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.239+0000 7fa9eca05700 1 -- 172.21.15.12:0/3501119661 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94320766826672] v28'25 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa9dc041040 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.239+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'24 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa9e400a5a0 con 0x55c8c3c958c0 2023-10-05T19:40:18.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.239+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94320766829248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd4f80 con 0x55c8c3c958c0 2023-10-05T19:40:18.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.239+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd5820 con 0x55c8c3ccd360 2023-10-05T19:40:18.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.239+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94320766829248] v28'25 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa9e400a5a0 con 0x55c8c3c958c0 2023-10-05T19:40:18.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.243+0000 7fa9ec204700 1 -- 172.21.15.12:0/3501119661 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'24 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa9d4006070 con 0x55c8c3ccd360 2023-10-05T19:40:18.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.243+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94320766832512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd53d0 con 0x55c8c3ccd360 2023-10-05T19:40:18.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.243+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd6600 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.243+0000 7fa9eca05700 1 -- 172.21.15.12:0/3501119661 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'24 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa9dc041040 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.243+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94320766837296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd6bb0 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.243+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd7710 con 0x55c8c3c958c0 2023-10-05T19:40:18.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.243+0000 7fa9ec204700 1 -- 172.21.15.12:0/3501119661 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94320766832512] v28'25 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa9d4006070 con 0x55c8c3ccd360 2023-10-05T19:40:18.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.247+0000 7fa9eca05700 1 -- 172.21.15.12:0/3501119661 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94320766837296] v28'25 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa9dc041040 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.247+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'24 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa9e400a5a0 con 0x55c8c3c958c0 2023-10-05T19:40:18.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.247+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94320766835744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd71e0 con 0x55c8c3c958c0 2023-10-05T19:40:18.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.247+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd8000 con 0x7fa9dc007030 2023-10-05T19:40:18.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.251+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94320766835744] v28'25 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa9e400a5a0 con 0x55c8c3c958c0 2023-10-05T19:40:18.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.251+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'24 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa9e400a5a0 con 0x7fa9dc007030 2023-10-05T19:40:18.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.251+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94320766838400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd7d10 con 0x7fa9dc007030 2023-10-05T19:40:18.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.251+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd85b0 con 0x7fa9dc007030 2023-10-05T19:40:18.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.251+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(95 notify.5 [watch watch cookie 94320766838400] v28'25 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa9e400a5a0 con 0x7fa9dc007030 2023-10-05T19:40:18.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.255+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(96 notify.6 [create] v28'24 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa9e400a5a0 con 0x7fa9dc007030 2023-10-05T19:40:18.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.255+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94320766843472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd8390 con 0x7fa9dc007030 2023-10-05T19:40:18.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.255+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd8f50 con 0x7fa9dc007030 2023-10-05T19:40:18.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.255+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(97 notify.6 [watch watch cookie 94320766843472] v28'25 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa9e400a5a0 con 0x7fa9dc007030 2023-10-05T19:40:18.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.255+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(98 notify.7 [create] v28'24 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa9e400a5a0 con 0x7fa9dc007030 2023-10-05T19:40:18.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.255+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94320766847520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd8390 con 0x7fa9dc007030 2023-10-05T19:40:18.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.255+0000 7fa9eeed0e40 20 add_watcher() i=0 2023-10-05T19:40:18.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.255+0000 7fa9eeed0e40 20 add_watcher() i=1 2023-10-05T19:40:18.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.255+0000 7fa9eeed0e40 20 add_watcher() i=2 2023-10-05T19:40:18.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.255+0000 7fa9eeed0e40 20 add_watcher() i=3 2023-10-05T19:40:18.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.255+0000 7fa9eeed0e40 20 add_watcher() i=4 2023-10-05T19:40:18.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.255+0000 7fa9eeed0e40 20 add_watcher() i=5 2023-10-05T19:40:18.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.255+0000 7fa9eeed0e40 20 add_watcher() i=6 2023-10-05T19:40:18.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.259+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94320766847520] v28'25 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa9e400a5a0 con 0x7fa9dc007030 2023-10-05T19:40:18.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.259+0000 7fa9eeed0e40 20 add_watcher() i=7 2023-10-05T19:40:18.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.259+0000 7fa9eeed0e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:18.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.259+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cdc2c0 con 0x55c8c3c958c0 2023-10-05T19:40:18.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.259+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fa9e400a5a0 con 0x55c8c3c958c0 2023-10-05T19:40:18.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.259+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94320766845040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd9780 con 0x55c8c3c958c0 2023-10-05T19:40:18.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.263+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94320766845040] v28'50 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fa9e400a5a0 con 0x55c8c3c958c0 2023-10-05T19:40:18.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.263+0000 7fa9be7e4700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:18.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.263+0000 7fa9eeed0e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:18.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.263+0000 7fa9eeed0e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:18.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.263+0000 7fa9eeed0e40 1 Processor -- start 2023-10-05T19:40:18.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.263+0000 7fa9eeed0e40 1 -- start start 2023-10-05T19:40:18.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.263+0000 7fa9eeed0e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c8c3ce6940 0x55c8c3ce6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.263+0000 7fa9eeed0e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8c3ce7250 con 0x55c8c3ce6940 2023-10-05T19:40:18.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.263+0000 7fa9ec204700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c8c3ce6940 0x55c8c3ce6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.263+0000 7fa9ec204700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c8c3ce6940 0x55c8c3ce6d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36636/0 (socket says 172.21.15.12:36636) 2023-10-05T19:40:18.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.263+0000 7fa9ec204700 1 -- 172.21.15.12:0/90406298 learned_addr learned my addr 172.21.15.12:0/90406298 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:18.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.263+0000 7fa9ec204700 1 -- 172.21.15.12:0/90406298 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8c3ce7390 con 0x55c8c3ce6940 2023-10-05T19:40:18.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.263+0000 7fa9ec204700 1 --2- 172.21.15.12:0/90406298 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c8c3ce6940 0x55c8c3ce6d10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x55c8c3cd01d0 tx=0x7fa9d401e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=443c379c09de33af server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.263+0000 7fa9bdfe3700 1 -- 172.21.15.12:0/90406298 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9d4013040 con 0x55c8c3ce6940 2023-10-05T19:40:18.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.263+0000 7fa9bdfe3700 1 -- 172.21.15.12:0/90406298 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d4022040 con 0x55c8c3ce6940 2023-10-05T19:40:18.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.263+0000 7fa9bdfe3700 1 -- 172.21.15.12:0/90406298 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9d400ee30 con 0x55c8c3ce6940 2023-10-05T19:40:18.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.267+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/90406298 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c8c3ce6940 msgr2=0x55c8c3ce6d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.267+0000 7fa9eeed0e40 1 --2- 172.21.15.12:0/90406298 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c8c3ce6940 0x55c8c3ce6d10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x55c8c3cd01d0 tx=0x7fa9d401e040 comp rx=0 tx=0).stop 2023-10-05T19:40:18.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.267+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/90406298 shutdown_connections 2023-10-05T19:40:18.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.267+0000 7fa9eeed0e40 1 --2- 172.21.15.12:0/90406298 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c8c3ce6940 0x55c8c3ce6d10 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.267+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/90406298 >> 172.21.15.12:0/90406298 conn(0x55c8c3ce1d00 msgr2=0x55c8c3cdfe50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:18.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.267+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/90406298 shutdown_connections 2023-10-05T19:40:18.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.267+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/90406298 wait complete. 2023-10-05T19:40:18.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.267+0000 7fa9eeed0e40 1 Processor -- start 2023-10-05T19:40:18.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.267+0000 7fa9eeed0e40 1 -- start start 2023-10-05T19:40:18.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.267+0000 7fa9eeed0e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c8c3ce6940 0x55c8c3cec4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.267+0000 7fa9eeed0e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8c3ced4e0 con 0x55c8c3ce6940 2023-10-05T19:40:18.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.267+0000 7fa9ec204700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c8c3ce6940 0x55c8c3cec4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.267+0000 7fa9ec204700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c8c3ce6940 0x55c8c3cec4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36648/0 (socket says 172.21.15.12:36648) 2023-10-05T19:40:18.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.267+0000 7fa9ec204700 1 -- 172.21.15.12:0/368035751 learned_addr learned my addr 172.21.15.12:0/368035751 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:18.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.267+0000 7fa9ec204700 1 -- 172.21.15.12:0/368035751 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d4034040 con 0x55c8c3ce6940 2023-10-05T19:40:18.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.267+0000 7fa9ec204700 1 --2- 172.21.15.12:0/368035751 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c8c3ce6940 0x55c8c3cec4e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa9d403e040 tx=0x7fa9d4040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.271+0000 7fa92ffff700 1 -- 172.21.15.12:0/368035751 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9d403be50 con 0x55c8c3ce6940 2023-10-05T19:40:18.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.271+0000 7fa92ffff700 1 -- 172.21.15.12:0/368035751 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d402c640 con 0x55c8c3ce6940 2023-10-05T19:40:18.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.271+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/368035751 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c8c3cecae0 con 0x55c8c3ce6940 2023-10-05T19:40:18.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.271+0000 7fa92ffff700 1 -- 172.21.15.12:0/368035751 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9d402c7b0 con 0x55c8c3ce6940 2023-10-05T19:40:18.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.271+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/368035751 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c8c3cecf20 con 0x55c8c3ce6940 2023-10-05T19:40:18.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.271+0000 7fa92ffff700 1 -- 172.21.15.12:0/368035751 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fa9d404d070 con 0x55c8c3ce6940 2023-10-05T19:40:18.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.271+0000 7fa92ffff700 1 --2- 172.21.15.12:0/368035751 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fa934030c00 0x7fa9340330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.271+0000 7fa92ffff700 1 -- 172.21.15.12:0/368035751 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7fa9d4059ab0 con 0x55c8c3ce6940 2023-10-05T19:40:18.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.271+0000 7fa9ed206700 1 --2- 172.21.15.12:0/368035751 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fa934030c00 0x7fa9340330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.271+0000 7fa9eeed0e40 5 note: GC not initialized 2023-10-05T19:40:18.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.271+0000 7fa9ed206700 1 --2- 172.21.15.12:0/368035751 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fa934030c00 0x7fa9340330b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x55c8c3cd85b0 tx=0x7fa9e400aea0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.275+0000 7fa92d7fa700 20 reqs_thread_entry: start 2023-10-05T19:40:18.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.275+0000 7fa9eeed0e40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:18.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.275+0000 7fa926ffd700 20 reqs_thread_entry: start 2023-10-05T19:40:18.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.275+0000 7fa9eeed0e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:18.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.275+0000 7fa9257fa700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:48 2023 2023-10-05T19:40:18.284 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:18.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.275+0000 7fa924ff9700 20 reqs_thread_entry: start 2023-10-05T19:40:18.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.275+0000 7fa9eeed0e40 20 get_system_obj_state: rctx=0x7ffe07e848a8 obj=default.rgw.meta:users.uid:foo state=0x55c8c3b6c0b0 s->prefetch_data=0 2023-10-05T19:40:18.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.275+0000 7fa9eeed0e40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2023-10-05T19:40:18.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.275+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3d54d50 con 0x55c8c3c958c0 2023-10-05T19:40:18.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.275+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(102 foo [call,getxattrs,stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x7fa9e400a5a0 con 0x55c8c3c958c0 2023-10-05T19:40:18.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.275+0000 7fa9eeed0e40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x0 2023-10-05T19:40:18.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.275+0000 7fa9eeed0e40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:18.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.275+0000 7fa9eeed0e40 20 get_system_obj_state: rctx=0x7ffe07e84598 obj=default.rgw.meta:users.email:foo@foo.com state=0x55c8c3b6c0b0 s->prefetch_data=0 2023-10-05T19:40:18.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.275+0000 7fa9eeed0e40 10 cache get: name=default.rgw.meta+users.email+foo@foo.com : miss 2023-10-05T19:40:18.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.275+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 8.3 8:c162d5d1:users.email::foo@foo.com:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3d54d50 con 0x55c8c3c958c0 2023-10-05T19:40:18.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.275+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(103 foo@foo.com [getxattrs,stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 197+0+0 (crc 0 0 0) 0x7fa9e400a5a0 con 0x55c8c3c958c0 2023-10-05T19:40:18.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.275+0000 7fa9eeed0e40 10 cache put: name=default.rgw.meta+users.email+foo@foo.com info.flags=0x0 2023-10-05T19:40:18.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.275+0000 7fa9eeed0e40 10 adding default.rgw.meta+users.email+foo@foo.com to cache LRU end 2023-10-05T19:40:18.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.275+0000 7fa9eeed0e40 20 get_system_obj_state: rctx=0x7ffe07e84598 obj=default.rgw.meta:users.keys:9te6NH5mcdcq0Tc5i8i1 state=0x55c8c3b6c0b0 s->prefetch_data=0 2023-10-05T19:40:18.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.275+0000 7fa9eeed0e40 10 cache get: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 : miss 2023-10-05T19:40:18.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.275+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:104 8.7 8:e4e0e270:users.keys::9te6NH5mcdcq0Tc5i8i1:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3d54d50 con 0x7fa9dc007030 2023-10-05T19:40:18.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.279+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(104 9te6NH5mcdcq0Tc5i8i1 [getxattrs,stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 206+0+0 (crc 0 0 0) 0x7fa9e400a5a0 con 0x7fa9dc007030 2023-10-05T19:40:18.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.279+0000 7fa9eeed0e40 10 cache put: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 info.flags=0x0 2023-10-05T19:40:18.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.279+0000 7fa9eeed0e40 10 adding default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 to cache LRU end 2023-10-05T19:40:18.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.279+0000 7fa9eeed0e40 20 get_system_obj_state: rctx=0x7ffe07e83e38 obj=default.rgw.meta:users.keys:9te6NH5mcdcq0Tc5i8i1 state=0x55c8c3b6c0b0 s->prefetch_data=0 2023-10-05T19:40:18.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.279+0000 7fa9eeed0e40 10 cache get: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 : hit (negative entry) 2023-10-05T19:40:18.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.279+0000 7fa9eeed0e40 20 get_system_obj_state: rctx=0x7ffe07e844d8 obj=default.rgw.meta:users.keys:9te6NH5mcdcq0Tc5i8i1 state=0x55c8c3b6c0b0 s->prefetch_data=0 2023-10-05T19:40:18.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.279+0000 7fa9eeed0e40 10 cache get: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 : hit (negative entry) 2023-10-05T19:40:18.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.279+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:105 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.set in=58b,writefull 0~357 in=357b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3d577e0 con 0x55c8c3c958c0 2023-10-05T19:40:18.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.283+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(105 foo [delete,create,call,writefull 0~357] v28'1 uv1 ondisk = 0) v8 ==== 273+0+0 (crc 0 0 0) 0x7fa9e400a5a0 con 0x55c8c3c958c0 2023-10-05T19:40:18.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.283+0000 7fa9eeed0e40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2023-10-05T19:40:18.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.283+0000 7fa9eeed0e40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:18.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.283+0000 7fa9eeed0e40 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2023-10-05T19:40:18.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.283+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:106 7.d 7:b4812045:::notify.4:head [notify cookie 94320767362080 in=545b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3d58180 con 0x55c8c3c958c0 2023-10-05T19:40:18.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.283+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== watch-notify(notify (1) cookie 94320766835744 notify 120259084288 ret 0) v3 ==== 575+0+0 (crc 0 0 0) 0x7fa9e402d040 con 0x55c8c3c958c0 2023-10-05T19:40:18.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.283+0000 7fa9eaa01700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084288 cookie 94320766835744 notifier 4273 bl.length()=533 2023-10-05T19:40:18.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.283+0000 7fa9eaa01700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2023-10-05T19:40:18.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.283+0000 7fa9eaa01700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:18.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.283+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(106 notify.4 [notify cookie 94320767362080 out=8b] v0'0 uv24 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fa9e400a5a0 con 0x55c8c3c958c0 2023-10-05T19:40:18.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.283+0000 7fa9eaa01700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:107 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fa9d0003d90 con 0x55c8c3c958c0 2023-10-05T19:40:18.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.283+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== osd_op_reply(107 notify.4 [notify-ack] v0'0 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa9e400a5a0 con 0x55c8c3c958c0 2023-10-05T19:40:18.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.283+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.1 v2:172.21.15.12:6808/29933 39 ==== watch-notify(notify_complete (2) cookie 94320767362080 notify 120259084288 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7fa9e4037030 con 0x55c8c3c958c0 2023-10-05T19:40:18.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.283+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:108 8.3 8:c162d5d1:users.email::foo@foo.com:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3d58040 con 0x55c8c3c958c0 2023-10-05T19:40:18.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.287+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.1 v2:172.21.15.12:6808/29933 40 ==== osd_op_reply(108 foo@foo.com [delete,create,writefull 0~7] v28'1 uv1 ondisk = 0) v8 ==== 239+0+0 (crc 0 0 0) 0x7fa9e400a5a0 con 0x55c8c3c958c0 2023-10-05T19:40:18.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.287+0000 7fa9eeed0e40 10 cache put: name=default.rgw.meta+users.email+foo@foo.com info.flags=0x7 2023-10-05T19:40:18.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.287+0000 7fa9eeed0e40 10 moving default.rgw.meta+users.email+foo@foo.com to cache LRU end 2023-10-05T19:40:18.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.287+0000 7fa9eeed0e40 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.email:foo@foo.com, ofs0, ns] 2023-10-05T19:40:18.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.287+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:109 7.e 7:7759931f:::notify.3:head [notify cookie 94320767362080 in=181b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3d58120 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.287+0000 7fa9eca05700 1 -- 172.21.15.12:0/3501119661 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== watch-notify(notify (1) cookie 94320766837296 notify 120259084288 ret 0) v3 ==== 211+0+0 (crc 0 0 0) 0x55c8c3cbb590 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.287+0000 7fa9ea200700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084288 cookie 94320766837296 notifier 4273 bl.length()=169 2023-10-05T19:40:18.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.287+0000 7fa9ea200700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+foo@foo.com info.flags=0x7 2023-10-05T19:40:18.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.287+0000 7fa9ea200700 10 rgw watcher librados: moving default.rgw.meta+users.email+foo@foo.com to cache LRU end 2023-10-05T19:40:18.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.287+0000 7fa9eca05700 1 -- 172.21.15.12:0/3501119661 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(109 notify.3 [notify cookie 94320767362080 out=8b] v0'0 uv24 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fa9dc041040 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.287+0000 7fa9ea200700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:110 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fa93c003450 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.287+0000 7fa9eca05700 1 -- 172.21.15.12:0/3501119661 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(110 notify.3 [notify-ack] v0'0 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa9dc041040 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.287+0000 7fa9eca05700 1 -- 172.21.15.12:0/3501119661 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== watch-notify(notify_complete (2) cookie 94320767362080 notify 120259084288 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55c8c3cbb150 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.287+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:111 8.7 8:e4e0e270:users.keys::9te6NH5mcdcq0Tc5i8i1:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3d538a0 con 0x7fa9dc007030 2023-10-05T19:40:18.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.291+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(111 9te6NH5mcdcq0Tc5i8i1 [delete,create,writefull 0~7] v28'1 uv1 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x7fa9e400a5a0 con 0x7fa9dc007030 2023-10-05T19:40:18.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.291+0000 7fa9eeed0e40 10 cache put: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 info.flags=0x7 2023-10-05T19:40:18.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.291+0000 7fa9eeed0e40 10 moving default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 to cache LRU end 2023-10-05T19:40:18.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.291+0000 7fa9eeed0e40 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.keys:9te6NH5mcdcq0Tc5i8i1, ofs0, ns] 2023-10-05T19:40:18.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.291+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:112 7.e 7:7759931f:::notify.3:head [notify cookie 94320767362080 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3d538a0 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.291+0000 7fa9eca05700 1 -- 172.21.15.12:0/3501119661 <== osd.2 v2:172.21.15.12:6800/29934 23 ==== watch-notify(notify (1) cookie 94320766837296 notify 120259084289 ret 0) v3 ==== 219+0+0 (crc 0 0 0) 0x55c8c3cbb150 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.291+0000 7fa9eaa01700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084289 cookie 94320766837296 notifier 4273 bl.length()=177 2023-10-05T19:40:18.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.291+0000 7fa9eaa01700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 info.flags=0x7 2023-10-05T19:40:18.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.291+0000 7fa9eaa01700 10 rgw watcher librados: moving default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 to cache LRU end 2023-10-05T19:40:18.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.291+0000 7fa9eaa01700 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:113 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fa9d0003d90 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.291+0000 7fa9eca05700 1 -- 172.21.15.12:0/3501119661 <== osd.2 v2:172.21.15.12:6800/29934 24 ==== osd_op_reply(112 notify.3 [notify cookie 94320767362080 out=8b] v0'0 uv24 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fa9dc041040 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.291+0000 7fa9eca05700 1 -- 172.21.15.12:0/3501119661 <== osd.2 v2:172.21.15.12:6800/29934 25 ==== osd_op_reply(113 notify.3 [notify-ack] v0'0 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa9dc041040 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.291+0000 7fa9eca05700 1 -- 172.21.15.12:0/3501119661 <== osd.2 v2:172.21.15.12:6800/29934 26 ==== watch-notify(notify_complete (2) cookie 94320767362080 notify 120259084289 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55c8c3c96a70 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.302 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:40:18.302 INFO:teuthology.orchestra.run.smithi012.stdout: "user_id": "foo", 2023-10-05T19:40:18.302 INFO:teuthology.orchestra.run.smithi012.stdout: "display_name": "Foo", 2023-10-05T19:40:18.302 INFO:teuthology.orchestra.run.smithi012.stdout: "email": "foo@foo.com", 2023-10-05T19:40:18.303 INFO:teuthology.orchestra.run.smithi012.stdout: "suspended": 0, 2023-10-05T19:40:18.303 INFO:teuthology.orchestra.run.smithi012.stdout: "max_buckets": 4, 2023-10-05T19:40:18.303 INFO:teuthology.orchestra.run.smithi012.stdout: "subusers": [], 2023-10-05T19:40:18.303 INFO:teuthology.orchestra.run.smithi012.stdout: "keys": [ 2023-10-05T19:40:18.303 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:18.303 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:40:18.303 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2023-10-05T19:40:18.303 INFO:teuthology.orchestra.run.smithi012.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2023-10-05T19:40:18.303 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:18.303 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:18.303 INFO:teuthology.orchestra.run.smithi012.stdout: "swift_keys": [], 2023-10-05T19:40:18.303 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": [], 2023-10-05T19:40:18.304 INFO:teuthology.orchestra.run.smithi012.stdout: "op_mask": "read, write, delete", 2023-10-05T19:40:18.304 INFO:teuthology.orchestra.run.smithi012.stdout: "default_placement": "", 2023-10-05T19:40:18.304 INFO:teuthology.orchestra.run.smithi012.stdout: "default_storage_class": "", 2023-10-05T19:40:18.304 INFO:teuthology.orchestra.run.smithi012.stdout: "placement_tags": [], 2023-10-05T19:40:18.304 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket_quota": { 2023-10-05T19:40:18.304 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:18.304 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:18.304 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:18.304 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:18.304 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:18.304 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:18.304 INFO:teuthology.orchestra.run.smithi012.stdout: "user_quota": { 2023-10-05T19:40:18.305 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:18.305 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:18.305 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:18.305 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:18.305 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:18.305 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:18.305 INFO:teuthology.orchestra.run.smithi012.stdout: "temp_url_keys": [], 2023-10-05T19:40:18.305 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "rgw", 2023-10-05T19:40:18.305 INFO:teuthology.orchestra.run.smithi012.stdout: "mfa_ids": [] 2023-10-05T19:40:18.305 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:40:18.305 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:40:18.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.295+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:114 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94320766826672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3caa830 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.295+0000 7fa9eca05700 1 -- 172.21.15.12:0/3501119661 <== osd.2 v2:172.21.15.12:6800/29934 27 ==== osd_op_reply(114 notify.0 [watch unwatch cookie 94320766826672] v28'26 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa9dc041040 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.295+0000 7fa9eeed0e40 20 remove_watcher() i=0 2023-10-05T19:40:18.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.295+0000 7fa9eeed0e40 2 removed watcher, disabling cache 2023-10-05T19:40:18.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.295+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:115 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94320766829248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd38b0 con 0x55c8c3c958c0 2023-10-05T19:40:18.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.299+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.1 v2:172.21.15.12:6808/29933 41 ==== osd_op_reply(115 notify.1 [watch unwatch cookie 94320766829248] v28'26 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa9e400a5a0 con 0x55c8c3c958c0 2023-10-05T19:40:18.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.299+0000 7fa9eeed0e40 20 remove_watcher() i=1 2023-10-05T19:40:18.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.299+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:116 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94320766832512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd38b0 con 0x55c8c3ccd360 2023-10-05T19:40:18.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.303+0000 7fa9ec204700 1 -- 172.21.15.12:0/3501119661 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(116 notify.2 [watch unwatch cookie 94320766832512] v28'26 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa9d4006070 con 0x55c8c3ccd360 2023-10-05T19:40:18.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.303+0000 7fa9eeed0e40 20 remove_watcher() i=2 2023-10-05T19:40:18.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.303+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:117 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94320766837296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd4f80 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.303+0000 7fa9eca05700 1 -- 172.21.15.12:0/3501119661 <== osd.2 v2:172.21.15.12:6800/29934 28 ==== osd_op_reply(117 notify.3 [watch unwatch cookie 94320766837296] v28'26 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa9dc041040 con 0x55c8c3cc8bc0 2023-10-05T19:40:18.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.303+0000 7fa9eeed0e40 20 remove_watcher() i=3 2023-10-05T19:40:18.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.303+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:118 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94320766835744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd38b0 con 0x55c8c3c958c0 2023-10-05T19:40:18.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.307+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.1 v2:172.21.15.12:6808/29933 42 ==== osd_op_reply(118 notify.4 [watch unwatch cookie 94320766835744] v28'26 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa9e400a5a0 con 0x55c8c3c958c0 2023-10-05T19:40:18.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.307+0000 7fa9eeed0e40 20 remove_watcher() i=4 2023-10-05T19:40:18.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.307+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:119 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94320766838400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd6230 con 0x7fa9dc007030 2023-10-05T19:40:18.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.311+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(119 notify.5 [watch unwatch cookie 94320766838400] v28'26 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa9e400a5a0 con 0x7fa9dc007030 2023-10-05T19:40:18.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.311+0000 7fa9eeed0e40 20 remove_watcher() i=5 2023-10-05T19:40:18.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.311+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:120 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94320766843472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd4f80 con 0x7fa9dc007030 2023-10-05T19:40:18.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.311+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.0 v2:172.21.15.8:6800/27563 30 ==== osd_op_reply(120 notify.6 [watch unwatch cookie 94320766843472] v28'26 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa9e400a5a0 con 0x7fa9dc007030 2023-10-05T19:40:18.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.311+0000 7fa9eeed0e40 20 remove_watcher() i=6 2023-10-05T19:40:18.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.311+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:121 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94320766847520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd5c20 con 0x7fa9dc007030 2023-10-05T19:40:18.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.315+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.0 v2:172.21.15.8:6800/27563 31 ==== osd_op_reply(121 notify.7 [watch unwatch cookie 94320766847520] v28'26 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa9e400a5a0 con 0x7fa9dc007030 2023-10-05T19:40:18.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.315+0000 7fa9eeed0e40 20 remove_watcher() i=7 2023-10-05T19:40:18.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.319+0000 7fa9be7e4700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:18.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.319+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:122 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94320766845040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c8c3cd38b0 con 0x55c8c3c958c0 2023-10-05T19:40:18.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.323+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 <== osd.1 v2:172.21.15.12:6808/29933 43 ==== osd_op_reply(122 data_loggenerations_metadata [watch unwatch cookie 94320766845040] v28'51 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fa9e400a5a0 con 0x55c8c3c958c0 2023-10-05T19:40:18.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.323+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fa9dc007030 msgr2=0x7fa9dc046050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.323+0000 7fa9eeed0e40 1 --2- 172.21.15.12:0/3501119661 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fa9dc007030 0x7fa9dc046050 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.323+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55c8c3c958c0 msgr2=0x55c8c3cc66d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.323+0000 7fa9eeed0e40 1 --2- 172.21.15.12:0/3501119661 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55c8c3c958c0 0x55c8c3cc66d0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.323+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55c8c3cc8bc0 msgr2=0x55c8c3ccafa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.323+0000 7fa9eeed0e40 1 --2- 172.21.15.12:0/3501119661 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55c8c3cc8bc0 0x55c8c3ccafa0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.323+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55c8c3ccd360 msgr2=0x55c8c3ccf770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.323+0000 7fa9eeed0e40 1 --2- 172.21.15.12:0/3501119661 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55c8c3ccd360 0x55c8c3ccf770 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.323+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fa9cc030ca0 msgr2=0x7fa9cc033150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.323+0000 7fa9eeed0e40 1 --2- 172.21.15.12:0/3501119661 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fa9cc030ca0 0x7fa9cc033150 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4004170 tx=0x7fa9d4001400 comp rx=0 tx=0).stop 2023-10-05T19:40:18.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.323+0000 7fa9ed206700 1 -- 172.21.15.12:0/3501119661 reap_dead start 2023-10-05T19:40:18.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.323+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c8c3c94fb0 msgr2=0x55c8c3cbac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.323+0000 7fa9eeed0e40 1 --2- 172.21.15.12:0/3501119661 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c8c3c94fb0 0x55c8c3cbac10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc01f040 tx=0x7fa9dc01c000 comp rx=0 tx=0).stop 2023-10-05T19:40:18.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.323+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 shutdown_connections 2023-10-05T19:40:18.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.323+0000 7fa9eeed0e40 1 --2- 172.21.15.12:0/3501119661 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c8c3c94fb0 0x55c8c3cbac10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.323+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 >> 172.21.15.12:0/3501119661 conn(0x55c8c3c08800 msgr2=0x55c8c3cae590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:18.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.323+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 shutdown_connections 2023-10-05T19:40:18.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.323+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/3501119661 wait complete. 2023-10-05T19:40:18.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.323+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/368035751 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fa934030c00 msgr2=0x7fa9340330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.323+0000 7fa9eeed0e40 1 --2- 172.21.15.12:0/368035751 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fa934030c00 0x7fa9340330b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x55c8c3cd85b0 tx=0x7fa9e400aea0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.323+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/368035751 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c8c3ce6940 msgr2=0x55c8c3cec4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.323+0000 7fa9eeed0e40 1 --2- 172.21.15.12:0/368035751 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c8c3ce6940 0x55c8c3cec4e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa9d403e040 tx=0x7fa9d4040000 comp rx=0 tx=0).stop 2023-10-05T19:40:18.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.323+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/368035751 shutdown_connections 2023-10-05T19:40:18.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.323+0000 7fa9eeed0e40 1 --2- 172.21.15.12:0/368035751 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fa934030c00 0x7fa9340330b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.323+0000 7fa9eeed0e40 1 --2- 172.21.15.12:0/368035751 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c8c3ce6940 0x55c8c3cec4e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.323+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/368035751 >> 172.21.15.12:0/368035751 conn(0x55c8c3ce1d00 msgr2=0x55c8c3cdfda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:18.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.323+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/368035751 shutdown_connections 2023-10-05T19:40:18.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.323+0000 7fa9eeed0e40 1 -- 172.21.15.12:0/368035751 wait complete. 2023-10-05T19:40:18.337 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:40:18.338 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--email', 'foo@foo.com'] 2023-10-05T19:40:18.338 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--email', 'foo@foo.com'] 2023-10-05T19:40:18.338 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid fud --display-name Fud --email foo@foo.com 2023-10-05T19:40:18.382 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:18.383 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:18.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.375+0000 7fd4061bae40 1 Processor -- start 2023-10-05T19:40:18.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.375+0000 7fd4061bae40 1 -- start start 2023-10-05T19:40:18.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.375+0000 7fd4061bae40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x5636d976edd0 0x5636d976f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.375+0000 7fd4061bae40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5636d96bbbd0 con 0x5636d976f6e0 2023-10-05T19:40:18.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.375+0000 7fd4061bae40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x5636d9678ee0 con 0x5636d976edd0 2023-10-05T19:40:18.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.375+0000 7fd4034ee700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x5636d976f6e0 0x5636d9770180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:47220/0 (socket says 172.21.15.12:47220) 2023-10-05T19:40:18.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.375+0000 7fd4034ee700 1 -- 172.21.15.12:0/202507630 learned_addr learned my addr 172.21.15.12:0/202507630 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:18.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.375+0000 7fd403cef700 1 --2- 172.21.15.12:0/202507630 >> v2:172.21.15.8:3300/0 conn(0x5636d976edd0 0x5636d976f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.375+0000 7fd402ced700 1 -- 172.21.15.12:0/202507630 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 242739055 0 0) 0x5636d96bbbd0 con 0x5636d976f6e0 2023-10-05T19:40:18.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.375+0000 7fd402ced700 1 -- 172.21.15.12:0/202507630 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3f0001e10 con 0x5636d976f6e0 2023-10-05T19:40:18.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.375+0000 7fd403cef700 1 -- 172.21.15.12:0/202507630 >> v1:172.21.15.8:6789/0 conn(0x5636d976f6e0 legacy=0x5636d9770180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.375+0000 7fd403cef700 1 -- 172.21.15.12:0/202507630 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636d9770890 con 0x5636d976edd0 2023-10-05T19:40:18.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.375+0000 7fd403cef700 1 --2- 172.21.15.12:0/202507630 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636d976edd0 0x5636d976f1a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc00a5d0 tx=0x7fd3fc005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91daacbf8f9e4f02 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.379+0000 7fd402ced700 1 -- 172.21.15.12:0/202507630 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd3fc007610 con 0x5636d976edd0 2023-10-05T19:40:18.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.379+0000 7fd402ced700 1 -- 172.21.15.12:0/202507630 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3fc007770 con 0x5636d976edd0 2023-10-05T19:40:18.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.379+0000 7fd402ced700 1 -- 172.21.15.12:0/202507630 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd3fc007a40 con 0x5636d976edd0 2023-10-05T19:40:18.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.379+0000 7fd4061bae40 1 -- 172.21.15.12:0/202507630 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636d976edd0 msgr2=0x5636d976f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.379+0000 7fd4061bae40 1 --2- 172.21.15.12:0/202507630 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636d976edd0 0x5636d976f1a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc00a5d0 tx=0x7fd3fc005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:18.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.379+0000 7fd4061bae40 1 -- 172.21.15.12:0/202507630 shutdown_connections 2023-10-05T19:40:18.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.379+0000 7fd4061bae40 1 --2- 172.21.15.12:0/202507630 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636d976edd0 0x5636d976f1a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.379+0000 7fd4061bae40 1 -- 172.21.15.12:0/202507630 >> 172.21.15.12:0/202507630 conn(0x5636d976a7f0 msgr2=0x5636d976cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:18.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.379+0000 7fd4061bae40 1 -- 172.21.15.12:0/202507630 shutdown_connections 2023-10-05T19:40:18.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.379+0000 7fd4061bae40 1 -- 172.21.15.12:0/202507630 wait complete. 2023-10-05T19:40:18.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.379+0000 7fd4061bae40 1 Processor -- start 2023-10-05T19:40:18.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.379+0000 7fd4061bae40 1 -- start start 2023-10-05T19:40:18.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.379+0000 7fd4061bae40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636d976edd0 0x5636d978dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.379+0000 7fd4061bae40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636d963ce30 con 0x5636d976edd0 2023-10-05T19:40:18.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.379+0000 7fd403cef700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636d976edd0 0x5636d978dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.379+0000 7fd403cef700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636d976edd0 0x5636d978dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36672/0 (socket says 172.21.15.12:36672) 2023-10-05T19:40:18.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.379+0000 7fd403cef700 1 -- 172.21.15.12:0/1077804724 learned_addr learned my addr 172.21.15.12:0/1077804724 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:18.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.379+0000 7fd403cef700 1 -- 172.21.15.12:0/1077804724 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3fc005cf0 con 0x5636d976edd0 2023-10-05T19:40:18.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.379+0000 7fd403cef700 1 --2- 172.21.15.12:0/1077804724 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636d976edd0 0x5636d978dd00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc000c00 tx=0x7fd3fc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a76abac0028592b server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.383+0000 7fd401ceb700 1 -- 172.21.15.12:0/1077804724 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd3fc0078e0 con 0x5636d976edd0 2023-10-05T19:40:18.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.383+0000 7fd401ceb700 1 -- 172.21.15.12:0/1077804724 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3fc007a40 con 0x5636d976edd0 2023-10-05T19:40:18.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.383+0000 7fd401ceb700 1 -- 172.21.15.12:0/1077804724 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd3fc007bd0 con 0x5636d976edd0 2023-10-05T19:40:18.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.383+0000 7fd4061bae40 1 -- 172.21.15.12:0/1077804724 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636d976edd0 msgr2=0x5636d978dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.383+0000 7fd4061bae40 1 --2- 172.21.15.12:0/1077804724 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636d976edd0 0x5636d978dd00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc000c00 tx=0x7fd3fc014000 comp rx=0 tx=0).stop 2023-10-05T19:40:18.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.383+0000 7fd4061bae40 1 -- 172.21.15.12:0/1077804724 shutdown_connections 2023-10-05T19:40:18.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.383+0000 7fd4061bae40 1 --2- 172.21.15.12:0/1077804724 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636d976edd0 0x5636d978dd00 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.383+0000 7fd4061bae40 1 -- 172.21.15.12:0/1077804724 >> 172.21.15.12:0/1077804724 conn(0x5636d976a7f0 msgr2=0x5636d9789840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:18.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.383+0000 7fd4061bae40 1 -- 172.21.15.12:0/1077804724 shutdown_connections 2023-10-05T19:40:18.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.383+0000 7fd4061bae40 1 -- 172.21.15.12:0/1077804724 wait complete. 2023-10-05T19:40:18.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.383+0000 7fd4061bae40 1 Processor -- start 2023-10-05T19:40:18.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.383+0000 7fd4061bae40 1 -- start start 2023-10-05T19:40:18.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.383+0000 7fd4061bae40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636d976edd0 0x5636d9794a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.383+0000 7fd4061bae40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636d960fed0 con 0x5636d976edd0 2023-10-05T19:40:18.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.383+0000 7fd403cef700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636d976edd0 0x5636d9794a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.383+0000 7fd403cef700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636d976edd0 0x5636d9794a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36682/0 (socket says 172.21.15.12:36682) 2023-10-05T19:40:18.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.383+0000 7fd403cef700 1 -- 172.21.15.12:0/3118385934 learned_addr learned my addr 172.21.15.12:0/3118385934 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:18.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.383+0000 7fd403cef700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636d9770890 con 0x5636d976edd0 2023-10-05T19:40:18.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.383+0000 7fd403cef700 1 --2- 172.21.15.12:0/3118385934 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636d976edd0 0x5636d9794a00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc01f040 tx=0x7fd3fc01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.383+0000 7fd3fbfff700 1 -- 172.21.15.12:0/3118385934 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd3fc007bb0 con 0x5636d976edd0 2023-10-05T19:40:18.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.383+0000 7fd3fbfff700 1 -- 172.21.15.12:0/3118385934 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3fc029070 con 0x5636d976edd0 2023-10-05T19:40:18.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.383+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5636d9794f40 con 0x5636d976edd0 2023-10-05T19:40:18.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.383+0000 7fd3fbfff700 1 -- 172.21.15.12:0/3118385934 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd3fc012610 con 0x5636d976edd0 2023-10-05T19:40:18.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.383+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5636d9795380 con 0x5636d976edd0 2023-10-05T19:40:18.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.387+0000 7fd4061bae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.387+0000 7fd3fbfff700 1 -- 172.21.15.12:0/3118385934 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fd3fc012840 con 0x5636d976edd0 2023-10-05T19:40:18.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.387+0000 7fd3fbfff700 1 --2- 172.21.15.12:0/3118385934 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd3e4030c50 0x7fd3e4033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.387+0000 7fd3fbfff700 1 -- 172.21.15.12:0/3118385934 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7fd3fc02b070 con 0x5636d976edd0 2023-10-05T19:40:18.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.387+0000 7fd4034ee700 1 --2- 172.21.15.12:0/3118385934 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd3e4030c50 0x7fd3e4033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.387+0000 7fd4061bae40 1 --2- 172.21.15.12:0/3118385934 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5636d976f6e0 0x5636d97a04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.387+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97a0b70 con 0x5636d976f6e0 2023-10-05T19:40:18.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.387+0000 7fd4044f0700 1 --2- 172.21.15.12:0/3118385934 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5636d976f6e0 0x5636d97a04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.387+0000 7fd4034ee700 1 --2- 172.21.15.12:0/3118385934 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd3e4030c50 0x7fd3e4033100 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec0044f0 tx=0x7fd3ec006690 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.387+0000 7fd4044f0700 1 --2- 172.21.15.12:0/3118385934 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5636d976f6e0 0x5636d97a04c0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.391+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd3f400a500 con 0x5636d976f6e0 2023-10-05T19:40:18.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.391+0000 7fd4061bae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:18.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.391+0000 7fd4061bae40 20 realm 2023-10-05T19:40:18.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.391+0000 7fd4061bae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.391+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d979db20 con 0x5636d976f6e0 2023-10-05T19:40:18.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.391+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd3f400a500 con 0x5636d976f6e0 2023-10-05T19:40:18.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.391+0000 7fd4061bae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:18.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.391+0000 7fd4061bae40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:18.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.391+0000 7fd4061bae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.391+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97a1650 con 0x5636d976f6e0 2023-10-05T19:40:18.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.391+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd3f400a500 con 0x5636d976f6e0 2023-10-05T19:40:18.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.391+0000 7fd4061bae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:18.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.391+0000 7fd4061bae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.391+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97a1260 con 0x5636d976f6e0 2023-10-05T19:40:18.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.391+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fd3f400a500 con 0x5636d976f6e0 2023-10-05T19:40:18.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.391+0000 7fd4061bae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:18.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.391+0000 7fd4061bae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.391+0000 7fd4061bae40 1 --2- 172.21.15.12:0/3118385934 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5636d97a29f0 0x5636d97a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.391+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97a5310 con 0x5636d97a29f0 2023-10-05T19:40:18.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.391+0000 7fd403cef700 1 --2- 172.21.15.12:0/3118385934 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5636d97a29f0 0x5636d97a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.391+0000 7fd403cef700 1 --2- 172.21.15.12:0/3118385934 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5636d97a29f0 0x5636d97a4dd0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.391+0000 7fd403cef700 1 -- 172.21.15.12:0/3118385934 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fd3fc01cc20 con 0x5636d97a29f0 2023-10-05T19:40:18.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.391+0000 7fd4061bae40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:18.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.391+0000 7fd4061bae40 20 searching for the correct realm 2023-10-05T19:40:18.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.391+0000 7fd4061bae40 1 --2- 172.21.15.12:0/3118385934 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5636d97a7190 0x5636d97a95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.391+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97a9c50 con 0x5636d97a7190 2023-10-05T19:40:18.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.395+0000 7fd4034ee700 1 --2- 172.21.15.12:0/3118385934 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5636d97a7190 0x5636d97a95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.395+0000 7fd4034ee700 1 --2- 172.21.15.12:0/3118385934 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5636d97a7190 0x5636d97a95a0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.395+0000 7fd4034ee700 1 -- 172.21.15.12:0/3118385934 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fd3ec004520 con 0x5636d97a7190 2023-10-05T19:40:18.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.395+0000 7fd4034ee700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97a9c50 con 0x5636d97a29f0 2023-10-05T19:40:18.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.395+0000 7fd403cef700 1 -- 172.21.15.12:0/3118385934 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3fc01cc20 con 0x5636d97a29f0 2023-10-05T19:40:18.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.395+0000 7fd403cef700 1 --2- 172.21.15.12:0/3118385934 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fd3fc007030 0x7fd3fc0113b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.395+0000 7fd403cef700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97a9c50 con 0x7fd3fc007030 2023-10-05T19:40:18.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.399+0000 7fd4044f0700 1 --2- 172.21.15.12:0/3118385934 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fd3fc007030 0x7fd3fc0113b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.399+0000 7fd4044f0700 1 --2- 172.21.15.12:0/3118385934 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fd3fc007030 0x7fd3fc0113b0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.399+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3f4014040 con 0x7fd3fc007030 2023-10-05T19:40:18.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.399+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97a9c50 con 0x5636d976f6e0 2023-10-05T19:40:18.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.399+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3f4014040 con 0x5636d976f6e0 2023-10-05T19:40:18.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.399+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97a9c50 con 0x5636d976f6e0 2023-10-05T19:40:18.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.399+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3f4014040 con 0x5636d976f6e0 2023-10-05T19:40:18.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.399+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97a9c50 con 0x7fd3fc007030 2023-10-05T19:40:18.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.403+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3f4014040 con 0x7fd3fc007030 2023-10-05T19:40:18.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.403+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97a9c50 con 0x7fd3fc007030 2023-10-05T19:40:18.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.403+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3f4014040 con 0x7fd3fc007030 2023-10-05T19:40:18.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.403+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97a9c50 con 0x5636d97a29f0 2023-10-05T19:40:18.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.403+0000 7fd403cef700 1 -- 172.21.15.12:0/3118385934 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3fc01cc20 con 0x5636d97a29f0 2023-10-05T19:40:18.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.403+0000 7fd403cef700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97a9c50 con 0x5636d97a7190 2023-10-05T19:40:18.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.403+0000 7fd4034ee700 1 -- 172.21.15.12:0/3118385934 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3ec004520 con 0x5636d97a7190 2023-10-05T19:40:18.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.403+0000 7fd4034ee700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97a9c50 con 0x5636d976f6e0 2023-10-05T19:40:18.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.403+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3f4014040 con 0x5636d976f6e0 2023-10-05T19:40:18.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.403+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97a9c50 con 0x7fd3fc007030 2023-10-05T19:40:18.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.407+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3f4014040 con 0x7fd3fc007030 2023-10-05T19:40:18.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.407+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97a9c50 con 0x5636d976f6e0 2023-10-05T19:40:18.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.407+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3f4014040 con 0x5636d976f6e0 2023-10-05T19:40:18.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.407+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97a9c50 con 0x5636d97a29f0 2023-10-05T19:40:18.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.407+0000 7fd403cef700 1 -- 172.21.15.12:0/3118385934 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3fc01cc20 con 0x5636d97a29f0 2023-10-05T19:40:18.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.407+0000 7fd403cef700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97a9c50 con 0x5636d97a7190 2023-10-05T19:40:18.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.407+0000 7fd4034ee700 1 -- 172.21.15.12:0/3118385934 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3ec004520 con 0x5636d97a7190 2023-10-05T19:40:18.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.407+0000 7fd4034ee700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97a9c50 con 0x5636d97a7190 2023-10-05T19:40:18.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.407+0000 7fd4034ee700 1 -- 172.21.15.12:0/3118385934 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3ec004520 con 0x5636d97a7190 2023-10-05T19:40:18.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.407+0000 7fd4034ee700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97a9c50 con 0x7fd3fc007030 2023-10-05T19:40:18.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.411+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3f4014040 con 0x7fd3fc007030 2023-10-05T19:40:18.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.411+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97a9c50 con 0x5636d97a7190 2023-10-05T19:40:18.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.411+0000 7fd4034ee700 1 -- 172.21.15.12:0/3118385934 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3ec004520 con 0x5636d97a7190 2023-10-05T19:40:18.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.411+0000 7fd4034ee700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97a9c50 con 0x5636d97a7190 2023-10-05T19:40:18.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.411+0000 7fd4034ee700 1 -- 172.21.15.12:0/3118385934 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3ec004520 con 0x5636d97a7190 2023-10-05T19:40:18.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.411+0000 7fd4034ee700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97a9c50 con 0x5636d976f6e0 2023-10-05T19:40:18.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.411+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3f4014040 con 0x5636d976f6e0 2023-10-05T19:40:18.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.411+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97a9c50 con 0x5636d976f6e0 2023-10-05T19:40:18.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.411+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3f4014040 con 0x5636d976f6e0 2023-10-05T19:40:18.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.411+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97a9c50 con 0x7fd3fc007030 2023-10-05T19:40:18.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.415+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3f4014040 con 0x7fd3fc007030 2023-10-05T19:40:18.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.415+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97a9c50 con 0x5636d97a7190 2023-10-05T19:40:18.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.415+0000 7fd4034ee700 1 -- 172.21.15.12:0/3118385934 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3ec004520 con 0x5636d97a7190 2023-10-05T19:40:18.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.415+0000 7fd4034ee700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97a9c50 con 0x5636d97a29f0 2023-10-05T19:40:18.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.415+0000 7fd403cef700 1 -- 172.21.15.12:0/3118385934 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3fc01cc20 con 0x5636d97a29f0 2023-10-05T19:40:18.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.415+0000 7fd403cef700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97a9c50 con 0x5636d976f6e0 2023-10-05T19:40:18.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.415+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3f4014040 con 0x5636d976f6e0 2023-10-05T19:40:18.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.415+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97a9c50 con 0x7fd3fc007030 2023-10-05T19:40:18.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.415+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3f4014040 con 0x7fd3fc007030 2023-10-05T19:40:18.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.415+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97a9c50 con 0x5636d97a7190 2023-10-05T19:40:18.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.419+0000 7fd4034ee700 1 -- 172.21.15.12:0/3118385934 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3ec004520 con 0x5636d97a7190 2023-10-05T19:40:18.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.419+0000 7fd4034ee700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97a9c50 con 0x7fd3fc007030 2023-10-05T19:40:18.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.419+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3f4014040 con 0x7fd3fc007030 2023-10-05T19:40:18.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.419+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97a9c50 con 0x5636d97a29f0 2023-10-05T19:40:18.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.419+0000 7fd403cef700 1 -- 172.21.15.12:0/3118385934 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3fc01cc20 con 0x5636d97a29f0 2023-10-05T19:40:18.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.419+0000 7fd403cef700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97a9c50 con 0x5636d97a7190 2023-10-05T19:40:18.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.419+0000 7fd4034ee700 1 -- 172.21.15.12:0/3118385934 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3ec004520 con 0x5636d97a7190 2023-10-05T19:40:18.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.419+0000 7fd4034ee700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97a9c50 con 0x7fd3fc007030 2023-10-05T19:40:18.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.423+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3f4014040 con 0x7fd3fc007030 2023-10-05T19:40:18.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.423+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97a9c50 con 0x5636d976f6e0 2023-10-05T19:40:18.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.423+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fd3f4014040 con 0x5636d976f6e0 2023-10-05T19:40:18.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.423+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97a9c50 con 0x5636d97a29f0 2023-10-05T19:40:18.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.423+0000 7fd403cef700 1 -- 172.21.15.12:0/3118385934 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fd3fc01cc20 con 0x5636d97a29f0 2023-10-05T19:40:18.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.423+0000 7fd4061bae40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:18.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.423+0000 7fd4061bae40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:18.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.423+0000 7fd4061bae40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:18.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.423+0000 7fd4061bae40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:18.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.423+0000 7fd4061bae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.423+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97a5f10 con 0x5636d976f6e0 2023-10-05T19:40:18.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.423+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd3f4014040 con 0x5636d976f6e0 2023-10-05T19:40:18.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.423+0000 7fd4061bae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:18.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.423+0000 7fd4061bae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.423+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97a5b40 con 0x5636d976f6e0 2023-10-05T19:40:18.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.423+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fd3f4014040 con 0x5636d976f6e0 2023-10-05T19:40:18.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.423+0000 7fd4061bae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:18.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.423+0000 7fd4061bae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.423+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97aa0c0 con 0x5636d97a7190 2023-10-05T19:40:18.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.427+0000 7fd4034ee700 1 -- 172.21.15.12:0/3118385934 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fd3ec004520 con 0x5636d97a7190 2023-10-05T19:40:18.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.427+0000 7fd4061bae40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:18.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.427+0000 7fd4061bae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.427+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97aa470 con 0x5636d97a7190 2023-10-05T19:40:18.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.427+0000 7fd4034ee700 1 -- 172.21.15.12:0/3118385934 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7fd3ec004520 con 0x5636d97a7190 2023-10-05T19:40:18.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.427+0000 7fd4061bae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:18.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.427+0000 7fd4061bae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.427+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97aa820 con 0x5636d976f6e0 2023-10-05T19:40:18.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.427+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd3f4014040 con 0x5636d976f6e0 2023-10-05T19:40:18.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.427+0000 7fd4061bae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:18.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.427+0000 7fd4061bae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.427+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97a6430 con 0x5636d976f6e0 2023-10-05T19:40:18.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.427+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fd3f4014040 con 0x5636d976f6e0 2023-10-05T19:40:18.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.427+0000 7fd4061bae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:18.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.427+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97a6430 con 0x5636d97a7190 2023-10-05T19:40:18.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.431+0000 7fd4034ee700 1 -- 172.21.15.12:0/3118385934 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fd3ec004520 con 0x5636d97a7190 2023-10-05T19:40:18.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.431+0000 7fd4034ee700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97a6430 con 0x5636d97a29f0 2023-10-05T19:40:18.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.431+0000 7fd403cef700 1 -- 172.21.15.12:0/3118385934 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3fc01cc20 con 0x5636d97a29f0 2023-10-05T19:40:18.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.431+0000 7fd403cef700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97a6430 con 0x7fd3fc007030 2023-10-05T19:40:18.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.431+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3f4014040 con 0x7fd3fc007030 2023-10-05T19:40:18.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.431+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97aa820 con 0x5636d976f6e0 2023-10-05T19:40:18.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.435+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3f4014040 con 0x5636d976f6e0 2023-10-05T19:40:18.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.435+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97aa820 con 0x5636d976f6e0 2023-10-05T19:40:18.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.435+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3f4014040 con 0x5636d976f6e0 2023-10-05T19:40:18.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.435+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97aa820 con 0x7fd3fc007030 2023-10-05T19:40:18.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.435+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3f4014040 con 0x7fd3fc007030 2023-10-05T19:40:18.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.435+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97aa820 con 0x7fd3fc007030 2023-10-05T19:40:18.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.435+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3f4014040 con 0x7fd3fc007030 2023-10-05T19:40:18.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.435+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97aa820 con 0x5636d97a29f0 2023-10-05T19:40:18.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.439+0000 7fd403cef700 1 -- 172.21.15.12:0/3118385934 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3fc01cc20 con 0x5636d97a29f0 2023-10-05T19:40:18.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.439+0000 7fd403cef700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97aa820 con 0x5636d97a7190 2023-10-05T19:40:18.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.439+0000 7fd4034ee700 1 -- 172.21.15.12:0/3118385934 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3ec02d040 con 0x5636d97a7190 2023-10-05T19:40:18.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.439+0000 7fd4034ee700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97aa820 con 0x5636d976f6e0 2023-10-05T19:40:18.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.439+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3f4014040 con 0x5636d976f6e0 2023-10-05T19:40:18.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.439+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97aa820 con 0x7fd3fc007030 2023-10-05T19:40:18.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.439+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3f4014040 con 0x7fd3fc007030 2023-10-05T19:40:18.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.439+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97aa820 con 0x5636d976f6e0 2023-10-05T19:40:18.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.439+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3f4014040 con 0x5636d976f6e0 2023-10-05T19:40:18.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.443+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97aa820 con 0x5636d97a29f0 2023-10-05T19:40:18.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.443+0000 7fd403cef700 1 -- 172.21.15.12:0/3118385934 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3fc01cc20 con 0x5636d97a29f0 2023-10-05T19:40:18.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.443+0000 7fd403cef700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97aa820 con 0x5636d97a7190 2023-10-05T19:40:18.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.443+0000 7fd4034ee700 1 -- 172.21.15.12:0/3118385934 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3ec02d040 con 0x5636d97a7190 2023-10-05T19:40:18.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.443+0000 7fd4034ee700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97aa820 con 0x5636d97a7190 2023-10-05T19:40:18.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.443+0000 7fd4034ee700 1 -- 172.21.15.12:0/3118385934 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3ec02d040 con 0x5636d97a7190 2023-10-05T19:40:18.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.443+0000 7fd4034ee700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97aa820 con 0x7fd3fc007030 2023-10-05T19:40:18.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.443+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3f4014040 con 0x7fd3fc007030 2023-10-05T19:40:18.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.443+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97aa820 con 0x5636d97a7190 2023-10-05T19:40:18.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.443+0000 7fd4034ee700 1 -- 172.21.15.12:0/3118385934 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3ec02d040 con 0x5636d97a7190 2023-10-05T19:40:18.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.443+0000 7fd4034ee700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97aa820 con 0x5636d97a7190 2023-10-05T19:40:18.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.447+0000 7fd4034ee700 1 -- 172.21.15.12:0/3118385934 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3ec02d040 con 0x5636d97a7190 2023-10-05T19:40:18.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.447+0000 7fd4034ee700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97aa820 con 0x5636d976f6e0 2023-10-05T19:40:18.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.447+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3f4014040 con 0x5636d976f6e0 2023-10-05T19:40:18.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.447+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97aa820 con 0x5636d976f6e0 2023-10-05T19:40:18.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.447+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3f4014040 con 0x5636d976f6e0 2023-10-05T19:40:18.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.447+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97aa820 con 0x7fd3fc007030 2023-10-05T19:40:18.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.447+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3f4014040 con 0x7fd3fc007030 2023-10-05T19:40:18.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.447+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97aa820 con 0x5636d97a7190 2023-10-05T19:40:18.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.447+0000 7fd4034ee700 1 -- 172.21.15.12:0/3118385934 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3ec02d040 con 0x5636d97a7190 2023-10-05T19:40:18.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.447+0000 7fd4034ee700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97aa820 con 0x5636d97a29f0 2023-10-05T19:40:18.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.451+0000 7fd403cef700 1 -- 172.21.15.12:0/3118385934 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3fc01cc20 con 0x5636d97a29f0 2023-10-05T19:40:18.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.451+0000 7fd403cef700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97aa820 con 0x5636d976f6e0 2023-10-05T19:40:18.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.451+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3f4014040 con 0x5636d976f6e0 2023-10-05T19:40:18.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.451+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97aa820 con 0x7fd3fc007030 2023-10-05T19:40:18.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.451+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3f4014040 con 0x7fd3fc007030 2023-10-05T19:40:18.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.451+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97aa820 con 0x5636d97a7190 2023-10-05T19:40:18.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.451+0000 7fd4034ee700 1 -- 172.21.15.12:0/3118385934 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3ec02d040 con 0x5636d97a7190 2023-10-05T19:40:18.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.451+0000 7fd4034ee700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97aa820 con 0x7fd3fc007030 2023-10-05T19:40:18.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.455+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3f4014040 con 0x7fd3fc007030 2023-10-05T19:40:18.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.455+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97aa820 con 0x5636d97a29f0 2023-10-05T19:40:18.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.455+0000 7fd403cef700 1 -- 172.21.15.12:0/3118385934 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3fc01cc20 con 0x5636d97a29f0 2023-10-05T19:40:18.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.455+0000 7fd403cef700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97aa820 con 0x5636d97a7190 2023-10-05T19:40:18.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.455+0000 7fd4034ee700 1 -- 172.21.15.12:0/3118385934 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3ec02d040 con 0x5636d97a7190 2023-10-05T19:40:18.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.455+0000 7fd4034ee700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97aa820 con 0x7fd3fc007030 2023-10-05T19:40:18.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.455+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd3f4014040 con 0x7fd3fc007030 2023-10-05T19:40:18.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.455+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97aa820 con 0x5636d976f6e0 2023-10-05T19:40:18.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.455+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fd3f4014040 con 0x5636d976f6e0 2023-10-05T19:40:18.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.455+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97aa820 con 0x5636d97a29f0 2023-10-05T19:40:18.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.459+0000 7fd403cef700 1 -- 172.21.15.12:0/3118385934 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fd3fc01cc20 con 0x5636d97a29f0 2023-10-05T19:40:18.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.459+0000 7fd4061bae40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:18.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.459+0000 7fd4061bae40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:18.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.459+0000 7fd4061bae40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:18.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.459+0000 7fd4061bae40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:18.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.459+0000 7fd4061bae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.459+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97a6430 con 0x5636d976f6e0 2023-10-05T19:40:18.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.459+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fd3f4014040 con 0x5636d976f6e0 2023-10-05T19:40:18.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.459+0000 7fd4061bae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:18.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.459+0000 7fd4061bae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.459+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97ab1c0 con 0x5636d97a29f0 2023-10-05T19:40:18.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.459+0000 7fd403cef700 1 -- 172.21.15.12:0/3118385934 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fd3fc01cc20 con 0x5636d97a29f0 2023-10-05T19:40:18.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.459+0000 7fd4061bae40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:18.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.459+0000 7fd4061bae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.459+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97abab0 con 0x5636d976f6e0 2023-10-05T19:40:18.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.459+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fd3f4014040 con 0x5636d976f6e0 2023-10-05T19:40:18.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.459+0000 7fd4061bae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:18.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.459+0000 7fd4061bae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.459+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97abab0 con 0x5636d97a7190 2023-10-05T19:40:18.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.463+0000 7fd4034ee700 1 -- 172.21.15.12:0/3118385934 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fd3ec02d040 con 0x5636d97a7190 2023-10-05T19:40:18.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.463+0000 7fd4061bae40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:18.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.463+0000 7fd4061bae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.463+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97ac220 con 0x7fd3fc007030 2023-10-05T19:40:18.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.463+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7fd3f4014040 con 0x7fd3fc007030 2023-10-05T19:40:18.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.463+0000 7fd4061bae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:18.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.463+0000 7fd4061bae40 20 zone default found 2023-10-05T19:40:18.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.463+0000 7fd4061bae40 4 Realm: () 2023-10-05T19:40:18.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.463+0000 7fd4061bae40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:18.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.463+0000 7fd4061bae40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:18.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.463+0000 7fd4061bae40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:18.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.463+0000 7fd4061bae40 20 zonegroup default 2023-10-05T19:40:18.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.463+0000 7fd4061bae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.463+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97abe60 con 0x7fd3fc007030 2023-10-05T19:40:18.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.463+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fd3f4014040 con 0x7fd3fc007030 2023-10-05T19:40:18.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.463+0000 7fd4061bae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:18.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.463+0000 7fd4061bae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.463+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97ad7e0 con 0x5636d97a7190 2023-10-05T19:40:18.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.463+0000 7fd4034ee700 1 -- 172.21.15.12:0/3118385934 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fd3ec02d040 con 0x5636d97a7190 2023-10-05T19:40:18.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.463+0000 7fd4061bae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:18.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.463+0000 7fd4061bae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.463+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97adb90 con 0x5636d97a7190 2023-10-05T19:40:18.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.463+0000 7fd4034ee700 1 -- 172.21.15.12:0/3118385934 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fd3ec03c040 con 0x5636d97a7190 2023-10-05T19:40:18.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.463+0000 7fd4061bae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:18.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.463+0000 7fd4061bae40 20 started sync module instance, tier type = 2023-10-05T19:40:18.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.463+0000 7fd4061bae40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:18.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.463+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97acea0 con 0x5636d97a29f0 2023-10-05T19:40:18.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.467+0000 7fd403cef700 1 -- 172.21.15.12:0/3118385934 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'27 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd3fc01cc20 con 0x5636d97a29f0 2023-10-05T19:40:18.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.467+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94793576927904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97ab6e0 con 0x5636d97a29f0 2023-10-05T19:40:18.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.467+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97ae960 con 0x5636d976f6e0 2023-10-05T19:40:18.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.471+0000 7fd403cef700 1 -- 172.21.15.12:0/3118385934 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94793576927904] v28'28 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd3fc01cc20 con 0x5636d97a29f0 2023-10-05T19:40:18.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.471+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'27 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd3f4014040 con 0x5636d976f6e0 2023-10-05T19:40:18.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.471+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94793576931136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97aed60 con 0x5636d976f6e0 2023-10-05T19:40:18.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.471+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97af600 con 0x5636d97a7190 2023-10-05T19:40:18.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.475+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94793576931136] v28'28 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd3f4014040 con 0x5636d976f6e0 2023-10-05T19:40:18.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.475+0000 7fd4034ee700 1 -- 172.21.15.12:0/3118385934 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'27 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd3ec03c040 con 0x5636d97a7190 2023-10-05T19:40:18.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.475+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94793576934752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97af1b0 con 0x5636d97a7190 2023-10-05T19:40:18.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.475+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97b03e0 con 0x5636d97a29f0 2023-10-05T19:40:18.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.479+0000 7fd403cef700 1 -- 172.21.15.12:0/3118385934 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'27 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd3fc01cc20 con 0x5636d97a29f0 2023-10-05T19:40:18.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.479+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94793576939248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97b0940 con 0x5636d97a29f0 2023-10-05T19:40:18.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.479+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97b14a0 con 0x5636d976f6e0 2023-10-05T19:40:18.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.479+0000 7fd4034ee700 1 -- 172.21.15.12:0/3118385934 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94793576934752] v28'28 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd3ec03c040 con 0x5636d97a7190 2023-10-05T19:40:18.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.479+0000 7fd403cef700 1 -- 172.21.15.12:0/3118385934 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94793576939248] v28'28 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd3fc01cc20 con 0x5636d97a29f0 2023-10-05T19:40:18.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.479+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'27 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd3f4014040 con 0x5636d976f6e0 2023-10-05T19:40:18.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.479+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94793576937984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97b0f70 con 0x5636d976f6e0 2023-10-05T19:40:18.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.479+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97b1d90 con 0x7fd3fc007030 2023-10-05T19:40:18.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.483+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94793576937984] v28'28 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd3f4014040 con 0x5636d976f6e0 2023-10-05T19:40:18.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.483+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'27 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd3f4014040 con 0x7fd3fc007030 2023-10-05T19:40:18.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.483+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94793576940496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97b1a40 con 0x7fd3fc007030 2023-10-05T19:40:18.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.483+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97b22e0 con 0x7fd3fc007030 2023-10-05T19:40:18.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.487+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(95 notify.5 [watch watch cookie 94793576940496] v28'28 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd3f4014040 con 0x7fd3fc007030 2023-10-05T19:40:18.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.487+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(96 notify.6 [create] v28'27 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd3f4014040 con 0x7fd3fc007030 2023-10-05T19:40:18.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.487+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94793576945536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97b20c0 con 0x7fd3fc007030 2023-10-05T19:40:18.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.487+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97b2b90 con 0x7fd3fc007030 2023-10-05T19:40:18.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.487+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(97 notify.6 [watch watch cookie 94793576945536] v28'28 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd3f4014040 con 0x7fd3fc007030 2023-10-05T19:40:18.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.491+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(98 notify.7 [create] v28'27 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd3f4014040 con 0x7fd3fc007030 2023-10-05T19:40:18.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.491+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94793576946640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97b20a0 con 0x7fd3fc007030 2023-10-05T19:40:18.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.491+0000 7fd4061bae40 20 add_watcher() i=0 2023-10-05T19:40:18.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.491+0000 7fd4061bae40 20 add_watcher() i=1 2023-10-05T19:40:18.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.491+0000 7fd4061bae40 20 add_watcher() i=2 2023-10-05T19:40:18.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.491+0000 7fd4061bae40 20 add_watcher() i=3 2023-10-05T19:40:18.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.491+0000 7fd4061bae40 20 add_watcher() i=4 2023-10-05T19:40:18.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.491+0000 7fd4061bae40 20 add_watcher() i=5 2023-10-05T19:40:18.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.491+0000 7fd4061bae40 20 add_watcher() i=6 2023-10-05T19:40:18.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.491+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94793576946640] v28'28 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd3f4014040 con 0x7fd3fc007030 2023-10-05T19:40:18.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.491+0000 7fd4061bae40 20 add_watcher() i=7 2023-10-05T19:40:18.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.491+0000 7fd4061bae40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:18.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.491+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97b5f40 con 0x5636d976f6e0 2023-10-05T19:40:18.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.491+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fd3f4014040 con 0x5636d976f6e0 2023-10-05T19:40:18.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.491+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94793576949696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97b3400 con 0x5636d976f6e0 2023-10-05T19:40:18.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.495+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94793576949696] v28'52 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fd3f4014040 con 0x5636d976f6e0 2023-10-05T19:40:18.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.495+0000 7fd3d57e2700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:18.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.495+0000 7fd4061bae40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:18.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.495+0000 7fd4061bae40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:18.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.495+0000 7fd4061bae40 1 Processor -- start 2023-10-05T19:40:18.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.495+0000 7fd4061bae40 1 -- start start 2023-10-05T19:40:18.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.495+0000 7fd4061bae40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636d96e32b0 0x5636d96e3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.495+0000 7fd4061bae40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636d96e3bc0 con 0x5636d96e32b0 2023-10-05T19:40:18.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.495+0000 7fd4034ee700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636d96e32b0 0x5636d96e3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.495+0000 7fd4034ee700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636d96e32b0 0x5636d96e3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36690/0 (socket says 172.21.15.12:36690) 2023-10-05T19:40:18.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.495+0000 7fd4034ee700 1 -- 172.21.15.12:0/3983362462 learned_addr learned my addr 172.21.15.12:0/3983362462 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:18.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.499+0000 7fd4034ee700 1 -- 172.21.15.12:0/3983362462 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636d96e3d00 con 0x5636d96e32b0 2023-10-05T19:40:18.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.499+0000 7fd4034ee700 1 --2- 172.21.15.12:0/3983362462 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636d96e32b0 0x5636d96e3680 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x5636d97a1e60 tx=0x7fd3ec027040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a43903e8a5a06ef server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.499+0000 7fd3d4fe1700 1 -- 172.21.15.12:0/3983362462 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd3ec039040 con 0x5636d96e32b0 2023-10-05T19:40:18.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.499+0000 7fd3d4fe1700 1 -- 172.21.15.12:0/3983362462 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3ec03a040 con 0x5636d96e32b0 2023-10-05T19:40:18.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.499+0000 7fd3d4fe1700 1 -- 172.21.15.12:0/3983362462 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd3ec02a080 con 0x5636d96e32b0 2023-10-05T19:40:18.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.499+0000 7fd4061bae40 1 -- 172.21.15.12:0/3983362462 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636d96e32b0 msgr2=0x5636d96e3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.499+0000 7fd4061bae40 1 --2- 172.21.15.12:0/3983362462 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636d96e32b0 0x5636d96e3680 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x5636d97a1e60 tx=0x7fd3ec027040 comp rx=0 tx=0).stop 2023-10-05T19:40:18.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.499+0000 7fd4061bae40 1 -- 172.21.15.12:0/3983362462 shutdown_connections 2023-10-05T19:40:18.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.499+0000 7fd4061bae40 1 --2- 172.21.15.12:0/3983362462 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636d96e32b0 0x5636d96e3680 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.499+0000 7fd4061bae40 1 -- 172.21.15.12:0/3983362462 >> 172.21.15.12:0/3983362462 conn(0x5636d97bb9f0 msgr2=0x5636d97b9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:18.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.499+0000 7fd4061bae40 1 -- 172.21.15.12:0/3983362462 shutdown_connections 2023-10-05T19:40:18.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.499+0000 7fd4061bae40 1 -- 172.21.15.12:0/3983362462 wait complete. 2023-10-05T19:40:18.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.499+0000 7fd4061bae40 1 Processor -- start 2023-10-05T19:40:18.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.499+0000 7fd4061bae40 1 -- start start 2023-10-05T19:40:18.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.499+0000 7fd4061bae40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636d96e32b0 0x5636d97c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.499+0000 7fd4061bae40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636d97c93c0 con 0x5636d96e32b0 2023-10-05T19:40:18.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.499+0000 7fd4034ee700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636d96e32b0 0x5636d97c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.503+0000 7fd4034ee700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636d96e32b0 0x5636d97c83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36704/0 (socket says 172.21.15.12:36704) 2023-10-05T19:40:18.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.503+0000 7fd4034ee700 1 -- 172.21.15.12:0/3117817563 learned_addr learned my addr 172.21.15.12:0/3117817563 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:18.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.503+0000 7fd4034ee700 1 -- 172.21.15.12:0/3117817563 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f0001e10 con 0x5636d96e32b0 2023-10-05T19:40:18.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.503+0000 7fd4034ee700 1 --2- 172.21.15.12:0/3117817563 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636d96e32b0 0x5636d97c83c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc02fd40 tx=0x7fd3ec03f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.503+0000 7fd34effd700 1 -- 172.21.15.12:0/3117817563 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd3ec039040 con 0x5636d96e32b0 2023-10-05T19:40:18.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.503+0000 7fd34effd700 1 -- 172.21.15.12:0/3117817563 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3ec02b040 con 0x5636d96e32b0 2023-10-05T19:40:18.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.503+0000 7fd4061bae40 1 -- 172.21.15.12:0/3117817563 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5636d97c89c0 con 0x5636d96e32b0 2023-10-05T19:40:18.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.503+0000 7fd34effd700 1 -- 172.21.15.12:0/3117817563 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd3ec01a040 con 0x5636d96e32b0 2023-10-05T19:40:18.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.503+0000 7fd4061bae40 1 -- 172.21.15.12:0/3117817563 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5636d97c8e00 con 0x5636d96e32b0 2023-10-05T19:40:18.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.503+0000 7fd34effd700 1 -- 172.21.15.12:0/3117817563 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fd3ec04c740 con 0x5636d96e32b0 2023-10-05T19:40:18.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.503+0000 7fd34effd700 1 --2- 172.21.15.12:0/3117817563 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd344030c50 0x7fd344033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.503+0000 7fd34effd700 1 -- 172.21.15.12:0/3117817563 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7fd3ec028040 con 0x5636d96e32b0 2023-10-05T19:40:18.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.503+0000 7fd4061bae40 5 note: GC not initialized 2023-10-05T19:40:18.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.503+0000 7fd4044f0700 1 --2- 172.21.15.12:0/3117817563 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd344030c50 0x7fd344033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.507+0000 7fd4044f0700 1 --2- 172.21.15.12:0/3117817563 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd344030c50 0x7fd344033100 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x5636d97b2350 tx=0x7fd3f400ab80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.507+0000 7fd341ffb700 20 reqs_thread_entry: start 2023-10-05T19:40:18.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.507+0000 7fd4061bae40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:18.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.507+0000 7fd343fff700 20 reqs_thread_entry: start 2023-10-05T19:40:18.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.507+0000 7fd4061bae40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:18.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.507+0000 7fd32ffff700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:48 2023 2023-10-05T19:40:18.517 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:18.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.507+0000 7fd32f7fe700 20 reqs_thread_entry: start 2023-10-05T19:40:18.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.507+0000 7fd4061bae40 20 get_system_obj_state: rctx=0x7ffef220bd08 obj=default.rgw.meta:users.uid:fud state=0x5636d96480b0 s->prefetch_data=0 2023-10-05T19:40:18.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.507+0000 7fd4061bae40 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2023-10-05T19:40:18.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.507+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:102 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d9830c20 con 0x7fd3fc007030 2023-10-05T19:40:18.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.507+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(102 fud [call,getxattrs,stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x7fd3f4014040 con 0x7fd3fc007030 2023-10-05T19:40:18.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.511+0000 7fd4061bae40 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x0 2023-10-05T19:40:18.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.511+0000 7fd4061bae40 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2023-10-05T19:40:18.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.511+0000 7fd4061bae40 20 get_system_obj_state: rctx=0x7ffef220b9f8 obj=default.rgw.meta:users.email:foo@foo.com state=0x5636d96480b0 s->prefetch_data=0 2023-10-05T19:40:18.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.511+0000 7fd4061bae40 10 cache get: name=default.rgw.meta+users.email+foo@foo.com : miss 2023-10-05T19:40:18.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.511+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 8.3 8:c162d5d1:users.email::foo@foo.com:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d9830c20 con 0x5636d976f6e0 2023-10-05T19:40:18.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.511+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(103 foo@foo.com [getxattrs out=4b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 197+0+20 (crc 0 0 0) 0x7fd3f4014040 con 0x5636d976f6e0 2023-10-05T19:40:18.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.511+0000 7fd4061bae40 10 cache put: name=default.rgw.meta+users.email+foo@foo.com info.flags=0x6 2023-10-05T19:40:18.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.511+0000 7fd4061bae40 10 adding default.rgw.meta+users.email+foo@foo.com to cache LRU end 2023-10-05T19:40:18.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.511+0000 7fd4061bae40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:18.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.511+0000 7fd4061bae40 10 cache get: name=default.rgw.meta+users.email+foo@foo.com : type miss (requested=0x1, cached=0x6) 2023-10-05T19:40:18.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.511+0000 7fd4061bae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.511+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:104 8.3 8:c162d5d1:users.email::foo@foo.com:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d9830c20 con 0x5636d976f6e0 2023-10-05T19:40:18.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.511+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(104 foo@foo.com [read 0~7 out=7b] v0'0 uv1 ondisk = 0) v8 ==== 155+0+7 (crc 0 0 0) 0x7fd3f4014040 con 0x5636d976f6e0 2023-10-05T19:40:18.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.511+0000 7fd4061bae40 20 rados_obj.operate() r=0 bl.length=7 2023-10-05T19:40:18.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.511+0000 7fd4061bae40 10 cache put: name=default.rgw.meta+users.email+foo@foo.com info.flags=0x1 2023-10-05T19:40:18.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.511+0000 7fd4061bae40 10 moving default.rgw.meta+users.email+foo@foo.com to cache LRU end 2023-10-05T19:40:18.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.511+0000 7fd4061bae40 20 get_system_obj_state: rctx=0x7ffef220b9f8 obj=default.rgw.meta:users.uid:foo state=0x5636d961e2b0 s->prefetch_data=0 2023-10-05T19:40:18.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.511+0000 7fd4061bae40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2023-10-05T19:40:18.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.511+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:105 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d98321b0 con 0x5636d976f6e0 2023-10-05T19:40:18.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.511+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(105 foo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7fd3f4014040 con 0x5636d976f6e0 2023-10-05T19:40:18.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.511+0000 7fd4061bae40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x16 2023-10-05T19:40:18.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.511+0000 7fd4061bae40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:18.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.511+0000 7fd4061bae40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:18.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.511+0000 7fd4061bae40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:18.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.511+0000 7fd4061bae40 10 cache get: name=default.rgw.meta+users.uid+foo : type miss (requested=0x11, cached=0x16) 2023-10-05T19:40:18.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.511+0000 7fd4061bae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.511+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:106 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d9834530 con 0x5636d976f6e0 2023-10-05T19:40:18.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.511+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(106 foo [call,call out=48b,read 0~357 out=357b] v0'0 uv1 ondisk = 0) v8 ==== 231+0+405 (crc 0 0 0) 0x7fd3f4014040 con 0x5636d976f6e0 2023-10-05T19:40:18.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.511+0000 7fd4061bae40 20 rados_obj.operate() r=0 bl.length=357 2023-10-05T19:40:18.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.511+0000 7fd4061bae40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x11 2023-10-05T19:40:18.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.511+0000 7fd4061bae40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:18.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.511+0000 7fd4061bae40 10 chain_cache_entry: cache_locator=default.rgw.meta+users.uid+foo 2023-10-05T19:40:18.523 INFO:teuthology.orchestra.run.smithi012.stderr:could not create user: unable to parse parameters, email: foo@foo.com is the email address of an existing user 2023-10-05T19:40:18.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.515+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:107 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94793576927904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d97ef010 con 0x5636d97a29f0 2023-10-05T19:40:18.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.515+0000 7fd403cef700 1 -- 172.21.15.12:0/3118385934 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(107 notify.0 [watch unwatch cookie 94793576927904] v28'29 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd3fc01cc20 con 0x5636d97a29f0 2023-10-05T19:40:18.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.515+0000 7fd4061bae40 20 remove_watcher() i=0 2023-10-05T19:40:18.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.515+0000 7fd4061bae40 2 removed watcher, disabling cache 2023-10-05T19:40:18.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.515+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:108 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94793576931136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d9784520 con 0x5636d976f6e0 2023-10-05T19:40:18.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.519+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(108 notify.1 [watch unwatch cookie 94793576931136] v28'29 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd3f4014040 con 0x5636d976f6e0 2023-10-05T19:40:18.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.519+0000 7fd4061bae40 20 remove_watcher() i=1 2023-10-05T19:40:18.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.519+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:109 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94793576934752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d9784520 con 0x5636d97a7190 2023-10-05T19:40:18.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.523+0000 7fd4034ee700 1 -- 172.21.15.12:0/3118385934 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(109 notify.2 [watch unwatch cookie 94793576934752] v28'29 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd3ec03c040 con 0x5636d97a7190 2023-10-05T19:40:18.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.523+0000 7fd4061bae40 20 remove_watcher() i=2 2023-10-05T19:40:18.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.523+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:110 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94793576939248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d9784520 con 0x5636d97a29f0 2023-10-05T19:40:18.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.523+0000 7fd403cef700 1 -- 172.21.15.12:0/3118385934 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(110 notify.3 [watch unwatch cookie 94793576939248] v28'29 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd3fc01cc20 con 0x5636d97a29f0 2023-10-05T19:40:18.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.527+0000 7fd4061bae40 20 remove_watcher() i=3 2023-10-05T19:40:18.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.527+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:111 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94793576937984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d9784520 con 0x5636d976f6e0 2023-10-05T19:40:18.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.527+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== osd_op_reply(111 notify.4 [watch unwatch cookie 94793576937984] v28'29 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd3f4014040 con 0x5636d976f6e0 2023-10-05T19:40:18.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.527+0000 7fd4061bae40 20 remove_watcher() i=4 2023-10-05T19:40:18.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.527+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:112 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94793576940496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d9784520 con 0x7fd3fc007030 2023-10-05T19:40:18.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.531+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(112 notify.5 [watch unwatch cookie 94793576940496] v28'29 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd3f4014040 con 0x7fd3fc007030 2023-10-05T19:40:18.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.531+0000 7fd4061bae40 20 remove_watcher() i=5 2023-10-05T19:40:18.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.531+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:113 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94793576945536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d9784520 con 0x7fd3fc007030 2023-10-05T19:40:18.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.535+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(113 notify.6 [watch unwatch cookie 94793576945536] v28'29 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd3f4014040 con 0x7fd3fc007030 2023-10-05T19:40:18.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.535+0000 7fd4061bae40 20 remove_watcher() i=6 2023-10-05T19:40:18.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.535+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:114 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94793576946640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d9784520 con 0x7fd3fc007030 2023-10-05T19:40:18.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.535+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.0 v2:172.21.15.8:6800/27563 30 ==== osd_op_reply(114 notify.7 [watch unwatch cookie 94793576946640] v28'29 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd3f4014040 con 0x7fd3fc007030 2023-10-05T19:40:18.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.535+0000 7fd4061bae40 20 remove_watcher() i=7 2023-10-05T19:40:18.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.539+0000 7fd3d57e2700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:18.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.539+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:115 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94793576949696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636d9784520 con 0x5636d976f6e0 2023-10-05T19:40:18.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.543+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 <== osd.1 v2:172.21.15.12:6808/29933 39 ==== osd_op_reply(115 data_loggenerations_metadata [watch unwatch cookie 94793576949696] v28'53 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fd3f4014040 con 0x5636d976f6e0 2023-10-05T19:40:18.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.543+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fd3fc007030 msgr2=0x7fd3fc0113b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.543+0000 7fd4061bae40 1 --2- 172.21.15.12:0/3118385934 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fd3fc007030 0x7fd3fc0113b0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.543+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5636d976f6e0 msgr2=0x5636d97a04c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.543+0000 7fd4061bae40 1 --2- 172.21.15.12:0/3118385934 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5636d976f6e0 0x5636d97a04c0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.543+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5636d97a29f0 msgr2=0x5636d97a4dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.543+0000 7fd4061bae40 1 --2- 172.21.15.12:0/3118385934 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5636d97a29f0 0x5636d97a4dd0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.543+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5636d97a7190 msgr2=0x5636d97a95a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.543+0000 7fd4061bae40 1 --2- 172.21.15.12:0/3118385934 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5636d97a7190 0x5636d97a95a0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.543+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd3e4030c50 msgr2=0x7fd3e4033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.543+0000 7fd4061bae40 1 --2- 172.21.15.12:0/3118385934 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd3e4030c50 0x7fd3e4033100 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec0044f0 tx=0x7fd3ec006690 comp rx=0 tx=0).stop 2023-10-05T19:40:18.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.543+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636d976edd0 msgr2=0x5636d9794a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.543+0000 7fd4044f0700 1 -- 172.21.15.12:0/3118385934 reap_dead start 2023-10-05T19:40:18.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.543+0000 7fd4061bae40 1 --2- 172.21.15.12:0/3118385934 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636d976edd0 0x5636d9794a00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc01f040 tx=0x7fd3fc01c000 comp rx=0 tx=0).stop 2023-10-05T19:40:18.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.543+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 shutdown_connections 2023-10-05T19:40:18.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.543+0000 7fd4061bae40 1 --2- 172.21.15.12:0/3118385934 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636d976edd0 0x5636d9794a00 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.543+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 >> 172.21.15.12:0/3118385934 conn(0x5636d976a7f0 msgr2=0x5636d97882c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:18.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.543+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 shutdown_connections 2023-10-05T19:40:18.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.543+0000 7fd4061bae40 1 -- 172.21.15.12:0/3118385934 wait complete. 2023-10-05T19:40:18.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.543+0000 7fd4061bae40 1 -- 172.21.15.12:0/3117817563 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd344030c50 msgr2=0x7fd344033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.543+0000 7fd4061bae40 1 --2- 172.21.15.12:0/3117817563 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd344030c50 0x7fd344033100 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x5636d97b2350 tx=0x7fd3f400ab80 comp rx=0 tx=0).stop 2023-10-05T19:40:18.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.543+0000 7fd4061bae40 1 -- 172.21.15.12:0/3117817563 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636d96e32b0 msgr2=0x5636d97c83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.543+0000 7fd4061bae40 1 --2- 172.21.15.12:0/3117817563 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636d96e32b0 0x5636d97c83c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc02fd40 tx=0x7fd3ec03f000 comp rx=0 tx=0).stop 2023-10-05T19:40:18.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.543+0000 7fd4061bae40 1 -- 172.21.15.12:0/3117817563 shutdown_connections 2023-10-05T19:40:18.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.543+0000 7fd4061bae40 1 --2- 172.21.15.12:0/3117817563 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd344030c50 0x7fd344033100 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.543+0000 7fd4061bae40 1 --2- 172.21.15.12:0/3117817563 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636d96e32b0 0x5636d97c83c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.543+0000 7fd4061bae40 1 -- 172.21.15.12:0/3117817563 >> 172.21.15.12:0/3117817563 conn(0x5636d97bb9f0 msgr2=0x5636d97b9a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:18.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.543+0000 7fd4061bae40 1 -- 172.21.15.12:0/3117817563 shutdown_connections 2023-10-05T19:40:18.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.543+0000 7fd4061bae40 1 -- 172.21.15.12:0/3117817563 wait complete. 2023-10-05T19:40:18.559 DEBUG:teuthology.orchestra.run:got remote process result: 240 2023-10-05T19:40:18.560 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2023-10-05T19:40:18.560 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2023-10-05T19:40:18.560 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2023-10-05T19:40:18.599 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:18.600 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:18.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.591+0000 7fea1bfdde40 1 Processor -- start 2023-10-05T19:40:18.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.591+0000 7fea1bfdde40 1 -- start start 2023-10-05T19:40:18.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.591+0000 7fea1bfdde40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x555b29cdd2e0 0x555b29cdf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.591+0000 7fea1bfdde40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x555b29c26bd0 con 0x555b29cd8d00 2023-10-05T19:40:18.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.591+0000 7fea1bfdde40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x555b29be3ee0 con 0x555b29cdd2e0 2023-10-05T19:40:18.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.595+0000 7fea19311700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x555b29cdd2e0 0x555b29cdf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.595+0000 7fea19b12700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x555b29cd8d00 0x555b29cd90d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:47222/0 (socket says 172.21.15.12:47222) 2023-10-05T19:40:18.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.595+0000 7fea19311700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x555b29cdd2e0 0x555b29cdf6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36720/0 (socket says 172.21.15.12:36720) 2023-10-05T19:40:18.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.595+0000 7fea19b12700 1 -- 172.21.15.12:0/1616045828 learned_addr learned my addr 172.21.15.12:0/1616045828 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:18.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.595+0000 7fea19311700 1 -- 172.21.15.12:0/1616045828 >> v1:172.21.15.8:6789/0 conn(0x555b29cd8d00 legacy=0x555b29cd90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.595+0000 7fea19311700 1 -- 172.21.15.12:0/1616045828 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b29cd9910 con 0x555b29cdd2e0 2023-10-05T19:40:18.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.595+0000 7fea19311700 1 --2- 172.21.15.12:0/1616045828 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555b29cdd2e0 0x555b29cdf6c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fea0400a5d0 tx=0x7fea04005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c31d0dd847fac7 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.595+0000 7fea18b10700 1 -- 172.21.15.12:0/1616045828 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea040086a0 con 0x555b29cdd2e0 2023-10-05T19:40:18.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.595+0000 7fea18b10700 1 -- 172.21.15.12:0/1616045828 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea04008800 con 0x555b29cdd2e0 2023-10-05T19:40:18.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.595+0000 7fea18b10700 1 -- 172.21.15.12:0/1616045828 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea04008ad0 con 0x555b29cdd2e0 2023-10-05T19:40:18.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.595+0000 7fea1bfdde40 1 -- 172.21.15.12:0/1616045828 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555b29cdd2e0 msgr2=0x555b29cdf6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.595+0000 7fea1bfdde40 1 --2- 172.21.15.12:0/1616045828 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555b29cdd2e0 0x555b29cdf6c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fea0400a5d0 tx=0x7fea04005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:18.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.595+0000 7fea1bfdde40 1 -- 172.21.15.12:0/1616045828 shutdown_connections 2023-10-05T19:40:18.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.595+0000 7fea1bfdde40 1 --2- 172.21.15.12:0/1616045828 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555b29cdd2e0 0x555b29cdf6c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.595+0000 7fea1bfdde40 1 -- 172.21.15.12:0/1616045828 >> 172.21.15.12:0/1616045828 conn(0x555b29cd4720 msgr2=0x555b29cd6b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:18.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.595+0000 7fea1bfdde40 1 -- 172.21.15.12:0/1616045828 shutdown_connections 2023-10-05T19:40:18.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.595+0000 7fea1bfdde40 1 -- 172.21.15.12:0/1616045828 wait complete. 2023-10-05T19:40:18.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.599+0000 7fea1bfdde40 1 Processor -- start 2023-10-05T19:40:18.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.599+0000 7fea1bfdde40 1 -- start start 2023-10-05T19:40:18.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.599+0000 7fea1bfdde40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555b29cd8d00 0x555b29c4db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.599+0000 7fea1bfdde40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b29ba7e30 con 0x555b29cd8d00 2023-10-05T19:40:18.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.599+0000 7fea19b12700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555b29cd8d00 0x555b29c4db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.599+0000 7fea19b12700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555b29cd8d00 0x555b29c4db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36736/0 (socket says 172.21.15.12:36736) 2023-10-05T19:40:18.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.599+0000 7fea19b12700 1 -- 172.21.15.12:0/2178079535 learned_addr learned my addr 172.21.15.12:0/2178079535 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:18.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.599+0000 7fea19b12700 1 -- 172.21.15.12:0/2178079535 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea04005cf0 con 0x555b29cd8d00 2023-10-05T19:40:18.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.599+0000 7fea19b12700 1 --2- 172.21.15.12:0/2178079535 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555b29cd8d00 0x555b29c4db20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fea0c006fa0 tx=0x7fea0c006b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ed95bbbc4c7724b server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.603+0000 7fea137fe700 1 -- 172.21.15.12:0/2178079535 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea0c006110 con 0x555b29cd8d00 2023-10-05T19:40:18.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.603+0000 7fea137fe700 1 -- 172.21.15.12:0/2178079535 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea0c006270 con 0x555b29cd8d00 2023-10-05T19:40:18.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.603+0000 7fea137fe700 1 -- 172.21.15.12:0/2178079535 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea0c004520 con 0x555b29cd8d00 2023-10-05T19:40:18.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.603+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2178079535 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555b29cd8d00 msgr2=0x555b29c4db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.603+0000 7fea1bfdde40 1 --2- 172.21.15.12:0/2178079535 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555b29cd8d00 0x555b29c4db20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fea0c006fa0 tx=0x7fea0c006b50 comp rx=0 tx=0).stop 2023-10-05T19:40:18.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.603+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2178079535 shutdown_connections 2023-10-05T19:40:18.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.603+0000 7fea1bfdde40 1 --2- 172.21.15.12:0/2178079535 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555b29cd8d00 0x555b29c4db20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.603+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2178079535 >> 172.21.15.12:0/2178079535 conn(0x555b29cd4720 msgr2=0x555b29cf3a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:18.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.603+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2178079535 shutdown_connections 2023-10-05T19:40:18.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.603+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2178079535 wait complete. 2023-10-05T19:40:18.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.603+0000 7fea1bfdde40 1 Processor -- start 2023-10-05T19:40:18.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.603+0000 7fea1bfdde40 1 -- start start 2023-10-05T19:40:18.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.603+0000 7fea1bfdde40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555b29cd8d00 0x555b29d01f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.603+0000 7fea1bfdde40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b29b7aed0 con 0x555b29cd8d00 2023-10-05T19:40:18.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.603+0000 7fea19b12700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555b29cd8d00 0x555b29d01f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.603+0000 7fea19b12700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555b29cd8d00 0x555b29d01f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36746/0 (socket says 172.21.15.12:36746) 2023-10-05T19:40:18.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.603+0000 7fea19b12700 1 -- 172.21.15.12:0/2693008780 learned_addr learned my addr 172.21.15.12:0/2693008780 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:18.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.607+0000 7fea19b12700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b29c26bd0 con 0x555b29cd8d00 2023-10-05T19:40:18.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.607+0000 7fea19b12700 1 --2- 172.21.15.12:0/2693008780 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555b29cd8d00 0x555b29d01f60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fea0c000f80 tx=0x7fea0c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.607+0000 7fea11ffb700 1 -- 172.21.15.12:0/2693008780 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea0c013420 con 0x555b29cd8d00 2023-10-05T19:40:18.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.607+0000 7fea11ffb700 1 -- 172.21.15.12:0/2693008780 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea0c013580 con 0x555b29cd8d00 2023-10-05T19:40:18.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.607+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555b29d024a0 con 0x555b29cd8d00 2023-10-05T19:40:18.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.607+0000 7fea11ffb700 1 -- 172.21.15.12:0/2693008780 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea0c013870 con 0x555b29cd8d00 2023-10-05T19:40:18.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.607+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555b29d028e0 con 0x555b29cd8d00 2023-10-05T19:40:18.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.607+0000 7fea11ffb700 1 -- 172.21.15.12:0/2693008780 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fea0c01a070 con 0x555b29cd8d00 2023-10-05T19:40:18.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.607+0000 7fea1bfdde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.611+0000 7fea11ffb700 1 --2- 172.21.15.12:0/2693008780 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe9fc030ca0 0x7fe9fc033150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.611+0000 7fea11ffb700 1 -- 172.21.15.12:0/2693008780 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7fea0c044b60 con 0x555b29cd8d00 2023-10-05T19:40:18.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.611+0000 7fea19311700 1 --2- 172.21.15.12:0/2693008780 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe9fc030ca0 0x7fe9fc033150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.611+0000 7fea1bfdde40 1 --2- 172.21.15.12:0/2693008780 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x555b29cdd2e0 0x555b29d0da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.611+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d0e120 con 0x555b29cdd2e0 2023-10-05T19:40:18.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.611+0000 7fea1a313700 1 --2- 172.21.15.12:0/2693008780 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x555b29cdd2e0 0x555b29d0da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.611+0000 7fea19311700 1 --2- 172.21.15.12:0/2693008780 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe9fc030ca0 0x7fe9fc033150 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fea04005a90 tx=0x7fea04015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.611+0000 7fea1a313700 1 --2- 172.21.15.12:0/2693008780 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x555b29cdd2e0 0x555b29d0da20 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.611+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fea1400a500 con 0x555b29cdd2e0 2023-10-05T19:40:18.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.611+0000 7fea1bfdde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:18.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.611+0000 7fea1bfdde40 20 realm 2023-10-05T19:40:18.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.611+0000 7fea1bfdde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.611+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d0b080 con 0x555b29cdd2e0 2023-10-05T19:40:18.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.611+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fea1400a500 con 0x555b29cdd2e0 2023-10-05T19:40:18.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.611+0000 7fea1bfdde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:18.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.611+0000 7fea1bfdde40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:18.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.611+0000 7fea1bfdde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.611+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d0ec00 con 0x555b29cdd2e0 2023-10-05T19:40:18.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.611+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fea1400a500 con 0x555b29cdd2e0 2023-10-05T19:40:18.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.611+0000 7fea1bfdde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:18.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.611+0000 7fea1bfdde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.611+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d0e7e0 con 0x555b29cdd2e0 2023-10-05T19:40:18.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.615+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fea1400a500 con 0x555b29cdd2e0 2023-10-05T19:40:18.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.615+0000 7fea1bfdde40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:18.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.615+0000 7fea1bfdde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.615+0000 7fea1bfdde40 1 --2- 172.21.15.12:0/2693008780 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x555b29d0fb90 0x555b29d11f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.615+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d126e0 con 0x555b29d0fb90 2023-10-05T19:40:18.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.615+0000 7fea19b12700 1 --2- 172.21.15.12:0/2693008780 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x555b29d0fb90 0x555b29d11f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.615+0000 7fea19b12700 1 --2- 172.21.15.12:0/2693008780 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x555b29d0fb90 0x555b29d11f70 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.615+0000 7fea19b12700 1 -- 172.21.15.12:0/2693008780 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fea0c025020 con 0x555b29d0fb90 2023-10-05T19:40:18.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.615+0000 7fea1bfdde40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:18.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.615+0000 7fea1bfdde40 20 searching for the correct realm 2023-10-05T19:40:18.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.615+0000 7fea1bfdde40 1 --2- 172.21.15.12:0/2693008780 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x555b29d14510 0x555b29d168f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.615+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d16e30 con 0x555b29d14510 2023-10-05T19:40:18.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.615+0000 7fea19311700 1 --2- 172.21.15.12:0/2693008780 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x555b29d14510 0x555b29d168f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.615+0000 7fea19311700 1 --2- 172.21.15.12:0/2693008780 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x555b29d14510 0x555b29d168f0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.619+0000 7fea19311700 1 -- 172.21.15.12:0/2693008780 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fea04008a40 con 0x555b29d14510 2023-10-05T19:40:18.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.619+0000 7fea19311700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d16e30 con 0x555b29d0fb90 2023-10-05T19:40:18.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.619+0000 7fea19b12700 1 -- 172.21.15.12:0/2693008780 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea0c025020 con 0x555b29d0fb90 2023-10-05T19:40:18.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.619+0000 7fea19b12700 1 --2- 172.21.15.12:0/2693008780 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fea0c01fc20 0x7fea0c01e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.619+0000 7fea19b12700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d16e30 con 0x7fea0c01fc20 2023-10-05T19:40:18.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.619+0000 7fea1a313700 1 --2- 172.21.15.12:0/2693008780 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fea0c01fc20 0x7fea0c01e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.619+0000 7fea1a313700 1 --2- 172.21.15.12:0/2693008780 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fea0c01fc20 0x7fea0c01e590 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.619+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea14014040 con 0x7fea0c01fc20 2023-10-05T19:40:18.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.619+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d16e30 con 0x555b29cdd2e0 2023-10-05T19:40:18.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.623+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea14014040 con 0x555b29cdd2e0 2023-10-05T19:40:18.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.623+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d16e30 con 0x555b29cdd2e0 2023-10-05T19:40:18.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.623+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea14014040 con 0x555b29cdd2e0 2023-10-05T19:40:18.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.623+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d16e30 con 0x7fea0c01fc20 2023-10-05T19:40:18.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.623+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea14014040 con 0x7fea0c01fc20 2023-10-05T19:40:18.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.623+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d16e30 con 0x7fea0c01fc20 2023-10-05T19:40:18.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.623+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea14014040 con 0x7fea0c01fc20 2023-10-05T19:40:18.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.623+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d16e30 con 0x555b29d0fb90 2023-10-05T19:40:18.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.623+0000 7fea19b12700 1 -- 172.21.15.12:0/2693008780 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea0c025020 con 0x555b29d0fb90 2023-10-05T19:40:18.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.623+0000 7fea19b12700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d16e30 con 0x555b29d14510 2023-10-05T19:40:18.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.627+0000 7fea19311700 1 -- 172.21.15.12:0/2693008780 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea04012450 con 0x555b29d14510 2023-10-05T19:40:18.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.627+0000 7fea19311700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d16e30 con 0x555b29cdd2e0 2023-10-05T19:40:18.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.627+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea14014040 con 0x555b29cdd2e0 2023-10-05T19:40:18.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.627+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d16e30 con 0x7fea0c01fc20 2023-10-05T19:40:18.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.627+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea14014040 con 0x7fea0c01fc20 2023-10-05T19:40:18.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.627+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d16e30 con 0x555b29cdd2e0 2023-10-05T19:40:18.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.627+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea14014040 con 0x555b29cdd2e0 2023-10-05T19:40:18.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.627+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d16e30 con 0x555b29d0fb90 2023-10-05T19:40:18.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.627+0000 7fea19b12700 1 -- 172.21.15.12:0/2693008780 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea0c025020 con 0x555b29d0fb90 2023-10-05T19:40:18.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.627+0000 7fea19b12700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d16e30 con 0x555b29d14510 2023-10-05T19:40:18.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.627+0000 7fea19311700 1 -- 172.21.15.12:0/2693008780 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea04012450 con 0x555b29d14510 2023-10-05T19:40:18.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.627+0000 7fea19311700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d16e30 con 0x555b29d14510 2023-10-05T19:40:18.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.631+0000 7fea19311700 1 -- 172.21.15.12:0/2693008780 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea04012450 con 0x555b29d14510 2023-10-05T19:40:18.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.631+0000 7fea19311700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d16e30 con 0x7fea0c01fc20 2023-10-05T19:40:18.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.631+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea14014040 con 0x7fea0c01fc20 2023-10-05T19:40:18.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.631+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d16e30 con 0x555b29d14510 2023-10-05T19:40:18.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.631+0000 7fea19311700 1 -- 172.21.15.12:0/2693008780 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea04012450 con 0x555b29d14510 2023-10-05T19:40:18.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.631+0000 7fea19311700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d16e30 con 0x555b29d14510 2023-10-05T19:40:18.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.631+0000 7fea19311700 1 -- 172.21.15.12:0/2693008780 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea04012450 con 0x555b29d14510 2023-10-05T19:40:18.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.631+0000 7fea19311700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d16e30 con 0x555b29cdd2e0 2023-10-05T19:40:18.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.635+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea14014040 con 0x555b29cdd2e0 2023-10-05T19:40:18.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.635+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d16e30 con 0x555b29cdd2e0 2023-10-05T19:40:18.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.635+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea14014040 con 0x555b29cdd2e0 2023-10-05T19:40:18.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.635+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d16e30 con 0x7fea0c01fc20 2023-10-05T19:40:18.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.635+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea14014040 con 0x7fea0c01fc20 2023-10-05T19:40:18.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.635+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d16e30 con 0x555b29d14510 2023-10-05T19:40:18.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.635+0000 7fea19311700 1 -- 172.21.15.12:0/2693008780 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea04012450 con 0x555b29d14510 2023-10-05T19:40:18.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.635+0000 7fea19311700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d16e30 con 0x555b29d0fb90 2023-10-05T19:40:18.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.635+0000 7fea19b12700 1 -- 172.21.15.12:0/2693008780 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea0c025020 con 0x555b29d0fb90 2023-10-05T19:40:18.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.635+0000 7fea19b12700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d16e30 con 0x555b29cdd2e0 2023-10-05T19:40:18.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.639+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea14014040 con 0x555b29cdd2e0 2023-10-05T19:40:18.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.639+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d16e30 con 0x7fea0c01fc20 2023-10-05T19:40:18.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.639+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea14014040 con 0x7fea0c01fc20 2023-10-05T19:40:18.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.639+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d16e30 con 0x555b29d14510 2023-10-05T19:40:18.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.639+0000 7fea19311700 1 -- 172.21.15.12:0/2693008780 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea04012450 con 0x555b29d14510 2023-10-05T19:40:18.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.639+0000 7fea19311700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d16e30 con 0x7fea0c01fc20 2023-10-05T19:40:18.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.639+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea14014040 con 0x7fea0c01fc20 2023-10-05T19:40:18.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.639+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d16e30 con 0x555b29d0fb90 2023-10-05T19:40:18.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.639+0000 7fea19b12700 1 -- 172.21.15.12:0/2693008780 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea0c025020 con 0x555b29d0fb90 2023-10-05T19:40:18.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.639+0000 7fea19b12700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d16e30 con 0x555b29d14510 2023-10-05T19:40:18.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.643+0000 7fea19311700 1 -- 172.21.15.12:0/2693008780 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea04012450 con 0x555b29d14510 2023-10-05T19:40:18.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.643+0000 7fea19311700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d16e30 con 0x7fea0c01fc20 2023-10-05T19:40:18.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.643+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea14014040 con 0x7fea0c01fc20 2023-10-05T19:40:18.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.643+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d16e30 con 0x555b29cdd2e0 2023-10-05T19:40:18.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.643+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fea14014040 con 0x555b29cdd2e0 2023-10-05T19:40:18.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.643+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d16e30 con 0x555b29d0fb90 2023-10-05T19:40:18.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.643+0000 7fea19b12700 1 -- 172.21.15.12:0/2693008780 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fea0c025020 con 0x555b29d0fb90 2023-10-05T19:40:18.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.643+0000 7fea1bfdde40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:18.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.643+0000 7fea1bfdde40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:18.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.643+0000 7fea1bfdde40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:18.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.643+0000 7fea1bfdde40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:18.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.643+0000 7fea1bfdde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.643+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d13290 con 0x555b29cdd2e0 2023-10-05T19:40:18.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.647+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fea14014040 con 0x555b29cdd2e0 2023-10-05T19:40:18.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.647+0000 7fea1bfdde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:18.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.647+0000 7fea1bfdde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.647+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d13be0 con 0x555b29cdd2e0 2023-10-05T19:40:18.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.647+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fea14014040 con 0x555b29cdd2e0 2023-10-05T19:40:18.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.647+0000 7fea1bfdde40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:18.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.647+0000 7fea1bfdde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.647+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d12e50 con 0x555b29d14510 2023-10-05T19:40:18.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.647+0000 7fea19311700 1 -- 172.21.15.12:0/2693008780 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fea04012450 con 0x555b29d14510 2023-10-05T19:40:18.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.647+0000 7fea1bfdde40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:18.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.647+0000 7fea1bfdde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.647+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d17760 con 0x555b29d14510 2023-10-05T19:40:18.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.647+0000 7fea19311700 1 -- 172.21.15.12:0/2693008780 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7fea04012450 con 0x555b29d14510 2023-10-05T19:40:18.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.647+0000 7fea1bfdde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:18.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.647+0000 7fea1bfdde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.647+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d17b10 con 0x555b29cdd2e0 2023-10-05T19:40:18.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.651+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fea14014040 con 0x555b29cdd2e0 2023-10-05T19:40:18.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.651+0000 7fea1bfdde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:18.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.651+0000 7fea1bfdde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.651+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d17ec0 con 0x555b29cdd2e0 2023-10-05T19:40:18.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.651+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fea14014040 con 0x555b29cdd2e0 2023-10-05T19:40:18.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.651+0000 7fea1bfdde40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:18.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.651+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d18ed0 con 0x555b29d14510 2023-10-05T19:40:18.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.651+0000 7fea19311700 1 -- 172.21.15.12:0/2693008780 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fea04012450 con 0x555b29d14510 2023-10-05T19:40:18.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.651+0000 7fea19311700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d18ed0 con 0x555b29d0fb90 2023-10-05T19:40:18.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.655+0000 7fea19b12700 1 -- 172.21.15.12:0/2693008780 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea0c025020 con 0x555b29d0fb90 2023-10-05T19:40:18.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.655+0000 7fea19b12700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d18ed0 con 0x7fea0c01fc20 2023-10-05T19:40:18.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.655+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea14014040 con 0x7fea0c01fc20 2023-10-05T19:40:18.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.655+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d17b10 con 0x555b29cdd2e0 2023-10-05T19:40:18.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.655+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea14014040 con 0x555b29cdd2e0 2023-10-05T19:40:18.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.655+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d17b10 con 0x555b29cdd2e0 2023-10-05T19:40:18.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.655+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea14014040 con 0x555b29cdd2e0 2023-10-05T19:40:18.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.655+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d17b10 con 0x7fea0c01fc20 2023-10-05T19:40:18.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.655+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea14014040 con 0x7fea0c01fc20 2023-10-05T19:40:18.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.655+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d17b10 con 0x7fea0c01fc20 2023-10-05T19:40:18.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.659+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea14014040 con 0x7fea0c01fc20 2023-10-05T19:40:18.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.659+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d17b10 con 0x555b29d0fb90 2023-10-05T19:40:18.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.659+0000 7fea19b12700 1 -- 172.21.15.12:0/2693008780 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea0c025020 con 0x555b29d0fb90 2023-10-05T19:40:18.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.659+0000 7fea19b12700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d17b10 con 0x555b29d14510 2023-10-05T19:40:18.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.659+0000 7fea19311700 1 -- 172.21.15.12:0/2693008780 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea04012450 con 0x555b29d14510 2023-10-05T19:40:18.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.659+0000 7fea19311700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d17b10 con 0x555b29cdd2e0 2023-10-05T19:40:18.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.659+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea14014040 con 0x555b29cdd2e0 2023-10-05T19:40:18.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.659+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d17b10 con 0x7fea0c01fc20 2023-10-05T19:40:18.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.659+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea14014040 con 0x7fea0c01fc20 2023-10-05T19:40:18.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.659+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d17b10 con 0x555b29cdd2e0 2023-10-05T19:40:18.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.659+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea14014040 con 0x555b29cdd2e0 2023-10-05T19:40:18.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.659+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d17b10 con 0x555b29d0fb90 2023-10-05T19:40:18.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.663+0000 7fea19b12700 1 -- 172.21.15.12:0/2693008780 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea0c025020 con 0x555b29d0fb90 2023-10-05T19:40:18.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.663+0000 7fea19b12700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d17b10 con 0x555b29d14510 2023-10-05T19:40:18.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.663+0000 7fea19311700 1 -- 172.21.15.12:0/2693008780 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea04012450 con 0x555b29d14510 2023-10-05T19:40:18.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.663+0000 7fea19311700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d17b10 con 0x555b29d14510 2023-10-05T19:40:18.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.663+0000 7fea19311700 1 -- 172.21.15.12:0/2693008780 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea04012450 con 0x555b29d14510 2023-10-05T19:40:18.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.663+0000 7fea19311700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d17b10 con 0x7fea0c01fc20 2023-10-05T19:40:18.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.663+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea14014040 con 0x7fea0c01fc20 2023-10-05T19:40:18.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.663+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d17b10 con 0x555b29d14510 2023-10-05T19:40:18.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.663+0000 7fea19311700 1 -- 172.21.15.12:0/2693008780 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea04012450 con 0x555b29d14510 2023-10-05T19:40:18.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.663+0000 7fea19311700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d17b10 con 0x555b29d14510 2023-10-05T19:40:18.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.667+0000 7fea19311700 1 -- 172.21.15.12:0/2693008780 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea04012450 con 0x555b29d14510 2023-10-05T19:40:18.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.667+0000 7fea19311700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d17b10 con 0x555b29cdd2e0 2023-10-05T19:40:18.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.667+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea14014040 con 0x555b29cdd2e0 2023-10-05T19:40:18.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.667+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d17b10 con 0x555b29cdd2e0 2023-10-05T19:40:18.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.667+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea14014040 con 0x555b29cdd2e0 2023-10-05T19:40:18.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.667+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d17b10 con 0x7fea0c01fc20 2023-10-05T19:40:18.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.667+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea14014040 con 0x7fea0c01fc20 2023-10-05T19:40:18.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.667+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d17b10 con 0x555b29d14510 2023-10-05T19:40:18.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.667+0000 7fea19311700 1 -- 172.21.15.12:0/2693008780 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea04012450 con 0x555b29d14510 2023-10-05T19:40:18.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.667+0000 7fea19311700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d17b10 con 0x555b29d0fb90 2023-10-05T19:40:18.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.671+0000 7fea19b12700 1 -- 172.21.15.12:0/2693008780 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea0c025020 con 0x555b29d0fb90 2023-10-05T19:40:18.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.671+0000 7fea19b12700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d17b10 con 0x555b29cdd2e0 2023-10-05T19:40:18.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.671+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea14014040 con 0x555b29cdd2e0 2023-10-05T19:40:18.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.671+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d17b10 con 0x7fea0c01fc20 2023-10-05T19:40:18.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.671+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea14014040 con 0x7fea0c01fc20 2023-10-05T19:40:18.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.671+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d17b10 con 0x555b29d14510 2023-10-05T19:40:18.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.671+0000 7fea19311700 1 -- 172.21.15.12:0/2693008780 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea04012450 con 0x555b29d14510 2023-10-05T19:40:18.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.671+0000 7fea19311700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d17b10 con 0x7fea0c01fc20 2023-10-05T19:40:18.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.671+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea14014040 con 0x7fea0c01fc20 2023-10-05T19:40:18.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.671+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d17b10 con 0x555b29d0fb90 2023-10-05T19:40:18.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.675+0000 7fea19b12700 1 -- 172.21.15.12:0/2693008780 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea0c025020 con 0x555b29d0fb90 2023-10-05T19:40:18.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.675+0000 7fea19b12700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d17b10 con 0x555b29d14510 2023-10-05T19:40:18.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.675+0000 7fea19311700 1 -- 172.21.15.12:0/2693008780 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea04012450 con 0x555b29d14510 2023-10-05T19:40:18.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.675+0000 7fea19311700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d17b10 con 0x7fea0c01fc20 2023-10-05T19:40:18.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.675+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fea14014040 con 0x7fea0c01fc20 2023-10-05T19:40:18.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.675+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d17b10 con 0x555b29cdd2e0 2023-10-05T19:40:18.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.675+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fea14014040 con 0x555b29cdd2e0 2023-10-05T19:40:18.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.675+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d17b10 con 0x555b29d0fb90 2023-10-05T19:40:18.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.679+0000 7fea19b12700 1 -- 172.21.15.12:0/2693008780 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fea0c025020 con 0x555b29d0fb90 2023-10-05T19:40:18.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.679+0000 7fea1bfdde40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:18.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.679+0000 7fea1bfdde40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:18.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.679+0000 7fea1bfdde40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:18.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.679+0000 7fea1bfdde40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:18.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.679+0000 7fea1bfdde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.679+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d17ec0 con 0x555b29cdd2e0 2023-10-05T19:40:18.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.679+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fea14014040 con 0x555b29cdd2e0 2023-10-05T19:40:18.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.679+0000 7fea1bfdde40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:18.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.679+0000 7fea1bfdde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.679+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d18270 con 0x555b29d0fb90 2023-10-05T19:40:18.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.679+0000 7fea19b12700 1 -- 172.21.15.12:0/2693008780 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fea0c025020 con 0x555b29d0fb90 2023-10-05T19:40:18.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.679+0000 7fea1bfdde40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:18.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.679+0000 7fea1bfdde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.679+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d18b10 con 0x555b29cdd2e0 2023-10-05T19:40:18.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.679+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fea14014040 con 0x555b29cdd2e0 2023-10-05T19:40:18.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.679+0000 7fea1bfdde40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:18.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.679+0000 7fea1bfdde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.679+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d18b10 con 0x555b29d14510 2023-10-05T19:40:18.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.679+0000 7fea19311700 1 -- 172.21.15.12:0/2693008780 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fea04012450 con 0x555b29d14510 2023-10-05T19:40:18.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.679+0000 7fea1bfdde40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:18.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.679+0000 7fea1bfdde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.679+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d186a0 con 0x7fea0c01fc20 2023-10-05T19:40:18.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.683+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7fea14014040 con 0x7fea0c01fc20 2023-10-05T19:40:18.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.683+0000 7fea1bfdde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:18.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.683+0000 7fea1bfdde40 20 zone default found 2023-10-05T19:40:18.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.683+0000 7fea1bfdde40 4 Realm: () 2023-10-05T19:40:18.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.683+0000 7fea1bfdde40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:18.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.683+0000 7fea1bfdde40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:18.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.683+0000 7fea1bfdde40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:18.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.683+0000 7fea1bfdde40 20 zonegroup default 2023-10-05T19:40:18.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.683+0000 7fea1bfdde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.683+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d186a0 con 0x7fea0c01fc20 2023-10-05T19:40:18.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.683+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fea14014040 con 0x7fea0c01fc20 2023-10-05T19:40:18.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.683+0000 7fea1bfdde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:18.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.683+0000 7fea1bfdde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.683+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d186a0 con 0x555b29d14510 2023-10-05T19:40:18.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.683+0000 7fea19311700 1 -- 172.21.15.12:0/2693008780 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fea04012450 con 0x555b29d14510 2023-10-05T19:40:18.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.683+0000 7fea1bfdde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:18.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.683+0000 7fea1bfdde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.683+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d1ad30 con 0x555b29d14510 2023-10-05T19:40:18.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.683+0000 7fea19311700 1 -- 172.21.15.12:0/2693008780 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fea04012450 con 0x555b29d14510 2023-10-05T19:40:18.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.683+0000 7fea1bfdde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:18.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.683+0000 7fea1bfdde40 20 started sync module instance, tier type = 2023-10-05T19:40:18.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.683+0000 7fea1bfdde40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:18.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.683+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d1a3f0 con 0x555b29d0fb90 2023-10-05T19:40:18.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.687+0000 7fea19b12700 1 -- 172.21.15.12:0/2693008780 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'30 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fea0c025020 con 0x555b29d0fb90 2023-10-05T19:40:18.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.687+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93850031990768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d1a020 con 0x555b29d0fb90 2023-10-05T19:40:18.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.687+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d1bac0 con 0x555b29cdd2e0 2023-10-05T19:40:18.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.691+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'30 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fea14014040 con 0x555b29cdd2e0 2023-10-05T19:40:18.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.691+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93850031994608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d1c0f0 con 0x555b29cdd2e0 2023-10-05T19:40:18.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.691+0000 7fea19b12700 1 -- 172.21.15.12:0/2693008780 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 93850031990768] v28'31 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fea0c025020 con 0x555b29d0fb90 2023-10-05T19:40:18.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.691+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d1cc50 con 0x555b29d14510 2023-10-05T19:40:18.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.695+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 93850031994608] v28'31 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fea14014040 con 0x555b29cdd2e0 2023-10-05T19:40:18.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.695+0000 7fea19311700 1 -- 172.21.15.12:0/2693008780 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'30 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fea04012450 con 0x555b29d14510 2023-10-05T19:40:18.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.695+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93850031993056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d1bb80 con 0x555b29d14510 2023-10-05T19:40:18.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.695+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d1d540 con 0x555b29d0fb90 2023-10-05T19:40:18.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.695+0000 7fea19b12700 1 -- 172.21.15.12:0/2693008780 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'30 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fea0c025020 con 0x555b29d0fb90 2023-10-05T19:40:18.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.695+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 93850032001104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d1da10 con 0x555b29d0fb90 2023-10-05T19:40:18.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.695+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d1e540 con 0x555b29cdd2e0 2023-10-05T19:40:18.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.699+0000 7fea19311700 1 -- 172.21.15.12:0/2693008780 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 93850031993056] v28'31 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fea04012450 con 0x555b29d14510 2023-10-05T19:40:18.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.699+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'30 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fea14014040 con 0x555b29cdd2e0 2023-10-05T19:40:18.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.699+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 93850031998752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d1e500 con 0x555b29cdd2e0 2023-10-05T19:40:18.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.699+0000 7fea19b12700 1 -- 172.21.15.12:0/2693008780 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 93850032001104] v28'31 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fea0c025020 con 0x555b29d0fb90 2023-10-05T19:40:18.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.699+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d1ea20 con 0x7fea0c01fc20 2023-10-05T19:40:18.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.703+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'30 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fea14014040 con 0x7fea0c01fc20 2023-10-05T19:40:18.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.703+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 93850032002208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d1ef10 con 0x7fea0c01fc20 2023-10-05T19:40:18.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.703+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d1f920 con 0x7fea0c01fc20 2023-10-05T19:40:18.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.703+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 93850031998752] v28'31 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fea14014040 con 0x555b29cdd2e0 2023-10-05T19:40:18.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.707+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(96 notify.6 [create] v28'30 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fea14014040 con 0x7fea0c01fc20 2023-10-05T19:40:18.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.707+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93850032005136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d1f920 con 0x7fea0c01fc20 2023-10-05T19:40:18.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.707+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d20330 con 0x7fea0c01fc20 2023-10-05T19:40:18.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.707+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(95 notify.5 [watch watch cookie 93850032002208] v28'31 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fea14014040 con 0x7fea0c01fc20 2023-10-05T19:40:18.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.707+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(97 notify.6 [watch watch cookie 93850032005136] v28'31 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fea14014040 con 0x7fea0c01fc20 2023-10-05T19:40:18.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.707+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(98 notify.7 [create] v28'30 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fea14014040 con 0x7fea0c01fc20 2023-10-05T19:40:18.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.707+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93850032010288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d1f880 con 0x7fea0c01fc20 2023-10-05T19:40:18.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.707+0000 7fea1bfdde40 20 add_watcher() i=0 2023-10-05T19:40:18.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.707+0000 7fea1bfdde40 20 add_watcher() i=1 2023-10-05T19:40:18.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.707+0000 7fea1bfdde40 20 add_watcher() i=2 2023-10-05T19:40:18.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.707+0000 7fea1bfdde40 20 add_watcher() i=3 2023-10-05T19:40:18.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.707+0000 7fea1bfdde40 20 add_watcher() i=4 2023-10-05T19:40:18.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.707+0000 7fea1bfdde40 20 add_watcher() i=5 2023-10-05T19:40:18.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.707+0000 7fea1bfdde40 20 add_watcher() i=6 2023-10-05T19:40:18.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.711+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 93850032010288] v28'31 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fea14014040 con 0x7fea0c01fc20 2023-10-05T19:40:18.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.711+0000 7fea1bfdde40 20 add_watcher() i=7 2023-10-05T19:40:18.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.711+0000 7fea1bfdde40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:18.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.711+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d1eb40 con 0x555b29cdd2e0 2023-10-05T19:40:18.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.711+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fea14014040 con 0x555b29cdd2e0 2023-10-05T19:40:18.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.711+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93850032012864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d20780 con 0x555b29cdd2e0 2023-10-05T19:40:18.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.715+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 93850032012864] v28'54 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fea14014040 con 0x555b29cdd2e0 2023-10-05T19:40:18.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.715+0000 7fe9eafe5700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:18.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.715+0000 7fea1bfdde40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:18.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.715+0000 7fea1bfdde40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:18.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.715+0000 7fea1bfdde40 1 Processor -- start 2023-10-05T19:40:18.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.715+0000 7fea1bfdde40 1 -- start start 2023-10-05T19:40:18.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.715+0000 7fea1bfdde40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555b29d2d4b0 0x555b29d2d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.715+0000 7fea1bfdde40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b29d2ddc0 con 0x555b29d2d4b0 2023-10-05T19:40:18.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.715+0000 7fea19311700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555b29d2d4b0 0x555b29d2d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.715+0000 7fea19311700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555b29d2d4b0 0x555b29d2d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36754/0 (socket says 172.21.15.12:36754) 2023-10-05T19:40:18.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.715+0000 7fea19311700 1 -- 172.21.15.12:0/1185139370 learned_addr learned my addr 172.21.15.12:0/1185139370 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:18.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.715+0000 7fea19311700 1 -- 172.21.15.12:0/1185139370 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b29d2df00 con 0x555b29d2d4b0 2023-10-05T19:40:18.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.715+0000 7fea19311700 1 --2- 172.21.15.12:0/1185139370 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555b29d2d4b0 0x555b29d2d880 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fea04002000 tx=0x7fea04012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5038f7ffd76108e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.715+0000 7fe9ea7e4700 1 -- 172.21.15.12:0/1185139370 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea04012c20 con 0x555b29d2d4b0 2023-10-05T19:40:18.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.715+0000 7fe9ea7e4700 1 -- 172.21.15.12:0/1185139370 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea04036040 con 0x555b29d2d4b0 2023-10-05T19:40:18.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.715+0000 7fe9ea7e4700 1 -- 172.21.15.12:0/1185139370 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea04030b90 con 0x555b29d2d4b0 2023-10-05T19:40:18.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.719+0000 7fea1bfdde40 1 -- 172.21.15.12:0/1185139370 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555b29d2d4b0 msgr2=0x555b29d2d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.719+0000 7fea1bfdde40 1 --2- 172.21.15.12:0/1185139370 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555b29d2d4b0 0x555b29d2d880 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fea04002000 tx=0x7fea04012000 comp rx=0 tx=0).stop 2023-10-05T19:40:18.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.719+0000 7fea1bfdde40 1 -- 172.21.15.12:0/1185139370 shutdown_connections 2023-10-05T19:40:18.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.719+0000 7fea1bfdde40 1 --2- 172.21.15.12:0/1185139370 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555b29d2d4b0 0x555b29d2d880 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.719+0000 7fea1bfdde40 1 -- 172.21.15.12:0/1185139370 >> 172.21.15.12:0/1185139370 conn(0x555b29d28870 msgr2=0x555b29d26a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:18.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.719+0000 7fea1bfdde40 1 -- 172.21.15.12:0/1185139370 shutdown_connections 2023-10-05T19:40:18.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.719+0000 7fea1bfdde40 1 -- 172.21.15.12:0/1185139370 wait complete. 2023-10-05T19:40:18.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.719+0000 7fea1bfdde40 1 Processor -- start 2023-10-05T19:40:18.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.719+0000 7fea1bfdde40 1 -- start start 2023-10-05T19:40:18.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.719+0000 7fea1bfdde40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555b29d2d4b0 0x555b29d32e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.719+0000 7fea1bfdde40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b29d33e00 con 0x555b29d2d4b0 2023-10-05T19:40:18.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.719+0000 7fea19311700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555b29d2d4b0 0x555b29d32e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.719+0000 7fea19311700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555b29d2d4b0 0x555b29d32e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36768/0 (socket says 172.21.15.12:36768) 2023-10-05T19:40:18.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.719+0000 7fea19311700 1 -- 172.21.15.12:0/4206524209 learned_addr learned my addr 172.21.15.12:0/4206524209 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:18.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.719+0000 7fea19311700 1 -- 172.21.15.12:0/4206524209 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b29cd9910 con 0x555b29d2d4b0 2023-10-05T19:40:18.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.719+0000 7fea19311700 1 --2- 172.21.15.12:0/4206524209 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555b29d2d4b0 0x555b29d32e00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fea04012fd0 tx=0x7fea04037000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.723+0000 7fe9e8fe1700 1 -- 172.21.15.12:0/4206524209 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea0404c400 con 0x555b29d2d4b0 2023-10-05T19:40:18.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.723+0000 7fe9e8fe1700 1 -- 172.21.15.12:0/4206524209 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea04046070 con 0x555b29d2d4b0 2023-10-05T19:40:18.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.723+0000 7fea1bfdde40 1 -- 172.21.15.12:0/4206524209 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555b29d33400 con 0x555b29d2d4b0 2023-10-05T19:40:18.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.723+0000 7fe9e8fe1700 1 -- 172.21.15.12:0/4206524209 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea0404c5c0 con 0x555b29d2d4b0 2023-10-05T19:40:18.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.723+0000 7fea1bfdde40 1 -- 172.21.15.12:0/4206524209 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555b29d337e0 con 0x555b29d2d4b0 2023-10-05T19:40:18.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.723+0000 7fe9e8fe1700 1 -- 172.21.15.12:0/4206524209 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fea0404c720 con 0x555b29d2d4b0 2023-10-05T19:40:18.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.723+0000 7fe9e8fe1700 1 --2- 172.21.15.12:0/4206524209 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe960030c50 0x7fe960033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.723+0000 7fe9e8fe1700 1 -- 172.21.15.12:0/4206524209 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7fea04056ab0 con 0x555b29d2d4b0 2023-10-05T19:40:18.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.723+0000 7fea1a313700 1 --2- 172.21.15.12:0/4206524209 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe960030c50 0x7fe960033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.723+0000 7fea1bfdde40 5 note: GC not initialized 2023-10-05T19:40:18.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.723+0000 7fea1a313700 1 --2- 172.21.15.12:0/4206524209 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe960030c50 0x7fe960033100 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x555b29d1ebf0 tx=0x7fea1400ab80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.727+0000 7fe959ffb700 20 reqs_thread_entry: start 2023-10-05T19:40:18.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.727+0000 7fe94b7fe700 20 reqs_thread_entry: start 2023-10-05T19:40:18.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.727+0000 7fea1bfdde40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:18.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.727+0000 7fea1bfdde40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:18.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.727+0000 7fe949ffb700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:48 2023 2023-10-05T19:40:18.737 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:18.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.727+0000 7fe9497fa700 20 reqs_thread_entry: start 2023-10-05T19:40:18.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.727+0000 7fea1bfdde40 20 get_system_obj_state: rctx=0x7ffe9ec73c88 obj=default.rgw.meta:users.uid:foo state=0x555b29bb30b0 s->prefetch_data=0 2023-10-05T19:40:18.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.727+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d9b6b0 con 0x555b29cdd2e0 2023-10-05T19:40:18.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.727+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(102 foo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7fea14014040 con 0x555b29cdd2e0 2023-10-05T19:40:18.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.727+0000 7fea1bfdde40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:18.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.727+0000 7fea1bfdde40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:18.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.727+0000 7fea1bfdde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.727+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d9b690 con 0x555b29cdd2e0 2023-10-05T19:40:18.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.731+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(103 foo [call,call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv1 ondisk = 0) v8 ==== 273+0+480 (crc 0 0 0) 0x7fea14014040 con 0x555b29cdd2e0 2023-10-05T19:40:18.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.731+0000 7fea1bfdde40 20 rados_obj.operate() r=0 bl.length=357 2023-10-05T19:40:18.739 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:40:18.739 INFO:teuthology.orchestra.run.smithi012.stdout: "user_id": "foo", 2023-10-05T19:40:18.740 INFO:teuthology.orchestra.run.smithi012.stdout: "display_name": "Foo", 2023-10-05T19:40:18.740 INFO:teuthology.orchestra.run.smithi012.stdout: "email": "foo@foo.com", 2023-10-05T19:40:18.740 INFO:teuthology.orchestra.run.smithi012.stdout: "suspended": 0, 2023-10-05T19:40:18.740 INFO:teuthology.orchestra.run.smithi012.stdout: "max_buckets": 4, 2023-10-05T19:40:18.740 INFO:teuthology.orchestra.run.smithi012.stdout: "subusers": [], 2023-10-05T19:40:18.740 INFO:teuthology.orchestra.run.smithi012.stdout: "keys": [ 2023-10-05T19:40:18.740 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:18.740 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:40:18.740 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2023-10-05T19:40:18.740 INFO:teuthology.orchestra.run.smithi012.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2023-10-05T19:40:18.741 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:18.741 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:18.741 INFO:teuthology.orchestra.run.smithi012.stdout: "swift_keys": [], 2023-10-05T19:40:18.741 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": [], 2023-10-05T19:40:18.741 INFO:teuthology.orchestra.run.smithi012.stdout: "op_mask": "read, write, delete", 2023-10-05T19:40:18.741 INFO:teuthology.orchestra.run.smithi012.stdout: "default_placement": "", 2023-10-05T19:40:18.741 INFO:teuthology.orchestra.run.smithi012.stdout: "default_storage_class": "", 2023-10-05T19:40:18.741 INFO:teuthology.orchestra.run.smithi012.stdout: "placement_tags": [], 2023-10-05T19:40:18.741 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket_quota": { 2023-10-05T19:40:18.741 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:18.741 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:18.742 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:18.742 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:18.742 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:18.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:18.742 INFO:teuthology.orchestra.run.smithi012.stdout: "user_quota": { 2023-10-05T19:40:18.742 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:18.742 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:18.742 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:18.742 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:18.742 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:18.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:18.742 INFO:teuthology.orchestra.run.smithi012.stdout: "temp_url_keys": [], 2023-10-05T19:40:18.743 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "rgw", 2023-10-05T19:40:18.743 INFO:teuthology.orchestra.run.smithi012.stdout: "mfa_ids": [] 2023-10-05T19:40:18.743 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:40:18.743 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:40:18.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.731+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:104 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93850031990768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d9b690 con 0x555b29d0fb90 2023-10-05T19:40:18.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.731+0000 7fea19b12700 1 -- 172.21.15.12:0/2693008780 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(104 notify.0 [watch unwatch cookie 93850031990768] v28'32 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fea0c025020 con 0x555b29d0fb90 2023-10-05T19:40:18.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.731+0000 7fea1bfdde40 20 remove_watcher() i=0 2023-10-05T19:40:18.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.731+0000 7fea1bfdde40 2 removed watcher, disabling cache 2023-10-05T19:40:18.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.731+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:105 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93850031994608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d9b690 con 0x555b29cdd2e0 2023-10-05T19:40:18.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.735+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(105 notify.1 [watch unwatch cookie 93850031994608] v28'32 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fea14014040 con 0x555b29cdd2e0 2023-10-05T19:40:18.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.735+0000 7fea1bfdde40 20 remove_watcher() i=1 2023-10-05T19:40:18.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.735+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:106 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93850031993056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d9b690 con 0x555b29d14510 2023-10-05T19:40:18.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.739+0000 7fea19311700 1 -- 172.21.15.12:0/2693008780 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(106 notify.2 [watch unwatch cookie 93850031993056] v28'32 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fea04040020 con 0x555b29d14510 2023-10-05T19:40:18.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.739+0000 7fea1bfdde40 20 remove_watcher() i=2 2023-10-05T19:40:18.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.739+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:107 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93850032001104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d9b690 con 0x555b29d0fb90 2023-10-05T19:40:18.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.739+0000 7fea19b12700 1 -- 172.21.15.12:0/2693008780 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(107 notify.3 [watch unwatch cookie 93850032001104] v28'32 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fea0c025020 con 0x555b29d0fb90 2023-10-05T19:40:18.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.739+0000 7fea1bfdde40 20 remove_watcher() i=3 2023-10-05T19:40:18.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.739+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:108 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93850031998752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d9b690 con 0x555b29cdd2e0 2023-10-05T19:40:18.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.743+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(108 notify.4 [watch unwatch cookie 93850031998752] v28'32 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fea14014040 con 0x555b29cdd2e0 2023-10-05T19:40:18.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.743+0000 7fea1bfdde40 20 remove_watcher() i=4 2023-10-05T19:40:18.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.743+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:109 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93850032002208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d9b690 con 0x7fea0c01fc20 2023-10-05T19:40:18.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.747+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(109 notify.5 [watch unwatch cookie 93850032002208] v28'32 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fea14014040 con 0x7fea0c01fc20 2023-10-05T19:40:18.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.747+0000 7fea1bfdde40 20 remove_watcher() i=5 2023-10-05T19:40:18.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.747+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:110 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93850032005136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d9b690 con 0x7fea0c01fc20 2023-10-05T19:40:18.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.751+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(110 notify.6 [watch unwatch cookie 93850032005136] v28'32 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fea14014040 con 0x7fea0c01fc20 2023-10-05T19:40:18.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.751+0000 7fea1bfdde40 20 remove_watcher() i=6 2023-10-05T19:40:18.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.751+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:111 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93850032010288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d9b690 con 0x7fea0c01fc20 2023-10-05T19:40:18.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.751+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(111 notify.7 [watch unwatch cookie 93850032010288] v28'32 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fea14014040 con 0x7fea0c01fc20 2023-10-05T19:40:18.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.751+0000 7fea1bfdde40 20 remove_watcher() i=7 2023-10-05T19:40:18.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.755+0000 7fe9eafe5700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:18.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.755+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:112 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93850032012864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555b29d9b690 con 0x555b29cdd2e0 2023-10-05T19:40:18.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.759+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(112 data_loggenerations_metadata [watch unwatch cookie 93850032012864] v28'55 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fea14014040 con 0x555b29cdd2e0 2023-10-05T19:40:18.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.759+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fea0c01fc20 msgr2=0x7fea0c01e590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.759+0000 7fea1bfdde40 1 --2- 172.21.15.12:0/2693008780 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fea0c01fc20 0x7fea0c01e590 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.759+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x555b29cdd2e0 msgr2=0x555b29d0da20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.759+0000 7fea1bfdde40 1 --2- 172.21.15.12:0/2693008780 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x555b29cdd2e0 0x555b29d0da20 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.759+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x555b29d0fb90 msgr2=0x555b29d11f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.759+0000 7fea1bfdde40 1 --2- 172.21.15.12:0/2693008780 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x555b29d0fb90 0x555b29d11f70 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.759+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x555b29d14510 msgr2=0x555b29d168f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.759+0000 7fea1bfdde40 1 --2- 172.21.15.12:0/2693008780 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x555b29d14510 0x555b29d168f0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.759+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe9fc030ca0 msgr2=0x7fe9fc033150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.759+0000 7fea1bfdde40 1 --2- 172.21.15.12:0/2693008780 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe9fc030ca0 0x7fe9fc033150 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fea04005a90 tx=0x7fea04015000 comp rx=0 tx=0).stop 2023-10-05T19:40:18.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.759+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555b29cd8d00 msgr2=0x555b29d01f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.759+0000 7fea1a313700 1 -- 172.21.15.12:0/2693008780 reap_dead start 2023-10-05T19:40:18.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.759+0000 7fea1bfdde40 1 --2- 172.21.15.12:0/2693008780 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555b29cd8d00 0x555b29d01f60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fea0c000f80 tx=0x7fea0c014000 comp rx=0 tx=0).stop 2023-10-05T19:40:18.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.759+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 shutdown_connections 2023-10-05T19:40:18.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.759+0000 7fea1bfdde40 1 --2- 172.21.15.12:0/2693008780 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555b29cd8d00 0x555b29d01f60 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.759+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 >> 172.21.15.12:0/2693008780 conn(0x555b29cd4720 msgr2=0x555b29cf24f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:18.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.759+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 shutdown_connections 2023-10-05T19:40:18.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.759+0000 7fea1bfdde40 1 -- 172.21.15.12:0/2693008780 wait complete. 2023-10-05T19:40:18.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.759+0000 7fea1bfdde40 1 -- 172.21.15.12:0/4206524209 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe960030c50 msgr2=0x7fe960033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.759+0000 7fea1bfdde40 1 --2- 172.21.15.12:0/4206524209 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe960030c50 0x7fe960033100 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x555b29d1ebf0 tx=0x7fea1400ab80 comp rx=0 tx=0).stop 2023-10-05T19:40:18.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.759+0000 7fea1bfdde40 1 -- 172.21.15.12:0/4206524209 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555b29d2d4b0 msgr2=0x555b29d32e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.759+0000 7fea1bfdde40 1 --2- 172.21.15.12:0/4206524209 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555b29d2d4b0 0x555b29d32e00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fea04012fd0 tx=0x7fea04037000 comp rx=0 tx=0).stop 2023-10-05T19:40:18.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.759+0000 7fea1bfdde40 1 -- 172.21.15.12:0/4206524209 shutdown_connections 2023-10-05T19:40:18.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.759+0000 7fea1bfdde40 1 --2- 172.21.15.12:0/4206524209 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe960030c50 0x7fe960033100 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.759+0000 7fea1bfdde40 1 --2- 172.21.15.12:0/4206524209 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555b29d2d4b0 0x555b29d32e00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.759+0000 7fea1bfdde40 1 -- 172.21.15.12:0/4206524209 >> 172.21.15.12:0/4206524209 conn(0x555b29d28870 msgr2=0x555b29d26970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:18.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.759+0000 7fea1bfdde40 1 -- 172.21.15.12:0/4206524209 shutdown_connections 2023-10-05T19:40:18.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.759+0000 7fea1bfdde40 1 -- 172.21.15.12:0/4206524209 wait complete. 2023-10-05T19:40:18.778 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:40:18.778 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'suspend', '--uid', 'foo'] 2023-10-05T19:40:18.779 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'suspend', '--uid', 'foo'] 2023-10-05T19:40:18.779 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user suspend --uid foo 2023-10-05T19:40:18.818 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:18.818 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:18.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.811+0000 7efda1eb7e40 1 Processor -- start 2023-10-05T19:40:18.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.811+0000 7efda1eb7e40 1 -- start start 2023-10-05T19:40:18.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.811+0000 7efda1eb7e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x564a31057910 0x564a310ebf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.811+0000 7efda1eb7e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x564a3102fbd0 con 0x564a31058c20 2023-10-05T19:40:18.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.811+0000 7efda1eb7e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x564a30fecee0 con 0x564a31057910 2023-10-05T19:40:18.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.811+0000 7efd9f1eb700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x564a31057910 0x564a310ebf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.811+0000 7efd9f9ec700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x564a31058c20 0x564a31057200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:47234/0 (socket says 172.21.15.12:47234) 2023-10-05T19:40:18.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.811+0000 7efd9f9ec700 1 -- 172.21.15.12:0/1067897711 learned_addr learned my addr 172.21.15.12:0/1067897711 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:18.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.811+0000 7efd9f1eb700 1 -- 172.21.15.12:0/1067897711 >> v1:172.21.15.8:6789/0 conn(0x564a31058c20 legacy=0x564a31057200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.811+0000 7efd9e9ea700 1 -- 172.21.15.12:0/1067897711 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3309464800 0 0) 0x564a3102fbd0 con 0x564a31058c20 2023-10-05T19:40:18.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.811+0000 7efd9f1eb700 1 -- 172.21.15.12:0/1067897711 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a310590b0 con 0x564a31057910 2023-10-05T19:40:18.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.811+0000 7efd9f1eb700 1 --2- 172.21.15.12:0/1067897711 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564a31057910 0x564a310ebf60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efd8800a520 tx=0x7efd88005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d6473d4c6577b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.815+0000 7efd9e9ea700 1 -- 172.21.15.12:0/1067897711 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd8800f590 con 0x564a31057910 2023-10-05T19:40:18.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.815+0000 7efd9e9ea700 1 -- 172.21.15.12:0/1067897711 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd8800f6f0 con 0x564a31057910 2023-10-05T19:40:18.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.815+0000 7efd9e9ea700 1 -- 172.21.15.12:0/1067897711 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd8800f880 con 0x564a31057910 2023-10-05T19:40:18.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.815+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1067897711 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564a31057910 msgr2=0x564a310ebf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.815+0000 7efda1eb7e40 1 --2- 172.21.15.12:0/1067897711 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564a31057910 0x564a310ebf60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efd8800a520 tx=0x7efd88005d40 comp rx=0 tx=0).stop 2023-10-05T19:40:18.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.815+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1067897711 shutdown_connections 2023-10-05T19:40:18.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.815+0000 7efda1eb7e40 1 --2- 172.21.15.12:0/1067897711 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564a31057910 0x564a310ebf60 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.815+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1067897711 >> 172.21.15.12:0/1067897711 conn(0x564a310de850 msgr2=0x564a310e0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:18.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.815+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1067897711 shutdown_connections 2023-10-05T19:40:18.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.815+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1067897711 wait complete. 2023-10-05T19:40:18.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.815+0000 7efda1eb7e40 1 Processor -- start 2023-10-05T19:40:18.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.815+0000 7efda1eb7e40 1 -- start start 2023-10-05T19:40:18.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.815+0000 7efda1eb7e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564a31057910 0x564a31103ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.815+0000 7efda1eb7e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a30fb0e30 con 0x564a31057910 2023-10-05T19:40:18.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.815+0000 7efd9f9ec700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564a31057910 0x564a31103ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.815+0000 7efd9f9ec700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564a31057910 0x564a31103ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36786/0 (socket says 172.21.15.12:36786) 2023-10-05T19:40:18.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.815+0000 7efd9f9ec700 1 -- 172.21.15.12:0/3082584489 learned_addr learned my addr 172.21.15.12:0/3082584489 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:18.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.815+0000 7efd9f9ec700 1 -- 172.21.15.12:0/3082584489 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd8800a0e0 con 0x564a31057910 2023-10-05T19:40:18.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.815+0000 7efd9f9ec700 1 --2- 172.21.15.12:0/3082584489 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564a31057910 0x564a31103ec0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7efd98001dc0 tx=0x7efd98001790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e66562cffae2be97 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.819+0000 7efd9d9e8700 1 -- 172.21.15.12:0/3082584489 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd9800bce0 con 0x564a31057910 2023-10-05T19:40:18.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.819+0000 7efd9d9e8700 1 -- 172.21.15.12:0/3082584489 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd9800be40 con 0x564a31057910 2023-10-05T19:40:18.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.819+0000 7efd9d9e8700 1 -- 172.21.15.12:0/3082584489 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd9800e5d0 con 0x564a31057910 2023-10-05T19:40:18.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.819+0000 7efda1eb7e40 1 -- 172.21.15.12:0/3082584489 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564a31057910 msgr2=0x564a31103ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.819+0000 7efda1eb7e40 1 --2- 172.21.15.12:0/3082584489 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564a31057910 0x564a31103ec0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7efd98001dc0 tx=0x7efd98001790 comp rx=0 tx=0).stop 2023-10-05T19:40:18.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.819+0000 7efda1eb7e40 1 -- 172.21.15.12:0/3082584489 shutdown_connections 2023-10-05T19:40:18.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.819+0000 7efda1eb7e40 1 --2- 172.21.15.12:0/3082584489 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564a31057910 0x564a31103ec0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.819+0000 7efda1eb7e40 1 -- 172.21.15.12:0/3082584489 >> 172.21.15.12:0/3082584489 conn(0x564a310de850 msgr2=0x564a310ffa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:18.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.819+0000 7efda1eb7e40 1 -- 172.21.15.12:0/3082584489 shutdown_connections 2023-10-05T19:40:18.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.819+0000 7efda1eb7e40 1 -- 172.21.15.12:0/3082584489 wait complete. 2023-10-05T19:40:18.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.819+0000 7efda1eb7e40 1 Processor -- start 2023-10-05T19:40:18.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.819+0000 7efda1eb7e40 1 -- start start 2023-10-05T19:40:18.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.819+0000 7efda1eb7e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564a31057910 0x564a3110aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.819+0000 7efda1eb7e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a30f83ed0 con 0x564a31057910 2023-10-05T19:40:18.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.819+0000 7efd9f9ec700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564a31057910 0x564a3110aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.819+0000 7efd9f9ec700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564a31057910 0x564a3110aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36802/0 (socket says 172.21.15.12:36802) 2023-10-05T19:40:18.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.819+0000 7efd9f9ec700 1 -- 172.21.15.12:0/1487343557 learned_addr learned my addr 172.21.15.12:0/1487343557 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:18.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.819+0000 7efd9f9ec700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd98001a70 con 0x564a31057910 2023-10-05T19:40:18.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.819+0000 7efd9f9ec700 1 --2- 172.21.15.12:0/1487343557 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564a31057910 0x564a3110aca0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efd980017d0 tx=0x7efd98016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.819+0000 7efd97fff700 1 -- 172.21.15.12:0/1487343557 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd9801f070 con 0x564a31057910 2023-10-05T19:40:18.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.819+0000 7efd97fff700 1 -- 172.21.15.12:0/1487343557 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd9801a550 con 0x564a31057910 2023-10-05T19:40:18.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.819+0000 7efd97fff700 1 -- 172.21.15.12:0/1487343557 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd9801a840 con 0x564a31057910 2023-10-05T19:40:18.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.819+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564a3110b1e0 con 0x564a31057910 2023-10-05T19:40:18.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.819+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564a3110b620 con 0x564a31057910 2023-10-05T19:40:18.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.823+0000 7efd97fff700 1 -- 172.21.15.12:0/1487343557 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7efd9801a9a0 con 0x564a31057910 2023-10-05T19:40:18.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.823+0000 7efd97fff700 1 --2- 172.21.15.12:0/1487343557 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7efd80030c50 0x7efd80033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.823+0000 7efd97fff700 1 -- 172.21.15.12:0/1487343557 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7efd98044fc0 con 0x564a31057910 2023-10-05T19:40:18.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.823+0000 7efda1eb7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.823+0000 7efd9f1eb700 1 --2- 172.21.15.12:0/1487343557 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7efd80030c50 0x7efd80033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.823+0000 7efda1eb7e40 1 --2- 172.21.15.12:0/1487343557 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x564a31058c20 0x564a31116710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.823+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a31116dc0 con 0x564a31058c20 2023-10-05T19:40:18.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.823+0000 7efda01ed700 1 --2- 172.21.15.12:0/1487343557 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x564a31058c20 0x564a31116710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.823+0000 7efd9f1eb700 1 --2- 172.21.15.12:0/1487343557 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7efd80030c50 0x7efd80033100 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efd88005d80 tx=0x7efd88014000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.823+0000 7efda01ed700 1 --2- 172.21.15.12:0/1487343557 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x564a31058c20 0x564a31116710 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.827+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7efd9000a500 con 0x564a31058c20 2023-10-05T19:40:18.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.827+0000 7efda1eb7e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:18.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.827+0000 7efda1eb7e40 20 realm 2023-10-05T19:40:18.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.827+0000 7efda1eb7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.827+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a31113d70 con 0x564a31058c20 2023-10-05T19:40:18.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.827+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7efd9000a500 con 0x564a31058c20 2023-10-05T19:40:18.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.827+0000 7efda1eb7e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:18.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.827+0000 7efda1eb7e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:18.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.827+0000 7efda1eb7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.827+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a311178a0 con 0x564a31058c20 2023-10-05T19:40:18.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.827+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7efd9000a500 con 0x564a31058c20 2023-10-05T19:40:18.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.827+0000 7efda1eb7e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:18.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.827+0000 7efda1eb7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.827+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a311174b0 con 0x564a31058c20 2023-10-05T19:40:18.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.827+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7efd9000a500 con 0x564a31058c20 2023-10-05T19:40:18.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.827+0000 7efda1eb7e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:18.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.827+0000 7efda1eb7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.827+0000 7efda1eb7e40 1 --2- 172.21.15.12:0/1487343557 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x564a31118c40 0x564a3111b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.827+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a3111b560 con 0x564a31118c40 2023-10-05T19:40:18.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.827+0000 7efd9f9ec700 1 --2- 172.21.15.12:0/1487343557 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x564a31118c40 0x564a3111b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.827+0000 7efd9f9ec700 1 --2- 172.21.15.12:0/1487343557 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x564a31118c40 0x564a3111b020 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.827+0000 7efd9f9ec700 1 -- 172.21.15.12:0/1487343557 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7efd98014070 con 0x564a31118c40 2023-10-05T19:40:18.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.827+0000 7efda1eb7e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:18.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.827+0000 7efda1eb7e40 20 searching for the correct realm 2023-10-05T19:40:18.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.827+0000 7efda1eb7e40 1 --2- 172.21.15.12:0/1487343557 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x564a3111d3e0 0x564a3111f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.827+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a3111fea0 con 0x564a3111d3e0 2023-10-05T19:40:18.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.831+0000 7efd9f1eb700 1 --2- 172.21.15.12:0/1487343557 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x564a3111d3e0 0x564a3111f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.831+0000 7efd9f1eb700 1 --2- 172.21.15.12:0/1487343557 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x564a3111d3e0 0x564a3111f7f0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.831+0000 7efd9f1eb700 1 -- 172.21.15.12:0/1487343557 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7efd8800fd60 con 0x564a3111d3e0 2023-10-05T19:40:18.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.831+0000 7efd9f1eb700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a3111fea0 con 0x564a31118c40 2023-10-05T19:40:18.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.831+0000 7efd9f9ec700 1 -- 172.21.15.12:0/1487343557 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd98014070 con 0x564a31118c40 2023-10-05T19:40:18.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.831+0000 7efd9f9ec700 1 --2- 172.21.15.12:0/1487343557 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7efd9800d750 0x7efd980165b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.831+0000 7efd9f9ec700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a3111fea0 con 0x7efd9800d750 2023-10-05T19:40:18.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.831+0000 7efda01ed700 1 --2- 172.21.15.12:0/1487343557 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7efd9800d750 0x7efd980165b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.835+0000 7efda01ed700 1 --2- 172.21.15.12:0/1487343557 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7efd9800d750 0x7efd980165b0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.835+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd90014040 con 0x7efd9800d750 2023-10-05T19:40:18.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.835+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a3111fea0 con 0x564a31058c20 2023-10-05T19:40:18.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.835+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd90014040 con 0x564a31058c20 2023-10-05T19:40:18.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.835+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a3111fea0 con 0x564a31058c20 2023-10-05T19:40:18.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.835+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd90014040 con 0x564a31058c20 2023-10-05T19:40:18.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.835+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a3111fea0 con 0x7efd9800d750 2023-10-05T19:40:18.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.835+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd90014040 con 0x7efd9800d750 2023-10-05T19:40:18.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.835+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a3111fea0 con 0x7efd9800d750 2023-10-05T19:40:18.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.839+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd90014040 con 0x7efd9800d750 2023-10-05T19:40:18.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.839+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a3111fea0 con 0x564a31118c40 2023-10-05T19:40:18.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.839+0000 7efd9f9ec700 1 -- 172.21.15.12:0/1487343557 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd98014070 con 0x564a31118c40 2023-10-05T19:40:18.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.839+0000 7efd9f9ec700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a3111fea0 con 0x564a3111d3e0 2023-10-05T19:40:18.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.839+0000 7efd9f1eb700 1 -- 172.21.15.12:0/1487343557 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd8800fd60 con 0x564a3111d3e0 2023-10-05T19:40:18.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.839+0000 7efd9f1eb700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a3111fea0 con 0x564a31058c20 2023-10-05T19:40:18.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.839+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd90014040 con 0x564a31058c20 2023-10-05T19:40:18.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.839+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a3111fea0 con 0x7efd9800d750 2023-10-05T19:40:18.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.839+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd90014040 con 0x7efd9800d750 2023-10-05T19:40:18.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.839+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a3111fea0 con 0x564a31058c20 2023-10-05T19:40:18.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.843+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd90014040 con 0x564a31058c20 2023-10-05T19:40:18.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.843+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a3111fea0 con 0x564a31118c40 2023-10-05T19:40:18.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.843+0000 7efd9f9ec700 1 -- 172.21.15.12:0/1487343557 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd98014070 con 0x564a31118c40 2023-10-05T19:40:18.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.843+0000 7efd9f9ec700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a3111fea0 con 0x564a3111d3e0 2023-10-05T19:40:18.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.843+0000 7efd9f1eb700 1 -- 172.21.15.12:0/1487343557 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd8800fd60 con 0x564a3111d3e0 2023-10-05T19:40:18.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.843+0000 7efd9f1eb700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a3111fea0 con 0x564a3111d3e0 2023-10-05T19:40:18.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.843+0000 7efd9f1eb700 1 -- 172.21.15.12:0/1487343557 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd8800fd60 con 0x564a3111d3e0 2023-10-05T19:40:18.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.843+0000 7efd9f1eb700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a3111fea0 con 0x7efd9800d750 2023-10-05T19:40:18.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.843+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd90014040 con 0x7efd9800d750 2023-10-05T19:40:18.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.843+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a3111fea0 con 0x564a3111d3e0 2023-10-05T19:40:18.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.847+0000 7efd9f1eb700 1 -- 172.21.15.12:0/1487343557 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd8800fd60 con 0x564a3111d3e0 2023-10-05T19:40:18.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.847+0000 7efd9f1eb700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a3111fea0 con 0x564a3111d3e0 2023-10-05T19:40:18.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.847+0000 7efd9f1eb700 1 -- 172.21.15.12:0/1487343557 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd8800fd60 con 0x564a3111d3e0 2023-10-05T19:40:18.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.847+0000 7efd9f1eb700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a3111fea0 con 0x564a31058c20 2023-10-05T19:40:18.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.847+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd90014040 con 0x564a31058c20 2023-10-05T19:40:18.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.847+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a3111fea0 con 0x564a31058c20 2023-10-05T19:40:18.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.847+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd90014040 con 0x564a31058c20 2023-10-05T19:40:18.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.847+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a3111fea0 con 0x7efd9800d750 2023-10-05T19:40:18.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.847+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd90014040 con 0x7efd9800d750 2023-10-05T19:40:18.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.847+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a3111fea0 con 0x564a3111d3e0 2023-10-05T19:40:18.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.851+0000 7efd9f1eb700 1 -- 172.21.15.12:0/1487343557 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd8800fd60 con 0x564a3111d3e0 2023-10-05T19:40:18.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.851+0000 7efd9f1eb700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a3111fea0 con 0x564a31118c40 2023-10-05T19:40:18.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.851+0000 7efd9f9ec700 1 -- 172.21.15.12:0/1487343557 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd98014070 con 0x564a31118c40 2023-10-05T19:40:18.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.851+0000 7efd9f9ec700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a3111fea0 con 0x564a31058c20 2023-10-05T19:40:18.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.851+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd90014040 con 0x564a31058c20 2023-10-05T19:40:18.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.851+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a3111fea0 con 0x7efd9800d750 2023-10-05T19:40:18.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.851+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd90014040 con 0x7efd9800d750 2023-10-05T19:40:18.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.851+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a3111fea0 con 0x564a3111d3e0 2023-10-05T19:40:18.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.851+0000 7efd9f1eb700 1 -- 172.21.15.12:0/1487343557 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd8800fd60 con 0x564a3111d3e0 2023-10-05T19:40:18.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.851+0000 7efd9f1eb700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a3111fea0 con 0x7efd9800d750 2023-10-05T19:40:18.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.855+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd90014040 con 0x7efd9800d750 2023-10-05T19:40:18.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.855+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a3111fea0 con 0x564a31118c40 2023-10-05T19:40:18.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.855+0000 7efd9f9ec700 1 -- 172.21.15.12:0/1487343557 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd98014070 con 0x564a31118c40 2023-10-05T19:40:18.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.855+0000 7efd9f9ec700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a3111fea0 con 0x564a3111d3e0 2023-10-05T19:40:18.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.855+0000 7efd9f1eb700 1 -- 172.21.15.12:0/1487343557 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd8800fd60 con 0x564a3111d3e0 2023-10-05T19:40:18.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.855+0000 7efd9f1eb700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a3111fea0 con 0x7efd9800d750 2023-10-05T19:40:18.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.855+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd90014040 con 0x7efd9800d750 2023-10-05T19:40:18.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.855+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a3111fea0 con 0x564a31058c20 2023-10-05T19:40:18.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.859+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7efd90014040 con 0x564a31058c20 2023-10-05T19:40:18.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.859+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a3111fea0 con 0x564a31118c40 2023-10-05T19:40:18.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.859+0000 7efd9f9ec700 1 -- 172.21.15.12:0/1487343557 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7efd98014070 con 0x564a31118c40 2023-10-05T19:40:18.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.859+0000 7efda1eb7e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:18.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.859+0000 7efda1eb7e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:18.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.859+0000 7efda1eb7e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:18.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.859+0000 7efda1eb7e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:18.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.859+0000 7efda1eb7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.859+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a3111c160 con 0x564a31058c20 2023-10-05T19:40:18.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.859+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7efd90014040 con 0x564a31058c20 2023-10-05T19:40:18.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.859+0000 7efda1eb7e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:18.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.859+0000 7efda1eb7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.859+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a3111bd90 con 0x564a31058c20 2023-10-05T19:40:18.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.859+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7efd90014040 con 0x564a31058c20 2023-10-05T19:40:18.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.859+0000 7efda1eb7e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:18.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.859+0000 7efda1eb7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.859+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a31120310 con 0x564a3111d3e0 2023-10-05T19:40:18.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.859+0000 7efd9f1eb700 1 -- 172.21.15.12:0/1487343557 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7efd8800fd60 con 0x564a3111d3e0 2023-10-05T19:40:18.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.859+0000 7efda1eb7e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:18.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.859+0000 7efda1eb7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.859+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a311206c0 con 0x564a3111d3e0 2023-10-05T19:40:18.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.863+0000 7efd9f1eb700 1 -- 172.21.15.12:0/1487343557 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7efd8800fd60 con 0x564a3111d3e0 2023-10-05T19:40:18.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.863+0000 7efda1eb7e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:18.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.863+0000 7efda1eb7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.863+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a31120a70 con 0x564a31058c20 2023-10-05T19:40:18.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.863+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7efd90014040 con 0x564a31058c20 2023-10-05T19:40:18.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.863+0000 7efda1eb7e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:18.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.863+0000 7efda1eb7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.863+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a31120e80 con 0x564a31058c20 2023-10-05T19:40:18.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.863+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7efd90014040 con 0x564a31058c20 2023-10-05T19:40:18.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.863+0000 7efda1eb7e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:18.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.863+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a31120e50 con 0x564a3111d3e0 2023-10-05T19:40:18.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.867+0000 7efd9f1eb700 1 -- 172.21.15.12:0/1487343557 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7efd8800fd60 con 0x564a3111d3e0 2023-10-05T19:40:18.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.867+0000 7efd9f1eb700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a31120e50 con 0x564a31118c40 2023-10-05T19:40:18.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.867+0000 7efd9f9ec700 1 -- 172.21.15.12:0/1487343557 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd98014070 con 0x564a31118c40 2023-10-05T19:40:18.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.867+0000 7efd9f9ec700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a31120e50 con 0x7efd9800d750 2023-10-05T19:40:18.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.867+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd90014040 con 0x7efd9800d750 2023-10-05T19:40:18.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.867+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a31120a70 con 0x564a31058c20 2023-10-05T19:40:18.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.867+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd90014040 con 0x564a31058c20 2023-10-05T19:40:18.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.867+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a31120a70 con 0x564a31058c20 2023-10-05T19:40:18.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.871+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd90014040 con 0x564a31058c20 2023-10-05T19:40:18.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.871+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a31120a70 con 0x7efd9800d750 2023-10-05T19:40:18.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.871+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd90014040 con 0x7efd9800d750 2023-10-05T19:40:18.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.871+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a31120a70 con 0x7efd9800d750 2023-10-05T19:40:18.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.871+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd90014040 con 0x7efd9800d750 2023-10-05T19:40:18.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.871+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a31120a70 con 0x564a31118c40 2023-10-05T19:40:18.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.871+0000 7efd9f9ec700 1 -- 172.21.15.12:0/1487343557 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd98014070 con 0x564a31118c40 2023-10-05T19:40:18.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.871+0000 7efd9f9ec700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a31120a70 con 0x564a3111d3e0 2023-10-05T19:40:18.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.875+0000 7efd9f1eb700 1 -- 172.21.15.12:0/1487343557 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd8800fd60 con 0x564a3111d3e0 2023-10-05T19:40:18.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.875+0000 7efd9f1eb700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7efd88039040 con 0x564a31058c20 2023-10-05T19:40:18.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.875+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd90014040 con 0x564a31058c20 2023-10-05T19:40:18.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.875+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7efd88039040 con 0x7efd9800d750 2023-10-05T19:40:18.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.875+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd90014040 con 0x7efd9800d750 2023-10-05T19:40:18.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.875+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7efd88039040 con 0x564a31058c20 2023-10-05T19:40:18.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.875+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd90014040 con 0x564a31058c20 2023-10-05T19:40:18.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.875+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7efd88039040 con 0x564a31118c40 2023-10-05T19:40:18.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.875+0000 7efd9f9ec700 1 -- 172.21.15.12:0/1487343557 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd98014070 con 0x564a31118c40 2023-10-05T19:40:18.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.875+0000 7efd9f9ec700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7efd88039040 con 0x564a3111d3e0 2023-10-05T19:40:18.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.879+0000 7efd9f1eb700 1 -- 172.21.15.12:0/1487343557 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd8800fd60 con 0x564a3111d3e0 2023-10-05T19:40:18.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.879+0000 7efd9f1eb700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7efd88039040 con 0x564a3111d3e0 2023-10-05T19:40:18.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.879+0000 7efd9f1eb700 1 -- 172.21.15.12:0/1487343557 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd8800fd60 con 0x564a3111d3e0 2023-10-05T19:40:18.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.879+0000 7efd9f1eb700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7efd88039040 con 0x7efd9800d750 2023-10-05T19:40:18.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.879+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd90014040 con 0x7efd9800d750 2023-10-05T19:40:18.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.879+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7efd88039040 con 0x564a3111d3e0 2023-10-05T19:40:18.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.879+0000 7efd9f1eb700 1 -- 172.21.15.12:0/1487343557 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd8800fd60 con 0x564a3111d3e0 2023-10-05T19:40:18.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.879+0000 7efd9f1eb700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7efd88039040 con 0x564a3111d3e0 2023-10-05T19:40:18.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.883+0000 7efd9f1eb700 1 -- 172.21.15.12:0/1487343557 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd8800fd60 con 0x564a3111d3e0 2023-10-05T19:40:18.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.883+0000 7efd9f1eb700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7efd88039040 con 0x564a31058c20 2023-10-05T19:40:18.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.883+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd90014040 con 0x564a31058c20 2023-10-05T19:40:18.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.883+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7efd88039040 con 0x564a31058c20 2023-10-05T19:40:18.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.883+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd90014040 con 0x564a31058c20 2023-10-05T19:40:18.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.883+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7efd88039040 con 0x7efd9800d750 2023-10-05T19:40:18.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.883+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd90014040 con 0x7efd9800d750 2023-10-05T19:40:18.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.883+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7efd88039040 con 0x564a3111d3e0 2023-10-05T19:40:18.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.883+0000 7efd9f1eb700 1 -- 172.21.15.12:0/1487343557 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd8800fd60 con 0x564a3111d3e0 2023-10-05T19:40:18.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.883+0000 7efd9f1eb700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7efd88039040 con 0x564a31118c40 2023-10-05T19:40:18.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.887+0000 7efd9f9ec700 1 -- 172.21.15.12:0/1487343557 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd98014070 con 0x564a31118c40 2023-10-05T19:40:18.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.887+0000 7efd9f9ec700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7efd88039040 con 0x564a31058c20 2023-10-05T19:40:18.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.887+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd90014040 con 0x564a31058c20 2023-10-05T19:40:18.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.887+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7efd88039040 con 0x7efd9800d750 2023-10-05T19:40:18.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.887+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd90014040 con 0x7efd9800d750 2023-10-05T19:40:18.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.887+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7efd88039040 con 0x564a3111d3e0 2023-10-05T19:40:18.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.887+0000 7efd9f1eb700 1 -- 172.21.15.12:0/1487343557 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd8800fd60 con 0x564a3111d3e0 2023-10-05T19:40:18.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.887+0000 7efd9f1eb700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7efd88039040 con 0x7efd9800d750 2023-10-05T19:40:18.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.891+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd90014040 con 0x7efd9800d750 2023-10-05T19:40:18.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.891+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7efd88039040 con 0x564a31118c40 2023-10-05T19:40:18.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.891+0000 7efd9f9ec700 1 -- 172.21.15.12:0/1487343557 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd98014070 con 0x564a31118c40 2023-10-05T19:40:18.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.891+0000 7efd9f9ec700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7efd88039040 con 0x564a3111d3e0 2023-10-05T19:40:18.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.891+0000 7efd9f1eb700 1 -- 172.21.15.12:0/1487343557 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd8800fd60 con 0x564a3111d3e0 2023-10-05T19:40:18.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.891+0000 7efd9f1eb700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7efd88039040 con 0x7efd9800d750 2023-10-05T19:40:18.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.891+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efd90014040 con 0x7efd9800d750 2023-10-05T19:40:18.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.891+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7efd88039040 con 0x564a31058c20 2023-10-05T19:40:18.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.891+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7efd90014040 con 0x564a31058c20 2023-10-05T19:40:18.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.895+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7efd88039040 con 0x564a31118c40 2023-10-05T19:40:18.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.895+0000 7efd9f9ec700 1 -- 172.21.15.12:0/1487343557 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7efd98014070 con 0x564a31118c40 2023-10-05T19:40:18.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.895+0000 7efda1eb7e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:18.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.895+0000 7efda1eb7e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:18.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.895+0000 7efda1eb7e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:18.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.895+0000 7efda1eb7e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:18.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.895+0000 7efda1eb7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.895+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a31120e50 con 0x564a31058c20 2023-10-05T19:40:18.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.895+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7efd90014040 con 0x564a31058c20 2023-10-05T19:40:18.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.895+0000 7efda1eb7e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:18.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.895+0000 7efda1eb7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.895+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a311212b0 con 0x564a31118c40 2023-10-05T19:40:18.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.895+0000 7efd9f9ec700 1 -- 172.21.15.12:0/1487343557 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7efd98014070 con 0x564a31118c40 2023-10-05T19:40:18.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.895+0000 7efda1eb7e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:18.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.895+0000 7efda1eb7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.895+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a31121ba0 con 0x564a31058c20 2023-10-05T19:40:18.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.895+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7efd90014040 con 0x564a31058c20 2023-10-05T19:40:18.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.895+0000 7efda1eb7e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:18.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.895+0000 7efda1eb7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.895+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a31121ba0 con 0x564a3111d3e0 2023-10-05T19:40:18.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.899+0000 7efd9f1eb700 1 -- 172.21.15.12:0/1487343557 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7efd8800fd60 con 0x564a3111d3e0 2023-10-05T19:40:18.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.899+0000 7efda1eb7e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:18.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.899+0000 7efda1eb7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.899+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a31122310 con 0x7efd9800d750 2023-10-05T19:40:18.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.899+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7efd90014040 con 0x7efd9800d750 2023-10-05T19:40:18.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.899+0000 7efda1eb7e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:18.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.899+0000 7efda1eb7e40 20 zone default found 2023-10-05T19:40:18.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.899+0000 7efda1eb7e40 4 Realm: () 2023-10-05T19:40:18.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.899+0000 7efda1eb7e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:18.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.899+0000 7efda1eb7e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:18.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.899+0000 7efda1eb7e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:18.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.899+0000 7efda1eb7e40 20 zonegroup default 2023-10-05T19:40:18.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.899+0000 7efda1eb7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.899+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a31122050 con 0x7efd9800d750 2023-10-05T19:40:18.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.899+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7efd90014040 con 0x7efd9800d750 2023-10-05T19:40:18.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.899+0000 7efda1eb7e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:18.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.899+0000 7efda1eb7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.899+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a311239d0 con 0x564a3111d3e0 2023-10-05T19:40:18.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.899+0000 7efd9f1eb700 1 -- 172.21.15.12:0/1487343557 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7efd8800fd60 con 0x564a3111d3e0 2023-10-05T19:40:18.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.899+0000 7efda1eb7e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:18.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.899+0000 7efda1eb7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.899+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a31123d80 con 0x564a3111d3e0 2023-10-05T19:40:18.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.899+0000 7efd9f1eb700 1 -- 172.21.15.12:0/1487343557 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7efd8800fd60 con 0x564a3111d3e0 2023-10-05T19:40:18.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.899+0000 7efda1eb7e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:18.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.899+0000 7efda1eb7e40 20 started sync module instance, tier type = 2023-10-05T19:40:18.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.899+0000 7efda1eb7e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:18.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.903+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a31123090 con 0x564a31118c40 2023-10-05T19:40:18.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.903+0000 7efd9f9ec700 1 -- 172.21.15.12:0/1487343557 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'33 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efd98014070 con 0x564a31118c40 2023-10-05T19:40:18.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.903+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94876650845328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a311217d0 con 0x564a31118c40 2023-10-05T19:40:18.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.903+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a31124b50 con 0x564a31058c20 2023-10-05T19:40:18.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.907+0000 7efd9f9ec700 1 -- 172.21.15.12:0/1487343557 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94876650845328] v28'34 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efd98014070 con 0x564a31118c40 2023-10-05T19:40:18.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.907+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'33 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efd90014040 con 0x564a31058c20 2023-10-05T19:40:18.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.907+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94876650848560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a31124f50 con 0x564a31058c20 2023-10-05T19:40:18.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.907+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a311257f0 con 0x564a3111d3e0 2023-10-05T19:40:18.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.911+0000 7efd9f1eb700 1 -- 172.21.15.12:0/1487343557 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'33 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efd8800fd60 con 0x564a3111d3e0 2023-10-05T19:40:18.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.911+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94876650852464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a31125bf0 con 0x564a3111d3e0 2023-10-05T19:40:18.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.911+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a31126640 con 0x564a31118c40 2023-10-05T19:40:18.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.911+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94876650848560] v28'34 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efd90014040 con 0x564a31058c20 2023-10-05T19:40:18.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.915+0000 7efd9f9ec700 1 -- 172.21.15.12:0/1487343557 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'33 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efd98014070 con 0x564a31118c40 2023-10-05T19:40:18.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.915+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94876650849664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a31126a40 con 0x564a31118c40 2023-10-05T19:40:18.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.915+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a31127320 con 0x564a31058c20 2023-10-05T19:40:18.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.915+0000 7efd9f1eb700 1 -- 172.21.15.12:0/1487343557 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94876650852464] v28'34 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efd8800fd60 con 0x564a3111d3e0 2023-10-05T19:40:18.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.915+0000 7efd9f9ec700 1 -- 172.21.15.12:0/1487343557 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94876650849664] v28'34 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efd98014070 con 0x564a31118c40 2023-10-05T19:40:18.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.915+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'33 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efd90014040 con 0x564a31058c20 2023-10-05T19:40:18.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.915+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94876650856720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a31126a40 con 0x564a31058c20 2023-10-05T19:40:18.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.915+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a31127c10 con 0x7efd9800d750 2023-10-05T19:40:18.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.919+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'33 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efd90014040 con 0x7efd9800d750 2023-10-05T19:40:18.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.919+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94876650861904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a31126a40 con 0x7efd9800d750 2023-10-05T19:40:18.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.919+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a31128640 con 0x7efd9800d750 2023-10-05T19:40:18.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.919+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94876650856720] v28'34 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efd90014040 con 0x564a31058c20 2023-10-05T19:40:18.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.919+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(95 notify.5 [watch watch cookie 94876650861904] v28'34 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efd90014040 con 0x7efd9800d750 2023-10-05T19:40:18.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.923+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(96 notify.6 [create] v28'33 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efd90014040 con 0x7efd9800d750 2023-10-05T19:40:18.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.923+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94876650863008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a31128110 con 0x7efd9800d750 2023-10-05T19:40:18.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.923+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a31128b20 con 0x7efd9800d750 2023-10-05T19:40:18.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.923+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(98 notify.7 [create] v28'33 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efd90014040 con 0x7efd9800d750 2023-10-05T19:40:18.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.923+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94876650866224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a31128f70 con 0x7efd9800d750 2023-10-05T19:40:18.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.923+0000 7efda1eb7e40 20 add_watcher() i=0 2023-10-05T19:40:18.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.923+0000 7efda1eb7e40 20 add_watcher() i=1 2023-10-05T19:40:18.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.923+0000 7efda1eb7e40 20 add_watcher() i=2 2023-10-05T19:40:18.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.923+0000 7efda1eb7e40 20 add_watcher() i=3 2023-10-05T19:40:18.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.923+0000 7efda1eb7e40 20 add_watcher() i=4 2023-10-05T19:40:18.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.923+0000 7efda1eb7e40 20 add_watcher() i=5 2023-10-05T19:40:18.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.923+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(97 notify.6 [watch watch cookie 94876650863008] v28'34 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efd90014040 con 0x7efd9800d750 2023-10-05T19:40:18.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.923+0000 7efda1eb7e40 20 add_watcher() i=6 2023-10-05T19:40:18.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.927+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94876650866224] v28'34 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efd90014040 con 0x7efd9800d750 2023-10-05T19:40:18.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.927+0000 7efda1eb7e40 20 add_watcher() i=7 2023-10-05T19:40:18.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.927+0000 7efda1eb7e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:18.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.927+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a3112bfa0 con 0x564a31058c20 2023-10-05T19:40:18.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.927+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7efd90014040 con 0x564a31058c20 2023-10-05T19:40:18.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.927+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94876650867328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a311293c0 con 0x564a31058c20 2023-10-05T19:40:18.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.931+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94876650867328] v28'56 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7efd90014040 con 0x564a31058c20 2023-10-05T19:40:18.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.931+0000 7efd717e2700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:18.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.931+0000 7efda1eb7e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:18.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.931+0000 7efda1eb7e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:18.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.931+0000 7efda1eb7e40 1 Processor -- start 2023-10-05T19:40:18.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.931+0000 7efda1eb7e40 1 -- start start 2023-10-05T19:40:18.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.931+0000 7efda1eb7e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564a31136450 0x564a31136820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.931+0000 7efda1eb7e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a31136d60 con 0x564a31136450 2023-10-05T19:40:18.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.931+0000 7efd9f1eb700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564a31136450 0x564a31136820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.931+0000 7efd9f1eb700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564a31136450 0x564a31136820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36810/0 (socket says 172.21.15.12:36810) 2023-10-05T19:40:18.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.931+0000 7efd9f1eb700 1 -- 172.21.15.12:0/2341790395 learned_addr learned my addr 172.21.15.12:0/2341790395 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:18.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.931+0000 7efd9f1eb700 1 -- 172.21.15.12:0/2341790395 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a31136ea0 con 0x564a31136450 2023-10-05T19:40:18.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.931+0000 7efd9f1eb700 1 --2- 172.21.15.12:0/2341790395 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564a31136450 0x564a31136820 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7efd88003000 tx=0x7efd88011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b974d03a627162d server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.931+0000 7efd70fe1700 1 -- 172.21.15.12:0/2341790395 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd8801cd00 con 0x564a31136450 2023-10-05T19:40:18.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.931+0000 7efd70fe1700 1 -- 172.21.15.12:0/2341790395 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd8801ce80 con 0x564a31136450 2023-10-05T19:40:18.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.931+0000 7efd70fe1700 1 -- 172.21.15.12:0/2341790395 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd88029420 con 0x564a31136450 2023-10-05T19:40:18.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.935+0000 7efda1eb7e40 1 -- 172.21.15.12:0/2341790395 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564a31136450 msgr2=0x564a31136820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.935+0000 7efda1eb7e40 1 --2- 172.21.15.12:0/2341790395 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564a31136450 0x564a31136820 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7efd88003000 tx=0x7efd88011000 comp rx=0 tx=0).stop 2023-10-05T19:40:18.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.935+0000 7efda1eb7e40 1 -- 172.21.15.12:0/2341790395 shutdown_connections 2023-10-05T19:40:18.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.935+0000 7efda1eb7e40 1 --2- 172.21.15.12:0/2341790395 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564a31136450 0x564a31136820 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.935+0000 7efda1eb7e40 1 -- 172.21.15.12:0/2341790395 >> 172.21.15.12:0/2341790395 conn(0x564a31131810 msgr2=0x564a3112f960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:18.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.935+0000 7efda1eb7e40 1 -- 172.21.15.12:0/2341790395 shutdown_connections 2023-10-05T19:40:18.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.935+0000 7efda1eb7e40 1 -- 172.21.15.12:0/2341790395 wait complete. 2023-10-05T19:40:18.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.935+0000 7efda1eb7e40 1 Processor -- start 2023-10-05T19:40:18.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.935+0000 7efda1eb7e40 1 -- start start 2023-10-05T19:40:18.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.935+0000 7efda1eb7e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564a31136450 0x564a310e6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.935+0000 7efda1eb7e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a310e3df0 con 0x564a31136450 2023-10-05T19:40:18.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.935+0000 7efd9f1eb700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564a31136450 0x564a310e6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.935+0000 7efd9f1eb700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564a31136450 0x564a310e6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36816/0 (socket says 172.21.15.12:36816) 2023-10-05T19:40:18.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.935+0000 7efd9f1eb700 1 -- 172.21.15.12:0/3421219123 learned_addr learned my addr 172.21.15.12:0/3421219123 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:18.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.935+0000 7efd9f1eb700 1 -- 172.21.15.12:0/3421219123 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd8802e040 con 0x564a31136450 2023-10-05T19:40:18.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.935+0000 7efd9f1eb700 1 --2- 172.21.15.12:0/3421219123 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564a31136450 0x564a310e6310 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7efd88047040 tx=0x7efd88042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.935+0000 7efceaffd700 1 -- 172.21.15.12:0/3421219123 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd88010bc0 con 0x564a31136450 2023-10-05T19:40:18.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.939+0000 7efceaffd700 1 -- 172.21.15.12:0/3421219123 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd88010d20 con 0x564a31136450 2023-10-05T19:40:18.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.939+0000 7efda1eb7e40 1 -- 172.21.15.12:0/3421219123 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564a310e6910 con 0x564a31136450 2023-10-05T19:40:18.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.939+0000 7efceaffd700 1 -- 172.21.15.12:0/3421219123 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd88050450 con 0x564a31136450 2023-10-05T19:40:18.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.939+0000 7efda1eb7e40 1 -- 172.21.15.12:0/3421219123 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564a310e2df0 con 0x564a31136450 2023-10-05T19:40:18.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.939+0000 7efceaffd700 1 -- 172.21.15.12:0/3421219123 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7efd8804e440 con 0x564a31136450 2023-10-05T19:40:18.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.939+0000 7efceaffd700 1 --2- 172.21.15.12:0/3421219123 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7efce0030c00 0x7efce00330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:18.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.939+0000 7efceaffd700 1 -- 172.21.15.12:0/3421219123 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7efd88051c70 con 0x564a31136450 2023-10-05T19:40:18.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.939+0000 7efda1eb7e40 5 note: GC not initialized 2023-10-05T19:40:18.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.939+0000 7efda01ed700 1 --2- 172.21.15.12:0/3421219123 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7efce0030c00 0x7efce00330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:18.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.939+0000 7efda01ed700 1 --2- 172.21.15.12:0/3421219123 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7efce0030c00 0x7efce00330b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x564a31125770 tx=0x7efd9000ab80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:18.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.943+0000 7efcdffff700 20 reqs_thread_entry: start 2023-10-05T19:40:18.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.943+0000 7efcddffb700 20 reqs_thread_entry: start 2023-10-05T19:40:18.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.943+0000 7efda1eb7e40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:18.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.943+0000 7efda1eb7e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:18.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.943+0000 7efccbfff700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:49 2023 2023-10-05T19:40:18.953 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:18.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.943+0000 7efccb7fe700 20 reqs_thread_entry: start 2023-10-05T19:40:18.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.943+0000 7efda1eb7e40 20 get_system_obj_state: rctx=0x7ffe673af4c8 obj=default.rgw.meta:users.uid:foo state=0x564a30fbc0b0 s->prefetch_data=0 2023-10-05T19:40:18.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.943+0000 7efda1eb7e40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2023-10-05T19:40:18.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.943+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a311a8ee0 con 0x564a31058c20 2023-10-05T19:40:18.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.943+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(102 foo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7efd90014040 con 0x564a31058c20 2023-10-05T19:40:18.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.943+0000 7efda1eb7e40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x16 2023-10-05T19:40:18.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.943+0000 7efda1eb7e40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:18.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.943+0000 7efda1eb7e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:18.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.943+0000 7efda1eb7e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:18.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.943+0000 7efda1eb7e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:18.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.943+0000 7efda1eb7e40 10 cache get: name=default.rgw.meta+users.uid+foo : type miss (requested=0x13, cached=0x16) 2023-10-05T19:40:18.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.943+0000 7efda1eb7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:18.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.943+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a311ab970 con 0x564a31058c20 2023-10-05T19:40:18.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.947+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(103 foo [call,call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv1 ondisk = 0) v8 ==== 273+0+480 (crc 0 0 0) 0x7efd90014040 con 0x564a31058c20 2023-10-05T19:40:18.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.947+0000 7efda1eb7e40 20 rados_obj.operate() r=0 bl.length=357 2023-10-05T19:40:18.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.947+0000 7efda1eb7e40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2023-10-05T19:40:18.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.947+0000 7efda1eb7e40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:18.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.947+0000 7efda1eb7e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:18.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.947+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:104 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a311ab970 con 0x564a31118c40 2023-10-05T19:40:18.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.947+0000 7efd9f9ec700 1 -- 172.21.15.12:0/1487343557 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(104 foo.buckets [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 155+0+0 (crc 0 0 0) 0x7efd98014070 con 0x564a31118c40 2023-10-05T19:40:18.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.947+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:105 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.set in=58b,writefull 0~357 in=357b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a311acbf0 con 0x564a31058c20 2023-10-05T19:40:18.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.951+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(105 foo [delete,create,call,writefull 0~357] v28'2 uv2 ondisk = 0) v8 ==== 273+0+0 (crc 0 0 0) 0x7efd90014040 con 0x564a31058c20 2023-10-05T19:40:18.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.951+0000 7efda1eb7e40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2023-10-05T19:40:18.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.951+0000 7efda1eb7e40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:18.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.951+0000 7efda1eb7e40 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2023-10-05T19:40:18.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.951+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:106 7.d 7:b4812045:::notify.4:head [notify cookie 94876651407104 in=545b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a311acbb0 con 0x564a31058c20 2023-10-05T19:40:18.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.951+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== watch-notify(notify (1) cookie 94876650856720 notify 120259084289 ret 0) v3 ==== 575+0+0 (crc 0 0 0) 0x7efd900320a0 con 0x564a31058c20 2023-10-05T19:40:18.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.951+0000 7efd9d1e7700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084289 cookie 94876650856720 notifier 4291 bl.length()=533 2023-10-05T19:40:18.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.951+0000 7efd9d1e7700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2023-10-05T19:40:18.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.951+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(106 notify.4 [notify cookie 94876651407104 out=8b] v0'0 uv33 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7efd90014040 con 0x564a31058c20 2023-10-05T19:40:18.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.951+0000 7efd9d1e7700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:18.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.951+0000 7efd9d1e7700 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:107 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7efd84003d90 con 0x564a31058c20 2023-10-05T19:40:18.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.951+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== osd_op_reply(107 notify.4 [notify-ack] v0'0 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efd90014040 con 0x564a31058c20 2023-10-05T19:40:18.961 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:40:18.961 INFO:teuthology.orchestra.run.smithi012.stdout: "user_id": "foo", 2023-10-05T19:40:18.961 INFO:teuthology.orchestra.run.smithi012.stdout: "display_name": "Foo", 2023-10-05T19:40:18.961 INFO:teuthology.orchestra.run.smithi012.stdout: "email": "foo@foo.com", 2023-10-05T19:40:18.961 INFO:teuthology.orchestra.run.smithi012.stdout: "suspended": 1, 2023-10-05T19:40:18.961 INFO:teuthology.orchestra.run.smithi012.stdout: "max_buckets": 4, 2023-10-05T19:40:18.961 INFO:teuthology.orchestra.run.smithi012.stdout: "subusers": [], 2023-10-05T19:40:18.961 INFO:teuthology.orchestra.run.smithi012.stdout: "keys": [ 2023-10-05T19:40:18.961 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:18.961 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:40:18.962 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2023-10-05T19:40:18.962 INFO:teuthology.orchestra.run.smithi012.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2023-10-05T19:40:18.962 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:18.962 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:18.962 INFO:teuthology.orchestra.run.smithi012.stdout: "swift_keys": [], 2023-10-05T19:40:18.962 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": [], 2023-10-05T19:40:18.962 INFO:teuthology.orchestra.run.smithi012.stdout: "op_mask": "read, write, delete", 2023-10-05T19:40:18.962 INFO:teuthology.orchestra.run.smithi012.stdout: "default_placement": "", 2023-10-05T19:40:18.962 INFO:teuthology.orchestra.run.smithi012.stdout: "default_storage_class": "", 2023-10-05T19:40:18.962 INFO:teuthology.orchestra.run.smithi012.stdout: "placement_tags": [], 2023-10-05T19:40:18.962 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket_quota": { 2023-10-05T19:40:18.963 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:18.963 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:18.963 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:18.963 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:18.963 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:18.963 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:18.963 INFO:teuthology.orchestra.run.smithi012.stdout: "user_quota": { 2023-10-05T19:40:18.963 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:18.963 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:18.963 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:18.963 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:18.964 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:18.964 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:18.964 INFO:teuthology.orchestra.run.smithi012.stdout: "temp_url_keys": [], 2023-10-05T19:40:18.964 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "rgw", 2023-10-05T19:40:18.964 INFO:teuthology.orchestra.run.smithi012.stdout: "mfa_ids": [] 2023-10-05T19:40:18.964 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:40:18.964 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:40:18.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.951+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.1 v2:172.21.15.12:6808/29933 39 ==== watch-notify(notify_complete (2) cookie 94876651407104 notify 120259084289 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7efd90032230 con 0x564a31058c20 2023-10-05T19:40:18.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.951+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:108 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94876650845328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a311acbf0 con 0x564a31118c40 2023-10-05T19:40:18.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.955+0000 7efd9f9ec700 1 -- 172.21.15.12:0/1487343557 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(108 notify.0 [watch unwatch cookie 94876650845328] v28'35 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efd98014070 con 0x564a31118c40 2023-10-05T19:40:18.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.955+0000 7efda1eb7e40 20 remove_watcher() i=0 2023-10-05T19:40:18.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.955+0000 7efda1eb7e40 2 removed watcher, disabling cache 2023-10-05T19:40:18.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.955+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:109 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94876650848560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a311acbf0 con 0x564a31058c20 2023-10-05T19:40:18.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.959+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.1 v2:172.21.15.12:6808/29933 40 ==== osd_op_reply(109 notify.1 [watch unwatch cookie 94876650848560] v28'35 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efd90014040 con 0x564a31058c20 2023-10-05T19:40:18.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.959+0000 7efda1eb7e40 20 remove_watcher() i=1 2023-10-05T19:40:18.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.959+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:110 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94876650852464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a311acbf0 con 0x564a3111d3e0 2023-10-05T19:40:18.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.959+0000 7efd9f1eb700 1 -- 172.21.15.12:0/1487343557 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(110 notify.2 [watch unwatch cookie 94876650852464] v28'35 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efd8804b040 con 0x564a3111d3e0 2023-10-05T19:40:18.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.959+0000 7efda1eb7e40 20 remove_watcher() i=2 2023-10-05T19:40:18.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.959+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:111 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94876650849664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a311acbf0 con 0x564a31118c40 2023-10-05T19:40:18.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.963+0000 7efd9f9ec700 1 -- 172.21.15.12:0/1487343557 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(111 notify.3 [watch unwatch cookie 94876650849664] v28'35 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efd98014070 con 0x564a31118c40 2023-10-05T19:40:18.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.963+0000 7efda1eb7e40 20 remove_watcher() i=3 2023-10-05T19:40:18.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.963+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:112 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94876650856720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a311acbf0 con 0x564a31058c20 2023-10-05T19:40:18.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.967+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.1 v2:172.21.15.12:6808/29933 41 ==== osd_op_reply(112 notify.4 [watch unwatch cookie 94876650856720] v28'35 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efd90014040 con 0x564a31058c20 2023-10-05T19:40:18.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.967+0000 7efda1eb7e40 20 remove_watcher() i=4 2023-10-05T19:40:18.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.967+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:113 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94876650861904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a311acbf0 con 0x7efd9800d750 2023-10-05T19:40:18.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.967+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(113 notify.5 [watch unwatch cookie 94876650861904] v28'35 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efd90014040 con 0x7efd9800d750 2023-10-05T19:40:18.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.967+0000 7efda1eb7e40 20 remove_watcher() i=5 2023-10-05T19:40:18.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.967+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:114 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94876650863008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a311acbf0 con 0x7efd9800d750 2023-10-05T19:40:18.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.971+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(114 notify.6 [watch unwatch cookie 94876650863008] v28'35 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efd90014040 con 0x7efd9800d750 2023-10-05T19:40:18.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.971+0000 7efda1eb7e40 20 remove_watcher() i=6 2023-10-05T19:40:18.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.971+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:115 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94876650866224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a311acbf0 con 0x7efd9800d750 2023-10-05T19:40:18.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.975+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(115 notify.7 [watch unwatch cookie 94876650866224] v28'35 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efd90014040 con 0x7efd9800d750 2023-10-05T19:40:18.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.975+0000 7efda1eb7e40 20 remove_watcher() i=7 2023-10-05T19:40:18.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.975+0000 7efd717e2700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:18.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.975+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:116 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94876650867328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x564a311acbf0 con 0x564a31058c20 2023-10-05T19:40:18.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.979+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 <== osd.1 v2:172.21.15.12:6808/29933 42 ==== osd_op_reply(116 data_loggenerations_metadata [watch unwatch cookie 94876650867328] v28'57 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7efd90014040 con 0x564a31058c20 2023-10-05T19:40:18.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.979+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7efd9800d750 msgr2=0x7efd980165b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.979+0000 7efda1eb7e40 1 --2- 172.21.15.12:0/1487343557 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7efd9800d750 0x7efd980165b0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.979+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x564a31058c20 msgr2=0x564a31116710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.979+0000 7efda1eb7e40 1 --2- 172.21.15.12:0/1487343557 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x564a31058c20 0x564a31116710 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.979+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x564a31118c40 msgr2=0x564a3111b020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.979+0000 7efda1eb7e40 1 --2- 172.21.15.12:0/1487343557 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x564a31118c40 0x564a3111b020 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.979+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x564a3111d3e0 msgr2=0x564a3111f7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.979+0000 7efda1eb7e40 1 --2- 172.21.15.12:0/1487343557 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x564a3111d3e0 0x564a3111f7f0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.979+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7efd80030c50 msgr2=0x7efd80033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.979+0000 7efda1eb7e40 1 --2- 172.21.15.12:0/1487343557 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7efd80030c50 0x7efd80033100 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efd88005d80 tx=0x7efd88014000 comp rx=0 tx=0).stop 2023-10-05T19:40:18.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.979+0000 7efda01ed700 1 -- 172.21.15.12:0/1487343557 reap_dead start 2023-10-05T19:40:18.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.979+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564a31057910 msgr2=0x564a3110aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.979+0000 7efda1eb7e40 1 --2- 172.21.15.12:0/1487343557 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564a31057910 0x564a3110aca0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efd980017d0 tx=0x7efd98016000 comp rx=0 tx=0).stop 2023-10-05T19:40:18.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.979+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 shutdown_connections 2023-10-05T19:40:18.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.979+0000 7efda1eb7e40 1 --2- 172.21.15.12:0/1487343557 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564a31057910 0x564a3110aca0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.979+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 >> 172.21.15.12:0/1487343557 conn(0x564a310de850 msgr2=0x564a310fe500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:18.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.983+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 shutdown_connections 2023-10-05T19:40:18.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.983+0000 7efda1eb7e40 1 -- 172.21.15.12:0/1487343557 wait complete. 2023-10-05T19:40:18.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.983+0000 7efda1eb7e40 1 -- 172.21.15.12:0/3421219123 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7efce0030c00 msgr2=0x7efce00330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.983+0000 7efda1eb7e40 1 --2- 172.21.15.12:0/3421219123 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7efce0030c00 0x7efce00330b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x564a31125770 tx=0x7efd9000ab80 comp rx=0 tx=0).stop 2023-10-05T19:40:18.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.983+0000 7efda1eb7e40 1 -- 172.21.15.12:0/3421219123 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564a31136450 msgr2=0x564a310e6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:18.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.983+0000 7efda1eb7e40 1 --2- 172.21.15.12:0/3421219123 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564a31136450 0x564a310e6310 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7efd88047040 tx=0x7efd88042000 comp rx=0 tx=0).stop 2023-10-05T19:40:18.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.983+0000 7efda1eb7e40 1 -- 172.21.15.12:0/3421219123 shutdown_connections 2023-10-05T19:40:18.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.983+0000 7efda1eb7e40 1 --2- 172.21.15.12:0/3421219123 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7efce0030c00 0x7efce00330b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.983+0000 7efda1eb7e40 1 --2- 172.21.15.12:0/3421219123 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564a31136450 0x564a310e6310 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:18.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.983+0000 7efda1eb7e40 1 -- 172.21.15.12:0/3421219123 >> 172.21.15.12:0/3421219123 conn(0x564a31131810 msgr2=0x564a3112f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:18.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.983+0000 7efda1eb7e40 1 -- 172.21.15.12:0/3421219123 shutdown_connections 2023-10-05T19:40:18.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:18.983+0000 7efda1eb7e40 1 -- 172.21.15.12:0/3421219123 wait complete. 2023-10-05T19:40:18.997 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:40:18.997 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2023-10-05T19:40:18.997 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2023-10-05T19:40:18.997 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2023-10-05T19:40:19.076 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:19.077 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:19.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.067+0000 7fe8508bde40 1 Processor -- start 2023-10-05T19:40:19.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.067+0000 7fe8508bde40 1 -- start start 2023-10-05T19:40:19.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.067+0000 7fe8508bde40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x557dd19a82e0 0x557dd19aa6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:19.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.067+0000 7fe8508bde40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x557dd18f1bd0 con 0x557dd19a3d00 2023-10-05T19:40:19.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.067+0000 7fe8508bde40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x557dd18aeee0 con 0x557dd19a82e0 2023-10-05T19:40:19.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.071+0000 7fe84dbf1700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x557dd19a82e0 0x557dd19aa6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:19.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.071+0000 7fe84e3f2700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x557dd19a3d00 0x557dd19a40d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:47244/0 (socket says 172.21.15.12:47244) 2023-10-05T19:40:19.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.071+0000 7fe84e3f2700 1 -- 172.21.15.12:0/3957464052 learned_addr learned my addr 172.21.15.12:0/3957464052 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:19.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.071+0000 7fe84d3f0700 1 -- 172.21.15.12:0/3957464052 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4008449253 0 0) 0x557dd18f1bd0 con 0x557dd19a3d00 2023-10-05T19:40:19.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.071+0000 7fe84d3f0700 1 -- 172.21.15.12:0/3957464052 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe83c001e10 con 0x557dd19a3d00 2023-10-05T19:40:19.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.071+0000 7fe84dbf1700 1 -- 172.21.15.12:0/3957464052 >> v1:172.21.15.8:6789/0 conn(0x557dd19a3d00 legacy=0x557dd19a40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.071+0000 7fe84dbf1700 1 -- 172.21.15.12:0/3957464052 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557dd19a4910 con 0x557dd19a82e0 2023-10-05T19:40:19.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.071+0000 7fe84dbf1700 1 --2- 172.21.15.12:0/3957464052 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557dd19a82e0 0x557dd19aa6c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe83800a520 tx=0x7fe838005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5afba59d12b681b server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:19.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.071+0000 7fe84d3f0700 1 -- 172.21.15.12:0/3957464052 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe83800f590 con 0x557dd19a82e0 2023-10-05T19:40:19.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.071+0000 7fe84d3f0700 1 -- 172.21.15.12:0/3957464052 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe83800f6f0 con 0x557dd19a82e0 2023-10-05T19:40:19.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.071+0000 7fe84d3f0700 1 -- 172.21.15.12:0/3957464052 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe83800f880 con 0x557dd19a82e0 2023-10-05T19:40:19.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.071+0000 7fe8508bde40 1 -- 172.21.15.12:0/3957464052 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557dd19a82e0 msgr2=0x557dd19aa6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.071+0000 7fe8508bde40 1 --2- 172.21.15.12:0/3957464052 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557dd19a82e0 0x557dd19aa6c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe83800a520 tx=0x7fe838005d40 comp rx=0 tx=0).stop 2023-10-05T19:40:19.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.071+0000 7fe8508bde40 1 -- 172.21.15.12:0/3957464052 shutdown_connections 2023-10-05T19:40:19.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.071+0000 7fe8508bde40 1 --2- 172.21.15.12:0/3957464052 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557dd19a82e0 0x557dd19aa6c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:19.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.071+0000 7fe8508bde40 1 -- 172.21.15.12:0/3957464052 >> 172.21.15.12:0/3957464052 conn(0x557dd199f720 msgr2=0x557dd19a1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:19.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.071+0000 7fe8508bde40 1 -- 172.21.15.12:0/3957464052 shutdown_connections 2023-10-05T19:40:19.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.071+0000 7fe8508bde40 1 -- 172.21.15.12:0/3957464052 wait complete. 2023-10-05T19:40:19.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.071+0000 7fe8508bde40 1 Processor -- start 2023-10-05T19:40:19.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.071+0000 7fe8508bde40 1 -- start start 2023-10-05T19:40:19.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.071+0000 7fe8508bde40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557dd19a3d00 0x557dd19c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:19.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.071+0000 7fe8508bde40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dd1872e30 con 0x557dd19a3d00 2023-10-05T19:40:19.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.075+0000 7fe84e3f2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557dd19a3d00 0x557dd19c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:19.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.075+0000 7fe84e3f2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557dd19a3d00 0x557dd19c8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36836/0 (socket says 172.21.15.12:36836) 2023-10-05T19:40:19.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.075+0000 7fe84e3f2700 1 -- 172.21.15.12:0/4288603394 learned_addr learned my addr 172.21.15.12:0/4288603394 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:19.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.075+0000 7fe84e3f2700 1 -- 172.21.15.12:0/4288603394 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe83800a0e0 con 0x557dd19a3d00 2023-10-05T19:40:19.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.075+0000 7fe84e3f2700 1 --2- 172.21.15.12:0/4288603394 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557dd19a3d00 0x557dd19c8930 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe848001c70 tx=0x7fe848001820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caec469990705582 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:19.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.075+0000 7fe847fff700 1 -- 172.21.15.12:0/4288603394 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe84800bc80 con 0x557dd19a3d00 2023-10-05T19:40:19.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.075+0000 7fe847fff700 1 -- 172.21.15.12:0/4288603394 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe84800be00 con 0x557dd19a3d00 2023-10-05T19:40:19.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.075+0000 7fe847fff700 1 -- 172.21.15.12:0/4288603394 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe84800e5f0 con 0x557dd19a3d00 2023-10-05T19:40:19.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.075+0000 7fe8508bde40 1 -- 172.21.15.12:0/4288603394 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557dd19a3d00 msgr2=0x557dd19c8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.075+0000 7fe8508bde40 1 --2- 172.21.15.12:0/4288603394 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557dd19a3d00 0x557dd19c8930 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe848001c70 tx=0x7fe848001820 comp rx=0 tx=0).stop 2023-10-05T19:40:19.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.075+0000 7fe8508bde40 1 -- 172.21.15.12:0/4288603394 shutdown_connections 2023-10-05T19:40:19.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.075+0000 7fe8508bde40 1 --2- 172.21.15.12:0/4288603394 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557dd19a3d00 0x557dd19c8930 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:19.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.075+0000 7fe8508bde40 1 -- 172.21.15.12:0/4288603394 >> 172.21.15.12:0/4288603394 conn(0x557dd199f720 msgr2=0x557dd1918e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:19.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.075+0000 7fe8508bde40 1 -- 172.21.15.12:0/4288603394 shutdown_connections 2023-10-05T19:40:19.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.075+0000 7fe8508bde40 1 -- 172.21.15.12:0/4288603394 wait complete. 2023-10-05T19:40:19.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.075+0000 7fe8508bde40 1 Processor -- start 2023-10-05T19:40:19.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.075+0000 7fe8508bde40 1 -- start start 2023-10-05T19:40:19.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.075+0000 7fe8508bde40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557dd19a3d00 0x557dd19ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:19.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.075+0000 7fe8508bde40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dd1845ed0 con 0x557dd19a3d00 2023-10-05T19:40:19.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.075+0000 7fe84e3f2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557dd19a3d00 0x557dd19ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:19.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.075+0000 7fe84e3f2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557dd19a3d00 0x557dd19ccdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36840/0 (socket says 172.21.15.12:36840) 2023-10-05T19:40:19.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.075+0000 7fe84e3f2700 1 -- 172.21.15.12:0/2674133507 learned_addr learned my addr 172.21.15.12:0/2674133507 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:19.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.079+0000 7fe84e3f2700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe83c001e10 con 0x557dd19a3d00 2023-10-05T19:40:19.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.079+0000 7fe84e3f2700 1 --2- 172.21.15.12:0/2674133507 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557dd19a3d00 0x557dd19ccdd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe84800b960 tx=0x7fe848016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:19.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.079+0000 7fe8467fc700 1 -- 172.21.15.12:0/2674133507 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe84801f070 con 0x557dd19a3d00 2023-10-05T19:40:19.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.079+0000 7fe8467fc700 1 -- 172.21.15.12:0/2674133507 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe848018640 con 0x557dd19a3d00 2023-10-05T19:40:19.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.079+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557dd19cd310 con 0x557dd19a3d00 2023-10-05T19:40:19.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.079+0000 7fe8467fc700 1 -- 172.21.15.12:0/2674133507 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe8480188f0 con 0x557dd19a3d00 2023-10-05T19:40:19.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.079+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557dd19cd750 con 0x557dd19a3d00 2023-10-05T19:40:19.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.083+0000 7fe8467fc700 1 -- 172.21.15.12:0/2674133507 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fe848018a50 con 0x557dd19a3d00 2023-10-05T19:40:19.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.083+0000 7fe8508bde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.083+0000 7fe8467fc700 1 --2- 172.21.15.12:0/2674133507 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe830030cf0 0x7fe8300331a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:19.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.083+0000 7fe8467fc700 1 -- 172.21.15.12:0/2674133507 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7fe848044e40 con 0x557dd19a3d00 2023-10-05T19:40:19.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.083+0000 7fe84dbf1700 1 --2- 172.21.15.12:0/2674133507 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe830030cf0 0x7fe8300331a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:19.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.083+0000 7fe8508bde40 1 --2- 172.21.15.12:0/2674133507 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x557dd19a82e0 0x557dd19d8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:19.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.083+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19d8ed0 con 0x557dd19a82e0 2023-10-05T19:40:19.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.083+0000 7fe84dbf1700 1 --2- 172.21.15.12:0/2674133507 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe830030cf0 0x7fe8300331a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe838005d80 tx=0x7fe838014000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:19.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.083+0000 7fe84ebf3700 1 --2- 172.21.15.12:0/2674133507 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x557dd19a82e0 0x557dd19d8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:19.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.083+0000 7fe84ebf3700 1 --2- 172.21.15.12:0/2674133507 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x557dd19a82e0 0x557dd19d8860 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:19.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.083+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe84000a500 con 0x557dd19a82e0 2023-10-05T19:40:19.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.083+0000 7fe8508bde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:19.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.083+0000 7fe8508bde40 20 realm 2023-10-05T19:40:19.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.083+0000 7fe8508bde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.083+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19d5ec0 con 0x557dd19a82e0 2023-10-05T19:40:19.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.083+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe84000a500 con 0x557dd19a82e0 2023-10-05T19:40:19.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.083+0000 7fe8508bde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:19.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.083+0000 7fe8508bde40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:19.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.083+0000 7fe8508bde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.083+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19d99b0 con 0x557dd19a82e0 2023-10-05T19:40:19.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.087+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe84000a500 con 0x557dd19a82e0 2023-10-05T19:40:19.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.087+0000 7fe8508bde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:19.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.087+0000 7fe8508bde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.087+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19d9590 con 0x557dd19a82e0 2023-10-05T19:40:19.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.087+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fe84000a500 con 0x557dd19a82e0 2023-10-05T19:40:19.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.087+0000 7fe8508bde40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:19.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.087+0000 7fe8508bde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.087+0000 7fe8508bde40 1 --2- 172.21.15.12:0/2674133507 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x557dd19da940 0x557dd19dcd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:19.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.087+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19dd490 con 0x557dd19da940 2023-10-05T19:40:19.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.087+0000 7fe84e3f2700 1 --2- 172.21.15.12:0/2674133507 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x557dd19da940 0x557dd19dcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:19.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.087+0000 7fe84e3f2700 1 --2- 172.21.15.12:0/2674133507 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x557dd19da940 0x557dd19dcd20 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:19.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.087+0000 7fe84e3f2700 1 -- 172.21.15.12:0/2674133507 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fe848014070 con 0x557dd19da940 2023-10-05T19:40:19.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.087+0000 7fe8508bde40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:19.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.087+0000 7fe8508bde40 20 searching for the correct realm 2023-10-05T19:40:19.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.087+0000 7fe8508bde40 1 --2- 172.21.15.12:0/2674133507 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x557dd19df2c0 0x557dd19e16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:19.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.087+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e1be0 con 0x557dd19df2c0 2023-10-05T19:40:19.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.087+0000 7fe84dbf1700 1 --2- 172.21.15.12:0/2674133507 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x557dd19df2c0 0x557dd19e16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:19.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.087+0000 7fe84dbf1700 1 --2- 172.21.15.12:0/2674133507 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x557dd19df2c0 0x557dd19e16a0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:19.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.091+0000 7fe84dbf1700 1 -- 172.21.15.12:0/2674133507 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fe83800fba0 con 0x557dd19df2c0 2023-10-05T19:40:19.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.091+0000 7fe84dbf1700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e1be0 con 0x557dd19da940 2023-10-05T19:40:19.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.091+0000 7fe84e3f2700 1 -- 172.21.15.12:0/2674133507 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe848014070 con 0x557dd19da940 2023-10-05T19:40:19.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.091+0000 7fe84e3f2700 1 --2- 172.21.15.12:0/2674133507 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fe848023740 0x7fe8480163b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:19.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.091+0000 7fe84e3f2700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e1be0 con 0x7fe848023740 2023-10-05T19:40:19.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.091+0000 7fe84ebf3700 1 --2- 172.21.15.12:0/2674133507 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fe848023740 0x7fe8480163b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:19.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.091+0000 7fe84ebf3700 1 --2- 172.21.15.12:0/2674133507 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fe848023740 0x7fe8480163b0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:19.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.095+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe840014040 con 0x7fe848023740 2023-10-05T19:40:19.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.095+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e1be0 con 0x557dd19a82e0 2023-10-05T19:40:19.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.095+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe840014040 con 0x557dd19a82e0 2023-10-05T19:40:19.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.095+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e1be0 con 0x557dd19a82e0 2023-10-05T19:40:19.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.095+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe840014040 con 0x557dd19a82e0 2023-10-05T19:40:19.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.095+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e1be0 con 0x7fe848023740 2023-10-05T19:40:19.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.095+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe840014040 con 0x7fe848023740 2023-10-05T19:40:19.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.095+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e1be0 con 0x7fe848023740 2023-10-05T19:40:19.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.095+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe840014040 con 0x7fe848023740 2023-10-05T19:40:19.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.095+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e1be0 con 0x557dd19da940 2023-10-05T19:40:19.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.099+0000 7fe84e3f2700 1 -- 172.21.15.12:0/2674133507 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe848014070 con 0x557dd19da940 2023-10-05T19:40:19.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.099+0000 7fe84e3f2700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e1be0 con 0x557dd19df2c0 2023-10-05T19:40:19.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.099+0000 7fe84dbf1700 1 -- 172.21.15.12:0/2674133507 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe83800fba0 con 0x557dd19df2c0 2023-10-05T19:40:19.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.099+0000 7fe84dbf1700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e1be0 con 0x557dd19a82e0 2023-10-05T19:40:19.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.099+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe840014040 con 0x557dd19a82e0 2023-10-05T19:40:19.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.099+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e1be0 con 0x7fe848023740 2023-10-05T19:40:19.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.099+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe840014040 con 0x7fe848023740 2023-10-05T19:40:19.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.099+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e1be0 con 0x557dd19a82e0 2023-10-05T19:40:19.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.099+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe840014040 con 0x557dd19a82e0 2023-10-05T19:40:19.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.099+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e1be0 con 0x557dd19da940 2023-10-05T19:40:19.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.099+0000 7fe84e3f2700 1 -- 172.21.15.12:0/2674133507 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe848014070 con 0x557dd19da940 2023-10-05T19:40:19.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.099+0000 7fe84e3f2700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e1be0 con 0x557dd19df2c0 2023-10-05T19:40:19.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.103+0000 7fe84dbf1700 1 -- 172.21.15.12:0/2674133507 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe83800fba0 con 0x557dd19df2c0 2023-10-05T19:40:19.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.103+0000 7fe84dbf1700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e1be0 con 0x557dd19df2c0 2023-10-05T19:40:19.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.103+0000 7fe84dbf1700 1 -- 172.21.15.12:0/2674133507 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe83800fba0 con 0x557dd19df2c0 2023-10-05T19:40:19.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.103+0000 7fe84dbf1700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e1be0 con 0x7fe848023740 2023-10-05T19:40:19.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.103+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe840014040 con 0x7fe848023740 2023-10-05T19:40:19.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.103+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e1be0 con 0x557dd19df2c0 2023-10-05T19:40:19.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.103+0000 7fe84dbf1700 1 -- 172.21.15.12:0/2674133507 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe83800fba0 con 0x557dd19df2c0 2023-10-05T19:40:19.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.103+0000 7fe84dbf1700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e1be0 con 0x557dd19df2c0 2023-10-05T19:40:19.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.103+0000 7fe84dbf1700 1 -- 172.21.15.12:0/2674133507 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe83800fba0 con 0x557dd19df2c0 2023-10-05T19:40:19.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.103+0000 7fe84dbf1700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e1be0 con 0x557dd19a82e0 2023-10-05T19:40:19.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.107+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe840014040 con 0x557dd19a82e0 2023-10-05T19:40:19.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.107+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e1be0 con 0x557dd19a82e0 2023-10-05T19:40:19.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.107+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe840014040 con 0x557dd19a82e0 2023-10-05T19:40:19.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.107+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e1be0 con 0x7fe848023740 2023-10-05T19:40:19.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.107+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe840014040 con 0x7fe848023740 2023-10-05T19:40:19.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.107+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e1be0 con 0x557dd19df2c0 2023-10-05T19:40:19.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.107+0000 7fe84dbf1700 1 -- 172.21.15.12:0/2674133507 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe83800fba0 con 0x557dd19df2c0 2023-10-05T19:40:19.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.107+0000 7fe84dbf1700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e1be0 con 0x557dd19da940 2023-10-05T19:40:19.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.111+0000 7fe84e3f2700 1 -- 172.21.15.12:0/2674133507 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe848014070 con 0x557dd19da940 2023-10-05T19:40:19.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.111+0000 7fe84e3f2700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e1be0 con 0x557dd19a82e0 2023-10-05T19:40:19.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.111+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe840014040 con 0x557dd19a82e0 2023-10-05T19:40:19.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.111+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e1be0 con 0x7fe848023740 2023-10-05T19:40:19.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.111+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe840014040 con 0x7fe848023740 2023-10-05T19:40:19.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.111+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e1be0 con 0x557dd19df2c0 2023-10-05T19:40:19.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.111+0000 7fe84dbf1700 1 -- 172.21.15.12:0/2674133507 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe83800fba0 con 0x557dd19df2c0 2023-10-05T19:40:19.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.111+0000 7fe84dbf1700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e1be0 con 0x7fe848023740 2023-10-05T19:40:19.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.111+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe840014040 con 0x7fe848023740 2023-10-05T19:40:19.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.111+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e1be0 con 0x557dd19da940 2023-10-05T19:40:19.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.115+0000 7fe84e3f2700 1 -- 172.21.15.12:0/2674133507 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe848014070 con 0x557dd19da940 2023-10-05T19:40:19.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.115+0000 7fe84e3f2700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e1be0 con 0x557dd19df2c0 2023-10-05T19:40:19.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.115+0000 7fe84dbf1700 1 -- 172.21.15.12:0/2674133507 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe83800fba0 con 0x557dd19df2c0 2023-10-05T19:40:19.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.115+0000 7fe84dbf1700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e1be0 con 0x7fe848023740 2023-10-05T19:40:19.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.115+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe840014040 con 0x7fe848023740 2023-10-05T19:40:19.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.115+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e1be0 con 0x557dd19a82e0 2023-10-05T19:40:19.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.115+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fe840014040 con 0x557dd19a82e0 2023-10-05T19:40:19.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.115+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e1be0 con 0x557dd19da940 2023-10-05T19:40:19.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.119+0000 7fe84e3f2700 1 -- 172.21.15.12:0/2674133507 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fe848014070 con 0x557dd19da940 2023-10-05T19:40:19.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.119+0000 7fe8508bde40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:19.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.119+0000 7fe8508bde40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:19.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.119+0000 7fe8508bde40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:19.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.119+0000 7fe8508bde40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:19.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.119+0000 7fe8508bde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.119+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19de010 con 0x557dd19a82e0 2023-10-05T19:40:19.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.119+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe840014040 con 0x557dd19a82e0 2023-10-05T19:40:19.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.119+0000 7fe8508bde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:19.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.119+0000 7fe8508bde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.119+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19ddbe0 con 0x557dd19a82e0 2023-10-05T19:40:19.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.119+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fe840014040 con 0x557dd19a82e0 2023-10-05T19:40:19.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.119+0000 7fe8508bde40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:19.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.119+0000 7fe8508bde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.119+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19de900 con 0x557dd19df2c0 2023-10-05T19:40:19.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.119+0000 7fe84dbf1700 1 -- 172.21.15.12:0/2674133507 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fe83800fba0 con 0x557dd19df2c0 2023-10-05T19:40:19.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.119+0000 7fe8508bde40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:19.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.119+0000 7fe8508bde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.119+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e2510 con 0x557dd19df2c0 2023-10-05T19:40:19.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.119+0000 7fe84dbf1700 1 -- 172.21.15.12:0/2674133507 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7fe83800fba0 con 0x557dd19df2c0 2023-10-05T19:40:19.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.119+0000 7fe8508bde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:19.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.119+0000 7fe8508bde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.119+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e28c0 con 0x557dd19a82e0 2023-10-05T19:40:19.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.119+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe840014040 con 0x557dd19a82e0 2023-10-05T19:40:19.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.119+0000 7fe8508bde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:19.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.119+0000 7fe8508bde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.119+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e2c70 con 0x557dd19a82e0 2023-10-05T19:40:19.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.123+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fe840014040 con 0x557dd19a82e0 2023-10-05T19:40:19.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.123+0000 7fe8508bde40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:19.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.123+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e3c80 con 0x557dd19df2c0 2023-10-05T19:40:19.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.123+0000 7fe84dbf1700 1 -- 172.21.15.12:0/2674133507 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fe83800fba0 con 0x557dd19df2c0 2023-10-05T19:40:19.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.123+0000 7fe84dbf1700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e3c80 con 0x557dd19da940 2023-10-05T19:40:19.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.127+0000 7fe84e3f2700 1 -- 172.21.15.12:0/2674133507 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe848014070 con 0x557dd19da940 2023-10-05T19:40:19.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.127+0000 7fe84e3f2700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e3c80 con 0x7fe848023740 2023-10-05T19:40:19.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.127+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe840014040 con 0x7fe848023740 2023-10-05T19:40:19.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.127+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e28c0 con 0x557dd19a82e0 2023-10-05T19:40:19.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.127+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe840014040 con 0x557dd19a82e0 2023-10-05T19:40:19.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.127+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e28c0 con 0x557dd19a82e0 2023-10-05T19:40:19.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.127+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe840014040 con 0x557dd19a82e0 2023-10-05T19:40:19.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.127+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e28c0 con 0x7fe848023740 2023-10-05T19:40:19.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.127+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe840014040 con 0x7fe848023740 2023-10-05T19:40:19.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.127+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e28c0 con 0x7fe848023740 2023-10-05T19:40:19.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.131+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe840014040 con 0x7fe848023740 2023-10-05T19:40:19.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.131+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e28c0 con 0x557dd19da940 2023-10-05T19:40:19.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.131+0000 7fe84e3f2700 1 -- 172.21.15.12:0/2674133507 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe848014070 con 0x557dd19da940 2023-10-05T19:40:19.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.131+0000 7fe84e3f2700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e28c0 con 0x557dd19df2c0 2023-10-05T19:40:19.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.131+0000 7fe84dbf1700 1 -- 172.21.15.12:0/2674133507 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe83800fba0 con 0x557dd19df2c0 2023-10-05T19:40:19.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.131+0000 7fe84dbf1700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e28c0 con 0x557dd19a82e0 2023-10-05T19:40:19.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.131+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe840014040 con 0x557dd19a82e0 2023-10-05T19:40:19.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.131+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e28c0 con 0x7fe848023740 2023-10-05T19:40:19.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.131+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe840014040 con 0x7fe848023740 2023-10-05T19:40:19.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.131+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e28c0 con 0x557dd19a82e0 2023-10-05T19:40:19.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.135+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe840014040 con 0x557dd19a82e0 2023-10-05T19:40:19.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.135+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e28c0 con 0x557dd19da940 2023-10-05T19:40:19.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.135+0000 7fe84e3f2700 1 -- 172.21.15.12:0/2674133507 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe848014070 con 0x557dd19da940 2023-10-05T19:40:19.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.135+0000 7fe84e3f2700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e28c0 con 0x557dd19df2c0 2023-10-05T19:40:19.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.135+0000 7fe84dbf1700 1 -- 172.21.15.12:0/2674133507 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe83800fba0 con 0x557dd19df2c0 2023-10-05T19:40:19.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.135+0000 7fe84dbf1700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e28c0 con 0x557dd19df2c0 2023-10-05T19:40:19.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.135+0000 7fe84dbf1700 1 -- 172.21.15.12:0/2674133507 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe83800fba0 con 0x557dd19df2c0 2023-10-05T19:40:19.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.135+0000 7fe84dbf1700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e28c0 con 0x7fe848023740 2023-10-05T19:40:19.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.135+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe840014040 con 0x7fe848023740 2023-10-05T19:40:19.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.135+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e28c0 con 0x557dd19df2c0 2023-10-05T19:40:19.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.139+0000 7fe84dbf1700 1 -- 172.21.15.12:0/2674133507 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe83800fba0 con 0x557dd19df2c0 2023-10-05T19:40:19.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.139+0000 7fe84dbf1700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e28c0 con 0x557dd19df2c0 2023-10-05T19:40:19.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.139+0000 7fe84dbf1700 1 -- 172.21.15.12:0/2674133507 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe83800fba0 con 0x557dd19df2c0 2023-10-05T19:40:19.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.139+0000 7fe84dbf1700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e28c0 con 0x557dd19a82e0 2023-10-05T19:40:19.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.139+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe840014040 con 0x557dd19a82e0 2023-10-05T19:40:19.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.139+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e28c0 con 0x557dd19a82e0 2023-10-05T19:40:19.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.139+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe840014040 con 0x557dd19a82e0 2023-10-05T19:40:19.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.139+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e28c0 con 0x7fe848023740 2023-10-05T19:40:19.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.139+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe840014040 con 0x7fe848023740 2023-10-05T19:40:19.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.139+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e28c0 con 0x557dd19df2c0 2023-10-05T19:40:19.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.143+0000 7fe84dbf1700 1 -- 172.21.15.12:0/2674133507 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe83800fba0 con 0x557dd19df2c0 2023-10-05T19:40:19.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.143+0000 7fe84dbf1700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e28c0 con 0x557dd19da940 2023-10-05T19:40:19.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.143+0000 7fe84e3f2700 1 -- 172.21.15.12:0/2674133507 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe848014070 con 0x557dd19da940 2023-10-05T19:40:19.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.143+0000 7fe84e3f2700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e28c0 con 0x557dd19a82e0 2023-10-05T19:40:19.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.143+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe840014040 con 0x557dd19a82e0 2023-10-05T19:40:19.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.143+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e28c0 con 0x7fe848023740 2023-10-05T19:40:19.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.143+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe840014040 con 0x7fe848023740 2023-10-05T19:40:19.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.143+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e28c0 con 0x557dd19df2c0 2023-10-05T19:40:19.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.147+0000 7fe84dbf1700 1 -- 172.21.15.12:0/2674133507 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe83800fba0 con 0x557dd19df2c0 2023-10-05T19:40:19.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.147+0000 7fe84dbf1700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e28c0 con 0x7fe848023740 2023-10-05T19:40:19.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.147+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe840014040 con 0x7fe848023740 2023-10-05T19:40:19.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.147+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e28c0 con 0x557dd19da940 2023-10-05T19:40:19.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.147+0000 7fe84e3f2700 1 -- 172.21.15.12:0/2674133507 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe848014070 con 0x557dd19da940 2023-10-05T19:40:19.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.147+0000 7fe84e3f2700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e28c0 con 0x557dd19df2c0 2023-10-05T19:40:19.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.147+0000 7fe84dbf1700 1 -- 172.21.15.12:0/2674133507 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe83800fba0 con 0x557dd19df2c0 2023-10-05T19:40:19.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.147+0000 7fe84dbf1700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e28c0 con 0x7fe848023740 2023-10-05T19:40:19.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.147+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe840014040 con 0x7fe848023740 2023-10-05T19:40:19.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.147+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e28c0 con 0x557dd19a82e0 2023-10-05T19:40:19.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.151+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fe840014040 con 0x557dd19a82e0 2023-10-05T19:40:19.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.151+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e28c0 con 0x557dd19da940 2023-10-05T19:40:19.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.151+0000 7fe84e3f2700 1 -- 172.21.15.12:0/2674133507 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fe848014070 con 0x557dd19da940 2023-10-05T19:40:19.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.151+0000 7fe8508bde40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:19.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.151+0000 7fe8508bde40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:19.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.151+0000 7fe8508bde40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:19.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.151+0000 7fe8508bde40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:19.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.151+0000 7fe8508bde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.151+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e2c70 con 0x557dd19a82e0 2023-10-05T19:40:19.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.151+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fe840014040 con 0x557dd19a82e0 2023-10-05T19:40:19.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.151+0000 7fe8508bde40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:19.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.151+0000 7fe8508bde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.151+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e3020 con 0x557dd19da940 2023-10-05T19:40:19.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.151+0000 7fe84e3f2700 1 -- 172.21.15.12:0/2674133507 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fe848014070 con 0x557dd19da940 2023-10-05T19:40:19.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.151+0000 7fe8508bde40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:19.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.151+0000 7fe8508bde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.151+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e38c0 con 0x557dd19a82e0 2023-10-05T19:40:19.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.155+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fe840014040 con 0x557dd19a82e0 2023-10-05T19:40:19.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.155+0000 7fe8508bde40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:19.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.155+0000 7fe8508bde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.155+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e38c0 con 0x557dd19df2c0 2023-10-05T19:40:19.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.155+0000 7fe84dbf1700 1 -- 172.21.15.12:0/2674133507 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fe83800fba0 con 0x557dd19df2c0 2023-10-05T19:40:19.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.155+0000 7fe8508bde40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:19.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.155+0000 7fe8508bde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.155+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e3400 con 0x7fe848023740 2023-10-05T19:40:19.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.155+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7fe840014040 con 0x7fe848023740 2023-10-05T19:40:19.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.155+0000 7fe8508bde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:19.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.155+0000 7fe8508bde40 20 zone default found 2023-10-05T19:40:19.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.155+0000 7fe8508bde40 4 Realm: () 2023-10-05T19:40:19.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.155+0000 7fe8508bde40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:19.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.155+0000 7fe8508bde40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:19.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.155+0000 7fe8508bde40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:19.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.155+0000 7fe8508bde40 20 zonegroup default 2023-10-05T19:40:19.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.155+0000 7fe8508bde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.155+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e3400 con 0x7fe848023740 2023-10-05T19:40:19.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.155+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fe840014040 con 0x7fe848023740 2023-10-05T19:40:19.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.155+0000 7fe8508bde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:19.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.155+0000 7fe8508bde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.155+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e3400 con 0x557dd19df2c0 2023-10-05T19:40:19.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.155+0000 7fe84dbf1700 1 -- 172.21.15.12:0/2674133507 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fe83800fba0 con 0x557dd19df2c0 2023-10-05T19:40:19.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.159+0000 7fe8508bde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:19.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.159+0000 7fe8508bde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.159+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e5ae0 con 0x557dd19df2c0 2023-10-05T19:40:19.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.159+0000 7fe84dbf1700 1 -- 172.21.15.12:0/2674133507 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fe83800fba0 con 0x557dd19df2c0 2023-10-05T19:40:19.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.159+0000 7fe8508bde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:19.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.159+0000 7fe8508bde40 20 started sync module instance, tier type = 2023-10-05T19:40:19.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.159+0000 7fe8508bde40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:19.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.159+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e51a0 con 0x557dd19da940 2023-10-05T19:40:19.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.159+0000 7fe84e3f2700 1 -- 172.21.15.12:0/2674133507 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'36 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe848014070 con 0x557dd19da940 2023-10-05T19:40:19.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.159+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93998876085424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e5e90 con 0x557dd19da940 2023-10-05T19:40:19.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.159+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e6730 con 0x557dd19a82e0 2023-10-05T19:40:19.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.163+0000 7fe84e3f2700 1 -- 172.21.15.12:0/2674133507 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 93998876085424] v28'37 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe848014070 con 0x557dd19da940 2023-10-05T19:40:19.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.163+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'36 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe840014040 con 0x557dd19a82e0 2023-10-05T19:40:19.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.163+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93998876090944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e56c0 con 0x557dd19a82e0 2023-10-05T19:40:19.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.163+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e7330 con 0x557dd19df2c0 2023-10-05T19:40:19.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.167+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 93998876090944] v28'37 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe840014040 con 0x557dd19a82e0 2023-10-05T19:40:19.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.167+0000 7fe84dbf1700 1 -- 172.21.15.12:0/2674133507 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'36 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe83800fba0 con 0x557dd19df2c0 2023-10-05T19:40:19.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.167+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93998876092048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e7730 con 0x557dd19df2c0 2023-10-05T19:40:19.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.167+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e7fd0 con 0x557dd19da940 2023-10-05T19:40:19.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.171+0000 7fe84e3f2700 1 -- 172.21.15.12:0/2674133507 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'36 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe848014070 con 0x557dd19da940 2023-10-05T19:40:19.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.171+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 93998876096592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e88c0 con 0x557dd19da940 2023-10-05T19:40:19.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.171+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e92d0 con 0x557dd19a82e0 2023-10-05T19:40:19.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.171+0000 7fe84dbf1700 1 -- 172.21.15.12:0/2674133507 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 93998876092048] v28'37 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe83800fba0 con 0x557dd19df2c0 2023-10-05T19:40:19.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.175+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'36 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe840014040 con 0x557dd19a82e0 2023-10-05T19:40:19.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.175+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 93998876093952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e92d0 con 0x557dd19a82e0 2023-10-05T19:40:19.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.175+0000 7fe84e3f2700 1 -- 172.21.15.12:0/2674133507 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 93998876096592] v28'37 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe848014070 con 0x557dd19da940 2023-10-05T19:40:19.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.175+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e9680 con 0x7fe848023740 2023-10-05T19:40:19.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.175+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 93998876093952] v28'37 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe840014040 con 0x557dd19a82e0 2023-10-05T19:40:19.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.179+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'36 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe840014040 con 0x7fe848023740 2023-10-05T19:40:19.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.179+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 93998876102112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e9720 con 0x7fe848023740 2023-10-05T19:40:19.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.179+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e9fc0 con 0x7fe848023740 2023-10-05T19:40:19.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.179+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(95 notify.5 [watch watch cookie 93998876102112] v28'37 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe840014040 con 0x7fe848023740 2023-10-05T19:40:19.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.179+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(96 notify.6 [create] v28'36 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe840014040 con 0x7fe848023740 2023-10-05T19:40:19.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.179+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93998876103776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e9da0 con 0x7fe848023740 2023-10-05T19:40:19.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.179+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19ea8b0 con 0x7fe848023740 2023-10-05T19:40:19.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.183+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(97 notify.6 [watch watch cookie 93998876103776] v28'37 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe840014040 con 0x7fe848023740 2023-10-05T19:40:19.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.183+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(98 notify.7 [create] v28'36 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe840014040 con 0x7fe848023740 2023-10-05T19:40:19.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.183+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93998876104880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19e9dc0 con 0x7fe848023740 2023-10-05T19:40:19.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.183+0000 7fe8508bde40 20 add_watcher() i=0 2023-10-05T19:40:19.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.183+0000 7fe8508bde40 20 add_watcher() i=1 2023-10-05T19:40:19.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.183+0000 7fe8508bde40 20 add_watcher() i=2 2023-10-05T19:40:19.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.183+0000 7fe8508bde40 20 add_watcher() i=3 2023-10-05T19:40:19.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.183+0000 7fe8508bde40 20 add_watcher() i=4 2023-10-05T19:40:19.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.183+0000 7fe8508bde40 20 add_watcher() i=5 2023-10-05T19:40:19.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.183+0000 7fe8508bde40 20 add_watcher() i=6 2023-10-05T19:40:19.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.187+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 93998876104880] v28'37 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe840014040 con 0x7fe848023740 2023-10-05T19:40:19.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.187+0000 7fe8508bde40 20 add_watcher() i=7 2023-10-05T19:40:19.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.187+0000 7fe8508bde40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:19.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.187+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19ee0e0 con 0x557dd19a82e0 2023-10-05T19:40:19.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.187+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fe840014040 con 0x557dd19a82e0 2023-10-05T19:40:19.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.187+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93998876109760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd19eb500 con 0x557dd19a82e0 2023-10-05T19:40:19.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.191+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 93998876109760] v28'58 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fe840014040 con 0x557dd19a82e0 2023-10-05T19:40:19.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.191+0000 7fe81ffdf700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:19.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.191+0000 7fe8508bde40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:19.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.191+0000 7fe8508bde40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:19.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.191+0000 7fe8508bde40 1 Processor -- start 2023-10-05T19:40:19.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.191+0000 7fe8508bde40 1 -- start start 2023-10-05T19:40:19.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.191+0000 7fe8508bde40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557dd19f8380 0x557dd19f8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:19.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.191+0000 7fe8508bde40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dd19f8c90 con 0x557dd19f8380 2023-10-05T19:40:19.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.191+0000 7fe84dbf1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557dd19f8380 0x557dd19f8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:19.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.191+0000 7fe84dbf1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557dd19f8380 0x557dd19f8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36852/0 (socket says 172.21.15.12:36852) 2023-10-05T19:40:19.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.191+0000 7fe84dbf1700 1 -- 172.21.15.12:0/4042537418 learned_addr learned my addr 172.21.15.12:0/4042537418 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:19.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.191+0000 7fe84dbf1700 1 -- 172.21.15.12:0/4042537418 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557dd19f8dd0 con 0x557dd19f8380 2023-10-05T19:40:19.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.191+0000 7fe84dbf1700 1 --2- 172.21.15.12:0/4042537418 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557dd19f8380 0x557dd19f8750 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x557dd19a4d80 tx=0x7fe838011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a833dcbb29158f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:19.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.191+0000 7fe81f7de700 1 -- 172.21.15.12:0/4042537418 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe83801cb00 con 0x557dd19f8380 2023-10-05T19:40:19.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.191+0000 7fe81f7de700 1 -- 172.21.15.12:0/4042537418 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe83801cc60 con 0x557dd19f8380 2023-10-05T19:40:19.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.191+0000 7fe81f7de700 1 -- 172.21.15.12:0/4042537418 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe83801ce10 con 0x557dd19f8380 2023-10-05T19:40:19.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.195+0000 7fe8508bde40 1 -- 172.21.15.12:0/4042537418 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557dd19f8380 msgr2=0x557dd19f8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.195+0000 7fe8508bde40 1 --2- 172.21.15.12:0/4042537418 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557dd19f8380 0x557dd19f8750 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x557dd19a4d80 tx=0x7fe838011000 comp rx=0 tx=0).stop 2023-10-05T19:40:19.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.195+0000 7fe8508bde40 1 -- 172.21.15.12:0/4042537418 shutdown_connections 2023-10-05T19:40:19.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.195+0000 7fe8508bde40 1 --2- 172.21.15.12:0/4042537418 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557dd19f8380 0x557dd19f8750 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:19.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.195+0000 7fe8508bde40 1 -- 172.21.15.12:0/4042537418 >> 172.21.15.12:0/4042537418 conn(0x557dd19f3740 msgr2=0x557dd19f18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:19.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.195+0000 7fe8508bde40 1 -- 172.21.15.12:0/4042537418 shutdown_connections 2023-10-05T19:40:19.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.195+0000 7fe8508bde40 1 -- 172.21.15.12:0/4042537418 wait complete. 2023-10-05T19:40:19.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.195+0000 7fe8508bde40 1 Processor -- start 2023-10-05T19:40:19.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.195+0000 7fe8508bde40 1 -- start start 2023-10-05T19:40:19.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.195+0000 7fe8508bde40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557dd19f8380 0x557dd19fdef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:19.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.195+0000 7fe8508bde40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dd19feef0 con 0x557dd19f8380 2023-10-05T19:40:19.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.195+0000 7fe84dbf1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557dd19f8380 0x557dd19fdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:19.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.195+0000 7fe84dbf1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557dd19f8380 0x557dd19fdef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36860/0 (socket says 172.21.15.12:36860) 2023-10-05T19:40:19.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.195+0000 7fe84dbf1700 1 -- 172.21.15.12:0/361904986 learned_addr learned my addr 172.21.15.12:0/361904986 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:19.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.195+0000 7fe84dbf1700 1 -- 172.21.15.12:0/361904986 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe83802e040 con 0x557dd19f8380 2023-10-05T19:40:19.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.199+0000 7fe84dbf1700 1 --2- 172.21.15.12:0/361904986 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557dd19f8380 0x557dd19fdef0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x557dd19f2460 tx=0x7fe838044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:19.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.199+0000 7fe81dfdb700 1 -- 172.21.15.12:0/361904986 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe838044b60 con 0x557dd19f8380 2023-10-05T19:40:19.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.199+0000 7fe81dfdb700 1 -- 172.21.15.12:0/361904986 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe838044ce0 con 0x557dd19f8380 2023-10-05T19:40:19.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.199+0000 7fe8508bde40 1 -- 172.21.15.12:0/361904986 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557dd19fe4f0 con 0x557dd19f8380 2023-10-05T19:40:19.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.199+0000 7fe81dfdb700 1 -- 172.21.15.12:0/361904986 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe838044e90 con 0x557dd19f8380 2023-10-05T19:40:19.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.199+0000 7fe8508bde40 1 -- 172.21.15.12:0/361904986 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557dd19fe990 con 0x557dd19f8380 2023-10-05T19:40:19.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.199+0000 7fe81dfdb700 1 -- 172.21.15.12:0/361904986 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fe838010420 con 0x557dd19f8380 2023-10-05T19:40:19.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.199+0000 7fe81dfdb700 1 --2- 172.21.15.12:0/361904986 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe794030bb0 0x7fe794033060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:19.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.199+0000 7fe81dfdb700 1 -- 172.21.15.12:0/361904986 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7fe838058b20 con 0x557dd19f8380 2023-10-05T19:40:19.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.199+0000 7fe84ebf3700 1 --2- 172.21.15.12:0/361904986 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe794030bb0 0x7fe794033060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:19.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.199+0000 7fe8508bde40 5 note: GC not initialized 2023-10-05T19:40:19.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.199+0000 7fe84ebf3700 1 --2- 172.21.15.12:0/361904986 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe794030bb0 0x7fe794033060 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x557dd19ea920 tx=0x7fe84000ab60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:19.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.203+0000 7fe78effd700 20 reqs_thread_entry: start 2023-10-05T19:40:19.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.203+0000 7fe8508bde40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:19.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.203+0000 7fe78cff9700 20 reqs_thread_entry: start 2023-10-05T19:40:19.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.203+0000 7fe8508bde40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:19.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.203+0000 7fe77effd700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:49 2023 2023-10-05T19:40:19.212 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:19.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.203+0000 7fe77e7fc700 20 reqs_thread_entry: start 2023-10-05T19:40:19.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.203+0000 7fe8508bde40 20 get_system_obj_state: rctx=0x7ffccc44c1d8 obj=default.rgw.meta:users.uid:foo state=0x557dd187e0b0 s->prefetch_data=0 2023-10-05T19:40:19.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.203+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd1a66760 con 0x557dd19a82e0 2023-10-05T19:40:19.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.203+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(102 foo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7fe840014040 con 0x557dd19a82e0 2023-10-05T19:40:19.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.203+0000 7fe8508bde40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:19.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.203+0000 7fe8508bde40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:19.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.203+0000 7fe8508bde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.203+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd1a66710 con 0x557dd19a82e0 2023-10-05T19:40:19.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.207+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(103 foo [call,call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv2 ondisk = 0) v8 ==== 273+0+480 (crc 0 0 0) 0x7fe840014040 con 0x557dd19a82e0 2023-10-05T19:40:19.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.207+0000 7fe8508bde40 20 rados_obj.operate() r=0 bl.length=357 2023-10-05T19:40:19.215 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:40:19.215 INFO:teuthology.orchestra.run.smithi012.stdout: "user_id": "foo", 2023-10-05T19:40:19.215 INFO:teuthology.orchestra.run.smithi012.stdout: "display_name": "Foo", 2023-10-05T19:40:19.216 INFO:teuthology.orchestra.run.smithi012.stdout: "email": "foo@foo.com", 2023-10-05T19:40:19.216 INFO:teuthology.orchestra.run.smithi012.stdout: "suspended": 1, 2023-10-05T19:40:19.216 INFO:teuthology.orchestra.run.smithi012.stdout: "max_buckets": 4, 2023-10-05T19:40:19.216 INFO:teuthology.orchestra.run.smithi012.stdout: "subusers": [], 2023-10-05T19:40:19.216 INFO:teuthology.orchestra.run.smithi012.stdout: "keys": [ 2023-10-05T19:40:19.216 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:19.216 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:40:19.216 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2023-10-05T19:40:19.216 INFO:teuthology.orchestra.run.smithi012.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2023-10-05T19:40:19.216 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:19.217 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:19.217 INFO:teuthology.orchestra.run.smithi012.stdout: "swift_keys": [], 2023-10-05T19:40:19.217 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": [], 2023-10-05T19:40:19.217 INFO:teuthology.orchestra.run.smithi012.stdout: "op_mask": "read, write, delete", 2023-10-05T19:40:19.217 INFO:teuthology.orchestra.run.smithi012.stdout: "default_placement": "", 2023-10-05T19:40:19.217 INFO:teuthology.orchestra.run.smithi012.stdout: "default_storage_class": "", 2023-10-05T19:40:19.217 INFO:teuthology.orchestra.run.smithi012.stdout: "placement_tags": [], 2023-10-05T19:40:19.217 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket_quota": { 2023-10-05T19:40:19.217 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:19.217 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:19.218 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:19.218 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:19.218 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:19.218 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:19.218 INFO:teuthology.orchestra.run.smithi012.stdout: "user_quota": { 2023-10-05T19:40:19.218 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:19.218 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:19.218 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:19.218 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:19.218 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:19.219 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:19.219 INFO:teuthology.orchestra.run.smithi012.stdout: "temp_url_keys": [], 2023-10-05T19:40:19.219 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "rgw", 2023-10-05T19:40:19.219 INFO:teuthology.orchestra.run.smithi012.stdout: "mfa_ids": [] 2023-10-05T19:40:19.219 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:40:19.219 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:40:19.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.207+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:104 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93998876085424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd1a66710 con 0x557dd19da940 2023-10-05T19:40:19.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.207+0000 7fe84e3f2700 1 -- 172.21.15.12:0/2674133507 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(104 notify.0 [watch unwatch cookie 93998876085424] v28'38 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe848014070 con 0x557dd19da940 2023-10-05T19:40:19.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.211+0000 7fe8508bde40 20 remove_watcher() i=0 2023-10-05T19:40:19.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.211+0000 7fe8508bde40 2 removed watcher, disabling cache 2023-10-05T19:40:19.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.211+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:105 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93998876090944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd1a66710 con 0x557dd19a82e0 2023-10-05T19:40:19.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.211+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(105 notify.1 [watch unwatch cookie 93998876090944] v28'38 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe840014040 con 0x557dd19a82e0 2023-10-05T19:40:19.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.211+0000 7fe8508bde40 20 remove_watcher() i=1 2023-10-05T19:40:19.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.211+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:106 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93998876092048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd1a66710 con 0x557dd19df2c0 2023-10-05T19:40:19.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.215+0000 7fe84dbf1700 1 -- 172.21.15.12:0/2674133507 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(106 notify.2 [watch unwatch cookie 93998876092048] v28'38 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe83804a020 con 0x557dd19df2c0 2023-10-05T19:40:19.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.215+0000 7fe8508bde40 20 remove_watcher() i=2 2023-10-05T19:40:19.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.215+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:107 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93998876096592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd1a66710 con 0x557dd19da940 2023-10-05T19:40:19.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.219+0000 7fe84e3f2700 1 -- 172.21.15.12:0/2674133507 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(107 notify.3 [watch unwatch cookie 93998876096592] v28'38 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe848014070 con 0x557dd19da940 2023-10-05T19:40:19.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.219+0000 7fe8508bde40 20 remove_watcher() i=3 2023-10-05T19:40:19.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.219+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:108 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93998876093952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd1a66710 con 0x557dd19a82e0 2023-10-05T19:40:19.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.219+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(108 notify.4 [watch unwatch cookie 93998876093952] v28'38 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe840014040 con 0x557dd19a82e0 2023-10-05T19:40:19.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.219+0000 7fe8508bde40 20 remove_watcher() i=4 2023-10-05T19:40:19.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.219+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:109 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93998876102112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd1a66710 con 0x7fe848023740 2023-10-05T19:40:19.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.223+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(109 notify.5 [watch unwatch cookie 93998876102112] v28'38 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe840014040 con 0x7fe848023740 2023-10-05T19:40:19.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.223+0000 7fe8508bde40 20 remove_watcher() i=5 2023-10-05T19:40:19.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.223+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:110 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93998876103776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd1a66710 con 0x7fe848023740 2023-10-05T19:40:19.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.227+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(110 notify.6 [watch unwatch cookie 93998876103776] v28'38 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe840014040 con 0x7fe848023740 2023-10-05T19:40:19.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.227+0000 7fe8508bde40 20 remove_watcher() i=6 2023-10-05T19:40:19.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.227+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:111 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93998876104880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd1a66710 con 0x7fe848023740 2023-10-05T19:40:19.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.231+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(111 notify.7 [watch unwatch cookie 93998876104880] v28'38 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe840014040 con 0x7fe848023740 2023-10-05T19:40:19.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.231+0000 7fe8508bde40 20 remove_watcher() i=7 2023-10-05T19:40:19.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.231+0000 7fe81ffdf700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:19.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.231+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:112 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93998876109760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x557dd1a66710 con 0x557dd19a82e0 2023-10-05T19:40:19.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.235+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(112 data_loggenerations_metadata [watch unwatch cookie 93998876109760] v28'59 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fe840014040 con 0x557dd19a82e0 2023-10-05T19:40:19.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.235+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fe848023740 msgr2=0x7fe8480163b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.235+0000 7fe8508bde40 1 --2- 172.21.15.12:0/2674133507 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fe848023740 0x7fe8480163b0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:19.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.235+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x557dd19a82e0 msgr2=0x557dd19d8860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.235+0000 7fe8508bde40 1 --2- 172.21.15.12:0/2674133507 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x557dd19a82e0 0x557dd19d8860 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:19.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.235+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x557dd19da940 msgr2=0x557dd19dcd20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.235+0000 7fe8508bde40 1 --2- 172.21.15.12:0/2674133507 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x557dd19da940 0x557dd19dcd20 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:19.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.235+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x557dd19df2c0 msgr2=0x557dd19e16a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.235+0000 7fe8508bde40 1 --2- 172.21.15.12:0/2674133507 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x557dd19df2c0 0x557dd19e16a0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:19.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.235+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe830030cf0 msgr2=0x7fe8300331a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.235+0000 7fe8508bde40 1 --2- 172.21.15.12:0/2674133507 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe830030cf0 0x7fe8300331a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe838005d80 tx=0x7fe838014000 comp rx=0 tx=0).stop 2023-10-05T19:40:19.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.235+0000 7fe84ebf3700 1 -- 172.21.15.12:0/2674133507 reap_dead start 2023-10-05T19:40:19.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.235+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557dd19a3d00 msgr2=0x557dd19ccdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.235+0000 7fe8508bde40 1 --2- 172.21.15.12:0/2674133507 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557dd19a3d00 0x557dd19ccdd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe84800b960 tx=0x7fe848016000 comp rx=0 tx=0).stop 2023-10-05T19:40:19.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.235+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 shutdown_connections 2023-10-05T19:40:19.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.235+0000 7fe8508bde40 1 --2- 172.21.15.12:0/2674133507 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557dd19a3d00 0x557dd19ccdd0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:19.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.235+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 >> 172.21.15.12:0/2674133507 conn(0x557dd199f720 msgr2=0x557dd19bd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:19.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.235+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 shutdown_connections 2023-10-05T19:40:19.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.235+0000 7fe8508bde40 1 -- 172.21.15.12:0/2674133507 wait complete. 2023-10-05T19:40:19.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.239+0000 7fe8508bde40 1 -- 172.21.15.12:0/361904986 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe794030bb0 msgr2=0x7fe794033060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.239+0000 7fe8508bde40 1 --2- 172.21.15.12:0/361904986 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe794030bb0 0x7fe794033060 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x557dd19ea920 tx=0x7fe84000ab60 comp rx=0 tx=0).stop 2023-10-05T19:40:19.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.239+0000 7fe8508bde40 1 -- 172.21.15.12:0/361904986 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557dd19f8380 msgr2=0x557dd19fdef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.239+0000 7fe8508bde40 1 --2- 172.21.15.12:0/361904986 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557dd19f8380 0x557dd19fdef0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x557dd19f2460 tx=0x7fe838044000 comp rx=0 tx=0).stop 2023-10-05T19:40:19.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.239+0000 7fe8508bde40 1 -- 172.21.15.12:0/361904986 shutdown_connections 2023-10-05T19:40:19.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.239+0000 7fe8508bde40 1 --2- 172.21.15.12:0/361904986 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe794030bb0 0x7fe794033060 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:19.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.239+0000 7fe8508bde40 1 --2- 172.21.15.12:0/361904986 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x557dd19f8380 0x557dd19fdef0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:19.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.239+0000 7fe8508bde40 1 -- 172.21.15.12:0/361904986 >> 172.21.15.12:0/361904986 conn(0x557dd19f3740 msgr2=0x557dd19f1840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:19.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.239+0000 7fe8508bde40 1 -- 172.21.15.12:0/361904986 shutdown_connections 2023-10-05T19:40:19.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.239+0000 7fe8508bde40 1 -- 172.21.15.12:0/361904986 wait complete. 2023-10-05T19:40:19.253 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:40:19.253 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'enable', '--uid', 'foo'] 2023-10-05T19:40:19.253 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'enable', '--uid', 'foo'] 2023-10-05T19:40:19.254 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user enable --uid foo 2023-10-05T19:40:19.289 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:19.289 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:19.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.283+0000 7f88e7524e40 1 Processor -- start 2023-10-05T19:40:19.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.283+0000 7f88e7524e40 1 -- start start 2023-10-05T19:40:19.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.283+0000 7f88e7524e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55583938c3d0 0x55583941abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:19.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.283+0000 7f88e7524e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x555839364bd0 con 0x55583941b110 2023-10-05T19:40:19.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.283+0000 7f88e7524e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x555839321ee0 con 0x55583938c3d0 2023-10-05T19:40:19.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.283+0000 7f88e4858700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x55583941b110 0x55583941d4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:47256/0 (socket says 172.21.15.12:47256) 2023-10-05T19:40:19.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.283+0000 7f88e4858700 1 -- 172.21.15.12:0/1022993514 learned_addr learned my addr 172.21.15.12:0/1022993514 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:19.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.283+0000 7f88e5059700 1 --2- 172.21.15.12:0/1022993514 >> v2:172.21.15.8:3300/0 conn(0x55583938c3d0 0x55583941abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:19.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.283+0000 7f88dffff700 1 -- 172.21.15.12:0/1022993514 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2261403689 0 0) 0x555839364bd0 con 0x55583941b110 2023-10-05T19:40:19.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.283+0000 7f88dffff700 1 -- 172.21.15.12:0/1022993514 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88d4001e10 con 0x55583941b110 2023-10-05T19:40:19.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.283+0000 7f88e5059700 1 -- 172.21.15.12:0/1022993514 >> v1:172.21.15.8:6789/0 conn(0x55583941b110 legacy=0x55583941d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.283+0000 7f88e5059700 1 -- 172.21.15.12:0/1022993514 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55583941fc20 con 0x55583938c3d0 2023-10-05T19:40:19.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.283+0000 7f88e5059700 1 --2- 172.21.15.12:0/1022993514 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55583938c3d0 0x55583941abd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f88d000a5d0 tx=0x7f88d0005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89f7942a1264ab34 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:19.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.283+0000 7f88dffff700 1 -- 172.21.15.12:0/1022993514 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f88d0007610 con 0x55583938c3d0 2023-10-05T19:40:19.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.283+0000 7f88dffff700 1 -- 172.21.15.12:0/1022993514 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d0007770 con 0x55583938c3d0 2023-10-05T19:40:19.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.283+0000 7f88dffff700 1 -- 172.21.15.12:0/1022993514 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f88d0007a40 con 0x55583938c3d0 2023-10-05T19:40:19.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.283+0000 7f88e7524e40 1 -- 172.21.15.12:0/1022993514 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55583938c3d0 msgr2=0x55583941abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.283+0000 7f88e7524e40 1 --2- 172.21.15.12:0/1022993514 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55583938c3d0 0x55583941abd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f88d000a5d0 tx=0x7f88d0005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:19.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.283+0000 7f88e7524e40 1 -- 172.21.15.12:0/1022993514 shutdown_connections 2023-10-05T19:40:19.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.283+0000 7f88e7524e40 1 --2- 172.21.15.12:0/1022993514 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55583938c3d0 0x55583941abd0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:19.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.283+0000 7f88e7524e40 1 -- 172.21.15.12:0/1022993514 >> 172.21.15.12:0/1022993514 conn(0x555839412720 msgr2=0x555839414b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:19.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.283+0000 7f88e7524e40 1 -- 172.21.15.12:0/1022993514 shutdown_connections 2023-10-05T19:40:19.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.283+0000 7f88e7524e40 1 -- 172.21.15.12:0/1022993514 wait complete. 2023-10-05T19:40:19.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.287+0000 7f88e7524e40 1 Processor -- start 2023-10-05T19:40:19.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.287+0000 7f88e7524e40 1 -- start start 2023-10-05T19:40:19.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.287+0000 7f88e7524e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55583938c3d0 0x555839418150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:19.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.287+0000 7f88e7524e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558392e5e30 con 0x55583938c3d0 2023-10-05T19:40:19.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.287+0000 7f88e5059700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55583938c3d0 0x555839418150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:19.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.287+0000 7f88e5059700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55583938c3d0 0x555839418150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36880/0 (socket says 172.21.15.12:36880) 2023-10-05T19:40:19.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.287+0000 7f88e5059700 1 -- 172.21.15.12:0/3039375273 learned_addr learned my addr 172.21.15.12:0/3039375273 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:19.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.287+0000 7f88e5059700 1 -- 172.21.15.12:0/3039375273 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d0005cf0 con 0x55583938c3d0 2023-10-05T19:40:19.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.287+0000 7f88e5059700 1 --2- 172.21.15.12:0/3039375273 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55583938c3d0 0x555839418150 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f88d0000c00 tx=0x7f88d0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee10e61376a7e6c server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:19.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.287+0000 7f88deffd700 1 -- 172.21.15.12:0/3039375273 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f88d00078e0 con 0x55583938c3d0 2023-10-05T19:40:19.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.287+0000 7f88deffd700 1 -- 172.21.15.12:0/3039375273 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d0007a40 con 0x55583938c3d0 2023-10-05T19:40:19.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.287+0000 7f88deffd700 1 -- 172.21.15.12:0/3039375273 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f88d0007bd0 con 0x55583938c3d0 2023-10-05T19:40:19.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.287+0000 7f88e7524e40 1 -- 172.21.15.12:0/3039375273 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55583938c3d0 msgr2=0x555839418150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.287+0000 7f88e7524e40 1 --2- 172.21.15.12:0/3039375273 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55583938c3d0 0x555839418150 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f88d0000c00 tx=0x7f88d0014000 comp rx=0 tx=0).stop 2023-10-05T19:40:19.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.287+0000 7f88e7524e40 1 -- 172.21.15.12:0/3039375273 shutdown_connections 2023-10-05T19:40:19.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.287+0000 7f88e7524e40 1 --2- 172.21.15.12:0/3039375273 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55583938c3d0 0x555839418150 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:19.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.287+0000 7f88e7524e40 1 -- 172.21.15.12:0/3039375273 >> 172.21.15.12:0/3039375273 conn(0x555839412720 msgr2=0x555839432900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:19.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.287+0000 7f88e7524e40 1 -- 172.21.15.12:0/3039375273 shutdown_connections 2023-10-05T19:40:19.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.287+0000 7f88e7524e40 1 -- 172.21.15.12:0/3039375273 wait complete. 2023-10-05T19:40:19.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.287+0000 7f88e7524e40 1 Processor -- start 2023-10-05T19:40:19.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.287+0000 7f88e7524e40 1 -- start start 2023-10-05T19:40:19.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.287+0000 7f88e7524e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55583938c3d0 0x55583943fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:19.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.287+0000 7f88e7524e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558392b8ed0 con 0x55583938c3d0 2023-10-05T19:40:19.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.291+0000 7f88e5059700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55583938c3d0 0x55583943fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:19.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.291+0000 7f88e5059700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55583938c3d0 0x55583943fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36882/0 (socket says 172.21.15.12:36882) 2023-10-05T19:40:19.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.291+0000 7f88e5059700 1 -- 172.21.15.12:0/343524609 learned_addr learned my addr 172.21.15.12:0/343524609 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:19.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.291+0000 7f88e5059700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55583941fc20 con 0x55583938c3d0 2023-10-05T19:40:19.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.291+0000 7f88e5059700 1 --2- 172.21.15.12:0/343524609 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55583938c3d0 0x55583943fc10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f88d001f040 tx=0x7f88d001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:19.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.291+0000 7f88dd7fa700 1 -- 172.21.15.12:0/343524609 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f88d00078e0 con 0x55583938c3d0 2023-10-05T19:40:19.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.291+0000 7f88dd7fa700 1 -- 172.21.15.12:0/343524609 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d0026650 con 0x55583938c3d0 2023-10-05T19:40:19.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.291+0000 7f88dd7fa700 1 -- 172.21.15.12:0/343524609 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f88d0026820 con 0x55583938c3d0 2023-10-05T19:40:19.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.291+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555839440150 con 0x55583938c3d0 2023-10-05T19:40:19.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.291+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555839440590 con 0x55583938c3d0 2023-10-05T19:40:19.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.291+0000 7f88dd7fa700 1 -- 172.21.15.12:0/343524609 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f88d0026980 con 0x55583938c3d0 2023-10-05T19:40:19.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.291+0000 7f88dd7fa700 1 --2- 172.21.15.12:0/343524609 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f88c8030c90 0x7f88c8033140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:19.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.291+0000 7f88dd7fa700 1 -- 172.21.15.12:0/343524609 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f88d0046930 con 0x55583938c3d0 2023-10-05T19:40:19.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.291+0000 7f88e4858700 1 --2- 172.21.15.12:0/343524609 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f88c8030c90 0x7f88c8033140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:19.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.295+0000 7f88e7524e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.295+0000 7f88e7524e40 1 --2- 172.21.15.12:0/343524609 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55583941b110 0x55583944b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:19.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.295+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55583944bd40 con 0x55583941b110 2023-10-05T19:40:19.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.295+0000 7f88e585a700 1 --2- 172.21.15.12:0/343524609 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55583941b110 0x55583944b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:19.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.295+0000 7f88e4858700 1 --2- 172.21.15.12:0/343524609 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f88c8030c90 0x7f88c8033140 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f88d80044f0 tx=0x7f88d80065f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:19.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.295+0000 7f88e585a700 1 --2- 172.21.15.12:0/343524609 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55583941b110 0x55583944b610 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:19.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.295+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f88e000a500 con 0x55583941b110 2023-10-05T19:40:19.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.295+0000 7f88e7524e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:19.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.295+0000 7f88e7524e40 20 realm 2023-10-05T19:40:19.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.295+0000 7f88e7524e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.295+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839448c40 con 0x55583941b110 2023-10-05T19:40:19.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.295+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f88e000a500 con 0x55583941b110 2023-10-05T19:40:19.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.295+0000 7f88e7524e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:19.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.295+0000 7f88e7524e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:19.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.295+0000 7f88e7524e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.295+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55583944c870 con 0x55583941b110 2023-10-05T19:40:19.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.295+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f88e000a500 con 0x55583941b110 2023-10-05T19:40:19.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.295+0000 7f88e7524e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:19.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.295+0000 7f88e7524e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.295+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55583944c480 con 0x55583941b110 2023-10-05T19:40:19.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.295+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f88e000a500 con 0x55583941b110 2023-10-05T19:40:19.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.295+0000 7f88e7524e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:19.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.295+0000 7f88e7524e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.295+0000 7f88e7524e40 1 --2- 172.21.15.12:0/343524609 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55583944dc10 0x55583944fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:19.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.295+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839450530 con 0x55583944dc10 2023-10-05T19:40:19.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.295+0000 7f88e5059700 1 --2- 172.21.15.12:0/343524609 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55583944dc10 0x55583944fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:19.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.295+0000 7f88e5059700 1 --2- 172.21.15.12:0/343524609 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55583944dc10 0x55583944fff0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:19.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.299+0000 7f88e5059700 1 -- 172.21.15.12:0/343524609 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f88d00129c0 con 0x55583944dc10 2023-10-05T19:40:19.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.299+0000 7f88e7524e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:19.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.299+0000 7f88e7524e40 20 searching for the correct realm 2023-10-05T19:40:19.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.299+0000 7f88e7524e40 1 --2- 172.21.15.12:0/343524609 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5558394523b0 0x5558394547c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:19.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.299+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839454e70 con 0x5558394523b0 2023-10-05T19:40:19.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.299+0000 7f88e4858700 1 --2- 172.21.15.12:0/343524609 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5558394523b0 0x5558394547c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:19.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.299+0000 7f88e4858700 1 --2- 172.21.15.12:0/343524609 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5558394523b0 0x5558394547c0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:19.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.299+0000 7f88e4858700 1 -- 172.21.15.12:0/343524609 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f88d8004520 con 0x5558394523b0 2023-10-05T19:40:19.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.299+0000 7f88e4858700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839454e70 con 0x55583944dc10 2023-10-05T19:40:19.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.303+0000 7f88e5059700 1 -- 172.21.15.12:0/343524609 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88d00129c0 con 0x55583944dc10 2023-10-05T19:40:19.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.303+0000 7f88e5059700 1 --2- 172.21.15.12:0/343524609 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f88d0007030 0x7f88d001c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:19.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.303+0000 7f88e5059700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839454e70 con 0x7f88d0007030 2023-10-05T19:40:19.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.303+0000 7f88e585a700 1 --2- 172.21.15.12:0/343524609 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f88d0007030 0x7f88d001c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:19.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.303+0000 7f88e585a700 1 --2- 172.21.15.12:0/343524609 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f88d0007030 0x7f88d001c5f0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:19.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.303+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88e0014040 con 0x7f88d0007030 2023-10-05T19:40:19.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.303+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839454e70 con 0x55583941b110 2023-10-05T19:40:19.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.303+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88e0014040 con 0x55583941b110 2023-10-05T19:40:19.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.303+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839454e70 con 0x55583941b110 2023-10-05T19:40:19.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.303+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88e0014040 con 0x55583941b110 2023-10-05T19:40:19.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.303+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839454e70 con 0x7f88d0007030 2023-10-05T19:40:19.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.307+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88e0014040 con 0x7f88d0007030 2023-10-05T19:40:19.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.307+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839454e70 con 0x7f88d0007030 2023-10-05T19:40:19.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.307+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88e0014040 con 0x7f88d0007030 2023-10-05T19:40:19.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.307+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839454e70 con 0x55583944dc10 2023-10-05T19:40:19.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.307+0000 7f88e5059700 1 -- 172.21.15.12:0/343524609 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88d00129c0 con 0x55583944dc10 2023-10-05T19:40:19.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.307+0000 7f88e5059700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839454e70 con 0x5558394523b0 2023-10-05T19:40:19.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.307+0000 7f88e4858700 1 -- 172.21.15.12:0/343524609 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88d8004520 con 0x5558394523b0 2023-10-05T19:40:19.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.307+0000 7f88e4858700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839454e70 con 0x55583941b110 2023-10-05T19:40:19.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.307+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88e0014040 con 0x55583941b110 2023-10-05T19:40:19.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.307+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839454e70 con 0x7f88d0007030 2023-10-05T19:40:19.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.311+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88e0014040 con 0x7f88d0007030 2023-10-05T19:40:19.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.311+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839454e70 con 0x55583941b110 2023-10-05T19:40:19.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.311+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88e0014040 con 0x55583941b110 2023-10-05T19:40:19.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.311+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839454e70 con 0x55583944dc10 2023-10-05T19:40:19.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.311+0000 7f88e5059700 1 -- 172.21.15.12:0/343524609 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88d00129c0 con 0x55583944dc10 2023-10-05T19:40:19.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.311+0000 7f88e5059700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839454e70 con 0x5558394523b0 2023-10-05T19:40:19.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.311+0000 7f88e4858700 1 -- 172.21.15.12:0/343524609 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88d8004520 con 0x5558394523b0 2023-10-05T19:40:19.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.311+0000 7f88e4858700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839454e70 con 0x5558394523b0 2023-10-05T19:40:19.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.311+0000 7f88e4858700 1 -- 172.21.15.12:0/343524609 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88d8004520 con 0x5558394523b0 2023-10-05T19:40:19.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.311+0000 7f88e4858700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839454e70 con 0x7f88d0007030 2023-10-05T19:40:19.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.315+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88e0014040 con 0x7f88d0007030 2023-10-05T19:40:19.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.315+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839454e70 con 0x5558394523b0 2023-10-05T19:40:19.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.315+0000 7f88e4858700 1 -- 172.21.15.12:0/343524609 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88d8004520 con 0x5558394523b0 2023-10-05T19:40:19.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.315+0000 7f88e4858700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839454e70 con 0x5558394523b0 2023-10-05T19:40:19.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.315+0000 7f88e4858700 1 -- 172.21.15.12:0/343524609 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88d8004520 con 0x5558394523b0 2023-10-05T19:40:19.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.315+0000 7f88e4858700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839454e70 con 0x55583941b110 2023-10-05T19:40:19.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.315+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88e0014040 con 0x55583941b110 2023-10-05T19:40:19.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.315+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839454e70 con 0x55583941b110 2023-10-05T19:40:19.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.315+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88e0014040 con 0x55583941b110 2023-10-05T19:40:19.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.315+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839454e70 con 0x7f88d0007030 2023-10-05T19:40:19.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.315+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88e0014040 con 0x7f88d0007030 2023-10-05T19:40:19.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.319+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839454e70 con 0x5558394523b0 2023-10-05T19:40:19.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.319+0000 7f88e4858700 1 -- 172.21.15.12:0/343524609 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88d8004520 con 0x5558394523b0 2023-10-05T19:40:19.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.319+0000 7f88e4858700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839454e70 con 0x55583944dc10 2023-10-05T19:40:19.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.319+0000 7f88e5059700 1 -- 172.21.15.12:0/343524609 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88d00129c0 con 0x55583944dc10 2023-10-05T19:40:19.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.319+0000 7f88e5059700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839454e70 con 0x55583941b110 2023-10-05T19:40:19.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.319+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88e0014040 con 0x55583941b110 2023-10-05T19:40:19.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.319+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839454e70 con 0x7f88d0007030 2023-10-05T19:40:19.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.319+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88e0014040 con 0x7f88d0007030 2023-10-05T19:40:19.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.319+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839454e70 con 0x5558394523b0 2023-10-05T19:40:19.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.323+0000 7f88e4858700 1 -- 172.21.15.12:0/343524609 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88d8004520 con 0x5558394523b0 2023-10-05T19:40:19.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.323+0000 7f88e4858700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839454e70 con 0x7f88d0007030 2023-10-05T19:40:19.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.323+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88e0014040 con 0x7f88d0007030 2023-10-05T19:40:19.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.323+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839454e70 con 0x55583944dc10 2023-10-05T19:40:19.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.323+0000 7f88e5059700 1 -- 172.21.15.12:0/343524609 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88d00129c0 con 0x55583944dc10 2023-10-05T19:40:19.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.323+0000 7f88e5059700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839454e70 con 0x5558394523b0 2023-10-05T19:40:19.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.323+0000 7f88e4858700 1 -- 172.21.15.12:0/343524609 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88d8004520 con 0x5558394523b0 2023-10-05T19:40:19.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.323+0000 7f88e4858700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839454e70 con 0x7f88d0007030 2023-10-05T19:40:19.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.327+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88e0014040 con 0x7f88d0007030 2023-10-05T19:40:19.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.327+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839454e70 con 0x55583941b110 2023-10-05T19:40:19.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.327+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f88e0014040 con 0x55583941b110 2023-10-05T19:40:19.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.327+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839454e70 con 0x55583944dc10 2023-10-05T19:40:19.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.327+0000 7f88e5059700 1 -- 172.21.15.12:0/343524609 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f88d00129c0 con 0x55583944dc10 2023-10-05T19:40:19.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.327+0000 7f88e7524e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:19.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.327+0000 7f88e7524e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:19.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.327+0000 7f88e7524e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:19.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.327+0000 7f88e7524e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:19.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.327+0000 7f88e7524e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.327+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839451130 con 0x55583941b110 2023-10-05T19:40:19.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.327+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f88e0014040 con 0x55583941b110 2023-10-05T19:40:19.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.327+0000 7f88e7524e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:19.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.327+0000 7f88e7524e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.327+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839450d60 con 0x55583941b110 2023-10-05T19:40:19.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.327+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f88e0014040 con 0x55583941b110 2023-10-05T19:40:19.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.327+0000 7f88e7524e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:19.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.327+0000 7f88e7524e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.327+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558394552e0 con 0x5558394523b0 2023-10-05T19:40:19.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.331+0000 7f88e4858700 1 -- 172.21.15.12:0/343524609 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f88d8004520 con 0x5558394523b0 2023-10-05T19:40:19.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.331+0000 7f88e7524e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:19.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.331+0000 7f88e7524e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.331+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839455690 con 0x5558394523b0 2023-10-05T19:40:19.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.331+0000 7f88e4858700 1 -- 172.21.15.12:0/343524609 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f88d8004520 con 0x5558394523b0 2023-10-05T19:40:19.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.331+0000 7f88e7524e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:19.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.331+0000 7f88e7524e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.331+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839455a40 con 0x55583941b110 2023-10-05T19:40:19.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.331+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f88e0014040 con 0x55583941b110 2023-10-05T19:40:19.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.331+0000 7f88e7524e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:19.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.331+0000 7f88e7524e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.331+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558394515c0 con 0x55583941b110 2023-10-05T19:40:19.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.331+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f88e0014040 con 0x55583941b110 2023-10-05T19:40:19.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.331+0000 7f88e7524e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:19.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.331+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839451590 con 0x5558394523b0 2023-10-05T19:40:19.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.335+0000 7f88e4858700 1 -- 172.21.15.12:0/343524609 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f88d8004520 con 0x5558394523b0 2023-10-05T19:40:19.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.335+0000 7f88e4858700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839451590 con 0x55583944dc10 2023-10-05T19:40:19.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.335+0000 7f88e5059700 1 -- 172.21.15.12:0/343524609 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88d00129c0 con 0x55583944dc10 2023-10-05T19:40:19.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.335+0000 7f88e5059700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839451590 con 0x7f88d0007030 2023-10-05T19:40:19.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.335+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88e0014040 con 0x7f88d0007030 2023-10-05T19:40:19.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.335+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839455a40 con 0x55583941b110 2023-10-05T19:40:19.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.335+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88e0014040 con 0x55583941b110 2023-10-05T19:40:19.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.335+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839455a40 con 0x55583941b110 2023-10-05T19:40:19.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.339+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88e0014040 con 0x55583941b110 2023-10-05T19:40:19.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.339+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839455a40 con 0x7f88d0007030 2023-10-05T19:40:19.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.339+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88e0014040 con 0x7f88d0007030 2023-10-05T19:40:19.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.339+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839455a40 con 0x7f88d0007030 2023-10-05T19:40:19.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.339+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88e0014040 con 0x7f88d0007030 2023-10-05T19:40:19.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.339+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839455a40 con 0x55583944dc10 2023-10-05T19:40:19.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.339+0000 7f88e5059700 1 -- 172.21.15.12:0/343524609 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88d00129c0 con 0x55583944dc10 2023-10-05T19:40:19.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.339+0000 7f88e5059700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839455a40 con 0x5558394523b0 2023-10-05T19:40:19.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.343+0000 7f88e4858700 1 -- 172.21.15.12:0/343524609 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88d802d040 con 0x5558394523b0 2023-10-05T19:40:19.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.343+0000 7f88e4858700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839455a40 con 0x55583941b110 2023-10-05T19:40:19.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.343+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88e0014040 con 0x55583941b110 2023-10-05T19:40:19.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.343+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839455a40 con 0x7f88d0007030 2023-10-05T19:40:19.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.343+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88e0014040 con 0x7f88d0007030 2023-10-05T19:40:19.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.343+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839455a40 con 0x55583941b110 2023-10-05T19:40:19.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.343+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88e0014040 con 0x55583941b110 2023-10-05T19:40:19.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.343+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839455a40 con 0x55583944dc10 2023-10-05T19:40:19.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.343+0000 7f88e5059700 1 -- 172.21.15.12:0/343524609 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88d00129c0 con 0x55583944dc10 2023-10-05T19:40:19.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.343+0000 7f88e5059700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839455a40 con 0x5558394523b0 2023-10-05T19:40:19.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.347+0000 7f88e4858700 1 -- 172.21.15.12:0/343524609 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88d802d040 con 0x5558394523b0 2023-10-05T19:40:19.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.347+0000 7f88e4858700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839455a40 con 0x5558394523b0 2023-10-05T19:40:19.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.347+0000 7f88e4858700 1 -- 172.21.15.12:0/343524609 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88d802d040 con 0x5558394523b0 2023-10-05T19:40:19.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.347+0000 7f88e4858700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839455a40 con 0x7f88d0007030 2023-10-05T19:40:19.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.347+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88e002e040 con 0x7f88d0007030 2023-10-05T19:40:19.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.347+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839455a40 con 0x5558394523b0 2023-10-05T19:40:19.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.347+0000 7f88e4858700 1 -- 172.21.15.12:0/343524609 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88d802d040 con 0x5558394523b0 2023-10-05T19:40:19.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.347+0000 7f88e4858700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839455a40 con 0x5558394523b0 2023-10-05T19:40:19.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.351+0000 7f88e4858700 1 -- 172.21.15.12:0/343524609 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88d802d040 con 0x5558394523b0 2023-10-05T19:40:19.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.351+0000 7f88e4858700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839455a40 con 0x55583941b110 2023-10-05T19:40:19.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.351+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88e002e040 con 0x55583941b110 2023-10-05T19:40:19.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.351+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839455a40 con 0x55583941b110 2023-10-05T19:40:19.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.351+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88e002e040 con 0x55583941b110 2023-10-05T19:40:19.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.351+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839455a40 con 0x7f88d0007030 2023-10-05T19:40:19.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.351+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88e002e040 con 0x7f88d0007030 2023-10-05T19:40:19.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.351+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839455a40 con 0x5558394523b0 2023-10-05T19:40:19.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.355+0000 7f88e4858700 1 -- 172.21.15.12:0/343524609 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88d802d040 con 0x5558394523b0 2023-10-05T19:40:19.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.355+0000 7f88e4858700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839455a40 con 0x55583944dc10 2023-10-05T19:40:19.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.355+0000 7f88e5059700 1 -- 172.21.15.12:0/343524609 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88d00129c0 con 0x55583944dc10 2023-10-05T19:40:19.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.355+0000 7f88e5059700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839455a40 con 0x55583941b110 2023-10-05T19:40:19.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.355+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88e002e040 con 0x55583941b110 2023-10-05T19:40:19.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.355+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839455a40 con 0x7f88d0007030 2023-10-05T19:40:19.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.355+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88e002e040 con 0x7f88d0007030 2023-10-05T19:40:19.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.355+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839455a40 con 0x5558394523b0 2023-10-05T19:40:19.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.355+0000 7f88e4858700 1 -- 172.21.15.12:0/343524609 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88d802d040 con 0x5558394523b0 2023-10-05T19:40:19.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.355+0000 7f88e4858700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839455a40 con 0x7f88d0007030 2023-10-05T19:40:19.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.359+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88e002e040 con 0x7f88d0007030 2023-10-05T19:40:19.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.359+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839455a40 con 0x55583944dc10 2023-10-05T19:40:19.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.359+0000 7f88e5059700 1 -- 172.21.15.12:0/343524609 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88d00129c0 con 0x55583944dc10 2023-10-05T19:40:19.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.359+0000 7f88e5059700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839455a40 con 0x5558394523b0 2023-10-05T19:40:19.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.359+0000 7f88e4858700 1 -- 172.21.15.12:0/343524609 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88d802d040 con 0x5558394523b0 2023-10-05T19:40:19.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.359+0000 7f88e4858700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839455a40 con 0x7f88d0007030 2023-10-05T19:40:19.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.359+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f88e002e040 con 0x7f88d0007030 2023-10-05T19:40:19.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.359+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839455a40 con 0x55583941b110 2023-10-05T19:40:19.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.363+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f88e002e040 con 0x55583941b110 2023-10-05T19:40:19.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.363+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839455a40 con 0x55583944dc10 2023-10-05T19:40:19.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.363+0000 7f88e5059700 1 -- 172.21.15.12:0/343524609 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f88d00129c0 con 0x55583944dc10 2023-10-05T19:40:19.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.363+0000 7f88e7524e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:19.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.363+0000 7f88e7524e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:19.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.363+0000 7f88e7524e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:19.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.363+0000 7f88e7524e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:19.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.363+0000 7f88e7524e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.363+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839451590 con 0x55583941b110 2023-10-05T19:40:19.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.363+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f88e002e040 con 0x55583941b110 2023-10-05T19:40:19.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.363+0000 7f88e7524e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:19.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.363+0000 7f88e7524e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.363+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839456330 con 0x55583944dc10 2023-10-05T19:40:19.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.363+0000 7f88e5059700 1 -- 172.21.15.12:0/343524609 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f88d00129c0 con 0x55583944dc10 2023-10-05T19:40:19.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.363+0000 7f88e7524e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:19.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.363+0000 7f88e7524e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.363+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839456c20 con 0x55583941b110 2023-10-05T19:40:19.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.363+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f88e002e040 con 0x55583941b110 2023-10-05T19:40:19.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.367+0000 7f88e7524e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:19.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.367+0000 7f88e7524e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.367+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839456c20 con 0x5558394523b0 2023-10-05T19:40:19.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.367+0000 7f88e4858700 1 -- 172.21.15.12:0/343524609 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f88d802d040 con 0x5558394523b0 2023-10-05T19:40:19.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.367+0000 7f88e7524e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:19.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.367+0000 7f88e7524e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.367+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839457390 con 0x7f88d0007030 2023-10-05T19:40:19.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.367+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f88e002e040 con 0x7f88d0007030 2023-10-05T19:40:19.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.367+0000 7f88e7524e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:19.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.367+0000 7f88e7524e40 20 zone default found 2023-10-05T19:40:19.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.367+0000 7f88e7524e40 4 Realm: () 2023-10-05T19:40:19.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.367+0000 7f88e7524e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:19.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.367+0000 7f88e7524e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:19.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.367+0000 7f88e7524e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:19.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.367+0000 7f88e7524e40 20 zonegroup default 2023-10-05T19:40:19.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.367+0000 7f88e7524e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.367+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839457340 con 0x7f88d0007030 2023-10-05T19:40:19.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.367+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f88e002e040 con 0x7f88d0007030 2023-10-05T19:40:19.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.367+0000 7f88e7524e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:19.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.367+0000 7f88e7524e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.367+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839458cc0 con 0x5558394523b0 2023-10-05T19:40:19.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.367+0000 7f88e4858700 1 -- 172.21.15.12:0/343524609 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f88d802d040 con 0x5558394523b0 2023-10-05T19:40:19.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.367+0000 7f88e7524e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:19.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.367+0000 7f88e7524e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.367+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839459070 con 0x5558394523b0 2023-10-05T19:40:19.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.371+0000 7f88e4858700 1 -- 172.21.15.12:0/343524609 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f88d803c040 con 0x5558394523b0 2023-10-05T19:40:19.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.371+0000 7f88e7524e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:19.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.371+0000 7f88e7524e40 20 started sync module instance, tier type = 2023-10-05T19:40:19.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.371+0000 7f88e7524e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:19.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.371+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839458380 con 0x55583944dc10 2023-10-05T19:40:19.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.371+0000 7f88e5059700 1 -- 172.21.15.12:0/343524609 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'39 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f88d00129c0 con 0x55583944dc10 2023-10-05T19:40:19.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.371+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93837406332720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839459420 con 0x55583944dc10 2023-10-05T19:40:19.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.371+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839459cc0 con 0x55583941b110 2023-10-05T19:40:19.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.375+0000 7f88e5059700 1 -- 172.21.15.12:0/343524609 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 93837406332720] v28'40 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f88d00129c0 con 0x55583944dc10 2023-10-05T19:40:19.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.375+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'39 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f88e002e040 con 0x55583941b110 2023-10-05T19:40:19.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.375+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93837406345168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558394588e0 con 0x55583941b110 2023-10-05T19:40:19.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.375+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55583945a8a0 con 0x5558394523b0 2023-10-05T19:40:19.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.379+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 93837406345168] v28'40 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f88e002e040 con 0x55583941b110 2023-10-05T19:40:19.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.379+0000 7f88e4858700 1 -- 172.21.15.12:0/343524609 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'39 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f88d803c040 con 0x5558394523b0 2023-10-05T19:40:19.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.379+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93837406348144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55583945aca0 con 0x5558394523b0 2023-10-05T19:40:19.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.379+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55583945b540 con 0x55583944dc10 2023-10-05T19:40:19.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.383+0000 7f88e5059700 1 -- 172.21.15.12:0/343524609 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'39 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f88d00129c0 con 0x55583944dc10 2023-10-05T19:40:19.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.383+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 93837406353728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55583945c280 con 0x55583944dc10 2023-10-05T19:40:19.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.383+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55583945cb60 con 0x55583941b110 2023-10-05T19:40:19.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.383+0000 7f88e4858700 1 -- 172.21.15.12:0/343524609 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 93837406348144] v28'40 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f88d803c040 con 0x5558394523b0 2023-10-05T19:40:19.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.383+0000 7f88e5059700 1 -- 172.21.15.12:0/343524609 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 93837406353728] v28'40 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f88d00129c0 con 0x55583944dc10 2023-10-05T19:40:19.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.387+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'39 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f88e002e040 con 0x55583941b110 2023-10-05T19:40:19.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.387+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 93837406346272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55583945c280 con 0x55583941b110 2023-10-05T19:40:19.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.387+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55583945d450 con 0x7f88d0007030 2023-10-05T19:40:19.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.387+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'39 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f88e002e040 con 0x7f88d0007030 2023-10-05T19:40:19.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.387+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 93837406358928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55583945d6d0 con 0x7f88d0007030 2023-10-05T19:40:19.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.387+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55583945df70 con 0x7f88d0007030 2023-10-05T19:40:19.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.387+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 93837406346272] v28'40 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f88e002e040 con 0x55583941b110 2023-10-05T19:40:19.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.391+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(96 notify.6 [create] v28'39 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f88e002e040 con 0x7f88d0007030 2023-10-05T19:40:19.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.391+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93837406361296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55583945e010 con 0x7f88d0007030 2023-10-05T19:40:19.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.391+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55583945ea20 con 0x7f88d0007030 2023-10-05T19:40:19.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.391+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(95 notify.5 [watch watch cookie 93837406358928] v28'40 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f88e002e040 con 0x7f88d0007030 2023-10-05T19:40:19.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.395+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(98 notify.7 [create] v28'39 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f88e002e040 con 0x7f88d0007030 2023-10-05T19:40:19.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.395+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93837406360032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55583945ea20 con 0x7f88d0007030 2023-10-05T19:40:19.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.395+0000 7f88e7524e40 20 add_watcher() i=0 2023-10-05T19:40:19.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.395+0000 7f88e7524e40 20 add_watcher() i=1 2023-10-05T19:40:19.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.395+0000 7f88e7524e40 20 add_watcher() i=2 2023-10-05T19:40:19.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.395+0000 7f88e7524e40 20 add_watcher() i=3 2023-10-05T19:40:19.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.395+0000 7f88e7524e40 20 add_watcher() i=4 2023-10-05T19:40:19.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.395+0000 7f88e7524e40 20 add_watcher() i=5 2023-10-05T19:40:19.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.395+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(97 notify.6 [watch watch cookie 93837406361296] v28'40 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f88e002e040 con 0x7f88d0007030 2023-10-05T19:40:19.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.395+0000 7f88e7524e40 20 add_watcher() i=6 2023-10-05T19:40:19.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.395+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 93837406360032] v28'40 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f88e002e040 con 0x7f88d0007030 2023-10-05T19:40:19.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.395+0000 7f88e7524e40 20 add_watcher() i=7 2023-10-05T19:40:19.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.395+0000 7f88e7524e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:19.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.395+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55583945dcf0 con 0x55583941b110 2023-10-05T19:40:19.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.395+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f88e002e040 con 0x55583941b110 2023-10-05T19:40:19.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.395+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93837406362864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55583945eb80 con 0x55583941b110 2023-10-05T19:40:19.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.399+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 93837406362864] v28'60 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f88e002e040 con 0x55583941b110 2023-10-05T19:40:19.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.399+0000 7f88b67e4700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:19.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.399+0000 7f88e7524e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:19.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.399+0000 7f88e7524e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:19.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.399+0000 7f88e7524e40 1 Processor -- start 2023-10-05T19:40:19.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.399+0000 7f88e7524e40 1 -- start start 2023-10-05T19:40:19.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.399+0000 7f88e7524e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55583946bde0 0x55583946c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:19.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.399+0000 7f88e7524e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55583946c6f0 con 0x55583946bde0 2023-10-05T19:40:19.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.399+0000 7f88e4858700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55583946bde0 0x55583946c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:19.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.399+0000 7f88e4858700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55583946bde0 0x55583946c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36898/0 (socket says 172.21.15.12:36898) 2023-10-05T19:40:19.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.399+0000 7f88e4858700 1 -- 172.21.15.12:0/2286898097 learned_addr learned my addr 172.21.15.12:0/2286898097 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:19.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.403+0000 7f88e4858700 1 -- 172.21.15.12:0/2286898097 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55583946c830 con 0x55583946bde0 2023-10-05T19:40:19.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.403+0000 7f88e4858700 1 --2- 172.21.15.12:0/2286898097 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55583946bde0 0x55583946c1b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f88d0011ff0 tx=0x7f88d8027040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4600a50510a290d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:19.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.403+0000 7f88b5fe3700 1 -- 172.21.15.12:0/2286898097 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f88d8039040 con 0x55583946bde0 2023-10-05T19:40:19.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.403+0000 7f88b5fe3700 1 -- 172.21.15.12:0/2286898097 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d803a040 con 0x55583946bde0 2023-10-05T19:40:19.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.403+0000 7f88b5fe3700 1 -- 172.21.15.12:0/2286898097 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f88d802a080 con 0x55583946bde0 2023-10-05T19:40:19.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.403+0000 7f88e7524e40 1 -- 172.21.15.12:0/2286898097 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55583946bde0 msgr2=0x55583946c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.403+0000 7f88e7524e40 1 --2- 172.21.15.12:0/2286898097 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55583946bde0 0x55583946c1b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f88d0011ff0 tx=0x7f88d8027040 comp rx=0 tx=0).stop 2023-10-05T19:40:19.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.403+0000 7f88e7524e40 1 -- 172.21.15.12:0/2286898097 shutdown_connections 2023-10-05T19:40:19.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.403+0000 7f88e7524e40 1 --2- 172.21.15.12:0/2286898097 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55583946bde0 0x55583946c1b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:19.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.403+0000 7f88e7524e40 1 -- 172.21.15.12:0/2286898097 >> 172.21.15.12:0/2286898097 conn(0x5558394671a0 msgr2=0x5558394652f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:19.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.403+0000 7f88e7524e40 1 -- 172.21.15.12:0/2286898097 shutdown_connections 2023-10-05T19:40:19.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.403+0000 7f88e7524e40 1 -- 172.21.15.12:0/2286898097 wait complete. 2023-10-05T19:40:19.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.403+0000 7f88e7524e40 1 Processor -- start 2023-10-05T19:40:19.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.403+0000 7f88e7524e40 1 -- start start 2023-10-05T19:40:19.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.407+0000 7f88e7524e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55583946bde0 0x555839471a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:19.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.407+0000 7f88e7524e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x555839472a40 con 0x55583946bde0 2023-10-05T19:40:19.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.407+0000 7f88e4858700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55583946bde0 0x555839471a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:19.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.407+0000 7f88e4858700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55583946bde0 0x555839471a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36902/0 (socket says 172.21.15.12:36902) 2023-10-05T19:40:19.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.407+0000 7f88e4858700 1 -- 172.21.15.12:0/1932053904 learned_addr learned my addr 172.21.15.12:0/1932053904 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:19.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.407+0000 7f88e4858700 1 -- 172.21.15.12:0/1932053904 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d4001e10 con 0x55583946bde0 2023-10-05T19:40:19.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.407+0000 7f88e4858700 1 --2- 172.21.15.12:0/1932053904 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55583946bde0 0x555839471a40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f88d8004fd0 tx=0x7f88d803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:19.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.407+0000 7f8827fff700 1 -- 172.21.15.12:0/1932053904 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f88d8039040 con 0x55583946bde0 2023-10-05T19:40:19.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.407+0000 7f8827fff700 1 -- 172.21.15.12:0/1932053904 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d802b040 con 0x55583946bde0 2023-10-05T19:40:19.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.407+0000 7f8827fff700 1 -- 172.21.15.12:0/1932053904 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f88d801a040 con 0x55583946bde0 2023-10-05T19:40:19.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.407+0000 7f88e7524e40 1 -- 172.21.15.12:0/1932053904 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555839472040 con 0x55583946bde0 2023-10-05T19:40:19.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.407+0000 7f88e7524e40 1 -- 172.21.15.12:0/1932053904 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555839472420 con 0x55583946bde0 2023-10-05T19:40:19.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.407+0000 7f8827fff700 1 -- 172.21.15.12:0/1932053904 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f88d804b730 con 0x55583946bde0 2023-10-05T19:40:19.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.407+0000 7f8827fff700 1 --2- 172.21.15.12:0/1932053904 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f882c030c50 0x7f882c033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:19.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.407+0000 7f8827fff700 1 -- 172.21.15.12:0/1932053904 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f88d8028040 con 0x55583946bde0 2023-10-05T19:40:19.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.411+0000 7f88e585a700 1 --2- 172.21.15.12:0/1932053904 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f882c030c50 0x7f882c033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:19.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.411+0000 7f88e7524e40 5 note: GC not initialized 2023-10-05T19:40:19.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.411+0000 7f88e585a700 1 --2- 172.21.15.12:0/1932053904 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f882c030c50 0x7f882c033100 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x55583945eae0 tx=0x7f88e0005500 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:19.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.411+0000 7f88257fa700 20 reqs_thread_entry: start 2023-10-05T19:40:19.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.411+0000 7f881effd700 20 reqs_thread_entry: start 2023-10-05T19:40:19.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.411+0000 7f88e7524e40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:19.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.411+0000 7f88e7524e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:19.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.411+0000 7f881cff9700 20 reqs_thread_entry: start 2023-10-05T19:40:19.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.411+0000 7f88e7524e40 20 get_system_obj_state: rctx=0x7fffcffb0008 obj=default.rgw.meta:users.uid:foo state=0x5558392f10b0 s->prefetch_data=0 2023-10-05T19:40:19.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.415+0000 7f88e7524e40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2023-10-05T19:40:19.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.415+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558394da230 con 0x55583941b110 2023-10-05T19:40:19.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.415+0000 7f881d7fa700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:49 2023 2023-10-05T19:40:19.422 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:19.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.415+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(102 foo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7f88e002e040 con 0x55583941b110 2023-10-05T19:40:19.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.415+0000 7f88e7524e40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x16 2023-10-05T19:40:19.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.415+0000 7f88e7524e40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:19.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.415+0000 7f88e7524e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:19.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.415+0000 7f88e7524e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:19.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.415+0000 7f88e7524e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:19.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.415+0000 7f88e7524e40 10 cache get: name=default.rgw.meta+users.uid+foo : type miss (requested=0x13, cached=0x16) 2023-10-05T19:40:19.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.415+0000 7f88e7524e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.415+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558394dccc0 con 0x55583941b110 2023-10-05T19:40:19.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.415+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(103 foo [call,call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv2 ondisk = 0) v8 ==== 273+0+480 (crc 0 0 0) 0x7f88e002e040 con 0x55583941b110 2023-10-05T19:40:19.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.415+0000 7f88e7524e40 20 rados_obj.operate() r=0 bl.length=357 2023-10-05T19:40:19.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.415+0000 7f88e7524e40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2023-10-05T19:40:19.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.415+0000 7f88e7524e40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:19.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.415+0000 7f88e7524e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:19.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.415+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:104 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558394dccc0 con 0x55583944dc10 2023-10-05T19:40:19.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.415+0000 7f88e5059700 1 -- 172.21.15.12:0/343524609 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(104 foo.buckets [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 155+0+0 (crc 0 0 0) 0x7f88d00129c0 con 0x55583944dc10 2023-10-05T19:40:19.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.415+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:105 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.set in=58b,writefull 0~357 in=357b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558394ddf40 con 0x55583941b110 2023-10-05T19:40:19.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.419+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(105 foo [delete,create,call,writefull 0~357] v28'3 uv3 ondisk = 0) v8 ==== 273+0+0 (crc 0 0 0) 0x7f88e002e040 con 0x55583941b110 2023-10-05T19:40:19.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.419+0000 7f88e7524e40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2023-10-05T19:40:19.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.419+0000 7f88e7524e40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:19.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.419+0000 7f88e7524e40 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2023-10-05T19:40:19.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.419+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:106 7.d 7:b4812045:::notify.4:head [notify cookie 93837406886480 in=545b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5558394dccc0 con 0x55583941b110 2023-10-05T19:40:19.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.419+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== watch-notify(notify (1) cookie 93837406346272 notify 120259084290 ret 0) v3 ==== 575+0+0 (crc 0 0 0) 0x7f88e00400a0 con 0x55583941b110 2023-10-05T19:40:19.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.419+0000 7f88de7fc700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084290 cookie 93837406346272 notifier 4303 bl.length()=533 2023-10-05T19:40:19.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.419+0000 7f88de7fc700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2023-10-05T19:40:19.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.419+0000 7f88de7fc700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:19.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.419+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(106 notify.4 [notify cookie 93837406886480 out=8b] v0'0 uv39 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f88e002e040 con 0x55583941b110 2023-10-05T19:40:19.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.419+0000 7f88de7fc700 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:107 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f88c4003d90 con 0x55583941b110 2023-10-05T19:40:19.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.419+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== osd_op_reply(107 notify.4 [notify-ack] v0'0 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f88e002e040 con 0x55583941b110 2023-10-05T19:40:19.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.419+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.1 v2:172.21.15.12:6808/29933 39 ==== watch-notify(notify_complete (2) cookie 93837406886480 notify 120259084290 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f88e0040230 con 0x55583941b110 2023-10-05T19:40:19.430 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:40:19.430 INFO:teuthology.orchestra.run.smithi012.stdout: "user_id": "foo", 2023-10-05T19:40:19.430 INFO:teuthology.orchestra.run.smithi012.stdout: "display_name": "Foo", 2023-10-05T19:40:19.431 INFO:teuthology.orchestra.run.smithi012.stdout: "email": "foo@foo.com", 2023-10-05T19:40:19.431 INFO:teuthology.orchestra.run.smithi012.stdout: "suspended": 0, 2023-10-05T19:40:19.431 INFO:teuthology.orchestra.run.smithi012.stdout: "max_buckets": 4, 2023-10-05T19:40:19.431 INFO:teuthology.orchestra.run.smithi012.stdout: "subusers": [], 2023-10-05T19:40:19.431 INFO:teuthology.orchestra.run.smithi012.stdout: "keys": [ 2023-10-05T19:40:19.431 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:19.431 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:40:19.431 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2023-10-05T19:40:19.431 INFO:teuthology.orchestra.run.smithi012.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2023-10-05T19:40:19.431 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:19.431 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:19.431 INFO:teuthology.orchestra.run.smithi012.stdout: "swift_keys": [], 2023-10-05T19:40:19.432 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": [], 2023-10-05T19:40:19.432 INFO:teuthology.orchestra.run.smithi012.stdout: "op_mask": "read, write, delete", 2023-10-05T19:40:19.432 INFO:teuthology.orchestra.run.smithi012.stdout: "default_placement": "", 2023-10-05T19:40:19.432 INFO:teuthology.orchestra.run.smithi012.stdout: "default_storage_class": "", 2023-10-05T19:40:19.432 INFO:teuthology.orchestra.run.smithi012.stdout: "placement_tags": [], 2023-10-05T19:40:19.432 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket_quota": { 2023-10-05T19:40:19.432 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:19.432 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:19.432 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:19.432 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:19.432 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:19.432 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:19.432 INFO:teuthology.orchestra.run.smithi012.stdout: "user_quota": { 2023-10-05T19:40:19.433 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:19.433 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:19.433 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:19.433 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:19.433 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:19.433 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:19.433 INFO:teuthology.orchestra.run.smithi012.stdout: "temp_url_keys": [], 2023-10-05T19:40:19.433 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "rgw", 2023-10-05T19:40:19.433 INFO:teuthology.orchestra.run.smithi012.stdout: "mfa_ids": [] 2023-10-05T19:40:19.433 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:40:19.433 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:40:19.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.423+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:108 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93837406332720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55583942d620 con 0x55583944dc10 2023-10-05T19:40:19.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.423+0000 7f88e5059700 1 -- 172.21.15.12:0/343524609 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(108 notify.0 [watch unwatch cookie 93837406332720] v28'41 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f88d00129c0 con 0x55583944dc10 2023-10-05T19:40:19.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.423+0000 7f88e7524e40 20 remove_watcher() i=0 2023-10-05T19:40:19.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.423+0000 7f88e7524e40 2 removed watcher, disabling cache 2023-10-05T19:40:19.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.423+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:109 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93837406345168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839456730 con 0x55583941b110 2023-10-05T19:40:19.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.427+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.1 v2:172.21.15.12:6808/29933 40 ==== osd_op_reply(109 notify.1 [watch unwatch cookie 93837406345168] v28'41 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f88e002e040 con 0x55583941b110 2023-10-05T19:40:19.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.427+0000 7f88e7524e40 20 remove_watcher() i=1 2023-10-05T19:40:19.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.427+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:110 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93837406348144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839456730 con 0x5558394523b0 2023-10-05T19:40:19.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.431+0000 7f88e4858700 1 -- 172.21.15.12:0/343524609 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(110 notify.2 [watch unwatch cookie 93837406348144] v28'41 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f88d803c040 con 0x5558394523b0 2023-10-05T19:40:19.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.431+0000 7f88e7524e40 20 remove_watcher() i=2 2023-10-05T19:40:19.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.431+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:111 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93837406353728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55583945a370 con 0x55583944dc10 2023-10-05T19:40:19.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.431+0000 7f88e5059700 1 -- 172.21.15.12:0/343524609 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(111 notify.3 [watch unwatch cookie 93837406353728] v28'41 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f88d00129c0 con 0x55583944dc10 2023-10-05T19:40:19.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.431+0000 7f88e7524e40 20 remove_watcher() i=3 2023-10-05T19:40:19.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.431+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:112 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93837406346272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839456730 con 0x55583941b110 2023-10-05T19:40:19.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.435+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.1 v2:172.21.15.12:6808/29933 41 ==== osd_op_reply(112 notify.4 [watch unwatch cookie 93837406346272] v28'41 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f88e002e040 con 0x55583941b110 2023-10-05T19:40:19.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.435+0000 7f88e7524e40 20 remove_watcher() i=4 2023-10-05T19:40:19.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.435+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:113 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93837406358928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839459cc0 con 0x7f88d0007030 2023-10-05T19:40:19.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.439+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(113 notify.5 [watch unwatch cookie 93837406358928] v28'41 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f88e002e040 con 0x7f88d0007030 2023-10-05T19:40:19.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.439+0000 7f88e7524e40 20 remove_watcher() i=5 2023-10-05T19:40:19.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.439+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:114 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93837406361296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839459cc0 con 0x7f88d0007030 2023-10-05T19:40:19.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.443+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(114 notify.6 [watch unwatch cookie 93837406361296] v28'41 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f88e002e040 con 0x7f88d0007030 2023-10-05T19:40:19.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.443+0000 7f88e7524e40 20 remove_watcher() i=6 2023-10-05T19:40:19.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.443+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:115 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93837406360032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x555839459cc0 con 0x7f88d0007030 2023-10-05T19:40:19.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.443+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(115 notify.7 [watch unwatch cookie 93837406360032] v28'41 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f88e002e040 con 0x7f88d0007030 2023-10-05T19:40:19.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.443+0000 7f88e7524e40 20 remove_watcher() i=7 2023-10-05T19:40:19.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.447+0000 7f88b67e4700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:19.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.447+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:116 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93837406362864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55583945d280 con 0x55583941b110 2023-10-05T19:40:19.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.451+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 <== osd.1 v2:172.21.15.12:6808/29933 42 ==== osd_op_reply(116 data_loggenerations_metadata [watch unwatch cookie 93837406362864] v28'61 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f88e002e040 con 0x55583941b110 2023-10-05T19:40:19.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.451+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f88d0007030 msgr2=0x7f88d001c5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.451+0000 7f88e7524e40 1 --2- 172.21.15.12:0/343524609 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f88d0007030 0x7f88d001c5f0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:19.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.451+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55583941b110 msgr2=0x55583944b610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.451+0000 7f88e7524e40 1 --2- 172.21.15.12:0/343524609 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55583941b110 0x55583944b610 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:19.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.451+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55583944dc10 msgr2=0x55583944fff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.451+0000 7f88e7524e40 1 --2- 172.21.15.12:0/343524609 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55583944dc10 0x55583944fff0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:19.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.451+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5558394523b0 msgr2=0x5558394547c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.451+0000 7f88e7524e40 1 --2- 172.21.15.12:0/343524609 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5558394523b0 0x5558394547c0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:19.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.451+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f88c8030c90 msgr2=0x7f88c8033140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.451+0000 7f88e7524e40 1 --2- 172.21.15.12:0/343524609 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f88c8030c90 0x7f88c8033140 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f88d80044f0 tx=0x7f88d80065f0 comp rx=0 tx=0).stop 2023-10-05T19:40:19.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.451+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55583938c3d0 msgr2=0x55583943fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.451+0000 7f88e585a700 1 -- 172.21.15.12:0/343524609 reap_dead start 2023-10-05T19:40:19.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.451+0000 7f88e7524e40 1 --2- 172.21.15.12:0/343524609 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55583938c3d0 0x55583943fc10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f88d001f040 tx=0x7f88d001c000 comp rx=0 tx=0).stop 2023-10-05T19:40:19.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.451+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 shutdown_connections 2023-10-05T19:40:19.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.451+0000 7f88e7524e40 1 --2- 172.21.15.12:0/343524609 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55583938c3d0 0x55583943fc10 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:19.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.451+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 >> 172.21.15.12:0/343524609 conn(0x555839412720 msgr2=0x555839431380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:19.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.451+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 shutdown_connections 2023-10-05T19:40:19.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.451+0000 7f88e7524e40 1 -- 172.21.15.12:0/343524609 wait complete. 2023-10-05T19:40:19.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.451+0000 7f88e7524e40 1 -- 172.21.15.12:0/1932053904 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f882c030c50 msgr2=0x7f882c033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.451+0000 7f88e7524e40 1 --2- 172.21.15.12:0/1932053904 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f882c030c50 0x7f882c033100 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x55583945eae0 tx=0x7f88e0005500 comp rx=0 tx=0).stop 2023-10-05T19:40:19.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.451+0000 7f88e7524e40 1 -- 172.21.15.12:0/1932053904 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55583946bde0 msgr2=0x555839471a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.451+0000 7f88e7524e40 1 --2- 172.21.15.12:0/1932053904 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55583946bde0 0x555839471a40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f88d8004fd0 tx=0x7f88d803f000 comp rx=0 tx=0).stop 2023-10-05T19:40:19.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.451+0000 7f88e7524e40 1 -- 172.21.15.12:0/1932053904 shutdown_connections 2023-10-05T19:40:19.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.451+0000 7f88e7524e40 1 --2- 172.21.15.12:0/1932053904 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f882c030c50 0x7f882c033100 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:19.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.451+0000 7f88e7524e40 1 --2- 172.21.15.12:0/1932053904 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55583946bde0 0x555839471a40 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:19.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.451+0000 7f88e7524e40 1 -- 172.21.15.12:0/1932053904 >> 172.21.15.12:0/1932053904 conn(0x5558394671a0 msgr2=0x555839465240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:19.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.451+0000 7f88e7524e40 1 -- 172.21.15.12:0/1932053904 shutdown_connections 2023-10-05T19:40:19.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.451+0000 7f88e7524e40 1 -- 172.21.15.12:0/1932053904 wait complete. 2023-10-05T19:40:19.465 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:40:19.466 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2023-10-05T19:40:19.466 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2023-10-05T19:40:19.466 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2023-10-05T19:40:19.507 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:19.507 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:19.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.499+0000 7f9981f72e40 1 Processor -- start 2023-10-05T19:40:19.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.499+0000 7f9981f72e40 1 -- start start 2023-10-05T19:40:19.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.499+0000 7f9981f72e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x56348498e2f0 0x5634849906d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:19.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.499+0000 7f9981f72e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5634848d6bd0 con 0x5634848fe3d0 2023-10-05T19:40:19.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.499+0000 7f9981f72e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x563484893ee0 con 0x56348498e2f0 2023-10-05T19:40:19.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.499+0000 7f997faa7700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x5634848fe3d0 0x56348498cbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:47262/0 (socket says 172.21.15.12:47262) 2023-10-05T19:40:19.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.499+0000 7f997faa7700 1 -- 172.21.15.12:0/1095027092 learned_addr learned my addr 172.21.15.12:0/1095027092 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:19.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.499+0000 7f997f2a6700 1 --2- 172.21.15.12:0/1095027092 >> v2:172.21.15.8:3300/0 conn(0x56348498e2f0 0x5634849906d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:19.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.499+0000 7f997eaa5700 1 -- 172.21.15.12:0/1095027092 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1147863238 0 0) 0x5634848d6bd0 con 0x5634848fe3d0 2023-10-05T19:40:19.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.499+0000 7f997eaa5700 1 -- 172.21.15.12:0/1095027092 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9968001e10 con 0x5634848fe3d0 2023-10-05T19:40:19.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.499+0000 7f997f2a6700 1 -- 172.21.15.12:0/1095027092 >> v1:172.21.15.8:6789/0 conn(0x5634848fe3d0 legacy=0x56348498cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.499+0000 7f997f2a6700 1 -- 172.21.15.12:0/1095027092 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563484991c20 con 0x56348498e2f0 2023-10-05T19:40:19.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.503+0000 7f997f2a6700 1 --2- 172.21.15.12:0/1095027092 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56348498e2f0 0x5634849906d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f997400a5d0 tx=0x7f9974005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=272860e0fb3fcf81 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:19.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.503+0000 7f997eaa5700 1 -- 172.21.15.12:0/1095027092 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9974007610 con 0x56348498e2f0 2023-10-05T19:40:19.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.503+0000 7f997eaa5700 1 -- 172.21.15.12:0/1095027092 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9974007770 con 0x56348498e2f0 2023-10-05T19:40:19.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.503+0000 7f997eaa5700 1 -- 172.21.15.12:0/1095027092 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9974007a20 con 0x56348498e2f0 2023-10-05T19:40:19.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.503+0000 7f9981f72e40 1 -- 172.21.15.12:0/1095027092 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56348498e2f0 msgr2=0x5634849906d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.503+0000 7f9981f72e40 1 --2- 172.21.15.12:0/1095027092 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56348498e2f0 0x5634849906d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f997400a5d0 tx=0x7f9974005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:19.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.503+0000 7f9981f72e40 1 -- 172.21.15.12:0/1095027092 shutdown_connections 2023-10-05T19:40:19.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.503+0000 7f9981f72e40 1 --2- 172.21.15.12:0/1095027092 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56348498e2f0 0x5634849906d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:19.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.503+0000 7f9981f72e40 1 -- 172.21.15.12:0/1095027092 >> 172.21.15.12:0/1095027092 conn(0x563484984720 msgr2=0x563484986b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:19.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.503+0000 7f9981f72e40 1 -- 172.21.15.12:0/1095027092 shutdown_connections 2023-10-05T19:40:19.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.503+0000 7f9981f72e40 1 -- 172.21.15.12:0/1095027092 wait complete. 2023-10-05T19:40:19.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.507+0000 7f9981f72e40 1 Processor -- start 2023-10-05T19:40:19.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.507+0000 7f9981f72e40 1 -- start start 2023-10-05T19:40:19.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.507+0000 7f9981f72e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5634848fe3d0 0x5634849a8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:19.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.507+0000 7f9981f72e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x563484857e30 con 0x5634848fe3d0 2023-10-05T19:40:19.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.507+0000 7f997faa7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5634848fe3d0 0x5634849a8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:19.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.507+0000 7f997faa7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5634848fe3d0 0x5634849a8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36920/0 (socket says 172.21.15.12:36920) 2023-10-05T19:40:19.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.507+0000 7f997faa7700 1 -- 172.21.15.12:0/2328894008 learned_addr learned my addr 172.21.15.12:0/2328894008 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:19.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.507+0000 7f997faa7700 1 -- 172.21.15.12:0/2328894008 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9974005cf0 con 0x5634848fe3d0 2023-10-05T19:40:19.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.507+0000 7f997faa7700 1 --2- 172.21.15.12:0/2328894008 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5634848fe3d0 0x5634849a8d90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9978006e30 tx=0x7f9978006a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baeb58f0768d93f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:19.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.507+0000 7f997daa3700 1 -- 172.21.15.12:0/2328894008 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9978006110 con 0x5634848fe3d0 2023-10-05T19:40:19.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.507+0000 7f997daa3700 1 -- 172.21.15.12:0/2328894008 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9978006290 con 0x5634848fe3d0 2023-10-05T19:40:19.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.507+0000 7f997daa3700 1 -- 172.21.15.12:0/2328894008 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9978004520 con 0x5634848fe3d0 2023-10-05T19:40:19.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.507+0000 7f9981f72e40 1 -- 172.21.15.12:0/2328894008 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5634848fe3d0 msgr2=0x5634849a8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.507+0000 7f9981f72e40 1 --2- 172.21.15.12:0/2328894008 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5634848fe3d0 0x5634849a8d90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9978006e30 tx=0x7f9978006a10 comp rx=0 tx=0).stop 2023-10-05T19:40:19.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.507+0000 7f9981f72e40 1 -- 172.21.15.12:0/2328894008 shutdown_connections 2023-10-05T19:40:19.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.507+0000 7f9981f72e40 1 --2- 172.21.15.12:0/2328894008 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5634848fe3d0 0x5634849a8d90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:19.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.507+0000 7f9981f72e40 1 -- 172.21.15.12:0/2328894008 >> 172.21.15.12:0/2328894008 conn(0x563484984720 msgr2=0x5634849a4950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:19.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.507+0000 7f9981f72e40 1 -- 172.21.15.12:0/2328894008 shutdown_connections 2023-10-05T19:40:19.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.507+0000 7f9981f72e40 1 -- 172.21.15.12:0/2328894008 wait complete. 2023-10-05T19:40:19.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.511+0000 7f9981f72e40 1 Processor -- start 2023-10-05T19:40:19.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.511+0000 7f9981f72e40 1 -- start start 2023-10-05T19:40:19.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.511+0000 7f9981f72e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5634848fe3d0 0x5634849afb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:19.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.511+0000 7f9981f72e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56348482aed0 con 0x5634848fe3d0 2023-10-05T19:40:19.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.511+0000 7f997faa7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5634848fe3d0 0x5634849afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:19.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.511+0000 7f997faa7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5634848fe3d0 0x5634849afb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36928/0 (socket says 172.21.15.12:36928) 2023-10-05T19:40:19.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.511+0000 7f997faa7700 1 -- 172.21.15.12:0/2160596884 learned_addr learned my addr 172.21.15.12:0/2160596884 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:19.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.511+0000 7f997faa7700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9968001e10 con 0x5634848fe3d0 2023-10-05T19:40:19.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.511+0000 7f997faa7700 1 --2- 172.21.15.12:0/2160596884 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5634848fe3d0 0x5634849afb70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f997800f000 tx=0x7f9978010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:19.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.511+0000 7f9967fff700 1 -- 172.21.15.12:0/2160596884 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9978006110 con 0x5634848fe3d0 2023-10-05T19:40:19.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.511+0000 7f9967fff700 1 -- 172.21.15.12:0/2160596884 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99780044f0 con 0x5634848fe3d0 2023-10-05T19:40:19.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.511+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5634849b00b0 con 0x5634848fe3d0 2023-10-05T19:40:19.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.511+0000 7f9967fff700 1 -- 172.21.15.12:0/2160596884 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9978014bd0 con 0x5634848fe3d0 2023-10-05T19:40:19.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.511+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5634849b04f0 con 0x5634848fe3d0 2023-10-05T19:40:19.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.511+0000 7f9981f72e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.515+0000 7f9967fff700 1 -- 172.21.15.12:0/2160596884 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f9978028460 con 0x5634848fe3d0 2023-10-05T19:40:19.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.515+0000 7f9967fff700 1 --2- 172.21.15.12:0/2160596884 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f996c030c50 0x7f996c033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:19.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.515+0000 7f9967fff700 1 -- 172.21.15.12:0/2160596884 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f9978044f80 con 0x5634848fe3d0 2023-10-05T19:40:19.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.515+0000 7f997f2a6700 1 --2- 172.21.15.12:0/2160596884 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f996c030c50 0x7f996c033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:19.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.515+0000 7f9981f72e40 1 --2- 172.21.15.12:0/2160596884 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x56348498e2f0 0x5634849bb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:19.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.515+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849bbd00 con 0x56348498e2f0 2023-10-05T19:40:19.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.515+0000 7f99802a8700 1 --2- 172.21.15.12:0/2160596884 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x56348498e2f0 0x5634849bb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:19.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.515+0000 7f997f2a6700 1 --2- 172.21.15.12:0/2160596884 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f996c030c50 0x7f996c033100 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9974005cc0 tx=0x7f9974014000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:19.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.515+0000 7f99802a8700 1 --2- 172.21.15.12:0/2160596884 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x56348498e2f0 0x5634849bb600 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:19.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.515+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f997000a500 con 0x56348498e2f0 2023-10-05T19:40:19.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.515+0000 7f9981f72e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:19.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.515+0000 7f9981f72e40 20 realm 2023-10-05T19:40:19.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.515+0000 7f9981f72e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.515+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849b8c20 con 0x56348498e2f0 2023-10-05T19:40:19.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.515+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f997000a500 con 0x56348498e2f0 2023-10-05T19:40:19.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.515+0000 7f9981f72e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:19.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.515+0000 7f9981f72e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:19.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.515+0000 7f9981f72e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.515+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849bc7e0 con 0x56348498e2f0 2023-10-05T19:40:19.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.515+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f997000a500 con 0x56348498e2f0 2023-10-05T19:40:19.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.515+0000 7f9981f72e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:19.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.515+0000 7f9981f72e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.515+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849bc390 con 0x56348498e2f0 2023-10-05T19:40:19.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.519+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f997000a500 con 0x56348498e2f0 2023-10-05T19:40:19.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.519+0000 7f9981f72e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:19.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.519+0000 7f9981f72e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.519+0000 7f9981f72e40 1 --2- 172.21.15.12:0/2160596884 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5634849bd730 0x5634849bfb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:19.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.519+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c0250 con 0x5634849bd730 2023-10-05T19:40:19.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.519+0000 7f997faa7700 1 --2- 172.21.15.12:0/2160596884 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5634849bd730 0x5634849bfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:19.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.519+0000 7f997faa7700 1 --2- 172.21.15.12:0/2160596884 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5634849bd730 0x5634849bfb10 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:19.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.519+0000 7f997faa7700 1 -- 172.21.15.12:0/2160596884 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f9978024020 con 0x5634849bd730 2023-10-05T19:40:19.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.519+0000 7f9981f72e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:19.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.519+0000 7f9981f72e40 20 searching for the correct realm 2023-10-05T19:40:19.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.519+0000 7f9981f72e40 1 --2- 172.21.15.12:0/2160596884 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5634849c2080 0x5634849c4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:19.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.519+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c49a0 con 0x5634849c2080 2023-10-05T19:40:19.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.519+0000 7f997f2a6700 1 --2- 172.21.15.12:0/2160596884 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5634849c2080 0x5634849c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:19.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.519+0000 7f997f2a6700 1 --2- 172.21.15.12:0/2160596884 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5634849c2080 0x5634849c4460 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:19.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.523+0000 7f997f2a6700 1 -- 172.21.15.12:0/2160596884 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f9974007a50 con 0x5634849c2080 2023-10-05T19:40:19.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.523+0000 7f997f2a6700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c49a0 con 0x5634849bd730 2023-10-05T19:40:19.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.523+0000 7f997faa7700 1 -- 172.21.15.12:0/2160596884 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9978024020 con 0x5634849bd730 2023-10-05T19:40:19.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.523+0000 7f997faa7700 1 --2- 172.21.15.12:0/2160596884 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f9978027be0 0x7f99780123c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:19.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.523+0000 7f997faa7700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c49a0 con 0x7f9978027be0 2023-10-05T19:40:19.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.523+0000 7f99802a8700 1 --2- 172.21.15.12:0/2160596884 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f9978027be0 0x7f99780123c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:19.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.527+0000 7f99802a8700 1 --2- 172.21.15.12:0/2160596884 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f9978027be0 0x7f99780123c0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:19.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.527+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9970014040 con 0x7f9978027be0 2023-10-05T19:40:19.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.527+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c49a0 con 0x56348498e2f0 2023-10-05T19:40:19.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.527+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9970014040 con 0x56348498e2f0 2023-10-05T19:40:19.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.527+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c49a0 con 0x56348498e2f0 2023-10-05T19:40:19.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.527+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9970014040 con 0x56348498e2f0 2023-10-05T19:40:19.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.527+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c49a0 con 0x7f9978027be0 2023-10-05T19:40:19.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.527+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9970014040 con 0x7f9978027be0 2023-10-05T19:40:19.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.527+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c49a0 con 0x7f9978027be0 2023-10-05T19:40:19.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.531+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9970014040 con 0x7f9978027be0 2023-10-05T19:40:19.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.531+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c49a0 con 0x5634849bd730 2023-10-05T19:40:19.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.531+0000 7f997faa7700 1 -- 172.21.15.12:0/2160596884 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9978024020 con 0x5634849bd730 2023-10-05T19:40:19.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.531+0000 7f997faa7700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c49a0 con 0x5634849c2080 2023-10-05T19:40:19.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.531+0000 7f997f2a6700 1 -- 172.21.15.12:0/2160596884 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9974007a50 con 0x5634849c2080 2023-10-05T19:40:19.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.531+0000 7f997f2a6700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c49a0 con 0x56348498e2f0 2023-10-05T19:40:19.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.531+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9970014040 con 0x56348498e2f0 2023-10-05T19:40:19.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.531+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c49a0 con 0x7f9978027be0 2023-10-05T19:40:19.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.531+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9970014040 con 0x7f9978027be0 2023-10-05T19:40:19.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.531+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c49a0 con 0x56348498e2f0 2023-10-05T19:40:19.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.535+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9970014040 con 0x56348498e2f0 2023-10-05T19:40:19.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.535+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c49a0 con 0x5634849bd730 2023-10-05T19:40:19.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.535+0000 7f997faa7700 1 -- 172.21.15.12:0/2160596884 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9978024020 con 0x5634849bd730 2023-10-05T19:40:19.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.535+0000 7f997faa7700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c49a0 con 0x5634849c2080 2023-10-05T19:40:19.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.535+0000 7f997f2a6700 1 -- 172.21.15.12:0/2160596884 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9974007a50 con 0x5634849c2080 2023-10-05T19:40:19.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.535+0000 7f997f2a6700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c49a0 con 0x5634849c2080 2023-10-05T19:40:19.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.535+0000 7f997f2a6700 1 -- 172.21.15.12:0/2160596884 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9974007a50 con 0x5634849c2080 2023-10-05T19:40:19.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.535+0000 7f997f2a6700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c49a0 con 0x7f9978027be0 2023-10-05T19:40:19.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.535+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9970014040 con 0x7f9978027be0 2023-10-05T19:40:19.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.535+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c49a0 con 0x5634849c2080 2023-10-05T19:40:19.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.535+0000 7f997f2a6700 1 -- 172.21.15.12:0/2160596884 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9974007a50 con 0x5634849c2080 2023-10-05T19:40:19.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.535+0000 7f997f2a6700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c49a0 con 0x5634849c2080 2023-10-05T19:40:19.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.539+0000 7f997f2a6700 1 -- 172.21.15.12:0/2160596884 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9974007a50 con 0x5634849c2080 2023-10-05T19:40:19.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.539+0000 7f997f2a6700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c49a0 con 0x56348498e2f0 2023-10-05T19:40:19.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.539+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9970014040 con 0x56348498e2f0 2023-10-05T19:40:19.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.539+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c49a0 con 0x56348498e2f0 2023-10-05T19:40:19.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.539+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9970014040 con 0x56348498e2f0 2023-10-05T19:40:19.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.539+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c49a0 con 0x7f9978027be0 2023-10-05T19:40:19.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.539+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9970014040 con 0x7f9978027be0 2023-10-05T19:40:19.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.539+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c49a0 con 0x5634849c2080 2023-10-05T19:40:19.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.543+0000 7f997f2a6700 1 -- 172.21.15.12:0/2160596884 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9974007a50 con 0x5634849c2080 2023-10-05T19:40:19.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.543+0000 7f997f2a6700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c49a0 con 0x5634849bd730 2023-10-05T19:40:19.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.543+0000 7f997faa7700 1 -- 172.21.15.12:0/2160596884 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9978024020 con 0x5634849bd730 2023-10-05T19:40:19.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.543+0000 7f997faa7700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c49a0 con 0x56348498e2f0 2023-10-05T19:40:19.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.543+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9970014040 con 0x56348498e2f0 2023-10-05T19:40:19.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.543+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c49a0 con 0x7f9978027be0 2023-10-05T19:40:19.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.543+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9970014040 con 0x7f9978027be0 2023-10-05T19:40:19.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.543+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c49a0 con 0x5634849c2080 2023-10-05T19:40:19.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.543+0000 7f997f2a6700 1 -- 172.21.15.12:0/2160596884 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9974007a50 con 0x5634849c2080 2023-10-05T19:40:19.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.543+0000 7f997f2a6700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c49a0 con 0x7f9978027be0 2023-10-05T19:40:19.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.547+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9970014040 con 0x7f9978027be0 2023-10-05T19:40:19.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.547+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c49a0 con 0x5634849bd730 2023-10-05T19:40:19.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.547+0000 7f997faa7700 1 -- 172.21.15.12:0/2160596884 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9978024020 con 0x5634849bd730 2023-10-05T19:40:19.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.547+0000 7f997faa7700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c49a0 con 0x5634849c2080 2023-10-05T19:40:19.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.547+0000 7f997f2a6700 1 -- 172.21.15.12:0/2160596884 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9974007a50 con 0x5634849c2080 2023-10-05T19:40:19.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.547+0000 7f997f2a6700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c49a0 con 0x7f9978027be0 2023-10-05T19:40:19.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.547+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9970014040 con 0x7f9978027be0 2023-10-05T19:40:19.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.547+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c49a0 con 0x56348498e2f0 2023-10-05T19:40:19.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.551+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f9970014040 con 0x56348498e2f0 2023-10-05T19:40:19.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.551+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c49a0 con 0x5634849bd730 2023-10-05T19:40:19.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.551+0000 7f997faa7700 1 -- 172.21.15.12:0/2160596884 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f9978024020 con 0x5634849bd730 2023-10-05T19:40:19.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.551+0000 7f9981f72e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:19.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.551+0000 7f9981f72e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:19.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.551+0000 7f9981f72e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:19.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.551+0000 7f9981f72e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:19.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.551+0000 7f9981f72e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.551+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c0e80 con 0x56348498e2f0 2023-10-05T19:40:19.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.551+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9970014040 con 0x56348498e2f0 2023-10-05T19:40:19.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.551+0000 7f9981f72e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:19.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.551+0000 7f9981f72e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.551+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c09f0 con 0x56348498e2f0 2023-10-05T19:40:19.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.551+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f9970014040 con 0x56348498e2f0 2023-10-05T19:40:19.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.551+0000 7f9981f72e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:19.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.551+0000 7f9981f72e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.551+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c17a0 con 0x5634849c2080 2023-10-05T19:40:19.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.551+0000 7f997f2a6700 1 -- 172.21.15.12:0/2160596884 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f9974007a50 con 0x5634849c2080 2023-10-05T19:40:19.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.555+0000 7f9981f72e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:19.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.555+0000 7f9981f72e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.555+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c52d0 con 0x5634849c2080 2023-10-05T19:40:19.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.555+0000 7f997f2a6700 1 -- 172.21.15.12:0/2160596884 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f9974007a50 con 0x5634849c2080 2023-10-05T19:40:19.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.555+0000 7f9981f72e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:19.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.555+0000 7f9981f72e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.555+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c5680 con 0x56348498e2f0 2023-10-05T19:40:19.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.555+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9970014040 con 0x56348498e2f0 2023-10-05T19:40:19.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.555+0000 7f9981f72e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:19.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.555+0000 7f9981f72e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.555+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c5a30 con 0x56348498e2f0 2023-10-05T19:40:19.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.555+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f9970014040 con 0x56348498e2f0 2023-10-05T19:40:19.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.555+0000 7f9981f72e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:19.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.555+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c6a40 con 0x5634849c2080 2023-10-05T19:40:19.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.559+0000 7f997f2a6700 1 -- 172.21.15.12:0/2160596884 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f9974007a50 con 0x5634849c2080 2023-10-05T19:40:19.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.559+0000 7f997f2a6700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c6a40 con 0x5634849bd730 2023-10-05T19:40:19.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.559+0000 7f997faa7700 1 -- 172.21.15.12:0/2160596884 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9978024020 con 0x5634849bd730 2023-10-05T19:40:19.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.559+0000 7f997faa7700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c6a40 con 0x7f9978027be0 2023-10-05T19:40:19.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.559+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9970014040 con 0x7f9978027be0 2023-10-05T19:40:19.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.559+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c5680 con 0x56348498e2f0 2023-10-05T19:40:19.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.559+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9970014040 con 0x56348498e2f0 2023-10-05T19:40:19.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.559+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c5680 con 0x56348498e2f0 2023-10-05T19:40:19.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.559+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9970014040 con 0x56348498e2f0 2023-10-05T19:40:19.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.559+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c5680 con 0x7f9978027be0 2023-10-05T19:40:19.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.563+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9970014040 con 0x7f9978027be0 2023-10-05T19:40:19.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.563+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c5680 con 0x7f9978027be0 2023-10-05T19:40:19.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.563+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9970014040 con 0x7f9978027be0 2023-10-05T19:40:19.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.563+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c5680 con 0x5634849bd730 2023-10-05T19:40:19.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.563+0000 7f997faa7700 1 -- 172.21.15.12:0/2160596884 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9978024020 con 0x5634849bd730 2023-10-05T19:40:19.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.563+0000 7f997faa7700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c5680 con 0x5634849c2080 2023-10-05T19:40:19.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.563+0000 7f997f2a6700 1 -- 172.21.15.12:0/2160596884 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9974007a50 con 0x5634849c2080 2023-10-05T19:40:19.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.563+0000 7f997f2a6700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c5680 con 0x56348498e2f0 2023-10-05T19:40:19.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.567+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9970014040 con 0x56348498e2f0 2023-10-05T19:40:19.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.567+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c5680 con 0x7f9978027be0 2023-10-05T19:40:19.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.567+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9970014040 con 0x7f9978027be0 2023-10-05T19:40:19.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.567+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c5680 con 0x56348498e2f0 2023-10-05T19:40:19.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.567+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9970014040 con 0x56348498e2f0 2023-10-05T19:40:19.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.567+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c5680 con 0x5634849bd730 2023-10-05T19:40:19.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.567+0000 7f997faa7700 1 -- 172.21.15.12:0/2160596884 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9978024020 con 0x5634849bd730 2023-10-05T19:40:19.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.567+0000 7f997faa7700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c5680 con 0x5634849c2080 2023-10-05T19:40:19.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.567+0000 7f997f2a6700 1 -- 172.21.15.12:0/2160596884 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9974007a50 con 0x5634849c2080 2023-10-05T19:40:19.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.567+0000 7f997f2a6700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c5680 con 0x5634849c2080 2023-10-05T19:40:19.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.571+0000 7f997f2a6700 1 -- 172.21.15.12:0/2160596884 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9974007a50 con 0x5634849c2080 2023-10-05T19:40:19.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.571+0000 7f997f2a6700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c5680 con 0x7f9978027be0 2023-10-05T19:40:19.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.571+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9970014040 con 0x7f9978027be0 2023-10-05T19:40:19.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.571+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c5680 con 0x5634849c2080 2023-10-05T19:40:19.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.571+0000 7f997f2a6700 1 -- 172.21.15.12:0/2160596884 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9974007a50 con 0x5634849c2080 2023-10-05T19:40:19.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.571+0000 7f997f2a6700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c5680 con 0x5634849c2080 2023-10-05T19:40:19.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.571+0000 7f997f2a6700 1 -- 172.21.15.12:0/2160596884 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9974007a50 con 0x5634849c2080 2023-10-05T19:40:19.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.571+0000 7f997f2a6700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c5680 con 0x56348498e2f0 2023-10-05T19:40:19.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.571+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9970014040 con 0x56348498e2f0 2023-10-05T19:40:19.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.571+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c5680 con 0x56348498e2f0 2023-10-05T19:40:19.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.575+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9970014040 con 0x56348498e2f0 2023-10-05T19:40:19.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.575+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c5680 con 0x7f9978027be0 2023-10-05T19:40:19.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.575+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9970014040 con 0x7f9978027be0 2023-10-05T19:40:19.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.575+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c5680 con 0x5634849c2080 2023-10-05T19:40:19.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.575+0000 7f997f2a6700 1 -- 172.21.15.12:0/2160596884 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9974007a50 con 0x5634849c2080 2023-10-05T19:40:19.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.575+0000 7f997f2a6700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c5680 con 0x5634849bd730 2023-10-05T19:40:19.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.575+0000 7f997faa7700 1 -- 172.21.15.12:0/2160596884 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9978024020 con 0x5634849bd730 2023-10-05T19:40:19.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.575+0000 7f997faa7700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c5680 con 0x56348498e2f0 2023-10-05T19:40:19.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.575+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9970014040 con 0x56348498e2f0 2023-10-05T19:40:19.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.575+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c5680 con 0x7f9978027be0 2023-10-05T19:40:19.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.579+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9970014040 con 0x7f9978027be0 2023-10-05T19:40:19.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.579+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c5680 con 0x5634849c2080 2023-10-05T19:40:19.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.579+0000 7f997f2a6700 1 -- 172.21.15.12:0/2160596884 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9974007a50 con 0x5634849c2080 2023-10-05T19:40:19.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.579+0000 7f997f2a6700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c5680 con 0x7f9978027be0 2023-10-05T19:40:19.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.579+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9970014040 con 0x7f9978027be0 2023-10-05T19:40:19.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.579+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c5680 con 0x5634849bd730 2023-10-05T19:40:19.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.579+0000 7f997faa7700 1 -- 172.21.15.12:0/2160596884 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9978024020 con 0x5634849bd730 2023-10-05T19:40:19.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.579+0000 7f997faa7700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c5680 con 0x5634849c2080 2023-10-05T19:40:19.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.579+0000 7f997f2a6700 1 -- 172.21.15.12:0/2160596884 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9974007a50 con 0x5634849c2080 2023-10-05T19:40:19.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.579+0000 7f997f2a6700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c5680 con 0x7f9978027be0 2023-10-05T19:40:19.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.583+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9970014040 con 0x7f9978027be0 2023-10-05T19:40:19.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.583+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c5680 con 0x56348498e2f0 2023-10-05T19:40:19.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.583+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f9970014040 con 0x56348498e2f0 2023-10-05T19:40:19.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.583+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c5680 con 0x5634849bd730 2023-10-05T19:40:19.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.583+0000 7f997faa7700 1 -- 172.21.15.12:0/2160596884 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f9978024020 con 0x5634849bd730 2023-10-05T19:40:19.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.583+0000 7f9981f72e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:19.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.583+0000 7f9981f72e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:19.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.583+0000 7f9981f72e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:19.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.583+0000 7f9981f72e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:19.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.583+0000 7f9981f72e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.583+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c5a30 con 0x56348498e2f0 2023-10-05T19:40:19.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.583+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f9970014040 con 0x56348498e2f0 2023-10-05T19:40:19.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.583+0000 7f9981f72e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:19.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.583+0000 7f9981f72e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.583+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c5de0 con 0x5634849bd730 2023-10-05T19:40:19.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.587+0000 7f997faa7700 1 -- 172.21.15.12:0/2160596884 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f9978024020 con 0x5634849bd730 2023-10-05T19:40:19.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.587+0000 7f9981f72e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:19.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.587+0000 7f9981f72e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.587+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c6680 con 0x56348498e2f0 2023-10-05T19:40:19.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.587+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f9970014040 con 0x56348498e2f0 2023-10-05T19:40:19.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.587+0000 7f9981f72e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:19.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.587+0000 7f9981f72e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.587+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c6680 con 0x5634849c2080 2023-10-05T19:40:19.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.587+0000 7f997f2a6700 1 -- 172.21.15.12:0/2160596884 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f9974007a50 con 0x5634849c2080 2023-10-05T19:40:19.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.587+0000 7f9981f72e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:19.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.587+0000 7f9981f72e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.587+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c6f20 con 0x7f9978027be0 2023-10-05T19:40:19.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.587+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f9970014040 con 0x7f9978027be0 2023-10-05T19:40:19.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.587+0000 7f9981f72e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:19.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.587+0000 7f9981f72e40 20 zone default found 2023-10-05T19:40:19.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.587+0000 7f9981f72e40 4 Realm: () 2023-10-05T19:40:19.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.587+0000 7f9981f72e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:19.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.587+0000 7f9981f72e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:19.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.587+0000 7f9981f72e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:19.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.587+0000 7f9981f72e40 20 zonegroup default 2023-10-05T19:40:19.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.587+0000 7f9981f72e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.587+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c6f20 con 0x7f9978027be0 2023-10-05T19:40:19.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.587+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f9970014040 con 0x7f9978027be0 2023-10-05T19:40:19.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.587+0000 7f9981f72e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:19.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.587+0000 7f9981f72e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.587+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c88a0 con 0x5634849c2080 2023-10-05T19:40:19.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.591+0000 7f997f2a6700 1 -- 172.21.15.12:0/2160596884 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f9974007a50 con 0x5634849c2080 2023-10-05T19:40:19.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.591+0000 7f9981f72e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:19.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.591+0000 7f9981f72e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.591+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c8c50 con 0x5634849c2080 2023-10-05T19:40:19.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.591+0000 7f997f2a6700 1 -- 172.21.15.12:0/2160596884 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f9974007a50 con 0x5634849c2080 2023-10-05T19:40:19.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.591+0000 7f9981f72e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:19.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.591+0000 7f9981f72e40 20 started sync module instance, tier type = 2023-10-05T19:40:19.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.591+0000 7f9981f72e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:19.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.591+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c7f60 con 0x5634849bd730 2023-10-05T19:40:19.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.595+0000 7f997faa7700 1 -- 172.21.15.12:0/2160596884 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'42 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9978024020 con 0x5634849bd730 2023-10-05T19:40:19.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.595+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94783563135600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c9000 con 0x5634849bd730 2023-10-05T19:40:19.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.595+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c98a0 con 0x56348498e2f0 2023-10-05T19:40:19.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.595+0000 7f997faa7700 1 -- 172.21.15.12:0/2160596884 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94783563135600] v28'43 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9978024020 con 0x5634849bd730 2023-10-05T19:40:19.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.595+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'42 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9970014040 con 0x56348498e2f0 2023-10-05T19:40:19.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.595+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94783563137888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c98a0 con 0x56348498e2f0 2023-10-05T19:40:19.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.595+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849ca2f0 con 0x5634849c2080 2023-10-05T19:40:19.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.599+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94783563137888] v28'43 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9970014040 con 0x56348498e2f0 2023-10-05T19:40:19.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.599+0000 7f997f2a6700 1 -- 172.21.15.12:0/2160596884 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'42 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9974007a50 con 0x5634849c2080 2023-10-05T19:40:19.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.599+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94783563142064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849c9f20 con 0x5634849c2080 2023-10-05T19:40:19.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.599+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849cb0d0 con 0x5634849bd730 2023-10-05T19:40:19.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.603+0000 7f997f2a6700 1 -- 172.21.15.12:0/2160596884 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94783563142064] v28'43 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9974007a50 con 0x5634849c2080 2023-10-05T19:40:19.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.603+0000 7f997faa7700 1 -- 172.21.15.12:0/2160596884 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'42 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9978024020 con 0x5634849bd730 2023-10-05T19:40:19.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.603+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94783563147280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849cac60 con 0x5634849bd730 2023-10-05T19:40:19.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.603+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849cc010 con 0x56348498e2f0 2023-10-05T19:40:19.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.607+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'42 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9970014040 con 0x56348498e2f0 2023-10-05T19:40:19.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.607+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94783563152096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849cbf30 con 0x56348498e2f0 2023-10-05T19:40:19.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.607+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849ccfa0 con 0x7f9978027be0 2023-10-05T19:40:19.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.607+0000 7f997faa7700 1 -- 172.21.15.12:0/2160596884 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94783563147280] v28'43 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9978024020 con 0x5634849bd730 2023-10-05T19:40:19.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.607+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94783563152096] v28'43 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9970014040 con 0x56348498e2f0 2023-10-05T19:40:19.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.607+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'42 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9970014040 con 0x7f9978027be0 2023-10-05T19:40:19.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.607+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94783563150464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849cca70 con 0x7f9978027be0 2023-10-05T19:40:19.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.607+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849cd890 con 0x7f9978027be0 2023-10-05T19:40:19.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.611+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(95 notify.5 [watch watch cookie 94783563150464] v28'43 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9970014040 con 0x7f9978027be0 2023-10-05T19:40:19.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.611+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(96 notify.6 [create] v28'42 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9970014040 con 0x7f9978027be0 2023-10-05T19:40:19.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.611+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94783563153760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849cca70 con 0x7f9978027be0 2023-10-05T19:40:19.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.611+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849cdd80 con 0x7f9978027be0 2023-10-05T19:40:19.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.615+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(97 notify.6 [watch watch cookie 94783563153760] v28'43 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9970014040 con 0x7f9978027be0 2023-10-05T19:40:19.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.615+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(98 notify.7 [create] v28'42 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9970014040 con 0x7f9978027be0 2023-10-05T19:40:19.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.615+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94783563158320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849cdc70 con 0x7f9978027be0 2023-10-05T19:40:19.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.615+0000 7f9981f72e40 20 add_watcher() i=0 2023-10-05T19:40:19.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.615+0000 7f9981f72e40 20 add_watcher() i=1 2023-10-05T19:40:19.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.615+0000 7f9981f72e40 20 add_watcher() i=2 2023-10-05T19:40:19.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.615+0000 7f9981f72e40 20 add_watcher() i=3 2023-10-05T19:40:19.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.615+0000 7f9981f72e40 20 add_watcher() i=4 2023-10-05T19:40:19.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.615+0000 7f9981f72e40 20 add_watcher() i=5 2023-10-05T19:40:19.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.615+0000 7f9981f72e40 20 add_watcher() i=6 2023-10-05T19:40:19.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.615+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94783563158320] v28'43 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9970014040 con 0x7f9978027be0 2023-10-05T19:40:19.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.615+0000 7f9981f72e40 20 add_watcher() i=7 2023-10-05T19:40:19.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.615+0000 7f9981f72e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:19.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.615+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849d0eb0 con 0x56348498e2f0 2023-10-05T19:40:19.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.619+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f9970014040 con 0x56348498e2f0 2023-10-05T19:40:19.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.619+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94783563159792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5634849ce230 con 0x56348498e2f0 2023-10-05T19:40:19.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.619+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94783563159792] v28'62 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f9970014040 con 0x56348498e2f0 2023-10-05T19:40:19.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.619+0000 7f99517e2700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:19.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.619+0000 7f9981f72e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:19.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.619+0000 7f9981f72e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:19.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.619+0000 7f9981f72e40 1 Processor -- start 2023-10-05T19:40:19.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.619+0000 7f9981f72e40 1 -- start start 2023-10-05T19:40:19.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.619+0000 7f9981f72e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5634849db360 0x5634849db730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:19.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.619+0000 7f9981f72e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634849dbc70 con 0x5634849db360 2023-10-05T19:40:19.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.623+0000 7f997f2a6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5634849db360 0x5634849db730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:19.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.623+0000 7f997f2a6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5634849db360 0x5634849db730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36930/0 (socket says 172.21.15.12:36930) 2023-10-05T19:40:19.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.623+0000 7f997f2a6700 1 -- 172.21.15.12:0/2961739821 learned_addr learned my addr 172.21.15.12:0/2961739821 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:19.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.623+0000 7f997f2a6700 1 -- 172.21.15.12:0/2961739821 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634849dbdb0 con 0x5634849db360 2023-10-05T19:40:19.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.623+0000 7f997f2a6700 1 --2- 172.21.15.12:0/2961739821 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5634849db360 0x5634849db730 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9974002000 tx=0x7f9974011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffeb128c44751ac2 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:19.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.623+0000 7f9950fe1700 1 -- 172.21.15.12:0/2961739821 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9974031040 con 0x5634849db360 2023-10-05T19:40:19.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.623+0000 7f9950fe1700 1 -- 172.21.15.12:0/2961739821 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9974041830 con 0x5634849db360 2023-10-05T19:40:19.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.623+0000 7f9950fe1700 1 -- 172.21.15.12:0/2961739821 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9974025040 con 0x5634849db360 2023-10-05T19:40:19.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.623+0000 7f9981f72e40 1 -- 172.21.15.12:0/2961739821 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5634849db360 msgr2=0x5634849db730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.623+0000 7f9981f72e40 1 --2- 172.21.15.12:0/2961739821 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5634849db360 0x5634849db730 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9974002000 tx=0x7f9974011000 comp rx=0 tx=0).stop 2023-10-05T19:40:19.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.623+0000 7f9981f72e40 1 -- 172.21.15.12:0/2961739821 shutdown_connections 2023-10-05T19:40:19.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.623+0000 7f9981f72e40 1 --2- 172.21.15.12:0/2961739821 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5634849db360 0x5634849db730 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:19.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.623+0000 7f9981f72e40 1 -- 172.21.15.12:0/2961739821 >> 172.21.15.12:0/2961739821 conn(0x5634849d6720 msgr2=0x5634849d48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:19.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.623+0000 7f9981f72e40 1 -- 172.21.15.12:0/2961739821 shutdown_connections 2023-10-05T19:40:19.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.623+0000 7f9981f72e40 1 -- 172.21.15.12:0/2961739821 wait complete. 2023-10-05T19:40:19.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.623+0000 7f9981f72e40 1 Processor -- start 2023-10-05T19:40:19.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.623+0000 7f9981f72e40 1 -- start start 2023-10-05T19:40:19.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.623+0000 7f9981f72e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5634849db360 0x5634849e0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:19.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.627+0000 7f9981f72e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634849e1e80 con 0x5634849db360 2023-10-05T19:40:19.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.627+0000 7f997f2a6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5634849db360 0x5634849e0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:19.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.627+0000 7f997f2a6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5634849db360 0x5634849e0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36936/0 (socket says 172.21.15.12:36936) 2023-10-05T19:40:19.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.627+0000 7f997f2a6700 1 -- 172.21.15.12:0/367076060 learned_addr learned my addr 172.21.15.12:0/367076060 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:19.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.627+0000 7f997f2a6700 1 -- 172.21.15.12:0/367076060 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563484991c20 con 0x5634849db360 2023-10-05T19:40:19.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.627+0000 7f997f2a6700 1 --2- 172.21.15.12:0/367076060 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5634849db360 0x5634849e0e80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f997400c010 tx=0x7f9974044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:19.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.627+0000 7f98caffd700 1 -- 172.21.15.12:0/367076060 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9974025070 con 0x5634849db360 2023-10-05T19:40:19.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.627+0000 7f98caffd700 1 -- 172.21.15.12:0/367076060 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f997402ce40 con 0x5634849db360 2023-10-05T19:40:19.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.627+0000 7f9981f72e40 1 -- 172.21.15.12:0/367076060 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5634849e1480 con 0x5634849db360 2023-10-05T19:40:19.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.627+0000 7f98caffd700 1 -- 172.21.15.12:0/367076060 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9974031040 con 0x5634849db360 2023-10-05T19:40:19.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.627+0000 7f9981f72e40 1 -- 172.21.15.12:0/367076060 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5634849e1920 con 0x5634849db360 2023-10-05T19:40:19.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.627+0000 7f98caffd700 1 -- 172.21.15.12:0/367076060 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f9974034040 con 0x5634849db360 2023-10-05T19:40:19.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.627+0000 7f98caffd700 1 --2- 172.21.15.12:0/367076060 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f98c0030bb0 0x7f98c0033060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:19.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.627+0000 7f98caffd700 1 -- 172.21.15.12:0/367076060 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f997405db00 con 0x5634849db360 2023-10-05T19:40:19.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.627+0000 7f99802a8700 1 --2- 172.21.15.12:0/367076060 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f98c0030bb0 0x7f98c0033060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:19.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.631+0000 7f9981f72e40 5 note: GC not initialized 2023-10-05T19:40:19.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.631+0000 7f99802a8700 1 --2- 172.21.15.12:0/367076060 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f98c0030bb0 0x7f98c0033060 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x5634849d0f60 tx=0x7f9970005760 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:19.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.631+0000 7f98bffff700 20 reqs_thread_entry: start 2023-10-05T19:40:19.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.631+0000 7f9981f72e40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:19.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.631+0000 7f98bdffb700 20 reqs_thread_entry: start 2023-10-05T19:40:19.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.631+0000 7f9981f72e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:19.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.631+0000 7f98abfff700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:49 2023 2023-10-05T19:40:19.641 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:19.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.631+0000 7f98ab7fe700 20 reqs_thread_entry: start 2023-10-05T19:40:19.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.631+0000 7f9981f72e40 20 get_system_obj_state: rctx=0x7fff6e04ee18 obj=default.rgw.meta:users.uid:foo state=0x5634848630b0 s->prefetch_data=0 2023-10-05T19:40:19.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.631+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563484a497d0 con 0x56348498e2f0 2023-10-05T19:40:19.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.635+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(102 foo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7f9970014040 con 0x56348498e2f0 2023-10-05T19:40:19.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.635+0000 7f9981f72e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:19.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.635+0000 7f9981f72e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:19.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.635+0000 7f9981f72e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.635+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563484a497a0 con 0x56348498e2f0 2023-10-05T19:40:19.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.635+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(103 foo [call,call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv3 ondisk = 0) v8 ==== 273+0+480 (crc 0 0 0) 0x7f9970014040 con 0x56348498e2f0 2023-10-05T19:40:19.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.635+0000 7f9981f72e40 20 rados_obj.operate() r=0 bl.length=357 2023-10-05T19:40:19.644 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:40:19.644 INFO:teuthology.orchestra.run.smithi012.stdout: "user_id": "foo", 2023-10-05T19:40:19.644 INFO:teuthology.orchestra.run.smithi012.stdout: "display_name": "Foo", 2023-10-05T19:40:19.644 INFO:teuthology.orchestra.run.smithi012.stdout: "email": "foo@foo.com", 2023-10-05T19:40:19.644 INFO:teuthology.orchestra.run.smithi012.stdout: "suspended": 0, 2023-10-05T19:40:19.644 INFO:teuthology.orchestra.run.smithi012.stdout: "max_buckets": 4, 2023-10-05T19:40:19.644 INFO:teuthology.orchestra.run.smithi012.stdout: "subusers": [], 2023-10-05T19:40:19.645 INFO:teuthology.orchestra.run.smithi012.stdout: "keys": [ 2023-10-05T19:40:19.645 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:19.645 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:40:19.645 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2023-10-05T19:40:19.645 INFO:teuthology.orchestra.run.smithi012.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2023-10-05T19:40:19.645 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:19.645 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:19.645 INFO:teuthology.orchestra.run.smithi012.stdout: "swift_keys": [], 2023-10-05T19:40:19.645 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": [], 2023-10-05T19:40:19.645 INFO:teuthology.orchestra.run.smithi012.stdout: "op_mask": "read, write, delete", 2023-10-05T19:40:19.645 INFO:teuthology.orchestra.run.smithi012.stdout: "default_placement": "", 2023-10-05T19:40:19.645 INFO:teuthology.orchestra.run.smithi012.stdout: "default_storage_class": "", 2023-10-05T19:40:19.646 INFO:teuthology.orchestra.run.smithi012.stdout: "placement_tags": [], 2023-10-05T19:40:19.646 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket_quota": { 2023-10-05T19:40:19.646 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:19.646 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:19.646 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:19.646 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:19.646 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:19.646 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:19.646 INFO:teuthology.orchestra.run.smithi012.stdout: "user_quota": { 2023-10-05T19:40:19.646 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:19.646 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:19.647 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:19.647 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:19.647 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:19.647 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:19.647 INFO:teuthology.orchestra.run.smithi012.stdout: "temp_url_keys": [], 2023-10-05T19:40:19.647 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "rgw", 2023-10-05T19:40:19.647 INFO:teuthology.orchestra.run.smithi012.stdout: "mfa_ids": [] 2023-10-05T19:40:19.647 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:40:19.647 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:40:19.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.635+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:104 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94783563135600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563484a497a0 con 0x5634849bd730 2023-10-05T19:40:19.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.639+0000 7f997faa7700 1 -- 172.21.15.12:0/2160596884 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(104 notify.0 [watch unwatch cookie 94783563135600] v28'44 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9978024020 con 0x5634849bd730 2023-10-05T19:40:19.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.639+0000 7f9981f72e40 20 remove_watcher() i=0 2023-10-05T19:40:19.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.639+0000 7f9981f72e40 2 removed watcher, disabling cache 2023-10-05T19:40:19.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.639+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:105 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94783563137888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563484a497a0 con 0x56348498e2f0 2023-10-05T19:40:19.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.639+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(105 notify.1 [watch unwatch cookie 94783563137888] v28'44 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9970014040 con 0x56348498e2f0 2023-10-05T19:40:19.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.639+0000 7f9981f72e40 20 remove_watcher() i=1 2023-10-05T19:40:19.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.639+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:106 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94783563142064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563484a497a0 con 0x5634849c2080 2023-10-05T19:40:19.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.643+0000 7f997f2a6700 1 -- 172.21.15.12:0/2160596884 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(106 notify.2 [watch unwatch cookie 94783563142064] v28'44 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f997404e070 con 0x5634849c2080 2023-10-05T19:40:19.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.643+0000 7f9981f72e40 20 remove_watcher() i=2 2023-10-05T19:40:19.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.643+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:107 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94783563147280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563484a497a0 con 0x5634849bd730 2023-10-05T19:40:19.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.647+0000 7f997faa7700 1 -- 172.21.15.12:0/2160596884 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(107 notify.3 [watch unwatch cookie 94783563147280] v28'44 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9978024020 con 0x5634849bd730 2023-10-05T19:40:19.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.647+0000 7f9981f72e40 20 remove_watcher() i=3 2023-10-05T19:40:19.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.647+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:108 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94783563152096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563484a497a0 con 0x56348498e2f0 2023-10-05T19:40:19.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.651+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(108 notify.4 [watch unwatch cookie 94783563152096] v28'44 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9970014040 con 0x56348498e2f0 2023-10-05T19:40:19.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.651+0000 7f9981f72e40 20 remove_watcher() i=4 2023-10-05T19:40:19.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.651+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:109 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94783563150464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563484a497a0 con 0x7f9978027be0 2023-10-05T19:40:19.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.651+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(109 notify.5 [watch unwatch cookie 94783563150464] v28'44 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9970014040 con 0x7f9978027be0 2023-10-05T19:40:19.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.651+0000 7f9981f72e40 20 remove_watcher() i=5 2023-10-05T19:40:19.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.651+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:110 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94783563153760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563484a497a0 con 0x7f9978027be0 2023-10-05T19:40:19.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.655+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(110 notify.6 [watch unwatch cookie 94783563153760] v28'44 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9970014040 con 0x7f9978027be0 2023-10-05T19:40:19.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.655+0000 7f9981f72e40 20 remove_watcher() i=6 2023-10-05T19:40:19.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.655+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:111 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94783563158320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563484a497a0 con 0x7f9978027be0 2023-10-05T19:40:19.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.659+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(111 notify.7 [watch unwatch cookie 94783563158320] v28'44 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9970014040 con 0x7f9978027be0 2023-10-05T19:40:19.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.659+0000 7f9981f72e40 20 remove_watcher() i=7 2023-10-05T19:40:19.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.659+0000 7f99517e2700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:19.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.659+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:112 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94783563159792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563484a08b50 con 0x56348498e2f0 2023-10-05T19:40:19.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.663+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(112 data_loggenerations_metadata [watch unwatch cookie 94783563159792] v28'63 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f9970014040 con 0x56348498e2f0 2023-10-05T19:40:19.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.663+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f9978027be0 msgr2=0x7f99780123c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.663+0000 7f9981f72e40 1 --2- 172.21.15.12:0/2160596884 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f9978027be0 0x7f99780123c0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:19.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.663+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x56348498e2f0 msgr2=0x5634849bb600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.663+0000 7f9981f72e40 1 --2- 172.21.15.12:0/2160596884 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x56348498e2f0 0x5634849bb600 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:19.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.663+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5634849bd730 msgr2=0x5634849bfb10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.663+0000 7f9981f72e40 1 --2- 172.21.15.12:0/2160596884 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5634849bd730 0x5634849bfb10 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:19.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.663+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5634849c2080 msgr2=0x5634849c4460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.663+0000 7f9981f72e40 1 --2- 172.21.15.12:0/2160596884 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5634849c2080 0x5634849c4460 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:19.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.663+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f996c030c50 msgr2=0x7f996c033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.663+0000 7f9981f72e40 1 --2- 172.21.15.12:0/2160596884 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f996c030c50 0x7f996c033100 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9974005cc0 tx=0x7f9974014000 comp rx=0 tx=0).stop 2023-10-05T19:40:19.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.663+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5634848fe3d0 msgr2=0x5634849afb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.663+0000 7f9981f72e40 1 --2- 172.21.15.12:0/2160596884 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5634848fe3d0 0x5634849afb70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f997800f000 tx=0x7f9978010000 comp rx=0 tx=0).stop 2023-10-05T19:40:19.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.663+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 reap_dead start 2023-10-05T19:40:19.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.663+0000 7f99802a8700 1 -- 172.21.15.12:0/2160596884 reap_dead start 2023-10-05T19:40:19.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.663+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 shutdown_connections 2023-10-05T19:40:19.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.663+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 >> 172.21.15.12:0/2160596884 conn(0x563484984720 msgr2=0x5634849a33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:19.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.663+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 shutdown_connections 2023-10-05T19:40:19.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.663+0000 7f9981f72e40 1 -- 172.21.15.12:0/2160596884 wait complete. 2023-10-05T19:40:19.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.663+0000 7f9981f72e40 1 -- 172.21.15.12:0/367076060 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f98c0030bb0 msgr2=0x7f98c0033060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.663+0000 7f9981f72e40 1 --2- 172.21.15.12:0/367076060 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f98c0030bb0 0x7f98c0033060 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x5634849d0f60 tx=0x7f9970005760 comp rx=0 tx=0).stop 2023-10-05T19:40:19.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.663+0000 7f9981f72e40 1 -- 172.21.15.12:0/367076060 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5634849db360 msgr2=0x5634849e0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.663+0000 7f9981f72e40 1 --2- 172.21.15.12:0/367076060 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5634849db360 0x5634849e0e80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f997400c010 tx=0x7f9974044000 comp rx=0 tx=0).stop 2023-10-05T19:40:19.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.663+0000 7f9981f72e40 1 -- 172.21.15.12:0/367076060 shutdown_connections 2023-10-05T19:40:19.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.663+0000 7f9981f72e40 1 --2- 172.21.15.12:0/367076060 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f98c0030bb0 0x7f98c0033060 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:19.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.663+0000 7f9981f72e40 1 --2- 172.21.15.12:0/367076060 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5634849db360 0x5634849e0e80 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:19.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.663+0000 7f9981f72e40 1 -- 172.21.15.12:0/367076060 >> 172.21.15.12:0/367076060 conn(0x5634849d6720 msgr2=0x5634849d4820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:19.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.663+0000 7f9981f72e40 1 -- 172.21.15.12:0/367076060 shutdown_connections 2023-10-05T19:40:19.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.663+0000 7f9981f72e40 1 -- 172.21.15.12:0/367076060 wait complete. 2023-10-05T19:40:19.679 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:40:19.680 INFO:tasks.util.rgw:rgwadmin: client.0 : ['key', 'create', '--uid', 'foo', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'] 2023-10-05T19:40:19.680 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'key', 'create', '--uid', 'foo', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'] 2023-10-05T19:40:19.680 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph key create --uid foo --access-key p5YnriCv1nAtykxBrupQ --secret Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh 2023-10-05T19:40:19.767 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:19.767 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:19.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.759+0000 7f2d2d23fe40 1 Processor -- start 2023-10-05T19:40:19.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.759+0000 7f2d2d23fe40 1 -- start start 2023-10-05T19:40:19.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.759+0000 7f2d2d23fe40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55ff1ba71bc0 0x55ff1ba701a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:19.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.759+0000 7f2d2d23fe40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55ff1ba48bd0 con 0x55ff1ba706e0 2023-10-05T19:40:19.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.759+0000 7f2d2d23fe40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x55ff1ba05ee0 con 0x55ff1ba71bc0 2023-10-05T19:40:19.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.759+0000 7f2d2ad74700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55ff1ba71bc0 0x55ff1ba701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:19.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.759+0000 7f2d2a573700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x55ff1ba706e0 0x55ff1bb03ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:47276/0 (socket says 172.21.15.12:47276) 2023-10-05T19:40:19.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.759+0000 7f2d2a573700 1 -- 172.21.15.12:0/4247118950 learned_addr learned my addr 172.21.15.12:0/4247118950 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:19.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.759+0000 7f2d29d72700 1 -- 172.21.15.12:0/4247118950 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1165228946 0 0) 0x55ff1ba48bd0 con 0x55ff1ba706e0 2023-10-05T19:40:19.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.759+0000 7f2d29d72700 1 -- 172.21.15.12:0/4247118950 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d0c001e10 con 0x55ff1ba706e0 2023-10-05T19:40:19.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.759+0000 7f2d2ad74700 1 -- 172.21.15.12:0/4247118950 >> v1:172.21.15.8:6789/0 conn(0x55ff1ba706e0 legacy=0x55ff1bb03ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.759+0000 7f2d2ad74700 1 -- 172.21.15.12:0/4247118950 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff1ba72050 con 0x55ff1ba71bc0 2023-10-05T19:40:19.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.763+0000 7f2d2ad74700 1 --2- 172.21.15.12:0/4247118950 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff1ba71bc0 0x55ff1ba701a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2d2400a5d0 tx=0x7f2d24005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4df7a3b04259b772 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:19.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.763+0000 7f2d29d72700 1 -- 172.21.15.12:0/4247118950 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d24007610 con 0x55ff1ba71bc0 2023-10-05T19:40:19.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.763+0000 7f2d29d72700 1 -- 172.21.15.12:0/4247118950 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d24007770 con 0x55ff1ba71bc0 2023-10-05T19:40:19.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.763+0000 7f2d29d72700 1 -- 172.21.15.12:0/4247118950 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d24007a40 con 0x55ff1ba71bc0 2023-10-05T19:40:19.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.763+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/4247118950 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff1ba71bc0 msgr2=0x55ff1ba701a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.763+0000 7f2d2d23fe40 1 --2- 172.21.15.12:0/4247118950 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff1ba71bc0 0x55ff1ba701a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2d2400a5d0 tx=0x7f2d24005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:19.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.763+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/4247118950 shutdown_connections 2023-10-05T19:40:19.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.763+0000 7f2d2d23fe40 1 --2- 172.21.15.12:0/4247118950 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff1ba71bc0 0x55ff1ba701a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:19.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.763+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/4247118950 >> 172.21.15.12:0/4247118950 conn(0x55ff1baf77f0 msgr2=0x55ff1baf9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:19.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.763+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/4247118950 shutdown_connections 2023-10-05T19:40:19.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.763+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/4247118950 wait complete. 2023-10-05T19:40:19.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.763+0000 7f2d2d23fe40 1 Processor -- start 2023-10-05T19:40:19.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.763+0000 7f2d2d23fe40 1 -- start start 2023-10-05T19:40:19.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.763+0000 7f2d2d23fe40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff1ba706e0 0x55ff1bb1d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:19.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.763+0000 7f2d2d23fe40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff1b9c9e30 con 0x55ff1ba706e0 2023-10-05T19:40:19.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.763+0000 7f2d2ad74700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff1ba706e0 0x55ff1bb1d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:19.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.763+0000 7f2d2ad74700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff1ba706e0 0x55ff1bb1d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36958/0 (socket says 172.21.15.12:36958) 2023-10-05T19:40:19.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.763+0000 7f2d2ad74700 1 -- 172.21.15.12:0/2751573771 learned_addr learned my addr 172.21.15.12:0/2751573771 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:19.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.763+0000 7f2d2ad74700 1 -- 172.21.15.12:0/2751573771 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d24005cf0 con 0x55ff1ba706e0 2023-10-05T19:40:19.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.763+0000 7f2d2ad74700 1 --2- 172.21.15.12:0/2751573771 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff1ba706e0 0x55ff1bb1d020 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2d24000c00 tx=0x7f2d24014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa1bcfb8e7587228 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:19.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.767+0000 7f2d28d70700 1 -- 172.21.15.12:0/2751573771 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d24021070 con 0x55ff1ba706e0 2023-10-05T19:40:19.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.767+0000 7f2d28d70700 1 -- 172.21.15.12:0/2751573771 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d240078e0 con 0x55ff1ba706e0 2023-10-05T19:40:19.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.767+0000 7f2d28d70700 1 -- 172.21.15.12:0/2751573771 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d24007bd0 con 0x55ff1ba706e0 2023-10-05T19:40:19.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.767+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/2751573771 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff1ba706e0 msgr2=0x55ff1bb1d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.767+0000 7f2d2d23fe40 1 --2- 172.21.15.12:0/2751573771 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff1ba706e0 0x55ff1bb1d020 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2d24000c00 tx=0x7f2d24014000 comp rx=0 tx=0).stop 2023-10-05T19:40:19.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.767+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/2751573771 shutdown_connections 2023-10-05T19:40:19.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.767+0000 7f2d2d23fe40 1 --2- 172.21.15.12:0/2751573771 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff1ba706e0 0x55ff1bb1d020 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:19.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.767+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/2751573771 >> 172.21.15.12:0/2751573771 conn(0x55ff1baf77f0 msgr2=0x55ff1bb18b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:19.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.767+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/2751573771 shutdown_connections 2023-10-05T19:40:19.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.767+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/2751573771 wait complete. 2023-10-05T19:40:19.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.767+0000 7f2d2d23fe40 1 Processor -- start 2023-10-05T19:40:19.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.767+0000 7f2d2d23fe40 1 -- start start 2023-10-05T19:40:19.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.767+0000 7f2d2d23fe40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff1ba706e0 0x55ff1bb23cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:19.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.767+0000 7f2d2d23fe40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff1b99ced0 con 0x55ff1ba706e0 2023-10-05T19:40:19.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.767+0000 7f2d2ad74700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff1ba706e0 0x55ff1bb23cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:19.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.767+0000 7f2d2ad74700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff1ba706e0 0x55ff1bb23cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36962/0 (socket says 172.21.15.12:36962) 2023-10-05T19:40:19.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.767+0000 7f2d2ad74700 1 -- 172.21.15.12:0/528765544 learned_addr learned my addr 172.21.15.12:0/528765544 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:19.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.767+0000 7f2d2ad74700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff1ba72050 con 0x55ff1ba706e0 2023-10-05T19:40:19.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.767+0000 7f2d2ad74700 1 --2- 172.21.15.12:0/528765544 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff1ba706e0 0x55ff1bb23cd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2d24005cc0 tx=0x7f2d24003050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:19.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.771+0000 7f2d22ffd700 1 -- 172.21.15.12:0/528765544 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d24021040 con 0x55ff1ba706e0 2023-10-05T19:40:19.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.771+0000 7f2d22ffd700 1 -- 172.21.15.12:0/528765544 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d24026400 con 0x55ff1ba706e0 2023-10-05T19:40:19.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.771+0000 7f2d22ffd700 1 -- 172.21.15.12:0/528765544 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d240266f0 con 0x55ff1ba706e0 2023-10-05T19:40:19.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.771+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ff1bb24210 con 0x55ff1ba706e0 2023-10-05T19:40:19.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.771+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff1bb24650 con 0x55ff1ba706e0 2023-10-05T19:40:19.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.771+0000 7f2d22ffd700 1 -- 172.21.15.12:0/528765544 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f2d24024040 con 0x55ff1ba706e0 2023-10-05T19:40:19.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.771+0000 7f2d22ffd700 1 --2- 172.21.15.12:0/528765544 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2d10030c50 0x7f2d10033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:19.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.771+0000 7f2d22ffd700 1 -- 172.21.15.12:0/528765544 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f2d24045f30 con 0x55ff1ba706e0 2023-10-05T19:40:19.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.771+0000 7f2d2a573700 1 --2- 172.21.15.12:0/528765544 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2d10030c50 0x7f2d10033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:19.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.771+0000 7f2d2d23fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.771+0000 7f2d2d23fe40 1 --2- 172.21.15.12:0/528765544 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55ff1ba71bc0 0x55ff1bb2f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:19.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.771+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb2fe20 con 0x55ff1ba71bc0 2023-10-05T19:40:19.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.771+0000 7f2d2b575700 1 --2- 172.21.15.12:0/528765544 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55ff1ba71bc0 0x55ff1bb2f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:19.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.775+0000 7f2d2a573700 1 --2- 172.21.15.12:0/528765544 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2d10030c50 0x7f2d10033100 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2d140044f0 tx=0x7f2d14006690 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:19.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.775+0000 7f2d2b575700 1 --2- 172.21.15.12:0/528765544 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55ff1ba71bc0 0x55ff1bb2f6f0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:19.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.775+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2d1c00a500 con 0x55ff1ba71bc0 2023-10-05T19:40:19.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.775+0000 7f2d2d23fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:19.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.775+0000 7f2d2d23fe40 20 realm 2023-10-05T19:40:19.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.775+0000 7f2d2d23fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.775+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb2cd20 con 0x55ff1ba71bc0 2023-10-05T19:40:19.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.775+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2d1c00a500 con 0x55ff1ba71bc0 2023-10-05T19:40:19.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.775+0000 7f2d2d23fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:19.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.775+0000 7f2d2d23fe40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:19.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.775+0000 7f2d2d23fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.775+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb30950 con 0x55ff1ba71bc0 2023-10-05T19:40:19.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.775+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2d1c00a500 con 0x55ff1ba71bc0 2023-10-05T19:40:19.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.775+0000 7f2d2d23fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:19.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.775+0000 7f2d2d23fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.775+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb30560 con 0x55ff1ba71bc0 2023-10-05T19:40:19.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.775+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f2d1c00a500 con 0x55ff1ba71bc0 2023-10-05T19:40:19.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.775+0000 7f2d2d23fe40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:19.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.775+0000 7f2d2d23fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.775+0000 7f2d2d23fe40 1 --2- 172.21.15.12:0/528765544 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55ff1bb31cf0 0x55ff1bb340d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:19.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.775+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb34610 con 0x55ff1bb31cf0 2023-10-05T19:40:19.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.775+0000 7f2d2ad74700 1 --2- 172.21.15.12:0/528765544 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55ff1bb31cf0 0x55ff1bb340d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:19.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.775+0000 7f2d2ad74700 1 --2- 172.21.15.12:0/528765544 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55ff1bb31cf0 0x55ff1bb340d0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:19.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.775+0000 7f2d2ad74700 1 -- 172.21.15.12:0/528765544 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f2d240067a0 con 0x55ff1bb31cf0 2023-10-05T19:40:19.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.775+0000 7f2d2d23fe40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:19.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.779+0000 7f2d2d23fe40 20 searching for the correct realm 2023-10-05T19:40:19.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.779+0000 7f2d2d23fe40 1 --2- 172.21.15.12:0/528765544 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55ff1bb36490 0x55ff1bb388a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:19.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.779+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb38f50 con 0x55ff1bb36490 2023-10-05T19:40:19.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.779+0000 7f2d2a573700 1 --2- 172.21.15.12:0/528765544 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55ff1bb36490 0x55ff1bb388a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:19.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.779+0000 7f2d2a573700 1 --2- 172.21.15.12:0/528765544 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55ff1bb36490 0x55ff1bb388a0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:19.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.779+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f2d14004520 con 0x55ff1bb36490 2023-10-05T19:40:19.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.779+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb38f50 con 0x55ff1bb31cf0 2023-10-05T19:40:19.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.779+0000 7f2d2ad74700 1 -- 172.21.15.12:0/528765544 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d240067a0 con 0x55ff1bb31cf0 2023-10-05T19:40:19.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.779+0000 7f2d2ad74700 1 --2- 172.21.15.12:0/528765544 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f2d24007030 0x7f2d24045880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:19.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.779+0000 7f2d2ad74700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb38f50 con 0x7f2d24007030 2023-10-05T19:40:19.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.779+0000 7f2d2b575700 1 --2- 172.21.15.12:0/528765544 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f2d24007030 0x7f2d24045880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:19.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.783+0000 7f2d2b575700 1 --2- 172.21.15.12:0/528765544 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f2d24007030 0x7f2d24045880 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:19.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.783+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d1c014040 con 0x7f2d24007030 2023-10-05T19:40:19.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.783+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb38f50 con 0x55ff1ba71bc0 2023-10-05T19:40:19.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.783+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d1c014040 con 0x55ff1ba71bc0 2023-10-05T19:40:19.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.783+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb38f50 con 0x55ff1ba71bc0 2023-10-05T19:40:19.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.783+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d1c014040 con 0x55ff1ba71bc0 2023-10-05T19:40:19.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.783+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb38f50 con 0x7f2d24007030 2023-10-05T19:40:19.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.783+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d1c014040 con 0x7f2d24007030 2023-10-05T19:40:19.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.783+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb38f50 con 0x7f2d24007030 2023-10-05T19:40:19.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.787+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d1c014040 con 0x7f2d24007030 2023-10-05T19:40:19.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.787+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb38f50 con 0x55ff1bb31cf0 2023-10-05T19:40:19.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.787+0000 7f2d2ad74700 1 -- 172.21.15.12:0/528765544 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d240067a0 con 0x55ff1bb31cf0 2023-10-05T19:40:19.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.787+0000 7f2d2ad74700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb38f50 con 0x55ff1bb36490 2023-10-05T19:40:19.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.787+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d14004520 con 0x55ff1bb36490 2023-10-05T19:40:19.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.787+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb38f50 con 0x55ff1ba71bc0 2023-10-05T19:40:19.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.787+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d1c014040 con 0x55ff1ba71bc0 2023-10-05T19:40:19.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.787+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb38f50 con 0x7f2d24007030 2023-10-05T19:40:19.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.787+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d1c014040 con 0x7f2d24007030 2023-10-05T19:40:19.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.787+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb38f50 con 0x55ff1ba71bc0 2023-10-05T19:40:19.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.791+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d1c014040 con 0x55ff1ba71bc0 2023-10-05T19:40:19.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.791+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb38f50 con 0x55ff1bb31cf0 2023-10-05T19:40:19.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.791+0000 7f2d2ad74700 1 -- 172.21.15.12:0/528765544 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d240067a0 con 0x55ff1bb31cf0 2023-10-05T19:40:19.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.791+0000 7f2d2ad74700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb38f50 con 0x55ff1bb36490 2023-10-05T19:40:19.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.791+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d14004520 con 0x55ff1bb36490 2023-10-05T19:40:19.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.791+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb38f50 con 0x55ff1bb36490 2023-10-05T19:40:19.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.791+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d14004520 con 0x55ff1bb36490 2023-10-05T19:40:19.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.791+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb38f50 con 0x7f2d24007030 2023-10-05T19:40:19.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.791+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d1c014040 con 0x7f2d24007030 2023-10-05T19:40:19.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.795+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb38f50 con 0x55ff1bb36490 2023-10-05T19:40:19.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.795+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d14004520 con 0x55ff1bb36490 2023-10-05T19:40:19.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.795+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb38f50 con 0x55ff1bb36490 2023-10-05T19:40:19.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.795+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d14004520 con 0x55ff1bb36490 2023-10-05T19:40:19.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.795+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb38f50 con 0x55ff1ba71bc0 2023-10-05T19:40:19.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.795+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d1c014040 con 0x55ff1ba71bc0 2023-10-05T19:40:19.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.795+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb38f50 con 0x55ff1ba71bc0 2023-10-05T19:40:19.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.795+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d1c014040 con 0x55ff1ba71bc0 2023-10-05T19:40:19.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.795+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb38f50 con 0x7f2d24007030 2023-10-05T19:40:19.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.799+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d1c014040 con 0x7f2d24007030 2023-10-05T19:40:19.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.799+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb38f50 con 0x55ff1bb36490 2023-10-05T19:40:19.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.799+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d14004520 con 0x55ff1bb36490 2023-10-05T19:40:19.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.799+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb38f50 con 0x55ff1bb31cf0 2023-10-05T19:40:19.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.799+0000 7f2d2ad74700 1 -- 172.21.15.12:0/528765544 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d240067a0 con 0x55ff1bb31cf0 2023-10-05T19:40:19.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.799+0000 7f2d2ad74700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb38f50 con 0x55ff1ba71bc0 2023-10-05T19:40:19.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.799+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d1c014040 con 0x55ff1ba71bc0 2023-10-05T19:40:19.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.799+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb38f50 con 0x7f2d24007030 2023-10-05T19:40:19.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.799+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d1c014040 con 0x7f2d24007030 2023-10-05T19:40:19.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.799+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb38f50 con 0x55ff1bb36490 2023-10-05T19:40:19.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.803+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d14004520 con 0x55ff1bb36490 2023-10-05T19:40:19.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.803+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb38f50 con 0x7f2d24007030 2023-10-05T19:40:19.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.803+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d1c014040 con 0x7f2d24007030 2023-10-05T19:40:19.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.803+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb38f50 con 0x55ff1bb31cf0 2023-10-05T19:40:19.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.803+0000 7f2d2ad74700 1 -- 172.21.15.12:0/528765544 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d240067a0 con 0x55ff1bb31cf0 2023-10-05T19:40:19.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.803+0000 7f2d2ad74700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb38f50 con 0x55ff1bb36490 2023-10-05T19:40:19.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.803+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d14004520 con 0x55ff1bb36490 2023-10-05T19:40:19.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.803+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb38f50 con 0x7f2d24007030 2023-10-05T19:40:19.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.807+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d1c014040 con 0x7f2d24007030 2023-10-05T19:40:19.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.807+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb38f50 con 0x55ff1ba71bc0 2023-10-05T19:40:19.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.807+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f2d1c014040 con 0x55ff1ba71bc0 2023-10-05T19:40:19.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.807+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb38f50 con 0x55ff1bb31cf0 2023-10-05T19:40:19.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.807+0000 7f2d2ad74700 1 -- 172.21.15.12:0/528765544 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f2d240067a0 con 0x55ff1bb31cf0 2023-10-05T19:40:19.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.807+0000 7f2d2d23fe40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:19.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.807+0000 7f2d2d23fe40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:19.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.807+0000 7f2d2d23fe40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:19.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.807+0000 7f2d2d23fe40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:19.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.807+0000 7f2d2d23fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.807+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb35210 con 0x55ff1ba71bc0 2023-10-05T19:40:19.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.807+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2d1c014040 con 0x55ff1ba71bc0 2023-10-05T19:40:19.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.807+0000 7f2d2d23fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:19.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.807+0000 7f2d2d23fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.807+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb34e40 con 0x55ff1ba71bc0 2023-10-05T19:40:19.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.807+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f2d1c014040 con 0x55ff1ba71bc0 2023-10-05T19:40:19.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.807+0000 7f2d2d23fe40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:19.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.807+0000 7f2d2d23fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.807+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb393c0 con 0x55ff1bb36490 2023-10-05T19:40:19.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.811+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f2d14004520 con 0x55ff1bb36490 2023-10-05T19:40:19.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.811+0000 7f2d2d23fe40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:19.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.811+0000 7f2d2d23fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.811+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb39770 con 0x55ff1bb36490 2023-10-05T19:40:19.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.811+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f2d14004520 con 0x55ff1bb36490 2023-10-05T19:40:19.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.811+0000 7f2d2d23fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:19.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.811+0000 7f2d2d23fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.811+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb39b20 con 0x55ff1ba71bc0 2023-10-05T19:40:19.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.811+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2d1c014040 con 0x55ff1ba71bc0 2023-10-05T19:40:19.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.811+0000 7f2d2d23fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:19.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.811+0000 7f2d2d23fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.811+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb35730 con 0x55ff1ba71bc0 2023-10-05T19:40:19.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.811+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f2d1c014040 con 0x55ff1ba71bc0 2023-10-05T19:40:19.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.811+0000 7f2d2d23fe40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:19.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.811+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb35730 con 0x55ff1bb36490 2023-10-05T19:40:19.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.815+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f2d14004520 con 0x55ff1bb36490 2023-10-05T19:40:19.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.815+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb35730 con 0x55ff1bb31cf0 2023-10-05T19:40:19.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.815+0000 7f2d2ad74700 1 -- 172.21.15.12:0/528765544 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d240067a0 con 0x55ff1bb31cf0 2023-10-05T19:40:19.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.815+0000 7f2d2ad74700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb35730 con 0x7f2d24007030 2023-10-05T19:40:19.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.815+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d1c014040 con 0x7f2d24007030 2023-10-05T19:40:19.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.815+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb39b20 con 0x55ff1ba71bc0 2023-10-05T19:40:19.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.819+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d1c014040 con 0x55ff1ba71bc0 2023-10-05T19:40:19.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.819+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb39b20 con 0x55ff1ba71bc0 2023-10-05T19:40:19.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.819+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d1c014040 con 0x55ff1ba71bc0 2023-10-05T19:40:19.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.819+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb39b20 con 0x7f2d24007030 2023-10-05T19:40:19.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.819+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d1c014040 con 0x7f2d24007030 2023-10-05T19:40:19.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.819+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb39b20 con 0x7f2d24007030 2023-10-05T19:40:19.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.819+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d1c014040 con 0x7f2d24007030 2023-10-05T19:40:19.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.819+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb39b20 con 0x55ff1bb31cf0 2023-10-05T19:40:19.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.819+0000 7f2d2ad74700 1 -- 172.21.15.12:0/528765544 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d240067a0 con 0x55ff1bb31cf0 2023-10-05T19:40:19.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.819+0000 7f2d2ad74700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb39b20 con 0x55ff1bb36490 2023-10-05T19:40:19.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.823+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d1402d040 con 0x55ff1bb36490 2023-10-05T19:40:19.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.823+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb39b20 con 0x55ff1ba71bc0 2023-10-05T19:40:19.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.823+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d1c014040 con 0x55ff1ba71bc0 2023-10-05T19:40:19.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.823+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb39b20 con 0x7f2d24007030 2023-10-05T19:40:19.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.823+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d1c014040 con 0x7f2d24007030 2023-10-05T19:40:19.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.823+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb39b20 con 0x55ff1ba71bc0 2023-10-05T19:40:19.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.823+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d1c014040 con 0x55ff1ba71bc0 2023-10-05T19:40:19.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.823+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb39b20 con 0x55ff1bb31cf0 2023-10-05T19:40:19.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.827+0000 7f2d2ad74700 1 -- 172.21.15.12:0/528765544 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d240067a0 con 0x55ff1bb31cf0 2023-10-05T19:40:19.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.827+0000 7f2d2ad74700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb39b20 con 0x55ff1bb36490 2023-10-05T19:40:19.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.827+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d1402d040 con 0x55ff1bb36490 2023-10-05T19:40:19.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.827+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb39b20 con 0x55ff1bb36490 2023-10-05T19:40:19.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.827+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d1402d040 con 0x55ff1bb36490 2023-10-05T19:40:19.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.827+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb39b20 con 0x7f2d24007030 2023-10-05T19:40:19.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.827+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d1c02e040 con 0x7f2d24007030 2023-10-05T19:40:19.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.827+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb39b20 con 0x55ff1bb36490 2023-10-05T19:40:19.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.827+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d1402d040 con 0x55ff1bb36490 2023-10-05T19:40:19.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.827+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb39b20 con 0x55ff1bb36490 2023-10-05T19:40:19.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.831+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d1402d040 con 0x55ff1bb36490 2023-10-05T19:40:19.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.831+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb39b20 con 0x55ff1ba71bc0 2023-10-05T19:40:19.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.831+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d1c02e040 con 0x55ff1ba71bc0 2023-10-05T19:40:19.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.831+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb39b20 con 0x55ff1ba71bc0 2023-10-05T19:40:19.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.831+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d1c02e040 con 0x55ff1ba71bc0 2023-10-05T19:40:19.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.831+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb39b20 con 0x7f2d24007030 2023-10-05T19:40:19.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.831+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d1c02e040 con 0x7f2d24007030 2023-10-05T19:40:19.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.831+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb39b20 con 0x55ff1bb36490 2023-10-05T19:40:19.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.835+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d1402d040 con 0x55ff1bb36490 2023-10-05T19:40:19.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.835+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb39b20 con 0x55ff1bb31cf0 2023-10-05T19:40:19.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.835+0000 7f2d2ad74700 1 -- 172.21.15.12:0/528765544 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d240067a0 con 0x55ff1bb31cf0 2023-10-05T19:40:19.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.835+0000 7f2d2ad74700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb39b20 con 0x55ff1ba71bc0 2023-10-05T19:40:19.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.835+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d1c02e040 con 0x55ff1ba71bc0 2023-10-05T19:40:19.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.835+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb39b20 con 0x7f2d24007030 2023-10-05T19:40:19.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.835+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d1c02e040 con 0x7f2d24007030 2023-10-05T19:40:19.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.835+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb39b20 con 0x55ff1bb36490 2023-10-05T19:40:19.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.835+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d1402d040 con 0x55ff1bb36490 2023-10-05T19:40:19.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.835+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb39b20 con 0x7f2d24007030 2023-10-05T19:40:19.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.839+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d1c02e040 con 0x7f2d24007030 2023-10-05T19:40:19.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.839+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb39b20 con 0x55ff1bb31cf0 2023-10-05T19:40:19.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.839+0000 7f2d2ad74700 1 -- 172.21.15.12:0/528765544 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d240067a0 con 0x55ff1bb31cf0 2023-10-05T19:40:19.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.839+0000 7f2d2ad74700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb39b20 con 0x55ff1bb36490 2023-10-05T19:40:19.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.839+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d1402d040 con 0x55ff1bb36490 2023-10-05T19:40:19.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.839+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb39b20 con 0x7f2d24007030 2023-10-05T19:40:19.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.839+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d1c02e040 con 0x7f2d24007030 2023-10-05T19:40:19.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.839+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb39b20 con 0x55ff1ba71bc0 2023-10-05T19:40:19.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.843+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f2d1c02e040 con 0x55ff1ba71bc0 2023-10-05T19:40:19.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.843+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb39b20 con 0x55ff1bb31cf0 2023-10-05T19:40:19.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.843+0000 7f2d2ad74700 1 -- 172.21.15.12:0/528765544 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f2d240067a0 con 0x55ff1bb31cf0 2023-10-05T19:40:19.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.843+0000 7f2d2d23fe40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:19.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.843+0000 7f2d2d23fe40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:19.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.843+0000 7f2d2d23fe40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:19.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.843+0000 7f2d2d23fe40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:19.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.843+0000 7f2d2d23fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.843+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb35730 con 0x55ff1ba71bc0 2023-10-05T19:40:19.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.843+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f2d1c02e040 con 0x55ff1ba71bc0 2023-10-05T19:40:19.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.843+0000 7f2d2d23fe40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:19.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.843+0000 7f2d2d23fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.843+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb3a4c0 con 0x55ff1bb31cf0 2023-10-05T19:40:19.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.843+0000 7f2d2ad74700 1 -- 172.21.15.12:0/528765544 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f2d240067a0 con 0x55ff1bb31cf0 2023-10-05T19:40:19.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.843+0000 7f2d2d23fe40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:19.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.843+0000 7f2d2d23fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.843+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb3adb0 con 0x55ff1ba71bc0 2023-10-05T19:40:19.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.847+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f2d1c02e040 con 0x55ff1ba71bc0 2023-10-05T19:40:19.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.847+0000 7f2d2d23fe40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:19.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.847+0000 7f2d2d23fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.847+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb3adb0 con 0x55ff1bb36490 2023-10-05T19:40:19.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.847+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f2d1402d040 con 0x55ff1bb36490 2023-10-05T19:40:19.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.847+0000 7f2d2d23fe40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:19.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.847+0000 7f2d2d23fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.847+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb3b520 con 0x7f2d24007030 2023-10-05T19:40:19.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.847+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f2d1c02e040 con 0x7f2d24007030 2023-10-05T19:40:19.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.847+0000 7f2d2d23fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:19.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.847+0000 7f2d2d23fe40 20 zone default found 2023-10-05T19:40:19.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.847+0000 7f2d2d23fe40 4 Realm: () 2023-10-05T19:40:19.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.847+0000 7f2d2d23fe40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:19.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.847+0000 7f2d2d23fe40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:19.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.847+0000 7f2d2d23fe40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:19.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.847+0000 7f2d2d23fe40 20 zonegroup default 2023-10-05T19:40:19.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.847+0000 7f2d2d23fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.847+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb3b160 con 0x7f2d24007030 2023-10-05T19:40:19.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.847+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f2d1c02e040 con 0x7f2d24007030 2023-10-05T19:40:19.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.847+0000 7f2d2d23fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:19.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.847+0000 7f2d2d23fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.847+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb3cae0 con 0x55ff1bb36490 2023-10-05T19:40:19.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.847+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f2d1402d040 con 0x55ff1bb36490 2023-10-05T19:40:19.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.851+0000 7f2d2d23fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:19.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.851+0000 7f2d2d23fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.851+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb3ce90 con 0x55ff1bb36490 2023-10-05T19:40:19.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.851+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f2d1403c040 con 0x55ff1bb36490 2023-10-05T19:40:19.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.851+0000 7f2d2d23fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:19.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.851+0000 7f2d2d23fe40 20 started sync module instance, tier type = 2023-10-05T19:40:19.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.851+0000 7f2d2d23fe40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:19.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.851+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb3c1a0 con 0x55ff1bb31cf0 2023-10-05T19:40:19.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.851+0000 7f2d2ad74700 1 -- 172.21.15.12:0/528765544 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'45 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2d240067a0 con 0x55ff1bb31cf0 2023-10-05T19:40:19.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.851+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94554169787728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb3a9e0 con 0x55ff1bb31cf0 2023-10-05T19:40:19.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.855+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb3dc50 con 0x55ff1ba71bc0 2023-10-05T19:40:19.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.855+0000 7f2d2ad74700 1 -- 172.21.15.12:0/528765544 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94554169787728] v28'46 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2d240067a0 con 0x55ff1bb31cf0 2023-10-05T19:40:19.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.855+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'45 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2d1c02e040 con 0x55ff1ba71bc0 2023-10-05T19:40:19.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.855+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94554169792352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb3dbb0 con 0x55ff1ba71bc0 2023-10-05T19:40:19.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.855+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb3ea30 con 0x55ff1bb36490 2023-10-05T19:40:19.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.859+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'45 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2d1403c040 con 0x55ff1bb36490 2023-10-05T19:40:19.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.859+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94554169795904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb3f2d0 con 0x55ff1bb36490 2023-10-05T19:40:19.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.859+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb3fb70 con 0x55ff1bb31cf0 2023-10-05T19:40:19.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.859+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94554169792352] v28'46 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2d1c02e040 con 0x55ff1ba71bc0 2023-10-05T19:40:19.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.863+0000 7f2d2ad74700 1 -- 172.21.15.12:0/528765544 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'45 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2d240067a0 con 0x55ff1bb31cf0 2023-10-05T19:40:19.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.863+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94554169794640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb3f7a0 con 0x55ff1bb31cf0 2023-10-05T19:40:19.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.863+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb40c10 con 0x55ff1ba71bc0 2023-10-05T19:40:19.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.863+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94554169795904] v28'46 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2d1403c040 con 0x55ff1bb36490 2023-10-05T19:40:19.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.863+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'45 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2d1c02e040 con 0x55ff1ba71bc0 2023-10-05T19:40:19.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.863+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94554169798112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb40bd0 con 0x55ff1ba71bc0 2023-10-05T19:40:19.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.863+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb415e0 con 0x7f2d24007030 2023-10-05T19:40:19.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.863+0000 7f2d2ad74700 1 -- 172.21.15.12:0/528765544 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94554169794640] v28'46 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2d240067a0 con 0x55ff1bb31cf0 2023-10-05T19:40:19.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.867+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94554169798112] v28'46 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2d1c02e040 con 0x55ff1ba71bc0 2023-10-05T19:40:19.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.867+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'45 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2d1c02e040 con 0x7f2d24007030 2023-10-05T19:40:19.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.867+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94554169804512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb40b30 con 0x7f2d24007030 2023-10-05T19:40:19.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.867+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb41ad0 con 0x7f2d24007030 2023-10-05T19:40:19.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.871+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(96 notify.6 [create] v28'45 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2d1c02e040 con 0x7f2d24007030 2023-10-05T19:40:19.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.871+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94554169808352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb41f20 con 0x7f2d24007030 2023-10-05T19:40:19.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.871+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb42930 con 0x7f2d24007030 2023-10-05T19:40:19.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.871+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(95 notify.5 [watch watch cookie 94554169804512] v28'46 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2d1c02e040 con 0x7f2d24007030 2023-10-05T19:40:19.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.871+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(98 notify.7 [create] v28'45 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2d1c02e040 con 0x7f2d24007030 2023-10-05T19:40:19.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.875+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94554169807088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb42930 con 0x7f2d24007030 2023-10-05T19:40:19.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.875+0000 7f2d2d23fe40 20 add_watcher() i=0 2023-10-05T19:40:19.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.875+0000 7f2d2d23fe40 20 add_watcher() i=1 2023-10-05T19:40:19.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.875+0000 7f2d2d23fe40 20 add_watcher() i=2 2023-10-05T19:40:19.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.875+0000 7f2d2d23fe40 20 add_watcher() i=3 2023-10-05T19:40:19.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.875+0000 7f2d2d23fe40 20 add_watcher() i=4 2023-10-05T19:40:19.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.875+0000 7f2d2d23fe40 20 add_watcher() i=5 2023-10-05T19:40:19.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.875+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(97 notify.6 [watch watch cookie 94554169808352] v28'46 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2d1c02e040 con 0x7f2d24007030 2023-10-05T19:40:19.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.875+0000 7f2d2d23fe40 20 add_watcher() i=6 2023-10-05T19:40:19.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.875+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94554169807088] v28'46 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2d1c02e040 con 0x7f2d24007030 2023-10-05T19:40:19.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.875+0000 7f2d2d23fe40 20 add_watcher() i=7 2023-10-05T19:40:19.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.875+0000 7f2d2d23fe40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:19.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.875+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb45960 con 0x55ff1ba71bc0 2023-10-05T19:40:19.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.875+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f2d1c02e040 con 0x55ff1ba71bc0 2023-10-05T19:40:19.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.875+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94554169812032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bb42d80 con 0x55ff1ba71bc0 2023-10-05T19:40:19.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.879+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94554169812032] v28'64 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f2d1c02e040 con 0x55ff1ba71bc0 2023-10-05T19:40:19.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.879+0000 7f2cfb7ee700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:19.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.879+0000 7f2d2d23fe40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:19.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.879+0000 7f2d2d23fe40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:19.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.879+0000 7f2d2d23fe40 1 Processor -- start 2023-10-05T19:40:19.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.879+0000 7f2d2d23fe40 1 -- start start 2023-10-05T19:40:19.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.879+0000 7f2d2d23fe40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff1bb4fdd0 0x55ff1bb501a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:19.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.879+0000 7f2d2d23fe40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff1bb506e0 con 0x55ff1bb4fdd0 2023-10-05T19:40:19.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.879+0000 7f2d2a573700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff1bb4fdd0 0x55ff1bb501a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:19.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.879+0000 7f2d2a573700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff1bb4fdd0 0x55ff1bb501a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36964/0 (socket says 172.21.15.12:36964) 2023-10-05T19:40:19.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.879+0000 7f2d2a573700 1 -- 172.21.15.12:0/1516381343 learned_addr learned my addr 172.21.15.12:0/1516381343 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:19.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.879+0000 7f2d2a573700 1 -- 172.21.15.12:0/1516381343 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff1bb50820 con 0x55ff1bb4fdd0 2023-10-05T19:40:19.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.883+0000 7f2d2a573700 1 --2- 172.21.15.12:0/1516381343 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff1bb4fdd0 0x55ff1bb501a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2d240311f0 tx=0x7f2d14027040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f772a54931d2418 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:19.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.883+0000 7f2cfafed700 1 -- 172.21.15.12:0/1516381343 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d14039040 con 0x55ff1bb4fdd0 2023-10-05T19:40:19.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.883+0000 7f2cfafed700 1 -- 172.21.15.12:0/1516381343 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d1403a040 con 0x55ff1bb4fdd0 2023-10-05T19:40:19.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.883+0000 7f2cfafed700 1 -- 172.21.15.12:0/1516381343 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d1402a080 con 0x55ff1bb4fdd0 2023-10-05T19:40:19.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.883+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/1516381343 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff1bb4fdd0 msgr2=0x55ff1bb501a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.883+0000 7f2d2d23fe40 1 --2- 172.21.15.12:0/1516381343 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff1bb4fdd0 0x55ff1bb501a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2d240311f0 tx=0x7f2d14027040 comp rx=0 tx=0).stop 2023-10-05T19:40:19.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.883+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/1516381343 shutdown_connections 2023-10-05T19:40:19.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.883+0000 7f2d2d23fe40 1 --2- 172.21.15.12:0/1516381343 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff1bb4fdd0 0x55ff1bb501a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:19.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.883+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/1516381343 >> 172.21.15.12:0/1516381343 conn(0x55ff1bb4b190 msgr2=0x55ff1bb492e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:19.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.883+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/1516381343 shutdown_connections 2023-10-05T19:40:19.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.883+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/1516381343 wait complete. 2023-10-05T19:40:19.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.883+0000 7f2d2d23fe40 1 Processor -- start 2023-10-05T19:40:19.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.883+0000 7f2d2d23fe40 1 -- start start 2023-10-05T19:40:19.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.883+0000 7f2d2d23fe40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff1bb4fdd0 0x55ff1bb559c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:19.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.883+0000 7f2d2d23fe40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff1bb569c0 con 0x55ff1bb4fdd0 2023-10-05T19:40:19.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.883+0000 7f2d2a573700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff1bb4fdd0 0x55ff1bb559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:19.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.883+0000 7f2d2a573700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff1bb4fdd0 0x55ff1bb559c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36978/0 (socket says 172.21.15.12:36978) 2023-10-05T19:40:19.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.887+0000 7f2d2a573700 1 -- 172.21.15.12:0/4206523050 learned_addr learned my addr 172.21.15.12:0/4206523050 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:19.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.887+0000 7f2d2a573700 1 -- 172.21.15.12:0/4206523050 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d0c001e10 con 0x55ff1bb4fdd0 2023-10-05T19:40:19.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.887+0000 7f2d2a573700 1 --2- 172.21.15.12:0/4206523050 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff1bb4fdd0 0x55ff1bb559c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x55ff1bb49e50 tx=0x7f2d14004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:19.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.887+0000 7f2cf97ea700 1 -- 172.21.15.12:0/4206523050 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d14039040 con 0x55ff1bb4fdd0 2023-10-05T19:40:19.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.887+0000 7f2cf97ea700 1 -- 172.21.15.12:0/4206523050 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d1402b040 con 0x55ff1bb4fdd0 2023-10-05T19:40:19.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.887+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/4206523050 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ff1bb55fc0 con 0x55ff1bb4fdd0 2023-10-05T19:40:19.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.887+0000 7f2cf97ea700 1 -- 172.21.15.12:0/4206523050 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d1401a040 con 0x55ff1bb4fdd0 2023-10-05T19:40:19.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.887+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/4206523050 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff1bb563a0 con 0x55ff1bb4fdd0 2023-10-05T19:40:19.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.887+0000 7f2cf97ea700 1 -- 172.21.15.12:0/4206523050 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f2d1404b9a0 con 0x55ff1bb4fdd0 2023-10-05T19:40:19.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.887+0000 7f2cf97ea700 1 --2- 172.21.15.12:0/4206523050 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2c70030c50 0x7f2c70033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:19.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.887+0000 7f2cf97ea700 1 -- 172.21.15.12:0/4206523050 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f2d14028040 con 0x55ff1bb4fdd0 2023-10-05T19:40:19.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.887+0000 7f2d2d23fe40 5 note: GC not initialized 2023-10-05T19:40:19.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.887+0000 7f2d2b575700 1 --2- 172.21.15.12:0/4206523050 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2c70030c50 0x7f2c70033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:19.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.891+0000 7f2d2b575700 1 --2- 172.21.15.12:0/4206523050 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2c70030c50 0x7f2c70033100 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x55ff1bb42100 tx=0x7f2d1c005500 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:19.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.891+0000 7f2c6a7fc700 20 reqs_thread_entry: start 2023-10-05T19:40:19.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.891+0000 7f2c63fff700 20 reqs_thread_entry: start 2023-10-05T19:40:19.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.891+0000 7f2d2d23fe40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:19.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.891+0000 7f2d2d23fe40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:19.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.891+0000 7f2c627fc700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:50 2023 2023-10-05T19:40:19.901 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:19.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.891+0000 7f2c61ffb700 20 reqs_thread_entry: start 2023-10-05T19:40:19.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.891+0000 7f2d2d23fe40 20 get_system_obj_state: rctx=0x7ffef4cac638 obj=default.rgw.meta:users.uid:foo state=0x55ff1b9d50b0 s->prefetch_data=0 2023-10-05T19:40:19.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.891+0000 7f2d2d23fe40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2023-10-05T19:40:19.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.891+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bbbe200 con 0x55ff1ba71bc0 2023-10-05T19:40:19.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.895+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(102 foo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7f2d1c02e040 con 0x55ff1ba71bc0 2023-10-05T19:40:19.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.895+0000 7f2d2d23fe40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x16 2023-10-05T19:40:19.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.895+0000 7f2d2d23fe40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:19.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.895+0000 7f2d2d23fe40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:19.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.895+0000 7f2d2d23fe40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:19.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.895+0000 7f2d2d23fe40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:19.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.895+0000 7f2d2d23fe40 10 cache get: name=default.rgw.meta+users.uid+foo : type miss (requested=0x13, cached=0x16) 2023-10-05T19:40:19.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.895+0000 7f2d2d23fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:19.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.895+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bbc0c90 con 0x55ff1ba71bc0 2023-10-05T19:40:19.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.895+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(103 foo [call,call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv3 ondisk = 0) v8 ==== 273+0+480 (crc 0 0 0) 0x7f2d1c02e040 con 0x55ff1ba71bc0 2023-10-05T19:40:19.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.895+0000 7f2d2d23fe40 20 rados_obj.operate() r=0 bl.length=357 2023-10-05T19:40:19.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.895+0000 7f2d2d23fe40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2023-10-05T19:40:19.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.895+0000 7f2d2d23fe40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:19.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.895+0000 7f2d2d23fe40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:19.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.895+0000 7f2d2d23fe40 20 get_system_obj_state: rctx=0x7ffef4cac0d8 obj=default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ state=0x55ff1b9d50b0 s->prefetch_data=0 2023-10-05T19:40:19.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.895+0000 7f2d2d23fe40 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : miss 2023-10-05T19:40:19.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.895+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:104 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bbbec70 con 0x7f2d24007030 2023-10-05T19:40:19.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.895+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(104 p5YnriCv1nAtykxBrupQ [getxattrs,stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 206+0+0 (crc 0 0 0) 0x7f2d1c02e040 con 0x7f2d24007030 2023-10-05T19:40:19.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.895+0000 7f2d2d23fe40 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x0 2023-10-05T19:40:19.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.895+0000 7f2d2d23fe40 10 adding default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2023-10-05T19:40:19.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.895+0000 7f2d2d23fe40 20 get_system_obj_state: rctx=0x7ffef4cac5f8 obj=default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ state=0x55ff1b9d50b0 s->prefetch_data=0 2023-10-05T19:40:19.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.895+0000 7f2d2d23fe40 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2023-10-05T19:40:19.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.895+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:105 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.set in=58b,writefull 0~459 in=459b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bbc1fd0 con 0x55ff1ba71bc0 2023-10-05T19:40:19.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.899+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(105 foo [delete,create,call,writefull 0~459] v28'4 uv4 ondisk = 0) v8 ==== 273+0+0 (crc 0 0 0) 0x7f2d1c02e040 con 0x55ff1ba71bc0 2023-10-05T19:40:19.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.899+0000 7f2d2d23fe40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2023-10-05T19:40:19.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.899+0000 7f2d2d23fe40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:19.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.899+0000 7f2d2d23fe40 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2023-10-05T19:40:19.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.899+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:106 7.d 7:b4812045:::notify.4:head [notify cookie 94554170333728 in=647b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bbc0cd0 con 0x55ff1ba71bc0 2023-10-05T19:40:19.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.899+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== watch-notify(notify (1) cookie 94554169798112 notify 120259084291 ret 0) v3 ==== 677+0+0 (crc 0 0 0) 0x7f2d1c040b40 con 0x55ff1ba71bc0 2023-10-05T19:40:19.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.899+0000 7f2d28d70700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084291 cookie 94554169798112 notifier 4315 bl.length()=635 2023-10-05T19:40:19.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.899+0000 7f2d28d70700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2023-10-05T19:40:19.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.899+0000 7f2d28d70700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:19.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.899+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(106 notify.4 [notify cookie 94554170333728 out=8b] v0'0 uv45 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f2d1c02e040 con 0x55ff1ba71bc0 2023-10-05T19:40:19.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.899+0000 7f2d28d70700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:107 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f2d04003d90 con 0x55ff1ba71bc0 2023-10-05T19:40:19.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.899+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== osd_op_reply(107 notify.4 [notify-ack] v0'0 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2d1c02e040 con 0x55ff1ba71bc0 2023-10-05T19:40:19.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.899+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.1 v2:172.21.15.12:6808/29933 39 ==== watch-notify(notify_complete (2) cookie 94554170333728 notify 120259084291 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f2d1c040d70 con 0x55ff1ba71bc0 2023-10-05T19:40:19.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.899+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:108 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bbc1a20 con 0x7f2d24007030 2023-10-05T19:40:19.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.903+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(108 p5YnriCv1nAtykxBrupQ [delete,create,writefull 0~7] v28'1 uv1 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x7f2d1c02e040 con 0x7f2d24007030 2023-10-05T19:40:19.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.903+0000 7f2d2d23fe40 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2023-10-05T19:40:19.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.903+0000 7f2d2d23fe40 10 moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2023-10-05T19:40:19.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.903+0000 7f2d2d23fe40 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2023-10-05T19:40:19.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.903+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:109 7.15 7:a93a5511:::notify.2:head [notify cookie 94554170331440 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bbc1a20 con 0x55ff1bb36490 2023-10-05T19:40:19.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.903+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== watch-notify(notify (1) cookie 94554169795904 notify 120259084288 ret 0) v3 ==== 219+0+0 (crc 0 0 0) 0x55ff1bb563a0 con 0x55ff1bb36490 2023-10-05T19:40:19.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.903+0000 7f2d23fff700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084288 cookie 94554169795904 notifier 4315 bl.length()=177 2023-10-05T19:40:19.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.903+0000 7f2d23fff700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2023-10-05T19:40:19.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.903+0000 7f2d23fff700 10 rgw watcher librados: moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2023-10-05T19:40:19.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.903+0000 7f2d23fff700 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:110 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f2c78003440 con 0x55ff1bb36490 2023-10-05T19:40:19.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.903+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 <== osd.3 v2:172.21.15.12:6816/29951 27 ==== osd_op_reply(109 notify.2 [notify cookie 94554170331440 out=8b] v0'0 uv45 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f2d1403c040 con 0x55ff1bb36490 2023-10-05T19:40:19.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.903+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 <== osd.3 v2:172.21.15.12:6816/29951 28 ==== watch-notify(notify_complete (2) cookie 94554170331440 notify 120259084288 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55ff1bb55fc0 con 0x55ff1bb36490 2023-10-05T19:40:19.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.903+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 <== osd.3 v2:172.21.15.12:6816/29951 29 ==== osd_op_reply(110 notify.2 [notify-ack] v0'0 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2d1403c040 con 0x55ff1bb36490 2023-10-05T19:40:19.913 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:40:19.914 INFO:teuthology.orchestra.run.smithi012.stdout: "user_id": "foo", 2023-10-05T19:40:19.914 INFO:teuthology.orchestra.run.smithi012.stdout: "display_name": "Foo", 2023-10-05T19:40:19.914 INFO:teuthology.orchestra.run.smithi012.stdout: "email": "foo@foo.com", 2023-10-05T19:40:19.914 INFO:teuthology.orchestra.run.smithi012.stdout: "suspended": 0, 2023-10-05T19:40:19.914 INFO:teuthology.orchestra.run.smithi012.stdout: "max_buckets": 4, 2023-10-05T19:40:19.914 INFO:teuthology.orchestra.run.smithi012.stdout: "subusers": [], 2023-10-05T19:40:19.914 INFO:teuthology.orchestra.run.smithi012.stdout: "keys": [ 2023-10-05T19:40:19.914 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:19.914 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:40:19.914 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2023-10-05T19:40:19.914 INFO:teuthology.orchestra.run.smithi012.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2023-10-05T19:40:19.915 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:19.915 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:19.915 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:40:19.915 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key": "p5YnriCv1nAtykxBrupQ", 2023-10-05T19:40:19.915 INFO:teuthology.orchestra.run.smithi012.stdout: "secret_key": "Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh" 2023-10-05T19:40:19.915 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:19.915 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:19.915 INFO:teuthology.orchestra.run.smithi012.stdout: "swift_keys": [], 2023-10-05T19:40:19.915 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": [], 2023-10-05T19:40:19.915 INFO:teuthology.orchestra.run.smithi012.stdout: "op_mask": "read, write, delete", 2023-10-05T19:40:19.915 INFO:teuthology.orchestra.run.smithi012.stdout: "default_placement": "", 2023-10-05T19:40:19.915 INFO:teuthology.orchestra.run.smithi012.stdout: "default_storage_class": "", 2023-10-05T19:40:19.916 INFO:teuthology.orchestra.run.smithi012.stdout: "placement_tags": [], 2023-10-05T19:40:19.916 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket_quota": { 2023-10-05T19:40:19.916 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:19.916 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:19.916 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:19.916 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:19.916 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:19.916 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:19.916 INFO:teuthology.orchestra.run.smithi012.stdout: "user_quota": { 2023-10-05T19:40:19.916 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:19.916 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:19.916 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:19.917 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:19.917 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:19.917 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:19.917 INFO:teuthology.orchestra.run.smithi012.stdout: "temp_url_keys": [], 2023-10-05T19:40:19.917 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "rgw", 2023-10-05T19:40:19.917 INFO:teuthology.orchestra.run.smithi012.stdout: "mfa_ids": [] 2023-10-05T19:40:19.917 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:40:19.917 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:40:19.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.903+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:111 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94554169787728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bbc1a20 con 0x55ff1bb31cf0 2023-10-05T19:40:19.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.907+0000 7f2d2ad74700 1 -- 172.21.15.12:0/528765544 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(111 notify.0 [watch unwatch cookie 94554169787728] v28'47 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2d240067a0 con 0x55ff1bb31cf0 2023-10-05T19:40:19.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.907+0000 7f2d2d23fe40 20 remove_watcher() i=0 2023-10-05T19:40:19.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.907+0000 7f2d2d23fe40 2 removed watcher, disabling cache 2023-10-05T19:40:19.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.907+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:112 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94554169792352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bbc1a20 con 0x55ff1ba71bc0 2023-10-05T19:40:19.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.911+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.1 v2:172.21.15.12:6808/29933 40 ==== osd_op_reply(112 notify.1 [watch unwatch cookie 94554169792352] v28'47 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2d1c02e040 con 0x55ff1ba71bc0 2023-10-05T19:40:19.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.911+0000 7f2d2d23fe40 20 remove_watcher() i=1 2023-10-05T19:40:19.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.911+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:113 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94554169795904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bbc1a20 con 0x55ff1bb36490 2023-10-05T19:40:19.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.915+0000 7f2d2a573700 1 -- 172.21.15.12:0/528765544 <== osd.3 v2:172.21.15.12:6816/29951 30 ==== osd_op_reply(113 notify.2 [watch unwatch cookie 94554169795904] v28'47 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2d1403c040 con 0x55ff1bb36490 2023-10-05T19:40:19.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.915+0000 7f2d2d23fe40 20 remove_watcher() i=2 2023-10-05T19:40:19.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.915+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:114 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94554169794640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bbc1a20 con 0x55ff1bb31cf0 2023-10-05T19:40:19.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.915+0000 7f2d2ad74700 1 -- 172.21.15.12:0/528765544 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(114 notify.3 [watch unwatch cookie 94554169794640] v28'47 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2d240067a0 con 0x55ff1bb31cf0 2023-10-05T19:40:19.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.915+0000 7f2d2d23fe40 20 remove_watcher() i=3 2023-10-05T19:40:19.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.915+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:115 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94554169798112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bbc1a20 con 0x55ff1ba71bc0 2023-10-05T19:40:19.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.919+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.1 v2:172.21.15.12:6808/29933 41 ==== osd_op_reply(115 notify.4 [watch unwatch cookie 94554169798112] v28'47 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2d1c02e040 con 0x55ff1ba71bc0 2023-10-05T19:40:19.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.919+0000 7f2d2d23fe40 20 remove_watcher() i=4 2023-10-05T19:40:19.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.919+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:116 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94554169804512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bbc1a20 con 0x7f2d24007030 2023-10-05T19:40:19.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.919+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(116 notify.5 [watch unwatch cookie 94554169804512] v28'47 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2d1c02e040 con 0x7f2d24007030 2023-10-05T19:40:19.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.919+0000 7f2d2d23fe40 20 remove_watcher() i=5 2023-10-05T19:40:19.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.919+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:117 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94554169808352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bbc1a20 con 0x7f2d24007030 2023-10-05T19:40:19.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.923+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.0 v2:172.21.15.8:6800/27563 30 ==== osd_op_reply(117 notify.6 [watch unwatch cookie 94554169808352] v28'47 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2d1c02e040 con 0x7f2d24007030 2023-10-05T19:40:19.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.923+0000 7f2d2d23fe40 20 remove_watcher() i=6 2023-10-05T19:40:19.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.923+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:118 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94554169807088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bbc1a20 con 0x7f2d24007030 2023-10-05T19:40:19.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.927+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.0 v2:172.21.15.8:6800/27563 31 ==== osd_op_reply(118 notify.7 [watch unwatch cookie 94554169807088] v28'47 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2d1c02e040 con 0x7f2d24007030 2023-10-05T19:40:19.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.927+0000 7f2d2d23fe40 20 remove_watcher() i=7 2023-10-05T19:40:19.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.927+0000 7f2cfb7ee700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:19.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.927+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:119 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94554169812032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ff1bbc1a20 con 0x55ff1ba71bc0 2023-10-05T19:40:19.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.931+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 <== osd.1 v2:172.21.15.12:6808/29933 42 ==== osd_op_reply(119 data_loggenerations_metadata [watch unwatch cookie 94554169812032] v28'65 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f2d1c02e040 con 0x55ff1ba71bc0 2023-10-05T19:40:19.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.931+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f2d24007030 msgr2=0x7f2d24045880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.931+0000 7f2d2d23fe40 1 --2- 172.21.15.12:0/528765544 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f2d24007030 0x7f2d24045880 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:19.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.931+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55ff1ba71bc0 msgr2=0x55ff1bb2f6f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.931+0000 7f2d2d23fe40 1 --2- 172.21.15.12:0/528765544 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55ff1ba71bc0 0x55ff1bb2f6f0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:19.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.931+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55ff1bb31cf0 msgr2=0x55ff1bb340d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.931+0000 7f2d2d23fe40 1 --2- 172.21.15.12:0/528765544 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55ff1bb31cf0 0x55ff1bb340d0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:19.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.931+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55ff1bb36490 msgr2=0x55ff1bb388a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.931+0000 7f2d2d23fe40 1 --2- 172.21.15.12:0/528765544 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55ff1bb36490 0x55ff1bb388a0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:19.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.931+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2d10030c50 msgr2=0x7f2d10033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.931+0000 7f2d2d23fe40 1 --2- 172.21.15.12:0/528765544 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2d10030c50 0x7f2d10033100 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2d140044f0 tx=0x7f2d14006690 comp rx=0 tx=0).stop 2023-10-05T19:40:19.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.931+0000 7f2d2b575700 1 -- 172.21.15.12:0/528765544 reap_dead start 2023-10-05T19:40:19.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.931+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff1ba706e0 msgr2=0x55ff1bb23cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.931+0000 7f2d2d23fe40 1 --2- 172.21.15.12:0/528765544 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff1ba706e0 0x55ff1bb23cd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2d24005cc0 tx=0x7f2d24003050 comp rx=0 tx=0).stop 2023-10-05T19:40:19.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.931+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 shutdown_connections 2023-10-05T19:40:19.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.931+0000 7f2d2d23fe40 1 --2- 172.21.15.12:0/528765544 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff1ba706e0 0x55ff1bb23cd0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:19.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.931+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 >> 172.21.15.12:0/528765544 conn(0x55ff1baf77f0 msgr2=0x55ff1bb175e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:19.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.931+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 shutdown_connections 2023-10-05T19:40:19.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.931+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/528765544 wait complete. 2023-10-05T19:40:19.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.931+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/4206523050 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2c70030c50 msgr2=0x7f2c70033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.931+0000 7f2d2d23fe40 1 --2- 172.21.15.12:0/4206523050 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2c70030c50 0x7f2c70033100 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x55ff1bb42100 tx=0x7f2d1c005500 comp rx=0 tx=0).stop 2023-10-05T19:40:19.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.931+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/4206523050 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff1bb4fdd0 msgr2=0x55ff1bb559c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.931+0000 7f2d2d23fe40 1 --2- 172.21.15.12:0/4206523050 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff1bb4fdd0 0x55ff1bb559c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x55ff1bb49e50 tx=0x7f2d14004f90 comp rx=0 tx=0).stop 2023-10-05T19:40:19.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.931+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/4206523050 shutdown_connections 2023-10-05T19:40:19.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.931+0000 7f2d2d23fe40 1 --2- 172.21.15.12:0/4206523050 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2c70030c50 0x7f2c70033100 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:19.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.931+0000 7f2d2d23fe40 1 --2- 172.21.15.12:0/4206523050 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ff1bb4fdd0 0x55ff1bb559c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:19.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.931+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/4206523050 >> 172.21.15.12:0/4206523050 conn(0x55ff1bb4b190 msgr2=0x55ff1bb49230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:19.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.935+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/4206523050 shutdown_connections 2023-10-05T19:40:19.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.935+0000 7f2d2d23fe40 1 -- 172.21.15.12:0/4206523050 wait complete. 2023-10-05T19:40:19.947 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}, {'user': 'foo', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:40:19.948 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2023-10-05T19:40:19.948 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2023-10-05T19:40:19.948 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2023-10-05T19:40:19.987 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:19.988 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:19.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.979+0000 7f198ba8fe40 1 Processor -- start 2023-10-05T19:40:19.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.979+0000 7f198ba8fe40 1 -- start start 2023-10-05T19:40:19.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.979+0000 7f198ba8fe40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55677ac66e40 0x55677ac67210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:19.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.979+0000 7f198ba8fe40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55677abb3bd0 con 0x55677ac67750 2023-10-05T19:40:19.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.979+0000 7f198ba8fe40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x55677ab70ee0 con 0x55677ac66e40 2023-10-05T19:40:19.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.979+0000 7f19895c4700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55677ac66e40 0x55677ac67210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:19.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.979+0000 7f1988dc3700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x55677ac67750 0x55677ac681f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:47290/0 (socket says 172.21.15.12:47290) 2023-10-05T19:40:19.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.979+0000 7f1988dc3700 1 -- 172.21.15.12:0/270073506 learned_addr learned my addr 172.21.15.12:0/270073506 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:19.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.983+0000 7f1983fff700 1 -- 172.21.15.12:0/270073506 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3258229540 0 0) 0x55677abb3bd0 con 0x55677ac67750 2023-10-05T19:40:19.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.983+0000 7f1983fff700 1 -- 172.21.15.12:0/270073506 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f196c001e10 con 0x55677ac67750 2023-10-05T19:40:19.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.983+0000 7f19895c4700 1 -- 172.21.15.12:0/270073506 >> v1:172.21.15.8:6789/0 conn(0x55677ac67750 legacy=0x55677ac681f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.983+0000 7f19895c4700 1 -- 172.21.15.12:0/270073506 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55677ac68900 con 0x55677ac66e40 2023-10-05T19:40:19.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.983+0000 7f19895c4700 1 --2- 172.21.15.12:0/270073506 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55677ac66e40 0x55677ac67210 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f197c00a5d0 tx=0x7f197c005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe4fd25d5bbc7e86 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:19.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.983+0000 7f1983fff700 1 -- 172.21.15.12:0/270073506 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f197c007610 con 0x55677ac66e40 2023-10-05T19:40:19.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.983+0000 7f1983fff700 1 -- 172.21.15.12:0/270073506 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f197c007770 con 0x55677ac66e40 2023-10-05T19:40:19.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.983+0000 7f1983fff700 1 -- 172.21.15.12:0/270073506 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f197c007a40 con 0x55677ac66e40 2023-10-05T19:40:19.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.983+0000 7f198ba8fe40 1 -- 172.21.15.12:0/270073506 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55677ac66e40 msgr2=0x55677ac67210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.983+0000 7f198ba8fe40 1 --2- 172.21.15.12:0/270073506 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55677ac66e40 0x55677ac67210 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f197c00a5d0 tx=0x7f197c005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:19.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.983+0000 7f198ba8fe40 1 -- 172.21.15.12:0/270073506 shutdown_connections 2023-10-05T19:40:19.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.983+0000 7f198ba8fe40 1 --2- 172.21.15.12:0/270073506 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55677ac66e40 0x55677ac67210 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:19.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.983+0000 7f198ba8fe40 1 -- 172.21.15.12:0/270073506 >> 172.21.15.12:0/270073506 conn(0x55677ac62860 msgr2=0x55677ac64c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:19.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.983+0000 7f198ba8fe40 1 -- 172.21.15.12:0/270073506 shutdown_connections 2023-10-05T19:40:19.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.983+0000 7f198ba8fe40 1 -- 172.21.15.12:0/270073506 wait complete. 2023-10-05T19:40:19.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.987+0000 7f198ba8fe40 1 Processor -- start 2023-10-05T19:40:19.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.987+0000 7f198ba8fe40 1 -- start start 2023-10-05T19:40:19.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.987+0000 7f198ba8fe40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55677ac66e40 0x55677ac85cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:19.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.987+0000 7f198ba8fe40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55677ab34e30 con 0x55677ac66e40 2023-10-05T19:40:19.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.987+0000 7f19895c4700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55677ac66e40 0x55677ac85cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:19.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.987+0000 7f19895c4700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55677ac66e40 0x55677ac85cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:36994/0 (socket says 172.21.15.12:36994) 2023-10-05T19:40:19.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.987+0000 7f19895c4700 1 -- 172.21.15.12:0/13446674 learned_addr learned my addr 172.21.15.12:0/13446674 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:19.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.987+0000 7f19895c4700 1 -- 172.21.15.12:0/13446674 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f197c005cf0 con 0x55677ac66e40 2023-10-05T19:40:19.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.987+0000 7f19895c4700 1 --2- 172.21.15.12:0/13446674 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55677ac66e40 0x55677ac85cf0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f197c000c00 tx=0x7f197c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2185bfc821cd6d7f server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:19.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.987+0000 7f1982ffd700 1 -- 172.21.15.12:0/13446674 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f197c0078e0 con 0x55677ac66e40 2023-10-05T19:40:19.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.987+0000 7f1982ffd700 1 -- 172.21.15.12:0/13446674 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f197c007a40 con 0x55677ac66e40 2023-10-05T19:40:19.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.987+0000 7f1982ffd700 1 -- 172.21.15.12:0/13446674 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f197c007bd0 con 0x55677ac66e40 2023-10-05T19:40:19.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.991+0000 7f198ba8fe40 1 -- 172.21.15.12:0/13446674 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55677ac66e40 msgr2=0x55677ac85cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:19.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.991+0000 7f198ba8fe40 1 --2- 172.21.15.12:0/13446674 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55677ac66e40 0x55677ac85cf0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f197c000c00 tx=0x7f197c014000 comp rx=0 tx=0).stop 2023-10-05T19:40:19.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.991+0000 7f198ba8fe40 1 -- 172.21.15.12:0/13446674 shutdown_connections 2023-10-05T19:40:19.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.991+0000 7f198ba8fe40 1 --2- 172.21.15.12:0/13446674 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55677ac66e40 0x55677ac85cf0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:19.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.991+0000 7f198ba8fe40 1 -- 172.21.15.12:0/13446674 >> 172.21.15.12:0/13446674 conn(0x55677ac62860 msgr2=0x55677ac818b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:19.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.991+0000 7f198ba8fe40 1 -- 172.21.15.12:0/13446674 shutdown_connections 2023-10-05T19:40:19.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.991+0000 7f198ba8fe40 1 -- 172.21.15.12:0/13446674 wait complete. 2023-10-05T19:40:19.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.991+0000 7f198ba8fe40 1 Processor -- start 2023-10-05T19:40:19.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.991+0000 7f198ba8fe40 1 -- start start 2023-10-05T19:40:20.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.991+0000 7f198ba8fe40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55677ac66e40 0x55677ac8c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.991+0000 7f198ba8fe40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55677ab07ed0 con 0x55677ac66e40 2023-10-05T19:40:20.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.991+0000 7f19895c4700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55677ac66e40 0x55677ac8c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.991+0000 7f19895c4700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55677ac66e40 0x55677ac8c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37010/0 (socket says 172.21.15.12:37010) 2023-10-05T19:40:20.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.991+0000 7f19895c4700 1 -- 172.21.15.12:0/3617685861 learned_addr learned my addr 172.21.15.12:0/3617685861 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:20.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.991+0000 7f19895c4700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55677ac68900 con 0x55677ac66e40 2023-10-05T19:40:20.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.991+0000 7f19895c4700 1 --2- 172.21.15.12:0/3617685861 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55677ac66e40 0x55677ac8c9b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f197c01f040 tx=0x7f197c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.991+0000 7f19817fa700 1 -- 172.21.15.12:0/3617685861 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f197c007bb0 con 0x55677ac66e40 2023-10-05T19:40:20.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.995+0000 7f19817fa700 1 -- 172.21.15.12:0/3617685861 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f197c029070 con 0x55677ac66e40 2023-10-05T19:40:20.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.995+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55677ac8cef0 con 0x55677ac66e40 2023-10-05T19:40:20.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.995+0000 7f19817fa700 1 -- 172.21.15.12:0/3617685861 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f197c012610 con 0x55677ac66e40 2023-10-05T19:40:20.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.995+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55677ac8d330 con 0x55677ac66e40 2023-10-05T19:40:20.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.995+0000 7f19817fa700 1 -- 172.21.15.12:0/3617685861 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f197c012770 con 0x55677ac66e40 2023-10-05T19:40:20.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.995+0000 7f19817fa700 1 --2- 172.21.15.12:0/3617685861 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1970030ca0 0x7f1970033150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.995+0000 7f198ba8fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.995+0000 7f19817fa700 1 -- 172.21.15.12:0/3617685861 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f197c02b070 con 0x55677ac66e40 2023-10-05T19:40:20.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.995+0000 7f1988dc3700 1 --2- 172.21.15.12:0/3617685861 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1970030ca0 0x7f1970033150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.995+0000 7f198ba8fe40 1 --2- 172.21.15.12:0/3617685861 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55677ac67750 0x55677ac98440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.995+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677ac98ab0 con 0x55677ac67750 2023-10-05T19:40:20.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.995+0000 7f1989dc5700 1 --2- 172.21.15.12:0/3617685861 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55677ac67750 0x55677ac98440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.995+0000 7f1988dc3700 1 --2- 172.21.15.12:0/3617685861 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1970030ca0 0x7f1970033150 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f19740044f0 tx=0x7f1974006690 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.999+0000 7f1989dc5700 1 --2- 172.21.15.12:0/3617685861 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55677ac67750 0x55677ac98440 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.999+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f198400a500 con 0x55677ac67750 2023-10-05T19:40:20.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.999+0000 7f198ba8fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.999+0000 7f198ba8fe40 20 realm 2023-10-05T19:40:20.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.999+0000 7f198ba8fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.999+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677ac95aa0 con 0x55677ac67750 2023-10-05T19:40:20.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.999+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f198400a500 con 0x55677ac67750 2023-10-05T19:40:20.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.999+0000 7f198ba8fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.999+0000 7f198ba8fe40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:20.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.999+0000 7f198ba8fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.999+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677ac99590 con 0x55677ac67750 2023-10-05T19:40:20.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.999+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f198400a500 con 0x55677ac67750 2023-10-05T19:40:20.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.999+0000 7f198ba8fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.999+0000 7f198ba8fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.999+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677ac99170 con 0x55677ac67750 2023-10-05T19:40:20.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.999+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f198400a500 con 0x55677ac67750 2023-10-05T19:40:20.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.999+0000 7f198ba8fe40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:20.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.999+0000 7f198ba8fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.999+0000 7f198ba8fe40 1 --2- 172.21.15.12:0/3617685861 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55677ac9a520 0x55677ac9c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.999+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677ac9d070 con 0x55677ac9a520 2023-10-05T19:40:20.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.999+0000 7f19895c4700 1 --2- 172.21.15.12:0/3617685861 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55677ac9a520 0x55677ac9c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:19.999+0000 7f19895c4700 1 --2- 172.21.15.12:0/3617685861 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55677ac9a520 0x55677ac9c900 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.003+0000 7f19895c4700 1 -- 172.21.15.12:0/3617685861 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f197c041040 con 0x55677ac9a520 2023-10-05T19:40:20.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.003+0000 7f198ba8fe40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:20.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.003+0000 7f198ba8fe40 20 searching for the correct realm 2023-10-05T19:40:20.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.003+0000 7f198ba8fe40 1 --2- 172.21.15.12:0/3617685861 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55677ac9eea0 0x55677aca1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.003+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca17c0 con 0x55677ac9eea0 2023-10-05T19:40:20.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.003+0000 7f1988dc3700 1 --2- 172.21.15.12:0/3617685861 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55677ac9eea0 0x55677aca1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.003+0000 7f1988dc3700 1 --2- 172.21.15.12:0/3617685861 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55677ac9eea0 0x55677aca1280 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.003+0000 7f1988dc3700 1 -- 172.21.15.12:0/3617685861 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f1974004520 con 0x55677ac9eea0 2023-10-05T19:40:20.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.003+0000 7f1988dc3700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca17c0 con 0x55677ac9a520 2023-10-05T19:40:20.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.007+0000 7f19895c4700 1 -- 172.21.15.12:0/3617685861 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f197c041040 con 0x55677ac9a520 2023-10-05T19:40:20.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.007+0000 7f19895c4700 1 --2- 172.21.15.12:0/3617685861 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f197c007030 0x7f197c046050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.007+0000 7f19895c4700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca17c0 con 0x7f197c007030 2023-10-05T19:40:20.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.007+0000 7f1989dc5700 1 --2- 172.21.15.12:0/3617685861 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f197c007030 0x7f197c046050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.007+0000 7f1989dc5700 1 --2- 172.21.15.12:0/3617685861 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f197c007030 0x7f197c046050 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.007+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1984014040 con 0x7f197c007030 2023-10-05T19:40:20.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.007+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca17c0 con 0x55677ac67750 2023-10-05T19:40:20.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.007+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1984014040 con 0x55677ac67750 2023-10-05T19:40:20.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.007+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca17c0 con 0x55677ac67750 2023-10-05T19:40:20.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.011+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1984014040 con 0x55677ac67750 2023-10-05T19:40:20.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.011+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca17c0 con 0x7f197c007030 2023-10-05T19:40:20.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.011+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1984014040 con 0x7f197c007030 2023-10-05T19:40:20.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.011+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca17c0 con 0x7f197c007030 2023-10-05T19:40:20.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.011+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1984014040 con 0x7f197c007030 2023-10-05T19:40:20.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.011+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca17c0 con 0x55677ac9a520 2023-10-05T19:40:20.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.011+0000 7f19895c4700 1 -- 172.21.15.12:0/3617685861 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f197c041040 con 0x55677ac9a520 2023-10-05T19:40:20.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.011+0000 7f19895c4700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca17c0 con 0x55677ac9eea0 2023-10-05T19:40:20.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.011+0000 7f1988dc3700 1 -- 172.21.15.12:0/3617685861 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1974004520 con 0x55677ac9eea0 2023-10-05T19:40:20.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.011+0000 7f1988dc3700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca17c0 con 0x55677ac67750 2023-10-05T19:40:20.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.015+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1984014040 con 0x55677ac67750 2023-10-05T19:40:20.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.015+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca17c0 con 0x7f197c007030 2023-10-05T19:40:20.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.015+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1984014040 con 0x7f197c007030 2023-10-05T19:40:20.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.015+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca17c0 con 0x55677ac67750 2023-10-05T19:40:20.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.015+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1984014040 con 0x55677ac67750 2023-10-05T19:40:20.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.015+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca17c0 con 0x55677ac9a520 2023-10-05T19:40:20.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.015+0000 7f19895c4700 1 -- 172.21.15.12:0/3617685861 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f197c041040 con 0x55677ac9a520 2023-10-05T19:40:20.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.015+0000 7f19895c4700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca17c0 con 0x55677ac9eea0 2023-10-05T19:40:20.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.015+0000 7f1988dc3700 1 -- 172.21.15.12:0/3617685861 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1974004520 con 0x55677ac9eea0 2023-10-05T19:40:20.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.015+0000 7f1988dc3700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca17c0 con 0x55677ac9eea0 2023-10-05T19:40:20.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.019+0000 7f1988dc3700 1 -- 172.21.15.12:0/3617685861 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1974004520 con 0x55677ac9eea0 2023-10-05T19:40:20.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.019+0000 7f1988dc3700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca17c0 con 0x7f197c007030 2023-10-05T19:40:20.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.019+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1984014040 con 0x7f197c007030 2023-10-05T19:40:20.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.019+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca17c0 con 0x55677ac9eea0 2023-10-05T19:40:20.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.019+0000 7f1988dc3700 1 -- 172.21.15.12:0/3617685861 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1974004520 con 0x55677ac9eea0 2023-10-05T19:40:20.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.019+0000 7f1988dc3700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca17c0 con 0x55677ac9eea0 2023-10-05T19:40:20.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.019+0000 7f1988dc3700 1 -- 172.21.15.12:0/3617685861 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1974004520 con 0x55677ac9eea0 2023-10-05T19:40:20.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.019+0000 7f1988dc3700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca17c0 con 0x55677ac67750 2023-10-05T19:40:20.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.019+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1984014040 con 0x55677ac67750 2023-10-05T19:40:20.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.019+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca17c0 con 0x55677ac67750 2023-10-05T19:40:20.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.019+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1984014040 con 0x55677ac67750 2023-10-05T19:40:20.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.019+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca17c0 con 0x7f197c007030 2023-10-05T19:40:20.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.023+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1984014040 con 0x7f197c007030 2023-10-05T19:40:20.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.023+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca17c0 con 0x55677ac9eea0 2023-10-05T19:40:20.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.023+0000 7f1988dc3700 1 -- 172.21.15.12:0/3617685861 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1974004520 con 0x55677ac9eea0 2023-10-05T19:40:20.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.023+0000 7f1988dc3700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca17c0 con 0x55677ac9a520 2023-10-05T19:40:20.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.023+0000 7f19895c4700 1 -- 172.21.15.12:0/3617685861 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f197c041040 con 0x55677ac9a520 2023-10-05T19:40:20.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.023+0000 7f19895c4700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca17c0 con 0x55677ac67750 2023-10-05T19:40:20.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.023+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1984014040 con 0x55677ac67750 2023-10-05T19:40:20.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.023+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca17c0 con 0x7f197c007030 2023-10-05T19:40:20.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.027+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1984014040 con 0x7f197c007030 2023-10-05T19:40:20.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.027+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca17c0 con 0x55677ac9eea0 2023-10-05T19:40:20.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.027+0000 7f1988dc3700 1 -- 172.21.15.12:0/3617685861 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1974004520 con 0x55677ac9eea0 2023-10-05T19:40:20.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.027+0000 7f1988dc3700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca17c0 con 0x7f197c007030 2023-10-05T19:40:20.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.027+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1984014040 con 0x7f197c007030 2023-10-05T19:40:20.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.027+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca17c0 con 0x55677ac9a520 2023-10-05T19:40:20.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.027+0000 7f19895c4700 1 -- 172.21.15.12:0/3617685861 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f197c041040 con 0x55677ac9a520 2023-10-05T19:40:20.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.027+0000 7f19895c4700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca17c0 con 0x55677ac9eea0 2023-10-05T19:40:20.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.027+0000 7f1988dc3700 1 -- 172.21.15.12:0/3617685861 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1974004520 con 0x55677ac9eea0 2023-10-05T19:40:20.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.027+0000 7f1988dc3700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca17c0 con 0x7f197c007030 2023-10-05T19:40:20.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.031+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1984014040 con 0x7f197c007030 2023-10-05T19:40:20.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.031+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca17c0 con 0x55677ac67750 2023-10-05T19:40:20.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.031+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f1984014040 con 0x55677ac67750 2023-10-05T19:40:20.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.031+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca17c0 con 0x55677ac9a520 2023-10-05T19:40:20.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.031+0000 7f19895c4700 1 -- 172.21.15.12:0/3617685861 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f197c041040 con 0x55677ac9a520 2023-10-05T19:40:20.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.031+0000 7f198ba8fe40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:20.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.031+0000 7f198ba8fe40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:20.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.031+0000 7f198ba8fe40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:20.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.031+0000 7f198ba8fe40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:20.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.031+0000 7f198ba8fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.031+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677ac9dc20 con 0x55677ac67750 2023-10-05T19:40:20.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.031+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1984014040 con 0x55677ac67750 2023-10-05T19:40:20.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.031+0000 7f198ba8fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.031+0000 7f198ba8fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.031+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677ac9e570 con 0x55677ac67750 2023-10-05T19:40:20.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.031+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f1984014040 con 0x55677ac67750 2023-10-05T19:40:20.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.031+0000 7f198ba8fe40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:20.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.031+0000 7f198ba8fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.031+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677ac9d7e0 con 0x55677ac9eea0 2023-10-05T19:40:20.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.035+0000 7f1988dc3700 1 -- 172.21.15.12:0/3617685861 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f1974004520 con 0x55677ac9eea0 2023-10-05T19:40:20.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.035+0000 7f198ba8fe40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:20.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.035+0000 7f198ba8fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.035+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca20f0 con 0x55677ac9eea0 2023-10-05T19:40:20.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.035+0000 7f1988dc3700 1 -- 172.21.15.12:0/3617685861 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f1974004520 con 0x55677ac9eea0 2023-10-05T19:40:20.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.035+0000 7f198ba8fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.035+0000 7f198ba8fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.035+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca24a0 con 0x55677ac67750 2023-10-05T19:40:20.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.035+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1984014040 con 0x55677ac67750 2023-10-05T19:40:20.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.035+0000 7f198ba8fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.035+0000 7f198ba8fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.035+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca2850 con 0x55677ac67750 2023-10-05T19:40:20.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.035+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f1984014040 con 0x55677ac67750 2023-10-05T19:40:20.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.035+0000 7f198ba8fe40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:20.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.035+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca3860 con 0x55677ac9eea0 2023-10-05T19:40:20.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.039+0000 7f1988dc3700 1 -- 172.21.15.12:0/3617685861 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f1974004520 con 0x55677ac9eea0 2023-10-05T19:40:20.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.039+0000 7f1988dc3700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca3860 con 0x55677ac9a520 2023-10-05T19:40:20.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.039+0000 7f19895c4700 1 -- 172.21.15.12:0/3617685861 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f197c041040 con 0x55677ac9a520 2023-10-05T19:40:20.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.039+0000 7f19895c4700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca3860 con 0x7f197c007030 2023-10-05T19:40:20.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.039+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1984014040 con 0x7f197c007030 2023-10-05T19:40:20.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.039+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca24a0 con 0x55677ac67750 2023-10-05T19:40:20.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.043+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1984014040 con 0x55677ac67750 2023-10-05T19:40:20.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.043+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca24a0 con 0x55677ac67750 2023-10-05T19:40:20.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.043+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1984014040 con 0x55677ac67750 2023-10-05T19:40:20.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.043+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca24a0 con 0x7f197c007030 2023-10-05T19:40:20.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.043+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1984014040 con 0x7f197c007030 2023-10-05T19:40:20.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.043+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca24a0 con 0x7f197c007030 2023-10-05T19:40:20.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.043+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1984014040 con 0x7f197c007030 2023-10-05T19:40:20.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.043+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca24a0 con 0x55677ac9a520 2023-10-05T19:40:20.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.043+0000 7f19895c4700 1 -- 172.21.15.12:0/3617685861 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f197c041040 con 0x55677ac9a520 2023-10-05T19:40:20.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.043+0000 7f19895c4700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca24a0 con 0x55677ac9eea0 2023-10-05T19:40:20.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.047+0000 7f1988dc3700 1 -- 172.21.15.12:0/3617685861 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f197402d040 con 0x55677ac9eea0 2023-10-05T19:40:20.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.047+0000 7f1988dc3700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca24a0 con 0x55677ac67750 2023-10-05T19:40:20.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.047+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1984014040 con 0x55677ac67750 2023-10-05T19:40:20.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.047+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca24a0 con 0x7f197c007030 2023-10-05T19:40:20.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.047+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1984014040 con 0x7f197c007030 2023-10-05T19:40:20.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.047+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca24a0 con 0x55677ac67750 2023-10-05T19:40:20.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.047+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1984014040 con 0x55677ac67750 2023-10-05T19:40:20.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.047+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca24a0 con 0x55677ac9a520 2023-10-05T19:40:20.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.051+0000 7f19895c4700 1 -- 172.21.15.12:0/3617685861 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f197c041040 con 0x55677ac9a520 2023-10-05T19:40:20.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.051+0000 7f19895c4700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca24a0 con 0x55677ac9eea0 2023-10-05T19:40:20.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.051+0000 7f1988dc3700 1 -- 172.21.15.12:0/3617685861 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f197402d040 con 0x55677ac9eea0 2023-10-05T19:40:20.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.051+0000 7f1988dc3700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca24a0 con 0x55677ac9eea0 2023-10-05T19:40:20.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.051+0000 7f1988dc3700 1 -- 172.21.15.12:0/3617685861 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f197402d040 con 0x55677ac9eea0 2023-10-05T19:40:20.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.051+0000 7f1988dc3700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca24a0 con 0x7f197c007030 2023-10-05T19:40:20.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.051+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1984014040 con 0x7f197c007030 2023-10-05T19:40:20.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.051+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca24a0 con 0x55677ac9eea0 2023-10-05T19:40:20.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.051+0000 7f1988dc3700 1 -- 172.21.15.12:0/3617685861 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f197402d040 con 0x55677ac9eea0 2023-10-05T19:40:20.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.051+0000 7f1988dc3700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f1974031040 con 0x55677ac9eea0 2023-10-05T19:40:20.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.055+0000 7f1988dc3700 1 -- 172.21.15.12:0/3617685861 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f197402d040 con 0x55677ac9eea0 2023-10-05T19:40:20.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.055+0000 7f1988dc3700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f1974031040 con 0x55677ac67750 2023-10-05T19:40:20.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.055+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1984014040 con 0x55677ac67750 2023-10-05T19:40:20.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.055+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f1974031040 con 0x55677ac67750 2023-10-05T19:40:20.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.055+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1984014040 con 0x55677ac67750 2023-10-05T19:40:20.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.055+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f1974031040 con 0x7f197c007030 2023-10-05T19:40:20.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.055+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1984014040 con 0x7f197c007030 2023-10-05T19:40:20.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.055+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f1974031040 con 0x55677ac9eea0 2023-10-05T19:40:20.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.059+0000 7f1988dc3700 1 -- 172.21.15.12:0/3617685861 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f197402d040 con 0x55677ac9eea0 2023-10-05T19:40:20.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.059+0000 7f1988dc3700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f1974031040 con 0x55677ac9a520 2023-10-05T19:40:20.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.059+0000 7f19895c4700 1 -- 172.21.15.12:0/3617685861 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f197c041040 con 0x55677ac9a520 2023-10-05T19:40:20.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.059+0000 7f19895c4700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f1974031040 con 0x55677ac67750 2023-10-05T19:40:20.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.059+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1984014040 con 0x55677ac67750 2023-10-05T19:40:20.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.059+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f1974031040 con 0x7f197c007030 2023-10-05T19:40:20.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.059+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1984014040 con 0x7f197c007030 2023-10-05T19:40:20.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.059+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f1974031040 con 0x55677ac9eea0 2023-10-05T19:40:20.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.059+0000 7f1988dc3700 1 -- 172.21.15.12:0/3617685861 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f197402d040 con 0x55677ac9eea0 2023-10-05T19:40:20.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.059+0000 7f1988dc3700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f1974031040 con 0x7f197c007030 2023-10-05T19:40:20.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.063+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1984014040 con 0x7f197c007030 2023-10-05T19:40:20.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.063+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f1974031040 con 0x55677ac9a520 2023-10-05T19:40:20.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.063+0000 7f19895c4700 1 -- 172.21.15.12:0/3617685861 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f197c041040 con 0x55677ac9a520 2023-10-05T19:40:20.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.063+0000 7f19895c4700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f1974031040 con 0x55677ac9eea0 2023-10-05T19:40:20.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.063+0000 7f1988dc3700 1 -- 172.21.15.12:0/3617685861 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f197402d040 con 0x55677ac9eea0 2023-10-05T19:40:20.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.063+0000 7f1988dc3700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f1974031040 con 0x7f197c007030 2023-10-05T19:40:20.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.063+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1984014040 con 0x7f197c007030 2023-10-05T19:40:20.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.063+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f1974031040 con 0x55677ac67750 2023-10-05T19:40:20.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.067+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f1984014040 con 0x55677ac67750 2023-10-05T19:40:20.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.067+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f1974031040 con 0x55677ac9a520 2023-10-05T19:40:20.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.067+0000 7f19895c4700 1 -- 172.21.15.12:0/3617685861 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f197c041040 con 0x55677ac9a520 2023-10-05T19:40:20.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.067+0000 7f198ba8fe40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:20.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.067+0000 7f198ba8fe40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:20.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.067+0000 7f198ba8fe40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:20.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.067+0000 7f198ba8fe40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:20.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.067+0000 7f198ba8fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.067+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca2850 con 0x55677ac67750 2023-10-05T19:40:20.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.067+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f1984014040 con 0x55677ac67750 2023-10-05T19:40:20.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.067+0000 7f198ba8fe40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:20.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.067+0000 7f198ba8fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.067+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca2c00 con 0x55677ac9a520 2023-10-05T19:40:20.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.067+0000 7f19895c4700 1 -- 172.21.15.12:0/3617685861 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f197c041040 con 0x55677ac9a520 2023-10-05T19:40:20.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.067+0000 7f198ba8fe40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:20.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.067+0000 7f198ba8fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.067+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca34a0 con 0x55677ac67750 2023-10-05T19:40:20.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.071+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f1984014040 con 0x55677ac67750 2023-10-05T19:40:20.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.071+0000 7f198ba8fe40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:20.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.071+0000 7f198ba8fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.071+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca34a0 con 0x55677ac9eea0 2023-10-05T19:40:20.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.071+0000 7f1988dc3700 1 -- 172.21.15.12:0/3617685861 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f197402d040 con 0x55677ac9eea0 2023-10-05T19:40:20.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.071+0000 7f198ba8fe40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:20.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.071+0000 7f198ba8fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.071+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca3d40 con 0x7f197c007030 2023-10-05T19:40:20.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.071+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f1984014040 con 0x7f197c007030 2023-10-05T19:40:20.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.071+0000 7f198ba8fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.071+0000 7f198ba8fe40 20 zone default found 2023-10-05T19:40:20.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.071+0000 7f198ba8fe40 4 Realm: () 2023-10-05T19:40:20.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.071+0000 7f198ba8fe40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:20.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.071+0000 7f198ba8fe40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:20.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.071+0000 7f198ba8fe40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:20.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.071+0000 7f198ba8fe40 20 zonegroup default 2023-10-05T19:40:20.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.071+0000 7f198ba8fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.071+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca3d40 con 0x7f197c007030 2023-10-05T19:40:20.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.071+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f1984014040 con 0x7f197c007030 2023-10-05T19:40:20.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.071+0000 7f198ba8fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.071+0000 7f198ba8fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.071+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca56c0 con 0x55677ac9eea0 2023-10-05T19:40:20.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.071+0000 7f1988dc3700 1 -- 172.21.15.12:0/3617685861 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f197402d040 con 0x55677ac9eea0 2023-10-05T19:40:20.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.071+0000 7f198ba8fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.071+0000 7f198ba8fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.071+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca5a70 con 0x55677ac9eea0 2023-10-05T19:40:20.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.075+0000 7f1988dc3700 1 -- 172.21.15.12:0/3617685861 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f197402d040 con 0x55677ac9eea0 2023-10-05T19:40:20.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.075+0000 7f198ba8fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.075+0000 7f198ba8fe40 20 started sync module instance, tier type = 2023-10-05T19:40:20.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.075+0000 7f198ba8fe40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:20.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.075+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca4d80 con 0x55677ac9a520 2023-10-05T19:40:20.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.075+0000 7f19895c4700 1 -- 172.21.15.12:0/3617685861 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'48 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f197c041040 con 0x55677ac9a520 2023-10-05T19:40:20.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.075+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93902930069648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca5e20 con 0x55677ac9a520 2023-10-05T19:40:20.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.075+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca66c0 con 0x55677ac67750 2023-10-05T19:40:20.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.079+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'48 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1984014040 con 0x55677ac67750 2023-10-05T19:40:20.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.079+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93902930076112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca6b10 con 0x55677ac67750 2023-10-05T19:40:20.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.079+0000 7f19895c4700 1 -- 172.21.15.12:0/3617685861 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 93902930069648] v28'49 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f197c041040 con 0x55677ac9a520 2023-10-05T19:40:20.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.079+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca74a0 con 0x55677ac9eea0 2023-10-05T19:40:20.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.083+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 93902930076112] v28'49 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1984014040 con 0x55677ac67750 2023-10-05T19:40:20.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.083+0000 7f1988dc3700 1 -- 172.21.15.12:0/3617685861 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'48 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f197402d040 con 0x55677ac9eea0 2023-10-05T19:40:20.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.083+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93902930079664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca6b10 con 0x55677ac9eea0 2023-10-05T19:40:20.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.083+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca7d80 con 0x55677ac9a520 2023-10-05T19:40:20.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.087+0000 7f19895c4700 1 -- 172.21.15.12:0/3617685861 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'48 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f197c041040 con 0x55677ac9a520 2023-10-05T19:40:20.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.087+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 93902930081872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca6b10 con 0x55677ac9a520 2023-10-05T19:40:20.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.087+0000 7f1988dc3700 1 -- 172.21.15.12:0/3617685861 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 93902930079664] v28'49 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f197402d040 con 0x55677ac9eea0 2023-10-05T19:40:20.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.087+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca8cd0 con 0x55677ac67750 2023-10-05T19:40:20.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.087+0000 7f19895c4700 1 -- 172.21.15.12:0/3617685861 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 93902930081872] v28'49 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f197c041040 con 0x55677ac9a520 2023-10-05T19:40:20.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.087+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'48 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1984014040 con 0x55677ac67750 2023-10-05T19:40:20.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.087+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 93902930077216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca6a70 con 0x55677ac67750 2023-10-05T19:40:20.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.087+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca91c0 con 0x7f197c007030 2023-10-05T19:40:20.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.091+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 93902930077216] v28'49 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1984014040 con 0x55677ac67750 2023-10-05T19:40:20.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.091+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'48 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1984014040 con 0x7f197c007030 2023-10-05T19:40:20.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.091+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 93902930085856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca9120 con 0x7f197c007030 2023-10-05T19:40:20.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.091+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca99c0 con 0x7f197c007030 2023-10-05T19:40:20.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.095+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(95 notify.5 [watch watch cookie 93902930085856] v28'49 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1984014040 con 0x7f197c007030 2023-10-05T19:40:20.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.095+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(96 notify.6 [create] v28'48 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1984014040 con 0x7f197c007030 2023-10-05T19:40:20.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.095+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93902930087520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca9820 con 0x7f197c007030 2023-10-05T19:40:20.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.095+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677acaa2f0 con 0x7f197c007030 2023-10-05T19:40:20.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.099+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(98 notify.7 [create] v28'48 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1984014040 con 0x7f197c007030 2023-10-05T19:40:20.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.099+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93902930091520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677acaa740 con 0x7f197c007030 2023-10-05T19:40:20.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.099+0000 7f198ba8fe40 20 add_watcher() i=0 2023-10-05T19:40:20.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.099+0000 7f198ba8fe40 20 add_watcher() i=1 2023-10-05T19:40:20.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.099+0000 7f198ba8fe40 20 add_watcher() i=2 2023-10-05T19:40:20.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.099+0000 7f198ba8fe40 20 add_watcher() i=3 2023-10-05T19:40:20.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.099+0000 7f198ba8fe40 20 add_watcher() i=4 2023-10-05T19:40:20.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.099+0000 7f198ba8fe40 20 add_watcher() i=5 2023-10-05T19:40:20.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.099+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(97 notify.6 [watch watch cookie 93902930087520] v28'49 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1984014040 con 0x7f197c007030 2023-10-05T19:40:20.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.099+0000 7f198ba8fe40 20 add_watcher() i=6 2023-10-05T19:40:20.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.099+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 93902930091520] v28'49 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1984014040 con 0x7f197c007030 2023-10-05T19:40:20.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.099+0000 7f198ba8fe40 20 add_watcher() i=7 2023-10-05T19:40:20.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.099+0000 7f198ba8fe40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:20.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.099+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677aca92f0 con 0x55677ac67750 2023-10-05T19:40:20.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.099+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f1984014040 con 0x55677ac67750 2023-10-05T19:40:20.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.099+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93902930092624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677acaab50 con 0x55677ac67750 2023-10-05T19:40:20.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.103+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 93902930092624] v28'66 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f1984014040 con 0x55677ac67750 2023-10-05T19:40:20.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.103+0000 7f1959feb700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:20.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.103+0000 7f198ba8fe40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:20.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.103+0000 7f198ba8fe40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:20.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.103+0000 7f198ba8fe40 1 Processor -- start 2023-10-05T19:40:20.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.103+0000 7f198ba8fe40 1 -- start start 2023-10-05T19:40:20.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.103+0000 7f198ba8fe40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55677acb7980 0x55677acb7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.103+0000 7f198ba8fe40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55677acb8290 con 0x55677acb7980 2023-10-05T19:40:20.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.103+0000 7f1988dc3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55677acb7980 0x55677acb7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.103+0000 7f1988dc3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55677acb7980 0x55677acb7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37022/0 (socket says 172.21.15.12:37022) 2023-10-05T19:40:20.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.103+0000 7f1988dc3700 1 -- 172.21.15.12:0/3992098816 learned_addr learned my addr 172.21.15.12:0/3992098816 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:20.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.107+0000 7f1988dc3700 1 -- 172.21.15.12:0/3992098816 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55677acb83d0 con 0x55677acb7980 2023-10-05T19:40:20.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.107+0000 7f1988dc3700 1 --2- 172.21.15.12:0/3992098816 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55677acb7980 0x55677acb7d50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f19840058f0 tx=0x7f1974027040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd863237b498bae server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.107+0000 7f19597ea700 1 -- 172.21.15.12:0/3992098816 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1974047070 con 0x55677acb7980 2023-10-05T19:40:20.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.107+0000 7f19597ea700 1 -- 172.21.15.12:0/3992098816 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1974039040 con 0x55677acb7980 2023-10-05T19:40:20.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.107+0000 7f19597ea700 1 -- 172.21.15.12:0/3992098816 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1974038040 con 0x55677acb7980 2023-10-05T19:40:20.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.107+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3992098816 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55677acb7980 msgr2=0x55677acb7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.107+0000 7f198ba8fe40 1 --2- 172.21.15.12:0/3992098816 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55677acb7980 0x55677acb7d50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f19840058f0 tx=0x7f1974027040 comp rx=0 tx=0).stop 2023-10-05T19:40:20.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.107+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3992098816 shutdown_connections 2023-10-05T19:40:20.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.107+0000 7f198ba8fe40 1 --2- 172.21.15.12:0/3992098816 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55677acb7980 0x55677acb7d50 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:20.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.107+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3992098816 >> 172.21.15.12:0/3992098816 conn(0x55677acb2d40 msgr2=0x55677acb0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:20.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.107+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3992098816 shutdown_connections 2023-10-05T19:40:20.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.107+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3992098816 wait complete. 2023-10-05T19:40:20.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.107+0000 7f198ba8fe40 1 Processor -- start 2023-10-05T19:40:20.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.107+0000 7f198ba8fe40 1 -- start start 2023-10-05T19:40:20.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.107+0000 7f198ba8fe40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55677acb7980 0x55677acbd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.107+0000 7f198ba8fe40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55677acbe4d0 con 0x55677acb7980 2023-10-05T19:40:20.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.107+0000 7f1988dc3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55677acb7980 0x55677acbd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.107+0000 7f1988dc3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55677acb7980 0x55677acbd4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37030/0 (socket says 172.21.15.12:37030) 2023-10-05T19:40:20.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.107+0000 7f1988dc3700 1 -- 172.21.15.12:0/66200056 learned_addr learned my addr 172.21.15.12:0/66200056 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:20.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.111+0000 7f1988dc3700 1 -- 172.21.15.12:0/66200056 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196c001e10 con 0x55677acb7980 2023-10-05T19:40:20.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.111+0000 7f1988dc3700 1 --2- 172.21.15.12:0/66200056 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55677acb7980 0x55677acbd4d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x55677acb1a60 tx=0x7f197403b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.111+0000 7f18d37fe700 1 -- 172.21.15.12:0/66200056 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1974038030 con 0x55677acb7980 2023-10-05T19:40:20.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.111+0000 7f18d37fe700 1 -- 172.21.15.12:0/66200056 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f197401b040 con 0x55677acb7980 2023-10-05T19:40:20.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.111+0000 7f198ba8fe40 1 -- 172.21.15.12:0/66200056 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55677acbdad0 con 0x55677acb7980 2023-10-05T19:40:20.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.111+0000 7f18d37fe700 1 -- 172.21.15.12:0/66200056 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f197402a040 con 0x55677acb7980 2023-10-05T19:40:20.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.111+0000 7f198ba8fe40 1 -- 172.21.15.12:0/66200056 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55677acbdf70 con 0x55677acb7980 2023-10-05T19:40:20.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.111+0000 7f18d37fe700 1 -- 172.21.15.12:0/66200056 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f1974025b90 con 0x55677acb7980 2023-10-05T19:40:20.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.111+0000 7f18d37fe700 1 --2- 172.21.15.12:0/66200056 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f18c8030c50 0x7f18c8033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.111+0000 7f18d37fe700 1 -- 172.21.15.12:0/66200056 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f1974028040 con 0x55677acb7980 2023-10-05T19:40:20.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.111+0000 7f1989dc5700 1 --2- 172.21.15.12:0/66200056 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f18c8030c50 0x7f18c8033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.115+0000 7f198ba8fe40 5 note: GC not initialized 2023-10-05T19:40:20.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.115+0000 7f1989dc5700 1 --2- 172.21.15.12:0/66200056 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f18c8030c50 0x7f18c8033100 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x55677aca97d0 tx=0x7f198400ab60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.115+0000 7f18d0ff9700 20 reqs_thread_entry: start 2023-10-05T19:40:20.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.115+0000 7f18c67fc700 20 reqs_thread_entry: start 2023-10-05T19:40:20.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.115+0000 7f198ba8fe40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:20.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.115+0000 7f198ba8fe40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:20.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.115+0000 7f18c4ff9700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:50 2023 2023-10-05T19:40:20.125 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:20.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.115+0000 7f18abfff700 20 reqs_thread_entry: start 2023-10-05T19:40:20.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.115+0000 7f198ba8fe40 20 get_system_obj_state: rctx=0x7ffdbd7c9b78 obj=default.rgw.meta:users.uid:foo state=0x55677ab400b0 s->prefetch_data=0 2023-10-05T19:40:20.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.115+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677ad25e00 con 0x55677ac67750 2023-10-05T19:40:20.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.119+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(102 foo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7f1984014040 con 0x55677ac67750 2023-10-05T19:40:20.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.119+0000 7f198ba8fe40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:20.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.119+0000 7f198ba8fe40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:20.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.119+0000 7f198ba8fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.119+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677ad25db0 con 0x55677ac67750 2023-10-05T19:40:20.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.119+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(103 foo [call,call out=48b,read 0~459 out=459b,getxattrs out=75b] v0'0 uv4 ondisk = 0) v8 ==== 273+0+582 (crc 0 0 0) 0x7f1984014040 con 0x55677ac67750 2023-10-05T19:40:20.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.119+0000 7f198ba8fe40 20 rados_obj.operate() r=0 bl.length=459 2023-10-05T19:40:20.128 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:40:20.128 INFO:teuthology.orchestra.run.smithi012.stdout: "user_id": "foo", 2023-10-05T19:40:20.128 INFO:teuthology.orchestra.run.smithi012.stdout: "display_name": "Foo", 2023-10-05T19:40:20.128 INFO:teuthology.orchestra.run.smithi012.stdout: "email": "foo@foo.com", 2023-10-05T19:40:20.128 INFO:teuthology.orchestra.run.smithi012.stdout: "suspended": 0, 2023-10-05T19:40:20.128 INFO:teuthology.orchestra.run.smithi012.stdout: "max_buckets": 4, 2023-10-05T19:40:20.128 INFO:teuthology.orchestra.run.smithi012.stdout: "subusers": [], 2023-10-05T19:40:20.128 INFO:teuthology.orchestra.run.smithi012.stdout: "keys": [ 2023-10-05T19:40:20.128 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:20.128 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:40:20.129 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2023-10-05T19:40:20.129 INFO:teuthology.orchestra.run.smithi012.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2023-10-05T19:40:20.129 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:20.129 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:20.129 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:40:20.129 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key": "p5YnriCv1nAtykxBrupQ", 2023-10-05T19:40:20.129 INFO:teuthology.orchestra.run.smithi012.stdout: "secret_key": "Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh" 2023-10-05T19:40:20.129 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:20.129 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:20.129 INFO:teuthology.orchestra.run.smithi012.stdout: "swift_keys": [], 2023-10-05T19:40:20.129 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": [], 2023-10-05T19:40:20.129 INFO:teuthology.orchestra.run.smithi012.stdout: "op_mask": "read, write, delete", 2023-10-05T19:40:20.130 INFO:teuthology.orchestra.run.smithi012.stdout: "default_placement": "", 2023-10-05T19:40:20.130 INFO:teuthology.orchestra.run.smithi012.stdout: "default_storage_class": "", 2023-10-05T19:40:20.130 INFO:teuthology.orchestra.run.smithi012.stdout: "placement_tags": [], 2023-10-05T19:40:20.130 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket_quota": { 2023-10-05T19:40:20.130 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:20.130 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:20.130 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:20.130 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:20.130 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:20.130 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:20.130 INFO:teuthology.orchestra.run.smithi012.stdout: "user_quota": { 2023-10-05T19:40:20.130 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:20.131 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:20.131 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:20.131 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:20.131 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:20.131 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:20.131 INFO:teuthology.orchestra.run.smithi012.stdout: "temp_url_keys": [], 2023-10-05T19:40:20.131 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "rgw", 2023-10-05T19:40:20.131 INFO:teuthology.orchestra.run.smithi012.stdout: "mfa_ids": [] 2023-10-05T19:40:20.131 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:40:20.131 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:40:20.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.119+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:104 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93902930069648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677ad25db0 con 0x55677ac9a520 2023-10-05T19:40:20.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.123+0000 7f19895c4700 1 -- 172.21.15.12:0/3617685861 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(104 notify.0 [watch unwatch cookie 93902930069648] v28'50 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f197c041040 con 0x55677ac9a520 2023-10-05T19:40:20.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.123+0000 7f198ba8fe40 20 remove_watcher() i=0 2023-10-05T19:40:20.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.123+0000 7f198ba8fe40 2 removed watcher, disabling cache 2023-10-05T19:40:20.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.123+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:105 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93902930076112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677ad25db0 con 0x55677ac67750 2023-10-05T19:40:20.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.123+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(105 notify.1 [watch unwatch cookie 93902930076112] v28'50 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1984014040 con 0x55677ac67750 2023-10-05T19:40:20.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.123+0000 7f198ba8fe40 20 remove_watcher() i=1 2023-10-05T19:40:20.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.123+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:106 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93902930079664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677ad25db0 con 0x55677ac9eea0 2023-10-05T19:40:20.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.127+0000 7f1988dc3700 1 -- 172.21.15.12:0/3617685861 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(106 notify.2 [watch unwatch cookie 93902930079664] v28'50 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f197403b070 con 0x55677ac9eea0 2023-10-05T19:40:20.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.127+0000 7f198ba8fe40 20 remove_watcher() i=2 2023-10-05T19:40:20.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.127+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:107 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93902930081872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677ad25db0 con 0x55677ac9a520 2023-10-05T19:40:20.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.131+0000 7f19895c4700 1 -- 172.21.15.12:0/3617685861 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(107 notify.3 [watch unwatch cookie 93902930081872] v28'50 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f197c041040 con 0x55677ac9a520 2023-10-05T19:40:20.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.131+0000 7f198ba8fe40 20 remove_watcher() i=3 2023-10-05T19:40:20.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.131+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:108 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93902930077216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677ad25db0 con 0x55677ac67750 2023-10-05T19:40:20.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.131+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(108 notify.4 [watch unwatch cookie 93902930077216] v28'50 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1984014040 con 0x55677ac67750 2023-10-05T19:40:20.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.131+0000 7f198ba8fe40 20 remove_watcher() i=4 2023-10-05T19:40:20.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.131+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:109 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93902930085856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677ad25db0 con 0x7f197c007030 2023-10-05T19:40:20.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.135+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(109 notify.5 [watch unwatch cookie 93902930085856] v28'50 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1984014040 con 0x7f197c007030 2023-10-05T19:40:20.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.135+0000 7f198ba8fe40 20 remove_watcher() i=5 2023-10-05T19:40:20.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.135+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:110 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93902930087520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677ad25db0 con 0x7f197c007030 2023-10-05T19:40:20.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.139+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(110 notify.6 [watch unwatch cookie 93902930087520] v28'50 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1984014040 con 0x7f197c007030 2023-10-05T19:40:20.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.139+0000 7f198ba8fe40 20 remove_watcher() i=6 2023-10-05T19:40:20.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.139+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:111 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93902930091520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677ad25db0 con 0x7f197c007030 2023-10-05T19:40:20.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.143+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(111 notify.7 [watch unwatch cookie 93902930091520] v28'50 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1984014040 con 0x7f197c007030 2023-10-05T19:40:20.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.143+0000 7f198ba8fe40 20 remove_watcher() i=7 2023-10-05T19:40:20.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.143+0000 7f1959feb700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:20.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.143+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:112 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93902930092624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55677ad25db0 con 0x55677ac67750 2023-10-05T19:40:20.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.147+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(112 data_loggenerations_metadata [watch unwatch cookie 93902930092624] v28'67 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f1984014040 con 0x55677ac67750 2023-10-05T19:40:20.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.147+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f197c007030 msgr2=0x7f197c046050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.147+0000 7f198ba8fe40 1 --2- 172.21.15.12:0/3617685861 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f197c007030 0x7f197c046050 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:20.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.147+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55677ac67750 msgr2=0x55677ac98440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.147+0000 7f198ba8fe40 1 --2- 172.21.15.12:0/3617685861 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55677ac67750 0x55677ac98440 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:20.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.147+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55677ac9a520 msgr2=0x55677ac9c900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.147+0000 7f198ba8fe40 1 --2- 172.21.15.12:0/3617685861 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55677ac9a520 0x55677ac9c900 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:20.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.147+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55677ac9eea0 msgr2=0x55677aca1280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.147+0000 7f198ba8fe40 1 --2- 172.21.15.12:0/3617685861 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55677ac9eea0 0x55677aca1280 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:20.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.147+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1970030ca0 msgr2=0x7f1970033150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.147+0000 7f198ba8fe40 1 --2- 172.21.15.12:0/3617685861 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1970030ca0 0x7f1970033150 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f19740044f0 tx=0x7f1974006690 comp rx=0 tx=0).stop 2023-10-05T19:40:20.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.147+0000 7f1989dc5700 1 -- 172.21.15.12:0/3617685861 reap_dead start 2023-10-05T19:40:20.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.147+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55677ac66e40 msgr2=0x55677ac8c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.147+0000 7f198ba8fe40 1 --2- 172.21.15.12:0/3617685861 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55677ac66e40 0x55677ac8c9b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f197c01f040 tx=0x7f197c01c000 comp rx=0 tx=0).stop 2023-10-05T19:40:20.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.147+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 shutdown_connections 2023-10-05T19:40:20.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.147+0000 7f198ba8fe40 1 --2- 172.21.15.12:0/3617685861 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55677ac66e40 0x55677ac8c9b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:20.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.147+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 >> 172.21.15.12:0/3617685861 conn(0x55677ac62860 msgr2=0x55677ac80330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:20.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.147+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 shutdown_connections 2023-10-05T19:40:20.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.147+0000 7f198ba8fe40 1 -- 172.21.15.12:0/3617685861 wait complete. 2023-10-05T19:40:20.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.147+0000 7f198ba8fe40 1 -- 172.21.15.12:0/66200056 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f18c8030c50 msgr2=0x7f18c8033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.147+0000 7f198ba8fe40 1 --2- 172.21.15.12:0/66200056 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f18c8030c50 0x7f18c8033100 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x55677aca97d0 tx=0x7f198400ab60 comp rx=0 tx=0).stop 2023-10-05T19:40:20.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.147+0000 7f198ba8fe40 1 -- 172.21.15.12:0/66200056 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55677acb7980 msgr2=0x55677acbd4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.147+0000 7f198ba8fe40 1 --2- 172.21.15.12:0/66200056 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55677acb7980 0x55677acbd4d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x55677acb1a60 tx=0x7f197403b000 comp rx=0 tx=0).stop 2023-10-05T19:40:20.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.147+0000 7f198ba8fe40 1 -- 172.21.15.12:0/66200056 shutdown_connections 2023-10-05T19:40:20.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.147+0000 7f198ba8fe40 1 --2- 172.21.15.12:0/66200056 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f18c8030c50 0x7f18c8033100 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:20.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.147+0000 7f198ba8fe40 1 --2- 172.21.15.12:0/66200056 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55677acb7980 0x55677acbd4d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:20.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.147+0000 7f198ba8fe40 1 -- 172.21.15.12:0/66200056 >> 172.21.15.12:0/66200056 conn(0x55677acb2d40 msgr2=0x55677acb0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:20.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.147+0000 7f198ba8fe40 1 -- 172.21.15.12:0/66200056 shutdown_connections 2023-10-05T19:40:20.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.147+0000 7f198ba8fe40 1 -- 172.21.15.12:0/66200056 wait complete. 2023-10-05T19:40:20.163 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}, {'user': 'foo', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:40:20.163 INFO:tasks.util.rgw:rgwadmin: client.0 : ['key', 'rm', '--uid', 'foo', '--access-key', 'p5YnriCv1nAtykxBrupQ'] 2023-10-05T19:40:20.163 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'key', 'rm', '--uid', 'foo', '--access-key', 'p5YnriCv1nAtykxBrupQ'] 2023-10-05T19:40:20.163 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph key rm --uid foo --access-key p5YnriCv1nAtykxBrupQ 2023-10-05T19:40:20.206 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:20.207 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:20.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.199+0000 7f75746e7e40 1 Processor -- start 2023-10-05T19:40:20.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.199+0000 7f75746e7e40 1 -- start start 2023-10-05T19:40:20.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.199+0000 7f75746e7e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x56121b04b300 0x56121b04d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.199+0000 7f75746e7e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x56121af93bd0 con 0x56121afbb3e0 2023-10-05T19:40:20.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.199+0000 7f75746e7e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x56121af50ee0 con 0x56121b04b300 2023-10-05T19:40:20.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.199+0000 7f7571a1b700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x56121b04b300 0x56121b04d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.199+0000 7f757221c700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x56121afbb3e0 0x56121b049be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:47294/0 (socket says 172.21.15.12:47294) 2023-10-05T19:40:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.199+0000 7f757221c700 1 -- 172.21.15.12:0/1610585891 learned_addr learned my addr 172.21.15.12:0/1610585891 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.199+0000 7f7571a1b700 1 -- 172.21.15.12:0/1610585891 >> v1:172.21.15.8:6789/0 conn(0x56121afbb3e0 legacy=0x56121b049be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.199+0000 7f757121a700 1 -- 172.21.15.12:0/1610585891 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2747123409 0 0) 0x56121af93bd0 con 0x56121afbb3e0 2023-10-05T19:40:20.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.199+0000 7f7571a1b700 1 -- 172.21.15.12:0/1610585891 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56121b04ec30 con 0x56121b04b300 2023-10-05T19:40:20.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.199+0000 7f7571a1b700 1 --2- 172.21.15.12:0/1610585891 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56121b04b300 0x56121b04d6e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f755c00a5d0 tx=0x7f755c005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdc31bf8b1ba5769 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.203+0000 7f757121a700 1 -- 172.21.15.12:0/1610585891 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f755c0086a0 con 0x56121b04b300 2023-10-05T19:40:20.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.203+0000 7f757121a700 1 -- 172.21.15.12:0/1610585891 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f755c008800 con 0x56121b04b300 2023-10-05T19:40:20.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.203+0000 7f757121a700 1 -- 172.21.15.12:0/1610585891 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f755c008ad0 con 0x56121b04b300 2023-10-05T19:40:20.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.203+0000 7f75746e7e40 1 -- 172.21.15.12:0/1610585891 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56121b04b300 msgr2=0x56121b04d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.203+0000 7f75746e7e40 1 --2- 172.21.15.12:0/1610585891 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56121b04b300 0x56121b04d6e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f755c00a5d0 tx=0x7f755c005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:20.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.203+0000 7f75746e7e40 1 -- 172.21.15.12:0/1610585891 shutdown_connections 2023-10-05T19:40:20.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.203+0000 7f75746e7e40 1 --2- 172.21.15.12:0/1610585891 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56121b04b300 0x56121b04d6e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:20.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.203+0000 7f75746e7e40 1 -- 172.21.15.12:0/1610585891 >> 172.21.15.12:0/1610585891 conn(0x56121b041730 msgr2=0x56121b043b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:20.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.203+0000 7f75746e7e40 1 -- 172.21.15.12:0/1610585891 shutdown_connections 2023-10-05T19:40:20.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.203+0000 7f75746e7e40 1 -- 172.21.15.12:0/1610585891 wait complete. 2023-10-05T19:40:20.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.203+0000 7f75746e7e40 1 Processor -- start 2023-10-05T19:40:20.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.203+0000 7f75746e7e40 1 -- start start 2023-10-05T19:40:20.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.203+0000 7f75746e7e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56121afbb3e0 0x56121b047160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.203+0000 7f75746e7e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56121af14e30 con 0x56121afbb3e0 2023-10-05T19:40:20.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.203+0000 7f757221c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56121afbb3e0 0x56121b047160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.203+0000 7f757221c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56121afbb3e0 0x56121b047160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37042/0 (socket says 172.21.15.12:37042) 2023-10-05T19:40:20.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.203+0000 7f757221c700 1 -- 172.21.15.12:0/673886943 learned_addr learned my addr 172.21.15.12:0/673886943 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:20.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.203+0000 7f757221c700 1 -- 172.21.15.12:0/673886943 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f755c005cf0 con 0x56121afbb3e0 2023-10-05T19:40:20.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.203+0000 7f757221c700 1 --2- 172.21.15.12:0/673886943 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56121afbb3e0 0x56121b047160 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7564001630 tx=0x7f75640069b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11343c05b3d07d63 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.207+0000 7f756bfff700 1 -- 172.21.15.12:0/673886943 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7564006320 con 0x56121afbb3e0 2023-10-05T19:40:20.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.207+0000 7f756bfff700 1 -- 172.21.15.12:0/673886943 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75640044c0 con 0x56121afbb3e0 2023-10-05T19:40:20.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.207+0000 7f756bfff700 1 -- 172.21.15.12:0/673886943 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7564004650 con 0x56121afbb3e0 2023-10-05T19:40:20.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.207+0000 7f75746e7e40 1 -- 172.21.15.12:0/673886943 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56121afbb3e0 msgr2=0x56121b047160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.207+0000 7f75746e7e40 1 --2- 172.21.15.12:0/673886943 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56121afbb3e0 0x56121b047160 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7564001630 tx=0x7f75640069b0 comp rx=0 tx=0).stop 2023-10-05T19:40:20.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.207+0000 7f75746e7e40 1 -- 172.21.15.12:0/673886943 shutdown_connections 2023-10-05T19:40:20.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.207+0000 7f75746e7e40 1 --2- 172.21.15.12:0/673886943 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56121afbb3e0 0x56121b047160 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:20.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.207+0000 7f75746e7e40 1 -- 172.21.15.12:0/673886943 >> 172.21.15.12:0/673886943 conn(0x56121b041730 msgr2=0x56121b061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:20.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.207+0000 7f75746e7e40 1 -- 172.21.15.12:0/673886943 shutdown_connections 2023-10-05T19:40:20.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.207+0000 7f75746e7e40 1 -- 172.21.15.12:0/673886943 wait complete. 2023-10-05T19:40:20.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.207+0000 7f75746e7e40 1 Processor -- start 2023-10-05T19:40:20.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.207+0000 7f75746e7e40 1 -- start start 2023-10-05T19:40:20.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.207+0000 7f75746e7e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56121afbb3e0 0x56121b06edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.207+0000 7f75746e7e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56121aee7ed0 con 0x56121afbb3e0 2023-10-05T19:40:20.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.207+0000 7f757221c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56121afbb3e0 0x56121b06edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.207+0000 7f757221c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56121afbb3e0 0x56121b06edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37048/0 (socket says 172.21.15.12:37048) 2023-10-05T19:40:20.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.207+0000 7f757221c700 1 -- 172.21.15.12:0/2383849624 learned_addr learned my addr 172.21.15.12:0/2383849624 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:20.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.211+0000 7f757221c700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7564006c90 con 0x56121afbb3e0 2023-10-05T19:40:20.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.211+0000 7f757221c700 1 --2- 172.21.15.12:0/2383849624 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56121afbb3e0 0x56121b06edf0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7564000f80 tx=0x7f7564014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.211+0000 7f756a7fc700 1 -- 172.21.15.12:0/2383849624 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f75640136c0 con 0x56121afbb3e0 2023-10-05T19:40:20.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.211+0000 7f756a7fc700 1 -- 172.21.15.12:0/2383849624 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7564013820 con 0x56121afbb3e0 2023-10-05T19:40:20.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.211+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56121b06f330 con 0x56121afbb3e0 2023-10-05T19:40:20.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.211+0000 7f756a7fc700 1 -- 172.21.15.12:0/2383849624 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7564013ad0 con 0x56121afbb3e0 2023-10-05T19:40:20.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.211+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56121b06f770 con 0x56121afbb3e0 2023-10-05T19:40:20.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.211+0000 7f75746e7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.211+0000 7f756a7fc700 1 -- 172.21.15.12:0/2383849624 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f756401a070 con 0x56121afbb3e0 2023-10-05T19:40:20.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.215+0000 7f756a7fc700 1 --2- 172.21.15.12:0/2383849624 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f7554030c50 0x7f7554033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.215+0000 7f756a7fc700 1 -- 172.21.15.12:0/2383849624 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f7564044d90 con 0x56121afbb3e0 2023-10-05T19:40:20.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.215+0000 7f7571a1b700 1 --2- 172.21.15.12:0/2383849624 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f7554030c50 0x7f7554033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.215+0000 7f75746e7e40 1 --2- 172.21.15.12:0/2383849624 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x56121b04b300 0x56121b07a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.215+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b07afb0 con 0x56121b04b300 2023-10-05T19:40:20.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.215+0000 7f7572a1d700 1 --2- 172.21.15.12:0/2383849624 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x56121b04b300 0x56121b07a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.215+0000 7f7571a1b700 1 --2- 172.21.15.12:0/2383849624 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f7554030c50 0x7f7554033100 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f755c005a90 tx=0x7f755c015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.215+0000 7f7572a1d700 1 --2- 172.21.15.12:0/2383849624 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x56121b04b300 0x56121b07a8b0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.215+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f756c00a500 con 0x56121b04b300 2023-10-05T19:40:20.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.215+0000 7f75746e7e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.215+0000 7f75746e7e40 20 realm 2023-10-05T19:40:20.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.215+0000 7f75746e7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.215+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b077f10 con 0x56121b04b300 2023-10-05T19:40:20.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.215+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f756c00a500 con 0x56121b04b300 2023-10-05T19:40:20.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.215+0000 7f75746e7e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.215+0000 7f75746e7e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:20.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.215+0000 7f75746e7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.215+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b07ba90 con 0x56121b04b300 2023-10-05T19:40:20.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.215+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f756c00a500 con 0x56121b04b300 2023-10-05T19:40:20.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.215+0000 7f75746e7e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.215+0000 7f75746e7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.215+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b07b6a0 con 0x56121b04b300 2023-10-05T19:40:20.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.219+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f756c00a500 con 0x56121b04b300 2023-10-05T19:40:20.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.219+0000 7f75746e7e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:20.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.219+0000 7f75746e7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.219+0000 7f75746e7e40 1 --2- 172.21.15.12:0/2383849624 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x56121b07ce30 0x56121b07f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.219+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b07f750 con 0x56121b07ce30 2023-10-05T19:40:20.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.219+0000 7f757221c700 1 --2- 172.21.15.12:0/2383849624 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x56121b07ce30 0x56121b07f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.219+0000 7f757221c700 1 --2- 172.21.15.12:0/2383849624 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x56121b07ce30 0x56121b07f210 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.219+0000 7f757221c700 1 -- 172.21.15.12:0/2383849624 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f7564025020 con 0x56121b07ce30 2023-10-05T19:40:20.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.219+0000 7f75746e7e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:20.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.219+0000 7f75746e7e40 20 searching for the correct realm 2023-10-05T19:40:20.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.219+0000 7f75746e7e40 1 --2- 172.21.15.12:0/2383849624 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x56121b0815d0 0x56121b0839e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.219+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084090 con 0x56121b0815d0 2023-10-05T19:40:20.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.219+0000 7f7571a1b700 1 --2- 172.21.15.12:0/2383849624 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x56121b0815d0 0x56121b0839e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.219+0000 7f7571a1b700 1 --2- 172.21.15.12:0/2383849624 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x56121b0815d0 0x56121b0839e0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.223+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f755c008a80 con 0x56121b0815d0 2023-10-05T19:40:20.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.223+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084090 con 0x56121b07ce30 2023-10-05T19:40:20.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.223+0000 7f757221c700 1 -- 172.21.15.12:0/2383849624 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7564025020 con 0x56121b07ce30 2023-10-05T19:40:20.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.223+0000 7f757221c700 1 --2- 172.21.15.12:0/2383849624 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f75640106e0 0x7f75640113e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.223+0000 7f757221c700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084090 con 0x7f75640106e0 2023-10-05T19:40:20.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.223+0000 7f7572a1d700 1 --2- 172.21.15.12:0/2383849624 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f75640106e0 0x7f75640113e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.223+0000 7f7572a1d700 1 --2- 172.21.15.12:0/2383849624 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f75640106e0 0x7f75640113e0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.223+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f756c014040 con 0x7f75640106e0 2023-10-05T19:40:20.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.223+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084090 con 0x56121b04b300 2023-10-05T19:40:20.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.227+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f756c014040 con 0x56121b04b300 2023-10-05T19:40:20.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.227+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084090 con 0x56121b04b300 2023-10-05T19:40:20.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.227+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f756c014040 con 0x56121b04b300 2023-10-05T19:40:20.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.227+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084090 con 0x7f75640106e0 2023-10-05T19:40:20.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.227+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f756c014040 con 0x7f75640106e0 2023-10-05T19:40:20.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.227+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084090 con 0x7f75640106e0 2023-10-05T19:40:20.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.227+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f756c014040 con 0x7f75640106e0 2023-10-05T19:40:20.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.227+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084090 con 0x56121b07ce30 2023-10-05T19:40:20.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.227+0000 7f757221c700 1 -- 172.21.15.12:0/2383849624 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7564025020 con 0x56121b07ce30 2023-10-05T19:40:20.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.227+0000 7f757221c700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084090 con 0x56121b0815d0 2023-10-05T19:40:20.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.231+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f755c008a80 con 0x56121b0815d0 2023-10-05T19:40:20.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.231+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084090 con 0x56121b04b300 2023-10-05T19:40:20.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.231+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f756c014040 con 0x56121b04b300 2023-10-05T19:40:20.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.231+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084090 con 0x7f75640106e0 2023-10-05T19:40:20.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.231+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f756c014040 con 0x7f75640106e0 2023-10-05T19:40:20.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.231+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084090 con 0x56121b04b300 2023-10-05T19:40:20.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.231+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f756c014040 con 0x56121b04b300 2023-10-05T19:40:20.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.231+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084090 con 0x56121b07ce30 2023-10-05T19:40:20.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.231+0000 7f757221c700 1 -- 172.21.15.12:0/2383849624 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7564025020 con 0x56121b07ce30 2023-10-05T19:40:20.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.231+0000 7f757221c700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084090 con 0x56121b0815d0 2023-10-05T19:40:20.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.235+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f755c008a80 con 0x56121b0815d0 2023-10-05T19:40:20.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.235+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084090 con 0x56121b0815d0 2023-10-05T19:40:20.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.235+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f755c008a80 con 0x56121b0815d0 2023-10-05T19:40:20.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.235+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084090 con 0x7f75640106e0 2023-10-05T19:40:20.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.235+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f756c014040 con 0x7f75640106e0 2023-10-05T19:40:20.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.235+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084090 con 0x56121b0815d0 2023-10-05T19:40:20.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.235+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f755c008a80 con 0x56121b0815d0 2023-10-05T19:40:20.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.235+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084090 con 0x56121b0815d0 2023-10-05T19:40:20.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.235+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f755c008a80 con 0x56121b0815d0 2023-10-05T19:40:20.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.235+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084090 con 0x56121b04b300 2023-10-05T19:40:20.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.239+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f756c014040 con 0x56121b04b300 2023-10-05T19:40:20.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.239+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084090 con 0x56121b04b300 2023-10-05T19:40:20.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.239+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f756c014040 con 0x56121b04b300 2023-10-05T19:40:20.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.239+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084090 con 0x7f75640106e0 2023-10-05T19:40:20.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.239+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f756c014040 con 0x7f75640106e0 2023-10-05T19:40:20.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.239+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084090 con 0x56121b0815d0 2023-10-05T19:40:20.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.239+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f755c008a80 con 0x56121b0815d0 2023-10-05T19:40:20.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.239+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084090 con 0x56121b07ce30 2023-10-05T19:40:20.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.239+0000 7f757221c700 1 -- 172.21.15.12:0/2383849624 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7564025020 con 0x56121b07ce30 2023-10-05T19:40:20.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.239+0000 7f757221c700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084090 con 0x56121b04b300 2023-10-05T19:40:20.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.243+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f756c014040 con 0x56121b04b300 2023-10-05T19:40:20.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.243+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084090 con 0x7f75640106e0 2023-10-05T19:40:20.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.243+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f756c014040 con 0x7f75640106e0 2023-10-05T19:40:20.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.243+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084090 con 0x56121b0815d0 2023-10-05T19:40:20.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.243+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f755c008a80 con 0x56121b0815d0 2023-10-05T19:40:20.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.243+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084090 con 0x7f75640106e0 2023-10-05T19:40:20.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.243+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f756c014040 con 0x7f75640106e0 2023-10-05T19:40:20.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.243+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084090 con 0x56121b07ce30 2023-10-05T19:40:20.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.247+0000 7f757221c700 1 -- 172.21.15.12:0/2383849624 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7564025020 con 0x56121b07ce30 2023-10-05T19:40:20.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.247+0000 7f757221c700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084090 con 0x56121b0815d0 2023-10-05T19:40:20.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.247+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f755c008a80 con 0x56121b0815d0 2023-10-05T19:40:20.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.247+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084090 con 0x7f75640106e0 2023-10-05T19:40:20.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.247+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f756c014040 con 0x7f75640106e0 2023-10-05T19:40:20.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.247+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084090 con 0x56121b04b300 2023-10-05T19:40:20.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.247+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f756c014040 con 0x56121b04b300 2023-10-05T19:40:20.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.247+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084090 con 0x56121b07ce30 2023-10-05T19:40:20.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.247+0000 7f757221c700 1 -- 172.21.15.12:0/2383849624 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f7564025020 con 0x56121b07ce30 2023-10-05T19:40:20.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.247+0000 7f75746e7e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:20.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.251+0000 7f75746e7e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:20.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.251+0000 7f75746e7e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:20.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.251+0000 7f75746e7e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:20.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.251+0000 7f75746e7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.251+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b080350 con 0x56121b04b300 2023-10-05T19:40:20.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.251+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f756c014040 con 0x56121b04b300 2023-10-05T19:40:20.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.251+0000 7f75746e7e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.251+0000 7f75746e7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.251+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b07ff80 con 0x56121b04b300 2023-10-05T19:40:20.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.251+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f756c014040 con 0x56121b04b300 2023-10-05T19:40:20.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.251+0000 7f75746e7e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:20.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.251+0000 7f75746e7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.251+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084500 con 0x56121b0815d0 2023-10-05T19:40:20.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.251+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f755c008a80 con 0x56121b0815d0 2023-10-05T19:40:20.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.251+0000 7f75746e7e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:20.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.251+0000 7f75746e7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.251+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b0848b0 con 0x56121b0815d0 2023-10-05T19:40:20.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.251+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f755c008a80 con 0x56121b0815d0 2023-10-05T19:40:20.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.251+0000 7f75746e7e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.251+0000 7f75746e7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.251+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084c60 con 0x56121b04b300 2023-10-05T19:40:20.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.251+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f756c014040 con 0x56121b04b300 2023-10-05T19:40:20.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.251+0000 7f75746e7e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.251+0000 7f75746e7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.251+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b085070 con 0x56121b04b300 2023-10-05T19:40:20.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.255+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f756c014040 con 0x56121b04b300 2023-10-05T19:40:20.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.255+0000 7f75746e7e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:20.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.255+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b085040 con 0x56121b0815d0 2023-10-05T19:40:20.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.255+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f755c008a80 con 0x56121b0815d0 2023-10-05T19:40:20.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.255+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b085040 con 0x56121b07ce30 2023-10-05T19:40:20.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.255+0000 7f757221c700 1 -- 172.21.15.12:0/2383849624 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7564025020 con 0x56121b07ce30 2023-10-05T19:40:20.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.255+0000 7f757221c700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b085040 con 0x7f75640106e0 2023-10-05T19:40:20.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.259+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f756c014040 con 0x7f75640106e0 2023-10-05T19:40:20.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.259+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084c60 con 0x56121b04b300 2023-10-05T19:40:20.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.259+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f756c014040 con 0x56121b04b300 2023-10-05T19:40:20.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.259+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084c60 con 0x56121b04b300 2023-10-05T19:40:20.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.259+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f756c014040 con 0x56121b04b300 2023-10-05T19:40:20.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.259+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084c60 con 0x7f75640106e0 2023-10-05T19:40:20.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.259+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f756c014040 con 0x7f75640106e0 2023-10-05T19:40:20.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.259+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084c60 con 0x7f75640106e0 2023-10-05T19:40:20.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.259+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f756c014040 con 0x7f75640106e0 2023-10-05T19:40:20.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.263+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084c60 con 0x56121b07ce30 2023-10-05T19:40:20.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.263+0000 7f757221c700 1 -- 172.21.15.12:0/2383849624 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7564025020 con 0x56121b07ce30 2023-10-05T19:40:20.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.263+0000 7f757221c700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084c60 con 0x56121b0815d0 2023-10-05T19:40:20.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.263+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f755c008a80 con 0x56121b0815d0 2023-10-05T19:40:20.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.263+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084c60 con 0x56121b04b300 2023-10-05T19:40:20.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.263+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f756c014040 con 0x56121b04b300 2023-10-05T19:40:20.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.263+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084c60 con 0x7f75640106e0 2023-10-05T19:40:20.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.263+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f756c014040 con 0x7f75640106e0 2023-10-05T19:40:20.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.263+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084c60 con 0x56121b04b300 2023-10-05T19:40:20.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.267+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f756c014040 con 0x56121b04b300 2023-10-05T19:40:20.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.267+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084c60 con 0x56121b07ce30 2023-10-05T19:40:20.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.267+0000 7f757221c700 1 -- 172.21.15.12:0/2383849624 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7564025020 con 0x56121b07ce30 2023-10-05T19:40:20.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.267+0000 7f757221c700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084c60 con 0x56121b0815d0 2023-10-05T19:40:20.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.267+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f755c008a80 con 0x56121b0815d0 2023-10-05T19:40:20.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.267+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084c60 con 0x56121b0815d0 2023-10-05T19:40:20.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.267+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f755c008a80 con 0x56121b0815d0 2023-10-05T19:40:20.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.267+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084c60 con 0x7f75640106e0 2023-10-05T19:40:20.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.267+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f756c014040 con 0x7f75640106e0 2023-10-05T19:40:20.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.271+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084c60 con 0x56121b0815d0 2023-10-05T19:40:20.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.271+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f755c008a80 con 0x56121b0815d0 2023-10-05T19:40:20.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.271+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084c60 con 0x56121b0815d0 2023-10-05T19:40:20.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.271+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f755c008a80 con 0x56121b0815d0 2023-10-05T19:40:20.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.271+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084c60 con 0x56121b04b300 2023-10-05T19:40:20.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.271+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f756c014040 con 0x56121b04b300 2023-10-05T19:40:20.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.271+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084c60 con 0x56121b04b300 2023-10-05T19:40:20.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.271+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f756c014040 con 0x56121b04b300 2023-10-05T19:40:20.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.271+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084c60 con 0x7f75640106e0 2023-10-05T19:40:20.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.271+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f756c014040 con 0x7f75640106e0 2023-10-05T19:40:20.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.275+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084c60 con 0x56121b0815d0 2023-10-05T19:40:20.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.275+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f755c008a80 con 0x56121b0815d0 2023-10-05T19:40:20.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.275+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084c60 con 0x56121b07ce30 2023-10-05T19:40:20.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.275+0000 7f757221c700 1 -- 172.21.15.12:0/2383849624 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7564025020 con 0x56121b07ce30 2023-10-05T19:40:20.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.275+0000 7f757221c700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084c60 con 0x56121b04b300 2023-10-05T19:40:20.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.275+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f756c014040 con 0x56121b04b300 2023-10-05T19:40:20.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.275+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084c60 con 0x7f75640106e0 2023-10-05T19:40:20.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.275+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f756c014040 con 0x7f75640106e0 2023-10-05T19:40:20.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.275+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084c60 con 0x56121b0815d0 2023-10-05T19:40:20.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.279+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f755c008a80 con 0x56121b0815d0 2023-10-05T19:40:20.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.279+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084c60 con 0x7f75640106e0 2023-10-05T19:40:20.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.279+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f756c014040 con 0x7f75640106e0 2023-10-05T19:40:20.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.279+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084c60 con 0x56121b07ce30 2023-10-05T19:40:20.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.279+0000 7f757221c700 1 -- 172.21.15.12:0/2383849624 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7564025020 con 0x56121b07ce30 2023-10-05T19:40:20.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.279+0000 7f757221c700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084c60 con 0x56121b0815d0 2023-10-05T19:40:20.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.279+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f755c008a80 con 0x56121b0815d0 2023-10-05T19:40:20.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.279+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084c60 con 0x7f75640106e0 2023-10-05T19:40:20.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.283+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f756c014040 con 0x7f75640106e0 2023-10-05T19:40:20.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.283+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084c60 con 0x56121b04b300 2023-10-05T19:40:20.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.283+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f756c014040 con 0x56121b04b300 2023-10-05T19:40:20.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.283+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b084c60 con 0x56121b07ce30 2023-10-05T19:40:20.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.283+0000 7f757221c700 1 -- 172.21.15.12:0/2383849624 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f7564025020 con 0x56121b07ce30 2023-10-05T19:40:20.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.283+0000 7f75746e7e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:20.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.283+0000 7f75746e7e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:20.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.283+0000 7f75746e7e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:20.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.283+0000 7f75746e7e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:20.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.283+0000 7f75746e7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.283+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b085040 con 0x56121b04b300 2023-10-05T19:40:20.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.283+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f756c014040 con 0x56121b04b300 2023-10-05T19:40:20.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.283+0000 7f75746e7e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:20.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.283+0000 7f75746e7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.283+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b0854a0 con 0x56121b07ce30 2023-10-05T19:40:20.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.283+0000 7f757221c700 1 -- 172.21.15.12:0/2383849624 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f7564025020 con 0x56121b07ce30 2023-10-05T19:40:20.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.283+0000 7f75746e7e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:20.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.283+0000 7f75746e7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.283+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b085d90 con 0x56121b04b300 2023-10-05T19:40:20.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.287+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f756c014040 con 0x56121b04b300 2023-10-05T19:40:20.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.287+0000 7f75746e7e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:20.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.287+0000 7f75746e7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.287+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b085d90 con 0x56121b0815d0 2023-10-05T19:40:20.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.287+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f755c008a80 con 0x56121b0815d0 2023-10-05T19:40:20.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.287+0000 7f75746e7e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:20.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.287+0000 7f75746e7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.287+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b086500 con 0x7f75640106e0 2023-10-05T19:40:20.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.287+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f756c014040 con 0x7f75640106e0 2023-10-05T19:40:20.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.287+0000 7f75746e7e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.287+0000 7f75746e7e40 20 zone default found 2023-10-05T19:40:20.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.287+0000 7f75746e7e40 4 Realm: () 2023-10-05T19:40:20.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.287+0000 7f75746e7e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:20.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.287+0000 7f75746e7e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:20.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.287+0000 7f75746e7e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:20.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.287+0000 7f75746e7e40 20 zonegroup default 2023-10-05T19:40:20.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.287+0000 7f75746e7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.287+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b086240 con 0x7f75640106e0 2023-10-05T19:40:20.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.287+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f756c014040 con 0x7f75640106e0 2023-10-05T19:40:20.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.287+0000 7f75746e7e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.287+0000 7f75746e7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.287+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b087bc0 con 0x56121b0815d0 2023-10-05T19:40:20.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.287+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f755c008a80 con 0x56121b0815d0 2023-10-05T19:40:20.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.291+0000 7f75746e7e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.291+0000 7f75746e7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.291+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b087f70 con 0x56121b0815d0 2023-10-05T19:40:20.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.291+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f755c008a80 con 0x56121b0815d0 2023-10-05T19:40:20.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.291+0000 7f75746e7e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.291+0000 7f75746e7e40 20 started sync module instance, tier type = 2023-10-05T19:40:20.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.291+0000 7f75746e7e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:20.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.291+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b087280 con 0x56121b07ce30 2023-10-05T19:40:20.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.291+0000 7f757221c700 1 -- 172.21.15.12:0/2383849624 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'51 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7564025020 con 0x56121b07ce30 2023-10-05T19:40:20.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.291+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94635762939520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b0859c0 con 0x56121b07ce30 2023-10-05T19:40:20.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.291+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b088d40 con 0x56121b04b300 2023-10-05T19:40:20.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.295+0000 7f757221c700 1 -- 172.21.15.12:0/2383849624 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94635762939520] v28'52 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7564025020 con 0x56121b07ce30 2023-10-05T19:40:20.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.295+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'51 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f756c014040 con 0x56121b04b300 2023-10-05T19:40:20.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.295+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94635762942752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b089140 con 0x56121b04b300 2023-10-05T19:40:20.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.295+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b0899e0 con 0x56121b0815d0 2023-10-05T19:40:20.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.299+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94635762942752] v28'52 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f756c014040 con 0x56121b04b300 2023-10-05T19:40:20.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.299+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'51 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f755c008a80 con 0x56121b0815d0 2023-10-05T19:40:20.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.299+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94635762946368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b0888d0 con 0x56121b0815d0 2023-10-05T19:40:20.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.299+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b08a7c0 con 0x56121b07ce30 2023-10-05T19:40:20.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.303+0000 7f757221c700 1 -- 172.21.15.12:0/2383849624 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'51 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7564025020 con 0x56121b07ce30 2023-10-05T19:40:20.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.303+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94635762946368] v28'52 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f755c008a80 con 0x56121b0815d0 2023-10-05T19:40:20.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.303+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94635762949600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b08ab70 con 0x56121b07ce30 2023-10-05T19:40:20.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.303+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b08b680 con 0x56121b04b300 2023-10-05T19:40:20.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.303+0000 7f757221c700 1 -- 172.21.15.12:0/2383849624 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94635762949600] v28'52 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7564025020 con 0x56121b07ce30 2023-10-05T19:40:20.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.303+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'51 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f756c014040 con 0x56121b04b300 2023-10-05T19:40:20.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.303+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94635762950704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b08b640 con 0x56121b04b300 2023-10-05T19:40:20.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.303+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b08c050 con 0x7f75640106e0 2023-10-05T19:40:20.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.307+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94635762950704] v28'52 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f756c014040 con 0x56121b04b300 2023-10-05T19:40:20.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.307+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'51 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f756c014040 con 0x7f75640106e0 2023-10-05T19:40:20.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.307+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94635762957712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b08b640 con 0x7f75640106e0 2023-10-05T19:40:20.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.307+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b08cab0 con 0x7f75640106e0 2023-10-05T19:40:20.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.311+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(95 notify.5 [watch watch cookie 94635762957712] v28'52 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f756c014040 con 0x7f75640106e0 2023-10-05T19:40:20.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.311+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(96 notify.6 [create] v28'51 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f756c014040 con 0x7f75640106e0 2023-10-05T19:40:20.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.311+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94635762954848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b08cab0 con 0x7f75640106e0 2023-10-05T19:40:20.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.311+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b08d390 con 0x7f75640106e0 2023-10-05T19:40:20.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.311+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(98 notify.7 [create] v28'51 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f756c014040 con 0x7f75640106e0 2023-10-05T19:40:20.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.311+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94635762961072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b08cf00 con 0x7f75640106e0 2023-10-05T19:40:20.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.311+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(97 notify.6 [watch watch cookie 94635762954848] v28'52 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f756c014040 con 0x7f75640106e0 2023-10-05T19:40:20.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.311+0000 7f75746e7e40 20 add_watcher() i=0 2023-10-05T19:40:20.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.311+0000 7f75746e7e40 20 add_watcher() i=1 2023-10-05T19:40:20.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.311+0000 7f75746e7e40 20 add_watcher() i=2 2023-10-05T19:40:20.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.311+0000 7f75746e7e40 20 add_watcher() i=3 2023-10-05T19:40:20.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.311+0000 7f75746e7e40 20 add_watcher() i=4 2023-10-05T19:40:20.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.311+0000 7f75746e7e40 20 add_watcher() i=5 2023-10-05T19:40:20.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.311+0000 7f75746e7e40 20 add_watcher() i=6 2023-10-05T19:40:20.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.315+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94635762961072] v28'52 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f756c014040 con 0x7f75640106e0 2023-10-05T19:40:20.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.315+0000 7f75746e7e40 20 add_watcher() i=7 2023-10-05T19:40:20.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.315+0000 7f75746e7e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:20.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.315+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b0907c0 con 0x56121b04b300 2023-10-05T19:40:20.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.315+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f756c014040 con 0x56121b04b300 2023-10-05T19:40:20.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.315+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94635762959808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b08dc30 con 0x56121b04b300 2023-10-05T19:40:20.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.319+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94635762959808] v28'68 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f756c014040 con 0x56121b04b300 2023-10-05T19:40:20.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.319+0000 7f7543fdf700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:20.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.319+0000 7f75746e7e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:20.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.319+0000 7f75746e7e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:20.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.319+0000 7f75746e7e40 1 Processor -- start 2023-10-05T19:40:20.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.319+0000 7f75746e7e40 1 -- start start 2023-10-05T19:40:20.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.319+0000 7f75746e7e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56121b09ad80 0x56121b09b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.319+0000 7f75746e7e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56121b09b690 con 0x56121b09ad80 2023-10-05T19:40:20.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.319+0000 7f7571a1b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56121b09ad80 0x56121b09b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.319+0000 7f7571a1b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56121b09ad80 0x56121b09b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37054/0 (socket says 172.21.15.12:37054) 2023-10-05T19:40:20.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.319+0000 7f7571a1b700 1 -- 172.21.15.12:0/1397959444 learned_addr learned my addr 172.21.15.12:0/1397959444 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:20.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.319+0000 7f7571a1b700 1 -- 172.21.15.12:0/1397959444 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56121b09b7d0 con 0x56121b09ad80 2023-10-05T19:40:20.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.319+0000 7f7571a1b700 1 --2- 172.21.15.12:0/1397959444 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56121b09ad80 0x56121b09b150 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x56121afbac70 tx=0x7f755c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bcbdcd35aef7465 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.323+0000 7f75437de700 1 -- 172.21.15.12:0/1397959444 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f755c030930 con 0x56121b09ad80 2023-10-05T19:40:20.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.323+0000 7f75437de700 1 -- 172.21.15.12:0/1397959444 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f755c030ab0 con 0x56121b09ad80 2023-10-05T19:40:20.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.323+0000 7f75437de700 1 -- 172.21.15.12:0/1397959444 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f755c030c60 con 0x56121b09ad80 2023-10-05T19:40:20.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.323+0000 7f75746e7e40 1 -- 172.21.15.12:0/1397959444 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56121b09ad80 msgr2=0x56121b09b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.323+0000 7f75746e7e40 1 --2- 172.21.15.12:0/1397959444 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56121b09ad80 0x56121b09b150 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x56121afbac70 tx=0x7f755c012000 comp rx=0 tx=0).stop 2023-10-05T19:40:20.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.323+0000 7f75746e7e40 1 -- 172.21.15.12:0/1397959444 shutdown_connections 2023-10-05T19:40:20.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.323+0000 7f75746e7e40 1 --2- 172.21.15.12:0/1397959444 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56121b09ad80 0x56121b09b150 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:20.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.323+0000 7f75746e7e40 1 -- 172.21.15.12:0/1397959444 >> 172.21.15.12:0/1397959444 conn(0x56121b096140 msgr2=0x56121b094290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:20.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.323+0000 7f75746e7e40 1 -- 172.21.15.12:0/1397959444 shutdown_connections 2023-10-05T19:40:20.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.323+0000 7f75746e7e40 1 -- 172.21.15.12:0/1397959444 wait complete. 2023-10-05T19:40:20.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.323+0000 7f75746e7e40 1 Processor -- start 2023-10-05T19:40:20.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.323+0000 7f75746e7e40 1 -- start start 2023-10-05T19:40:20.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.323+0000 7f75746e7e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56121b09ad80 0x56121b0a08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.323+0000 7f75746e7e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56121b0a18d0 con 0x56121b09ad80 2023-10-05T19:40:20.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.323+0000 7f7571a1b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56121b09ad80 0x56121b0a08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.323+0000 7f7571a1b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56121b09ad80 0x56121b0a08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37070/0 (socket says 172.21.15.12:37070) 2023-10-05T19:40:20.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.323+0000 7f7571a1b700 1 -- 172.21.15.12:0/3886885821 learned_addr learned my addr 172.21.15.12:0/3886885821 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:20.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.327+0000 7f7571a1b700 1 -- 172.21.15.12:0/3886885821 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56121b04ec30 con 0x56121b09ad80 2023-10-05T19:40:20.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.327+0000 7f7571a1b700 1 --2- 172.21.15.12:0/3886885821 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56121b09ad80 0x56121b0a08d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x56121b094e00 tx=0x7f755c034000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.327+0000 7f7541fdb700 1 -- 172.21.15.12:0/3886885821 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f755c027690 con 0x56121b09ad80 2023-10-05T19:40:20.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.327+0000 7f7541fdb700 1 -- 172.21.15.12:0/3886885821 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f755c0277f0 con 0x56121b09ad80 2023-10-05T19:40:20.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.327+0000 7f75746e7e40 1 -- 172.21.15.12:0/3886885821 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56121b0a0ed0 con 0x56121b09ad80 2023-10-05T19:40:20.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.327+0000 7f7541fdb700 1 -- 172.21.15.12:0/3886885821 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f755c0279a0 con 0x56121b09ad80 2023-10-05T19:40:20.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.327+0000 7f75746e7e40 1 -- 172.21.15.12:0/3886885821 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56121b0a1310 con 0x56121b09ad80 2023-10-05T19:40:20.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.327+0000 7f7541fdb700 1 -- 172.21.15.12:0/3886885821 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f755c027b00 con 0x56121b09ad80 2023-10-05T19:40:20.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.327+0000 7f7541fdb700 1 --2- 172.21.15.12:0/3886885821 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f74b8030c50 0x7f74b8033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.327+0000 7f7541fdb700 1 -- 172.21.15.12:0/3886885821 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f755c054ab0 con 0x56121b09ad80 2023-10-05T19:40:20.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.327+0000 7f7572a1d700 1 --2- 172.21.15.12:0/3886885821 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f74b8030c50 0x7f74b8033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.327+0000 7f75746e7e40 5 note: GC not initialized 2023-10-05T19:40:20.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.331+0000 7f7572a1d700 1 --2- 172.21.15.12:0/3886885821 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f74b8030c50 0x7f74b8033100 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x56121b08d590 tx=0x7f756c00a9b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.331+0000 7f74b2ffd700 20 reqs_thread_entry: start 2023-10-05T19:40:20.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.331+0000 7f75746e7e40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:20.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.331+0000 7f74b0ff9700 20 reqs_thread_entry: start 2023-10-05T19:40:20.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.331+0000 7f75746e7e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:20.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.331+0000 7f74a2ffd700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:50 2023 2023-10-05T19:40:20.342 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:20.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.331+0000 7f74a27fc700 20 reqs_thread_entry: start 2023-10-05T19:40:20.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.331+0000 7f75746e7e40 20 get_system_obj_state: rctx=0x7ffd8fc545e8 obj=default.rgw.meta:users.uid:foo state=0x56121af200b0 s->prefetch_data=0 2023-10-05T19:40:20.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.331+0000 7f75746e7e40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2023-10-05T19:40:20.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.331+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b109160 con 0x56121b04b300 2023-10-05T19:40:20.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.335+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(102 foo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7f756c014040 con 0x56121b04b300 2023-10-05T19:40:20.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.335+0000 7f75746e7e40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x16 2023-10-05T19:40:20.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.335+0000 7f75746e7e40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:20.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.335+0000 7f75746e7e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:20.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.335+0000 7f75746e7e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:20.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.335+0000 7f75746e7e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:20.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.335+0000 7f75746e7e40 10 cache get: name=default.rgw.meta+users.uid+foo : type miss (requested=0x13, cached=0x16) 2023-10-05T19:40:20.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.335+0000 7f75746e7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.335+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b10bbf0 con 0x56121b04b300 2023-10-05T19:40:20.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.335+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(103 foo [call,call out=48b,read 0~459 out=459b,getxattrs out=75b] v0'0 uv4 ondisk = 0) v8 ==== 273+0+582 (crc 0 0 0) 0x7f756c014040 con 0x56121b04b300 2023-10-05T19:40:20.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.335+0000 7f75746e7e40 20 rados_obj.operate() r=0 bl.length=459 2023-10-05T19:40:20.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.335+0000 7f75746e7e40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2023-10-05T19:40:20.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.335+0000 7f75746e7e40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:20.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.335+0000 7f75746e7e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:20.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.335+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:104 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.set in=58b,writefull 0~357 in=357b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b108490 con 0x56121b04b300 2023-10-05T19:40:20.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.339+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(104 foo [delete,create,call,writefull 0~357] v28'5 uv5 ondisk = 0) v8 ==== 273+0+0 (crc 0 0 0) 0x7f756c014040 con 0x56121b04b300 2023-10-05T19:40:20.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.339+0000 7f75746e7e40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2023-10-05T19:40:20.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.339+0000 7f75746e7e40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:20.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.339+0000 7f75746e7e40 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2023-10-05T19:40:20.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.339+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:105 7.d 7:b4812045:::notify.4:head [notify cookie 94635763466464 in=545b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b10cc00 con 0x56121b04b300 2023-10-05T19:40:20.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.339+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== watch-notify(notify (1) cookie 94635762950704 notify 120259084292 ret 0) v3 ==== 575+0+0 (crc 0 0 0) 0x7f756c032600 con 0x56121b04b300 2023-10-05T19:40:20.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.339+0000 7f756bfff700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084292 cookie 94635762950704 notifier 4327 bl.length()=533 2023-10-05T19:40:20.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.339+0000 7f756bfff700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2023-10-05T19:40:20.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.339+0000 7f756bfff700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:20.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.339+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(105 notify.4 [notify cookie 94635763466464 out=8b] v0'0 uv51 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f756c014040 con 0x56121b04b300 2023-10-05T19:40:20.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.339+0000 7f756bfff700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:106 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f7558003d90 con 0x56121b04b300 2023-10-05T19:40:20.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.339+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== osd_op_reply(106 notify.4 [notify-ack] v0'0 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f756c014040 con 0x56121b04b300 2023-10-05T19:40:20.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.339+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.1 v2:172.21.15.12:6808/29933 39 ==== watch-notify(notify_complete (2) cookie 94635763466464 notify 120259084292 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f756c0264a0 con 0x56121b04b300 2023-10-05T19:40:20.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.339+0000 7f75746e7e40 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2023-10-05T19:40:20.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.339+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:107 7.15 7:a93a5511:::notify.2:head [notify cookie 94635763466464 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b10c710 con 0x56121b0815d0 2023-10-05T19:40:20.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.339+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== watch-notify(notify (1) cookie 94635762946368 notify 120259084289 ret 0) v3 ==== 212+0+0 (crc 0 0 0) 0x56121b0a1310 con 0x56121b0815d0 2023-10-05T19:40:20.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.339+0000 7f756b7fe700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084289 cookie 94635762946368 notifier 4327 bl.length()=170 2023-10-05T19:40:20.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.339+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 <== osd.3 v2:172.21.15.12:6816/29951 27 ==== osd_op_reply(107 notify.2 [notify cookie 94635763466464 out=8b] v0'0 uv51 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f755c008a80 con 0x56121b0815d0 2023-10-05T19:40:20.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.339+0000 7f756b7fe700 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:108 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f74c00032d0 con 0x56121b0815d0 2023-10-05T19:40:20.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.343+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 <== osd.3 v2:172.21.15.12:6816/29951 28 ==== osd_op_reply(108 notify.2 [notify-ack] v0'0 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f755c008a80 con 0x56121b0815d0 2023-10-05T19:40:20.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.343+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 <== osd.3 v2:172.21.15.12:6816/29951 29 ==== watch-notify(notify_complete (2) cookie 94635763466464 notify 120259084289 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x56121b0a0ed0 con 0x56121b0815d0 2023-10-05T19:40:20.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.343+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:109 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b1080e0 con 0x7f75640106e0 2023-10-05T19:40:20.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.343+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(109 p5YnriCv1nAtykxBrupQ [delete] v28'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f756c014040 con 0x7f75640106e0 2023-10-05T19:40:20.354 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:40:20.354 INFO:teuthology.orchestra.run.smithi012.stdout: "user_id": "foo", 2023-10-05T19:40:20.354 INFO:teuthology.orchestra.run.smithi012.stdout: "display_name": "Foo", 2023-10-05T19:40:20.354 INFO:teuthology.orchestra.run.smithi012.stdout: "email": "foo@foo.com", 2023-10-05T19:40:20.354 INFO:teuthology.orchestra.run.smithi012.stdout: "suspended": 0, 2023-10-05T19:40:20.354 INFO:teuthology.orchestra.run.smithi012.stdout: "max_buckets": 4, 2023-10-05T19:40:20.354 INFO:teuthology.orchestra.run.smithi012.stdout: "subusers": [], 2023-10-05T19:40:20.354 INFO:teuthology.orchestra.run.smithi012.stdout: "keys": [ 2023-10-05T19:40:20.354 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:20.354 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:40:20.355 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2023-10-05T19:40:20.355 INFO:teuthology.orchestra.run.smithi012.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2023-10-05T19:40:20.355 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:20.355 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:20.355 INFO:teuthology.orchestra.run.smithi012.stdout: "swift_keys": [], 2023-10-05T19:40:20.355 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": [], 2023-10-05T19:40:20.355 INFO:teuthology.orchestra.run.smithi012.stdout: "op_mask": "read, write, delete", 2023-10-05T19:40:20.355 INFO:teuthology.orchestra.run.smithi012.stdout: "default_placement": "", 2023-10-05T19:40:20.355 INFO:teuthology.orchestra.run.smithi012.stdout: "default_storage_class": "", 2023-10-05T19:40:20.355 INFO:teuthology.orchestra.run.smithi012.stdout: "placement_tags": [], 2023-10-05T19:40:20.355 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket_quota": { 2023-10-05T19:40:20.355 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:20.356 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:20.356 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:20.356 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:20.356 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:20.356 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:20.356 INFO:teuthology.orchestra.run.smithi012.stdout: "user_quota": { 2023-10-05T19:40:20.356 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:20.356 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:20.356 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:20.356 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:20.356 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:20.356 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:20.357 INFO:teuthology.orchestra.run.smithi012.stdout: "temp_url_keys": [], 2023-10-05T19:40:20.357 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "rgw", 2023-10-05T19:40:20.357 INFO:teuthology.orchestra.run.smithi012.stdout: "mfa_ids": [] 2023-10-05T19:40:20.357 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:40:20.357 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:40:20.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.347+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:110 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94635762939520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b1080e0 con 0x56121b07ce30 2023-10-05T19:40:20.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.347+0000 7f757221c700 1 -- 172.21.15.12:0/2383849624 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(110 notify.0 [watch unwatch cookie 94635762939520] v28'53 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7564025020 con 0x56121b07ce30 2023-10-05T19:40:20.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.347+0000 7f75746e7e40 20 remove_watcher() i=0 2023-10-05T19:40:20.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.347+0000 7f75746e7e40 2 removed watcher, disabling cache 2023-10-05T19:40:20.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.347+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:111 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94635762942752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b1080e0 con 0x56121b04b300 2023-10-05T19:40:20.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.351+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.1 v2:172.21.15.12:6808/29933 40 ==== osd_op_reply(111 notify.1 [watch unwatch cookie 94635762942752] v28'53 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f756c014040 con 0x56121b04b300 2023-10-05T19:40:20.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.351+0000 7f75746e7e40 20 remove_watcher() i=1 2023-10-05T19:40:20.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.351+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:112 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94635762946368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b1080e0 con 0x56121b0815d0 2023-10-05T19:40:20.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.355+0000 7f7571a1b700 1 -- 172.21.15.12:0/2383849624 <== osd.3 v2:172.21.15.12:6816/29951 30 ==== osd_op_reply(112 notify.2 [watch unwatch cookie 94635762946368] v28'53 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f755c008a80 con 0x56121b0815d0 2023-10-05T19:40:20.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.355+0000 7f75746e7e40 20 remove_watcher() i=2 2023-10-05T19:40:20.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.355+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:113 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94635762949600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b1080e0 con 0x56121b07ce30 2023-10-05T19:40:20.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.355+0000 7f757221c700 1 -- 172.21.15.12:0/2383849624 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(113 notify.3 [watch unwatch cookie 94635762949600] v28'53 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7564025020 con 0x56121b07ce30 2023-10-05T19:40:20.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.355+0000 7f75746e7e40 20 remove_watcher() i=3 2023-10-05T19:40:20.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.355+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:114 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94635762950704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b1080e0 con 0x56121b04b300 2023-10-05T19:40:20.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.359+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.1 v2:172.21.15.12:6808/29933 41 ==== osd_op_reply(114 notify.4 [watch unwatch cookie 94635762950704] v28'53 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f756c014040 con 0x56121b04b300 2023-10-05T19:40:20.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.359+0000 7f75746e7e40 20 remove_watcher() i=4 2023-10-05T19:40:20.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.359+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:115 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94635762957712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b08a2d0 con 0x7f75640106e0 2023-10-05T19:40:20.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.363+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(115 notify.5 [watch unwatch cookie 94635762957712] v28'53 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f756c014040 con 0x7f75640106e0 2023-10-05T19:40:20.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.363+0000 7f75746e7e40 20 remove_watcher() i=5 2023-10-05T19:40:20.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.363+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:116 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94635762954848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b08a2d0 con 0x7f75640106e0 2023-10-05T19:40:20.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.363+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(116 notify.6 [watch unwatch cookie 94635762954848] v28'53 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f756c014040 con 0x7f75640106e0 2023-10-05T19:40:20.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.363+0000 7f75746e7e40 20 remove_watcher() i=6 2023-10-05T19:40:20.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.363+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:117 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94635762961072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b08a2d0 con 0x7f75640106e0 2023-10-05T19:40:20.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.367+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.0 v2:172.21.15.8:6800/27563 30 ==== osd_op_reply(117 notify.7 [watch unwatch cookie 94635762961072] v28'53 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f756c014040 con 0x7f75640106e0 2023-10-05T19:40:20.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.367+0000 7f75746e7e40 20 remove_watcher() i=7 2023-10-05T19:40:20.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.371+0000 7f7543fdf700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:20.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.371+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:118 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94635762959808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x56121b08a2d0 con 0x56121b04b300 2023-10-05T19:40:20.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.371+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 <== osd.1 v2:172.21.15.12:6808/29933 42 ==== osd_op_reply(118 data_loggenerations_metadata [watch unwatch cookie 94635762959808] v28'69 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f756c014040 con 0x56121b04b300 2023-10-05T19:40:20.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.371+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f75640106e0 msgr2=0x7f75640113e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.371+0000 7f75746e7e40 1 --2- 172.21.15.12:0/2383849624 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f75640106e0 0x7f75640113e0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:20.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.371+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x56121b04b300 msgr2=0x56121b07a8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.371+0000 7f75746e7e40 1 --2- 172.21.15.12:0/2383849624 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x56121b04b300 0x56121b07a8b0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:20.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.371+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x56121b07ce30 msgr2=0x56121b07f210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.371+0000 7f75746e7e40 1 --2- 172.21.15.12:0/2383849624 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x56121b07ce30 0x56121b07f210 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:20.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.371+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x56121b0815d0 msgr2=0x56121b0839e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.371+0000 7f75746e7e40 1 --2- 172.21.15.12:0/2383849624 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x56121b0815d0 0x56121b0839e0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:20.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.371+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f7554030c50 msgr2=0x7f7554033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.371+0000 7f75746e7e40 1 --2- 172.21.15.12:0/2383849624 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f7554030c50 0x7f7554033100 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f755c005a90 tx=0x7f755c015000 comp rx=0 tx=0).stop 2023-10-05T19:40:20.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.371+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56121afbb3e0 msgr2=0x56121b06edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.371+0000 7f7572a1d700 1 -- 172.21.15.12:0/2383849624 reap_dead start 2023-10-05T19:40:20.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.371+0000 7f75746e7e40 1 --2- 172.21.15.12:0/2383849624 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56121afbb3e0 0x56121b06edf0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7564000f80 tx=0x7f7564014000 comp rx=0 tx=0).stop 2023-10-05T19:40:20.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.371+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 shutdown_connections 2023-10-05T19:40:20.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.371+0000 7f75746e7e40 1 --2- 172.21.15.12:0/2383849624 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56121afbb3e0 0x56121b06edf0 secure :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7564000f80 tx=0x7f7564014000 comp rx=0 tx=0).stop 2023-10-05T19:40:20.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.371+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 >> 172.21.15.12:0/2383849624 conn(0x56121b041730 msgr2=0x56121b060490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:20.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.371+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 shutdown_connections 2023-10-05T19:40:20.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.371+0000 7f75746e7e40 1 -- 172.21.15.12:0/2383849624 wait complete. 2023-10-05T19:40:20.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.375+0000 7f75746e7e40 1 -- 172.21.15.12:0/3886885821 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f74b8030c50 msgr2=0x7f74b8033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.375+0000 7f75746e7e40 1 --2- 172.21.15.12:0/3886885821 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f74b8030c50 0x7f74b8033100 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x56121b08d590 tx=0x7f756c00a9b0 comp rx=0 tx=0).stop 2023-10-05T19:40:20.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.375+0000 7f75746e7e40 1 -- 172.21.15.12:0/3886885821 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56121b09ad80 msgr2=0x56121b0a08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.375+0000 7f75746e7e40 1 --2- 172.21.15.12:0/3886885821 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56121b09ad80 0x56121b0a08d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x56121b094e00 tx=0x7f755c034000 comp rx=0 tx=0).stop 2023-10-05T19:40:20.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.375+0000 7f75746e7e40 1 -- 172.21.15.12:0/3886885821 shutdown_connections 2023-10-05T19:40:20.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.375+0000 7f75746e7e40 1 --2- 172.21.15.12:0/3886885821 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f74b8030c50 0x7f74b8033100 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:20.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.375+0000 7f75746e7e40 1 --2- 172.21.15.12:0/3886885821 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56121b09ad80 0x56121b0a08d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:20.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.375+0000 7f75746e7e40 1 -- 172.21.15.12:0/3886885821 >> 172.21.15.12:0/3886885821 conn(0x56121b096140 msgr2=0x56121b0941e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:20.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.375+0000 7f75746e7e40 1 -- 172.21.15.12:0/3886885821 shutdown_connections 2023-10-05T19:40:20.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.375+0000 7f75746e7e40 1 -- 172.21.15.12:0/3886885821 wait complete. 2023-10-05T19:40:20.388 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:40:20.389 INFO:tasks.util.rgw:rgwadmin: client.0 : ['subuser', 'create', '--subuser', 'foo:foo1', '--access', 'full'] 2023-10-05T19:40:20.389 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'subuser', 'create', '--subuser', 'foo:foo1', '--access', 'full'] 2023-10-05T19:40:20.389 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph subuser create --subuser foo:foo1 --access full 2023-10-05T19:40:20.428 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:20.428 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:20.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.419+0000 7f1be6279e40 1 Processor -- start 2023-10-05T19:40:20.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.419+0000 7f1be6279e40 1 -- start start 2023-10-05T19:40:20.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.419+0000 7f1be6279e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x563ff86a8b50 0x563ff87359f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.419+0000 7f1be6279e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x563ff8681bd0 con 0x563ff8735f30 2023-10-05T19:40:20.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.419+0000 7f1be6279e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x563ff863eee0 con 0x563ff86a8b50 2023-10-05T19:40:20.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.423+0000 7f1be3dae700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x563ff86a8b50 0x563ff87359f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.423+0000 7f1be35ad700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x563ff8735f30 0x563ff8738340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:47304/0 (socket says 172.21.15.12:47304) 2023-10-05T19:40:20.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.423+0000 7f1be35ad700 1 -- 172.21.15.12:0/2483102713 learned_addr learned my addr 172.21.15.12:0/2483102713 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:20.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.423+0000 7f1be2dac700 1 -- 172.21.15.12:0/2483102713 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3192033552 0 0) 0x563ff8681bd0 con 0x563ff8735f30 2023-10-05T19:40:20.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.423+0000 7f1be2dac700 1 -- 172.21.15.12:0/2483102713 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1bc4001e10 con 0x563ff8735f30 2023-10-05T19:40:20.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.423+0000 7f1be3dae700 1 -- 172.21.15.12:0/2483102713 >> v1:172.21.15.8:6789/0 conn(0x563ff8735f30 legacy=0x563ff8738340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.423+0000 7f1be3dae700 1 -- 172.21.15.12:0/2483102713 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ff8739a60 con 0x563ff86a8b50 2023-10-05T19:40:20.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.423+0000 7f1be3dae700 1 --2- 172.21.15.12:0/2483102713 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ff86a8b50 0x563ff87359f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1bd400a5d0 tx=0x7f1bd4005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1493f9702b6ac122 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.423+0000 7f1be2dac700 1 -- 172.21.15.12:0/2483102713 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1bd4007610 con 0x563ff86a8b50 2023-10-05T19:40:20.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.423+0000 7f1be2dac700 1 -- 172.21.15.12:0/2483102713 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bd4007770 con 0x563ff86a8b50 2023-10-05T19:40:20.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.423+0000 7f1be2dac700 1 -- 172.21.15.12:0/2483102713 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1bd4007a40 con 0x563ff86a8b50 2023-10-05T19:40:20.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.423+0000 7f1be6279e40 1 -- 172.21.15.12:0/2483102713 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ff86a8b50 msgr2=0x563ff87359f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.423+0000 7f1be6279e40 1 --2- 172.21.15.12:0/2483102713 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ff86a8b50 0x563ff87359f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1bd400a5d0 tx=0x7f1bd4005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:20.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.423+0000 7f1be6279e40 1 -- 172.21.15.12:0/2483102713 shutdown_connections 2023-10-05T19:40:20.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.423+0000 7f1be6279e40 1 --2- 172.21.15.12:0/2483102713 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ff86a8b50 0x563ff87359f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:20.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.423+0000 7f1be6279e40 1 -- 172.21.15.12:0/2483102713 >> 172.21.15.12:0/2483102713 conn(0x563ff872f750 msgr2=0x563ff8731b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:20.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.423+0000 7f1be6279e40 1 -- 172.21.15.12:0/2483102713 shutdown_connections 2023-10-05T19:40:20.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.423+0000 7f1be6279e40 1 -- 172.21.15.12:0/2483102713 wait complete. 2023-10-05T19:40:20.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.423+0000 7f1be6279e40 1 Processor -- start 2023-10-05T19:40:20.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.423+0000 7f1be6279e40 1 -- start start 2023-10-05T19:40:20.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.423+0000 7f1be6279e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ff86a8b50 0x563ff875a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.423+0000 7f1be6279e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ff8602e30 con 0x563ff86a8b50 2023-10-05T19:40:20.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.427+0000 7f1be3dae700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ff86a8b50 0x563ff875a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.427+0000 7f1be3dae700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ff86a8b50 0x563ff875a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37088/0 (socket says 172.21.15.12:37088) 2023-10-05T19:40:20.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.427+0000 7f1be3dae700 1 -- 172.21.15.12:0/3328896042 learned_addr learned my addr 172.21.15.12:0/3328896042 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:20.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.427+0000 7f1be3dae700 1 -- 172.21.15.12:0/3328896042 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd4005cf0 con 0x563ff86a8b50 2023-10-05T19:40:20.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.427+0000 7f1be3dae700 1 --2- 172.21.15.12:0/3328896042 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ff86a8b50 0x563ff875a360 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4005cc0 tx=0x7f1bd4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb35b8fe7b357878 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.427+0000 7f1be1daa700 1 -- 172.21.15.12:0/3328896042 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1bd40078e0 con 0x563ff86a8b50 2023-10-05T19:40:20.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.427+0000 7f1be1daa700 1 -- 172.21.15.12:0/3328896042 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bd4007a40 con 0x563ff86a8b50 2023-10-05T19:40:20.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.427+0000 7f1be1daa700 1 -- 172.21.15.12:0/3328896042 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1bd4007bd0 con 0x563ff86a8b50 2023-10-05T19:40:20.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.427+0000 7f1be6279e40 1 -- 172.21.15.12:0/3328896042 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ff86a8b50 msgr2=0x563ff875a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.427+0000 7f1be6279e40 1 --2- 172.21.15.12:0/3328896042 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ff86a8b50 0x563ff875a360 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4005cc0 tx=0x7f1bd4014000 comp rx=0 tx=0).stop 2023-10-05T19:40:20.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.427+0000 7f1be6279e40 1 -- 172.21.15.12:0/3328896042 shutdown_connections 2023-10-05T19:40:20.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.427+0000 7f1be6279e40 1 --2- 172.21.15.12:0/3328896042 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ff86a8b50 0x563ff875a360 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:20.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.427+0000 7f1be6279e40 1 -- 172.21.15.12:0/3328896042 >> 172.21.15.12:0/3328896042 conn(0x563ff872f750 msgr2=0x563ff8755f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:20.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.427+0000 7f1be6279e40 1 -- 172.21.15.12:0/3328896042 shutdown_connections 2023-10-05T19:40:20.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.427+0000 7f1be6279e40 1 -- 172.21.15.12:0/3328896042 wait complete. 2023-10-05T19:40:20.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.427+0000 7f1be6279e40 1 Processor -- start 2023-10-05T19:40:20.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.427+0000 7f1be6279e40 1 -- start start 2023-10-05T19:40:20.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.427+0000 7f1be6279e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ff86a8b50 0x563ff8761020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.427+0000 7f1be6279e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ff85d5ed0 con 0x563ff86a8b50 2023-10-05T19:40:20.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.427+0000 7f1be3dae700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ff86a8b50 0x563ff8761020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.427+0000 7f1be3dae700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ff86a8b50 0x563ff8761020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37090/0 (socket says 172.21.15.12:37090) 2023-10-05T19:40:20.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.427+0000 7f1be3dae700 1 -- 172.21.15.12:0/1842477014 learned_addr learned my addr 172.21.15.12:0/1842477014 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:20.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.427+0000 7f1be3dae700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ff8739a60 con 0x563ff86a8b50 2023-10-05T19:40:20.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.427+0000 7f1be3dae700 1 --2- 172.21.15.12:0/1842477014 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ff86a8b50 0x563ff8761020 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1bd401f040 tx=0x7f1bd401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.431+0000 7f1bdbfff700 1 -- 172.21.15.12:0/1842477014 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1bd40078e0 con 0x563ff86a8b50 2023-10-05T19:40:20.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.431+0000 7f1bdbfff700 1 -- 172.21.15.12:0/1842477014 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bd4026650 con 0x563ff86a8b50 2023-10-05T19:40:20.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.431+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563ff8761560 con 0x563ff86a8b50 2023-10-05T19:40:20.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.431+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ff87619a0 con 0x563ff86a8b50 2023-10-05T19:40:20.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.431+0000 7f1bdbfff700 1 -- 172.21.15.12:0/1842477014 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1bd4026820 con 0x563ff86a8b50 2023-10-05T19:40:20.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.431+0000 7f1be6279e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.431+0000 7f1bdbfff700 1 -- 172.21.15.12:0/1842477014 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f1bd4026980 con 0x563ff86a8b50 2023-10-05T19:40:20.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.431+0000 7f1bdbfff700 1 --2- 172.21.15.12:0/1842477014 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1bc8030c90 0x7f1bc8033140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.431+0000 7f1bdbfff700 1 -- 172.21.15.12:0/1842477014 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f1bd4046910 con 0x563ff86a8b50 2023-10-05T19:40:20.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.431+0000 7f1be35ad700 1 --2- 172.21.15.12:0/1842477014 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1bc8030c90 0x7f1bc8033140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.431+0000 7f1be6279e40 1 --2- 172.21.15.12:0/1842477014 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x563ff8735f30 0x563ff876cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.431+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff876d150 con 0x563ff8735f30 2023-10-05T19:40:20.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.431+0000 7f1be45af700 1 --2- 172.21.15.12:0/1842477014 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x563ff8735f30 0x563ff876cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.431+0000 7f1be35ad700 1 --2- 172.21.15.12:0/1842477014 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1bc8030c90 0x7f1bc8033140 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc004170 tx=0x7f1bcc001400 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.435+0000 7f1be45af700 1 --2- 172.21.15.12:0/1842477014 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x563ff8735f30 0x563ff876cae0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.435+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x563ff8735f30 2023-10-05T19:40:20.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.435+0000 7f1be6279e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.435+0000 7f1be6279e40 20 realm 2023-10-05T19:40:20.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.435+0000 7f1be6279e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.435+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff876a140 con 0x563ff8735f30 2023-10-05T19:40:20.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.435+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x563ff8735f30 2023-10-05T19:40:20.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.435+0000 7f1be6279e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.435+0000 7f1be6279e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:20.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.435+0000 7f1be6279e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.435+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff876dc30 con 0x563ff8735f30 2023-10-05T19:40:20.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.435+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x563ff8735f30 2023-10-05T19:40:20.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.435+0000 7f1be6279e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.435+0000 7f1be6279e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.435+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff876d840 con 0x563ff8735f30 2023-10-05T19:40:20.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.435+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x563ff8735f30 2023-10-05T19:40:20.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.435+0000 7f1be6279e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:20.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.435+0000 7f1be6279e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.435+0000 7f1be6279e40 1 --2- 172.21.15.12:0/1842477014 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x563ff876efd0 0x563ff87713b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.435+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87718f0 con 0x563ff876efd0 2023-10-05T19:40:20.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.435+0000 7f1be3dae700 1 --2- 172.21.15.12:0/1842477014 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x563ff876efd0 0x563ff87713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.435+0000 7f1be3dae700 1 --2- 172.21.15.12:0/1842477014 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x563ff876efd0 0x563ff87713b0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.435+0000 7f1be3dae700 1 -- 172.21.15.12:0/1842477014 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f1bd40129c0 con 0x563ff876efd0 2023-10-05T19:40:20.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.435+0000 7f1be6279e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:20.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.435+0000 7f1be6279e40 20 searching for the correct realm 2023-10-05T19:40:20.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.435+0000 7f1be6279e40 1 --2- 172.21.15.12:0/1842477014 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x563ff87736e0 0x563ff8775af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.439+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87761a0 con 0x563ff87736e0 2023-10-05T19:40:20.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.439+0000 7f1be35ad700 1 --2- 172.21.15.12:0/1842477014 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x563ff87736e0 0x563ff8775af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.439+0000 7f1be35ad700 1 --2- 172.21.15.12:0/1842477014 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x563ff87736e0 0x563ff8775af0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.439+0000 7f1be35ad700 1 -- 172.21.15.12:0/1842477014 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f1bcc006070 con 0x563ff87736e0 2023-10-05T19:40:20.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.439+0000 7f1be35ad700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87761a0 con 0x563ff876efd0 2023-10-05T19:40:20.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.439+0000 7f1be3dae700 1 -- 172.21.15.12:0/1842477014 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bd40129c0 con 0x563ff876efd0 2023-10-05T19:40:20.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.439+0000 7f1be3dae700 1 --2- 172.21.15.12:0/1842477014 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f1bd4007030 0x7f1bd40115e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.439+0000 7f1be3dae700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87761a0 con 0x7f1bd4007030 2023-10-05T19:40:20.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.443+0000 7f1be45af700 1 --2- 172.21.15.12:0/1842477014 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f1bd4007030 0x7f1bd40115e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.443+0000 7f1be45af700 1 --2- 172.21.15.12:0/1842477014 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f1bd4007030 0x7f1bd40115e0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.443+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x7f1bd4007030 2023-10-05T19:40:20.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.443+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87761a0 con 0x563ff8735f30 2023-10-05T19:40:20.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.443+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x563ff8735f30 2023-10-05T19:40:20.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.443+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87761a0 con 0x563ff8735f30 2023-10-05T19:40:20.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.443+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x563ff8735f30 2023-10-05T19:40:20.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.443+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87761a0 con 0x7f1bd4007030 2023-10-05T19:40:20.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.447+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x7f1bd4007030 2023-10-05T19:40:20.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.447+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87761a0 con 0x7f1bd4007030 2023-10-05T19:40:20.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.447+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x7f1bd4007030 2023-10-05T19:40:20.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.447+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87761a0 con 0x563ff876efd0 2023-10-05T19:40:20.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.447+0000 7f1be3dae700 1 -- 172.21.15.12:0/1842477014 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bd4040040 con 0x563ff876efd0 2023-10-05T19:40:20.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.447+0000 7f1be3dae700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87761a0 con 0x563ff87736e0 2023-10-05T19:40:20.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.447+0000 7f1be35ad700 1 -- 172.21.15.12:0/1842477014 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bcc006070 con 0x563ff87736e0 2023-10-05T19:40:20.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.447+0000 7f1be35ad700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87761a0 con 0x563ff8735f30 2023-10-05T19:40:20.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.447+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x563ff8735f30 2023-10-05T19:40:20.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.447+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87761a0 con 0x7f1bd4007030 2023-10-05T19:40:20.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.451+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x7f1bd4007030 2023-10-05T19:40:20.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.451+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87761a0 con 0x563ff8735f30 2023-10-05T19:40:20.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.451+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x563ff8735f30 2023-10-05T19:40:20.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.451+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87761a0 con 0x563ff876efd0 2023-10-05T19:40:20.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.451+0000 7f1be3dae700 1 -- 172.21.15.12:0/1842477014 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bd4040040 con 0x563ff876efd0 2023-10-05T19:40:20.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.451+0000 7f1be3dae700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87761a0 con 0x563ff87736e0 2023-10-05T19:40:20.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.451+0000 7f1be35ad700 1 -- 172.21.15.12:0/1842477014 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bcc006070 con 0x563ff87736e0 2023-10-05T19:40:20.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.451+0000 7f1be35ad700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87761a0 con 0x563ff87736e0 2023-10-05T19:40:20.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.451+0000 7f1be35ad700 1 -- 172.21.15.12:0/1842477014 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bcc006070 con 0x563ff87736e0 2023-10-05T19:40:20.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.451+0000 7f1be35ad700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87761a0 con 0x7f1bd4007030 2023-10-05T19:40:20.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.451+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x7f1bd4007030 2023-10-05T19:40:20.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.451+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87761a0 con 0x563ff87736e0 2023-10-05T19:40:20.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.451+0000 7f1be35ad700 1 -- 172.21.15.12:0/1842477014 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bcc006070 con 0x563ff87736e0 2023-10-05T19:40:20.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.451+0000 7f1be35ad700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87761a0 con 0x563ff87736e0 2023-10-05T19:40:20.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.455+0000 7f1be35ad700 1 -- 172.21.15.12:0/1842477014 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bcc006070 con 0x563ff87736e0 2023-10-05T19:40:20.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.455+0000 7f1be35ad700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87761a0 con 0x563ff8735f30 2023-10-05T19:40:20.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.455+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x563ff8735f30 2023-10-05T19:40:20.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.455+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87761a0 con 0x563ff8735f30 2023-10-05T19:40:20.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.455+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x563ff8735f30 2023-10-05T19:40:20.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.455+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87761a0 con 0x7f1bd4007030 2023-10-05T19:40:20.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.455+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x7f1bd4007030 2023-10-05T19:40:20.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.455+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87761a0 con 0x563ff87736e0 2023-10-05T19:40:20.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.455+0000 7f1be35ad700 1 -- 172.21.15.12:0/1842477014 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bcc006070 con 0x563ff87736e0 2023-10-05T19:40:20.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.455+0000 7f1be35ad700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87761a0 con 0x563ff876efd0 2023-10-05T19:40:20.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.459+0000 7f1be3dae700 1 -- 172.21.15.12:0/1842477014 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bd4040040 con 0x563ff876efd0 2023-10-05T19:40:20.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.459+0000 7f1be3dae700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87761a0 con 0x563ff8735f30 2023-10-05T19:40:20.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.459+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x563ff8735f30 2023-10-05T19:40:20.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.459+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87761a0 con 0x7f1bd4007030 2023-10-05T19:40:20.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.459+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x7f1bd4007030 2023-10-05T19:40:20.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.459+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87761a0 con 0x563ff87736e0 2023-10-05T19:40:20.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.459+0000 7f1be35ad700 1 -- 172.21.15.12:0/1842477014 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bcc006070 con 0x563ff87736e0 2023-10-05T19:40:20.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.459+0000 7f1be35ad700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87761a0 con 0x7f1bd4007030 2023-10-05T19:40:20.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.459+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x7f1bd4007030 2023-10-05T19:40:20.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.459+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87761a0 con 0x563ff876efd0 2023-10-05T19:40:20.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.463+0000 7f1be3dae700 1 -- 172.21.15.12:0/1842477014 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bd4040040 con 0x563ff876efd0 2023-10-05T19:40:20.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.463+0000 7f1be3dae700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87761a0 con 0x563ff87736e0 2023-10-05T19:40:20.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.463+0000 7f1be35ad700 1 -- 172.21.15.12:0/1842477014 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bcc006070 con 0x563ff87736e0 2023-10-05T19:40:20.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.463+0000 7f1be35ad700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87761a0 con 0x7f1bd4007030 2023-10-05T19:40:20.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.463+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x7f1bd4007030 2023-10-05T19:40:20.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.463+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87761a0 con 0x563ff8735f30 2023-10-05T19:40:20.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.463+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x563ff8735f30 2023-10-05T19:40:20.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.463+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87761a0 con 0x563ff876efd0 2023-10-05T19:40:20.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.463+0000 7f1be3dae700 1 -- 172.21.15.12:0/1842477014 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f1bd4040040 con 0x563ff876efd0 2023-10-05T19:40:20.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.463+0000 7f1be6279e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:20.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.463+0000 7f1be6279e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:20.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.463+0000 7f1be6279e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:20.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.463+0000 7f1be6279e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:20.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.463+0000 7f1be6279e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.463+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff8772460 con 0x563ff8735f30 2023-10-05T19:40:20.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.467+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x563ff8735f30 2023-10-05T19:40:20.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.467+0000 7f1be6279e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.467+0000 7f1be6279e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.467+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff8772090 con 0x563ff8735f30 2023-10-05T19:40:20.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.467+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x563ff8735f30 2023-10-05T19:40:20.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.467+0000 7f1be6279e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:20.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.467+0000 7f1be6279e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.467+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff8776610 con 0x563ff87736e0 2023-10-05T19:40:20.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.467+0000 7f1be35ad700 1 -- 172.21.15.12:0/1842477014 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f1bcc006070 con 0x563ff87736e0 2023-10-05T19:40:20.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.467+0000 7f1be6279e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:20.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.467+0000 7f1be6279e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.467+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87769c0 con 0x563ff87736e0 2023-10-05T19:40:20.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.467+0000 7f1be35ad700 1 -- 172.21.15.12:0/1842477014 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f1bcc006070 con 0x563ff87736e0 2023-10-05T19:40:20.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.467+0000 7f1be6279e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.467+0000 7f1be6279e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.467+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff8776d70 con 0x563ff8735f30 2023-10-05T19:40:20.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.467+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x563ff8735f30 2023-10-05T19:40:20.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.467+0000 7f1be6279e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.467+0000 7f1be6279e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.467+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87728f0 con 0x563ff8735f30 2023-10-05T19:40:20.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.467+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x563ff8735f30 2023-10-05T19:40:20.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.467+0000 7f1be6279e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:20.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.467+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87728c0 con 0x563ff87736e0 2023-10-05T19:40:20.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.471+0000 7f1be35ad700 1 -- 172.21.15.12:0/1842477014 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f1bcc006070 con 0x563ff87736e0 2023-10-05T19:40:20.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.471+0000 7f1be35ad700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87728c0 con 0x563ff876efd0 2023-10-05T19:40:20.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.471+0000 7f1be3dae700 1 -- 172.21.15.12:0/1842477014 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bd4040040 con 0x563ff876efd0 2023-10-05T19:40:20.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.471+0000 7f1be3dae700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87728c0 con 0x7f1bd4007030 2023-10-05T19:40:20.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.471+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x7f1bd4007030 2023-10-05T19:40:20.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.475+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff8776d70 con 0x563ff8735f30 2023-10-05T19:40:20.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.475+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x563ff8735f30 2023-10-05T19:40:20.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.475+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff8776d70 con 0x563ff8735f30 2023-10-05T19:40:20.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.475+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x563ff8735f30 2023-10-05T19:40:20.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.475+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff8776d70 con 0x7f1bd4007030 2023-10-05T19:40:20.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.475+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x7f1bd4007030 2023-10-05T19:40:20.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.475+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff8776d70 con 0x7f1bd4007030 2023-10-05T19:40:20.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.475+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x7f1bd4007030 2023-10-05T19:40:20.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.475+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff8776d70 con 0x563ff876efd0 2023-10-05T19:40:20.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.479+0000 7f1be3dae700 1 -- 172.21.15.12:0/1842477014 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bd4040040 con 0x563ff876efd0 2023-10-05T19:40:20.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.479+0000 7f1be3dae700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff8776d70 con 0x563ff87736e0 2023-10-05T19:40:20.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.479+0000 7f1be35ad700 1 -- 172.21.15.12:0/1842477014 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bcc006070 con 0x563ff87736e0 2023-10-05T19:40:20.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.479+0000 7f1be35ad700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff8776d70 con 0x563ff8735f30 2023-10-05T19:40:20.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.479+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x563ff8735f30 2023-10-05T19:40:20.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.479+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff8776d70 con 0x7f1bd4007030 2023-10-05T19:40:20.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.479+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x7f1bd4007030 2023-10-05T19:40:20.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.479+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff8776d70 con 0x563ff8735f30 2023-10-05T19:40:20.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.479+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x563ff8735f30 2023-10-05T19:40:20.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.479+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff8776d70 con 0x563ff876efd0 2023-10-05T19:40:20.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.483+0000 7f1be3dae700 1 -- 172.21.15.12:0/1842477014 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bd4040040 con 0x563ff876efd0 2023-10-05T19:40:20.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.483+0000 7f1be3dae700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff8776d70 con 0x563ff87736e0 2023-10-05T19:40:20.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.483+0000 7f1be35ad700 1 -- 172.21.15.12:0/1842477014 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bcc006070 con 0x563ff87736e0 2023-10-05T19:40:20.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.483+0000 7f1be35ad700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff8776d70 con 0x563ff87736e0 2023-10-05T19:40:20.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.483+0000 7f1be35ad700 1 -- 172.21.15.12:0/1842477014 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bcc006070 con 0x563ff87736e0 2023-10-05T19:40:20.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.483+0000 7f1be35ad700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff8776d70 con 0x7f1bd4007030 2023-10-05T19:40:20.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.483+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x7f1bd4007030 2023-10-05T19:40:20.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.483+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff8776d70 con 0x563ff87736e0 2023-10-05T19:40:20.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.483+0000 7f1be35ad700 1 -- 172.21.15.12:0/1842477014 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bcc006070 con 0x563ff87736e0 2023-10-05T19:40:20.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.483+0000 7f1be35ad700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff8776d70 con 0x563ff87736e0 2023-10-05T19:40:20.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.487+0000 7f1be35ad700 1 -- 172.21.15.12:0/1842477014 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bcc006070 con 0x563ff87736e0 2023-10-05T19:40:20.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.487+0000 7f1be35ad700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff8776d70 con 0x563ff8735f30 2023-10-05T19:40:20.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.487+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x563ff8735f30 2023-10-05T19:40:20.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.487+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff8776d70 con 0x563ff8735f30 2023-10-05T19:40:20.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.487+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x563ff8735f30 2023-10-05T19:40:20.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.487+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff8776d70 con 0x7f1bd4007030 2023-10-05T19:40:20.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.487+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x7f1bd4007030 2023-10-05T19:40:20.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.487+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff8776d70 con 0x563ff87736e0 2023-10-05T19:40:20.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.487+0000 7f1be35ad700 1 -- 172.21.15.12:0/1842477014 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bcc006070 con 0x563ff87736e0 2023-10-05T19:40:20.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.487+0000 7f1be35ad700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff8776d70 con 0x563ff876efd0 2023-10-05T19:40:20.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.487+0000 7f1be3dae700 1 -- 172.21.15.12:0/1842477014 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bd4040040 con 0x563ff876efd0 2023-10-05T19:40:20.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.487+0000 7f1be3dae700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff8776d70 con 0x563ff8735f30 2023-10-05T19:40:20.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.491+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x563ff8735f30 2023-10-05T19:40:20.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.491+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff8776d70 con 0x7f1bd4007030 2023-10-05T19:40:20.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.491+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x7f1bd4007030 2023-10-05T19:40:20.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.491+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff8776d70 con 0x563ff87736e0 2023-10-05T19:40:20.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.491+0000 7f1be35ad700 1 -- 172.21.15.12:0/1842477014 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bcc006070 con 0x563ff87736e0 2023-10-05T19:40:20.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.491+0000 7f1be35ad700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff8776d70 con 0x7f1bd4007030 2023-10-05T19:40:20.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.491+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x7f1bd4007030 2023-10-05T19:40:20.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.491+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff8776d70 con 0x563ff876efd0 2023-10-05T19:40:20.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.495+0000 7f1be3dae700 1 -- 172.21.15.12:0/1842477014 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bd4040040 con 0x563ff876efd0 2023-10-05T19:40:20.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.495+0000 7f1be3dae700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff8776d70 con 0x563ff87736e0 2023-10-05T19:40:20.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.495+0000 7f1be35ad700 1 -- 172.21.15.12:0/1842477014 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bcc006070 con 0x563ff87736e0 2023-10-05T19:40:20.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.495+0000 7f1be35ad700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff8776d70 con 0x7f1bd4007030 2023-10-05T19:40:20.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.495+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x7f1bd4007030 2023-10-05T19:40:20.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.495+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff8776d70 con 0x563ff8735f30 2023-10-05T19:40:20.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.495+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x563ff8735f30 2023-10-05T19:40:20.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.495+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff8776d70 con 0x563ff876efd0 2023-10-05T19:40:20.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.499+0000 7f1be3dae700 1 -- 172.21.15.12:0/1842477014 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f1bd4040040 con 0x563ff876efd0 2023-10-05T19:40:20.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.499+0000 7f1be6279e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:20.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.499+0000 7f1be6279e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:20.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.499+0000 7f1be6279e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:20.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.499+0000 7f1be6279e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:20.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.499+0000 7f1be6279e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.499+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87728c0 con 0x563ff8735f30 2023-10-05T19:40:20.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.499+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x563ff8735f30 2023-10-05T19:40:20.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.499+0000 7f1be6279e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:20.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.499+0000 7f1be6279e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.499+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff8777660 con 0x563ff876efd0 2023-10-05T19:40:20.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.499+0000 7f1be3dae700 1 -- 172.21.15.12:0/1842477014 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f1bd4040040 con 0x563ff876efd0 2023-10-05T19:40:20.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.499+0000 7f1be6279e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:20.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.499+0000 7f1be6279e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.499+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff8777f50 con 0x563ff8735f30 2023-10-05T19:40:20.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.499+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x563ff8735f30 2023-10-05T19:40:20.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.499+0000 7f1be6279e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:20.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.499+0000 7f1be6279e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.499+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff8777f50 con 0x563ff87736e0 2023-10-05T19:40:20.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.499+0000 7f1be35ad700 1 -- 172.21.15.12:0/1842477014 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f1bcc006070 con 0x563ff87736e0 2023-10-05T19:40:20.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.499+0000 7f1be6279e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:20.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.499+0000 7f1be6279e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.499+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87786c0 con 0x7f1bd4007030 2023-10-05T19:40:20.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.503+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x7f1bd4007030 2023-10-05T19:40:20.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.503+0000 7f1be6279e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.503+0000 7f1be6279e40 20 zone default found 2023-10-05T19:40:20.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.503+0000 7f1be6279e40 4 Realm: () 2023-10-05T19:40:20.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.503+0000 7f1be6279e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:20.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.503+0000 7f1be6279e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:20.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.503+0000 7f1be6279e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:20.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.503+0000 7f1be6279e40 20 zonegroup default 2023-10-05T19:40:20.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.503+0000 7f1be6279e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.503+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff8778670 con 0x7f1bd4007030 2023-10-05T19:40:20.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.503+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x7f1bd4007030 2023-10-05T19:40:20.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.503+0000 7f1be6279e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.503+0000 7f1be6279e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.503+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff8779ff0 con 0x563ff87736e0 2023-10-05T19:40:20.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.503+0000 7f1be35ad700 1 -- 172.21.15.12:0/1842477014 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f1bcc006070 con 0x563ff87736e0 2023-10-05T19:40:20.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.503+0000 7f1be6279e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.503+0000 7f1be6279e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.503+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff877a3a0 con 0x563ff87736e0 2023-10-05T19:40:20.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.503+0000 7f1be35ad700 1 -- 172.21.15.12:0/1842477014 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f1bcc006070 con 0x563ff87736e0 2023-10-05T19:40:20.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.503+0000 7f1be6279e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.503+0000 7f1be6279e40 20 started sync module instance, tier type = 2023-10-05T19:40:20.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.503+0000 7f1be6279e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:20.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.503+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87796b0 con 0x563ff876efd0 2023-10-05T19:40:20.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.507+0000 7f1be3dae700 1 -- 172.21.15.12:0/1842477014 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'54 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1bd4040040 con 0x563ff876efd0 2023-10-05T19:40:20.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.507+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94832751508064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff877a750 con 0x563ff876efd0 2023-10-05T19:40:20.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.507+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff877aff0 con 0x563ff8735f30 2023-10-05T19:40:20.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.511+0000 7f1be3dae700 1 -- 172.21.15.12:0/1842477014 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94832751508064] v28'55 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1bd4040040 con 0x563ff876efd0 2023-10-05T19:40:20.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.511+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'54 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x563ff8735f30 2023-10-05T19:40:20.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.511+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94832751520512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff8779bd0 con 0x563ff8735f30 2023-10-05T19:40:20.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.511+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff877bb00 con 0x563ff87736e0 2023-10-05T19:40:20.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.515+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94832751520512] v28'55 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x563ff8735f30 2023-10-05T19:40:20.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.515+0000 7f1be35ad700 1 -- 172.21.15.12:0/1842477014 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'54 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1bcc006070 con 0x563ff87736e0 2023-10-05T19:40:20.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.515+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94832751523344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff877beb0 con 0x563ff87736e0 2023-10-05T19:40:20.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.515+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff877c8c0 con 0x563ff876efd0 2023-10-05T19:40:20.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.515+0000 7f1be3dae700 1 -- 172.21.15.12:0/1842477014 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'54 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1bd4040040 con 0x563ff876efd0 2023-10-05T19:40:20.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.519+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94832751529072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff877c4f0 con 0x563ff876efd0 2023-10-05T19:40:20.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.519+0000 7f1be35ad700 1 -- 172.21.15.12:0/1842477014 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94832751523344] v28'55 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1bcc006070 con 0x563ff87736e0 2023-10-05T19:40:20.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.519+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff877ddc0 con 0x563ff8735f30 2023-10-05T19:40:20.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.519+0000 7f1be3dae700 1 -- 172.21.15.12:0/1842477014 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94832751529072] v28'55 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1bd4040040 con 0x563ff876efd0 2023-10-05T19:40:20.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.519+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'54 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x563ff8735f30 2023-10-05T19:40:20.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.519+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94832751521616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff877c4f0 con 0x563ff8735f30 2023-10-05T19:40:20.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.519+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff877da00 con 0x7f1bd4007030 2023-10-05T19:40:20.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.523+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94832751521616] v28'55 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x563ff8735f30 2023-10-05T19:40:20.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.523+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'54 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x7f1bd4007030 2023-10-05T19:40:20.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.523+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94832751530176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff877e630 con 0x7f1bd4007030 2023-10-05T19:40:20.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.523+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff877f040 con 0x7f1bd4007030 2023-10-05T19:40:20.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.527+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(96 notify.6 [create] v28'54 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x7f1bd4007030 2023-10-05T19:40:20.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.527+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94832751536976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff877f490 con 0x7f1bd4007030 2023-10-05T19:40:20.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.527+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff877ff70 con 0x7f1bd4007030 2023-10-05T19:40:20.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.527+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(95 notify.5 [watch watch cookie 94832751530176] v28'55 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x7f1bd4007030 2023-10-05T19:40:20.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.527+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(98 notify.7 [create] v28'54 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x7f1bd4007030 2023-10-05T19:40:20.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.527+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94832751534320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff877ff30 con 0x7f1bd4007030 2023-10-05T19:40:20.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.527+0000 7f1be6279e40 20 add_watcher() i=0 2023-10-05T19:40:20.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.527+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(97 notify.6 [watch watch cookie 94832751536976] v28'55 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x7f1bd4007030 2023-10-05T19:40:20.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.527+0000 7f1be6279e40 20 add_watcher() i=1 2023-10-05T19:40:20.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.527+0000 7f1be6279e40 20 add_watcher() i=2 2023-10-05T19:40:20.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.527+0000 7f1be6279e40 20 add_watcher() i=3 2023-10-05T19:40:20.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.527+0000 7f1be6279e40 20 add_watcher() i=4 2023-10-05T19:40:20.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.527+0000 7f1be6279e40 20 add_watcher() i=5 2023-10-05T19:40:20.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.527+0000 7f1be6279e40 20 add_watcher() i=6 2023-10-05T19:40:20.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.531+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94832751534320] v28'55 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x7f1bd4007030 2023-10-05T19:40:20.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.531+0000 7f1be6279e40 20 add_watcher() i=7 2023-10-05T19:40:20.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.531+0000 7f1be6279e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:20.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.531+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff8782f60 con 0x563ff8735f30 2023-10-05T19:40:20.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.531+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x563ff8735f30 2023-10-05T19:40:20.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.531+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94832751540800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff8780380 con 0x563ff8735f30 2023-10-05T19:40:20.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.535+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94832751540800] v28'70 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x563ff8735f30 2023-10-05T19:40:20.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.535+0000 7f1b2ffff700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:20.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.535+0000 7f1be6279e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:20.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.535+0000 7f1be6279e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:20.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.535+0000 7f1be6279e40 1 Processor -- start 2023-10-05T19:40:20.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.535+0000 7f1be6279e40 1 -- start start 2023-10-05T19:40:20.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.535+0000 7f1be6279e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ff878d3d0 0x563ff878d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.535+0000 7f1be6279e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ff878dce0 con 0x563ff878d3d0 2023-10-05T19:40:20.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.535+0000 7f1be35ad700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ff878d3d0 0x563ff878d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.535+0000 7f1be35ad700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ff878d3d0 0x563ff878d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37100/0 (socket says 172.21.15.12:37100) 2023-10-05T19:40:20.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.535+0000 7f1be35ad700 1 -- 172.21.15.12:0/621056785 learned_addr learned my addr 172.21.15.12:0/621056785 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:20.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.539+0000 7f1be35ad700 1 -- 172.21.15.12:0/621056785 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ff878de20 con 0x563ff878d3d0 2023-10-05T19:40:20.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.539+0000 7f1be35ad700 1 --2- 172.21.15.12:0/621056785 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ff878d3d0 0x563ff878d7a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1bd401cf90 tx=0x7f1bcc01e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b560daa0fdfa77c server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.539+0000 7f1b2f7fe700 1 -- 172.21.15.12:0/621056785 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1bcc013040 con 0x563ff878d3d0 2023-10-05T19:40:20.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.539+0000 7f1b2f7fe700 1 -- 172.21.15.12:0/621056785 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bcc022040 con 0x563ff878d3d0 2023-10-05T19:40:20.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.539+0000 7f1b2f7fe700 1 -- 172.21.15.12:0/621056785 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1bcc03b8e0 con 0x563ff878d3d0 2023-10-05T19:40:20.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.539+0000 7f1be6279e40 1 -- 172.21.15.12:0/621056785 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ff878d3d0 msgr2=0x563ff878d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.539+0000 7f1be6279e40 1 --2- 172.21.15.12:0/621056785 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ff878d3d0 0x563ff878d7a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1bd401cf90 tx=0x7f1bcc01e040 comp rx=0 tx=0).stop 2023-10-05T19:40:20.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.539+0000 7f1be6279e40 1 -- 172.21.15.12:0/621056785 shutdown_connections 2023-10-05T19:40:20.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.539+0000 7f1be6279e40 1 --2- 172.21.15.12:0/621056785 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ff878d3d0 0x563ff878d7a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:20.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.539+0000 7f1be6279e40 1 -- 172.21.15.12:0/621056785 >> 172.21.15.12:0/621056785 conn(0x563ff8788790 msgr2=0x563ff87868e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:20.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.539+0000 7f1be6279e40 1 -- 172.21.15.12:0/621056785 shutdown_connections 2023-10-05T19:40:20.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.539+0000 7f1be6279e40 1 -- 172.21.15.12:0/621056785 wait complete. 2023-10-05T19:40:20.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.539+0000 7f1be6279e40 1 Processor -- start 2023-10-05T19:40:20.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.539+0000 7f1be6279e40 1 -- start start 2023-10-05T19:40:20.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.543+0000 7f1be6279e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ff878d3d0 0x563ff8793030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.543+0000 7f1be6279e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ff8794030 con 0x563ff878d3d0 2023-10-05T19:40:20.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.543+0000 7f1be35ad700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ff878d3d0 0x563ff8793030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.543+0000 7f1be35ad700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ff878d3d0 0x563ff8793030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37116/0 (socket says 172.21.15.12:37116) 2023-10-05T19:40:20.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.543+0000 7f1be35ad700 1 -- 172.21.15.12:0/806876521 learned_addr learned my addr 172.21.15.12:0/806876521 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:20.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.543+0000 7f1be35ad700 1 -- 172.21.15.12:0/806876521 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bcc034040 con 0x563ff878d3d0 2023-10-05T19:40:20.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.543+0000 7f1be35ad700 1 --2- 172.21.15.12:0/806876521 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ff878d3d0 0x563ff8793030 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x563ff8787450 tx=0x7f1bcc03d030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.543+0000 7f1b2dffb700 1 -- 172.21.15.12:0/806876521 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1bcc02c640 con 0x563ff878d3d0 2023-10-05T19:40:20.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.543+0000 7f1b2dffb700 1 -- 172.21.15.12:0/806876521 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bcc02c7a0 con 0x563ff878d3d0 2023-10-05T19:40:20.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.543+0000 7f1b2dffb700 1 -- 172.21.15.12:0/806876521 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1bcc02c910 con 0x563ff878d3d0 2023-10-05T19:40:20.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.543+0000 7f1be6279e40 1 -- 172.21.15.12:0/806876521 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563ff8793630 con 0x563ff878d3d0 2023-10-05T19:40:20.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.543+0000 7f1be6279e40 1 -- 172.21.15.12:0/806876521 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ff8793a10 con 0x563ff878d3d0 2023-10-05T19:40:20.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.543+0000 7f1b2dffb700 1 -- 172.21.15.12:0/806876521 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f1bcc02ca70 con 0x563ff878d3d0 2023-10-05T19:40:20.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.547+0000 7f1b2dffb700 1 --2- 172.21.15.12:0/806876521 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1b20030ca0 0x7f1b20033150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.547+0000 7f1b2dffb700 1 -- 172.21.15.12:0/806876521 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f1bcc059ab0 con 0x563ff878d3d0 2023-10-05T19:40:20.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.547+0000 7f1be45af700 1 --2- 172.21.15.12:0/806876521 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1b20030ca0 0x7f1b20033150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.547+0000 7f1be6279e40 5 note: GC not initialized 2023-10-05T19:40:20.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.547+0000 7f1be45af700 1 --2- 172.21.15.12:0/806876521 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1b20030ca0 0x7f1b20033150 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x563ff877f170 tx=0x7f1bdc00af00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.547+0000 7f1b1effd700 20 reqs_thread_entry: start 2023-10-05T19:40:20.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.547+0000 7f1b1cff9700 20 reqs_thread_entry: start 2023-10-05T19:40:20.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.547+0000 7f1be6279e40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:20.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.547+0000 7f1be6279e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:20.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.547+0000 7f1b16ffd700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:50 2023 2023-10-05T19:40:20.558 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:20.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.551+0000 7f1be6279e40 20 get_system_obj_state: rctx=0x7fff3d932058 obj=default.rgw.meta:users.uid:foo state=0x563ff860e0b0 s->prefetch_data=0 2023-10-05T19:40:20.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.547+0000 7f1b167fc700 20 reqs_thread_entry: start 2023-10-05T19:40:20.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.551+0000 7f1be6279e40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2023-10-05T19:40:20.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.551+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87fb960 con 0x563ff8735f30 2023-10-05T19:40:20.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.551+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(102 foo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x563ff8735f30 2023-10-05T19:40:20.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.551+0000 7f1be6279e40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x16 2023-10-05T19:40:20.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.551+0000 7f1be6279e40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:20.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.551+0000 7f1be6279e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:20.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.551+0000 7f1be6279e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:20.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.551+0000 7f1be6279e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:20.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.551+0000 7f1be6279e40 10 cache get: name=default.rgw.meta+users.uid+foo : type miss (requested=0x13, cached=0x16) 2023-10-05T19:40:20.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.551+0000 7f1be6279e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.551+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87fe3f0 con 0x563ff8735f30 2023-10-05T19:40:20.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.551+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(103 foo [call,call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv5 ondisk = 0) v8 ==== 273+0+480 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x563ff8735f30 2023-10-05T19:40:20.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.551+0000 7f1be6279e40 20 rados_obj.operate() r=0 bl.length=357 2023-10-05T19:40:20.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.551+0000 7f1be6279e40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2023-10-05T19:40:20.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.551+0000 7f1be6279e40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:20.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.551+0000 7f1be6279e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:20.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.551+0000 7f1be6279e40 20 get_system_obj_state: rctx=0x7fff3d9318a8 obj=default.rgw.meta:users.swift:foo:foo1 state=0x563ff860e0b0 s->prefetch_data=0 2023-10-05T19:40:20.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.551+0000 7f1be6279e40 10 cache get: name=default.rgw.meta+users.swift+foo:foo1 : miss 2023-10-05T19:40:20.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.551+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:104 8.8 8:17d68d04:users.swift::foo%3afoo1:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87fc3d0 con 0x563ff8735f30 2023-10-05T19:40:20.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.551+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(104 foo:foo1 [getxattrs,stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 194+0+0 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x563ff8735f30 2023-10-05T19:40:20.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.551+0000 7f1be6279e40 10 cache put: name=default.rgw.meta+users.swift+foo:foo1 info.flags=0x0 2023-10-05T19:40:20.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.551+0000 7f1be6279e40 10 adding default.rgw.meta+users.swift+foo:foo1 to cache LRU end 2023-10-05T19:40:20.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.551+0000 7f1be6279e40 20 get_system_obj_state: rctx=0x7fff3d931f48 obj=default.rgw.meta:users.swift:foo:foo1 state=0x563ff860e0b0 s->prefetch_data=0 2023-10-05T19:40:20.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.551+0000 7f1be6279e40 10 cache get: name=default.rgw.meta+users.swift+foo:foo1 : hit (negative entry) 2023-10-05T19:40:20.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.551+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:105 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.set in=58b,writefull 0~513 in=513b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87ff670 con 0x563ff8735f30 2023-10-05T19:40:20.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.555+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(105 foo [delete,create,call,writefull 0~513] v28'6 uv6 ondisk = 0) v8 ==== 273+0+0 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x563ff8735f30 2023-10-05T19:40:20.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.555+0000 7f1be6279e40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2023-10-05T19:40:20.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.555+0000 7f1be6279e40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:20.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.555+0000 7f1be6279e40 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2023-10-05T19:40:20.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.555+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:106 7.d 7:b4812045:::notify.4:head [notify cookie 94832752063168 in=701b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87fe430 con 0x563ff8735f30 2023-10-05T19:40:20.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.555+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== watch-notify(notify (1) cookie 94832751521616 notify 120259084293 ret 0) v3 ==== 731+0+0 (crc 0 0 0) 0x7f1bdc02d040 con 0x563ff8735f30 2023-10-05T19:40:20.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.555+0000 7f1be1daa700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084293 cookie 94832751521616 notifier 4333 bl.length()=689 2023-10-05T19:40:20.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.555+0000 7f1be1daa700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2023-10-05T19:40:20.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.555+0000 7f1be1daa700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:20.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.555+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== osd_op_reply(106 notify.4 [notify cookie 94832752063168 out=8b] v0'0 uv54 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x563ff8735f30 2023-10-05T19:40:20.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.555+0000 7f1be1daa700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:107 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f1bbc003d90 con 0x563ff8735f30 2023-10-05T19:40:20.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.555+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.1 v2:172.21.15.12:6808/29933 39 ==== watch-notify(notify_complete (2) cookie 94832752063168 notify 120259084293 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f1bdc037e50 con 0x563ff8735f30 2023-10-05T19:40:20.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.555+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.1 v2:172.21.15.12:6808/29933 40 ==== osd_op_reply(107 notify.4 [notify-ack] v0'0 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x563ff8735f30 2023-10-05T19:40:20.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.559+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:108 8.8 8:17d68d04:users.swift::foo%3afoo1:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87ff2c0 con 0x563ff8735f30 2023-10-05T19:40:20.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.559+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.1 v2:172.21.15.12:6808/29933 41 ==== osd_op_reply(108 foo:foo1 [delete,create,writefull 0~7] v28'1 uv1 ondisk = 0) v8 ==== 236+0+0 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x563ff8735f30 2023-10-05T19:40:20.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.559+0000 7f1be6279e40 10 cache put: name=default.rgw.meta+users.swift+foo:foo1 info.flags=0x7 2023-10-05T19:40:20.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.559+0000 7f1be6279e40 10 moving default.rgw.meta+users.swift+foo:foo1 to cache LRU end 2023-10-05T19:40:20.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.559+0000 7f1be6279e40 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.swift:foo:foo1, ofs0, ns] 2023-10-05T19:40:20.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.559+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:109 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94832752063168 in=178b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87fe0d0 con 0x563ff876efd0 2023-10-05T19:40:20.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.559+0000 7f1be3dae700 1 -- 172.21.15.12:0/1842477014 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== watch-notify(notify (1) cookie 94832751508064 notify 120259084290 ret 0) v3 ==== 208+0+0 (crc 0 0 0) 0x563ff87619a0 con 0x563ff876efd0 2023-10-05T19:40:20.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.559+0000 7f1be15a9700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084290 cookie 94832751508064 notifier 4333 bl.length()=166 2023-10-05T19:40:20.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.559+0000 7f1be15a9700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.swift+foo:foo1 info.flags=0x7 2023-10-05T19:40:20.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.559+0000 7f1be15a9700 10 rgw watcher librados: moving default.rgw.meta+users.swift+foo:foo1 to cache LRU end 2023-10-05T19:40:20.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.559+0000 7f1be15a9700 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:110 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f1b34003410 con 0x563ff876efd0 2023-10-05T19:40:20.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.559+0000 7f1be3dae700 1 -- 172.21.15.12:0/1842477014 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(109 notify.0 [notify cookie 94832752063168 out=8b] v0'0 uv54 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f1bd4040040 con 0x563ff876efd0 2023-10-05T19:40:20.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.563+0000 7f1be3dae700 1 -- 172.21.15.12:0/1842477014 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== watch-notify(notify_complete (2) cookie 94832752063168 notify 120259084290 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x563ff8761560 con 0x563ff876efd0 2023-10-05T19:40:20.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.563+0000 7f1be3dae700 1 -- 172.21.15.12:0/1842477014 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(110 notify.0 [notify-ack] v0'0 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1bd4040040 con 0x563ff876efd0 2023-10-05T19:40:20.571 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:40:20.571 INFO:teuthology.orchestra.run.smithi012.stdout: "user_id": "foo", 2023-10-05T19:40:20.571 INFO:teuthology.orchestra.run.smithi012.stdout: "display_name": "Foo", 2023-10-05T19:40:20.571 INFO:teuthology.orchestra.run.smithi012.stdout: "email": "foo@foo.com", 2023-10-05T19:40:20.572 INFO:teuthology.orchestra.run.smithi012.stdout: "suspended": 0, 2023-10-05T19:40:20.572 INFO:teuthology.orchestra.run.smithi012.stdout: "max_buckets": 4, 2023-10-05T19:40:20.572 INFO:teuthology.orchestra.run.smithi012.stdout: "subusers": [ 2023-10-05T19:40:20.572 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:20.572 INFO:teuthology.orchestra.run.smithi012.stdout: "id": "foo:foo1", 2023-10-05T19:40:20.572 INFO:teuthology.orchestra.run.smithi012.stdout: "permissions": "full-control" 2023-10-05T19:40:20.572 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:20.572 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:20.572 INFO:teuthology.orchestra.run.smithi012.stdout: "keys": [ 2023-10-05T19:40:20.572 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:20.572 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:40:20.572 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2023-10-05T19:40:20.573 INFO:teuthology.orchestra.run.smithi012.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2023-10-05T19:40:20.573 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:20.573 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:20.573 INFO:teuthology.orchestra.run.smithi012.stdout: "swift_keys": [ 2023-10-05T19:40:20.573 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:20.573 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo:foo1", 2023-10-05T19:40:20.573 INFO:teuthology.orchestra.run.smithi012.stdout: "secret_key": "j0cinr5ujBbUfiYljXZjcj4CDXSQaTv5V1BLODHJ" 2023-10-05T19:40:20.573 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:20.573 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:20.573 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": [], 2023-10-05T19:40:20.573 INFO:teuthology.orchestra.run.smithi012.stdout: "op_mask": "read, write, delete", 2023-10-05T19:40:20.573 INFO:teuthology.orchestra.run.smithi012.stdout: "default_placement": "", 2023-10-05T19:40:20.574 INFO:teuthology.orchestra.run.smithi012.stdout: "default_storage_class": "", 2023-10-05T19:40:20.574 INFO:teuthology.orchestra.run.smithi012.stdout: "placement_tags": [], 2023-10-05T19:40:20.574 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket_quota": { 2023-10-05T19:40:20.574 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:20.574 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:20.574 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:20.574 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:20.574 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:20.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:20.574 INFO:teuthology.orchestra.run.smithi012.stdout: "user_quota": { 2023-10-05T19:40:20.574 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:20.574 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:20.575 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:20.575 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:20.575 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:20.575 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:20.575 INFO:teuthology.orchestra.run.smithi012.stdout: "temp_url_keys": [], 2023-10-05T19:40:20.575 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "rgw", 2023-10-05T19:40:20.575 INFO:teuthology.orchestra.run.smithi012.stdout: "mfa_ids": [] 2023-10-05T19:40:20.575 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:40:20.575 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:40:20.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.563+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:111 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94832751508064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87ff2c0 con 0x563ff876efd0 2023-10-05T19:40:20.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.563+0000 7f1be3dae700 1 -- 172.21.15.12:0/1842477014 <== osd.2 v2:172.21.15.12:6800/29934 23 ==== osd_op_reply(111 notify.0 [watch unwatch cookie 94832751508064] v28'56 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1bd4040040 con 0x563ff876efd0 2023-10-05T19:40:20.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.563+0000 7f1be6279e40 20 remove_watcher() i=0 2023-10-05T19:40:20.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.563+0000 7f1be6279e40 2 removed watcher, disabling cache 2023-10-05T19:40:20.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.563+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:112 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94832751520512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87ff2c0 con 0x563ff8735f30 2023-10-05T19:40:20.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.567+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.1 v2:172.21.15.12:6808/29933 42 ==== osd_op_reply(112 notify.1 [watch unwatch cookie 94832751520512] v28'56 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x563ff8735f30 2023-10-05T19:40:20.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.567+0000 7f1be6279e40 20 remove_watcher() i=1 2023-10-05T19:40:20.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.567+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:113 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94832751523344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87ff2c0 con 0x563ff87736e0 2023-10-05T19:40:20.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.571+0000 7f1be35ad700 1 -- 172.21.15.12:0/1842477014 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(113 notify.2 [watch unwatch cookie 94832751523344] v28'56 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1bcc006070 con 0x563ff87736e0 2023-10-05T19:40:20.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.571+0000 7f1be6279e40 20 remove_watcher() i=2 2023-10-05T19:40:20.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.571+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:114 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94832751529072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff877b610 con 0x563ff876efd0 2023-10-05T19:40:20.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.575+0000 7f1be3dae700 1 -- 172.21.15.12:0/1842477014 <== osd.2 v2:172.21.15.12:6800/29934 24 ==== osd_op_reply(114 notify.3 [watch unwatch cookie 94832751529072] v28'56 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1bd4040040 con 0x563ff876efd0 2023-10-05T19:40:20.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.575+0000 7f1be6279e40 20 remove_watcher() i=3 2023-10-05T19:40:20.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.575+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:115 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94832751521616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87ff2c0 con 0x563ff8735f30 2023-10-05T19:40:20.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.575+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.1 v2:172.21.15.12:6808/29933 43 ==== osd_op_reply(115 notify.4 [watch unwatch cookie 94832751521616] v28'56 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x563ff8735f30 2023-10-05T19:40:20.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.575+0000 7f1be6279e40 20 remove_watcher() i=4 2023-10-05T19:40:20.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.575+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:116 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94832751530176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff877cc70 con 0x7f1bd4007030 2023-10-05T19:40:20.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.579+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(116 notify.5 [watch unwatch cookie 94832751530176] v28'56 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x7f1bd4007030 2023-10-05T19:40:20.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.579+0000 7f1be6279e40 20 remove_watcher() i=5 2023-10-05T19:40:20.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.579+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:117 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94832751536976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87ff2c0 con 0x7f1bd4007030 2023-10-05T19:40:20.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.583+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(117 notify.6 [watch unwatch cookie 94832751536976] v28'56 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x7f1bd4007030 2023-10-05T19:40:20.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.583+0000 7f1be6279e40 20 remove_watcher() i=6 2023-10-05T19:40:20.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.583+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:118 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94832751534320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87ff2c0 con 0x7f1bd4007030 2023-10-05T19:40:20.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.583+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(118 notify.7 [watch unwatch cookie 94832751534320] v28'56 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x7f1bd4007030 2023-10-05T19:40:20.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.583+0000 7f1be6279e40 20 remove_watcher() i=7 2023-10-05T19:40:20.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.587+0000 7f1b2ffff700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:20.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.587+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:119 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94832751540800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ff87ff2c0 con 0x563ff8735f30 2023-10-05T19:40:20.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.591+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 <== osd.1 v2:172.21.15.12:6808/29933 44 ==== osd_op_reply(119 data_loggenerations_metadata [watch unwatch cookie 94832751540800] v28'71 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f1bdc00a5a0 con 0x563ff8735f30 2023-10-05T19:40:20.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.591+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f1bd4007030 msgr2=0x7f1bd40115e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.591+0000 7f1be6279e40 1 --2- 172.21.15.12:0/1842477014 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f1bd4007030 0x7f1bd40115e0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:20.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.591+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x563ff8735f30 msgr2=0x563ff876cae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.591+0000 7f1be6279e40 1 --2- 172.21.15.12:0/1842477014 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x563ff8735f30 0x563ff876cae0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:20.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.591+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x563ff876efd0 msgr2=0x563ff87713b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.591+0000 7f1be6279e40 1 --2- 172.21.15.12:0/1842477014 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x563ff876efd0 0x563ff87713b0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:20.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.591+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x563ff87736e0 msgr2=0x563ff8775af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.591+0000 7f1be6279e40 1 --2- 172.21.15.12:0/1842477014 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x563ff87736e0 0x563ff8775af0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:20.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.591+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1bc8030c90 msgr2=0x7f1bc8033140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.591+0000 7f1be6279e40 1 --2- 172.21.15.12:0/1842477014 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1bc8030c90 0x7f1bc8033140 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc004170 tx=0x7f1bcc001400 comp rx=0 tx=0).stop 2023-10-05T19:40:20.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.591+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ff86a8b50 msgr2=0x563ff8761020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.591+0000 7f1be6279e40 1 --2- 172.21.15.12:0/1842477014 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ff86a8b50 0x563ff8761020 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1bd401f040 tx=0x7f1bd401c000 comp rx=0 tx=0).stop 2023-10-05T19:40:20.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.591+0000 7f1be45af700 1 -- 172.21.15.12:0/1842477014 reap_dead start 2023-10-05T19:40:20.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.591+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 shutdown_connections 2023-10-05T19:40:20.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.591+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 >> 172.21.15.12:0/1842477014 conn(0x563ff872f750 msgr2=0x563ff87549a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:20.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.591+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 shutdown_connections 2023-10-05T19:40:20.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.591+0000 7f1be6279e40 1 -- 172.21.15.12:0/1842477014 wait complete. 2023-10-05T19:40:20.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.591+0000 7f1be6279e40 1 -- 172.21.15.12:0/806876521 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1b20030ca0 msgr2=0x7f1b20033150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.591+0000 7f1be6279e40 1 --2- 172.21.15.12:0/806876521 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1b20030ca0 0x7f1b20033150 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x563ff877f170 tx=0x7f1bdc00af00 comp rx=0 tx=0).stop 2023-10-05T19:40:20.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.591+0000 7f1be6279e40 1 -- 172.21.15.12:0/806876521 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ff878d3d0 msgr2=0x563ff8793030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.591+0000 7f1be6279e40 1 --2- 172.21.15.12:0/806876521 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ff878d3d0 0x563ff8793030 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x563ff8787450 tx=0x7f1bcc03d030 comp rx=0 tx=0).stop 2023-10-05T19:40:20.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.591+0000 7f1be6279e40 1 -- 172.21.15.12:0/806876521 shutdown_connections 2023-10-05T19:40:20.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.591+0000 7f1be6279e40 1 --2- 172.21.15.12:0/806876521 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1b20030ca0 0x7f1b20033150 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:20.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.591+0000 7f1be6279e40 1 --2- 172.21.15.12:0/806876521 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ff878d3d0 0x563ff8793030 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:20.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.591+0000 7f1be6279e40 1 -- 172.21.15.12:0/806876521 >> 172.21.15.12:0/806876521 conn(0x563ff8788790 msgr2=0x563ff8786830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:20.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.591+0000 7f1be6279e40 1 -- 172.21.15.12:0/806876521 shutdown_connections 2023-10-05T19:40:20.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.591+0000 7f1be6279e40 1 -- 172.21.15.12:0/806876521 wait complete. 2023-10-05T19:40:20.606 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'j0cinr5ujBbUfiYljXZjcj4CDXSQaTv5V1BLODHJ'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:40:20.606 INFO:tasks.util.rgw:rgwadmin: client.0 : ['subuser', 'modify', '--subuser', 'foo:foo1', '--secret', 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', '--key-type', 'swift'] 2023-10-05T19:40:20.606 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'subuser', 'modify', '--subuser', 'foo:foo1', '--secret', 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', '--key-type', 'swift'] 2023-10-05T19:40:20.607 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph subuser modify --subuser foo:foo1 --secret gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL --key-type swift 2023-10-05T19:40:20.650 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:20.651 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:20.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.643+0000 7f655a7ffe40 1 Processor -- start 2023-10-05T19:40:20.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.643+0000 7f655a7ffe40 1 -- start start 2023-10-05T19:40:20.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.643+0000 7f655a7ffe40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55f9e9ce03b0 0x55f9e9ce2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.643+0000 7f655a7ffe40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55f9e9c28bd0 con 0x55f9e9cdbdd0 2023-10-05T19:40:20.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.643+0000 7f655a7ffe40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x55f9e9be5ee0 con 0x55f9e9ce03b0 2023-10-05T19:40:20.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.643+0000 7f6557b33700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55f9e9ce03b0 0x55f9e9ce2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.643+0000 7f6558334700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x55f9e9cdbdd0 0x55f9e9cdc1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:47310/0 (socket says 172.21.15.12:47310) 2023-10-05T19:40:20.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.643+0000 7f6558334700 1 -- 172.21.15.12:0/595984443 learned_addr learned my addr 172.21.15.12:0/595984443 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:20.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.643+0000 7f6557b33700 1 -- 172.21.15.12:0/595984443 >> v1:172.21.15.8:6789/0 conn(0x55f9e9cdbdd0 legacy=0x55f9e9cdc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.643+0000 7f6557b33700 1 -- 172.21.15.12:0/595984443 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9e9cdc9e0 con 0x55f9e9ce03b0 2023-10-05T19:40:20.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.643+0000 7f6557b33700 1 --2- 172.21.15.12:0/595984443 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f9e9ce03b0 0x55f9e9ce2790 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f654000a520 tx=0x7f6540005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=924c368f49f8262 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.647+0000 7f6557332700 1 -- 172.21.15.12:0/595984443 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f654000f590 con 0x55f9e9ce03b0 2023-10-05T19:40:20.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.647+0000 7f6557332700 1 -- 172.21.15.12:0/595984443 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f654000f6f0 con 0x55f9e9ce03b0 2023-10-05T19:40:20.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.647+0000 7f6557332700 1 -- 172.21.15.12:0/595984443 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f654000f880 con 0x55f9e9ce03b0 2023-10-05T19:40:20.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.647+0000 7f655a7ffe40 1 -- 172.21.15.12:0/595984443 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f9e9ce03b0 msgr2=0x55f9e9ce2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.647+0000 7f655a7ffe40 1 --2- 172.21.15.12:0/595984443 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f9e9ce03b0 0x55f9e9ce2790 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f654000a520 tx=0x7f6540005d40 comp rx=0 tx=0).stop 2023-10-05T19:40:20.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.647+0000 7f655a7ffe40 1 -- 172.21.15.12:0/595984443 shutdown_connections 2023-10-05T19:40:20.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.647+0000 7f655a7ffe40 1 --2- 172.21.15.12:0/595984443 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f9e9ce03b0 0x55f9e9ce2790 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:20.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.647+0000 7f655a7ffe40 1 -- 172.21.15.12:0/595984443 >> 172.21.15.12:0/595984443 conn(0x55f9e9cd77f0 msgr2=0x55f9e9cd9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:20.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.647+0000 7f655a7ffe40 1 -- 172.21.15.12:0/595984443 shutdown_connections 2023-10-05T19:40:20.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.647+0000 7f655a7ffe40 1 -- 172.21.15.12:0/595984443 wait complete. 2023-10-05T19:40:20.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.647+0000 7f655a7ffe40 1 Processor -- start 2023-10-05T19:40:20.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.647+0000 7f655a7ffe40 1 -- start start 2023-10-05T19:40:20.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.647+0000 7f655a7ffe40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f9e9cdbdd0 0x55f9e9cfafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.647+0000 7f655a7ffe40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9e9ba9e30 con 0x55f9e9cdbdd0 2023-10-05T19:40:20.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.647+0000 7f6558334700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f9e9cdbdd0 0x55f9e9cfafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.647+0000 7f6558334700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f9e9cdbdd0 0x55f9e9cfafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37136/0 (socket says 172.21.15.12:37136) 2023-10-05T19:40:20.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.647+0000 7f6558334700 1 -- 172.21.15.12:0/592264218 learned_addr learned my addr 172.21.15.12:0/592264218 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:20.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.647+0000 7f6558334700 1 -- 172.21.15.12:0/592264218 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f654000a0e0 con 0x55f9e9cdbdd0 2023-10-05T19:40:20.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.647+0000 7f6558334700 1 --2- 172.21.15.12:0/592264218 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f9e9cdbdd0 0x55f9e9cfafa0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6548001bf0 tx=0x7f65480017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12d6b8bbd299ef52 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.647+0000 7f6556330700 1 -- 172.21.15.12:0/592264218 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f654800bbf0 con 0x55f9e9cdbdd0 2023-10-05T19:40:20.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.647+0000 7f6556330700 1 -- 172.21.15.12:0/592264218 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f654800bd50 con 0x55f9e9cdbdd0 2023-10-05T19:40:20.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.647+0000 7f6556330700 1 -- 172.21.15.12:0/592264218 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f654800e470 con 0x55f9e9cdbdd0 2023-10-05T19:40:20.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.651+0000 7f655a7ffe40 1 -- 172.21.15.12:0/592264218 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f9e9cdbdd0 msgr2=0x55f9e9cfafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.651+0000 7f655a7ffe40 1 --2- 172.21.15.12:0/592264218 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f9e9cdbdd0 0x55f9e9cfafa0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6548001bf0 tx=0x7f65480017a0 comp rx=0 tx=0).stop 2023-10-05T19:40:20.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.651+0000 7f655a7ffe40 1 -- 172.21.15.12:0/592264218 shutdown_connections 2023-10-05T19:40:20.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.651+0000 7f655a7ffe40 1 --2- 172.21.15.12:0/592264218 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f9e9cdbdd0 0x55f9e9cfafa0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:20.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.651+0000 7f655a7ffe40 1 -- 172.21.15.12:0/592264218 >> 172.21.15.12:0/592264218 conn(0x55f9e9cd77f0 msgr2=0x55f9e9cf6ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:20.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.651+0000 7f655a7ffe40 1 -- 172.21.15.12:0/592264218 shutdown_connections 2023-10-05T19:40:20.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.651+0000 7f655a7ffe40 1 -- 172.21.15.12:0/592264218 wait complete. 2023-10-05T19:40:20.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.651+0000 7f655a7ffe40 1 Processor -- start 2023-10-05T19:40:20.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.651+0000 7f655a7ffe40 1 -- start start 2023-10-05T19:40:20.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.651+0000 7f655a7ffe40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f9e9cdbdd0 0x55f9e9c51a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.651+0000 7f655a7ffe40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9e9b7ced0 con 0x55f9e9cdbdd0 2023-10-05T19:40:20.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.651+0000 7f6558334700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f9e9cdbdd0 0x55f9e9c51a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.651+0000 7f6558334700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f9e9cdbdd0 0x55f9e9c51a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37150/0 (socket says 172.21.15.12:37150) 2023-10-05T19:40:20.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.651+0000 7f6558334700 1 -- 172.21.15.12:0/1573777961 learned_addr learned my addr 172.21.15.12:0/1573777961 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:20.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.651+0000 7f6558334700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9e9c28bd0 con 0x55f9e9cdbdd0 2023-10-05T19:40:20.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.651+0000 7f6558334700 1 --2- 172.21.15.12:0/1573777961 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f9e9cdbdd0 0x55f9e9c51a90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55f9e9cf7600 tx=0x7f6548016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.651+0000 7f6554b2d700 1 -- 172.21.15.12:0/1573777961 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f654801f070 con 0x55f9e9cdbdd0 2023-10-05T19:40:20.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.651+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f9e9c51fd0 con 0x55f9e9cdbdd0 2023-10-05T19:40:20.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.651+0000 7f6554b2d700 1 -- 172.21.15.12:0/1573777961 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f654801a2e0 con 0x55f9e9cdbdd0 2023-10-05T19:40:20.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.651+0000 7f6554b2d700 1 -- 172.21.15.12:0/1573777961 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f654800bec0 con 0x55f9e9cdbdd0 2023-10-05T19:40:20.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.651+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f9e9c50390 con 0x55f9e9cdbdd0 2023-10-05T19:40:20.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.655+0000 7f655a7ffe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.655+0000 7f6554b2d700 1 -- 172.21.15.12:0/1573777961 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f654801a960 con 0x55f9e9cdbdd0 2023-10-05T19:40:20.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.655+0000 7f6554b2d700 1 --2- 172.21.15.12:0/1573777961 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f6538030c00 0x7f65380330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.655+0000 7f6554b2d700 1 -- 172.21.15.12:0/1573777961 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f6548045050 con 0x55f9e9cdbdd0 2023-10-05T19:40:20.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.655+0000 7f655a7ffe40 1 --2- 172.21.15.12:0/1573777961 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55f9e9ce03b0 0x55f9e9d0fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.655+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d10100 con 0x55f9e9ce03b0 2023-10-05T19:40:20.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.655+0000 7f6557b33700 1 --2- 172.21.15.12:0/1573777961 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f6538030c00 0x7f65380330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.655+0000 7f6558b35700 1 --2- 172.21.15.12:0/1573777961 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55f9e9ce03b0 0x55f9e9d0fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.655+0000 7f6557b33700 1 --2- 172.21.15.12:0/1573777961 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f6538030c00 0x7f65380330b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6540005d80 tx=0x7f6540014000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.655+0000 7f6558b35700 1 --2- 172.21.15.12:0/1573777961 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55f9e9ce03b0 0x55f9e9d0fa50 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.655+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f655000a500 con 0x55f9e9ce03b0 2023-10-05T19:40:20.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.655+0000 7f655a7ffe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.655+0000 7f655a7ffe40 20 realm 2023-10-05T19:40:20.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.655+0000 7f655a7ffe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.655+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d0d0b0 con 0x55f9e9ce03b0 2023-10-05T19:40:20.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.659+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f655000a500 con 0x55f9e9ce03b0 2023-10-05T19:40:20.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.659+0000 7f655a7ffe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.659+0000 7f655a7ffe40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:20.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.659+0000 7f655a7ffe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.659+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d10be0 con 0x55f9e9ce03b0 2023-10-05T19:40:20.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.659+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f655000a500 con 0x55f9e9ce03b0 2023-10-05T19:40:20.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.659+0000 7f655a7ffe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.659+0000 7f655a7ffe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.659+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d107f0 con 0x55f9e9ce03b0 2023-10-05T19:40:20.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.659+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f655000a500 con 0x55f9e9ce03b0 2023-10-05T19:40:20.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.659+0000 7f655a7ffe40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:20.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.659+0000 7f655a7ffe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.659+0000 7f655a7ffe40 1 --2- 172.21.15.12:0/1573777961 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55f9e9d11f80 0x55f9e9d14360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.659+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d148a0 con 0x55f9e9d11f80 2023-10-05T19:40:20.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.659+0000 7f6558334700 1 --2- 172.21.15.12:0/1573777961 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55f9e9d11f80 0x55f9e9d14360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.659+0000 7f6558334700 1 --2- 172.21.15.12:0/1573777961 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55f9e9d11f80 0x55f9e9d14360 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.659+0000 7f6558334700 1 -- 172.21.15.12:0/1573777961 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f6548014070 con 0x55f9e9d11f80 2023-10-05T19:40:20.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.659+0000 7f655a7ffe40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:20.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.659+0000 7f655a7ffe40 20 searching for the correct realm 2023-10-05T19:40:20.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.659+0000 7f655a7ffe40 1 --2- 172.21.15.12:0/1573777961 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55f9e9d16720 0x55f9e9d18b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.659+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d191e0 con 0x55f9e9d16720 2023-10-05T19:40:20.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.659+0000 7f6557b33700 1 --2- 172.21.15.12:0/1573777961 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55f9e9d16720 0x55f9e9d18b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.659+0000 7f6557b33700 1 --2- 172.21.15.12:0/1573777961 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55f9e9d16720 0x55f9e9d18b30 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.663+0000 7f6557b33700 1 -- 172.21.15.12:0/1573777961 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f654000fcc0 con 0x55f9e9d16720 2023-10-05T19:40:20.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.663+0000 7f6557b33700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d191e0 con 0x55f9e9d11f80 2023-10-05T19:40:20.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.663+0000 7f6558334700 1 -- 172.21.15.12:0/1573777961 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6548014070 con 0x55f9e9d11f80 2023-10-05T19:40:20.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.663+0000 7f6558334700 1 --2- 172.21.15.12:0/1573777961 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f654802d020 0x7f654803d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.663+0000 7f6558334700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d191e0 con 0x7f654802d020 2023-10-05T19:40:20.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.663+0000 7f6558b35700 1 --2- 172.21.15.12:0/1573777961 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f654802d020 0x7f654803d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.663+0000 7f6558b35700 1 --2- 172.21.15.12:0/1573777961 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f654802d020 0x7f654803d920 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.667+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6550014040 con 0x7f654802d020 2023-10-05T19:40:20.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.667+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d191e0 con 0x55f9e9ce03b0 2023-10-05T19:40:20.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.667+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6550014040 con 0x55f9e9ce03b0 2023-10-05T19:40:20.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.667+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d191e0 con 0x55f9e9ce03b0 2023-10-05T19:40:20.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.667+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6550014040 con 0x55f9e9ce03b0 2023-10-05T19:40:20.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.667+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d191e0 con 0x7f654802d020 2023-10-05T19:40:20.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.667+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6550014040 con 0x7f654802d020 2023-10-05T19:40:20.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.667+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d191e0 con 0x7f654802d020 2023-10-05T19:40:20.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.667+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6550014040 con 0x7f654802d020 2023-10-05T19:40:20.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.667+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d191e0 con 0x55f9e9d11f80 2023-10-05T19:40:20.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.671+0000 7f6558334700 1 -- 172.21.15.12:0/1573777961 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6548014070 con 0x55f9e9d11f80 2023-10-05T19:40:20.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.671+0000 7f6558334700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d191e0 con 0x55f9e9d16720 2023-10-05T19:40:20.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.671+0000 7f6557b33700 1 -- 172.21.15.12:0/1573777961 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f654000fcc0 con 0x55f9e9d16720 2023-10-05T19:40:20.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.671+0000 7f6557b33700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d191e0 con 0x55f9e9ce03b0 2023-10-05T19:40:20.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.671+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6550014040 con 0x55f9e9ce03b0 2023-10-05T19:40:20.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.671+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d191e0 con 0x7f654802d020 2023-10-05T19:40:20.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.671+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6550014040 con 0x7f654802d020 2023-10-05T19:40:20.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.671+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d191e0 con 0x55f9e9ce03b0 2023-10-05T19:40:20.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.675+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6550014040 con 0x55f9e9ce03b0 2023-10-05T19:40:20.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.675+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d191e0 con 0x55f9e9d11f80 2023-10-05T19:40:20.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.675+0000 7f6558334700 1 -- 172.21.15.12:0/1573777961 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6548014070 con 0x55f9e9d11f80 2023-10-05T19:40:20.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.675+0000 7f6558334700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d191e0 con 0x55f9e9d16720 2023-10-05T19:40:20.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.675+0000 7f6557b33700 1 -- 172.21.15.12:0/1573777961 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f654000fcc0 con 0x55f9e9d16720 2023-10-05T19:40:20.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.675+0000 7f6557b33700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d191e0 con 0x55f9e9d16720 2023-10-05T19:40:20.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.675+0000 7f6557b33700 1 -- 172.21.15.12:0/1573777961 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f654000fcc0 con 0x55f9e9d16720 2023-10-05T19:40:20.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.675+0000 7f6557b33700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d191e0 con 0x7f654802d020 2023-10-05T19:40:20.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.675+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6550014040 con 0x7f654802d020 2023-10-05T19:40:20.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.675+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d191e0 con 0x55f9e9d16720 2023-10-05T19:40:20.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.679+0000 7f6557b33700 1 -- 172.21.15.12:0/1573777961 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f654000fcc0 con 0x55f9e9d16720 2023-10-05T19:40:20.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.679+0000 7f6557b33700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d191e0 con 0x55f9e9d16720 2023-10-05T19:40:20.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.679+0000 7f6557b33700 1 -- 172.21.15.12:0/1573777961 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f654000fcc0 con 0x55f9e9d16720 2023-10-05T19:40:20.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.679+0000 7f6557b33700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d191e0 con 0x55f9e9ce03b0 2023-10-05T19:40:20.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.679+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6550014040 con 0x55f9e9ce03b0 2023-10-05T19:40:20.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.679+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d191e0 con 0x55f9e9ce03b0 2023-10-05T19:40:20.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.679+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6550014040 con 0x55f9e9ce03b0 2023-10-05T19:40:20.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.679+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d191e0 con 0x7f654802d020 2023-10-05T19:40:20.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.679+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6550014040 con 0x7f654802d020 2023-10-05T19:40:20.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.679+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d191e0 con 0x55f9e9d16720 2023-10-05T19:40:20.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.683+0000 7f6557b33700 1 -- 172.21.15.12:0/1573777961 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f654000fcc0 con 0x55f9e9d16720 2023-10-05T19:40:20.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.683+0000 7f6557b33700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d191e0 con 0x55f9e9d11f80 2023-10-05T19:40:20.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.683+0000 7f6558334700 1 -- 172.21.15.12:0/1573777961 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6548014070 con 0x55f9e9d11f80 2023-10-05T19:40:20.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.683+0000 7f6558334700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d191e0 con 0x55f9e9ce03b0 2023-10-05T19:40:20.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.683+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6550014040 con 0x55f9e9ce03b0 2023-10-05T19:40:20.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.683+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d191e0 con 0x7f654802d020 2023-10-05T19:40:20.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.683+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6550014040 con 0x7f654802d020 2023-10-05T19:40:20.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.683+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d191e0 con 0x55f9e9d16720 2023-10-05T19:40:20.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.683+0000 7f6557b33700 1 -- 172.21.15.12:0/1573777961 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f654000fcc0 con 0x55f9e9d16720 2023-10-05T19:40:20.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.683+0000 7f6557b33700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d191e0 con 0x7f654802d020 2023-10-05T19:40:20.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.687+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6550014040 con 0x7f654802d020 2023-10-05T19:40:20.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.687+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d191e0 con 0x55f9e9d11f80 2023-10-05T19:40:20.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.687+0000 7f6558334700 1 -- 172.21.15.12:0/1573777961 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6548014070 con 0x55f9e9d11f80 2023-10-05T19:40:20.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.687+0000 7f6558334700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d191e0 con 0x55f9e9d16720 2023-10-05T19:40:20.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.687+0000 7f6557b33700 1 -- 172.21.15.12:0/1573777961 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f654000fcc0 con 0x55f9e9d16720 2023-10-05T19:40:20.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.687+0000 7f6557b33700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d191e0 con 0x7f654802d020 2023-10-05T19:40:20.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.687+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6550014040 con 0x7f654802d020 2023-10-05T19:40:20.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.687+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d191e0 con 0x55f9e9ce03b0 2023-10-05T19:40:20.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.687+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f6550014040 con 0x55f9e9ce03b0 2023-10-05T19:40:20.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.687+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d191e0 con 0x55f9e9d11f80 2023-10-05T19:40:20.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.687+0000 7f6558334700 1 -- 172.21.15.12:0/1573777961 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f6548014070 con 0x55f9e9d11f80 2023-10-05T19:40:20.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.687+0000 7f655a7ffe40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:20.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.687+0000 7f655a7ffe40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:20.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.687+0000 7f655a7ffe40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:20.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.687+0000 7f655a7ffe40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:20.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.687+0000 7f655a7ffe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.687+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d15470 con 0x55f9e9ce03b0 2023-10-05T19:40:20.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.691+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6550014040 con 0x55f9e9ce03b0 2023-10-05T19:40:20.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.691+0000 7f655a7ffe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.691+0000 7f655a7ffe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.691+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d150a0 con 0x55f9e9ce03b0 2023-10-05T19:40:20.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.691+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f6550014040 con 0x55f9e9ce03b0 2023-10-05T19:40:20.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.691+0000 7f655a7ffe40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:20.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.691+0000 7f655a7ffe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.691+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d19650 con 0x55f9e9d16720 2023-10-05T19:40:20.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.691+0000 7f6557b33700 1 -- 172.21.15.12:0/1573777961 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f654000fcc0 con 0x55f9e9d16720 2023-10-05T19:40:20.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.691+0000 7f655a7ffe40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:20.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.691+0000 7f655a7ffe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.691+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d19a00 con 0x55f9e9d16720 2023-10-05T19:40:20.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.691+0000 7f6557b33700 1 -- 172.21.15.12:0/1573777961 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f654000fcc0 con 0x55f9e9d16720 2023-10-05T19:40:20.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.691+0000 7f655a7ffe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.691+0000 7f655a7ffe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.691+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d19db0 con 0x55f9e9ce03b0 2023-10-05T19:40:20.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.691+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6550014040 con 0x55f9e9ce03b0 2023-10-05T19:40:20.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.691+0000 7f655a7ffe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.691+0000 7f655a7ffe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.691+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d1a190 con 0x55f9e9ce03b0 2023-10-05T19:40:20.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.691+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f6550014040 con 0x55f9e9ce03b0 2023-10-05T19:40:20.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.691+0000 7f655a7ffe40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:20.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.691+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d1a160 con 0x55f9e9d16720 2023-10-05T19:40:20.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.695+0000 7f6557b33700 1 -- 172.21.15.12:0/1573777961 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f654000fcc0 con 0x55f9e9d16720 2023-10-05T19:40:20.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.695+0000 7f6557b33700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d1a160 con 0x55f9e9d11f80 2023-10-05T19:40:20.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.695+0000 7f6558334700 1 -- 172.21.15.12:0/1573777961 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6548014070 con 0x55f9e9d11f80 2023-10-05T19:40:20.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.695+0000 7f6558334700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d1a160 con 0x7f654802d020 2023-10-05T19:40:20.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.699+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6550014040 con 0x7f654802d020 2023-10-05T19:40:20.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.699+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d19db0 con 0x55f9e9ce03b0 2023-10-05T19:40:20.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.699+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6550014040 con 0x55f9e9ce03b0 2023-10-05T19:40:20.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.699+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d19db0 con 0x55f9e9ce03b0 2023-10-05T19:40:20.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.699+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6550014040 con 0x55f9e9ce03b0 2023-10-05T19:40:20.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.699+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d19db0 con 0x7f654802d020 2023-10-05T19:40:20.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.699+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6550014040 con 0x7f654802d020 2023-10-05T19:40:20.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.699+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d19db0 con 0x7f654802d020 2023-10-05T19:40:20.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.699+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6550014040 con 0x7f654802d020 2023-10-05T19:40:20.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.699+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d19db0 con 0x55f9e9d11f80 2023-10-05T19:40:20.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.699+0000 7f6558334700 1 -- 172.21.15.12:0/1573777961 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6548014070 con 0x55f9e9d11f80 2023-10-05T19:40:20.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.699+0000 7f6558334700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d19db0 con 0x55f9e9d16720 2023-10-05T19:40:20.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.703+0000 7f6557b33700 1 -- 172.21.15.12:0/1573777961 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f654000fcc0 con 0x55f9e9d16720 2023-10-05T19:40:20.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.703+0000 7f6557b33700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6540039040 con 0x55f9e9ce03b0 2023-10-05T19:40:20.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.703+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6550014040 con 0x55f9e9ce03b0 2023-10-05T19:40:20.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.703+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6540039040 con 0x7f654802d020 2023-10-05T19:40:20.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.703+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6550014040 con 0x7f654802d020 2023-10-05T19:40:20.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.703+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6540039040 con 0x55f9e9ce03b0 2023-10-05T19:40:20.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.703+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6550014040 con 0x55f9e9ce03b0 2023-10-05T19:40:20.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.703+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6540039040 con 0x55f9e9d11f80 2023-10-05T19:40:20.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.703+0000 7f6558334700 1 -- 172.21.15.12:0/1573777961 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6548014070 con 0x55f9e9d11f80 2023-10-05T19:40:20.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.703+0000 7f6558334700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6540039040 con 0x55f9e9d16720 2023-10-05T19:40:20.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.707+0000 7f6557b33700 1 -- 172.21.15.12:0/1573777961 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f654000fcc0 con 0x55f9e9d16720 2023-10-05T19:40:20.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.707+0000 7f6557b33700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6540039040 con 0x55f9e9d16720 2023-10-05T19:40:20.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.707+0000 7f6557b33700 1 -- 172.21.15.12:0/1573777961 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f654000fcc0 con 0x55f9e9d16720 2023-10-05T19:40:20.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.707+0000 7f6557b33700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6540039040 con 0x7f654802d020 2023-10-05T19:40:20.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.707+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6550014040 con 0x7f654802d020 2023-10-05T19:40:20.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.707+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6540039040 con 0x55f9e9d16720 2023-10-05T19:40:20.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.707+0000 7f6557b33700 1 -- 172.21.15.12:0/1573777961 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f654000fcc0 con 0x55f9e9d16720 2023-10-05T19:40:20.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.707+0000 7f6557b33700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6540039040 con 0x55f9e9d16720 2023-10-05T19:40:20.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.707+0000 7f6557b33700 1 -- 172.21.15.12:0/1573777961 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f654000fcc0 con 0x55f9e9d16720 2023-10-05T19:40:20.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.707+0000 7f6557b33700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6540039040 con 0x55f9e9ce03b0 2023-10-05T19:40:20.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.711+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6550014040 con 0x55f9e9ce03b0 2023-10-05T19:40:20.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.711+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6540039040 con 0x55f9e9ce03b0 2023-10-05T19:40:20.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.711+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6550014040 con 0x55f9e9ce03b0 2023-10-05T19:40:20.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.711+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6540039040 con 0x7f654802d020 2023-10-05T19:40:20.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.711+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6550014040 con 0x7f654802d020 2023-10-05T19:40:20.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.711+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6540039040 con 0x55f9e9d16720 2023-10-05T19:40:20.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.711+0000 7f6557b33700 1 -- 172.21.15.12:0/1573777961 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f654000fcc0 con 0x55f9e9d16720 2023-10-05T19:40:20.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.711+0000 7f6557b33700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6540039040 con 0x55f9e9d11f80 2023-10-05T19:40:20.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.711+0000 7f6558334700 1 -- 172.21.15.12:0/1573777961 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6548014070 con 0x55f9e9d11f80 2023-10-05T19:40:20.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.711+0000 7f6558334700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6540039040 con 0x55f9e9ce03b0 2023-10-05T19:40:20.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.715+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6550014040 con 0x55f9e9ce03b0 2023-10-05T19:40:20.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.715+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6540039040 con 0x7f654802d020 2023-10-05T19:40:20.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.715+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6550014040 con 0x7f654802d020 2023-10-05T19:40:20.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.715+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6540039040 con 0x55f9e9d16720 2023-10-05T19:40:20.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.715+0000 7f6557b33700 1 -- 172.21.15.12:0/1573777961 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f654000fcc0 con 0x55f9e9d16720 2023-10-05T19:40:20.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.715+0000 7f6557b33700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6540039040 con 0x7f654802d020 2023-10-05T19:40:20.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.715+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6550014040 con 0x7f654802d020 2023-10-05T19:40:20.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.715+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6540039040 con 0x55f9e9d11f80 2023-10-05T19:40:20.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.719+0000 7f6558334700 1 -- 172.21.15.12:0/1573777961 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6548014070 con 0x55f9e9d11f80 2023-10-05T19:40:20.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.719+0000 7f6558334700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6540039040 con 0x55f9e9d16720 2023-10-05T19:40:20.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.719+0000 7f6557b33700 1 -- 172.21.15.12:0/1573777961 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f654000fcc0 con 0x55f9e9d16720 2023-10-05T19:40:20.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.719+0000 7f6557b33700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6540039040 con 0x7f654802d020 2023-10-05T19:40:20.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.719+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6550014040 con 0x7f654802d020 2023-10-05T19:40:20.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.719+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6540039040 con 0x55f9e9ce03b0 2023-10-05T19:40:20.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.719+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f6550014040 con 0x55f9e9ce03b0 2023-10-05T19:40:20.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.719+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6540039040 con 0x55f9e9d11f80 2023-10-05T19:40:20.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.719+0000 7f6558334700 1 -- 172.21.15.12:0/1573777961 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f654804d080 con 0x55f9e9d11f80 2023-10-05T19:40:20.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.719+0000 7f655a7ffe40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:20.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.719+0000 7f655a7ffe40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:20.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.719+0000 7f655a7ffe40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:20.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.719+0000 7f655a7ffe40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:20.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.719+0000 7f655a7ffe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.719+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d1a160 con 0x55f9e9ce03b0 2023-10-05T19:40:20.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.723+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f6550014040 con 0x55f9e9ce03b0 2023-10-05T19:40:20.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.723+0000 7f655a7ffe40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:20.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.723+0000 7f655a7ffe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.723+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d1a5c0 con 0x55f9e9d11f80 2023-10-05T19:40:20.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.723+0000 7f6558334700 1 -- 172.21.15.12:0/1573777961 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f654804d080 con 0x55f9e9d11f80 2023-10-05T19:40:20.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.723+0000 7f655a7ffe40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:20.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.723+0000 7f655a7ffe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.723+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d1aeb0 con 0x55f9e9ce03b0 2023-10-05T19:40:20.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.723+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f6550014040 con 0x55f9e9ce03b0 2023-10-05T19:40:20.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.723+0000 7f655a7ffe40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:20.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.723+0000 7f655a7ffe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.723+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d1aeb0 con 0x55f9e9d16720 2023-10-05T19:40:20.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.723+0000 7f6557b33700 1 -- 172.21.15.12:0/1573777961 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f654000fcc0 con 0x55f9e9d16720 2023-10-05T19:40:20.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.723+0000 7f655a7ffe40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:20.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.723+0000 7f655a7ffe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.723+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d1b620 con 0x7f654802d020 2023-10-05T19:40:20.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.727+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f6550014040 con 0x7f654802d020 2023-10-05T19:40:20.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.727+0000 7f655a7ffe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.727+0000 7f655a7ffe40 20 zone default found 2023-10-05T19:40:20.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.727+0000 7f655a7ffe40 4 Realm: () 2023-10-05T19:40:20.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.727+0000 7f655a7ffe40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:20.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.727+0000 7f655a7ffe40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:20.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.727+0000 7f655a7ffe40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:20.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.727+0000 7f655a7ffe40 20 zonegroup default 2023-10-05T19:40:20.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.727+0000 7f655a7ffe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.727+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d1b420 con 0x7f654802d020 2023-10-05T19:40:20.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.727+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f6550014040 con 0x7f654802d020 2023-10-05T19:40:20.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.727+0000 7f655a7ffe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.727+0000 7f655a7ffe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.727+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d1cda0 con 0x55f9e9d16720 2023-10-05T19:40:20.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.727+0000 7f6557b33700 1 -- 172.21.15.12:0/1573777961 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f654000fcc0 con 0x55f9e9d16720 2023-10-05T19:40:20.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.727+0000 7f655a7ffe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.727+0000 7f655a7ffe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.727+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d1d150 con 0x55f9e9d16720 2023-10-05T19:40:20.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.727+0000 7f6557b33700 1 -- 172.21.15.12:0/1573777961 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f654000fcc0 con 0x55f9e9d16720 2023-10-05T19:40:20.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.727+0000 7f655a7ffe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.727+0000 7f655a7ffe40 20 started sync module instance, tier type = 2023-10-05T19:40:20.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.727+0000 7f655a7ffe40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:20.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.727+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d1c460 con 0x55f9e9d11f80 2023-10-05T19:40:20.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.731+0000 7f6558334700 1 -- 172.21.15.12:0/1573777961 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'57 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f654804d080 con 0x55f9e9d11f80 2023-10-05T19:40:20.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.731+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94531858049472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d1d500 con 0x55f9e9d11f80 2023-10-05T19:40:20.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.731+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d1dde0 con 0x55f9e9ce03b0 2023-10-05T19:40:20.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.735+0000 7f6558334700 1 -- 172.21.15.12:0/1573777961 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94531858049472] v28'58 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f654804d080 con 0x55f9e9d11f80 2023-10-05T19:40:20.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.735+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'57 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6550014040 con 0x55f9e9ce03b0 2023-10-05T19:40:20.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.735+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94531858057232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d1e1e0 con 0x55f9e9ce03b0 2023-10-05T19:40:20.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.735+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d1ea80 con 0x55f9e9d16720 2023-10-05T19:40:20.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.735+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94531858057232] v28'58 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6550014040 con 0x55f9e9ce03b0 2023-10-05T19:40:20.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.739+0000 7f6557b33700 1 -- 172.21.15.12:0/1573777961 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'57 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f654000fcc0 con 0x55f9e9d16720 2023-10-05T19:40:20.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.739+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94531858063840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d1d9d0 con 0x55f9e9d16720 2023-10-05T19:40:20.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.739+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d1f370 con 0x55f9e9d11f80 2023-10-05T19:40:20.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.739+0000 7f6557b33700 1 -- 172.21.15.12:0/1573777961 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94531858063840] v28'58 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f654000fcc0 con 0x55f9e9d16720 2023-10-05T19:40:20.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.739+0000 7f6558334700 1 -- 172.21.15.12:0/1573777961 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'57 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f654804d080 con 0x55f9e9d11f80 2023-10-05T19:40:20.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.739+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94531858067072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d1f720 con 0x55f9e9d11f80 2023-10-05T19:40:20.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.739+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d1ffc0 con 0x55f9e9ce03b0 2023-10-05T19:40:20.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.743+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'57 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6550014040 con 0x55f9e9ce03b0 2023-10-05T19:40:20.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.743+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94531858069840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d20350 con 0x55f9e9ce03b0 2023-10-05T19:40:20.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.743+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d20eb0 con 0x7f654802d020 2023-10-05T19:40:20.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.743+0000 7f6558334700 1 -- 172.21.15.12:0/1573777961 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94531858067072] v28'58 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f654804d080 con 0x55f9e9d11f80 2023-10-05T19:40:20.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.747+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'57 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6550014040 con 0x7f654802d020 2023-10-05T19:40:20.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.747+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94531858064944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d20e70 con 0x7f654802d020 2023-10-05T19:40:20.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.747+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d21910 con 0x7f654802d020 2023-10-05T19:40:20.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.747+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94531858069840] v28'58 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6550014040 con 0x55f9e9ce03b0 2023-10-05T19:40:20.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.747+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(95 notify.5 [watch watch cookie 94531858064944] v28'58 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6550014040 con 0x7f654802d020 2023-10-05T19:40:20.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.747+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(96 notify.6 [create] v28'57 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6550014040 con 0x7f654802d020 2023-10-05T19:40:20.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.747+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94531858071280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d21420 con 0x7f654802d020 2023-10-05T19:40:20.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.747+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d21cc0 con 0x7f654802d020 2023-10-05T19:40:20.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.751+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(97 notify.6 [watch watch cookie 94531858071280] v28'58 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6550014040 con 0x7f654802d020 2023-10-05T19:40:20.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.751+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(98 notify.7 [create] v28'57 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6550014040 con 0x7f654802d020 2023-10-05T19:40:20.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.751+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94531858076704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d220c0 con 0x7f654802d020 2023-10-05T19:40:20.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.751+0000 7f655a7ffe40 20 add_watcher() i=0 2023-10-05T19:40:20.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.751+0000 7f655a7ffe40 20 add_watcher() i=1 2023-10-05T19:40:20.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.751+0000 7f655a7ffe40 20 add_watcher() i=2 2023-10-05T19:40:20.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.751+0000 7f655a7ffe40 20 add_watcher() i=3 2023-10-05T19:40:20.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.751+0000 7f655a7ffe40 20 add_watcher() i=4 2023-10-05T19:40:20.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.751+0000 7f655a7ffe40 20 add_watcher() i=5 2023-10-05T19:40:20.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.751+0000 7f655a7ffe40 20 add_watcher() i=6 2023-10-05T19:40:20.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.755+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94531858076704] v28'58 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6550014040 con 0x7f654802d020 2023-10-05T19:40:20.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.755+0000 7f655a7ffe40 20 add_watcher() i=7 2023-10-05T19:40:20.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.755+0000 7f655a7ffe40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:20.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.755+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d250f0 con 0x55f9e9ce03b0 2023-10-05T19:40:20.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.755+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f6550014040 con 0x55f9e9ce03b0 2023-10-05T19:40:20.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.755+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94531858077808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d22560 con 0x55f9e9ce03b0 2023-10-05T19:40:20.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.759+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94531858077808] v28'72 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f6550014040 con 0x55f9e9ce03b0 2023-10-05T19:40:20.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.759+0000 7f6529fe3700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:20.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.759+0000 7f655a7ffe40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:20.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.759+0000 7f655a7ffe40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:20.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.759+0000 7f655a7ffe40 1 Processor -- start 2023-10-05T19:40:20.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.759+0000 7f655a7ffe40 1 -- start start 2023-10-05T19:40:20.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.759+0000 7f655a7ffe40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f9e9d2f6d0 0x55f9e9d2faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.759+0000 7f655a7ffe40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9e9d2ffe0 con 0x55f9e9d2f6d0 2023-10-05T19:40:20.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.759+0000 7f6557b33700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f9e9d2f6d0 0x55f9e9d2faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.759+0000 7f6557b33700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f9e9d2f6d0 0x55f9e9d2faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37154/0 (socket says 172.21.15.12:37154) 2023-10-05T19:40:20.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.759+0000 7f6557b33700 1 -- 172.21.15.12:0/2041757896 learned_addr learned my addr 172.21.15.12:0/2041757896 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:20.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.759+0000 7f6557b33700 1 -- 172.21.15.12:0/2041757896 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9e9d30120 con 0x55f9e9d2f6d0 2023-10-05T19:40:20.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.759+0000 7f6557b33700 1 --2- 172.21.15.12:0/2041757896 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f9e9d2f6d0 0x55f9e9d2faa0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x55f9e9cdce50 tx=0x7f6540011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=939a09807d65e397 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.759+0000 7f65297e2700 1 -- 172.21.15.12:0/2041757896 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f654001ccf0 con 0x55f9e9d2f6d0 2023-10-05T19:40:20.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.759+0000 7f65297e2700 1 -- 172.21.15.12:0/2041757896 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f654001ce50 con 0x55f9e9d2f6d0 2023-10-05T19:40:20.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.759+0000 7f65297e2700 1 -- 172.21.15.12:0/2041757896 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6540029420 con 0x55f9e9d2f6d0 2023-10-05T19:40:20.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.763+0000 7f655a7ffe40 1 -- 172.21.15.12:0/2041757896 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f9e9d2f6d0 msgr2=0x55f9e9d2faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.763+0000 7f655a7ffe40 1 --2- 172.21.15.12:0/2041757896 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f9e9d2f6d0 0x55f9e9d2faa0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x55f9e9cdce50 tx=0x7f6540011000 comp rx=0 tx=0).stop 2023-10-05T19:40:20.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.763+0000 7f655a7ffe40 1 -- 172.21.15.12:0/2041757896 shutdown_connections 2023-10-05T19:40:20.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.763+0000 7f655a7ffe40 1 --2- 172.21.15.12:0/2041757896 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f9e9d2f6d0 0x55f9e9d2faa0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:20.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.763+0000 7f655a7ffe40 1 -- 172.21.15.12:0/2041757896 >> 172.21.15.12:0/2041757896 conn(0x55f9e9d2aa90 msgr2=0x55f9e9d28be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:20.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.763+0000 7f655a7ffe40 1 -- 172.21.15.12:0/2041757896 shutdown_connections 2023-10-05T19:40:20.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.763+0000 7f655a7ffe40 1 -- 172.21.15.12:0/2041757896 wait complete. 2023-10-05T19:40:20.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.763+0000 7f655a7ffe40 1 Processor -- start 2023-10-05T19:40:20.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.763+0000 7f655a7ffe40 1 -- start start 2023-10-05T19:40:20.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.763+0000 7f655a7ffe40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f9e9d2f6d0 0x55f9e9d35220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.763+0000 7f655a7ffe40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9e9d36220 con 0x55f9e9d2f6d0 2023-10-05T19:40:20.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.763+0000 7f6557b33700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f9e9d2f6d0 0x55f9e9d35220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.763+0000 7f6557b33700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f9e9d2f6d0 0x55f9e9d35220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37162/0 (socket says 172.21.15.12:37162) 2023-10-05T19:40:20.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.763+0000 7f6557b33700 1 -- 172.21.15.12:0/3258276684 learned_addr learned my addr 172.21.15.12:0/3258276684 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:20.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.763+0000 7f6557b33700 1 -- 172.21.15.12:0/3258276684 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f654002e040 con 0x55f9e9d2f6d0 2023-10-05T19:40:20.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.763+0000 7f6557b33700 1 --2- 172.21.15.12:0/3258276684 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f9e9d2f6d0 0x55f9e9d35220 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6540047040 tx=0x7f6540042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.763+0000 7f64a37fe700 1 -- 172.21.15.12:0/3258276684 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f654001cc70 con 0x55f9e9d2f6d0 2023-10-05T19:40:20.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.763+0000 7f64a37fe700 1 -- 172.21.15.12:0/3258276684 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6540029630 con 0x55f9e9d2f6d0 2023-10-05T19:40:20.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.763+0000 7f655a7ffe40 1 -- 172.21.15.12:0/3258276684 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f9e9d35820 con 0x55f9e9d2f6d0 2023-10-05T19:40:20.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.763+0000 7f64a37fe700 1 -- 172.21.15.12:0/3258276684 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65400297a0 con 0x55f9e9d2f6d0 2023-10-05T19:40:20.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.763+0000 7f655a7ffe40 1 -- 172.21.15.12:0/3258276684 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f9e9d35cc0 con 0x55f9e9d2f6d0 2023-10-05T19:40:20.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.767+0000 7f64a37fe700 1 -- 172.21.15.12:0/3258276684 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f654004b040 con 0x55f9e9d2f6d0 2023-10-05T19:40:20.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.767+0000 7f64a37fe700 1 --2- 172.21.15.12:0/3258276684 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f6498030c00 0x7f64980330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.767+0000 7f64a37fe700 1 -- 172.21.15.12:0/3258276684 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f6540058b00 con 0x55f9e9d2f6d0 2023-10-05T19:40:20.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.767+0000 7f6558b35700 1 --2- 172.21.15.12:0/3258276684 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f6498030c00 0x7f64980330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.767+0000 7f655a7ffe40 5 note: GC not initialized 2023-10-05T19:40:20.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.767+0000 7f6558b35700 1 --2- 172.21.15.12:0/3258276684 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f6498030c00 0x7f64980330b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x55f9e9d20da0 tx=0x7f655000a9b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.767+0000 7f64a0ff9700 20 reqs_thread_entry: start 2023-10-05T19:40:20.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.771+0000 7f655a7ffe40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:20.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.771+0000 7f64967fc700 20 reqs_thread_entry: start 2023-10-05T19:40:20.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.771+0000 7f655a7ffe40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:20.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.771+0000 7f6494ff9700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:51 2023 2023-10-05T19:40:20.779 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:20.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.771+0000 7f655a7ffe40 20 get_system_obj_state: rctx=0x7ffe01520f38 obj=default.rgw.meta:users.uid:foo state=0x55f9e9bb50b0 s->prefetch_data=0 2023-10-05T19:40:20.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.771+0000 7f655a7ffe40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2023-10-05T19:40:20.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.771+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d9dad0 con 0x55f9e9ce03b0 2023-10-05T19:40:20.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.771+0000 7f647bfff700 20 reqs_thread_entry: start 2023-10-05T19:40:20.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.771+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(102 foo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7f6550014040 con 0x55f9e9ce03b0 2023-10-05T19:40:20.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.771+0000 7f655a7ffe40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x16 2023-10-05T19:40:20.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.771+0000 7f655a7ffe40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:20.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.771+0000 7f655a7ffe40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:20.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.771+0000 7f655a7ffe40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:20.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.771+0000 7f655a7ffe40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:20.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.771+0000 7f655a7ffe40 10 cache get: name=default.rgw.meta+users.uid+foo : type miss (requested=0x13, cached=0x16) 2023-10-05T19:40:20.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.771+0000 7f655a7ffe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.771+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9da0560 con 0x55f9e9ce03b0 2023-10-05T19:40:20.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.771+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(103 foo [call,call out=48b,read 0~513 out=513b,getxattrs out=75b] v0'0 uv6 ondisk = 0) v8 ==== 273+0+636 (crc 0 0 0) 0x7f6550014040 con 0x55f9e9ce03b0 2023-10-05T19:40:20.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.771+0000 7f655a7ffe40 20 rados_obj.operate() r=0 bl.length=513 2023-10-05T19:40:20.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.771+0000 7f655a7ffe40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2023-10-05T19:40:20.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.771+0000 7f655a7ffe40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:20.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.771+0000 7f655a7ffe40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:20.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.771+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:104 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.set in=58b,writefull 0~513 in=513b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d9cb60 con 0x55f9e9ce03b0 2023-10-05T19:40:20.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.775+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(104 foo [delete,create,call,writefull 0~513] v28'7 uv7 ondisk = 0) v8 ==== 273+0+0 (crc 0 0 0) 0x7f6550014040 con 0x55f9e9ce03b0 2023-10-05T19:40:20.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.775+0000 7f655a7ffe40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2023-10-05T19:40:20.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.775+0000 7f655a7ffe40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:20.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.775+0000 7f655a7ffe40 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2023-10-05T19:40:20.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.775+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:105 7.d 7:b4812045:::notify.4:head [notify cookie 94531858581104 in=701b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d9cb60 con 0x55f9e9ce03b0 2023-10-05T19:40:20.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.775+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== watch-notify(notify (1) cookie 94531858069840 notify 120259084294 ret 0) v3 ==== 731+0+0 (crc 0 0 0) 0x7f65500330d0 con 0x55f9e9ce03b0 2023-10-05T19:40:20.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.775+0000 7f6556330700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084294 cookie 94531858069840 notifier 4339 bl.length()=689 2023-10-05T19:40:20.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.775+0000 7f6556330700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2023-10-05T19:40:20.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.775+0000 7f6556330700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:20.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.775+0000 7f6556330700 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:106 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f653c003d90 con 0x55f9e9ce03b0 2023-10-05T19:40:20.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.775+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(105 notify.4 [notify cookie 94531858581104 out=8b] v0'0 uv57 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f6550014040 con 0x55f9e9ce03b0 2023-10-05T19:40:20.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.775+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== watch-notify(notify_complete (2) cookie 94531858581104 notify 120259084294 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f6550033260 con 0x55f9e9ce03b0 2023-10-05T19:40:20.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.775+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.1 v2:172.21.15.12:6808/29933 39 ==== osd_op_reply(106 notify.4 [notify-ack] v0'0 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6550014040 con 0x55f9e9ce03b0 2023-10-05T19:40:20.786 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:40:20.786 INFO:teuthology.orchestra.run.smithi012.stdout: "user_id": "foo", 2023-10-05T19:40:20.786 INFO:teuthology.orchestra.run.smithi012.stdout: "display_name": "Foo", 2023-10-05T19:40:20.786 INFO:teuthology.orchestra.run.smithi012.stdout: "email": "foo@foo.com", 2023-10-05T19:40:20.786 INFO:teuthology.orchestra.run.smithi012.stdout: "suspended": 0, 2023-10-05T19:40:20.786 INFO:teuthology.orchestra.run.smithi012.stdout: "max_buckets": 4, 2023-10-05T19:40:20.786 INFO:teuthology.orchestra.run.smithi012.stdout: "subusers": [ 2023-10-05T19:40:20.787 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:20.787 INFO:teuthology.orchestra.run.smithi012.stdout: "id": "foo:foo1", 2023-10-05T19:40:20.787 INFO:teuthology.orchestra.run.smithi012.stdout: "permissions": "full-control" 2023-10-05T19:40:20.787 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:20.787 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:20.787 INFO:teuthology.orchestra.run.smithi012.stdout: "keys": [ 2023-10-05T19:40:20.787 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:20.787 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:40:20.787 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2023-10-05T19:40:20.787 INFO:teuthology.orchestra.run.smithi012.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2023-10-05T19:40:20.787 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:20.787 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:20.788 INFO:teuthology.orchestra.run.smithi012.stdout: "swift_keys": [ 2023-10-05T19:40:20.788 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:20.788 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo:foo1", 2023-10-05T19:40:20.788 INFO:teuthology.orchestra.run.smithi012.stdout: "secret_key": "gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL" 2023-10-05T19:40:20.788 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:20.788 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:20.788 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": [], 2023-10-05T19:40:20.788 INFO:teuthology.orchestra.run.smithi012.stdout: "op_mask": "read, write, delete", 2023-10-05T19:40:20.788 INFO:teuthology.orchestra.run.smithi012.stdout: "default_placement": "", 2023-10-05T19:40:20.788 INFO:teuthology.orchestra.run.smithi012.stdout: "default_storage_class": "", 2023-10-05T19:40:20.788 INFO:teuthology.orchestra.run.smithi012.stdout: "placement_tags": [], 2023-10-05T19:40:20.788 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket_quota": { 2023-10-05T19:40:20.789 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:20.789 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:20.789 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:20.789 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:20.789 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:20.789 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:20.789 INFO:teuthology.orchestra.run.smithi012.stdout: "user_quota": { 2023-10-05T19:40:20.789 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:20.789 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:20.789 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:20.789 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:20.789 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:20.790 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:20.790 INFO:teuthology.orchestra.run.smithi012.stdout: "temp_url_keys": [], 2023-10-05T19:40:20.790 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "rgw", 2023-10-05T19:40:20.790 INFO:teuthology.orchestra.run.smithi012.stdout: "mfa_ids": [] 2023-10-05T19:40:20.790 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:40:20.790 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:40:20.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.779+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:107 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94531858049472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9cf1780 con 0x55f9e9d11f80 2023-10-05T19:40:20.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.779+0000 7f6558334700 1 -- 172.21.15.12:0/1573777961 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(107 notify.0 [watch unwatch cookie 94531858049472] v28'59 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f654804d080 con 0x55f9e9d11f80 2023-10-05T19:40:20.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.779+0000 7f655a7ffe40 20 remove_watcher() i=0 2023-10-05T19:40:20.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.779+0000 7f655a7ffe40 2 removed watcher, disabling cache 2023-10-05T19:40:20.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.779+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:108 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94531858057232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9cf1780 con 0x55f9e9ce03b0 2023-10-05T19:40:20.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.783+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.1 v2:172.21.15.12:6808/29933 40 ==== osd_op_reply(108 notify.1 [watch unwatch cookie 94531858057232] v28'59 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6550014040 con 0x55f9e9ce03b0 2023-10-05T19:40:20.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.783+0000 7f655a7ffe40 20 remove_watcher() i=1 2023-10-05T19:40:20.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.783+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:109 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94531858063840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9cf1780 con 0x55f9e9d16720 2023-10-05T19:40:20.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.783+0000 7f6557b33700 1 -- 172.21.15.12:0/1573777961 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(109 notify.2 [watch unwatch cookie 94531858063840] v28'59 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f654000fcc0 con 0x55f9e9d16720 2023-10-05T19:40:20.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.783+0000 7f655a7ffe40 20 remove_watcher() i=2 2023-10-05T19:40:20.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.783+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:110 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94531858067072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9cf1780 con 0x55f9e9d11f80 2023-10-05T19:40:20.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.787+0000 7f6558334700 1 -- 172.21.15.12:0/1573777961 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(110 notify.3 [watch unwatch cookie 94531858067072] v28'59 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f654804d080 con 0x55f9e9d11f80 2023-10-05T19:40:20.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.787+0000 7f655a7ffe40 20 remove_watcher() i=3 2023-10-05T19:40:20.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.787+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:111 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94531858069840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9cf1780 con 0x55f9e9ce03b0 2023-10-05T19:40:20.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.791+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.1 v2:172.21.15.12:6808/29933 41 ==== osd_op_reply(111 notify.4 [watch unwatch cookie 94531858069840] v28'59 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6550014040 con 0x55f9e9ce03b0 2023-10-05T19:40:20.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.791+0000 7f655a7ffe40 20 remove_watcher() i=4 2023-10-05T19:40:20.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.791+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:112 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94531858064944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9cf1780 con 0x7f654802d020 2023-10-05T19:40:20.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.791+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(112 notify.5 [watch unwatch cookie 94531858064944] v28'59 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6550014040 con 0x7f654802d020 2023-10-05T19:40:20.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.791+0000 7f655a7ffe40 20 remove_watcher() i=5 2023-10-05T19:40:20.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.791+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:113 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94531858071280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9d1e6d0 con 0x7f654802d020 2023-10-05T19:40:20.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.795+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(113 notify.6 [watch unwatch cookie 94531858071280] v28'59 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6550014040 con 0x7f654802d020 2023-10-05T19:40:20.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.795+0000 7f655a7ffe40 20 remove_watcher() i=6 2023-10-05T19:40:20.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.795+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:114 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94531858076704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9cf1780 con 0x7f654802d020 2023-10-05T19:40:20.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.799+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(114 notify.7 [watch unwatch cookie 94531858076704] v28'59 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6550014040 con 0x7f654802d020 2023-10-05T19:40:20.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.799+0000 7f655a7ffe40 20 remove_watcher() i=7 2023-10-05T19:40:20.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.799+0000 7f6529fe3700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:20.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.799+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:115 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94531858077808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f9e9cf1780 con 0x55f9e9ce03b0 2023-10-05T19:40:20.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.803+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 <== osd.1 v2:172.21.15.12:6808/29933 42 ==== osd_op_reply(115 data_loggenerations_metadata [watch unwatch cookie 94531858077808] v28'73 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f6550014040 con 0x55f9e9ce03b0 2023-10-05T19:40:20.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.803+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f654802d020 msgr2=0x7f654803d920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.803+0000 7f655a7ffe40 1 --2- 172.21.15.12:0/1573777961 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f654802d020 0x7f654803d920 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:20.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.803+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55f9e9ce03b0 msgr2=0x55f9e9d0fa50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.803+0000 7f655a7ffe40 1 --2- 172.21.15.12:0/1573777961 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55f9e9ce03b0 0x55f9e9d0fa50 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:20.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.803+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55f9e9d11f80 msgr2=0x55f9e9d14360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.803+0000 7f655a7ffe40 1 --2- 172.21.15.12:0/1573777961 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55f9e9d11f80 0x55f9e9d14360 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:20.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.803+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55f9e9d16720 msgr2=0x55f9e9d18b30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.803+0000 7f655a7ffe40 1 --2- 172.21.15.12:0/1573777961 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55f9e9d16720 0x55f9e9d18b30 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:20.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.803+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f6538030c00 msgr2=0x7f65380330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.803+0000 7f655a7ffe40 1 --2- 172.21.15.12:0/1573777961 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f6538030c00 0x7f65380330b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6540005d80 tx=0x7f6540014000 comp rx=0 tx=0).stop 2023-10-05T19:40:20.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.803+0000 7f6558b35700 1 -- 172.21.15.12:0/1573777961 reap_dead start 2023-10-05T19:40:20.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.803+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f9e9cdbdd0 msgr2=0x55f9e9c51a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.803+0000 7f655a7ffe40 1 --2- 172.21.15.12:0/1573777961 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f9e9cdbdd0 0x55f9e9c51a90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55f9e9cf7600 tx=0x7f6548016000 comp rx=0 tx=0).stop 2023-10-05T19:40:20.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.803+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 shutdown_connections 2023-10-05T19:40:20.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.803+0000 7f655a7ffe40 1 --2- 172.21.15.12:0/1573777961 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f9e9cdbdd0 0x55f9e9c51a90 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:20.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.803+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 >> 172.21.15.12:0/1573777961 conn(0x55f9e9cd77f0 msgr2=0x55f9e9cf5560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:20.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.803+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 shutdown_connections 2023-10-05T19:40:20.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.803+0000 7f655a7ffe40 1 -- 172.21.15.12:0/1573777961 wait complete. 2023-10-05T19:40:20.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.803+0000 7f655a7ffe40 1 -- 172.21.15.12:0/3258276684 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f6498030c00 msgr2=0x7f64980330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.803+0000 7f655a7ffe40 1 --2- 172.21.15.12:0/3258276684 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f6498030c00 0x7f64980330b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x55f9e9d20da0 tx=0x7f655000a9b0 comp rx=0 tx=0).stop 2023-10-05T19:40:20.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.803+0000 7f655a7ffe40 1 -- 172.21.15.12:0/3258276684 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f9e9d2f6d0 msgr2=0x55f9e9d35220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.803+0000 7f655a7ffe40 1 --2- 172.21.15.12:0/3258276684 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f9e9d2f6d0 0x55f9e9d35220 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6540047040 tx=0x7f6540042000 comp rx=0 tx=0).stop 2023-10-05T19:40:20.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.803+0000 7f655a7ffe40 1 -- 172.21.15.12:0/3258276684 shutdown_connections 2023-10-05T19:40:20.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.803+0000 7f655a7ffe40 1 --2- 172.21.15.12:0/3258276684 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f6498030c00 0x7f64980330b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:20.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.803+0000 7f655a7ffe40 1 --2- 172.21.15.12:0/3258276684 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f9e9d2f6d0 0x55f9e9d35220 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:20.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.803+0000 7f655a7ffe40 1 -- 172.21.15.12:0/3258276684 >> 172.21.15.12:0/3258276684 conn(0x55f9e9d2aa90 msgr2=0x55f9e9d28b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:20.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.803+0000 7f655a7ffe40 1 -- 172.21.15.12:0/3258276684 shutdown_connections 2023-10-05T19:40:20.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.803+0000 7f655a7ffe40 1 -- 172.21.15.12:0/3258276684 wait complete. 2023-10-05T19:40:20.819 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:40:20.819 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2023-10-05T19:40:20.819 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2023-10-05T19:40:20.819 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2023-10-05T19:40:20.855 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:20.855 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:20.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.847+0000 7f935a1f1e40 1 Processor -- start 2023-10-05T19:40:20.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.847+0000 7f935a1f1e40 1 -- start start 2023-10-05T19:40:20.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.847+0000 7f935a1f1e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x563ebfaa1010 0x563ebfaa13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.847+0000 7f935a1f1e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x563ebf9ebbd0 con 0x563ebfaa1920 2023-10-05T19:40:20.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.847+0000 7f935a1f1e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x563ebf9a8ee0 con 0x563ebfaa1010 2023-10-05T19:40:20.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.847+0000 7f9357d26700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x563ebfaa1010 0x563ebfaa13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.847+0000 7f9357525700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x563ebfaa1920 0x563ebfaa23c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:47312/0 (socket says 172.21.15.12:47312) 2023-10-05T19:40:20.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.847+0000 7f9357525700 1 -- 172.21.15.12:0/2753618270 learned_addr learned my addr 172.21.15.12:0/2753618270 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:20.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.851+0000 7f9356d24700 1 -- 172.21.15.12:0/2753618270 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3930321384 0 0) 0x563ebf9ebbd0 con 0x563ebfaa1920 2023-10-05T19:40:20.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.851+0000 7f9356d24700 1 -- 172.21.15.12:0/2753618270 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9344001e10 con 0x563ebfaa1920 2023-10-05T19:40:20.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.851+0000 7f9357d26700 1 -- 172.21.15.12:0/2753618270 >> v1:172.21.15.8:6789/0 conn(0x563ebfaa1920 legacy=0x563ebfaa23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.851+0000 7f9357d26700 1 -- 172.21.15.12:0/2753618270 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ebfaa2ad0 con 0x563ebfaa1010 2023-10-05T19:40:20.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.851+0000 7f9357d26700 1 --2- 172.21.15.12:0/2753618270 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ebfaa1010 0x563ebfaa13e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f935000a520 tx=0x7f9350005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93347ee5ea1ec306 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.851+0000 7f9356d24700 1 -- 172.21.15.12:0/2753618270 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f935000f590 con 0x563ebfaa1010 2023-10-05T19:40:20.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.851+0000 7f9356d24700 1 -- 172.21.15.12:0/2753618270 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f935000f6f0 con 0x563ebfaa1010 2023-10-05T19:40:20.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.851+0000 7f9356d24700 1 -- 172.21.15.12:0/2753618270 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f935000f860 con 0x563ebfaa1010 2023-10-05T19:40:20.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.851+0000 7f935a1f1e40 1 -- 172.21.15.12:0/2753618270 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ebfaa1010 msgr2=0x563ebfaa13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.851+0000 7f935a1f1e40 1 --2- 172.21.15.12:0/2753618270 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ebfaa1010 0x563ebfaa13e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f935000a520 tx=0x7f9350005d40 comp rx=0 tx=0).stop 2023-10-05T19:40:20.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.851+0000 7f935a1f1e40 1 -- 172.21.15.12:0/2753618270 shutdown_connections 2023-10-05T19:40:20.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.851+0000 7f935a1f1e40 1 --2- 172.21.15.12:0/2753618270 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ebfaa1010 0x563ebfaa13e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:20.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.851+0000 7f935a1f1e40 1 -- 172.21.15.12:0/2753618270 >> 172.21.15.12:0/2753618270 conn(0x563ebfa14860 msgr2=0x563ebfa14c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:20.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.851+0000 7f935a1f1e40 1 -- 172.21.15.12:0/2753618270 shutdown_connections 2023-10-05T19:40:20.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.851+0000 7f935a1f1e40 1 -- 172.21.15.12:0/2753618270 wait complete. 2023-10-05T19:40:20.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.851+0000 7f935a1f1e40 1 Processor -- start 2023-10-05T19:40:20.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.851+0000 7f935a1f1e40 1 -- start start 2023-10-05T19:40:20.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.851+0000 7f935a1f1e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ebfaa1010 0x563ebfabfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.851+0000 7f935a1f1e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ebf96ce30 con 0x563ebfaa1010 2023-10-05T19:40:20.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.851+0000 7f9357d26700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ebfaa1010 0x563ebfabfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.851+0000 7f9357d26700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ebfaa1010 0x563ebfabfea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37174/0 (socket says 172.21.15.12:37174) 2023-10-05T19:40:20.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.851+0000 7f9357d26700 1 -- 172.21.15.12:0/2181402821 learned_addr learned my addr 172.21.15.12:0/2181402821 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:20.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.855+0000 7f9357d26700 1 -- 172.21.15.12:0/2181402821 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f935000a0e0 con 0x563ebfaa1010 2023-10-05T19:40:20.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.855+0000 7f9357d26700 1 --2- 172.21.15.12:0/2181402821 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ebfaa1010 0x563ebfabfea0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f935000a470 tx=0x7f9350014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=425a4dd963f1d662 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.855+0000 7f9355d22700 1 -- 172.21.15.12:0/2181402821 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f935000f860 con 0x563ebfaa1010 2023-10-05T19:40:20.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.855+0000 7f9355d22700 1 -- 172.21.15.12:0/2181402821 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f935000f9c0 con 0x563ebfaa1010 2023-10-05T19:40:20.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.855+0000 7f9355d22700 1 -- 172.21.15.12:0/2181402821 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f935000fb90 con 0x563ebfaa1010 2023-10-05T19:40:20.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.855+0000 7f935a1f1e40 1 -- 172.21.15.12:0/2181402821 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ebfaa1010 msgr2=0x563ebfabfea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.855+0000 7f935a1f1e40 1 --2- 172.21.15.12:0/2181402821 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ebfaa1010 0x563ebfabfea0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f935000a470 tx=0x7f9350014000 comp rx=0 tx=0).stop 2023-10-05T19:40:20.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.855+0000 7f935a1f1e40 1 -- 172.21.15.12:0/2181402821 shutdown_connections 2023-10-05T19:40:20.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.855+0000 7f935a1f1e40 1 --2- 172.21.15.12:0/2181402821 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ebfaa1010 0x563ebfabfea0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:20.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.855+0000 7f935a1f1e40 1 -- 172.21.15.12:0/2181402821 >> 172.21.15.12:0/2181402821 conn(0x563ebfa14860 msgr2=0x563ebfabba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:20.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.855+0000 7f935a1f1e40 1 -- 172.21.15.12:0/2181402821 shutdown_connections 2023-10-05T19:40:20.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.855+0000 7f935a1f1e40 1 -- 172.21.15.12:0/2181402821 wait complete. 2023-10-05T19:40:20.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.855+0000 7f935a1f1e40 1 Processor -- start 2023-10-05T19:40:20.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.855+0000 7f935a1f1e40 1 -- start start 2023-10-05T19:40:20.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.855+0000 7f935a1f1e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ebfaa1010 0x563ebfac6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.855+0000 7f935a1f1e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ebf93fed0 con 0x563ebfaa1010 2023-10-05T19:40:20.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.855+0000 7f9357d26700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ebfaa1010 0x563ebfac6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.855+0000 7f9357d26700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ebfaa1010 0x563ebfac6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37180/0 (socket says 172.21.15.12:37180) 2023-10-05T19:40:20.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.855+0000 7f9357d26700 1 -- 172.21.15.12:0/760245333 learned_addr learned my addr 172.21.15.12:0/760245333 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:20.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.855+0000 7f9357d26700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ebfaa2ad0 con 0x563ebfaa1010 2023-10-05T19:40:20.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.855+0000 7f9357d26700 1 --2- 172.21.15.12:0/760245333 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ebfaa1010 0x563ebfac6b80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f935000adc0 tx=0x7f935001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.859+0000 7f934ffff700 1 -- 172.21.15.12:0/760245333 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f935002f430 con 0x563ebfaa1010 2023-10-05T19:40:20.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.859+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563ebfac70c0 con 0x563ebfaa1010 2023-10-05T19:40:20.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.859+0000 7f934ffff700 1 -- 172.21.15.12:0/760245333 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f935002f590 con 0x563ebfaa1010 2023-10-05T19:40:20.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.859+0000 7f934ffff700 1 -- 172.21.15.12:0/760245333 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f935002f740 con 0x563ebfaa1010 2023-10-05T19:40:20.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.859+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ebfac7560 con 0x563ebfaa1010 2023-10-05T19:40:20.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.859+0000 7f935a1f1e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.859+0000 7f934ffff700 1 -- 172.21.15.12:0/760245333 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f9350010420 con 0x563ebfaa1010 2023-10-05T19:40:20.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.859+0000 7f934ffff700 1 --2- 172.21.15.12:0/760245333 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f9338030c00 0x7f93380330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.859+0000 7f934ffff700 1 -- 172.21.15.12:0/760245333 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f935002b070 con 0x563ebfaa1010 2023-10-05T19:40:20.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.859+0000 7f935a1f1e40 1 --2- 172.21.15.12:0/760245333 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x563ebfaa1920 0x563ebfad2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.859+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfad2d60 con 0x563ebfaa1920 2023-10-05T19:40:20.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.859+0000 7f9357525700 1 --2- 172.21.15.12:0/760245333 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f9338030c00 0x7f93380330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.859+0000 7f9358527700 1 --2- 172.21.15.12:0/760245333 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x563ebfaa1920 0x563ebfad2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.863+0000 7f9358527700 1 --2- 172.21.15.12:0/760245333 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x563ebfaa1920 0x563ebfad2740 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.863+0000 7f9357525700 1 --2- 172.21.15.12:0/760245333 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f9338030c00 0x7f93380330b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9340004170 tx=0x7f9340001400 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.863+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f934800a500 con 0x563ebfaa1920 2023-10-05T19:40:20.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.863+0000 7f935a1f1e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.863+0000 7f935a1f1e40 20 realm 2023-10-05T19:40:20.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.863+0000 7f935a1f1e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.863+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfacfda0 con 0x563ebfaa1920 2023-10-05T19:40:20.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.863+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f934800a500 con 0x563ebfaa1920 2023-10-05T19:40:20.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.863+0000 7f935a1f1e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.863+0000 7f935a1f1e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:20.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.863+0000 7f935a1f1e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.863+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfad3840 con 0x563ebfaa1920 2023-10-05T19:40:20.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.863+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f934800a500 con 0x563ebfaa1920 2023-10-05T19:40:20.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.863+0000 7f935a1f1e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.863+0000 7f935a1f1e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.863+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfad3420 con 0x563ebfaa1920 2023-10-05T19:40:20.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.863+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f934800a500 con 0x563ebfaa1920 2023-10-05T19:40:20.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.863+0000 7f935a1f1e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:20.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.863+0000 7f935a1f1e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.863+0000 7f935a1f1e40 1 --2- 172.21.15.12:0/760245333 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x563ebfad47d0 0x563ebfad6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.863+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfad7320 con 0x563ebfad47d0 2023-10-05T19:40:20.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.863+0000 7f9357d26700 1 --2- 172.21.15.12:0/760245333 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x563ebfad47d0 0x563ebfad6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.863+0000 7f9357d26700 1 --2- 172.21.15.12:0/760245333 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x563ebfad47d0 0x563ebfad6bb0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.867+0000 7f9357d26700 1 -- 172.21.15.12:0/760245333 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f9350024400 con 0x563ebfad47d0 2023-10-05T19:40:20.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.867+0000 7f935a1f1e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:20.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.867+0000 7f935a1f1e40 20 searching for the correct realm 2023-10-05T19:40:20.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.867+0000 7f935a1f1e40 1 --2- 172.21.15.12:0/760245333 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x563ebfad9150 0x563ebfadb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.867+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadba70 con 0x563ebfad9150 2023-10-05T19:40:20.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.867+0000 7f9357525700 1 --2- 172.21.15.12:0/760245333 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x563ebfad9150 0x563ebfadb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.867+0000 7f9357525700 1 --2- 172.21.15.12:0/760245333 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x563ebfad9150 0x563ebfadb530 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.871+0000 7f9357525700 1 -- 172.21.15.12:0/760245333 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f9340006070 con 0x563ebfad9150 2023-10-05T19:40:20.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.871+0000 7f9357525700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadba70 con 0x563ebfad47d0 2023-10-05T19:40:20.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.871+0000 7f9357d26700 1 -- 172.21.15.12:0/760245333 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9350024400 con 0x563ebfad47d0 2023-10-05T19:40:20.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.871+0000 7f9357d26700 1 --2- 172.21.15.12:0/760245333 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f935001abe0 0x7f93500383b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.871+0000 7f9357d26700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadba70 con 0x7f935001abe0 2023-10-05T19:40:20.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.871+0000 7f9358527700 1 --2- 172.21.15.12:0/760245333 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f935001abe0 0x7f93500383b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.871+0000 7f9358527700 1 --2- 172.21.15.12:0/760245333 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f935001abe0 0x7f93500383b0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.871+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9348014040 con 0x7f935001abe0 2023-10-05T19:40:20.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.871+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadba70 con 0x563ebfaa1920 2023-10-05T19:40:20.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.871+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9348014040 con 0x563ebfaa1920 2023-10-05T19:40:20.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.871+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadba70 con 0x563ebfaa1920 2023-10-05T19:40:20.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.875+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9348014040 con 0x563ebfaa1920 2023-10-05T19:40:20.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.875+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadba70 con 0x7f935001abe0 2023-10-05T19:40:20.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.875+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9348014040 con 0x7f935001abe0 2023-10-05T19:40:20.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.875+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadba70 con 0x7f935001abe0 2023-10-05T19:40:20.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.875+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9348014040 con 0x7f935001abe0 2023-10-05T19:40:20.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.875+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadba70 con 0x563ebfad47d0 2023-10-05T19:40:20.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.875+0000 7f9357d26700 1 -- 172.21.15.12:0/760245333 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9350024400 con 0x563ebfad47d0 2023-10-05T19:40:20.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.875+0000 7f9357d26700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadba70 con 0x563ebfad9150 2023-10-05T19:40:20.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.875+0000 7f9357525700 1 -- 172.21.15.12:0/760245333 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9340006070 con 0x563ebfad9150 2023-10-05T19:40:20.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.875+0000 7f9357525700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadba70 con 0x563ebfaa1920 2023-10-05T19:40:20.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.879+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9348014040 con 0x563ebfaa1920 2023-10-05T19:40:20.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.879+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadba70 con 0x7f935001abe0 2023-10-05T19:40:20.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.879+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9348014040 con 0x7f935001abe0 2023-10-05T19:40:20.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.879+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadba70 con 0x563ebfaa1920 2023-10-05T19:40:20.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.879+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9348014040 con 0x563ebfaa1920 2023-10-05T19:40:20.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.879+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadba70 con 0x563ebfad47d0 2023-10-05T19:40:20.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.879+0000 7f9357d26700 1 -- 172.21.15.12:0/760245333 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9350024400 con 0x563ebfad47d0 2023-10-05T19:40:20.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.879+0000 7f9357d26700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadba70 con 0x563ebfad9150 2023-10-05T19:40:20.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.879+0000 7f9357525700 1 -- 172.21.15.12:0/760245333 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9340006070 con 0x563ebfad9150 2023-10-05T19:40:20.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.879+0000 7f9357525700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadba70 con 0x563ebfad9150 2023-10-05T19:40:20.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.883+0000 7f9357525700 1 -- 172.21.15.12:0/760245333 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9340006070 con 0x563ebfad9150 2023-10-05T19:40:20.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.883+0000 7f9357525700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadba70 con 0x7f935001abe0 2023-10-05T19:40:20.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.883+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9348014040 con 0x7f935001abe0 2023-10-05T19:40:20.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.883+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadba70 con 0x563ebfad9150 2023-10-05T19:40:20.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.883+0000 7f9357525700 1 -- 172.21.15.12:0/760245333 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9340006070 con 0x563ebfad9150 2023-10-05T19:40:20.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.883+0000 7f9357525700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadba70 con 0x563ebfad9150 2023-10-05T19:40:20.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.883+0000 7f9357525700 1 -- 172.21.15.12:0/760245333 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9340006070 con 0x563ebfad9150 2023-10-05T19:40:20.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.883+0000 7f9357525700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadba70 con 0x563ebfaa1920 2023-10-05T19:40:20.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.883+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9348014040 con 0x563ebfaa1920 2023-10-05T19:40:20.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.883+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadba70 con 0x563ebfaa1920 2023-10-05T19:40:20.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.887+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9348014040 con 0x563ebfaa1920 2023-10-05T19:40:20.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.887+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadba70 con 0x7f935001abe0 2023-10-05T19:40:20.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.887+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9348014040 con 0x7f935001abe0 2023-10-05T19:40:20.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.887+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadba70 con 0x563ebfad9150 2023-10-05T19:40:20.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.887+0000 7f9357525700 1 -- 172.21.15.12:0/760245333 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9340006070 con 0x563ebfad9150 2023-10-05T19:40:20.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.887+0000 7f9357525700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadba70 con 0x563ebfad47d0 2023-10-05T19:40:20.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.887+0000 7f9357d26700 1 -- 172.21.15.12:0/760245333 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9350024400 con 0x563ebfad47d0 2023-10-05T19:40:20.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.887+0000 7f9357d26700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadba70 con 0x563ebfaa1920 2023-10-05T19:40:20.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.887+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9348014040 con 0x563ebfaa1920 2023-10-05T19:40:20.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.887+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadba70 con 0x7f935001abe0 2023-10-05T19:40:20.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.891+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9348014040 con 0x7f935001abe0 2023-10-05T19:40:20.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.891+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadba70 con 0x563ebfad9150 2023-10-05T19:40:20.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.891+0000 7f9357525700 1 -- 172.21.15.12:0/760245333 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9340006070 con 0x563ebfad9150 2023-10-05T19:40:20.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.891+0000 7f9357525700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadba70 con 0x7f935001abe0 2023-10-05T19:40:20.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.891+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9348014040 con 0x7f935001abe0 2023-10-05T19:40:20.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.891+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadba70 con 0x563ebfad47d0 2023-10-05T19:40:20.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.891+0000 7f9357d26700 1 -- 172.21.15.12:0/760245333 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9350024400 con 0x563ebfad47d0 2023-10-05T19:40:20.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.891+0000 7f9357d26700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadba70 con 0x563ebfad9150 2023-10-05T19:40:20.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.895+0000 7f9357525700 1 -- 172.21.15.12:0/760245333 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9340006070 con 0x563ebfad9150 2023-10-05T19:40:20.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.895+0000 7f9357525700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadba70 con 0x7f935001abe0 2023-10-05T19:40:20.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.895+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9348014040 con 0x7f935001abe0 2023-10-05T19:40:20.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.895+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadba70 con 0x563ebfaa1920 2023-10-05T19:40:20.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.895+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f9348014040 con 0x563ebfaa1920 2023-10-05T19:40:20.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.895+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadba70 con 0x563ebfad47d0 2023-10-05T19:40:20.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.895+0000 7f9357d26700 1 -- 172.21.15.12:0/760245333 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f9350024400 con 0x563ebfad47d0 2023-10-05T19:40:20.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.895+0000 7f935a1f1e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:20.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.895+0000 7f935a1f1e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:20.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.895+0000 7f935a1f1e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:20.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.895+0000 7f935a1f1e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:20.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.895+0000 7f935a1f1e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.895+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfad7ea0 con 0x563ebfaa1920 2023-10-05T19:40:20.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.895+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9348014040 con 0x563ebfaa1920 2023-10-05T19:40:20.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.895+0000 7f935a1f1e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.895+0000 7f935a1f1e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.895+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfad7a70 con 0x563ebfaa1920 2023-10-05T19:40:20.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.899+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f9348014040 con 0x563ebfaa1920 2023-10-05T19:40:20.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.899+0000 7f935a1f1e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:20.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.899+0000 7f935a1f1e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.899+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfad87d0 con 0x563ebfad9150 2023-10-05T19:40:20.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.899+0000 7f9357525700 1 -- 172.21.15.12:0/760245333 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f9340006070 con 0x563ebfad9150 2023-10-05T19:40:20.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.899+0000 7f935a1f1e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:20.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.899+0000 7f935a1f1e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.899+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadc3a0 con 0x563ebfad9150 2023-10-05T19:40:20.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.899+0000 7f9357525700 1 -- 172.21.15.12:0/760245333 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f9340006070 con 0x563ebfad9150 2023-10-05T19:40:20.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.899+0000 7f935a1f1e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.899+0000 7f935a1f1e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.899+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadc750 con 0x563ebfaa1920 2023-10-05T19:40:20.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.899+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9348014040 con 0x563ebfaa1920 2023-10-05T19:40:20.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.899+0000 7f935a1f1e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.899+0000 7f935a1f1e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.899+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadcb00 con 0x563ebfaa1920 2023-10-05T19:40:20.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.899+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f9348014040 con 0x563ebfaa1920 2023-10-05T19:40:20.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.899+0000 7f935a1f1e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:20.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.899+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfaddb10 con 0x563ebfad9150 2023-10-05T19:40:20.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.903+0000 7f9357525700 1 -- 172.21.15.12:0/760245333 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f9340006070 con 0x563ebfad9150 2023-10-05T19:40:20.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.903+0000 7f9357525700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfaddb10 con 0x563ebfad47d0 2023-10-05T19:40:20.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.903+0000 7f9357d26700 1 -- 172.21.15.12:0/760245333 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9350024400 con 0x563ebfad47d0 2023-10-05T19:40:20.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.903+0000 7f9357d26700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfaddb10 con 0x7f935001abe0 2023-10-05T19:40:20.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.903+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9348014040 con 0x7f935001abe0 2023-10-05T19:40:20.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.907+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadc750 con 0x563ebfaa1920 2023-10-05T19:40:20.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.907+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9348014040 con 0x563ebfaa1920 2023-10-05T19:40:20.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.907+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadc750 con 0x563ebfaa1920 2023-10-05T19:40:20.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.907+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9348014040 con 0x563ebfaa1920 2023-10-05T19:40:20.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.907+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadc750 con 0x7f935001abe0 2023-10-05T19:40:20.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.907+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9348014040 con 0x7f935001abe0 2023-10-05T19:40:20.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.907+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadc750 con 0x7f935001abe0 2023-10-05T19:40:20.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.907+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9348014040 con 0x7f935001abe0 2023-10-05T19:40:20.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.907+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadc750 con 0x563ebfad47d0 2023-10-05T19:40:20.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.911+0000 7f9357d26700 1 -- 172.21.15.12:0/760245333 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9350024400 con 0x563ebfad47d0 2023-10-05T19:40:20.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.911+0000 7f9357d26700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadc750 con 0x563ebfad9150 2023-10-05T19:40:20.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.911+0000 7f9357525700 1 -- 172.21.15.12:0/760245333 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9340006070 con 0x563ebfad9150 2023-10-05T19:40:20.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.911+0000 7f9357525700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadc750 con 0x563ebfaa1920 2023-10-05T19:40:20.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.911+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9348014040 con 0x563ebfaa1920 2023-10-05T19:40:20.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.911+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadc750 con 0x7f935001abe0 2023-10-05T19:40:20.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.911+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9348014040 con 0x7f935001abe0 2023-10-05T19:40:20.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.911+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadc750 con 0x563ebfaa1920 2023-10-05T19:40:20.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.911+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9348014040 con 0x563ebfaa1920 2023-10-05T19:40:20.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.911+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadc750 con 0x563ebfad47d0 2023-10-05T19:40:20.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.915+0000 7f9357d26700 1 -- 172.21.15.12:0/760245333 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9350024400 con 0x563ebfad47d0 2023-10-05T19:40:20.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.915+0000 7f9357d26700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadc750 con 0x563ebfad9150 2023-10-05T19:40:20.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.915+0000 7f9357525700 1 -- 172.21.15.12:0/760245333 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9340006070 con 0x563ebfad9150 2023-10-05T19:40:20.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.915+0000 7f9357525700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadc750 con 0x563ebfad9150 2023-10-05T19:40:20.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.915+0000 7f9357525700 1 -- 172.21.15.12:0/760245333 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9340006070 con 0x563ebfad9150 2023-10-05T19:40:20.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.915+0000 7f9357525700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadc750 con 0x7f935001abe0 2023-10-05T19:40:20.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.915+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9348014040 con 0x7f935001abe0 2023-10-05T19:40:20.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.915+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadc750 con 0x563ebfad9150 2023-10-05T19:40:20.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.919+0000 7f9357525700 1 -- 172.21.15.12:0/760245333 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9340006070 con 0x563ebfad9150 2023-10-05T19:40:20.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.919+0000 7f9357525700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadc750 con 0x563ebfad9150 2023-10-05T19:40:20.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.919+0000 7f9357525700 1 -- 172.21.15.12:0/760245333 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9340006070 con 0x563ebfad9150 2023-10-05T19:40:20.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.919+0000 7f9357525700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadc750 con 0x563ebfaa1920 2023-10-05T19:40:20.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.919+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9348014040 con 0x563ebfaa1920 2023-10-05T19:40:20.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.919+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadc750 con 0x563ebfaa1920 2023-10-05T19:40:20.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.919+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9348014040 con 0x563ebfaa1920 2023-10-05T19:40:20.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.919+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadc750 con 0x7f935001abe0 2023-10-05T19:40:20.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.919+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9348014040 con 0x7f935001abe0 2023-10-05T19:40:20.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.919+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadc750 con 0x563ebfad9150 2023-10-05T19:40:20.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.923+0000 7f9357525700 1 -- 172.21.15.12:0/760245333 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9340006070 con 0x563ebfad9150 2023-10-05T19:40:20.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.923+0000 7f9357525700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadc750 con 0x563ebfad47d0 2023-10-05T19:40:20.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.923+0000 7f9357d26700 1 -- 172.21.15.12:0/760245333 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9350024400 con 0x563ebfad47d0 2023-10-05T19:40:20.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.923+0000 7f9357d26700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadc750 con 0x563ebfaa1920 2023-10-05T19:40:20.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.923+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9348014040 con 0x563ebfaa1920 2023-10-05T19:40:20.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.923+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadc750 con 0x7f935001abe0 2023-10-05T19:40:20.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.923+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9348014040 con 0x7f935001abe0 2023-10-05T19:40:20.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.923+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadc750 con 0x563ebfad9150 2023-10-05T19:40:20.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.923+0000 7f9357525700 1 -- 172.21.15.12:0/760245333 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9340006070 con 0x563ebfad9150 2023-10-05T19:40:20.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.923+0000 7f9357525700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadc750 con 0x7f935001abe0 2023-10-05T19:40:20.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.927+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9348014040 con 0x7f935001abe0 2023-10-05T19:40:20.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.927+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadc750 con 0x563ebfad47d0 2023-10-05T19:40:20.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.927+0000 7f9357d26700 1 -- 172.21.15.12:0/760245333 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9350024400 con 0x563ebfad47d0 2023-10-05T19:40:20.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.927+0000 7f9357d26700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadc750 con 0x563ebfad9150 2023-10-05T19:40:20.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.927+0000 7f9357525700 1 -- 172.21.15.12:0/760245333 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9340006070 con 0x563ebfad9150 2023-10-05T19:40:20.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.927+0000 7f9357525700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadc750 con 0x7f935001abe0 2023-10-05T19:40:20.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.927+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9348014040 con 0x7f935001abe0 2023-10-05T19:40:20.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.927+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadc750 con 0x563ebfaa1920 2023-10-05T19:40:20.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.927+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f9348014040 con 0x563ebfaa1920 2023-10-05T19:40:20.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.927+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadc750 con 0x563ebfad47d0 2023-10-05T19:40:20.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.931+0000 7f9357d26700 1 -- 172.21.15.12:0/760245333 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f9350024400 con 0x563ebfad47d0 2023-10-05T19:40:20.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.931+0000 7f935a1f1e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:20.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.931+0000 7f935a1f1e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:20.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.931+0000 7f935a1f1e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:20.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.931+0000 7f935a1f1e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:20.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.931+0000 7f935a1f1e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.931+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadcb00 con 0x563ebfaa1920 2023-10-05T19:40:20.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.931+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f9348014040 con 0x563ebfaa1920 2023-10-05T19:40:20.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.931+0000 7f935a1f1e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:20.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.931+0000 7f935a1f1e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.931+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadceb0 con 0x563ebfad47d0 2023-10-05T19:40:20.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.931+0000 7f9357d26700 1 -- 172.21.15.12:0/760245333 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f9350024400 con 0x563ebfad47d0 2023-10-05T19:40:20.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.931+0000 7f935a1f1e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:20.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.931+0000 7f935a1f1e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.931+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadd750 con 0x563ebfaa1920 2023-10-05T19:40:20.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.931+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f9348014040 con 0x563ebfaa1920 2023-10-05T19:40:20.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.931+0000 7f935a1f1e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:20.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.931+0000 7f935a1f1e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.931+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadd750 con 0x563ebfad9150 2023-10-05T19:40:20.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.935+0000 7f9357525700 1 -- 172.21.15.12:0/760245333 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f9340006070 con 0x563ebfad9150 2023-10-05T19:40:20.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.935+0000 7f935a1f1e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:20.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.935+0000 7f935a1f1e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.935+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadd2e0 con 0x7f935001abe0 2023-10-05T19:40:20.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.935+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f9348014040 con 0x7f935001abe0 2023-10-05T19:40:20.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.935+0000 7f935a1f1e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.935+0000 7f935a1f1e40 20 zone default found 2023-10-05T19:40:20.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.935+0000 7f935a1f1e40 4 Realm: () 2023-10-05T19:40:20.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.935+0000 7f935a1f1e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:20.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.935+0000 7f935a1f1e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:20.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.935+0000 7f935a1f1e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:20.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.935+0000 7f935a1f1e40 20 zonegroup default 2023-10-05T19:40:20.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.935+0000 7f935a1f1e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.935+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadd2e0 con 0x7f935001abe0 2023-10-05T19:40:20.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.935+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f9348014040 con 0x7f935001abe0 2023-10-05T19:40:20.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.935+0000 7f935a1f1e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.935+0000 7f935a1f1e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.935+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadd2e0 con 0x563ebfad9150 2023-10-05T19:40:20.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.935+0000 7f9357525700 1 -- 172.21.15.12:0/760245333 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f9340006070 con 0x563ebfad9150 2023-10-05T19:40:20.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.935+0000 7f935a1f1e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.935+0000 7f935a1f1e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.935+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadf970 con 0x563ebfad9150 2023-10-05T19:40:20.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.939+0000 7f9357525700 1 -- 172.21.15.12:0/760245333 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f9340006070 con 0x563ebfad9150 2023-10-05T19:40:20.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.939+0000 7f935a1f1e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:20.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.939+0000 7f935a1f1e40 20 started sync module instance, tier type = 2023-10-05T19:40:20.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.939+0000 7f935a1f1e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:20.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.939+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadf030 con 0x563ebfad47d0 2023-10-05T19:40:20.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.939+0000 7f9357d26700 1 -- 172.21.15.12:0/760245333 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'60 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9350024400 con 0x563ebfad47d0 2023-10-05T19:40:20.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.939+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94827503809632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfadec60 con 0x563ebfad47d0 2023-10-05T19:40:20.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.939+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfae0700 con 0x563ebfaa1920 2023-10-05T19:40:20.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.943+0000 7f9357d26700 1 -- 172.21.15.12:0/760245333 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94827503809632] v28'61 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9350024400 con 0x563ebfad47d0 2023-10-05T19:40:20.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.943+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'60 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9348014040 con 0x563ebfaa1920 2023-10-05T19:40:20.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.943+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94827503812160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfae07c0 con 0x563ebfaa1920 2023-10-05T19:40:20.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.943+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfae1350 con 0x563ebfad9150 2023-10-05T19:40:20.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.947+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94827503812160] v28'61 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9348014040 con 0x563ebfaa1920 2023-10-05T19:40:20.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.947+0000 7f9357525700 1 -- 172.21.15.12:0/760245333 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'60 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9340006070 con 0x563ebfad9150 2023-10-05T19:40:20.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.947+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94827503813264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfae1750 con 0x563ebfad9150 2023-10-05T19:40:20.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.947+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfae1ff0 con 0x563ebfad47d0 2023-10-05T19:40:20.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.951+0000 7f9357d26700 1 -- 172.21.15.12:0/760245333 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'60 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9350024400 con 0x563ebfad47d0 2023-10-05T19:40:20.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.951+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94827503819808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfae28e0 con 0x563ebfad47d0 2023-10-05T19:40:20.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.951+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfae31c0 con 0x563ebfaa1920 2023-10-05T19:40:20.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.951+0000 7f9357525700 1 -- 172.21.15.12:0/760245333 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94827503813264] v28'61 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9340006070 con 0x563ebfad9150 2023-10-05T19:40:20.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.955+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'60 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9348014040 con 0x563ebfaa1920 2023-10-05T19:40:20.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.955+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94827503816224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfae2dd0 con 0x563ebfaa1920 2023-10-05T19:40:20.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.955+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfae3c20 con 0x7f935001abe0 2023-10-05T19:40:20.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.955+0000 7f9357d26700 1 -- 172.21.15.12:0/760245333 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94827503819808] v28'61 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9350024400 con 0x563ebfad47d0 2023-10-05T19:40:20.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.955+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'60 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9348014040 con 0x7f935001abe0 2023-10-05T19:40:20.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.955+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94827503821808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfae3c20 con 0x7f935001abe0 2023-10-05T19:40:20.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.955+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfae4630 con 0x7f935001abe0 2023-10-05T19:40:20.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.955+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94827503816224] v28'61 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9348014040 con 0x563ebfaa1920 2023-10-05T19:40:20.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.959+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(95 notify.5 [watch watch cookie 94827503821808] v28'61 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9348014040 con 0x7f935001abe0 2023-10-05T19:40:20.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.959+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(96 notify.6 [create] v28'60 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9348014040 con 0x7f935001abe0 2023-10-05T19:40:20.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.959+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94827503826736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfae3b80 con 0x7f935001abe0 2023-10-05T19:40:20.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.959+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfae4b20 con 0x7f935001abe0 2023-10-05T19:40:20.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.959+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(98 notify.7 [create] v28'60 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9348014040 con 0x7f935001abe0 2023-10-05T19:40:20.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.959+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94827503830576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfae4f70 con 0x7f935001abe0 2023-10-05T19:40:20.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.963+0000 7f935a1f1e40 20 add_watcher() i=0 2023-10-05T19:40:20.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.963+0000 7f935a1f1e40 20 add_watcher() i=1 2023-10-05T19:40:20.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.963+0000 7f935a1f1e40 20 add_watcher() i=2 2023-10-05T19:40:20.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.963+0000 7f935a1f1e40 20 add_watcher() i=3 2023-10-05T19:40:20.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.963+0000 7f935a1f1e40 20 add_watcher() i=4 2023-10-05T19:40:20.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.963+0000 7f935a1f1e40 20 add_watcher() i=5 2023-10-05T19:40:20.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.963+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(97 notify.6 [watch watch cookie 94827503826736] v28'61 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9348014040 con 0x7f935001abe0 2023-10-05T19:40:20.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.963+0000 7f935a1f1e40 20 add_watcher() i=6 2023-10-05T19:40:20.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.963+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94827503830576] v28'61 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9348014040 con 0x7f935001abe0 2023-10-05T19:40:20.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.963+0000 7f935a1f1e40 20 add_watcher() i=7 2023-10-05T19:40:20.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.963+0000 7f935a1f1e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:20.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.963+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfae7fa0 con 0x563ebfaa1920 2023-10-05T19:40:20.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.963+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f9348014040 con 0x563ebfaa1920 2023-10-05T19:40:20.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.963+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94827503831680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfae53c0 con 0x563ebfaa1920 2023-10-05T19:40:20.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.967+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94827503831680] v28'74 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f9348014040 con 0x563ebfaa1920 2023-10-05T19:40:20.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.967+0000 7f93297e2700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:20.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.967+0000 7f935a1f1e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:20.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.967+0000 7f935a1f1e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:20.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.967+0000 7f935a1f1e40 1 Processor -- start 2023-10-05T19:40:20.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.967+0000 7f935a1f1e40 1 -- start start 2023-10-05T19:40:20.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.967+0000 7f935a1f1e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ebfaf20e0 0x563ebfaf24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.967+0000 7f935a1f1e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ebfaf29f0 con 0x563ebfaf20e0 2023-10-05T19:40:20.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.967+0000 7f9357525700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ebfaf20e0 0x563ebfaf24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.967+0000 7f9357525700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ebfaf20e0 0x563ebfaf24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37188/0 (socket says 172.21.15.12:37188) 2023-10-05T19:40:20.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.967+0000 7f9357525700 1 -- 172.21.15.12:0/2122331103 learned_addr learned my addr 172.21.15.12:0/2122331103 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:20.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.967+0000 7f9357525700 1 -- 172.21.15.12:0/2122331103 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ebfaf2b30 con 0x563ebfaf20e0 2023-10-05T19:40:20.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.971+0000 7f9357525700 1 --2- 172.21.15.12:0/2122331103 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ebfaf20e0 0x563ebfaf24b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f935003a260 tx=0x7f934001e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d470f0efb3f3dfc7 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.971+0000 7f9328fe1700 1 -- 172.21.15.12:0/2122331103 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9340013040 con 0x563ebfaf20e0 2023-10-05T19:40:20.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.971+0000 7f9328fe1700 1 -- 172.21.15.12:0/2122331103 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9340022040 con 0x563ebfaf20e0 2023-10-05T19:40:20.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.971+0000 7f9328fe1700 1 -- 172.21.15.12:0/2122331103 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f934000e7e0 con 0x563ebfaf20e0 2023-10-05T19:40:20.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.971+0000 7f935a1f1e40 1 -- 172.21.15.12:0/2122331103 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ebfaf20e0 msgr2=0x563ebfaf24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:20.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.971+0000 7f935a1f1e40 1 --2- 172.21.15.12:0/2122331103 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ebfaf20e0 0x563ebfaf24b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f935003a260 tx=0x7f934001e040 comp rx=0 tx=0).stop 2023-10-05T19:40:20.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.971+0000 7f935a1f1e40 1 -- 172.21.15.12:0/2122331103 shutdown_connections 2023-10-05T19:40:20.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.971+0000 7f935a1f1e40 1 --2- 172.21.15.12:0/2122331103 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ebfaf20e0 0x563ebfaf24b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:20.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.971+0000 7f935a1f1e40 1 -- 172.21.15.12:0/2122331103 >> 172.21.15.12:0/2122331103 conn(0x563ebfaed4a0 msgr2=0x563ebfaeb650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:20.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.971+0000 7f935a1f1e40 1 -- 172.21.15.12:0/2122331103 shutdown_connections 2023-10-05T19:40:20.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.971+0000 7f935a1f1e40 1 -- 172.21.15.12:0/2122331103 wait complete. 2023-10-05T19:40:20.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.971+0000 7f935a1f1e40 1 Processor -- start 2023-10-05T19:40:20.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.971+0000 7f935a1f1e40 1 -- start start 2023-10-05T19:40:20.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.971+0000 7f935a1f1e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ebfaf20e0 0x563ebfaf7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.971+0000 7f935a1f1e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ebfaf8c60 con 0x563ebfaf20e0 2023-10-05T19:40:20.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.971+0000 7f9357525700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ebfaf20e0 0x563ebfaf7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.971+0000 7f9357525700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ebfaf20e0 0x563ebfaf7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37202/0 (socket says 172.21.15.12:37202) 2023-10-05T19:40:20.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.971+0000 7f9357525700 1 -- 172.21.15.12:0/1106024477 learned_addr learned my addr 172.21.15.12:0/1106024477 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:20.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.975+0000 7f9357525700 1 -- 172.21.15.12:0/1106024477 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9340034040 con 0x563ebfaf20e0 2023-10-05T19:40:20.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.975+0000 7f9357525700 1 --2- 172.21.15.12:0/1106024477 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ebfaf20e0 0x563ebfaf7c60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x563ebfaec1c0 tx=0x7f934003d030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.975+0000 7f92a2ffd700 1 -- 172.21.15.12:0/1106024477 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f934002c640 con 0x563ebfaf20e0 2023-10-05T19:40:20.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.975+0000 7f92a2ffd700 1 -- 172.21.15.12:0/1106024477 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f934002c7c0 con 0x563ebfaf20e0 2023-10-05T19:40:20.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.975+0000 7f935a1f1e40 1 -- 172.21.15.12:0/1106024477 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563ebfaf8260 con 0x563ebfaf20e0 2023-10-05T19:40:20.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.975+0000 7f92a2ffd700 1 -- 172.21.15.12:0/1106024477 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f934002c970 con 0x563ebfaf20e0 2023-10-05T19:40:20.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.975+0000 7f935a1f1e40 1 -- 172.21.15.12:0/1106024477 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ebfaf86a0 con 0x563ebfaf20e0 2023-10-05T19:40:20.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.975+0000 7f92a2ffd700 1 -- 172.21.15.12:0/1106024477 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f934004d070 con 0x563ebfaf20e0 2023-10-05T19:40:20.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.975+0000 7f92a2ffd700 1 --2- 172.21.15.12:0/1106024477 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f9298030c00 0x7f92980330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:20.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.975+0000 7f92a2ffd700 1 -- 172.21.15.12:0/1106024477 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f9340051b30 con 0x563ebfaf20e0 2023-10-05T19:40:20.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.975+0000 7f935a1f1e40 5 note: GC not initialized 2023-10-05T19:40:20.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.975+0000 7f9358527700 1 --2- 172.21.15.12:0/1106024477 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f9298030c00 0x7f92980330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:20.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.979+0000 7f9358527700 1 --2- 172.21.15.12:0/1106024477 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f9298030c00 0x7f92980330b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x563ebfae3c30 tx=0x7f934800a9b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:20.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.979+0000 7f9295ffb700 20 reqs_thread_entry: start 2023-10-05T19:40:20.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.979+0000 7f935a1f1e40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:20.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.979+0000 7f9297fff700 20 reqs_thread_entry: start 2023-10-05T19:40:20.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.979+0000 7f935a1f1e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:20.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.979+0000 7f9283fff700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:51 2023 2023-10-05T19:40:20.988 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:20.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.979+0000 7f92837fe700 20 reqs_thread_entry: start 2023-10-05T19:40:20.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.979+0000 7f935a1f1e40 20 get_system_obj_state: rctx=0x7ffd1fd6ba78 obj=default.rgw.meta:users.uid:foo state=0x563ebf9780b0 s->prefetch_data=0 2023-10-05T19:40:20.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.979+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfb60470 con 0x563ebfaa1920 2023-10-05T19:40:20.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.979+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(102 foo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7f9348014040 con 0x563ebfaa1920 2023-10-05T19:40:20.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.979+0000 7f935a1f1e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:20.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.979+0000 7f935a1f1e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:20.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.979+0000 7f935a1f1e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:20.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.979+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfb60420 con 0x563ebfaa1920 2023-10-05T19:40:20.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.983+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(103 foo [call,call out=48b,read 0~513 out=513b,getxattrs out=75b] v0'0 uv7 ondisk = 0) v8 ==== 273+0+636 (crc 0 0 0) 0x7f9348014040 con 0x563ebfaa1920 2023-10-05T19:40:20.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.983+0000 7f935a1f1e40 20 rados_obj.operate() r=0 bl.length=513 2023-10-05T19:40:20.990 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:40:20.991 INFO:teuthology.orchestra.run.smithi012.stdout: "user_id": "foo", 2023-10-05T19:40:20.991 INFO:teuthology.orchestra.run.smithi012.stdout: "display_name": "Foo", 2023-10-05T19:40:20.991 INFO:teuthology.orchestra.run.smithi012.stdout: "email": "foo@foo.com", 2023-10-05T19:40:20.991 INFO:teuthology.orchestra.run.smithi012.stdout: "suspended": 0, 2023-10-05T19:40:20.991 INFO:teuthology.orchestra.run.smithi012.stdout: "max_buckets": 4, 2023-10-05T19:40:20.991 INFO:teuthology.orchestra.run.smithi012.stdout: "subusers": [ 2023-10-05T19:40:20.991 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:20.991 INFO:teuthology.orchestra.run.smithi012.stdout: "id": "foo:foo1", 2023-10-05T19:40:20.991 INFO:teuthology.orchestra.run.smithi012.stdout: "permissions": "full-control" 2023-10-05T19:40:20.991 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:20.991 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:20.992 INFO:teuthology.orchestra.run.smithi012.stdout: "keys": [ 2023-10-05T19:40:20.992 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:20.992 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:40:20.992 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2023-10-05T19:40:20.992 INFO:teuthology.orchestra.run.smithi012.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2023-10-05T19:40:20.992 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:20.992 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:20.992 INFO:teuthology.orchestra.run.smithi012.stdout: "swift_keys": [ 2023-10-05T19:40:20.992 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:20.992 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo:foo1", 2023-10-05T19:40:20.992 INFO:teuthology.orchestra.run.smithi012.stdout: "secret_key": "gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL" 2023-10-05T19:40:20.992 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:20.993 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:20.993 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": [], 2023-10-05T19:40:20.993 INFO:teuthology.orchestra.run.smithi012.stdout: "op_mask": "read, write, delete", 2023-10-05T19:40:20.993 INFO:teuthology.orchestra.run.smithi012.stdout: "default_placement": "", 2023-10-05T19:40:20.993 INFO:teuthology.orchestra.run.smithi012.stdout: "default_storage_class": "", 2023-10-05T19:40:20.993 INFO:teuthology.orchestra.run.smithi012.stdout: "placement_tags": [], 2023-10-05T19:40:20.993 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket_quota": { 2023-10-05T19:40:20.993 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:20.993 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:20.993 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:20.993 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:20.993 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:20.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:20.994 INFO:teuthology.orchestra.run.smithi012.stdout: "user_quota": { 2023-10-05T19:40:20.994 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:20.994 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:20.994 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:20.994 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:20.994 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:20.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:20.994 INFO:teuthology.orchestra.run.smithi012.stdout: "temp_url_keys": [], 2023-10-05T19:40:20.994 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "rgw", 2023-10-05T19:40:20.994 INFO:teuthology.orchestra.run.smithi012.stdout: "mfa_ids": [] 2023-10-05T19:40:20.994 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:40:20.995 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:40:20.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.983+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:104 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94827503809632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfb60420 con 0x563ebfad47d0 2023-10-05T19:40:20.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.983+0000 7f9357d26700 1 -- 172.21.15.12:0/760245333 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(104 notify.0 [watch unwatch cookie 94827503809632] v28'62 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9350024400 con 0x563ebfad47d0 2023-10-05T19:40:20.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.983+0000 7f935a1f1e40 20 remove_watcher() i=0 2023-10-05T19:40:20.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.983+0000 7f935a1f1e40 2 removed watcher, disabling cache 2023-10-05T19:40:20.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.983+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:105 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94827503812160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfb60420 con 0x563ebfaa1920 2023-10-05T19:40:20.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.987+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(105 notify.1 [watch unwatch cookie 94827503812160] v28'62 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9348014040 con 0x563ebfaa1920 2023-10-05T19:40:20.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.987+0000 7f935a1f1e40 20 remove_watcher() i=1 2023-10-05T19:40:20.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.987+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:106 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94827503813264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfb60420 con 0x563ebfad9150 2023-10-05T19:40:20.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.991+0000 7f9357525700 1 -- 172.21.15.12:0/760245333 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(106 notify.2 [watch unwatch cookie 94827503813264] v28'62 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9340006070 con 0x563ebfad9150 2023-10-05T19:40:20.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.991+0000 7f935a1f1e40 20 remove_watcher() i=2 2023-10-05T19:40:20.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.991+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:107 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94827503819808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfb60420 con 0x563ebfad47d0 2023-10-05T19:40:21.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.991+0000 7f9357d26700 1 -- 172.21.15.12:0/760245333 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(107 notify.3 [watch unwatch cookie 94827503819808] v28'62 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9350024400 con 0x563ebfad47d0 2023-10-05T19:40:21.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.991+0000 7f935a1f1e40 20 remove_watcher() i=3 2023-10-05T19:40:21.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.991+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:108 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94827503816224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfb60420 con 0x563ebfaa1920 2023-10-05T19:40:21.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.995+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(108 notify.4 [watch unwatch cookie 94827503816224] v28'62 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9348014040 con 0x563ebfaa1920 2023-10-05T19:40:21.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.995+0000 7f935a1f1e40 20 remove_watcher() i=4 2023-10-05T19:40:21.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.995+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:109 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94827503821808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfb60420 con 0x7f935001abe0 2023-10-05T19:40:21.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.999+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(109 notify.5 [watch unwatch cookie 94827503821808] v28'62 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9348014040 con 0x7f935001abe0 2023-10-05T19:40:21.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.999+0000 7f935a1f1e40 20 remove_watcher() i=5 2023-10-05T19:40:21.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:20.999+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:110 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94827503826736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfb60420 con 0x7f935001abe0 2023-10-05T19:40:21.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.003+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(110 notify.6 [watch unwatch cookie 94827503826736] v28'62 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9348014040 con 0x7f935001abe0 2023-10-05T19:40:21.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.003+0000 7f935a1f1e40 20 remove_watcher() i=6 2023-10-05T19:40:21.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.003+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:111 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94827503830576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfb60420 con 0x7f935001abe0 2023-10-05T19:40:21.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.003+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(111 notify.7 [watch unwatch cookie 94827503830576] v28'62 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9348014040 con 0x7f935001abe0 2023-10-05T19:40:21.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.003+0000 7f935a1f1e40 20 remove_watcher() i=7 2023-10-05T19:40:21.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.007+0000 7f93297e2700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:21.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.007+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:112 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94827503831680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x563ebfb60420 con 0x563ebfaa1920 2023-10-05T19:40:21.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.007+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(112 data_loggenerations_metadata [watch unwatch cookie 94827503831680] v28'75 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f9348014040 con 0x563ebfaa1920 2023-10-05T19:40:21.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.011+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f935001abe0 msgr2=0x7f93500383b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.011+0000 7f935a1f1e40 1 --2- 172.21.15.12:0/760245333 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f935001abe0 0x7f93500383b0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.011+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x563ebfaa1920 msgr2=0x563ebfad2740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.011+0000 7f935a1f1e40 1 --2- 172.21.15.12:0/760245333 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x563ebfaa1920 0x563ebfad2740 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.011+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x563ebfad47d0 msgr2=0x563ebfad6bb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.011+0000 7f935a1f1e40 1 --2- 172.21.15.12:0/760245333 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x563ebfad47d0 0x563ebfad6bb0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.011+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x563ebfad9150 msgr2=0x563ebfadb530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.011+0000 7f935a1f1e40 1 --2- 172.21.15.12:0/760245333 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x563ebfad9150 0x563ebfadb530 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.011+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f9338030c00 msgr2=0x7f93380330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.011+0000 7f935a1f1e40 1 --2- 172.21.15.12:0/760245333 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f9338030c00 0x7f93380330b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9340004170 tx=0x7f9340001400 comp rx=0 tx=0).stop 2023-10-05T19:40:21.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.011+0000 7f9358527700 1 -- 172.21.15.12:0/760245333 reap_dead start 2023-10-05T19:40:21.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.011+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ebfaa1010 msgr2=0x563ebfac6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.011+0000 7f935a1f1e40 1 --2- 172.21.15.12:0/760245333 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ebfaa1010 0x563ebfac6b80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f935000adc0 tx=0x7f935001d000 comp rx=0 tx=0).stop 2023-10-05T19:40:21.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.011+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 shutdown_connections 2023-10-05T19:40:21.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.011+0000 7f935a1f1e40 1 --2- 172.21.15.12:0/760245333 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ebfaa1010 0x563ebfac6b80 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.011+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 >> 172.21.15.12:0/760245333 conn(0x563ebfa14860 msgr2=0x563ebfaba4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:21.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.011+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 shutdown_connections 2023-10-05T19:40:21.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.011+0000 7f935a1f1e40 1 -- 172.21.15.12:0/760245333 wait complete. 2023-10-05T19:40:21.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.011+0000 7f935a1f1e40 1 -- 172.21.15.12:0/1106024477 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f9298030c00 msgr2=0x7f92980330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.011+0000 7f935a1f1e40 1 --2- 172.21.15.12:0/1106024477 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f9298030c00 0x7f92980330b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x563ebfae3c30 tx=0x7f934800a9b0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.011+0000 7f935a1f1e40 1 -- 172.21.15.12:0/1106024477 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ebfaf20e0 msgr2=0x563ebfaf7c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.011+0000 7f935a1f1e40 1 --2- 172.21.15.12:0/1106024477 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ebfaf20e0 0x563ebfaf7c60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x563ebfaec1c0 tx=0x7f934003d030 comp rx=0 tx=0).stop 2023-10-05T19:40:21.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.011+0000 7f935a1f1e40 1 -- 172.21.15.12:0/1106024477 shutdown_connections 2023-10-05T19:40:21.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.011+0000 7f935a1f1e40 1 --2- 172.21.15.12:0/1106024477 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f9298030c00 0x7f92980330b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.011+0000 7f935a1f1e40 1 --2- 172.21.15.12:0/1106024477 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563ebfaf20e0 0x563ebfaf7c60 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.011+0000 7f935a1f1e40 1 -- 172.21.15.12:0/1106024477 >> 172.21.15.12:0/1106024477 conn(0x563ebfaed4a0 msgr2=0x563ebfaeb5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:21.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.011+0000 7f935a1f1e40 1 -- 172.21.15.12:0/1106024477 shutdown_connections 2023-10-05T19:40:21.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.011+0000 7f935a1f1e40 1 -- 172.21.15.12:0/1106024477 wait complete. 2023-10-05T19:40:21.026 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:40:21.026 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2023-10-05T19:40:21.026 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2023-10-05T19:40:21.026 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2023-10-05T19:40:21.110 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:21.111 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:21.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.103+0000 7f39aba95e40 1 Processor -- start 2023-10-05T19:40:21.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.103+0000 7f39aba95e40 1 -- start start 2023-10-05T19:40:21.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.103+0000 7f39aba95e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55eb48700600 0x55eb487029e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.103+0000 7f39aba95e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55eb48646bd0 con 0x55eb486fc020 2023-10-05T19:40:21.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.103+0000 7f39aba95e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x55eb48603ee0 con 0x55eb48700600 2023-10-05T19:40:21.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.103+0000 7f39a8dc9700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55eb48700600 0x55eb487029e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.103+0000 7f39a95ca700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x55eb486fc020 0x55eb486fc3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:47320/0 (socket says 172.21.15.12:47320) 2023-10-05T19:40:21.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.103+0000 7f39a8dc9700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55eb48700600 0x55eb487029e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37216/0 (socket says 172.21.15.12:37216) 2023-10-05T19:40:21.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.103+0000 7f39a95ca700 1 -- 172.21.15.12:0/1132963689 learned_addr learned my addr 172.21.15.12:0/1132963689 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:21.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.103+0000 7f39a8dc9700 1 -- 172.21.15.12:0/1132963689 >> v1:172.21.15.8:6789/0 conn(0x55eb486fc020 legacy=0x55eb486fc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.103+0000 7f39a8dc9700 1 -- 172.21.15.12:0/1132963689 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb486fcc30 con 0x55eb48700600 2023-10-05T19:40:21.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.103+0000 7f39a8dc9700 1 --2- 172.21.15.12:0/1132963689 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eb48700600 0x55eb487029e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f399400a5d0 tx=0x7f3994005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2f88b036afe41a server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.107+0000 7f39a3fff700 1 -- 172.21.15.12:0/1132963689 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39940086a0 con 0x55eb48700600 2023-10-05T19:40:21.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.107+0000 7f39a3fff700 1 -- 172.21.15.12:0/1132963689 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3994008800 con 0x55eb48700600 2023-10-05T19:40:21.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.107+0000 7f39a3fff700 1 -- 172.21.15.12:0/1132963689 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3994008ad0 con 0x55eb48700600 2023-10-05T19:40:21.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.107+0000 7f39aba95e40 1 -- 172.21.15.12:0/1132963689 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eb48700600 msgr2=0x55eb487029e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.107+0000 7f39aba95e40 1 --2- 172.21.15.12:0/1132963689 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eb48700600 0x55eb487029e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f399400a5d0 tx=0x7f3994005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:21.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.107+0000 7f39aba95e40 1 -- 172.21.15.12:0/1132963689 shutdown_connections 2023-10-05T19:40:21.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.107+0000 7f39aba95e40 1 --2- 172.21.15.12:0/1132963689 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eb48700600 0x55eb487029e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.107+0000 7f39aba95e40 1 -- 172.21.15.12:0/1132963689 >> 172.21.15.12:0/1132963689 conn(0x55eb4866f870 msgr2=0x55eb486fa510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:21.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.107+0000 7f39aba95e40 1 -- 172.21.15.12:0/1132963689 shutdown_connections 2023-10-05T19:40:21.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.107+0000 7f39aba95e40 1 -- 172.21.15.12:0/1132963689 wait complete. 2023-10-05T19:40:21.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.107+0000 7f39aba95e40 1 Processor -- start 2023-10-05T19:40:21.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.107+0000 7f39aba95e40 1 -- start start 2023-10-05T19:40:21.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.107+0000 7f39aba95e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eb486fc020 0x55eb4871b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.107+0000 7f39aba95e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb485c7e30 con 0x55eb486fc020 2023-10-05T19:40:21.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.107+0000 7f39a95ca700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eb486fc020 0x55eb4871b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.107+0000 7f39a95ca700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eb486fc020 0x55eb4871b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37220/0 (socket says 172.21.15.12:37220) 2023-10-05T19:40:21.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.107+0000 7f39a95ca700 1 -- 172.21.15.12:0/3760045832 learned_addr learned my addr 172.21.15.12:0/3760045832 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:21.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.107+0000 7f39a95ca700 1 -- 172.21.15.12:0/3760045832 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3994005cf0 con 0x55eb486fc020 2023-10-05T19:40:21.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.107+0000 7f39a95ca700 1 --2- 172.21.15.12:0/3760045832 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eb486fc020 0x55eb4871b100 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f399c001bf0 tx=0x7f399c0017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07f9356cfab0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.107+0000 7f39a2ffd700 1 -- 172.21.15.12:0/3760045832 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f399c00bbf0 con 0x55eb486fc020 2023-10-05T19:40:21.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.111+0000 7f39a2ffd700 1 -- 172.21.15.12:0/3760045832 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f399c00bd50 con 0x55eb486fc020 2023-10-05T19:40:21.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.111+0000 7f39a2ffd700 1 -- 172.21.15.12:0/3760045832 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f399c00e470 con 0x55eb486fc020 2023-10-05T19:40:21.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.111+0000 7f39aba95e40 1 -- 172.21.15.12:0/3760045832 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eb486fc020 msgr2=0x55eb4871b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.111+0000 7f39aba95e40 1 --2- 172.21.15.12:0/3760045832 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eb486fc020 0x55eb4871b100 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f399c001bf0 tx=0x7f399c0017a0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.111+0000 7f39aba95e40 1 -- 172.21.15.12:0/3760045832 shutdown_connections 2023-10-05T19:40:21.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.111+0000 7f39aba95e40 1 --2- 172.21.15.12:0/3760045832 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eb486fc020 0x55eb4871b100 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.111+0000 7f39aba95e40 1 -- 172.21.15.12:0/3760045832 >> 172.21.15.12:0/3760045832 conn(0x55eb4866f870 msgr2=0x55eb48716cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:21.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.111+0000 7f39aba95e40 1 -- 172.21.15.12:0/3760045832 shutdown_connections 2023-10-05T19:40:21.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.111+0000 7f39aba95e40 1 -- 172.21.15.12:0/3760045832 wait complete. 2023-10-05T19:40:21.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.111+0000 7f39aba95e40 1 Processor -- start 2023-10-05T19:40:21.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.111+0000 7f39aba95e40 1 -- start start 2023-10-05T19:40:21.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.111+0000 7f39aba95e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eb48700600 0x55eb48722040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.111+0000 7f39aba95e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb4859aed0 con 0x55eb48700600 2023-10-05T19:40:21.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.111+0000 7f39a95ca700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eb48700600 0x55eb48722040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.111+0000 7f39a95ca700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eb48700600 0x55eb48722040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37228/0 (socket says 172.21.15.12:37228) 2023-10-05T19:40:21.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.111+0000 7f39a95ca700 1 -- 172.21.15.12:0/1282781615 learned_addr learned my addr 172.21.15.12:0/1282781615 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:21.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.111+0000 7f39a95ca700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb48646bd0 con 0x55eb48700600 2023-10-05T19:40:21.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.111+0000 7f39a95ca700 1 --2- 172.21.15.12:0/1282781615 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eb48700600 0x55eb48722040 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x55eb487177e0 tx=0x7f399c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.111+0000 7f39a17fa700 1 -- 172.21.15.12:0/1282781615 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f399c01f070 con 0x55eb48700600 2023-10-05T19:40:21.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.111+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55eb48722580 con 0x55eb48700600 2023-10-05T19:40:21.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.111+0000 7f39a17fa700 1 -- 172.21.15.12:0/1282781615 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f399c01a2e0 con 0x55eb48700600 2023-10-05T19:40:21.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.111+0000 7f39a17fa700 1 -- 172.21.15.12:0/1282781615 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f399c00bec0 con 0x55eb48700600 2023-10-05T19:40:21.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.111+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eb487229c0 con 0x55eb48700600 2023-10-05T19:40:21.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.115+0000 7f39a17fa700 1 -- 172.21.15.12:0/1282781615 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f399c01a960 con 0x55eb48700600 2023-10-05T19:40:21.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.115+0000 7f39a17fa700 1 --2- 172.21.15.12:0/1282781615 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f398c030bb0 0x7f398c033060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.115+0000 7f39a17fa700 1 -- 172.21.15.12:0/1282781615 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f399c044ec0 con 0x55eb48700600 2023-10-05T19:40:21.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.115+0000 7f39aba95e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.115+0000 7f39a8dc9700 1 --2- 172.21.15.12:0/1282781615 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f398c030bb0 0x7f398c033060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.115+0000 7f39aba95e40 1 --2- 172.21.15.12:0/1282781615 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55eb4872bac0 0x55eb4872dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.115+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb4872e510 con 0x55eb4872bac0 2023-10-05T19:40:21.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.115+0000 7f39a9dcb700 1 --2- 172.21.15.12:0/1282781615 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55eb4872bac0 0x55eb4872dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.115+0000 7f39a8dc9700 1 --2- 172.21.15.12:0/1282781615 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f398c030bb0 0x7f398c033060 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3994005a90 tx=0x7f3994015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.115+0000 7f39a9dcb700 1 --2- 172.21.15.12:0/1282781615 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55eb4872bac0 0x55eb4872dea0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.119+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f39a400a5a0 con 0x55eb4872bac0 2023-10-05T19:40:21.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.119+0000 7f39aba95e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:21.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.119+0000 7f39aba95e40 20 realm 2023-10-05T19:40:21.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.119+0000 7f39aba95e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.119+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb4872b130 con 0x55eb4872bac0 2023-10-05T19:40:21.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.119+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f39a400a5a0 con 0x55eb4872bac0 2023-10-05T19:40:21.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.119+0000 7f39aba95e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:21.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.119+0000 7f39aba95e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:21.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.119+0000 7f39aba95e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.119+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb4872eff0 con 0x55eb4872bac0 2023-10-05T19:40:21.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.119+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f39a400a5a0 con 0x55eb4872bac0 2023-10-05T19:40:21.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.119+0000 7f39aba95e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:21.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.119+0000 7f39aba95e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.119+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb4872ebd0 con 0x55eb4872bac0 2023-10-05T19:40:21.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.119+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f39a400a5a0 con 0x55eb4872bac0 2023-10-05T19:40:21.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.119+0000 7f39aba95e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:21.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.119+0000 7f39aba95e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.119+0000 7f39aba95e40 1 --2- 172.21.15.12:0/1282781615 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55eb4872ff80 0x55eb48732360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.119+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48732ad0 con 0x55eb4872ff80 2023-10-05T19:40:21.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.119+0000 7f39a95ca700 1 --2- 172.21.15.12:0/1282781615 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55eb4872ff80 0x55eb48732360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.119+0000 7f39a95ca700 1 --2- 172.21.15.12:0/1282781615 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55eb4872ff80 0x55eb48732360 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.123+0000 7f39a95ca700 1 -- 172.21.15.12:0/1282781615 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f399c014070 con 0x55eb4872ff80 2023-10-05T19:40:21.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.123+0000 7f39aba95e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:21.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.123+0000 7f39aba95e40 20 searching for the correct realm 2023-10-05T19:40:21.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.123+0000 7f39aba95e40 1 --2- 172.21.15.12:0/1282781615 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55eb48734900 0x55eb48736ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.123+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737220 con 0x55eb48734900 2023-10-05T19:40:21.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.123+0000 7f39a8dc9700 1 --2- 172.21.15.12:0/1282781615 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55eb48734900 0x55eb48736ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.123+0000 7f39a8dc9700 1 --2- 172.21.15.12:0/1282781615 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55eb48734900 0x55eb48736ce0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.123+0000 7f39a8dc9700 1 -- 172.21.15.12:0/1282781615 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f3994012450 con 0x55eb48734900 2023-10-05T19:40:21.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.123+0000 7f39a8dc9700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737220 con 0x55eb4872ff80 2023-10-05T19:40:21.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.123+0000 7f39a95ca700 1 -- 172.21.15.12:0/1282781615 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f399c014070 con 0x55eb4872ff80 2023-10-05T19:40:21.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.123+0000 7f39a95ca700 1 --2- 172.21.15.12:0/1282781615 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f399c02d020 0x7f399c03d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.123+0000 7f39a95ca700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737220 con 0x7f399c02d020 2023-10-05T19:40:21.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.127+0000 7f39a9dcb700 1 --2- 172.21.15.12:0/1282781615 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f399c02d020 0x7f399c03d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.127+0000 7f39a9dcb700 1 --2- 172.21.15.12:0/1282781615 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f399c02d020 0x7f399c03d790 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.127+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39a400a5a0 con 0x7f399c02d020 2023-10-05T19:40:21.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.127+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737220 con 0x55eb4872bac0 2023-10-05T19:40:21.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.127+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39a400a5a0 con 0x55eb4872bac0 2023-10-05T19:40:21.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.127+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737220 con 0x55eb4872bac0 2023-10-05T19:40:21.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.127+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39a400a5a0 con 0x55eb4872bac0 2023-10-05T19:40:21.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.127+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737220 con 0x7f399c02d020 2023-10-05T19:40:21.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.127+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39a400a5a0 con 0x7f399c02d020 2023-10-05T19:40:21.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.127+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737220 con 0x7f399c02d020 2023-10-05T19:40:21.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.131+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39a400a5a0 con 0x7f399c02d020 2023-10-05T19:40:21.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.131+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737220 con 0x55eb4872ff80 2023-10-05T19:40:21.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.131+0000 7f39a95ca700 1 -- 172.21.15.12:0/1282781615 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f399c014070 con 0x55eb4872ff80 2023-10-05T19:40:21.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.131+0000 7f39a95ca700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737220 con 0x55eb48734900 2023-10-05T19:40:21.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.131+0000 7f39a8dc9700 1 -- 172.21.15.12:0/1282781615 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3994012450 con 0x55eb48734900 2023-10-05T19:40:21.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.131+0000 7f39a8dc9700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737220 con 0x55eb4872bac0 2023-10-05T19:40:21.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.131+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39a400a5a0 con 0x55eb4872bac0 2023-10-05T19:40:21.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.131+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737220 con 0x7f399c02d020 2023-10-05T19:40:21.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.131+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39a400a5a0 con 0x7f399c02d020 2023-10-05T19:40:21.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.131+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737220 con 0x55eb4872bac0 2023-10-05T19:40:21.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.135+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39a400a5a0 con 0x55eb4872bac0 2023-10-05T19:40:21.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.135+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737220 con 0x55eb4872ff80 2023-10-05T19:40:21.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.135+0000 7f39a95ca700 1 -- 172.21.15.12:0/1282781615 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f399c014070 con 0x55eb4872ff80 2023-10-05T19:40:21.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.135+0000 7f39a95ca700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737220 con 0x55eb48734900 2023-10-05T19:40:21.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.135+0000 7f39a8dc9700 1 -- 172.21.15.12:0/1282781615 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3994012450 con 0x55eb48734900 2023-10-05T19:40:21.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.135+0000 7f39a8dc9700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737220 con 0x55eb48734900 2023-10-05T19:40:21.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.135+0000 7f39a8dc9700 1 -- 172.21.15.12:0/1282781615 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3994012450 con 0x55eb48734900 2023-10-05T19:40:21.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.135+0000 7f39a8dc9700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737220 con 0x7f399c02d020 2023-10-05T19:40:21.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.135+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39a400a5a0 con 0x7f399c02d020 2023-10-05T19:40:21.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.135+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737220 con 0x55eb48734900 2023-10-05T19:40:21.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.135+0000 7f39a8dc9700 1 -- 172.21.15.12:0/1282781615 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3994012450 con 0x55eb48734900 2023-10-05T19:40:21.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.139+0000 7f39a8dc9700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737220 con 0x55eb48734900 2023-10-05T19:40:21.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.139+0000 7f39a8dc9700 1 -- 172.21.15.12:0/1282781615 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3994012450 con 0x55eb48734900 2023-10-05T19:40:21.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.139+0000 7f39a8dc9700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737220 con 0x55eb4872bac0 2023-10-05T19:40:21.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.139+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39a400a5a0 con 0x55eb4872bac0 2023-10-05T19:40:21.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.139+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737220 con 0x55eb4872bac0 2023-10-05T19:40:21.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.139+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39a400a5a0 con 0x55eb4872bac0 2023-10-05T19:40:21.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.139+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737220 con 0x7f399c02d020 2023-10-05T19:40:21.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.139+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39a400a5a0 con 0x7f399c02d020 2023-10-05T19:40:21.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.139+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737220 con 0x55eb48734900 2023-10-05T19:40:21.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.139+0000 7f39a8dc9700 1 -- 172.21.15.12:0/1282781615 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3994012450 con 0x55eb48734900 2023-10-05T19:40:21.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.139+0000 7f39a8dc9700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737220 con 0x55eb4872ff80 2023-10-05T19:40:21.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.143+0000 7f39a95ca700 1 -- 172.21.15.12:0/1282781615 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f399c014070 con 0x55eb4872ff80 2023-10-05T19:40:21.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.143+0000 7f39a95ca700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737220 con 0x55eb4872bac0 2023-10-05T19:40:21.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.143+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39a400a5a0 con 0x55eb4872bac0 2023-10-05T19:40:21.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.143+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737220 con 0x7f399c02d020 2023-10-05T19:40:21.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.143+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39a400a5a0 con 0x7f399c02d020 2023-10-05T19:40:21.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.143+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737220 con 0x55eb48734900 2023-10-05T19:40:21.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.143+0000 7f39a8dc9700 1 -- 172.21.15.12:0/1282781615 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3994012450 con 0x55eb48734900 2023-10-05T19:40:21.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.143+0000 7f39a8dc9700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737220 con 0x7f399c02d020 2023-10-05T19:40:21.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.147+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39a400a5a0 con 0x7f399c02d020 2023-10-05T19:40:21.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.147+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737220 con 0x55eb4872ff80 2023-10-05T19:40:21.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.147+0000 7f39a95ca700 1 -- 172.21.15.12:0/1282781615 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f399c014070 con 0x55eb4872ff80 2023-10-05T19:40:21.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.147+0000 7f39a95ca700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737220 con 0x55eb48734900 2023-10-05T19:40:21.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.147+0000 7f39a8dc9700 1 -- 172.21.15.12:0/1282781615 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3994012450 con 0x55eb48734900 2023-10-05T19:40:21.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.147+0000 7f39a8dc9700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737220 con 0x7f399c02d020 2023-10-05T19:40:21.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.147+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39a400a5a0 con 0x7f399c02d020 2023-10-05T19:40:21.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.147+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737220 con 0x55eb4872bac0 2023-10-05T19:40:21.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.147+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f39a400a5a0 con 0x55eb4872bac0 2023-10-05T19:40:21.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.147+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737220 con 0x55eb4872ff80 2023-10-05T19:40:21.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.151+0000 7f39a95ca700 1 -- 172.21.15.12:0/1282781615 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f399c014070 con 0x55eb4872ff80 2023-10-05T19:40:21.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.151+0000 7f39aba95e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:21.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.151+0000 7f39aba95e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:21.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.151+0000 7f39aba95e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:21.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.151+0000 7f39aba95e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:21.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.151+0000 7f39aba95e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.151+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48733680 con 0x55eb4872bac0 2023-10-05T19:40:21.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.151+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f39a400a5a0 con 0x55eb4872bac0 2023-10-05T19:40:21.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.151+0000 7f39aba95e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:21.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.151+0000 7f39aba95e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.151+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48733fd0 con 0x55eb4872bac0 2023-10-05T19:40:21.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.151+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f39a400a5a0 con 0x55eb4872bac0 2023-10-05T19:40:21.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.151+0000 7f39aba95e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:21.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.151+0000 7f39aba95e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.151+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48733240 con 0x55eb48734900 2023-10-05T19:40:21.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.151+0000 7f39a8dc9700 1 -- 172.21.15.12:0/1282781615 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f3994012450 con 0x55eb48734900 2023-10-05T19:40:21.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.151+0000 7f39aba95e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:21.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.151+0000 7f39aba95e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.151+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737b50 con 0x55eb48734900 2023-10-05T19:40:21.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.151+0000 7f39a8dc9700 1 -- 172.21.15.12:0/1282781615 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f3994012450 con 0x55eb48734900 2023-10-05T19:40:21.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.151+0000 7f39aba95e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:21.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.151+0000 7f39aba95e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.151+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737f00 con 0x55eb4872bac0 2023-10-05T19:40:21.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.155+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f39a400a5a0 con 0x55eb4872bac0 2023-10-05T19:40:21.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.155+0000 7f39aba95e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:21.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.155+0000 7f39aba95e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.155+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb487382b0 con 0x55eb4872bac0 2023-10-05T19:40:21.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.155+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f39a400a5a0 con 0x55eb4872bac0 2023-10-05T19:40:21.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.155+0000 7f39aba95e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:21.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.155+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb487392c0 con 0x55eb48734900 2023-10-05T19:40:21.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.159+0000 7f39a8dc9700 1 -- 172.21.15.12:0/1282781615 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f3994012450 con 0x55eb48734900 2023-10-05T19:40:21.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.159+0000 7f39a8dc9700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb487392c0 con 0x55eb4872ff80 2023-10-05T19:40:21.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.159+0000 7f39a95ca700 1 -- 172.21.15.12:0/1282781615 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f399c014070 con 0x55eb4872ff80 2023-10-05T19:40:21.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.159+0000 7f39a95ca700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb487392c0 con 0x7f399c02d020 2023-10-05T19:40:21.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.159+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39a400a5a0 con 0x7f399c02d020 2023-10-05T19:40:21.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.159+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737f00 con 0x55eb4872bac0 2023-10-05T19:40:21.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.159+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39a400a5a0 con 0x55eb4872bac0 2023-10-05T19:40:21.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.159+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737f00 con 0x55eb4872bac0 2023-10-05T19:40:21.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.159+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39a400a5a0 con 0x55eb4872bac0 2023-10-05T19:40:21.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.159+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737f00 con 0x7f399c02d020 2023-10-05T19:40:21.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.163+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39a400a5a0 con 0x7f399c02d020 2023-10-05T19:40:21.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.163+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737f00 con 0x7f399c02d020 2023-10-05T19:40:21.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.163+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39a400a5a0 con 0x7f399c02d020 2023-10-05T19:40:21.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.163+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737f00 con 0x55eb4872ff80 2023-10-05T19:40:21.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.163+0000 7f39a95ca700 1 -- 172.21.15.12:0/1282781615 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f399c014070 con 0x55eb4872ff80 2023-10-05T19:40:21.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.163+0000 7f39a95ca700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737f00 con 0x55eb48734900 2023-10-05T19:40:21.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.163+0000 7f39a8dc9700 1 -- 172.21.15.12:0/1282781615 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3994012450 con 0x55eb48734900 2023-10-05T19:40:21.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.163+0000 7f39a8dc9700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737f00 con 0x55eb4872bac0 2023-10-05T19:40:21.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.167+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39a400a5a0 con 0x55eb4872bac0 2023-10-05T19:40:21.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.167+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737f00 con 0x7f399c02d020 2023-10-05T19:40:21.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.167+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39a400a5a0 con 0x7f399c02d020 2023-10-05T19:40:21.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.167+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737f00 con 0x55eb4872bac0 2023-10-05T19:40:21.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.167+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39a400a5a0 con 0x55eb4872bac0 2023-10-05T19:40:21.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.167+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737f00 con 0x55eb4872ff80 2023-10-05T19:40:21.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.167+0000 7f39a95ca700 1 -- 172.21.15.12:0/1282781615 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f399c014070 con 0x55eb4872ff80 2023-10-05T19:40:21.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.167+0000 7f39a95ca700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737f00 con 0x55eb48734900 2023-10-05T19:40:21.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.171+0000 7f39a8dc9700 1 -- 172.21.15.12:0/1282781615 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3994012450 con 0x55eb48734900 2023-10-05T19:40:21.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.171+0000 7f39a8dc9700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737f00 con 0x55eb48734900 2023-10-05T19:40:21.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.171+0000 7f39a8dc9700 1 -- 172.21.15.12:0/1282781615 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3994012450 con 0x55eb48734900 2023-10-05T19:40:21.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.171+0000 7f39a8dc9700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737f00 con 0x7f399c02d020 2023-10-05T19:40:21.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.171+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39a400a5a0 con 0x7f399c02d020 2023-10-05T19:40:21.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.171+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737f00 con 0x55eb48734900 2023-10-05T19:40:21.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.171+0000 7f39a8dc9700 1 -- 172.21.15.12:0/1282781615 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3994012450 con 0x55eb48734900 2023-10-05T19:40:21.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.171+0000 7f39a8dc9700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737f00 con 0x55eb48734900 2023-10-05T19:40:21.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.171+0000 7f39a8dc9700 1 -- 172.21.15.12:0/1282781615 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3994012450 con 0x55eb48734900 2023-10-05T19:40:21.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.171+0000 7f39a8dc9700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737f00 con 0x55eb4872bac0 2023-10-05T19:40:21.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.175+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39a400a5a0 con 0x55eb4872bac0 2023-10-05T19:40:21.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.175+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737f00 con 0x55eb4872bac0 2023-10-05T19:40:21.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.175+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39a400a5a0 con 0x55eb4872bac0 2023-10-05T19:40:21.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.175+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737f00 con 0x7f399c02d020 2023-10-05T19:40:21.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.175+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39a400a5a0 con 0x7f399c02d020 2023-10-05T19:40:21.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.175+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737f00 con 0x55eb48734900 2023-10-05T19:40:21.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.175+0000 7f39a8dc9700 1 -- 172.21.15.12:0/1282781615 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3994012450 con 0x55eb48734900 2023-10-05T19:40:21.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.175+0000 7f39a8dc9700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737f00 con 0x55eb4872ff80 2023-10-05T19:40:21.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.179+0000 7f39a95ca700 1 -- 172.21.15.12:0/1282781615 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f399c014070 con 0x55eb4872ff80 2023-10-05T19:40:21.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.179+0000 7f39a95ca700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737f00 con 0x55eb4872bac0 2023-10-05T19:40:21.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.179+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39a400a5a0 con 0x55eb4872bac0 2023-10-05T19:40:21.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.179+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737f00 con 0x7f399c02d020 2023-10-05T19:40:21.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.179+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39a400a5a0 con 0x7f399c02d020 2023-10-05T19:40:21.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.179+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737f00 con 0x55eb48734900 2023-10-05T19:40:21.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.179+0000 7f39a8dc9700 1 -- 172.21.15.12:0/1282781615 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3994012450 con 0x55eb48734900 2023-10-05T19:40:21.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.179+0000 7f39a8dc9700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737f00 con 0x7f399c02d020 2023-10-05T19:40:21.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.183+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39a400a5a0 con 0x7f399c02d020 2023-10-05T19:40:21.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.183+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737f00 con 0x55eb4872ff80 2023-10-05T19:40:21.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.183+0000 7f39a95ca700 1 -- 172.21.15.12:0/1282781615 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f399c014070 con 0x55eb4872ff80 2023-10-05T19:40:21.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.183+0000 7f39a95ca700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737f00 con 0x55eb48734900 2023-10-05T19:40:21.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.183+0000 7f39a8dc9700 1 -- 172.21.15.12:0/1282781615 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3994012450 con 0x55eb48734900 2023-10-05T19:40:21.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.183+0000 7f39a8dc9700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737f00 con 0x7f399c02d020 2023-10-05T19:40:21.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.183+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39a400a5a0 con 0x7f399c02d020 2023-10-05T19:40:21.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.183+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737f00 con 0x55eb4872bac0 2023-10-05T19:40:21.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.187+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f39a400a5a0 con 0x55eb4872bac0 2023-10-05T19:40:21.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.187+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48737f00 con 0x55eb4872ff80 2023-10-05T19:40:21.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.187+0000 7f39a95ca700 1 -- 172.21.15.12:0/1282781615 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f399c014070 con 0x55eb4872ff80 2023-10-05T19:40:21.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.187+0000 7f39aba95e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:21.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.187+0000 7f39aba95e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:21.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.187+0000 7f39aba95e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:21.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.187+0000 7f39aba95e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:21.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.187+0000 7f39aba95e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.187+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb487382b0 con 0x55eb4872bac0 2023-10-05T19:40:21.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.187+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f39a400a5a0 con 0x55eb4872bac0 2023-10-05T19:40:21.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.187+0000 7f39aba95e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:21.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.187+0000 7f39aba95e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.187+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48738660 con 0x55eb4872ff80 2023-10-05T19:40:21.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.187+0000 7f39a95ca700 1 -- 172.21.15.12:0/1282781615 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f399c014070 con 0x55eb4872ff80 2023-10-05T19:40:21.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.187+0000 7f39aba95e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:21.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.187+0000 7f39aba95e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.187+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48738f00 con 0x55eb4872bac0 2023-10-05T19:40:21.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.187+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f39a400a5a0 con 0x55eb4872bac0 2023-10-05T19:40:21.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.187+0000 7f39aba95e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:21.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.187+0000 7f39aba95e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.187+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48738f00 con 0x55eb48734900 2023-10-05T19:40:21.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.191+0000 7f39a8dc9700 1 -- 172.21.15.12:0/1282781615 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f3994012450 con 0x55eb48734900 2023-10-05T19:40:21.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.191+0000 7f39aba95e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:21.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.191+0000 7f39aba95e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.191+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb487392b0 con 0x7f399c02d020 2023-10-05T19:40:21.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.191+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f39a400a5a0 con 0x7f399c02d020 2023-10-05T19:40:21.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.191+0000 7f39aba95e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:21.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.191+0000 7f39aba95e40 20 zone default found 2023-10-05T19:40:21.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.191+0000 7f39aba95e40 4 Realm: () 2023-10-05T19:40:21.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.191+0000 7f39aba95e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:21.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.191+0000 7f39aba95e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:21.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.191+0000 7f39aba95e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:21.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.191+0000 7f39aba95e40 20 zonegroup default 2023-10-05T19:40:21.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.191+0000 7f39aba95e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.191+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb487392b0 con 0x7f399c02d020 2023-10-05T19:40:21.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.191+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f39a400a5a0 con 0x7f399c02d020 2023-10-05T19:40:21.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.191+0000 7f39aba95e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:21.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.191+0000 7f39aba95e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.191+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb4873ac30 con 0x55eb48734900 2023-10-05T19:40:21.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.191+0000 7f39a8dc9700 1 -- 172.21.15.12:0/1282781615 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f3994012450 con 0x55eb48734900 2023-10-05T19:40:21.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.191+0000 7f39aba95e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:21.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.191+0000 7f39aba95e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.191+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb4873afe0 con 0x55eb48734900 2023-10-05T19:40:21.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.195+0000 7f39a8dc9700 1 -- 172.21.15.12:0/1282781615 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f3994012450 con 0x55eb48734900 2023-10-05T19:40:21.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.195+0000 7f39aba95e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:21.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.195+0000 7f39aba95e40 20 started sync module instance, tier type = 2023-10-05T19:40:21.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.195+0000 7f39aba95e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:21.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.195+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb4873a2f0 con 0x55eb4872ff80 2023-10-05T19:40:21.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.195+0000 7f39a95ca700 1 -- 172.21.15.12:0/1282781615 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'63 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f399c014070 con 0x55eb4872ff80 2023-10-05T19:40:21.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.195+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94469021214448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48738b30 con 0x55eb4872ff80 2023-10-05T19:40:21.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.195+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb4873bd70 con 0x55eb4872bac0 2023-10-05T19:40:21.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.199+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'63 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39a400a5a0 con 0x55eb4872bac0 2023-10-05T19:40:21.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.199+0000 7f39a95ca700 1 -- 172.21.15.12:0/1282781615 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94469021214448] v28'64 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f399c014070 con 0x55eb4872ff80 2023-10-05T19:40:21.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.199+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94469021219232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb4873c3f0 con 0x55eb4872bac0 2023-10-05T19:40:21.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.199+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb4873cd10 con 0x55eb48734900 2023-10-05T19:40:21.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.203+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94469021219232] v28'64 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39a400a5a0 con 0x55eb4872bac0 2023-10-05T19:40:21.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.203+0000 7f39a8dc9700 1 -- 172.21.15.12:0/1282781615 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'63 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3994012450 con 0x55eb48734900 2023-10-05T19:40:21.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.203+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94469021222896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb4873c370 con 0x55eb48734900 2023-10-05T19:40:21.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.203+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb4873d720 con 0x55eb4872ff80 2023-10-05T19:40:21.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.207+0000 7f39a8dc9700 1 -- 172.21.15.12:0/1282781615 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94469021222896] v28'64 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3994012450 con 0x55eb48734900 2023-10-05T19:40:21.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.207+0000 7f39a95ca700 1 -- 172.21.15.12:0/1282781615 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'63 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f399c014070 con 0x55eb4872ff80 2023-10-05T19:40:21.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.207+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94469021225520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb4873bfa0 con 0x55eb4872ff80 2023-10-05T19:40:21.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.207+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb4873e500 con 0x55eb4872bac0 2023-10-05T19:40:21.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.207+0000 7f39a95ca700 1 -- 172.21.15.12:0/1282781615 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94469021225520] v28'64 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f399c014070 con 0x55eb4872ff80 2023-10-05T19:40:21.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.207+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'63 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39a400a5a0 con 0x55eb4872bac0 2023-10-05T19:40:21.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.207+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94469021227808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb4873e460 con 0x55eb4872bac0 2023-10-05T19:40:21.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.207+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb4873ed00 con 0x7f399c02d020 2023-10-05T19:40:21.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.211+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94469021227808] v28'64 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39a400a5a0 con 0x55eb4872bac0 2023-10-05T19:40:21.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.211+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'63 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39a400a5a0 con 0x7f399c02d020 2023-10-05T19:40:21.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.211+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94469021229472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb4873eae0 con 0x7f399c02d020 2023-10-05T19:40:21.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.211+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb4873f760 con 0x7f399c02d020 2023-10-05T19:40:21.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.215+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(95 notify.5 [watch watch cookie 94469021229472] v28'64 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39a400a5a0 con 0x7f399c02d020 2023-10-05T19:40:21.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.215+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(96 notify.6 [create] v28'63 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39a400a5a0 con 0x7f399c02d020 2023-10-05T19:40:21.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.215+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94469021231136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb4873f760 con 0x7f399c02d020 2023-10-05T19:40:21.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.215+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48740170 con 0x7f399c02d020 2023-10-05T19:40:21.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.215+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(98 notify.7 [create] v28'63 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39a400a5a0 con 0x7f399c02d020 2023-10-05T19:40:21.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.215+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94469021236352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb487405c0 con 0x7f399c02d020 2023-10-05T19:40:21.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.215+0000 7f39aba95e40 20 add_watcher() i=0 2023-10-05T19:40:21.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.215+0000 7f39aba95e40 20 add_watcher() i=1 2023-10-05T19:40:21.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.215+0000 7f39aba95e40 20 add_watcher() i=2 2023-10-05T19:40:21.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.215+0000 7f39aba95e40 20 add_watcher() i=3 2023-10-05T19:40:21.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.215+0000 7f39aba95e40 20 add_watcher() i=4 2023-10-05T19:40:21.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.215+0000 7f39aba95e40 20 add_watcher() i=5 2023-10-05T19:40:21.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.219+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(97 notify.6 [watch watch cookie 94469021231136] v28'64 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39a400a5a0 con 0x7f399c02d020 2023-10-05T19:40:21.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.219+0000 7f39aba95e40 20 add_watcher() i=6 2023-10-05T19:40:21.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.219+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94469021236352] v28'64 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39a400a5a0 con 0x7f399c02d020 2023-10-05T19:40:21.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.219+0000 7f39aba95e40 20 add_watcher() i=7 2023-10-05T19:40:21.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.219+0000 7f39aba95e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:21.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.219+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb487435f0 con 0x55eb4872bac0 2023-10-05T19:40:21.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.219+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f39a400a5a0 con 0x55eb4872bac0 2023-10-05T19:40:21.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.219+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94469021237456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb48740a10 con 0x55eb4872bac0 2023-10-05T19:40:21.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.223+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94469021237456] v28'76 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f39a400a5a0 con 0x55eb4872bac0 2023-10-05T19:40:21.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.223+0000 7f397a7e4700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:21.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.223+0000 7f39aba95e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:21.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.223+0000 7f39aba95e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:21.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.223+0000 7f39aba95e40 1 Processor -- start 2023-10-05T19:40:21.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.223+0000 7f39aba95e40 1 -- start start 2023-10-05T19:40:21.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.223+0000 7f39aba95e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eb4874d720 0x55eb4874daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.223+0000 7f39aba95e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb4874e030 con 0x55eb4874d720 2023-10-05T19:40:21.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.223+0000 7f39a8dc9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eb4874d720 0x55eb4874daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.223+0000 7f39a8dc9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eb4874d720 0x55eb4874daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37234/0 (socket says 172.21.15.12:37234) 2023-10-05T19:40:21.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.223+0000 7f39a8dc9700 1 -- 172.21.15.12:0/2793225051 learned_addr learned my addr 172.21.15.12:0/2793225051 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:21.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.223+0000 7f39a8dc9700 1 -- 172.21.15.12:0/2793225051 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb4874e170 con 0x55eb4874d720 2023-10-05T19:40:21.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.223+0000 7f39a8dc9700 1 --2- 172.21.15.12:0/2793225051 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eb4874d720 0x55eb4874daf0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x55eb486fd0a0 tx=0x7f3994012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cff4cf906300a5f server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.227+0000 7f3979fe3700 1 -- 172.21.15.12:0/2793225051 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3994012d80 con 0x55eb4874d720 2023-10-05T19:40:21.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.227+0000 7f3979fe3700 1 -- 172.21.15.12:0/2793225051 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3994036040 con 0x55eb4874d720 2023-10-05T19:40:21.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.227+0000 7f3979fe3700 1 -- 172.21.15.12:0/2793225051 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3994030b90 con 0x55eb4874d720 2023-10-05T19:40:21.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.227+0000 7f39aba95e40 1 -- 172.21.15.12:0/2793225051 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eb4874d720 msgr2=0x55eb4874daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.227+0000 7f39aba95e40 1 --2- 172.21.15.12:0/2793225051 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eb4874d720 0x55eb4874daf0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x55eb486fd0a0 tx=0x7f3994012000 comp rx=0 tx=0).stop 2023-10-05T19:40:21.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.227+0000 7f39aba95e40 1 -- 172.21.15.12:0/2793225051 shutdown_connections 2023-10-05T19:40:21.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.227+0000 7f39aba95e40 1 --2- 172.21.15.12:0/2793225051 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eb4874d720 0x55eb4874daf0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.227+0000 7f39aba95e40 1 -- 172.21.15.12:0/2793225051 >> 172.21.15.12:0/2793225051 conn(0x55eb48748ae0 msgr2=0x55eb48746c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:21.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.227+0000 7f39aba95e40 1 -- 172.21.15.12:0/2793225051 shutdown_connections 2023-10-05T19:40:21.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.227+0000 7f39aba95e40 1 -- 172.21.15.12:0/2793225051 wait complete. 2023-10-05T19:40:21.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.227+0000 7f39aba95e40 1 Processor -- start 2023-10-05T19:40:21.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.227+0000 7f39aba95e40 1 -- start start 2023-10-05T19:40:21.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.227+0000 7f39aba95e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eb4874d720 0x55eb48753180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.227+0000 7f39aba95e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb48754180 con 0x55eb4874d720 2023-10-05T19:40:21.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.227+0000 7f39a8dc9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eb4874d720 0x55eb48753180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.227+0000 7f39a8dc9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eb4874d720 0x55eb48753180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37242/0 (socket says 172.21.15.12:37242) 2023-10-05T19:40:21.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.227+0000 7f39a8dc9700 1 -- 172.21.15.12:0/2245448875 learned_addr learned my addr 172.21.15.12:0/2245448875 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:21.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.227+0000 7f39a8dc9700 1 -- 172.21.15.12:0/2245448875 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb486fcc30 con 0x55eb4874d720 2023-10-05T19:40:21.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.227+0000 7f39a8dc9700 1 --2- 172.21.15.12:0/2245448875 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eb4874d720 0x55eb48753180 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x55eb48747800 tx=0x7f3994037000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.231+0000 7f38ebfff700 1 -- 172.21.15.12:0/2245448875 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3994027ea0 con 0x55eb4874d720 2023-10-05T19:40:21.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.231+0000 7f38ebfff700 1 -- 172.21.15.12:0/2245448875 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3994012e00 con 0x55eb4874d720 2023-10-05T19:40:21.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.231+0000 7f38ebfff700 1 -- 172.21.15.12:0/2245448875 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f399404c3f0 con 0x55eb4874d720 2023-10-05T19:40:21.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.231+0000 7f39aba95e40 1 -- 172.21.15.12:0/2245448875 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55eb48753780 con 0x55eb4874d720 2023-10-05T19:40:21.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.231+0000 7f39aba95e40 1 -- 172.21.15.12:0/2245448875 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eb48753c20 con 0x55eb4874d720 2023-10-05T19:40:21.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.231+0000 7f38ebfff700 1 -- 172.21.15.12:0/2245448875 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f3994055440 con 0x55eb4874d720 2023-10-05T19:40:21.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.231+0000 7f38ebfff700 1 --2- 172.21.15.12:0/2245448875 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f38f0030c00 0x7f38f00330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.231+0000 7f38ebfff700 1 -- 172.21.15.12:0/2245448875 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f399404dd90 con 0x55eb4874d720 2023-10-05T19:40:21.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.231+0000 7f39a9dcb700 1 --2- 172.21.15.12:0/2245448875 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f38f0030c00 0x7f38f00330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.231+0000 7f39aba95e40 5 note: GC not initialized 2023-10-05T19:40:21.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.231+0000 7f39a9dcb700 1 --2- 172.21.15.12:0/2245448875 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f38f0030c00 0x7f38f00330b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x55eb4873f550 tx=0x7f39a400ae20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.235+0000 7f38e97fa700 20 reqs_thread_entry: start 2023-10-05T19:40:21.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.235+0000 7f39aba95e40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:21.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.235+0000 7f38e2ffd700 20 reqs_thread_entry: start 2023-10-05T19:40:21.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.235+0000 7f39aba95e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:21.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.235+0000 7f38e17fa700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:51 2023 2023-10-05T19:40:21.244 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:21.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.235+0000 7f39aba95e40 20 get_system_obj_state: rctx=0x7ffe6ebb4818 obj=default.rgw.meta:users.uid:foo state=0x55eb485d30b0 s->prefetch_data=0 2023-10-05T19:40:21.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.235+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb487bba50 con 0x55eb4872bac0 2023-10-05T19:40:21.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.235+0000 7f38e0ff9700 20 reqs_thread_entry: start 2023-10-05T19:40:21.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.235+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(102 foo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7f39a400a5a0 con 0x55eb4872bac0 2023-10-05T19:40:21.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.235+0000 7f39aba95e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:21.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.235+0000 7f39aba95e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:21.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.235+0000 7f39aba95e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.235+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb487be4e0 con 0x55eb4872bac0 2023-10-05T19:40:21.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.235+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(103 foo [call,call out=48b,read 0~513 out=513b,getxattrs out=75b] v0'0 uv7 ondisk = 0) v8 ==== 273+0+636 (crc 0 0 0) 0x7f39a400a5a0 con 0x55eb4872bac0 2023-10-05T19:40:21.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.235+0000 7f39aba95e40 20 rados_obj.operate() r=0 bl.length=513 2023-10-05T19:40:21.246 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:40:21.246 INFO:teuthology.orchestra.run.smithi012.stdout: "user_id": "foo", 2023-10-05T19:40:21.246 INFO:teuthology.orchestra.run.smithi012.stdout: "display_name": "Foo", 2023-10-05T19:40:21.246 INFO:teuthology.orchestra.run.smithi012.stdout: "email": "foo@foo.com", 2023-10-05T19:40:21.246 INFO:teuthology.orchestra.run.smithi012.stdout: "suspended": 0, 2023-10-05T19:40:21.246 INFO:teuthology.orchestra.run.smithi012.stdout: "max_buckets": 4, 2023-10-05T19:40:21.246 INFO:teuthology.orchestra.run.smithi012.stdout: "subusers": [ 2023-10-05T19:40:21.247 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:21.247 INFO:teuthology.orchestra.run.smithi012.stdout: "id": "foo:foo1", 2023-10-05T19:40:21.247 INFO:teuthology.orchestra.run.smithi012.stdout: "permissions": "full-control" 2023-10-05T19:40:21.247 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:21.247 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:21.247 INFO:teuthology.orchestra.run.smithi012.stdout: "keys": [ 2023-10-05T19:40:21.247 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:21.247 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:40:21.247 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2023-10-05T19:40:21.247 INFO:teuthology.orchestra.run.smithi012.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2023-10-05T19:40:21.247 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:21.248 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:21.248 INFO:teuthology.orchestra.run.smithi012.stdout: "swift_keys": [ 2023-10-05T19:40:21.248 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:21.248 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo:foo1", 2023-10-05T19:40:21.248 INFO:teuthology.orchestra.run.smithi012.stdout: "secret_key": "gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL" 2023-10-05T19:40:21.248 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:21.248 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:21.248 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": [], 2023-10-05T19:40:21.248 INFO:teuthology.orchestra.run.smithi012.stdout: "op_mask": "read, write, delete", 2023-10-05T19:40:21.248 INFO:teuthology.orchestra.run.smithi012.stdout: "default_placement": "", 2023-10-05T19:40:21.248 INFO:teuthology.orchestra.run.smithi012.stdout: "default_storage_class": "", 2023-10-05T19:40:21.248 INFO:teuthology.orchestra.run.smithi012.stdout: "placement_tags": [], 2023-10-05T19:40:21.249 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket_quota": { 2023-10-05T19:40:21.249 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:21.249 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:21.249 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:21.249 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:21.249 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:21.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:21.249 INFO:teuthology.orchestra.run.smithi012.stdout: "user_quota": { 2023-10-05T19:40:21.249 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:21.249 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:21.249 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:21.249 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:21.249 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:21.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:21.250 INFO:teuthology.orchestra.run.smithi012.stdout: "temp_url_keys": [], 2023-10-05T19:40:21.250 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "rgw", 2023-10-05T19:40:21.250 INFO:teuthology.orchestra.run.smithi012.stdout: "mfa_ids": [] 2023-10-05T19:40:21.250 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:40:21.250 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:40:21.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.239+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:104 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94469021214448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb487119e0 con 0x55eb4872ff80 2023-10-05T19:40:21.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.239+0000 7f39a95ca700 1 -- 172.21.15.12:0/1282781615 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(104 notify.0 [watch unwatch cookie 94469021214448] v28'65 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f399c014070 con 0x55eb4872ff80 2023-10-05T19:40:21.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.239+0000 7f39aba95e40 20 remove_watcher() i=0 2023-10-05T19:40:21.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.239+0000 7f39aba95e40 2 removed watcher, disabling cache 2023-10-05T19:40:21.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.239+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:105 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94469021219232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb487119e0 con 0x55eb4872bac0 2023-10-05T19:40:21.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.243+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(105 notify.1 [watch unwatch cookie 94469021219232] v28'65 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39a400a5a0 con 0x55eb4872bac0 2023-10-05T19:40:21.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.243+0000 7f39aba95e40 20 remove_watcher() i=1 2023-10-05T19:40:21.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.243+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:106 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94469021222896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb487119e0 con 0x55eb48734900 2023-10-05T19:40:21.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.247+0000 7f39a8dc9700 1 -- 172.21.15.12:0/1282781615 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(106 notify.2 [watch unwatch cookie 94469021222896] v28'65 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3994040020 con 0x55eb48734900 2023-10-05T19:40:21.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.247+0000 7f39aba95e40 20 remove_watcher() i=2 2023-10-05T19:40:21.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.247+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:107 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94469021225520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb487119e0 con 0x55eb4872ff80 2023-10-05T19:40:21.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.247+0000 7f39a95ca700 1 -- 172.21.15.12:0/1282781615 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(107 notify.3 [watch unwatch cookie 94469021225520] v28'65 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f399c014070 con 0x55eb4872ff80 2023-10-05T19:40:21.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.247+0000 7f39aba95e40 20 remove_watcher() i=3 2023-10-05T19:40:21.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.247+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:108 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94469021227808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb487119e0 con 0x55eb4872bac0 2023-10-05T19:40:21.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.251+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(108 notify.4 [watch unwatch cookie 94469021227808] v28'65 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39a400a5a0 con 0x55eb4872bac0 2023-10-05T19:40:21.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.251+0000 7f39aba95e40 20 remove_watcher() i=4 2023-10-05T19:40:21.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.251+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:109 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94469021229472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb487119e0 con 0x7f399c02d020 2023-10-05T19:40:21.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.255+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(109 notify.5 [watch unwatch cookie 94469021229472] v28'65 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39a400a5a0 con 0x7f399c02d020 2023-10-05T19:40:21.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.255+0000 7f39aba95e40 20 remove_watcher() i=5 2023-10-05T19:40:21.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.255+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:110 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94469021231136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb487119e0 con 0x7f399c02d020 2023-10-05T19:40:21.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.255+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(110 notify.6 [watch unwatch cookie 94469021231136] v28'65 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39a400a5a0 con 0x7f399c02d020 2023-10-05T19:40:21.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.255+0000 7f39aba95e40 20 remove_watcher() i=6 2023-10-05T19:40:21.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.255+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:111 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94469021236352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb487119e0 con 0x7f399c02d020 2023-10-05T19:40:21.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.259+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(111 notify.7 [watch unwatch cookie 94469021236352] v28'65 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39a400a5a0 con 0x7f399c02d020 2023-10-05T19:40:21.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.259+0000 7f39aba95e40 20 remove_watcher() i=7 2023-10-05T19:40:21.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.263+0000 7f397a7e4700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:21.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.263+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:112 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94469021237456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55eb487119e0 con 0x55eb4872bac0 2023-10-05T19:40:21.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.263+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(112 data_loggenerations_metadata [watch unwatch cookie 94469021237456] v28'77 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f39a400a5a0 con 0x55eb4872bac0 2023-10-05T19:40:21.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.263+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f399c02d020 msgr2=0x7f399c03d790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.263+0000 7f39aba95e40 1 --2- 172.21.15.12:0/1282781615 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f399c02d020 0x7f399c03d790 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.263+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55eb4872bac0 msgr2=0x55eb4872dea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.263+0000 7f39aba95e40 1 --2- 172.21.15.12:0/1282781615 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55eb4872bac0 0x55eb4872dea0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.263+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55eb4872ff80 msgr2=0x55eb48732360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.263+0000 7f39aba95e40 1 --2- 172.21.15.12:0/1282781615 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55eb4872ff80 0x55eb48732360 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.263+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55eb48734900 msgr2=0x55eb48736ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.263+0000 7f39aba95e40 1 --2- 172.21.15.12:0/1282781615 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55eb48734900 0x55eb48736ce0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.263+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f398c030bb0 msgr2=0x7f398c033060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.263+0000 7f39aba95e40 1 --2- 172.21.15.12:0/1282781615 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f398c030bb0 0x7f398c033060 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3994005a90 tx=0x7f3994015000 comp rx=0 tx=0).stop 2023-10-05T19:40:21.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.263+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eb48700600 msgr2=0x55eb48722040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.263+0000 7f39aba95e40 1 --2- 172.21.15.12:0/1282781615 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eb48700600 0x55eb48722040 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x55eb487177e0 tx=0x7f399c016000 comp rx=0 tx=0).stop 2023-10-05T19:40:21.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.263+0000 7f39a9dcb700 1 -- 172.21.15.12:0/1282781615 reap_dead start 2023-10-05T19:40:21.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.263+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 shutdown_connections 2023-10-05T19:40:21.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.263+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 >> 172.21.15.12:0/1282781615 conn(0x55eb4866f870 msgr2=0x55eb48715740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:21.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.267+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 shutdown_connections 2023-10-05T19:40:21.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.267+0000 7f39aba95e40 1 -- 172.21.15.12:0/1282781615 wait complete. 2023-10-05T19:40:21.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.267+0000 7f39aba95e40 1 -- 172.21.15.12:0/2245448875 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f38f0030c00 msgr2=0x7f38f00330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.267+0000 7f39aba95e40 1 --2- 172.21.15.12:0/2245448875 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f38f0030c00 0x7f38f00330b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x55eb4873f550 tx=0x7f39a400ae20 comp rx=0 tx=0).stop 2023-10-05T19:40:21.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.267+0000 7f39aba95e40 1 -- 172.21.15.12:0/2245448875 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eb4874d720 msgr2=0x55eb48753180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.267+0000 7f39aba95e40 1 --2- 172.21.15.12:0/2245448875 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eb4874d720 0x55eb48753180 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x55eb48747800 tx=0x7f3994037000 comp rx=0 tx=0).stop 2023-10-05T19:40:21.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.267+0000 7f39aba95e40 1 -- 172.21.15.12:0/2245448875 shutdown_connections 2023-10-05T19:40:21.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.267+0000 7f39aba95e40 1 --2- 172.21.15.12:0/2245448875 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f38f0030c00 0x7f38f00330b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.267+0000 7f39aba95e40 1 --2- 172.21.15.12:0/2245448875 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eb4874d720 0x55eb48753180 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.267+0000 7f39aba95e40 1 -- 172.21.15.12:0/2245448875 >> 172.21.15.12:0/2245448875 conn(0x55eb48748ae0 msgr2=0x55eb48746be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:21.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.267+0000 7f39aba95e40 1 -- 172.21.15.12:0/2245448875 shutdown_connections 2023-10-05T19:40:21.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.267+0000 7f39aba95e40 1 -- 172.21.15.12:0/2245448875 wait complete. 2023-10-05T19:40:21.280 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:40:21.280 INFO:tasks.util.rgw:rgwadmin: client.0 : ['subuser', 'create', '--subuser', 'foo:foo2', '--secret', 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', '--key-type', 'swift'] 2023-10-05T19:40:21.280 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'subuser', 'create', '--subuser', 'foo:foo2', '--secret', 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', '--key-type', 'swift'] 2023-10-05T19:40:21.281 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph subuser create --subuser foo:foo2 --secret ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy --key-type swift 2023-10-05T19:40:21.326 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:21.326 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:21.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.319+0000 7f4177e7de40 1 Processor -- start 2023-10-05T19:40:21.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.319+0000 7f4177e7de40 1 -- start start 2023-10-05T19:40:21.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.319+0000 7f4177e7de40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55e223275dd0 0x55e2232761a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.319+0000 7f4177e7de40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55e2231c2bd0 con 0x55e2232766e0 2023-10-05T19:40:21.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.319+0000 7f4177e7de40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x55e22317fee0 con 0x55e223275dd0 2023-10-05T19:40:21.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.319+0000 7f41759b2700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55e223275dd0 0x55e2232761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.319+0000 7f41751b1700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x55e2232766e0 0x55e223277180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:47334/0 (socket says 172.21.15.12:47334) 2023-10-05T19:40:21.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.319+0000 7f41751b1700 1 -- 172.21.15.12:0/2522263322 learned_addr learned my addr 172.21.15.12:0/2522263322 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:21.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.319+0000 7f41749b0700 1 -- 172.21.15.12:0/2522263322 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4089514712 0 0) 0x55e2231c2bd0 con 0x55e2232766e0 2023-10-05T19:40:21.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.319+0000 7f41749b0700 1 -- 172.21.15.12:0/2522263322 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4164001e10 con 0x55e2232766e0 2023-10-05T19:40:21.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.319+0000 7f41759b2700 1 -- 172.21.15.12:0/2522263322 >> v1:172.21.15.8:6789/0 conn(0x55e2232766e0 legacy=0x55e223277180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.319+0000 7f41759b2700 1 -- 172.21.15.12:0/2522263322 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e223277890 con 0x55e223275dd0 2023-10-05T19:40:21.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.319+0000 7f41759b2700 1 --2- 172.21.15.12:0/2522263322 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e223275dd0 0x55e2232761a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f416800a520 tx=0x7f4168005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbedd5f0a73d4ab0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.319+0000 7f41749b0700 1 -- 172.21.15.12:0/2522263322 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f416800f590 con 0x55e223275dd0 2023-10-05T19:40:21.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.319+0000 7f41749b0700 1 -- 172.21.15.12:0/2522263322 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f416800f6f0 con 0x55e223275dd0 2023-10-05T19:40:21.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.319+0000 7f41749b0700 1 -- 172.21.15.12:0/2522263322 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f416800f860 con 0x55e223275dd0 2023-10-05T19:40:21.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.319+0000 7f4177e7de40 1 -- 172.21.15.12:0/2522263322 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e223275dd0 msgr2=0x55e2232761a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.319+0000 7f4177e7de40 1 --2- 172.21.15.12:0/2522263322 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e223275dd0 0x55e2232761a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f416800a520 tx=0x7f4168005d40 comp rx=0 tx=0).stop 2023-10-05T19:40:21.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.323+0000 7f4177e7de40 1 -- 172.21.15.12:0/2522263322 shutdown_connections 2023-10-05T19:40:21.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.323+0000 7f4177e7de40 1 --2- 172.21.15.12:0/2522263322 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e223275dd0 0x55e2232761a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.323+0000 7f4177e7de40 1 -- 172.21.15.12:0/2522263322 >> 172.21.15.12:0/2522263322 conn(0x55e2232717f0 msgr2=0x55e223273c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:21.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.323+0000 7f4177e7de40 1 -- 172.21.15.12:0/2522263322 shutdown_connections 2023-10-05T19:40:21.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.323+0000 7f4177e7de40 1 -- 172.21.15.12:0/2522263322 wait complete. 2023-10-05T19:40:21.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.323+0000 7f4177e7de40 1 Processor -- start 2023-10-05T19:40:21.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.323+0000 7f4177e7de40 1 -- start start 2023-10-05T19:40:21.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.323+0000 7f4177e7de40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e223275dd0 0x55e2231eb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.323+0000 7f4177e7de40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e223143e30 con 0x55e223275dd0 2023-10-05T19:40:21.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.323+0000 7f41759b2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e223275dd0 0x55e2231eb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.323+0000 7f41759b2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e223275dd0 0x55e2231eb880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37268/0 (socket says 172.21.15.12:37268) 2023-10-05T19:40:21.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.323+0000 7f41759b2700 1 -- 172.21.15.12:0/1747426509 learned_addr learned my addr 172.21.15.12:0/1747426509 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:21.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.323+0000 7f41759b2700 1 -- 172.21.15.12:0/1747426509 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f416800a0e0 con 0x55e223275dd0 2023-10-05T19:40:21.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.323+0000 7f41759b2700 1 --2- 172.21.15.12:0/1747426509 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e223275dd0 0x55e2231eb880 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f416800a470 tx=0x7f4168014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e46a2948d9d3c4e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.323+0000 7f416f7fe700 1 -- 172.21.15.12:0/1747426509 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f416800f860 con 0x55e223275dd0 2023-10-05T19:40:21.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.323+0000 7f416f7fe700 1 -- 172.21.15.12:0/1747426509 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f416800f9c0 con 0x55e223275dd0 2023-10-05T19:40:21.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.323+0000 7f416f7fe700 1 -- 172.21.15.12:0/1747426509 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f416800fb90 con 0x55e223275dd0 2023-10-05T19:40:21.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.323+0000 7f4177e7de40 1 -- 172.21.15.12:0/1747426509 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e223275dd0 msgr2=0x55e2231eb880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.323+0000 7f4177e7de40 1 --2- 172.21.15.12:0/1747426509 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e223275dd0 0x55e2231eb880 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f416800a470 tx=0x7f4168014000 comp rx=0 tx=0).stop 2023-10-05T19:40:21.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.323+0000 7f4177e7de40 1 -- 172.21.15.12:0/1747426509 shutdown_connections 2023-10-05T19:40:21.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.323+0000 7f4177e7de40 1 --2- 172.21.15.12:0/1747426509 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e223275dd0 0x55e2231eb880 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.323+0000 7f4177e7de40 1 -- 172.21.15.12:0/1747426509 >> 172.21.15.12:0/1747426509 conn(0x55e2232717f0 msgr2=0x55e223290870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:21.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.323+0000 7f4177e7de40 1 -- 172.21.15.12:0/1747426509 shutdown_connections 2023-10-05T19:40:21.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.323+0000 7f4177e7de40 1 -- 172.21.15.12:0/1747426509 wait complete. 2023-10-05T19:40:21.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.327+0000 7f4177e7de40 1 Processor -- start 2023-10-05T19:40:21.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.327+0000 7f4177e7de40 1 -- start start 2023-10-05T19:40:21.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.327+0000 7f4177e7de40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e223275dd0 0x55e22329db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.327+0000 7f4177e7de40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e223116ed0 con 0x55e223275dd0 2023-10-05T19:40:21.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.327+0000 7f41759b2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e223275dd0 0x55e22329db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.327+0000 7f41759b2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e223275dd0 0x55e22329db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37280/0 (socket says 172.21.15.12:37280) 2023-10-05T19:40:21.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.327+0000 7f41759b2700 1 -- 172.21.15.12:0/71787349 learned_addr learned my addr 172.21.15.12:0/71787349 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:21.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.327+0000 7f41759b2700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e223277890 con 0x55e223275dd0 2023-10-05T19:40:21.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.327+0000 7f41759b2700 1 --2- 172.21.15.12:0/71787349 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e223275dd0 0x55e22329db80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f416800adc0 tx=0x7f416801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.327+0000 7f416dffb700 1 -- 172.21.15.12:0/71787349 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f416802f430 con 0x55e223275dd0 2023-10-05T19:40:21.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.327+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e22329e0c0 con 0x55e223275dd0 2023-10-05T19:40:21.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.327+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e22329e560 con 0x55e223275dd0 2023-10-05T19:40:21.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.327+0000 7f416dffb700 1 -- 172.21.15.12:0/71787349 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f416802f5b0 con 0x55e223275dd0 2023-10-05T19:40:21.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.327+0000 7f416dffb700 1 -- 172.21.15.12:0/71787349 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f416802f760 con 0x55e223275dd0 2023-10-05T19:40:21.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.331+0000 7f4177e7de40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.331+0000 7f416dffb700 1 -- 172.21.15.12:0/71787349 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f4168010420 con 0x55e223275dd0 2023-10-05T19:40:21.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.331+0000 7f416dffb700 1 --2- 172.21.15.12:0/71787349 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f4158030c00 0x7f41580330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.331+0000 7f416dffb700 1 -- 172.21.15.12:0/71787349 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f416802b070 con 0x55e223275dd0 2023-10-05T19:40:21.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.331+0000 7f41751b1700 1 --2- 172.21.15.12:0/71787349 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f4158030c00 0x7f41580330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.331+0000 7f4177e7de40 1 --2- 172.21.15.12:0/71787349 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55e2232766e0 0x55e2232a9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.331+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232a9d80 con 0x55e2232766e0 2023-10-05T19:40:21.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.331+0000 7f41761b3700 1 --2- 172.21.15.12:0/71787349 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55e2232766e0 0x55e2232a9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.331+0000 7f41751b1700 1 --2- 172.21.15.12:0/71787349 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f4158030c00 0x7f41580330b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4160004170 tx=0x7f4160001400 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.331+0000 7f41761b3700 1 --2- 172.21.15.12:0/71787349 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55e2232766e0 0x55e2232a9710 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.331+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f417000a500 con 0x55e2232766e0 2023-10-05T19:40:21.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.331+0000 7f4177e7de40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:21.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.331+0000 7f4177e7de40 20 realm 2023-10-05T19:40:21.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.331+0000 7f4177e7de40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.331+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232a6d70 con 0x55e2232766e0 2023-10-05T19:40:21.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.335+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f417000a500 con 0x55e2232766e0 2023-10-05T19:40:21.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.335+0000 7f4177e7de40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:21.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.335+0000 7f4177e7de40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:21.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.335+0000 7f4177e7de40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.335+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232aa8b0 con 0x55e2232766e0 2023-10-05T19:40:21.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.335+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f417000a500 con 0x55e2232766e0 2023-10-05T19:40:21.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.335+0000 7f4177e7de40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:21.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.335+0000 7f4177e7de40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.335+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232aa4c0 con 0x55e2232766e0 2023-10-05T19:40:21.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.335+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f417000a500 con 0x55e2232766e0 2023-10-05T19:40:21.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.335+0000 7f4177e7de40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:21.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.335+0000 7f4177e7de40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.335+0000 7f4177e7de40 1 --2- 172.21.15.12:0/71787349 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55e2232abc50 0x55e2232ae030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.335+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232ae570 con 0x55e2232abc50 2023-10-05T19:40:21.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.335+0000 7f41759b2700 1 --2- 172.21.15.12:0/71787349 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55e2232abc50 0x55e2232ae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.335+0000 7f41759b2700 1 --2- 172.21.15.12:0/71787349 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55e2232abc50 0x55e2232ae030 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.335+0000 7f41759b2700 1 -- 172.21.15.12:0/71787349 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f41680387d0 con 0x55e2232abc50 2023-10-05T19:40:21.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.335+0000 7f4177e7de40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:21.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.335+0000 7f4177e7de40 20 searching for the correct realm 2023-10-05T19:40:21.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.335+0000 7f4177e7de40 1 --2- 172.21.15.12:0/71787349 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55e2232b03f0 0x55e2232b2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.335+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b2eb0 con 0x55e2232b03f0 2023-10-05T19:40:21.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.335+0000 7f41751b1700 1 --2- 172.21.15.12:0/71787349 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55e2232b03f0 0x55e2232b2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.339+0000 7f41751b1700 1 --2- 172.21.15.12:0/71787349 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55e2232b03f0 0x55e2232b2800 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.339+0000 7f41751b1700 1 -- 172.21.15.12:0/71787349 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f4160006070 con 0x55e2232b03f0 2023-10-05T19:40:21.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.339+0000 7f41751b1700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b2eb0 con 0x55e2232abc50 2023-10-05T19:40:21.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.339+0000 7f41759b2700 1 -- 172.21.15.12:0/71787349 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f41680387d0 con 0x55e2232abc50 2023-10-05T19:40:21.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.339+0000 7f41759b2700 1 --2- 172.21.15.12:0/71787349 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f416801abe0 0x7f41680289f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.339+0000 7f41759b2700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b2eb0 con 0x7f416801abe0 2023-10-05T19:40:21.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.339+0000 7f41761b3700 1 --2- 172.21.15.12:0/71787349 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f416801abe0 0x7f41680289f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.343+0000 7f41761b3700 1 --2- 172.21.15.12:0/71787349 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f416801abe0 0x7f41680289f0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.343+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4170014040 con 0x7f416801abe0 2023-10-05T19:40:21.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.343+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b2eb0 con 0x55e2232766e0 2023-10-05T19:40:21.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.343+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4170014040 con 0x55e2232766e0 2023-10-05T19:40:21.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.343+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b2eb0 con 0x55e2232766e0 2023-10-05T19:40:21.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.343+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4170014040 con 0x55e2232766e0 2023-10-05T19:40:21.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.343+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b2eb0 con 0x7f416801abe0 2023-10-05T19:40:21.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.343+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4170014040 con 0x7f416801abe0 2023-10-05T19:40:21.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.343+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b2eb0 con 0x7f416801abe0 2023-10-05T19:40:21.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.343+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4170014040 con 0x7f416801abe0 2023-10-05T19:40:21.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.343+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b2eb0 con 0x55e2232abc50 2023-10-05T19:40:21.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.347+0000 7f41759b2700 1 -- 172.21.15.12:0/71787349 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f41680387d0 con 0x55e2232abc50 2023-10-05T19:40:21.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.347+0000 7f41759b2700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b2eb0 con 0x55e2232b03f0 2023-10-05T19:40:21.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.347+0000 7f41751b1700 1 -- 172.21.15.12:0/71787349 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4160006070 con 0x55e2232b03f0 2023-10-05T19:40:21.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.347+0000 7f41751b1700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b2eb0 con 0x55e2232766e0 2023-10-05T19:40:21.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.347+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4170014040 con 0x55e2232766e0 2023-10-05T19:40:21.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.347+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b2eb0 con 0x7f416801abe0 2023-10-05T19:40:21.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.347+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4170014040 con 0x7f416801abe0 2023-10-05T19:40:21.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.347+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b2eb0 con 0x55e2232766e0 2023-10-05T19:40:21.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.347+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4170014040 con 0x55e2232766e0 2023-10-05T19:40:21.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.347+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b2eb0 con 0x55e2232abc50 2023-10-05T19:40:21.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.351+0000 7f41759b2700 1 -- 172.21.15.12:0/71787349 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f41680387d0 con 0x55e2232abc50 2023-10-05T19:40:21.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.351+0000 7f41759b2700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b2eb0 con 0x55e2232b03f0 2023-10-05T19:40:21.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.351+0000 7f41751b1700 1 -- 172.21.15.12:0/71787349 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4160006070 con 0x55e2232b03f0 2023-10-05T19:40:21.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.351+0000 7f41751b1700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b2eb0 con 0x55e2232b03f0 2023-10-05T19:40:21.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.351+0000 7f41751b1700 1 -- 172.21.15.12:0/71787349 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4160006070 con 0x55e2232b03f0 2023-10-05T19:40:21.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.351+0000 7f41751b1700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b2eb0 con 0x7f416801abe0 2023-10-05T19:40:21.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.351+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4170014040 con 0x7f416801abe0 2023-10-05T19:40:21.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.351+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b2eb0 con 0x55e2232b03f0 2023-10-05T19:40:21.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.351+0000 7f41751b1700 1 -- 172.21.15.12:0/71787349 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4160006070 con 0x55e2232b03f0 2023-10-05T19:40:21.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.351+0000 7f41751b1700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b2eb0 con 0x55e2232b03f0 2023-10-05T19:40:21.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.355+0000 7f41751b1700 1 -- 172.21.15.12:0/71787349 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4160006070 con 0x55e2232b03f0 2023-10-05T19:40:21.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.355+0000 7f41751b1700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b2eb0 con 0x55e2232766e0 2023-10-05T19:40:21.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.355+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4170014040 con 0x55e2232766e0 2023-10-05T19:40:21.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.355+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b2eb0 con 0x55e2232766e0 2023-10-05T19:40:21.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.355+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4170014040 con 0x55e2232766e0 2023-10-05T19:40:21.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.355+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b2eb0 con 0x7f416801abe0 2023-10-05T19:40:21.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.355+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4170014040 con 0x7f416801abe0 2023-10-05T19:40:21.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.355+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b2eb0 con 0x55e2232b03f0 2023-10-05T19:40:21.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.355+0000 7f41751b1700 1 -- 172.21.15.12:0/71787349 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4160006070 con 0x55e2232b03f0 2023-10-05T19:40:21.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.355+0000 7f41751b1700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b2eb0 con 0x55e2232abc50 2023-10-05T19:40:21.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.359+0000 7f41759b2700 1 -- 172.21.15.12:0/71787349 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f41680387d0 con 0x55e2232abc50 2023-10-05T19:40:21.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.359+0000 7f41759b2700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b2eb0 con 0x55e2232766e0 2023-10-05T19:40:21.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.359+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4170014040 con 0x55e2232766e0 2023-10-05T19:40:21.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.359+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b2eb0 con 0x7f416801abe0 2023-10-05T19:40:21.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.359+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4170014040 con 0x7f416801abe0 2023-10-05T19:40:21.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.359+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b2eb0 con 0x55e2232b03f0 2023-10-05T19:40:21.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.359+0000 7f41751b1700 1 -- 172.21.15.12:0/71787349 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4160006070 con 0x55e2232b03f0 2023-10-05T19:40:21.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.359+0000 7f41751b1700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b2eb0 con 0x7f416801abe0 2023-10-05T19:40:21.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.363+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4170014040 con 0x7f416801abe0 2023-10-05T19:40:21.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.363+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b2eb0 con 0x55e2232abc50 2023-10-05T19:40:21.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.363+0000 7f41759b2700 1 -- 172.21.15.12:0/71787349 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f41680387d0 con 0x55e2232abc50 2023-10-05T19:40:21.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.363+0000 7f41759b2700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b2eb0 con 0x55e2232b03f0 2023-10-05T19:40:21.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.363+0000 7f41751b1700 1 -- 172.21.15.12:0/71787349 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4160006070 con 0x55e2232b03f0 2023-10-05T19:40:21.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.363+0000 7f41751b1700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b2eb0 con 0x7f416801abe0 2023-10-05T19:40:21.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.363+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4170014040 con 0x7f416801abe0 2023-10-05T19:40:21.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.363+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b2eb0 con 0x55e2232766e0 2023-10-05T19:40:21.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.363+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f4170014040 con 0x55e2232766e0 2023-10-05T19:40:21.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.363+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b2eb0 con 0x55e2232abc50 2023-10-05T19:40:21.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.367+0000 7f41759b2700 1 -- 172.21.15.12:0/71787349 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f41680387d0 con 0x55e2232abc50 2023-10-05T19:40:21.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.367+0000 7f4177e7de40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:21.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.367+0000 7f4177e7de40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:21.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.367+0000 7f4177e7de40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:21.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.367+0000 7f4177e7de40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:21.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.367+0000 7f4177e7de40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.367+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232af170 con 0x55e2232766e0 2023-10-05T19:40:21.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.367+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4170014040 con 0x55e2232766e0 2023-10-05T19:40:21.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.367+0000 7f4177e7de40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:21.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.367+0000 7f4177e7de40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.367+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232aeda0 con 0x55e2232766e0 2023-10-05T19:40:21.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.367+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f4170014040 con 0x55e2232766e0 2023-10-05T19:40:21.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.367+0000 7f4177e7de40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:21.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.367+0000 7f4177e7de40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.367+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b3320 con 0x55e2232b03f0 2023-10-05T19:40:21.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.367+0000 7f41751b1700 1 -- 172.21.15.12:0/71787349 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f4160006070 con 0x55e2232b03f0 2023-10-05T19:40:21.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.367+0000 7f4177e7de40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:21.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.367+0000 7f4177e7de40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.367+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b36d0 con 0x55e2232b03f0 2023-10-05T19:40:21.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.367+0000 7f41751b1700 1 -- 172.21.15.12:0/71787349 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f4160006070 con 0x55e2232b03f0 2023-10-05T19:40:21.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.367+0000 7f4177e7de40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:21.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.367+0000 7f4177e7de40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.367+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b3a80 con 0x55e2232766e0 2023-10-05T19:40:21.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.371+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4170014040 con 0x55e2232766e0 2023-10-05T19:40:21.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.371+0000 7f4177e7de40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:21.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.371+0000 7f4177e7de40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.371+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232af600 con 0x55e2232766e0 2023-10-05T19:40:21.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.371+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f4170014040 con 0x55e2232766e0 2023-10-05T19:40:21.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.371+0000 7f4177e7de40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:21.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.371+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232af5d0 con 0x55e2232b03f0 2023-10-05T19:40:21.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.371+0000 7f41751b1700 1 -- 172.21.15.12:0/71787349 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f4160006070 con 0x55e2232b03f0 2023-10-05T19:40:21.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.371+0000 7f41751b1700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232af5d0 con 0x55e2232abc50 2023-10-05T19:40:21.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.375+0000 7f41759b2700 1 -- 172.21.15.12:0/71787349 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f41680387d0 con 0x55e2232abc50 2023-10-05T19:40:21.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.375+0000 7f41759b2700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232af5d0 con 0x7f416801abe0 2023-10-05T19:40:21.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.375+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4170014040 con 0x7f416801abe0 2023-10-05T19:40:21.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.375+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b3a80 con 0x55e2232766e0 2023-10-05T19:40:21.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.375+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4170014040 con 0x55e2232766e0 2023-10-05T19:40:21.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.375+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b3a80 con 0x55e2232766e0 2023-10-05T19:40:21.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.375+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4170014040 con 0x55e2232766e0 2023-10-05T19:40:21.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.375+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b3a80 con 0x7f416801abe0 2023-10-05T19:40:21.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.379+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4170014040 con 0x7f416801abe0 2023-10-05T19:40:21.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.379+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b3a80 con 0x7f416801abe0 2023-10-05T19:40:21.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.379+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4170014040 con 0x7f416801abe0 2023-10-05T19:40:21.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.379+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b3a80 con 0x55e2232abc50 2023-10-05T19:40:21.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.379+0000 7f41759b2700 1 -- 172.21.15.12:0/71787349 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f41680387d0 con 0x55e2232abc50 2023-10-05T19:40:21.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.379+0000 7f41759b2700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b3a80 con 0x55e2232b03f0 2023-10-05T19:40:21.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.379+0000 7f41751b1700 1 -- 172.21.15.12:0/71787349 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4160006070 con 0x55e2232b03f0 2023-10-05T19:40:21.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.379+0000 7f41751b1700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b3a80 con 0x55e2232766e0 2023-10-05T19:40:21.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.379+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4170014040 con 0x55e2232766e0 2023-10-05T19:40:21.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.379+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b3a80 con 0x7f416801abe0 2023-10-05T19:40:21.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.383+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4170014040 con 0x7f416801abe0 2023-10-05T19:40:21.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.383+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b3a80 con 0x55e2232766e0 2023-10-05T19:40:21.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.383+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4170014040 con 0x55e2232766e0 2023-10-05T19:40:21.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.383+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b3a80 con 0x55e2232abc50 2023-10-05T19:40:21.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.383+0000 7f41759b2700 1 -- 172.21.15.12:0/71787349 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f41680387d0 con 0x55e2232abc50 2023-10-05T19:40:21.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.383+0000 7f41759b2700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b3a80 con 0x55e2232b03f0 2023-10-05T19:40:21.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.383+0000 7f41751b1700 1 -- 172.21.15.12:0/71787349 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4160006070 con 0x55e2232b03f0 2023-10-05T19:40:21.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.383+0000 7f41751b1700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b3a80 con 0x55e2232b03f0 2023-10-05T19:40:21.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.383+0000 7f41751b1700 1 -- 172.21.15.12:0/71787349 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4160006070 con 0x55e2232b03f0 2023-10-05T19:40:21.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.383+0000 7f41751b1700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b3a80 con 0x7f416801abe0 2023-10-05T19:40:21.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.387+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f417002e040 con 0x7f416801abe0 2023-10-05T19:40:21.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.387+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b3a80 con 0x55e2232b03f0 2023-10-05T19:40:21.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.387+0000 7f41751b1700 1 -- 172.21.15.12:0/71787349 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4160006070 con 0x55e2232b03f0 2023-10-05T19:40:21.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.387+0000 7f41751b1700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b3a80 con 0x55e2232b03f0 2023-10-05T19:40:21.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.387+0000 7f41751b1700 1 -- 172.21.15.12:0/71787349 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4160006070 con 0x55e2232b03f0 2023-10-05T19:40:21.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.387+0000 7f41751b1700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b3a80 con 0x55e2232766e0 2023-10-05T19:40:21.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.387+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f417002e040 con 0x55e2232766e0 2023-10-05T19:40:21.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.387+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b3a80 con 0x55e2232766e0 2023-10-05T19:40:21.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.391+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f417002e040 con 0x55e2232766e0 2023-10-05T19:40:21.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.391+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b3a80 con 0x7f416801abe0 2023-10-05T19:40:21.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.391+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f417002e040 con 0x7f416801abe0 2023-10-05T19:40:21.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.391+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b3a80 con 0x55e2232b03f0 2023-10-05T19:40:21.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.391+0000 7f41751b1700 1 -- 172.21.15.12:0/71787349 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4160006070 con 0x55e2232b03f0 2023-10-05T19:40:21.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.391+0000 7f41751b1700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b3a80 con 0x55e2232abc50 2023-10-05T19:40:21.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.391+0000 7f41759b2700 1 -- 172.21.15.12:0/71787349 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f41680387d0 con 0x55e2232abc50 2023-10-05T19:40:21.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.391+0000 7f41759b2700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b3a80 con 0x55e2232766e0 2023-10-05T19:40:21.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.395+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f417002e040 con 0x55e2232766e0 2023-10-05T19:40:21.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.395+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b3a80 con 0x7f416801abe0 2023-10-05T19:40:21.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.395+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f417002e040 con 0x7f416801abe0 2023-10-05T19:40:21.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.395+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b3a80 con 0x55e2232b03f0 2023-10-05T19:40:21.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.395+0000 7f41751b1700 1 -- 172.21.15.12:0/71787349 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4160006070 con 0x55e2232b03f0 2023-10-05T19:40:21.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.395+0000 7f41751b1700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b3a80 con 0x7f416801abe0 2023-10-05T19:40:21.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.395+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f417002e040 con 0x7f416801abe0 2023-10-05T19:40:21.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.395+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b3a80 con 0x55e2232abc50 2023-10-05T19:40:21.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.399+0000 7f41759b2700 1 -- 172.21.15.12:0/71787349 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f41680387d0 con 0x55e2232abc50 2023-10-05T19:40:21.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.399+0000 7f41759b2700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b3a80 con 0x55e2232b03f0 2023-10-05T19:40:21.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.399+0000 7f41751b1700 1 -- 172.21.15.12:0/71787349 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4160006070 con 0x55e2232b03f0 2023-10-05T19:40:21.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.399+0000 7f41751b1700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b3a80 con 0x7f416801abe0 2023-10-05T19:40:21.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.399+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f417002e040 con 0x7f416801abe0 2023-10-05T19:40:21.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.399+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b3a80 con 0x55e2232766e0 2023-10-05T19:40:21.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.399+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f417002e040 con 0x55e2232766e0 2023-10-05T19:40:21.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.399+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b3a80 con 0x55e2232abc50 2023-10-05T19:40:21.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.403+0000 7f41759b2700 1 -- 172.21.15.12:0/71787349 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f41680387d0 con 0x55e2232abc50 2023-10-05T19:40:21.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.403+0000 7f4177e7de40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:21.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.403+0000 7f4177e7de40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:21.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.403+0000 7f4177e7de40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:21.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.403+0000 7f4177e7de40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:21.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.403+0000 7f4177e7de40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.403+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232af5d0 con 0x55e2232766e0 2023-10-05T19:40:21.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.403+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f417002e040 con 0x55e2232766e0 2023-10-05T19:40:21.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.403+0000 7f4177e7de40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:21.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.403+0000 7f4177e7de40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.403+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b4370 con 0x55e2232abc50 2023-10-05T19:40:21.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.403+0000 7f41759b2700 1 -- 172.21.15.12:0/71787349 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f41680387d0 con 0x55e2232abc50 2023-10-05T19:40:21.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.403+0000 7f4177e7de40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:21.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.403+0000 7f4177e7de40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.403+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b4c60 con 0x55e2232766e0 2023-10-05T19:40:21.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.403+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f417002e040 con 0x55e2232766e0 2023-10-05T19:40:21.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.403+0000 7f4177e7de40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:21.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.403+0000 7f4177e7de40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.403+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b4c60 con 0x55e2232b03f0 2023-10-05T19:40:21.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.403+0000 7f41751b1700 1 -- 172.21.15.12:0/71787349 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f4160006070 con 0x55e2232b03f0 2023-10-05T19:40:21.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.403+0000 7f4177e7de40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:21.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.403+0000 7f4177e7de40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.403+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b53d0 con 0x7f416801abe0 2023-10-05T19:40:21.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.407+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f417002e040 con 0x7f416801abe0 2023-10-05T19:40:21.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.407+0000 7f4177e7de40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:21.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.407+0000 7f4177e7de40 20 zone default found 2023-10-05T19:40:21.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.407+0000 7f4177e7de40 4 Realm: () 2023-10-05T19:40:21.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.407+0000 7f4177e7de40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:21.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.407+0000 7f4177e7de40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:21.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.407+0000 7f4177e7de40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:21.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.407+0000 7f4177e7de40 20 zonegroup default 2023-10-05T19:40:21.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.407+0000 7f4177e7de40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.407+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b5380 con 0x7f416801abe0 2023-10-05T19:40:21.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.407+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f417002e040 con 0x7f416801abe0 2023-10-05T19:40:21.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.407+0000 7f4177e7de40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:21.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.407+0000 7f4177e7de40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.407+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b6d00 con 0x55e2232b03f0 2023-10-05T19:40:21.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.407+0000 7f41751b1700 1 -- 172.21.15.12:0/71787349 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f4160006070 con 0x55e2232b03f0 2023-10-05T19:40:21.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.407+0000 7f4177e7de40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:21.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.407+0000 7f4177e7de40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.407+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b70b0 con 0x55e2232b03f0 2023-10-05T19:40:21.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.407+0000 7f41751b1700 1 -- 172.21.15.12:0/71787349 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f4160006070 con 0x55e2232b03f0 2023-10-05T19:40:21.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.407+0000 7f4177e7de40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:21.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.407+0000 7f4177e7de40 20 started sync module instance, tier type = 2023-10-05T19:40:21.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.407+0000 7f4177e7de40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:21.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.407+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b63c0 con 0x55e2232abc50 2023-10-05T19:40:21.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.411+0000 7f41759b2700 1 -- 172.21.15.12:0/71787349 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'66 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f41680387d0 con 0x55e2232abc50 2023-10-05T19:40:21.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.411+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94429741008752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b7460 con 0x55e2232abc50 2023-10-05T19:40:21.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.411+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b7d00 con 0x55e2232766e0 2023-10-05T19:40:21.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.415+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'66 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f417002e040 con 0x55e2232766e0 2023-10-05T19:40:21.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.415+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94429741021200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b8260 con 0x55e2232766e0 2023-10-05T19:40:21.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.415+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b8dc0 con 0x55e2232b03f0 2023-10-05T19:40:21.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.415+0000 7f41759b2700 1 -- 172.21.15.12:0/71787349 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94429741008752] v28'67 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f41680387d0 con 0x55e2232abc50 2023-10-05T19:40:21.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.419+0000 7f41751b1700 1 -- 172.21.15.12:0/71787349 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'66 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4160006070 con 0x55e2232b03f0 2023-10-05T19:40:21.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.419+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94429741025424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b91c0 con 0x55e2232b03f0 2023-10-05T19:40:21.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.419+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b9aa0 con 0x55e2232abc50 2023-10-05T19:40:21.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.419+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94429741021200] v28'67 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f417002e040 con 0x55e2232766e0 2023-10-05T19:40:21.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.419+0000 7f41751b1700 1 -- 172.21.15.12:0/71787349 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94429741025424] v28'67 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4160006070 con 0x55e2232b03f0 2023-10-05T19:40:21.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.419+0000 7f41759b2700 1 -- 172.21.15.12:0/71787349 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'66 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f41680387d0 con 0x55e2232abc50 2023-10-05T19:40:21.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.419+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94429741028720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232b6770 con 0x55e2232abc50 2023-10-05T19:40:21.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.419+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232ba390 con 0x55e2232766e0 2023-10-05T19:40:21.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.423+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'66 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f417002e040 con 0x55e2232766e0 2023-10-05T19:40:21.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.423+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94429741031072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232ba7e0 con 0x55e2232766e0 2023-10-05T19:40:21.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.423+0000 7f41759b2700 1 -- 172.21.15.12:0/71787349 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94429741028720] v28'67 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f41680387d0 con 0x55e2232abc50 2023-10-05T19:40:21.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.423+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232bb1f0 con 0x7f416801abe0 2023-10-05T19:40:21.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.427+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94429741031072] v28'67 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f417002e040 con 0x55e2232766e0 2023-10-05T19:40:21.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.427+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'66 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f417002e040 con 0x7f416801abe0 2023-10-05T19:40:21.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.427+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94429741032176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232ba740 con 0x7f416801abe0 2023-10-05T19:40:21.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.427+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232bb6e0 con 0x7f416801abe0 2023-10-05T19:40:21.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.431+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(96 notify.6 [create] v28'66 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f417002e040 con 0x7f416801abe0 2023-10-05T19:40:21.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.431+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94429741036016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232bbb30 con 0x7f416801abe0 2023-10-05T19:40:21.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.431+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232bc410 con 0x7f416801abe0 2023-10-05T19:40:21.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.431+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(95 notify.5 [watch watch cookie 94429741032176] v28'67 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f417002e040 con 0x7f416801abe0 2023-10-05T19:40:21.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.431+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(98 notify.7 [create] v28'66 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f417002e040 con 0x7f416801abe0 2023-10-05T19:40:21.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.431+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94429741034752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232bc020 con 0x7f416801abe0 2023-10-05T19:40:21.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.431+0000 7f4177e7de40 20 add_watcher() i=0 2023-10-05T19:40:21.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.431+0000 7f4177e7de40 20 add_watcher() i=1 2023-10-05T19:40:21.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.431+0000 7f4177e7de40 20 add_watcher() i=2 2023-10-05T19:40:21.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.431+0000 7f4177e7de40 20 add_watcher() i=3 2023-10-05T19:40:21.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.431+0000 7f4177e7de40 20 add_watcher() i=4 2023-10-05T19:40:21.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.431+0000 7f4177e7de40 20 add_watcher() i=5 2023-10-05T19:40:21.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.431+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(97 notify.6 [watch watch cookie 94429741036016] v28'67 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f417002e040 con 0x7f416801abe0 2023-10-05T19:40:21.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.431+0000 7f4177e7de40 20 add_watcher() i=6 2023-10-05T19:40:21.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.435+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94429741034752] v28'67 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f417002e040 con 0x7f416801abe0 2023-10-05T19:40:21.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.435+0000 7f4177e7de40 20 add_watcher() i=7 2023-10-05T19:40:21.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.435+0000 7f4177e7de40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:21.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.435+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232bbd00 con 0x55e2232766e0 2023-10-05T19:40:21.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.435+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f417002e040 con 0x55e2232766e0 2023-10-05T19:40:21.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.435+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94429741037584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232bc810 con 0x55e2232766e0 2023-10-05T19:40:21.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.439+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94429741037584] v28'78 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f417002e040 con 0x55e2232766e0 2023-10-05T19:40:21.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.439+0000 7f4146fe5700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:21.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.439+0000 7f4177e7de40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:21.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.439+0000 7f4177e7de40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:21.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.439+0000 7f4177e7de40 1 Processor -- start 2023-10-05T19:40:21.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.439+0000 7f4177e7de40 1 -- start start 2023-10-05T19:40:21.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.439+0000 7f4177e7de40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e2232c9850 0x55e2232c9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.439+0000 7f4177e7de40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2232ca160 con 0x55e2232c9850 2023-10-05T19:40:21.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.439+0000 7f41751b1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e2232c9850 0x55e2232c9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.439+0000 7f41751b1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e2232c9850 0x55e2232c9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37286/0 (socket says 172.21.15.12:37286) 2023-10-05T19:40:21.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.439+0000 7f41751b1700 1 -- 172.21.15.12:0/2513037757 learned_addr learned my addr 172.21.15.12:0/2513037757 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:21.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.439+0000 7f41751b1700 1 -- 172.21.15.12:0/2513037757 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2232ca2a0 con 0x55e2232c9850 2023-10-05T19:40:21.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.439+0000 7f41751b1700 1 --2- 172.21.15.12:0/2513037757 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e2232c9850 0x55e2232c9c20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4168038a20 tx=0x7f416001e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6352afb005d2593 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.439+0000 7f41467e4700 1 -- 172.21.15.12:0/2513037757 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4160013040 con 0x55e2232c9850 2023-10-05T19:40:21.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.439+0000 7f41467e4700 1 -- 172.21.15.12:0/2513037757 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4160022040 con 0x55e2232c9850 2023-10-05T19:40:21.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.439+0000 7f41467e4700 1 -- 172.21.15.12:0/2513037757 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f416000eae0 con 0x55e2232c9850 2023-10-05T19:40:21.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.443+0000 7f4177e7de40 1 -- 172.21.15.12:0/2513037757 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e2232c9850 msgr2=0x55e2232c9c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.443+0000 7f4177e7de40 1 --2- 172.21.15.12:0/2513037757 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e2232c9850 0x55e2232c9c20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4168038a20 tx=0x7f416001e040 comp rx=0 tx=0).stop 2023-10-05T19:40:21.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.443+0000 7f4177e7de40 1 -- 172.21.15.12:0/2513037757 shutdown_connections 2023-10-05T19:40:21.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.443+0000 7f4177e7de40 1 --2- 172.21.15.12:0/2513037757 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e2232c9850 0x55e2232c9c20 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.443+0000 7f4177e7de40 1 -- 172.21.15.12:0/2513037757 >> 172.21.15.12:0/2513037757 conn(0x55e2232c4c10 msgr2=0x55e2232c2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:21.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.443+0000 7f4177e7de40 1 -- 172.21.15.12:0/2513037757 shutdown_connections 2023-10-05T19:40:21.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.443+0000 7f4177e7de40 1 -- 172.21.15.12:0/2513037757 wait complete. 2023-10-05T19:40:21.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.443+0000 7f4177e7de40 1 Processor -- start 2023-10-05T19:40:21.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.443+0000 7f4177e7de40 1 -- start start 2023-10-05T19:40:21.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.443+0000 7f4177e7de40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e2232c9850 0x55e2232cf460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.443+0000 7f4177e7de40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2232d0460 con 0x55e2232c9850 2023-10-05T19:40:21.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.443+0000 7f41751b1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e2232c9850 0x55e2232cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.443+0000 7f41751b1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e2232c9850 0x55e2232cf460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37290/0 (socket says 172.21.15.12:37290) 2023-10-05T19:40:21.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.443+0000 7f41751b1700 1 -- 172.21.15.12:0/1191977922 learned_addr learned my addr 172.21.15.12:0/1191977922 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:21.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.443+0000 7f41751b1700 1 -- 172.21.15.12:0/1191977922 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4160034040 con 0x55e2232c9850 2023-10-05T19:40:21.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.443+0000 7f41751b1700 1 --2- 172.21.15.12:0/1191977922 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e2232c9850 0x55e2232cf460 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55e2232ae980 tx=0x7f416003d030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.443+0000 7f4144fe1700 1 -- 172.21.15.12:0/1191977922 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f416002c640 con 0x55e2232c9850 2023-10-05T19:40:21.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.443+0000 7f4144fe1700 1 -- 172.21.15.12:0/1191977922 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f416002c7c0 con 0x55e2232c9850 2023-10-05T19:40:21.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.443+0000 7f4144fe1700 1 -- 172.21.15.12:0/1191977922 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f416002c970 con 0x55e2232c9850 2023-10-05T19:40:21.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.443+0000 7f4177e7de40 1 -- 172.21.15.12:0/1191977922 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e2232cfa60 con 0x55e2232c9850 2023-10-05T19:40:21.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.443+0000 7f4177e7de40 1 -- 172.21.15.12:0/1191977922 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e2232cfe40 con 0x55e2232c9850 2023-10-05T19:40:21.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.447+0000 7f4144fe1700 1 -- 172.21.15.12:0/1191977922 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f416002cad0 con 0x55e2232c9850 2023-10-05T19:40:21.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.447+0000 7f4144fe1700 1 --2- 172.21.15.12:0/1191977922 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f40bc030ca0 0x7f40bc033150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.447+0000 7f4144fe1700 1 -- 172.21.15.12:0/1191977922 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f4160059ab0 con 0x55e2232c9850 2023-10-05T19:40:21.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.447+0000 7f41761b3700 1 --2- 172.21.15.12:0/1191977922 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f40bc030ca0 0x7f40bc033150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.447+0000 7f4177e7de40 5 note: GC not initialized 2023-10-05T19:40:21.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.447+0000 7f41761b3700 1 --2- 172.21.15.12:0/1191977922 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f40bc030ca0 0x7f40bc033150 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x55e2232bc180 tx=0x7f4170005500 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.451+0000 7f40b5ffb700 20 reqs_thread_entry: start 2023-10-05T19:40:21.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.451+0000 7f4177e7de40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:21.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.451+0000 7f40a77fe700 20 reqs_thread_entry: start 2023-10-05T19:40:21.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.451+0000 7f4177e7de40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:21.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.451+0000 7f40a5ffb700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:51 2023 2023-10-05T19:40:21.459 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:21.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.451+0000 7f40a57fa700 20 reqs_thread_entry: start 2023-10-05T19:40:21.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.451+0000 7f4177e7de40 20 get_system_obj_state: rctx=0x7ffcba218e28 obj=default.rgw.meta:users.uid:foo state=0x55e22314f0b0 s->prefetch_data=0 2023-10-05T19:40:21.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.451+0000 7f4177e7de40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2023-10-05T19:40:21.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.451+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e223337e30 con 0x55e2232766e0 2023-10-05T19:40:21.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.451+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(102 foo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7f417002e040 con 0x55e2232766e0 2023-10-05T19:40:21.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.451+0000 7f4177e7de40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x16 2023-10-05T19:40:21.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.451+0000 7f4177e7de40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:21.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.451+0000 7f4177e7de40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:21.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.451+0000 7f4177e7de40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:21.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.451+0000 7f4177e7de40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:21.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.451+0000 7f4177e7de40 10 cache get: name=default.rgw.meta+users.uid+foo : type miss (requested=0x13, cached=0x16) 2023-10-05T19:40:21.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.451+0000 7f4177e7de40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.451+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e22333a8c0 con 0x55e2232766e0 2023-10-05T19:40:21.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.451+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(103 foo [call,call out=48b,read 0~513 out=513b,getxattrs out=75b] v0'0 uv7 ondisk = 0) v8 ==== 273+0+636 (crc 0 0 0) 0x7f417002e040 con 0x55e2232766e0 2023-10-05T19:40:21.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.451+0000 7f4177e7de40 20 rados_obj.operate() r=0 bl.length=513 2023-10-05T19:40:21.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.451+0000 7f4177e7de40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2023-10-05T19:40:21.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.451+0000 7f4177e7de40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:21.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.451+0000 7f4177e7de40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:21.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.451+0000 7f4177e7de40 20 get_system_obj_state: rctx=0x7ffcba218678 obj=default.rgw.meta:users.swift:foo:foo2 state=0x55e22314f0b0 s->prefetch_data=0 2023-10-05T19:40:21.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.451+0000 7f4177e7de40 10 cache get: name=default.rgw.meta+users.swift+foo:foo2 : miss 2023-10-05T19:40:21.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.451+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:104 8.2 8:45671577:users.swift::foo%3afoo2:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e223336ec0 con 0x55e2232abc50 2023-10-05T19:40:21.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.451+0000 7f41759b2700 1 -- 172.21.15.12:0/71787349 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(104 foo:foo2 [getxattrs,stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 194+0+0 (crc 0 0 0) 0x7f41680387d0 con 0x55e2232abc50 2023-10-05T19:40:21.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.455+0000 7f4177e7de40 10 cache put: name=default.rgw.meta+users.swift+foo:foo2 info.flags=0x0 2023-10-05T19:40:21.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.455+0000 7f4177e7de40 10 adding default.rgw.meta+users.swift+foo:foo2 to cache LRU end 2023-10-05T19:40:21.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.455+0000 7f4177e7de40 20 get_system_obj_state: rctx=0x7ffcba218d18 obj=default.rgw.meta:users.swift:foo:foo2 state=0x55e22314f0b0 s->prefetch_data=0 2023-10-05T19:40:21.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.455+0000 7f4177e7de40 10 cache get: name=default.rgw.meta+users.swift+foo:foo2 : hit (negative entry) 2023-10-05T19:40:21.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.455+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:105 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.set in=58b,writefull 0~621 in=621b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2233372c0 con 0x55e2232766e0 2023-10-05T19:40:21.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.455+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(105 foo [delete,create,call,writefull 0~621] v28'8 uv8 ondisk = 0) v8 ==== 273+0+0 (crc 0 0 0) 0x7f417002e040 con 0x55e2232766e0 2023-10-05T19:40:21.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.455+0000 7f4177e7de40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2023-10-05T19:40:21.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.455+0000 7f4177e7de40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:21.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.455+0000 7f4177e7de40 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2023-10-05T19:40:21.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.455+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:106 7.d 7:b4812045:::notify.4:head [notify cookie 94429741541840 in=809b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e22333b8d0 con 0x55e2232766e0 2023-10-05T19:40:21.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.455+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== watch-notify(notify (1) cookie 94429741031072 notify 120259084295 ret 0) v3 ==== 839+0+0 (crc 0 0 0) 0x7f41700400a0 con 0x55e2232766e0 2023-10-05T19:40:21.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.455+0000 7f416effd700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084295 cookie 94429741031072 notifier 4357 bl.length()=797 2023-10-05T19:40:21.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.455+0000 7f416effd700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2023-10-05T19:40:21.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.455+0000 7f416effd700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:21.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.455+0000 7f416effd700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:107 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f4154003d90 con 0x55e2232766e0 2023-10-05T19:40:21.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.459+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(106 notify.4 [notify cookie 94429741541840 out=8b] v0'0 uv66 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f417002e040 con 0x55e2232766e0 2023-10-05T19:40:21.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.459+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== watch-notify(notify_complete (2) cookie 94429741541840 notify 120259084295 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f4170040230 con 0x55e2232766e0 2023-10-05T19:40:21.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.459+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.1 v2:172.21.15.12:6808/29933 39 ==== osd_op_reply(107 notify.4 [notify-ack] v0'0 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f417002e040 con 0x55e2232766e0 2023-10-05T19:40:21.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.459+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:108 8.2 8:45671577:users.swift::foo%3afoo2:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2233369d0 con 0x55e2232abc50 2023-10-05T19:40:21.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.463+0000 7f41759b2700 1 -- 172.21.15.12:0/71787349 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(108 foo:foo2 [delete,create,writefull 0~7] v28'1 uv1 ondisk = 0) v8 ==== 236+0+0 (crc 0 0 0) 0x7f41680387d0 con 0x55e2232abc50 2023-10-05T19:40:21.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.463+0000 7f4177e7de40 10 cache put: name=default.rgw.meta+users.swift+foo:foo2 info.flags=0x7 2023-10-05T19:40:21.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.463+0000 7f4177e7de40 10 moving default.rgw.meta+users.swift+foo:foo2 to cache LRU end 2023-10-05T19:40:21.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.463+0000 7f4177e7de40 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.swift:foo:foo2, ofs0, ns] 2023-10-05T19:40:21.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.463+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:109 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94429741541840 in=178b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e22333bcf0 con 0x55e2232abc50 2023-10-05T19:40:21.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.463+0000 7f41759b2700 1 -- 172.21.15.12:0/71787349 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== watch-notify(notify (1) cookie 94429741008752 notify 120259084291 ret 0) v3 ==== 208+0+0 (crc 0 0 0) 0x55e22329e560 con 0x55e2232abc50 2023-10-05T19:40:21.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.463+0000 7f416f7fe700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084291 cookie 94429741008752 notifier 4357 bl.length()=166 2023-10-05T19:40:21.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.463+0000 7f416f7fe700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.swift+foo:foo2 info.flags=0x7 2023-10-05T19:40:21.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.463+0000 7f416f7fe700 10 rgw watcher librados: moving default.rgw.meta+users.swift+foo:foo2 to cache LRU end 2023-10-05T19:40:21.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.463+0000 7f416f7fe700 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:110 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f40c4003410 con 0x55e2232abc50 2023-10-05T19:40:21.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.463+0000 7f41759b2700 1 -- 172.21.15.12:0/71787349 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(109 notify.0 [notify cookie 94429741541840 out=8b] v0'0 uv66 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f41680387d0 con 0x55e2232abc50 2023-10-05T19:40:21.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.463+0000 7f41759b2700 1 -- 172.21.15.12:0/71787349 <== osd.2 v2:172.21.15.12:6800/29934 23 ==== watch-notify(notify_complete (2) cookie 94429741541840 notify 120259084291 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55e22329e0c0 con 0x55e2232abc50 2023-10-05T19:40:21.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.463+0000 7f41759b2700 1 -- 172.21.15.12:0/71787349 <== osd.2 v2:172.21.15.12:6800/29934 24 ==== osd_op_reply(110 notify.0 [notify-ack] v0'0 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f41680387d0 con 0x55e2232abc50 2023-10-05T19:40:21.472 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:40:21.472 INFO:teuthology.orchestra.run.smithi012.stdout: "user_id": "foo", 2023-10-05T19:40:21.472 INFO:teuthology.orchestra.run.smithi012.stdout: "display_name": "Foo", 2023-10-05T19:40:21.472 INFO:teuthology.orchestra.run.smithi012.stdout: "email": "foo@foo.com", 2023-10-05T19:40:21.473 INFO:teuthology.orchestra.run.smithi012.stdout: "suspended": 0, 2023-10-05T19:40:21.473 INFO:teuthology.orchestra.run.smithi012.stdout: "max_buckets": 4, 2023-10-05T19:40:21.473 INFO:teuthology.orchestra.run.smithi012.stdout: "subusers": [ 2023-10-05T19:40:21.473 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:21.473 INFO:teuthology.orchestra.run.smithi012.stdout: "id": "foo:foo1", 2023-10-05T19:40:21.473 INFO:teuthology.orchestra.run.smithi012.stdout: "permissions": "full-control" 2023-10-05T19:40:21.473 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:21.473 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:21.473 INFO:teuthology.orchestra.run.smithi012.stdout: "id": "foo:foo2", 2023-10-05T19:40:21.473 INFO:teuthology.orchestra.run.smithi012.stdout: "permissions": "" 2023-10-05T19:40:21.473 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:21.474 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:21.474 INFO:teuthology.orchestra.run.smithi012.stdout: "keys": [ 2023-10-05T19:40:21.474 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:21.474 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:40:21.474 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2023-10-05T19:40:21.474 INFO:teuthology.orchestra.run.smithi012.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2023-10-05T19:40:21.474 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:21.474 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:21.474 INFO:teuthology.orchestra.run.smithi012.stdout: "swift_keys": [ 2023-10-05T19:40:21.474 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:21.474 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo:foo1", 2023-10-05T19:40:21.474 INFO:teuthology.orchestra.run.smithi012.stdout: "secret_key": "gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL" 2023-10-05T19:40:21.475 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:21.475 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:21.475 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo:foo2", 2023-10-05T19:40:21.475 INFO:teuthology.orchestra.run.smithi012.stdout: "secret_key": "ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy" 2023-10-05T19:40:21.475 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:21.475 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:21.475 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": [], 2023-10-05T19:40:21.475 INFO:teuthology.orchestra.run.smithi012.stdout: "op_mask": "read, write, delete", 2023-10-05T19:40:21.475 INFO:teuthology.orchestra.run.smithi012.stdout: "default_placement": "", 2023-10-05T19:40:21.475 INFO:teuthology.orchestra.run.smithi012.stdout: "default_storage_class": "", 2023-10-05T19:40:21.475 INFO:teuthology.orchestra.run.smithi012.stdout: "placement_tags": [], 2023-10-05T19:40:21.475 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket_quota": { 2023-10-05T19:40:21.475 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:21.476 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:21.476 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:21.476 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:21.476 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:21.476 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:21.476 INFO:teuthology.orchestra.run.smithi012.stdout: "user_quota": { 2023-10-05T19:40:21.476 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:21.476 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:21.476 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:21.476 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:21.476 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:21.476 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:21.477 INFO:teuthology.orchestra.run.smithi012.stdout: "temp_url_keys": [], 2023-10-05T19:40:21.477 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "rgw", 2023-10-05T19:40:21.477 INFO:teuthology.orchestra.run.smithi012.stdout: "mfa_ids": [] 2023-10-05T19:40:21.477 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:40:21.477 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:40:21.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.463+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:111 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94429741008752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e22328b4d0 con 0x55e2232abc50 2023-10-05T19:40:21.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.467+0000 7f41759b2700 1 -- 172.21.15.12:0/71787349 <== osd.2 v2:172.21.15.12:6800/29934 25 ==== osd_op_reply(111 notify.0 [watch unwatch cookie 94429741008752] v28'68 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f41680387d0 con 0x55e2232abc50 2023-10-05T19:40:21.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.467+0000 7f4177e7de40 20 remove_watcher() i=0 2023-10-05T19:40:21.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.467+0000 7f4177e7de40 2 removed watcher, disabling cache 2023-10-05T19:40:21.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.467+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:112 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94429741021200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232af5d0 con 0x55e2232766e0 2023-10-05T19:40:21.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.467+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.1 v2:172.21.15.12:6808/29933 40 ==== osd_op_reply(112 notify.1 [watch unwatch cookie 94429741021200] v28'68 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f417002e040 con 0x55e2232766e0 2023-10-05T19:40:21.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.471+0000 7f4177e7de40 20 remove_watcher() i=1 2023-10-05T19:40:21.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.471+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:113 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94429741025424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232af5d0 con 0x55e2232b03f0 2023-10-05T19:40:21.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.471+0000 7f41751b1700 1 -- 172.21.15.12:0/71787349 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(113 notify.2 [watch unwatch cookie 94429741025424] v28'68 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4160006070 con 0x55e2232b03f0 2023-10-05T19:40:21.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.471+0000 7f4177e7de40 20 remove_watcher() i=2 2023-10-05T19:40:21.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.471+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:114 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94429741028720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232af5d0 con 0x55e2232abc50 2023-10-05T19:40:21.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.475+0000 7f41759b2700 1 -- 172.21.15.12:0/71787349 <== osd.2 v2:172.21.15.12:6800/29934 26 ==== osd_op_reply(114 notify.3 [watch unwatch cookie 94429741028720] v28'68 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f41680387d0 con 0x55e2232abc50 2023-10-05T19:40:21.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.475+0000 7f4177e7de40 20 remove_watcher() i=3 2023-10-05T19:40:21.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.475+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:115 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94429741031072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232af5d0 con 0x55e2232766e0 2023-10-05T19:40:21.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.475+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.1 v2:172.21.15.12:6808/29933 41 ==== osd_op_reply(115 notify.4 [watch unwatch cookie 94429741031072] v28'68 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f417002e040 con 0x55e2232766e0 2023-10-05T19:40:21.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.475+0000 7f4177e7de40 20 remove_watcher() i=4 2023-10-05T19:40:21.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.475+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:116 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94429741032176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232af5d0 con 0x7f416801abe0 2023-10-05T19:40:21.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.479+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(116 notify.5 [watch unwatch cookie 94429741032176] v28'68 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f417002e040 con 0x7f416801abe0 2023-10-05T19:40:21.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.479+0000 7f4177e7de40 20 remove_watcher() i=5 2023-10-05T19:40:21.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.479+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:117 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94429741036016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232af5d0 con 0x7f416801abe0 2023-10-05T19:40:21.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.483+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(117 notify.6 [watch unwatch cookie 94429741036016] v28'68 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f417002e040 con 0x7f416801abe0 2023-10-05T19:40:21.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.483+0000 7f4177e7de40 20 remove_watcher() i=6 2023-10-05T19:40:21.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.483+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:118 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94429741034752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232af5d0 con 0x7f416801abe0 2023-10-05T19:40:21.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.487+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(118 notify.7 [watch unwatch cookie 94429741034752] v28'68 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f417002e040 con 0x7f416801abe0 2023-10-05T19:40:21.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.487+0000 7f4177e7de40 20 remove_watcher() i=7 2023-10-05T19:40:21.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.487+0000 7f4146fe5700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:21.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.487+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:119 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94429741037584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e2232af5d0 con 0x55e2232766e0 2023-10-05T19:40:21.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.491+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 <== osd.1 v2:172.21.15.12:6808/29933 42 ==== osd_op_reply(119 data_loggenerations_metadata [watch unwatch cookie 94429741037584] v28'79 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f417002e040 con 0x55e2232766e0 2023-10-05T19:40:21.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.491+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f416801abe0 msgr2=0x7f41680289f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.491+0000 7f4177e7de40 1 --2- 172.21.15.12:0/71787349 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f416801abe0 0x7f41680289f0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.491+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55e2232766e0 msgr2=0x55e2232a9710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.491+0000 7f4177e7de40 1 --2- 172.21.15.12:0/71787349 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55e2232766e0 0x55e2232a9710 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.491+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55e2232abc50 msgr2=0x55e2232ae030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.491+0000 7f4177e7de40 1 --2- 172.21.15.12:0/71787349 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55e2232abc50 0x55e2232ae030 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.491+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55e2232b03f0 msgr2=0x55e2232b2800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.491+0000 7f4177e7de40 1 --2- 172.21.15.12:0/71787349 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55e2232b03f0 0x55e2232b2800 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.491+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f4158030c00 msgr2=0x7f41580330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.491+0000 7f4177e7de40 1 --2- 172.21.15.12:0/71787349 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f4158030c00 0x7f41580330b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4160004170 tx=0x7f4160001400 comp rx=0 tx=0).stop 2023-10-05T19:40:21.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.491+0000 7f41761b3700 1 -- 172.21.15.12:0/71787349 reap_dead start 2023-10-05T19:40:21.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.491+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e223275dd0 msgr2=0x55e22329db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.491+0000 7f4177e7de40 1 --2- 172.21.15.12:0/71787349 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e223275dd0 0x55e22329db80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f416800adc0 tx=0x7f416801d000 comp rx=0 tx=0).stop 2023-10-05T19:40:21.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.491+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 shutdown_connections 2023-10-05T19:40:21.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.491+0000 7f4177e7de40 1 --2- 172.21.15.12:0/71787349 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e223275dd0 0x55e22329db80 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.491+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 >> 172.21.15.12:0/71787349 conn(0x55e2232717f0 msgr2=0x55e22328f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:21.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.491+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 shutdown_connections 2023-10-05T19:40:21.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.491+0000 7f4177e7de40 1 -- 172.21.15.12:0/71787349 wait complete. 2023-10-05T19:40:21.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.491+0000 7f4177e7de40 1 -- 172.21.15.12:0/1191977922 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f40bc030ca0 msgr2=0x7f40bc033150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.491+0000 7f4177e7de40 1 --2- 172.21.15.12:0/1191977922 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f40bc030ca0 0x7f40bc033150 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x55e2232bc180 tx=0x7f4170005500 comp rx=0 tx=0).stop 2023-10-05T19:40:21.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.491+0000 7f4177e7de40 1 -- 172.21.15.12:0/1191977922 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e2232c9850 msgr2=0x55e2232cf460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.491+0000 7f4177e7de40 1 --2- 172.21.15.12:0/1191977922 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e2232c9850 0x55e2232cf460 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55e2232ae980 tx=0x7f416003d030 comp rx=0 tx=0).stop 2023-10-05T19:40:21.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.491+0000 7f4177e7de40 1 -- 172.21.15.12:0/1191977922 shutdown_connections 2023-10-05T19:40:21.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.491+0000 7f4177e7de40 1 --2- 172.21.15.12:0/1191977922 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f40bc030ca0 0x7f40bc033150 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.491+0000 7f4177e7de40 1 --2- 172.21.15.12:0/1191977922 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e2232c9850 0x55e2232cf460 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.491+0000 7f4177e7de40 1 -- 172.21.15.12:0/1191977922 >> 172.21.15.12:0/1191977922 conn(0x55e2232c4c10 msgr2=0x55e2232c2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:21.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.491+0000 7f4177e7de40 1 -- 172.21.15.12:0/1191977922 shutdown_connections 2023-10-05T19:40:21.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.491+0000 7f4177e7de40 1 -- 172.21.15.12:0/1191977922 wait complete. 2023-10-05T19:40:21.506 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}, {'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL'}, {'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:40:21.507 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2023-10-05T19:40:21.507 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2023-10-05T19:40:21.507 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2023-10-05T19:40:21.547 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:21.547 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:21.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.539+0000 7f667cb8ee40 1 Processor -- start 2023-10-05T19:40:21.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.539+0000 7f667cb8ee40 1 -- start start 2023-10-05T19:40:21.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.539+0000 7f667cb8ee40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x5608e7d80410 0x5608e7d7efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.539+0000 7f667cb8ee40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5608e7cccbd0 con 0x5608e7d807e0 2023-10-05T19:40:21.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.539+0000 7f667cb8ee40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x5608e7c89ee0 con 0x5608e7d80410 2023-10-05T19:40:21.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.539+0000 7f667a6c3700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x5608e7d80410 0x5608e7d7efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.539+0000 7f6679ec2700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x5608e7d807e0 0x5608e7d7f500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:47340/0 (socket says 172.21.15.12:47340) 2023-10-05T19:40:21.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.539+0000 7f6679ec2700 1 -- 172.21.15.12:0/3952931400 learned_addr learned my addr 172.21.15.12:0/3952931400 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:21.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.539+0000 7f667a6c3700 1 -- 172.21.15.12:0/3952931400 >> v1:172.21.15.8:6789/0 conn(0x5608e7d807e0 legacy=0x5608e7d7f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.539+0000 7f667a6c3700 1 -- 172.21.15.12:0/3952931400 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608e7d7fcd0 con 0x5608e7d80410 2023-10-05T19:40:21.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.543+0000 7f667a6c3700 1 --2- 172.21.15.12:0/3952931400 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5608e7d80410 0x5608e7d7efc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f667400a5d0 tx=0x7f6674005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4717ea0dc9afdfec server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.543+0000 7f66796c1700 1 -- 172.21.15.12:0/3952931400 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f66740086a0 con 0x5608e7d80410 2023-10-05T19:40:21.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.543+0000 7f66796c1700 1 -- 172.21.15.12:0/3952931400 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6674008800 con 0x5608e7d80410 2023-10-05T19:40:21.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.543+0000 7f66796c1700 1 -- 172.21.15.12:0/3952931400 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6674008ad0 con 0x5608e7d80410 2023-10-05T19:40:21.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.543+0000 7f667cb8ee40 1 -- 172.21.15.12:0/3952931400 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5608e7d80410 msgr2=0x5608e7d7efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.543+0000 7f667cb8ee40 1 --2- 172.21.15.12:0/3952931400 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5608e7d80410 0x5608e7d7efc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f667400a5d0 tx=0x7f6674005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:21.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.543+0000 7f667cb8ee40 1 -- 172.21.15.12:0/3952931400 shutdown_connections 2023-10-05T19:40:21.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.543+0000 7f667cb8ee40 1 --2- 172.21.15.12:0/3952931400 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5608e7d80410 0x5608e7d7efc0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.543+0000 7f667cb8ee40 1 -- 172.21.15.12:0/3952931400 >> 172.21.15.12:0/3952931400 conn(0x5608e7d7a720 msgr2=0x5608e7d7cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:21.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.543+0000 7f667cb8ee40 1 -- 172.21.15.12:0/3952931400 shutdown_connections 2023-10-05T19:40:21.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.543+0000 7f667cb8ee40 1 -- 172.21.15.12:0/3952931400 wait complete. 2023-10-05T19:40:21.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.543+0000 7f667cb8ee40 1 Processor -- start 2023-10-05T19:40:21.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.543+0000 7f667cb8ee40 1 -- start start 2023-10-05T19:40:21.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.543+0000 7f667cb8ee40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5608e7d80410 0x5608e7da1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.543+0000 7f667cb8ee40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e7c4de30 con 0x5608e7d80410 2023-10-05T19:40:21.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.543+0000 7f667a6c3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5608e7d80410 0x5608e7da1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.547+0000 7f667a6c3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5608e7d80410 0x5608e7da1110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37320/0 (socket says 172.21.15.12:37320) 2023-10-05T19:40:21.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.547+0000 7f667a6c3700 1 -- 172.21.15.12:0/1814726347 learned_addr learned my addr 172.21.15.12:0/1814726347 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:21.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.547+0000 7f667a6c3700 1 -- 172.21.15.12:0/1814726347 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6674005cf0 con 0x5608e7d80410 2023-10-05T19:40:21.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.547+0000 7f667a6c3700 1 --2- 172.21.15.12:0/1814726347 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5608e7d80410 0x5608e7da1110 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6674005a90 tx=0x7f6674015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25ee18f14d9732de server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.547+0000 7f6673fff700 1 -- 172.21.15.12:0/1814726347 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6674008a40 con 0x5608e7d80410 2023-10-05T19:40:21.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.547+0000 7f6673fff700 1 -- 172.21.15.12:0/1814726347 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6674008bc0 con 0x5608e7d80410 2023-10-05T19:40:21.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.547+0000 7f6673fff700 1 -- 172.21.15.12:0/1814726347 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6674012450 con 0x5608e7d80410 2023-10-05T19:40:21.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.547+0000 7f667cb8ee40 1 -- 172.21.15.12:0/1814726347 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5608e7d80410 msgr2=0x5608e7da1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.547+0000 7f667cb8ee40 1 --2- 172.21.15.12:0/1814726347 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5608e7d80410 0x5608e7da1110 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6674005a90 tx=0x7f6674015000 comp rx=0 tx=0).stop 2023-10-05T19:40:21.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.547+0000 7f667cb8ee40 1 -- 172.21.15.12:0/1814726347 shutdown_connections 2023-10-05T19:40:21.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.547+0000 7f667cb8ee40 1 --2- 172.21.15.12:0/1814726347 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5608e7d80410 0x5608e7da1110 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.547+0000 7f667cb8ee40 1 -- 172.21.15.12:0/1814726347 >> 172.21.15.12:0/1814726347 conn(0x5608e7d7a720 msgr2=0x5608e7d9ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:21.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.547+0000 7f667cb8ee40 1 -- 172.21.15.12:0/1814726347 shutdown_connections 2023-10-05T19:40:21.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.547+0000 7f667cb8ee40 1 -- 172.21.15.12:0/1814726347 wait complete. 2023-10-05T19:40:21.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.547+0000 7f667cb8ee40 1 Processor -- start 2023-10-05T19:40:21.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.547+0000 7f667cb8ee40 1 -- start start 2023-10-05T19:40:21.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.551+0000 7f667cb8ee40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5608e7d80410 0x5608e7da7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.551+0000 7f667cb8ee40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e7c20ed0 con 0x5608e7d80410 2023-10-05T19:40:21.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.551+0000 7f667a6c3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5608e7d80410 0x5608e7da7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.551+0000 7f667a6c3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5608e7d80410 0x5608e7da7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37330/0 (socket says 172.21.15.12:37330) 2023-10-05T19:40:21.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.551+0000 7f667a6c3700 1 -- 172.21.15.12:0/2787179272 learned_addr learned my addr 172.21.15.12:0/2787179272 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:21.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.551+0000 7f667a6c3700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608e7d7fcd0 con 0x5608e7d80410 2023-10-05T19:40:21.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.551+0000 7f667a6c3700 1 --2- 172.21.15.12:0/2787179272 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5608e7d80410 0x5608e7da7d20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f667400c010 tx=0x7f667401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.551+0000 7f66727fc700 1 -- 172.21.15.12:0/2787179272 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6674008e50 con 0x5608e7d80410 2023-10-05T19:40:21.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.551+0000 7f66727fc700 1 -- 172.21.15.12:0/2787179272 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6674025660 con 0x5608e7d80410 2023-10-05T19:40:21.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.551+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5608e7da8260 con 0x5608e7d80410 2023-10-05T19:40:21.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.551+0000 7f66727fc700 1 -- 172.21.15.12:0/2787179272 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6674025950 con 0x5608e7d80410 2023-10-05T19:40:21.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.551+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5608e7da8700 con 0x5608e7d80410 2023-10-05T19:40:21.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.555+0000 7f66727fc700 1 -- 172.21.15.12:0/2787179272 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f6674019070 con 0x5608e7d80410 2023-10-05T19:40:21.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.555+0000 7f667cb8ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.555+0000 7f66727fc700 1 --2- 172.21.15.12:0/2787179272 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f665c030c00 0x7f665c0330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.555+0000 7f66727fc700 1 -- 172.21.15.12:0/2787179272 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f6674046610 con 0x5608e7d80410 2023-10-05T19:40:21.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.555+0000 7f6679ec2700 1 --2- 172.21.15.12:0/2787179272 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f665c030c00 0x7f665c0330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.555+0000 7f667cb8ee40 1 --2- 172.21.15.12:0/2787179272 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5608e7d807e0 0x5608e7db3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.555+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7db3ec0 con 0x5608e7d807e0 2023-10-05T19:40:21.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.555+0000 7f667aec4700 1 --2- 172.21.15.12:0/2787179272 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5608e7d807e0 0x5608e7db3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.555+0000 7f6679ec2700 1 --2- 172.21.15.12:0/2787179272 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f665c030c00 0x7f665c0330b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f66640044f0 tx=0x7f66640067c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.555+0000 7f667aec4700 1 --2- 172.21.15.12:0/2787179272 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5608e7d807e0 0x5608e7db3810 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.555+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f666c00a500 con 0x5608e7d807e0 2023-10-05T19:40:21.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.555+0000 7f667cb8ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:21.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.555+0000 7f667cb8ee40 20 realm 2023-10-05T19:40:21.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.555+0000 7f667cb8ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.555+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7db0e70 con 0x5608e7d807e0 2023-10-05T19:40:21.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.555+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f666c00a500 con 0x5608e7d807e0 2023-10-05T19:40:21.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.555+0000 7f667cb8ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:21.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.555+0000 7f667cb8ee40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:21.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.555+0000 7f667cb8ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.555+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7db49a0 con 0x5608e7d807e0 2023-10-05T19:40:21.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.559+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f666c00a500 con 0x5608e7d807e0 2023-10-05T19:40:21.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.559+0000 7f667cb8ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:21.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.559+0000 7f667cb8ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.559+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7db4580 con 0x5608e7d807e0 2023-10-05T19:40:21.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.559+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f666c00a500 con 0x5608e7d807e0 2023-10-05T19:40:21.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.559+0000 7f667cb8ee40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:21.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.559+0000 7f667cb8ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.559+0000 7f667cb8ee40 1 --2- 172.21.15.12:0/2787179272 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5608e7db5930 0x5608e7db7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.559+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7db8480 con 0x5608e7db5930 2023-10-05T19:40:21.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.559+0000 7f667a6c3700 1 --2- 172.21.15.12:0/2787179272 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5608e7db5930 0x5608e7db7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.559+0000 7f667a6c3700 1 --2- 172.21.15.12:0/2787179272 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5608e7db5930 0x5608e7db7d10 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.559+0000 7f667a6c3700 1 -- 172.21.15.12:0/2787179272 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f6674011570 con 0x5608e7db5930 2023-10-05T19:40:21.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.559+0000 7f667cb8ee40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:21.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.559+0000 7f667cb8ee40 20 searching for the correct realm 2023-10-05T19:40:21.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.559+0000 7f667cb8ee40 1 --2- 172.21.15.12:0/2787179272 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5608e7dba2b0 0x5608e7dbc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.559+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbcbd0 con 0x5608e7dba2b0 2023-10-05T19:40:21.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.559+0000 7f6679ec2700 1 --2- 172.21.15.12:0/2787179272 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5608e7dba2b0 0x5608e7dbc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.563+0000 7f6679ec2700 1 --2- 172.21.15.12:0/2787179272 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5608e7dba2b0 0x5608e7dbc690 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.563+0000 7f6679ec2700 1 -- 172.21.15.12:0/2787179272 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f6664004520 con 0x5608e7dba2b0 2023-10-05T19:40:21.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.563+0000 7f6679ec2700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbcbd0 con 0x5608e7db5930 2023-10-05T19:40:21.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.563+0000 7f667a6c3700 1 -- 172.21.15.12:0/2787179272 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6674011570 con 0x5608e7db5930 2023-10-05T19:40:21.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.563+0000 7f667a6c3700 1 --2- 172.21.15.12:0/2787179272 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f6674007030 0x7f6674011830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.563+0000 7f667a6c3700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbcbd0 con 0x7f6674007030 2023-10-05T19:40:21.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.563+0000 7f667aec4700 1 --2- 172.21.15.12:0/2787179272 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f6674007030 0x7f6674011830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.567+0000 7f667aec4700 1 --2- 172.21.15.12:0/2787179272 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f6674007030 0x7f6674011830 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.567+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f666c014040 con 0x7f6674007030 2023-10-05T19:40:21.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.567+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbcbd0 con 0x5608e7d807e0 2023-10-05T19:40:21.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.567+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f666c014040 con 0x5608e7d807e0 2023-10-05T19:40:21.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.567+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbcbd0 con 0x5608e7d807e0 2023-10-05T19:40:21.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.567+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f666c014040 con 0x5608e7d807e0 2023-10-05T19:40:21.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.567+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbcbd0 con 0x7f6674007030 2023-10-05T19:40:21.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.567+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f666c014040 con 0x7f6674007030 2023-10-05T19:40:21.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.567+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbcbd0 con 0x7f6674007030 2023-10-05T19:40:21.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.571+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f666c014040 con 0x7f6674007030 2023-10-05T19:40:21.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.571+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbcbd0 con 0x5608e7db5930 2023-10-05T19:40:21.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.571+0000 7f667a6c3700 1 -- 172.21.15.12:0/2787179272 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6674011570 con 0x5608e7db5930 2023-10-05T19:40:21.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.571+0000 7f667a6c3700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbcbd0 con 0x5608e7dba2b0 2023-10-05T19:40:21.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.571+0000 7f6679ec2700 1 -- 172.21.15.12:0/2787179272 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6664004520 con 0x5608e7dba2b0 2023-10-05T19:40:21.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.571+0000 7f6679ec2700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbcbd0 con 0x5608e7d807e0 2023-10-05T19:40:21.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.571+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f666c014040 con 0x5608e7d807e0 2023-10-05T19:40:21.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.571+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbcbd0 con 0x7f6674007030 2023-10-05T19:40:21.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.575+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f666c014040 con 0x7f6674007030 2023-10-05T19:40:21.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.575+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbcbd0 con 0x5608e7d807e0 2023-10-05T19:40:21.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.575+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f666c014040 con 0x5608e7d807e0 2023-10-05T19:40:21.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.575+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbcbd0 con 0x5608e7db5930 2023-10-05T19:40:21.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.575+0000 7f667a6c3700 1 -- 172.21.15.12:0/2787179272 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6674011570 con 0x5608e7db5930 2023-10-05T19:40:21.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.575+0000 7f667a6c3700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbcbd0 con 0x5608e7dba2b0 2023-10-05T19:40:21.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.575+0000 7f6679ec2700 1 -- 172.21.15.12:0/2787179272 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6664004520 con 0x5608e7dba2b0 2023-10-05T19:40:21.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.575+0000 7f6679ec2700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbcbd0 con 0x5608e7dba2b0 2023-10-05T19:40:21.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.575+0000 7f6679ec2700 1 -- 172.21.15.12:0/2787179272 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6664004520 con 0x5608e7dba2b0 2023-10-05T19:40:21.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.579+0000 7f6679ec2700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbcbd0 con 0x7f6674007030 2023-10-05T19:40:21.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.579+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f666c014040 con 0x7f6674007030 2023-10-05T19:40:21.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.579+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbcbd0 con 0x5608e7dba2b0 2023-10-05T19:40:21.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.579+0000 7f6679ec2700 1 -- 172.21.15.12:0/2787179272 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6664004520 con 0x5608e7dba2b0 2023-10-05T19:40:21.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.579+0000 7f6679ec2700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbcbd0 con 0x5608e7dba2b0 2023-10-05T19:40:21.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.579+0000 7f6679ec2700 1 -- 172.21.15.12:0/2787179272 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6664004520 con 0x5608e7dba2b0 2023-10-05T19:40:21.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.579+0000 7f6679ec2700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbcbd0 con 0x5608e7d807e0 2023-10-05T19:40:21.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.579+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f666c014040 con 0x5608e7d807e0 2023-10-05T19:40:21.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.579+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbcbd0 con 0x5608e7d807e0 2023-10-05T19:40:21.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.583+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f666c014040 con 0x5608e7d807e0 2023-10-05T19:40:21.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.583+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbcbd0 con 0x7f6674007030 2023-10-05T19:40:21.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.583+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f666c014040 con 0x7f6674007030 2023-10-05T19:40:21.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.583+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbcbd0 con 0x5608e7dba2b0 2023-10-05T19:40:21.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.583+0000 7f6679ec2700 1 -- 172.21.15.12:0/2787179272 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6664004520 con 0x5608e7dba2b0 2023-10-05T19:40:21.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.583+0000 7f6679ec2700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbcbd0 con 0x5608e7db5930 2023-10-05T19:40:21.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.583+0000 7f667a6c3700 1 -- 172.21.15.12:0/2787179272 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6674011570 con 0x5608e7db5930 2023-10-05T19:40:21.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.583+0000 7f667a6c3700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbcbd0 con 0x5608e7d807e0 2023-10-05T19:40:21.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.587+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f666c014040 con 0x5608e7d807e0 2023-10-05T19:40:21.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.587+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbcbd0 con 0x7f6674007030 2023-10-05T19:40:21.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.587+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f666c014040 con 0x7f6674007030 2023-10-05T19:40:21.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.587+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbcbd0 con 0x5608e7dba2b0 2023-10-05T19:40:21.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.587+0000 7f6679ec2700 1 -- 172.21.15.12:0/2787179272 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6664004520 con 0x5608e7dba2b0 2023-10-05T19:40:21.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.587+0000 7f6679ec2700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbcbd0 con 0x7f6674007030 2023-10-05T19:40:21.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.587+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f666c014040 con 0x7f6674007030 2023-10-05T19:40:21.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.587+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbcbd0 con 0x5608e7db5930 2023-10-05T19:40:21.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.591+0000 7f667a6c3700 1 -- 172.21.15.12:0/2787179272 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6674011570 con 0x5608e7db5930 2023-10-05T19:40:21.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.591+0000 7f667a6c3700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbcbd0 con 0x5608e7dba2b0 2023-10-05T19:40:21.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.591+0000 7f6679ec2700 1 -- 172.21.15.12:0/2787179272 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6664004520 con 0x5608e7dba2b0 2023-10-05T19:40:21.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.591+0000 7f6679ec2700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbcbd0 con 0x7f6674007030 2023-10-05T19:40:21.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.591+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f666c014040 con 0x7f6674007030 2023-10-05T19:40:21.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.591+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbcbd0 con 0x5608e7d807e0 2023-10-05T19:40:21.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.591+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f666c014040 con 0x5608e7d807e0 2023-10-05T19:40:21.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.591+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbcbd0 con 0x5608e7db5930 2023-10-05T19:40:21.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.595+0000 7f667a6c3700 1 -- 172.21.15.12:0/2787179272 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f6674011570 con 0x5608e7db5930 2023-10-05T19:40:21.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.595+0000 7f667cb8ee40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:21.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.595+0000 7f667cb8ee40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:21.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.595+0000 7f667cb8ee40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:21.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.595+0000 7f667cb8ee40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:21.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.595+0000 7f667cb8ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.595+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7db9060 con 0x5608e7d807e0 2023-10-05T19:40:21.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.595+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f666c014040 con 0x5608e7d807e0 2023-10-05T19:40:21.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.595+0000 7f667cb8ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:21.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.595+0000 7f667cb8ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.595+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7db99b0 con 0x5608e7d807e0 2023-10-05T19:40:21.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.595+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f666c014040 con 0x5608e7d807e0 2023-10-05T19:40:21.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.595+0000 7f667cb8ee40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:21.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.595+0000 7f667cb8ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.595+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7db8c20 con 0x5608e7dba2b0 2023-10-05T19:40:21.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.595+0000 7f6679ec2700 1 -- 172.21.15.12:0/2787179272 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f6664004520 con 0x5608e7dba2b0 2023-10-05T19:40:21.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.595+0000 7f667cb8ee40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:21.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.595+0000 7f667cb8ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.595+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbd470 con 0x5608e7dba2b0 2023-10-05T19:40:21.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.595+0000 7f6679ec2700 1 -- 172.21.15.12:0/2787179272 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f6664004520 con 0x5608e7dba2b0 2023-10-05T19:40:21.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.595+0000 7f667cb8ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:21.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.595+0000 7f667cb8ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.595+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbd820 con 0x5608e7d807e0 2023-10-05T19:40:21.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.599+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f666c014040 con 0x5608e7d807e0 2023-10-05T19:40:21.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.599+0000 7f667cb8ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:21.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.599+0000 7f667cb8ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.599+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbdbd0 con 0x5608e7d807e0 2023-10-05T19:40:21.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.599+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f666c014040 con 0x5608e7d807e0 2023-10-05T19:40:21.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.599+0000 7f667cb8ee40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:21.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.599+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbebe0 con 0x5608e7dba2b0 2023-10-05T19:40:21.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.599+0000 7f6679ec2700 1 -- 172.21.15.12:0/2787179272 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f6664004520 con 0x5608e7dba2b0 2023-10-05T19:40:21.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.603+0000 7f6679ec2700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbebe0 con 0x5608e7db5930 2023-10-05T19:40:21.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.603+0000 7f667a6c3700 1 -- 172.21.15.12:0/2787179272 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6674011570 con 0x5608e7db5930 2023-10-05T19:40:21.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.603+0000 7f667a6c3700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbebe0 con 0x7f6674007030 2023-10-05T19:40:21.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.603+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f666c014040 con 0x7f6674007030 2023-10-05T19:40:21.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.603+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbd820 con 0x5608e7d807e0 2023-10-05T19:40:21.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.603+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f666c014040 con 0x5608e7d807e0 2023-10-05T19:40:21.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.603+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbd820 con 0x5608e7d807e0 2023-10-05T19:40:21.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.603+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f666c014040 con 0x5608e7d807e0 2023-10-05T19:40:21.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.603+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbd820 con 0x7f6674007030 2023-10-05T19:40:21.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.607+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f666c014040 con 0x7f6674007030 2023-10-05T19:40:21.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.607+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbd820 con 0x7f6674007030 2023-10-05T19:40:21.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.607+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f666c014040 con 0x7f6674007030 2023-10-05T19:40:21.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.607+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbd820 con 0x5608e7db5930 2023-10-05T19:40:21.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.607+0000 7f667a6c3700 1 -- 172.21.15.12:0/2787179272 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6674011570 con 0x5608e7db5930 2023-10-05T19:40:21.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.607+0000 7f667a6c3700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbd820 con 0x5608e7dba2b0 2023-10-05T19:40:21.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.607+0000 7f6679ec2700 1 -- 172.21.15.12:0/2787179272 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f666402d040 con 0x5608e7dba2b0 2023-10-05T19:40:21.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.607+0000 7f6679ec2700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbd820 con 0x5608e7d807e0 2023-10-05T19:40:21.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.607+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f666c014040 con 0x5608e7d807e0 2023-10-05T19:40:21.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.607+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbd820 con 0x7f6674007030 2023-10-05T19:40:21.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.611+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f666c014040 con 0x7f6674007030 2023-10-05T19:40:21.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.611+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbd820 con 0x5608e7d807e0 2023-10-05T19:40:21.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.611+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f666c014040 con 0x5608e7d807e0 2023-10-05T19:40:21.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.611+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbd820 con 0x5608e7db5930 2023-10-05T19:40:21.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.611+0000 7f667a6c3700 1 -- 172.21.15.12:0/2787179272 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6674011570 con 0x5608e7db5930 2023-10-05T19:40:21.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.611+0000 7f667a6c3700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbd820 con 0x5608e7dba2b0 2023-10-05T19:40:21.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.611+0000 7f6679ec2700 1 -- 172.21.15.12:0/2787179272 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f666402d040 con 0x5608e7dba2b0 2023-10-05T19:40:21.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.611+0000 7f6679ec2700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbd820 con 0x5608e7dba2b0 2023-10-05T19:40:21.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.615+0000 7f6679ec2700 1 -- 172.21.15.12:0/2787179272 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f666402d040 con 0x5608e7dba2b0 2023-10-05T19:40:21.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.615+0000 7f6679ec2700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbd820 con 0x7f6674007030 2023-10-05T19:40:21.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.615+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f666c014040 con 0x7f6674007030 2023-10-05T19:40:21.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.615+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbd820 con 0x5608e7dba2b0 2023-10-05T19:40:21.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.615+0000 7f6679ec2700 1 -- 172.21.15.12:0/2787179272 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f666402d040 con 0x5608e7dba2b0 2023-10-05T19:40:21.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.615+0000 7f6679ec2700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6664031040 con 0x5608e7dba2b0 2023-10-05T19:40:21.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.615+0000 7f6679ec2700 1 -- 172.21.15.12:0/2787179272 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f666402d040 con 0x5608e7dba2b0 2023-10-05T19:40:21.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.615+0000 7f6679ec2700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6664031040 con 0x5608e7d807e0 2023-10-05T19:40:21.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.619+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f666c014040 con 0x5608e7d807e0 2023-10-05T19:40:21.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.619+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6664031040 con 0x5608e7d807e0 2023-10-05T19:40:21.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.619+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f666c014040 con 0x5608e7d807e0 2023-10-05T19:40:21.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.619+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6664031040 con 0x7f6674007030 2023-10-05T19:40:21.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.619+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f666c014040 con 0x7f6674007030 2023-10-05T19:40:21.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.619+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6664031040 con 0x5608e7dba2b0 2023-10-05T19:40:21.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.619+0000 7f6679ec2700 1 -- 172.21.15.12:0/2787179272 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f666402d040 con 0x5608e7dba2b0 2023-10-05T19:40:21.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.619+0000 7f6679ec2700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6664031040 con 0x5608e7db5930 2023-10-05T19:40:21.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.619+0000 7f667a6c3700 1 -- 172.21.15.12:0/2787179272 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6674011570 con 0x5608e7db5930 2023-10-05T19:40:21.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.623+0000 7f667a6c3700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6664031040 con 0x5608e7d807e0 2023-10-05T19:40:21.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.623+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f666c014040 con 0x5608e7d807e0 2023-10-05T19:40:21.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.623+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6664031040 con 0x7f6674007030 2023-10-05T19:40:21.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.623+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f666c014040 con 0x7f6674007030 2023-10-05T19:40:21.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.623+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6664031040 con 0x5608e7dba2b0 2023-10-05T19:40:21.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.623+0000 7f6679ec2700 1 -- 172.21.15.12:0/2787179272 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f666402d040 con 0x5608e7dba2b0 2023-10-05T19:40:21.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.623+0000 7f6679ec2700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6664031040 con 0x7f6674007030 2023-10-05T19:40:21.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.623+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f666c014040 con 0x7f6674007030 2023-10-05T19:40:21.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.623+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6664031040 con 0x5608e7db5930 2023-10-05T19:40:21.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.627+0000 7f667a6c3700 1 -- 172.21.15.12:0/2787179272 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6674011570 con 0x5608e7db5930 2023-10-05T19:40:21.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.627+0000 7f667a6c3700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6664031040 con 0x5608e7dba2b0 2023-10-05T19:40:21.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.627+0000 7f6679ec2700 1 -- 172.21.15.12:0/2787179272 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f666402d040 con 0x5608e7dba2b0 2023-10-05T19:40:21.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.627+0000 7f6679ec2700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6664031040 con 0x7f6674007030 2023-10-05T19:40:21.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.627+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f666c014040 con 0x7f6674007030 2023-10-05T19:40:21.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.627+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6664031040 con 0x5608e7d807e0 2023-10-05T19:40:21.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.631+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f666c014040 con 0x5608e7d807e0 2023-10-05T19:40:21.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.631+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6664031040 con 0x5608e7db5930 2023-10-05T19:40:21.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.631+0000 7f667a6c3700 1 -- 172.21.15.12:0/2787179272 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f6674011570 con 0x5608e7db5930 2023-10-05T19:40:21.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.631+0000 7f667cb8ee40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:21.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.631+0000 7f667cb8ee40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:21.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.631+0000 7f667cb8ee40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:21.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.631+0000 7f667cb8ee40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:21.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.631+0000 7f667cb8ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.631+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbdbd0 con 0x5608e7d807e0 2023-10-05T19:40:21.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.631+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f666c014040 con 0x5608e7d807e0 2023-10-05T19:40:21.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.631+0000 7f667cb8ee40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:21.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.631+0000 7f667cb8ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.631+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbdf80 con 0x5608e7db5930 2023-10-05T19:40:21.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.631+0000 7f667a6c3700 1 -- 172.21.15.12:0/2787179272 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f6674011570 con 0x5608e7db5930 2023-10-05T19:40:21.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.631+0000 7f667cb8ee40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:21.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.631+0000 7f667cb8ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.631+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbe820 con 0x5608e7d807e0 2023-10-05T19:40:21.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.631+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f666c014040 con 0x5608e7d807e0 2023-10-05T19:40:21.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.631+0000 7f667cb8ee40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:21.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.631+0000 7f667cb8ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.631+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbe820 con 0x5608e7dba2b0 2023-10-05T19:40:21.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.635+0000 7f6679ec2700 1 -- 172.21.15.12:0/2787179272 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f666402d040 con 0x5608e7dba2b0 2023-10-05T19:40:21.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.635+0000 7f667cb8ee40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:21.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.635+0000 7f667cb8ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.635+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbebd0 con 0x7f6674007030 2023-10-05T19:40:21.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.635+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f666c014040 con 0x7f6674007030 2023-10-05T19:40:21.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.635+0000 7f667cb8ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:21.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.635+0000 7f667cb8ee40 20 zone default found 2023-10-05T19:40:21.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.635+0000 7f667cb8ee40 4 Realm: () 2023-10-05T19:40:21.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.635+0000 7f667cb8ee40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:21.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.635+0000 7f667cb8ee40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:21.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.635+0000 7f667cb8ee40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:21.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.635+0000 7f667cb8ee40 20 zonegroup default 2023-10-05T19:40:21.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.635+0000 7f667cb8ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.635+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbebd0 con 0x7f6674007030 2023-10-05T19:40:21.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.635+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f666c014040 con 0x7f6674007030 2023-10-05T19:40:21.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.635+0000 7f667cb8ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:21.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.635+0000 7f667cb8ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.635+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dc0550 con 0x5608e7dba2b0 2023-10-05T19:40:21.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.635+0000 7f6679ec2700 1 -- 172.21.15.12:0/2787179272 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f666402d040 con 0x5608e7dba2b0 2023-10-05T19:40:21.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.635+0000 7f667cb8ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:21.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.635+0000 7f667cb8ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.635+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dc0900 con 0x5608e7dba2b0 2023-10-05T19:40:21.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.639+0000 7f6679ec2700 1 -- 172.21.15.12:0/2787179272 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f666402d040 con 0x5608e7dba2b0 2023-10-05T19:40:21.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.639+0000 7f667cb8ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:21.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.639+0000 7f667cb8ee40 20 started sync module instance, tier type = 2023-10-05T19:40:21.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.639+0000 7f667cb8ee40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:21.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.639+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbfc10 con 0x5608e7db5930 2023-10-05T19:40:21.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.639+0000 7f667a6c3700 1 -- 172.21.15.12:0/2787179272 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'69 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6674011570 con 0x5608e7db5930 2023-10-05T19:40:21.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.643+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94596249681936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dbe450 con 0x5608e7db5930 2023-10-05T19:40:21.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.643+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dc1690 con 0x5608e7d807e0 2023-10-05T19:40:21.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.643+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'69 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f666c014040 con 0x5608e7d807e0 2023-10-05T19:40:21.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.643+0000 7f667a6c3700 1 -- 172.21.15.12:0/2787179272 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94596249681936] v28'70 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6674011570 con 0x5608e7db5930 2023-10-05T19:40:21.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.643+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94596249685456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dc1a90 con 0x5608e7d807e0 2023-10-05T19:40:21.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.643+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dc2370 con 0x5608e7dba2b0 2023-10-05T19:40:21.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.647+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94596249685456] v28'70 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f666c014040 con 0x5608e7d807e0 2023-10-05T19:40:21.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.647+0000 7f6679ec2700 1 -- 172.21.15.12:0/2787179272 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'69 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f666402d040 con 0x5608e7dba2b0 2023-10-05T19:40:21.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.647+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94596249686912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dc2720 con 0x5608e7dba2b0 2023-10-05T19:40:21.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.647+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dc2fc0 con 0x5608e7db5930 2023-10-05T19:40:21.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.651+0000 7f667a6c3700 1 -- 172.21.15.12:0/2787179272 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'69 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6674011570 con 0x5608e7db5930 2023-10-05T19:40:21.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.651+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94596249692224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dc38b0 con 0x5608e7db5930 2023-10-05T19:40:21.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.651+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dc42c0 con 0x5608e7d807e0 2023-10-05T19:40:21.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.651+0000 7f6679ec2700 1 -- 172.21.15.12:0/2787179272 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94596249686912] v28'70 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f666402d040 con 0x5608e7dba2b0 2023-10-05T19:40:21.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.651+0000 7f667a6c3700 1 -- 172.21.15.12:0/2787179272 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94596249692224] v28'70 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6674011570 con 0x5608e7db5930 2023-10-05T19:40:21.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.651+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'69 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f666c014040 con 0x5608e7d807e0 2023-10-05T19:40:21.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.651+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94596249695168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dc3810 con 0x5608e7d807e0 2023-10-05T19:40:21.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.651+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dc47b0 con 0x7f6674007030 2023-10-05T19:40:21.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.655+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'69 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f666c014040 con 0x7f6674007030 2023-10-05T19:40:21.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.655+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94596249699008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dc4c00 con 0x7f6674007030 2023-10-05T19:40:21.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.655+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dc5610 con 0x7f6674007030 2023-10-05T19:40:21.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.655+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94596249695168] v28'70 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f666c014040 con 0x5608e7d807e0 2023-10-05T19:40:21.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.659+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(96 notify.6 [create] v28'69 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f666c014040 con 0x7f6674007030 2023-10-05T19:40:21.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.659+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94596249697744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dc5610 con 0x7f6674007030 2023-10-05T19:40:21.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.659+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dc6020 con 0x7f6674007030 2023-10-05T19:40:21.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.659+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(95 notify.5 [watch watch cookie 94596249699008] v28'70 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f666c014040 con 0x7f6674007030 2023-10-05T19:40:21.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.659+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(98 notify.7 [create] v28'69 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f666c014040 con 0x7f6674007030 2023-10-05T19:40:21.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.659+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(97 notify.6 [watch watch cookie 94596249697744] v28'70 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f666c014040 con 0x7f6674007030 2023-10-05T19:40:21.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.659+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94596249700112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dc5b30 con 0x7f6674007030 2023-10-05T19:40:21.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.659+0000 7f667cb8ee40 20 add_watcher() i=0 2023-10-05T19:40:21.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.659+0000 7f667cb8ee40 20 add_watcher() i=1 2023-10-05T19:40:21.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.659+0000 7f667cb8ee40 20 add_watcher() i=2 2023-10-05T19:40:21.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.659+0000 7f667cb8ee40 20 add_watcher() i=3 2023-10-05T19:40:21.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.659+0000 7f667cb8ee40 20 add_watcher() i=4 2023-10-05T19:40:21.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.659+0000 7f667cb8ee40 20 add_watcher() i=5 2023-10-05T19:40:21.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.659+0000 7f667cb8ee40 20 add_watcher() i=6 2023-10-05T19:40:21.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.663+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94596249700112] v28'70 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f666c014040 con 0x7f6674007030 2023-10-05T19:40:21.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.663+0000 7f667cb8ee40 20 add_watcher() i=7 2023-10-05T19:40:21.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.663+0000 7f667cb8ee40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:21.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.663+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dc8b90 con 0x5608e7d807e0 2023-10-05T19:40:21.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.663+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f666c014040 con 0x5608e7d807e0 2023-10-05T19:40:21.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.663+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94596249702688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7dc6020 con 0x5608e7d807e0 2023-10-05T19:40:21.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.667+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94596249702688] v28'80 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f666c014040 con 0x5608e7d807e0 2023-10-05T19:40:21.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.667+0000 7f664bfdf700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:21.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.667+0000 7f667cb8ee40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:21.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.667+0000 7f667cb8ee40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:21.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.667+0000 7f667cb8ee40 1 Processor -- start 2023-10-05T19:40:21.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.667+0000 7f667cb8ee40 1 -- start start 2023-10-05T19:40:21.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.667+0000 7f667cb8ee40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5608e7dd3070 0x5608e7dd3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.667+0000 7f667cb8ee40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e7dd3980 con 0x5608e7dd3070 2023-10-05T19:40:21.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.667+0000 7f6679ec2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5608e7dd3070 0x5608e7dd3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.667+0000 7f6679ec2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5608e7dd3070 0x5608e7dd3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37342/0 (socket says 172.21.15.12:37342) 2023-10-05T19:40:21.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.667+0000 7f6679ec2700 1 -- 172.21.15.12:0/1075456214 learned_addr learned my addr 172.21.15.12:0/1075456214 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:21.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.667+0000 7f6679ec2700 1 -- 172.21.15.12:0/1075456214 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608e7dd3ac0 con 0x5608e7dd3070 2023-10-05T19:40:21.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.667+0000 7f6679ec2700 1 --2- 172.21.15.12:0/1075456214 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5608e7dd3070 0x5608e7dd3440 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6664000f80 tx=0x7f6664027040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d243323b2a241e38 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.667+0000 7f664b7de700 1 -- 172.21.15.12:0/1075456214 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6664047070 con 0x5608e7dd3070 2023-10-05T19:40:21.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.671+0000 7f664b7de700 1 -- 172.21.15.12:0/1075456214 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6664039040 con 0x5608e7dd3070 2023-10-05T19:40:21.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.671+0000 7f664b7de700 1 -- 172.21.15.12:0/1075456214 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6664038040 con 0x5608e7dd3070 2023-10-05T19:40:21.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.671+0000 7f667cb8ee40 1 -- 172.21.15.12:0/1075456214 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5608e7dd3070 msgr2=0x5608e7dd3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.671+0000 7f667cb8ee40 1 --2- 172.21.15.12:0/1075456214 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5608e7dd3070 0x5608e7dd3440 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6664000f80 tx=0x7f6664027040 comp rx=0 tx=0).stop 2023-10-05T19:40:21.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.671+0000 7f667cb8ee40 1 -- 172.21.15.12:0/1075456214 shutdown_connections 2023-10-05T19:40:21.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.671+0000 7f667cb8ee40 1 --2- 172.21.15.12:0/1075456214 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5608e7dd3070 0x5608e7dd3440 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.671+0000 7f667cb8ee40 1 -- 172.21.15.12:0/1075456214 >> 172.21.15.12:0/1075456214 conn(0x5608e7dce430 msgr2=0x5608e7dcc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:21.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.671+0000 7f667cb8ee40 1 -- 172.21.15.12:0/1075456214 shutdown_connections 2023-10-05T19:40:21.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.671+0000 7f667cb8ee40 1 -- 172.21.15.12:0/1075456214 wait complete. 2023-10-05T19:40:21.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.671+0000 7f667cb8ee40 1 Processor -- start 2023-10-05T19:40:21.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.671+0000 7f667cb8ee40 1 -- start start 2023-10-05T19:40:21.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.671+0000 7f667cb8ee40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5608e7dd3070 0x5608e7dd8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.671+0000 7f667cb8ee40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e7dd9b80 con 0x5608e7dd3070 2023-10-05T19:40:21.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.671+0000 7f6679ec2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5608e7dd3070 0x5608e7dd8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.671+0000 7f6679ec2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5608e7dd3070 0x5608e7dd8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37356/0 (socket says 172.21.15.12:37356) 2023-10-05T19:40:21.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.671+0000 7f6679ec2700 1 -- 172.21.15.12:0/565328111 learned_addr learned my addr 172.21.15.12:0/565328111 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:21.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.671+0000 7f6679ec2700 1 -- 172.21.15.12:0/565328111 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608e7cccbd0 con 0x5608e7dd3070 2023-10-05T19:40:21.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.671+0000 7f6679ec2700 1 --2- 172.21.15.12:0/565328111 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5608e7dd3070 0x5608e7dd8b80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6664001d40 tx=0x7f6664042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.675+0000 7f6649fdb700 1 -- 172.21.15.12:0/565328111 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6664038030 con 0x5608e7dd3070 2023-10-05T19:40:21.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.675+0000 7f6649fdb700 1 -- 172.21.15.12:0/565328111 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f666401b040 con 0x5608e7dd3070 2023-10-05T19:40:21.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.675+0000 7f667cb8ee40 1 -- 172.21.15.12:0/565328111 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5608e7dd9180 con 0x5608e7dd3070 2023-10-05T19:40:21.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.675+0000 7f6649fdb700 1 -- 172.21.15.12:0/565328111 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f666402a040 con 0x5608e7dd3070 2023-10-05T19:40:21.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.675+0000 7f667cb8ee40 1 -- 172.21.15.12:0/565328111 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5608e7dd9620 con 0x5608e7dd3070 2023-10-05T19:40:21.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.675+0000 7f6649fdb700 1 -- 172.21.15.12:0/565328111 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f666404a020 con 0x5608e7dd3070 2023-10-05T19:40:21.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.675+0000 7f6649fdb700 1 --2- 172.21.15.12:0/565328111 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f65c0030bb0 0x7f65c0033060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.675+0000 7f6649fdb700 1 -- 172.21.15.12:0/565328111 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f6664028040 con 0x5608e7dd3070 2023-10-05T19:40:21.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.675+0000 7f667aec4700 1 --2- 172.21.15.12:0/565328111 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f65c0030bb0 0x7f65c0033060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.675+0000 7f667cb8ee40 5 note: GC not initialized 2023-10-05T19:40:21.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.675+0000 7f667aec4700 1 --2- 172.21.15.12:0/565328111 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f65c0030bb0 0x7f65c0033060 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x5608e7dc55e0 tx=0x7f666c00af80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.679+0000 7f65baffd700 20 reqs_thread_entry: start 2023-10-05T19:40:21.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.679+0000 7f65b8ff9700 20 reqs_thread_entry: start 2023-10-05T19:40:21.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.679+0000 7f667cb8ee40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:21.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.679+0000 7f667cb8ee40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:21.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.679+0000 7f65aaffd700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:52 2023 2023-10-05T19:40:21.688 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:21.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.679+0000 7f65aa7fc700 20 reqs_thread_entry: start 2023-10-05T19:40:21.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.679+0000 7f667cb8ee40 20 get_system_obj_state: rctx=0x7ffd27560ec8 obj=default.rgw.meta:users.uid:foo state=0x5608e7c590b0 s->prefetch_data=0 2023-10-05T19:40:21.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.679+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7e414f0 con 0x5608e7d807e0 2023-10-05T19:40:21.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.679+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(102 foo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7f666c014040 con 0x5608e7d807e0 2023-10-05T19:40:21.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.679+0000 7f667cb8ee40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:21.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.679+0000 7f667cb8ee40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:21.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.679+0000 7f667cb8ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.679+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7e414d0 con 0x5608e7d807e0 2023-10-05T19:40:21.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.679+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(103 foo [call,call out=48b,read 0~621 out=621b,getxattrs out=75b] v0'0 uv8 ondisk = 0) v8 ==== 273+0+744 (crc 0 0 0) 0x7f666c014040 con 0x5608e7d807e0 2023-10-05T19:40:21.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.679+0000 7f667cb8ee40 20 rados_obj.operate() r=0 bl.length=621 2023-10-05T19:40:21.690 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:40:21.690 INFO:teuthology.orchestra.run.smithi012.stdout: "user_id": "foo", 2023-10-05T19:40:21.690 INFO:teuthology.orchestra.run.smithi012.stdout: "display_name": "Foo", 2023-10-05T19:40:21.690 INFO:teuthology.orchestra.run.smithi012.stdout: "email": "foo@foo.com", 2023-10-05T19:40:21.690 INFO:teuthology.orchestra.run.smithi012.stdout: "suspended": 0, 2023-10-05T19:40:21.690 INFO:teuthology.orchestra.run.smithi012.stdout: "max_buckets": 4, 2023-10-05T19:40:21.691 INFO:teuthology.orchestra.run.smithi012.stdout: "subusers": [ 2023-10-05T19:40:21.691 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:21.691 INFO:teuthology.orchestra.run.smithi012.stdout: "id": "foo:foo1", 2023-10-05T19:40:21.691 INFO:teuthology.orchestra.run.smithi012.stdout: "permissions": "full-control" 2023-10-05T19:40:21.691 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:21.691 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:21.691 INFO:teuthology.orchestra.run.smithi012.stdout: "id": "foo:foo2", 2023-10-05T19:40:21.691 INFO:teuthology.orchestra.run.smithi012.stdout: "permissions": "" 2023-10-05T19:40:21.691 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:21.691 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:21.691 INFO:teuthology.orchestra.run.smithi012.stdout: "keys": [ 2023-10-05T19:40:21.691 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:21.692 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:40:21.692 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2023-10-05T19:40:21.692 INFO:teuthology.orchestra.run.smithi012.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2023-10-05T19:40:21.692 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:21.692 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:21.692 INFO:teuthology.orchestra.run.smithi012.stdout: "swift_keys": [ 2023-10-05T19:40:21.692 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:21.692 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo:foo1", 2023-10-05T19:40:21.692 INFO:teuthology.orchestra.run.smithi012.stdout: "secret_key": "gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL" 2023-10-05T19:40:21.692 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:21.692 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:21.692 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo:foo2", 2023-10-05T19:40:21.693 INFO:teuthology.orchestra.run.smithi012.stdout: "secret_key": "ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy" 2023-10-05T19:40:21.693 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:21.693 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:21.693 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": [], 2023-10-05T19:40:21.693 INFO:teuthology.orchestra.run.smithi012.stdout: "op_mask": "read, write, delete", 2023-10-05T19:40:21.693 INFO:teuthology.orchestra.run.smithi012.stdout: "default_placement": "", 2023-10-05T19:40:21.693 INFO:teuthology.orchestra.run.smithi012.stdout: "default_storage_class": "", 2023-10-05T19:40:21.693 INFO:teuthology.orchestra.run.smithi012.stdout: "placement_tags": [], 2023-10-05T19:40:21.693 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket_quota": { 2023-10-05T19:40:21.693 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:21.693 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:21.693 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:21.694 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:21.694 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:21.694 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:21.694 INFO:teuthology.orchestra.run.smithi012.stdout: "user_quota": { 2023-10-05T19:40:21.694 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:21.694 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:21.694 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:21.694 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:21.694 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:21.694 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:21.694 INFO:teuthology.orchestra.run.smithi012.stdout: "temp_url_keys": [], 2023-10-05T19:40:21.694 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "rgw", 2023-10-05T19:40:21.695 INFO:teuthology.orchestra.run.smithi012.stdout: "mfa_ids": [] 2023-10-05T19:40:21.695 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:40:21.695 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:40:21.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.683+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:104 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94596249681936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7e414d0 con 0x5608e7db5930 2023-10-05T19:40:21.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.683+0000 7f667a6c3700 1 -- 172.21.15.12:0/2787179272 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(104 notify.0 [watch unwatch cookie 94596249681936] v28'71 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6674011570 con 0x5608e7db5930 2023-10-05T19:40:21.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.683+0000 7f667cb8ee40 20 remove_watcher() i=0 2023-10-05T19:40:21.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.683+0000 7f667cb8ee40 2 removed watcher, disabling cache 2023-10-05T19:40:21.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.683+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:105 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94596249685456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7e414d0 con 0x5608e7d807e0 2023-10-05T19:40:21.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.687+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(105 notify.1 [watch unwatch cookie 94596249685456] v28'71 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f666c014040 con 0x5608e7d807e0 2023-10-05T19:40:21.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.687+0000 7f667cb8ee40 20 remove_watcher() i=1 2023-10-05T19:40:21.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.687+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:106 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94596249686912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7e414d0 con 0x5608e7dba2b0 2023-10-05T19:40:21.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.691+0000 7f6679ec2700 1 -- 172.21.15.12:0/2787179272 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(106 notify.2 [watch unwatch cookie 94596249686912] v28'71 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f666403b070 con 0x5608e7dba2b0 2023-10-05T19:40:21.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.691+0000 7f667cb8ee40 20 remove_watcher() i=2 2023-10-05T19:40:21.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.691+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:107 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94596249692224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7e414d0 con 0x5608e7db5930 2023-10-05T19:40:21.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.691+0000 7f667a6c3700 1 -- 172.21.15.12:0/2787179272 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(107 notify.3 [watch unwatch cookie 94596249692224] v28'71 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6674011570 con 0x5608e7db5930 2023-10-05T19:40:21.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.691+0000 7f667cb8ee40 20 remove_watcher() i=3 2023-10-05T19:40:21.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.691+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:108 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94596249695168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7e414d0 con 0x5608e7d807e0 2023-10-05T19:40:21.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.695+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(108 notify.4 [watch unwatch cookie 94596249695168] v28'71 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f666c014040 con 0x5608e7d807e0 2023-10-05T19:40:21.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.695+0000 7f667cb8ee40 20 remove_watcher() i=4 2023-10-05T19:40:21.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.695+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:109 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94596249699008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7e414d0 con 0x7f6674007030 2023-10-05T19:40:21.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.699+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(109 notify.5 [watch unwatch cookie 94596249699008] v28'71 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f666c014040 con 0x7f6674007030 2023-10-05T19:40:21.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.699+0000 7f667cb8ee40 20 remove_watcher() i=5 2023-10-05T19:40:21.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.699+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:110 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94596249697744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7e414d0 con 0x7f6674007030 2023-10-05T19:40:21.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.703+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(110 notify.6 [watch unwatch cookie 94596249697744] v28'71 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f666c014040 con 0x7f6674007030 2023-10-05T19:40:21.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.703+0000 7f667cb8ee40 20 remove_watcher() i=6 2023-10-05T19:40:21.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.703+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:111 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94596249700112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7e414d0 con 0x7f6674007030 2023-10-05T19:40:21.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.707+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(111 notify.7 [watch unwatch cookie 94596249700112] v28'71 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f666c014040 con 0x7f6674007030 2023-10-05T19:40:21.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.707+0000 7f667cb8ee40 20 remove_watcher() i=7 2023-10-05T19:40:21.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.707+0000 7f664bfdf700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:21.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.707+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:112 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94596249702688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5608e7e414d0 con 0x5608e7d807e0 2023-10-05T19:40:21.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.711+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(112 data_loggenerations_metadata [watch unwatch cookie 94596249702688] v28'81 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f666c014040 con 0x5608e7d807e0 2023-10-05T19:40:21.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.711+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f6674007030 msgr2=0x7f6674011830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.711+0000 7f667cb8ee40 1 --2- 172.21.15.12:0/2787179272 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f6674007030 0x7f6674011830 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.711+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5608e7d807e0 msgr2=0x5608e7db3810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.711+0000 7f667cb8ee40 1 --2- 172.21.15.12:0/2787179272 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5608e7d807e0 0x5608e7db3810 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.711+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5608e7db5930 msgr2=0x5608e7db7d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.711+0000 7f667cb8ee40 1 --2- 172.21.15.12:0/2787179272 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5608e7db5930 0x5608e7db7d10 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.711+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5608e7dba2b0 msgr2=0x5608e7dbc690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.711+0000 7f667cb8ee40 1 --2- 172.21.15.12:0/2787179272 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5608e7dba2b0 0x5608e7dbc690 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.711+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f665c030c00 msgr2=0x7f665c0330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.711+0000 7f667cb8ee40 1 --2- 172.21.15.12:0/2787179272 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f665c030c00 0x7f665c0330b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f66640044f0 tx=0x7f66640067c0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.711+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5608e7d80410 msgr2=0x5608e7da7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.711+0000 7f667cb8ee40 1 --2- 172.21.15.12:0/2787179272 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5608e7d80410 0x5608e7da7d20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f667400c010 tx=0x7f667401c000 comp rx=0 tx=0).stop 2023-10-05T19:40:21.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.711+0000 7f667aec4700 1 -- 172.21.15.12:0/2787179272 reap_dead start 2023-10-05T19:40:21.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.711+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 shutdown_connections 2023-10-05T19:40:21.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.711+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 >> 172.21.15.12:0/2787179272 conn(0x5608e7d7a720 msgr2=0x5608e7d9b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:21.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.711+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 shutdown_connections 2023-10-05T19:40:21.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.711+0000 7f667cb8ee40 1 -- 172.21.15.12:0/2787179272 wait complete. 2023-10-05T19:40:21.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.711+0000 7f667cb8ee40 1 -- 172.21.15.12:0/565328111 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f65c0030bb0 msgr2=0x7f65c0033060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.711+0000 7f667cb8ee40 1 --2- 172.21.15.12:0/565328111 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f65c0030bb0 0x7f65c0033060 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x5608e7dc55e0 tx=0x7f666c00af80 comp rx=0 tx=0).stop 2023-10-05T19:40:21.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.711+0000 7f667cb8ee40 1 -- 172.21.15.12:0/565328111 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5608e7dd3070 msgr2=0x5608e7dd8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.711+0000 7f667cb8ee40 1 --2- 172.21.15.12:0/565328111 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5608e7dd3070 0x5608e7dd8b80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6664001d40 tx=0x7f6664042000 comp rx=0 tx=0).stop 2023-10-05T19:40:21.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.711+0000 7f667cb8ee40 1 -- 172.21.15.12:0/565328111 shutdown_connections 2023-10-05T19:40:21.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.711+0000 7f667cb8ee40 1 --2- 172.21.15.12:0/565328111 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f65c0030bb0 0x7f65c0033060 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.711+0000 7f667cb8ee40 1 --2- 172.21.15.12:0/565328111 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5608e7dd3070 0x5608e7dd8b80 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.711+0000 7f667cb8ee40 1 -- 172.21.15.12:0/565328111 >> 172.21.15.12:0/565328111 conn(0x5608e7dce430 msgr2=0x5608e7dcc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:21.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.711+0000 7f667cb8ee40 1 -- 172.21.15.12:0/565328111 shutdown_connections 2023-10-05T19:40:21.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.711+0000 7f667cb8ee40 1 -- 172.21.15.12:0/565328111 wait complete. 2023-10-05T19:40:21.726 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}, {'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL'}, {'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:40:21.727 INFO:tasks.util.rgw:rgwadmin: client.0 : ['key', 'rm', '--subuser', 'foo:foo1', '--key-type', 'swift'] 2023-10-05T19:40:21.727 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'key', 'rm', '--subuser', 'foo:foo1', '--key-type', 'swift'] 2023-10-05T19:40:21.727 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph key rm --subuser foo:foo1 --key-type swift 2023-10-05T19:40:21.765 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:21.765 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:21.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.759+0000 7fd54505fe40 1 Processor -- start 2023-10-05T19:40:21.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.759+0000 7fd54505fe40 1 -- start start 2023-10-05T19:40:21.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.759+0000 7fd54505fe40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x558655501d10 0x5586555020e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.759+0000 7fd54505fe40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55865544fbd0 con 0x558655502620 2023-10-05T19:40:21.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.759+0000 7fd54505fe40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x55865540cee0 con 0x558655501d10 2023-10-05T19:40:21.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.759+0000 7fd542b94700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x558655501d10 0x5586555020e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.759+0000 7fd542393700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x558655502620 0x5586555030c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:47342/0 (socket says 172.21.15.12:47342) 2023-10-05T19:40:21.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.759+0000 7fd542393700 1 -- 172.21.15.12:0/570675203 learned_addr learned my addr 172.21.15.12:0/570675203 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:21.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.759+0000 7fd541b92700 1 -- 172.21.15.12:0/570675203 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1671675737 0 0) 0x55865544fbd0 con 0x558655502620 2023-10-05T19:40:21.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.759+0000 7fd541b92700 1 -- 172.21.15.12:0/570675203 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd530001e10 con 0x558655502620 2023-10-05T19:40:21.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.759+0000 7fd542b94700 1 -- 172.21.15.12:0/570675203 >> v1:172.21.15.8:6789/0 conn(0x558655502620 legacy=0x5586555030c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.759+0000 7fd542b94700 1 -- 172.21.15.12:0/570675203 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586555037d0 con 0x558655501d10 2023-10-05T19:40:21.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.759+0000 7fd542b94700 1 --2- 172.21.15.12:0/570675203 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558655501d10 0x5586555020e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd53c00a5d0 tx=0x7fd53c005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e13106db2ecd35d server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.759+0000 7fd541b92700 1 -- 172.21.15.12:0/570675203 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd53c007610 con 0x558655501d10 2023-10-05T19:40:21.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.759+0000 7fd541b92700 1 -- 172.21.15.12:0/570675203 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd53c007770 con 0x558655501d10 2023-10-05T19:40:21.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.759+0000 7fd541b92700 1 -- 172.21.15.12:0/570675203 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd53c007a40 con 0x558655501d10 2023-10-05T19:40:21.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.759+0000 7fd54505fe40 1 -- 172.21.15.12:0/570675203 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558655501d10 msgr2=0x5586555020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.759+0000 7fd54505fe40 1 --2- 172.21.15.12:0/570675203 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558655501d10 0x5586555020e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd53c00a5d0 tx=0x7fd53c005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:21.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.759+0000 7fd54505fe40 1 -- 172.21.15.12:0/570675203 shutdown_connections 2023-10-05T19:40:21.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.759+0000 7fd54505fe40 1 --2- 172.21.15.12:0/570675203 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558655501d10 0x5586555020e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.759+0000 7fd54505fe40 1 -- 172.21.15.12:0/570675203 >> 172.21.15.12:0/570675203 conn(0x5586554fd730 msgr2=0x5586554ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:21.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.759+0000 7fd54505fe40 1 -- 172.21.15.12:0/570675203 shutdown_connections 2023-10-05T19:40:21.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.759+0000 7fd54505fe40 1 -- 172.21.15.12:0/570675203 wait complete. 2023-10-05T19:40:21.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.763+0000 7fd54505fe40 1 Processor -- start 2023-10-05T19:40:21.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.763+0000 7fd54505fe40 1 -- start start 2023-10-05T19:40:21.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.763+0000 7fd54505fe40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558655501d10 0x5586555268e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.763+0000 7fd54505fe40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586553d0e30 con 0x558655501d10 2023-10-05T19:40:21.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.763+0000 7fd542b94700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558655501d10 0x5586555268e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.763+0000 7fd542b94700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558655501d10 0x5586555268e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37372/0 (socket says 172.21.15.12:37372) 2023-10-05T19:40:21.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.763+0000 7fd542b94700 1 -- 172.21.15.12:0/735070217 learned_addr learned my addr 172.21.15.12:0/735070217 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:21.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.763+0000 7fd542b94700 1 -- 172.21.15.12:0/735070217 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd53c005cf0 con 0x558655501d10 2023-10-05T19:40:21.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.763+0000 7fd542b94700 1 --2- 172.21.15.12:0/735070217 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558655501d10 0x5586555268e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd53c000c00 tx=0x7fd53c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c72290d8c02004d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.763+0000 7fd540b90700 1 -- 172.21.15.12:0/735070217 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd53c007a00 con 0x558655501d10 2023-10-05T19:40:21.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.763+0000 7fd540b90700 1 -- 172.21.15.12:0/735070217 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd53c007b60 con 0x558655501d10 2023-10-05T19:40:21.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.763+0000 7fd540b90700 1 -- 172.21.15.12:0/735070217 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd53c007d10 con 0x558655501d10 2023-10-05T19:40:21.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.763+0000 7fd54505fe40 1 -- 172.21.15.12:0/735070217 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558655501d10 msgr2=0x5586555268e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.763+0000 7fd54505fe40 1 --2- 172.21.15.12:0/735070217 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558655501d10 0x5586555268e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd53c000c00 tx=0x7fd53c014000 comp rx=0 tx=0).stop 2023-10-05T19:40:21.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.763+0000 7fd54505fe40 1 -- 172.21.15.12:0/735070217 shutdown_connections 2023-10-05T19:40:21.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.763+0000 7fd54505fe40 1 --2- 172.21.15.12:0/735070217 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558655501d10 0x5586555268e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.763+0000 7fd54505fe40 1 -- 172.21.15.12:0/735070217 >> 172.21.15.12:0/735070217 conn(0x5586554fd730 msgr2=0x558655476e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:21.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.763+0000 7fd54505fe40 1 -- 172.21.15.12:0/735070217 shutdown_connections 2023-10-05T19:40:21.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.763+0000 7fd54505fe40 1 -- 172.21.15.12:0/735070217 wait complete. 2023-10-05T19:40:21.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.763+0000 7fd54505fe40 1 Processor -- start 2023-10-05T19:40:21.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.763+0000 7fd54505fe40 1 -- start start 2023-10-05T19:40:21.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.763+0000 7fd54505fe40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558655502620 0x55865552ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.763+0000 7fd54505fe40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586553a3ed0 con 0x558655502620 2023-10-05T19:40:21.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.767+0000 7fd542b94700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558655502620 0x55865552ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.767+0000 7fd542b94700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558655502620 0x55865552ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37388/0 (socket says 172.21.15.12:37388) 2023-10-05T19:40:21.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.767+0000 7fd542b94700 1 -- 172.21.15.12:0/1232190650 learned_addr learned my addr 172.21.15.12:0/1232190650 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:21.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.767+0000 7fd542b94700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586555037d0 con 0x558655502620 2023-10-05T19:40:21.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.767+0000 7fd542b94700 1 --2- 172.21.15.12:0/1232190650 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558655502620 0x55865552ad80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd53c00afd0 tx=0x7fd53c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.767+0000 7fd53affd700 1 -- 172.21.15.12:0/1232190650 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd53c007a00 con 0x558655502620 2023-10-05T19:40:21.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.767+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55865552b2c0 con 0x558655502620 2023-10-05T19:40:21.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.767+0000 7fd53affd700 1 -- 172.21.15.12:0/1232190650 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd53c029070 con 0x558655502620 2023-10-05T19:40:21.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.767+0000 7fd53affd700 1 -- 172.21.15.12:0/1232190650 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd53c007cd0 con 0x558655502620 2023-10-05T19:40:21.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.767+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55865552b6a0 con 0x558655502620 2023-10-05T19:40:21.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.767+0000 7fd53affd700 1 -- 172.21.15.12:0/1232190650 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fd53c012610 con 0x558655502620 2023-10-05T19:40:21.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.771+0000 7fd53affd700 1 --2- 172.21.15.12:0/1232190650 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd524030c00 0x7fd5240330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.771+0000 7fd53affd700 1 -- 172.21.15.12:0/1232190650 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7fd53c0466a0 con 0x558655502620 2023-10-05T19:40:21.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.771+0000 7fd54505fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.771+0000 7fd542393700 1 --2- 172.21.15.12:0/1232190650 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd524030c00 0x7fd5240330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.771+0000 7fd54505fe40 1 --2- 172.21.15.12:0/1232190650 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5586555347a0 0x558655536b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.771+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655537230 con 0x5586555347a0 2023-10-05T19:40:21.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.771+0000 7fd543395700 1 --2- 172.21.15.12:0/1232190650 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5586555347a0 0x558655536b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.771+0000 7fd542393700 1 --2- 172.21.15.12:0/1232190650 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd524030c00 0x7fd5240330b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd52c0044f0 tx=0x7fd52c006690 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.771+0000 7fd543395700 1 --2- 172.21.15.12:0/1232190650 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5586555347a0 0x558655536b80 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.771+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd53400a500 con 0x5586555347a0 2023-10-05T19:40:21.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.771+0000 7fd54505fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:21.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.771+0000 7fd54505fe40 20 realm 2023-10-05T19:40:21.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.771+0000 7fd54505fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.771+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655533e10 con 0x5586555347a0 2023-10-05T19:40:21.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.771+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd53400a500 con 0x5586555347a0 2023-10-05T19:40:21.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.771+0000 7fd54505fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:21.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.771+0000 7fd54505fe40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:21.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.771+0000 7fd54505fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.771+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655537d10 con 0x5586555347a0 2023-10-05T19:40:21.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.771+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd53400a500 con 0x5586555347a0 2023-10-05T19:40:21.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.771+0000 7fd54505fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:21.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.771+0000 7fd54505fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.771+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655537920 con 0x5586555347a0 2023-10-05T19:40:21.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.775+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fd53400a500 con 0x5586555347a0 2023-10-05T19:40:21.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.775+0000 7fd54505fe40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:21.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.775+0000 7fd54505fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.775+0000 7fd54505fe40 1 --2- 172.21.15.12:0/1232190650 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5586555390b0 0x55865553b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.775+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55865553b9d0 con 0x5586555390b0 2023-10-05T19:40:21.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.775+0000 7fd542b94700 1 --2- 172.21.15.12:0/1232190650 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5586555390b0 0x55865553b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.775+0000 7fd542b94700 1 --2- 172.21.15.12:0/1232190650 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5586555390b0 0x55865553b490 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.775+0000 7fd542b94700 1 -- 172.21.15.12:0/1232190650 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fd53c035070 con 0x5586555390b0 2023-10-05T19:40:21.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.775+0000 7fd54505fe40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:21.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.775+0000 7fd54505fe40 20 searching for the correct realm 2023-10-05T19:40:21.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.775+0000 7fd54505fe40 1 --2- 172.21.15.12:0/1232190650 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55865553d850 0x55865553fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.775+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540310 con 0x55865553d850 2023-10-05T19:40:21.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.775+0000 7fd542393700 1 --2- 172.21.15.12:0/1232190650 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55865553d850 0x55865553fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.775+0000 7fd542393700 1 --2- 172.21.15.12:0/1232190650 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55865553d850 0x55865553fc60 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.779+0000 7fd542393700 1 -- 172.21.15.12:0/1232190650 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fd52c004520 con 0x55865553d850 2023-10-05T19:40:21.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.779+0000 7fd542393700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540310 con 0x5586555390b0 2023-10-05T19:40:21.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.779+0000 7fd542b94700 1 -- 172.21.15.12:0/1232190650 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd53c035070 con 0x5586555390b0 2023-10-05T19:40:21.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.779+0000 7fd542b94700 1 --2- 172.21.15.12:0/1232190650 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x558655501d10 0x7fd53c0460b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.779+0000 7fd542b94700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540310 con 0x558655501d10 2023-10-05T19:40:21.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.779+0000 7fd543395700 1 --2- 172.21.15.12:0/1232190650 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x558655501d10 0x7fd53c0460b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.779+0000 7fd543395700 1 --2- 172.21.15.12:0/1232190650 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x558655501d10 0x7fd53c0460b0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.779+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd534014040 con 0x558655501d10 2023-10-05T19:40:21.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.779+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540310 con 0x5586555347a0 2023-10-05T19:40:21.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.783+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd534014040 con 0x5586555347a0 2023-10-05T19:40:21.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.783+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540310 con 0x5586555347a0 2023-10-05T19:40:21.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.783+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd534014040 con 0x5586555347a0 2023-10-05T19:40:21.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.783+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540310 con 0x558655501d10 2023-10-05T19:40:21.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.783+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd534014040 con 0x558655501d10 2023-10-05T19:40:21.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.783+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540310 con 0x558655501d10 2023-10-05T19:40:21.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.783+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd534014040 con 0x558655501d10 2023-10-05T19:40:21.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.783+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540310 con 0x5586555390b0 2023-10-05T19:40:21.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.783+0000 7fd542b94700 1 -- 172.21.15.12:0/1232190650 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd53c035070 con 0x5586555390b0 2023-10-05T19:40:21.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.783+0000 7fd542b94700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540310 con 0x55865553d850 2023-10-05T19:40:21.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.787+0000 7fd542393700 1 -- 172.21.15.12:0/1232190650 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd52c004520 con 0x55865553d850 2023-10-05T19:40:21.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.787+0000 7fd542393700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540310 con 0x5586555347a0 2023-10-05T19:40:21.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.787+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd534014040 con 0x5586555347a0 2023-10-05T19:40:21.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.787+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540310 con 0x558655501d10 2023-10-05T19:40:21.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.787+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd534014040 con 0x558655501d10 2023-10-05T19:40:21.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.787+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540310 con 0x5586555347a0 2023-10-05T19:40:21.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.787+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd534014040 con 0x5586555347a0 2023-10-05T19:40:21.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.787+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540310 con 0x5586555390b0 2023-10-05T19:40:21.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.787+0000 7fd542b94700 1 -- 172.21.15.12:0/1232190650 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd53c035070 con 0x5586555390b0 2023-10-05T19:40:21.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.787+0000 7fd542b94700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540310 con 0x55865553d850 2023-10-05T19:40:21.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.787+0000 7fd542393700 1 -- 172.21.15.12:0/1232190650 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd52c004520 con 0x55865553d850 2023-10-05T19:40:21.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.787+0000 7fd542393700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540310 con 0x55865553d850 2023-10-05T19:40:21.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.791+0000 7fd542393700 1 -- 172.21.15.12:0/1232190650 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd52c004520 con 0x55865553d850 2023-10-05T19:40:21.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.791+0000 7fd542393700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540310 con 0x558655501d10 2023-10-05T19:40:21.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.791+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd534014040 con 0x558655501d10 2023-10-05T19:40:21.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.791+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540310 con 0x55865553d850 2023-10-05T19:40:21.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.791+0000 7fd542393700 1 -- 172.21.15.12:0/1232190650 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd52c004520 con 0x55865553d850 2023-10-05T19:40:21.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.791+0000 7fd542393700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540310 con 0x55865553d850 2023-10-05T19:40:21.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.791+0000 7fd542393700 1 -- 172.21.15.12:0/1232190650 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd52c004520 con 0x55865553d850 2023-10-05T19:40:21.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.791+0000 7fd542393700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540310 con 0x5586555347a0 2023-10-05T19:40:21.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.791+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd534014040 con 0x5586555347a0 2023-10-05T19:40:21.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.791+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540310 con 0x5586555347a0 2023-10-05T19:40:21.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.795+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd534014040 con 0x5586555347a0 2023-10-05T19:40:21.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.795+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540310 con 0x558655501d10 2023-10-05T19:40:21.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.795+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd534014040 con 0x558655501d10 2023-10-05T19:40:21.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.795+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540310 con 0x55865553d850 2023-10-05T19:40:21.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.795+0000 7fd542393700 1 -- 172.21.15.12:0/1232190650 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd52c004520 con 0x55865553d850 2023-10-05T19:40:21.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.795+0000 7fd542393700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540310 con 0x5586555390b0 2023-10-05T19:40:21.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.795+0000 7fd542b94700 1 -- 172.21.15.12:0/1232190650 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd53c035070 con 0x5586555390b0 2023-10-05T19:40:21.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.795+0000 7fd542b94700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540310 con 0x5586555347a0 2023-10-05T19:40:21.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.799+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd534014040 con 0x5586555347a0 2023-10-05T19:40:21.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.799+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540310 con 0x558655501d10 2023-10-05T19:40:21.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.799+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd534014040 con 0x558655501d10 2023-10-05T19:40:21.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.799+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540310 con 0x55865553d850 2023-10-05T19:40:21.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.799+0000 7fd542393700 1 -- 172.21.15.12:0/1232190650 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd52c004520 con 0x55865553d850 2023-10-05T19:40:21.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.799+0000 7fd542393700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540310 con 0x558655501d10 2023-10-05T19:40:21.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.799+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd534014040 con 0x558655501d10 2023-10-05T19:40:21.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.799+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540310 con 0x5586555390b0 2023-10-05T19:40:21.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.799+0000 7fd542b94700 1 -- 172.21.15.12:0/1232190650 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd53c035070 con 0x5586555390b0 2023-10-05T19:40:21.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.799+0000 7fd542b94700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540310 con 0x55865553d850 2023-10-05T19:40:21.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.803+0000 7fd542393700 1 -- 172.21.15.12:0/1232190650 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd52c004520 con 0x55865553d850 2023-10-05T19:40:21.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.803+0000 7fd542393700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540310 con 0x558655501d10 2023-10-05T19:40:21.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.803+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd534014040 con 0x558655501d10 2023-10-05T19:40:21.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.803+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540310 con 0x5586555347a0 2023-10-05T19:40:21.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.803+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fd534014040 con 0x5586555347a0 2023-10-05T19:40:21.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.803+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540310 con 0x5586555390b0 2023-10-05T19:40:21.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.803+0000 7fd542b94700 1 -- 172.21.15.12:0/1232190650 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fd53c035070 con 0x5586555390b0 2023-10-05T19:40:21.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.803+0000 7fd54505fe40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:21.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.803+0000 7fd54505fe40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:21.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.803+0000 7fd54505fe40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:21.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.803+0000 7fd54505fe40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:21.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.803+0000 7fd54505fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.803+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55865553c5a0 con 0x5586555347a0 2023-10-05T19:40:21.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.807+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd534014040 con 0x5586555347a0 2023-10-05T19:40:21.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.807+0000 7fd54505fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:21.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.807+0000 7fd54505fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.807+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55865553c1d0 con 0x5586555347a0 2023-10-05T19:40:21.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.807+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fd534014040 con 0x5586555347a0 2023-10-05T19:40:21.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.807+0000 7fd54505fe40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:21.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.807+0000 7fd54505fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.807+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540780 con 0x55865553d850 2023-10-05T19:40:21.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.807+0000 7fd542393700 1 -- 172.21.15.12:0/1232190650 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fd52c004520 con 0x55865553d850 2023-10-05T19:40:21.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.807+0000 7fd54505fe40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:21.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.807+0000 7fd54505fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.807+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540b30 con 0x55865553d850 2023-10-05T19:40:21.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.807+0000 7fd542393700 1 -- 172.21.15.12:0/1232190650 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7fd52c004520 con 0x55865553d850 2023-10-05T19:40:21.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.807+0000 7fd54505fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:21.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.807+0000 7fd54505fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.807+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540ee0 con 0x5586555347a0 2023-10-05T19:40:21.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.807+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd534014040 con 0x5586555347a0 2023-10-05T19:40:21.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.807+0000 7fd54505fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:21.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.811+0000 7fd54505fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.811+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55865553cac0 con 0x5586555347a0 2023-10-05T19:40:21.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.811+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fd534014040 con 0x5586555347a0 2023-10-05T19:40:21.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.811+0000 7fd54505fe40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:21.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.811+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55865553cac0 con 0x55865553d850 2023-10-05T19:40:21.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.811+0000 7fd542393700 1 -- 172.21.15.12:0/1232190650 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fd52c004520 con 0x55865553d850 2023-10-05T19:40:21.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.811+0000 7fd542393700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55865553cac0 con 0x5586555390b0 2023-10-05T19:40:21.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.815+0000 7fd542b94700 1 -- 172.21.15.12:0/1232190650 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd53c035070 con 0x5586555390b0 2023-10-05T19:40:21.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.815+0000 7fd542b94700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55865553cac0 con 0x558655501d10 2023-10-05T19:40:21.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.815+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd534014040 con 0x558655501d10 2023-10-05T19:40:21.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.815+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540ee0 con 0x5586555347a0 2023-10-05T19:40:21.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.815+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd534014040 con 0x5586555347a0 2023-10-05T19:40:21.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.815+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540ee0 con 0x5586555347a0 2023-10-05T19:40:21.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.815+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd534014040 con 0x5586555347a0 2023-10-05T19:40:21.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.815+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540ee0 con 0x558655501d10 2023-10-05T19:40:21.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.815+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd534014040 con 0x558655501d10 2023-10-05T19:40:21.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.815+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540ee0 con 0x558655501d10 2023-10-05T19:40:21.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.819+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd534014040 con 0x558655501d10 2023-10-05T19:40:21.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.819+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540ee0 con 0x5586555390b0 2023-10-05T19:40:21.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.819+0000 7fd542b94700 1 -- 172.21.15.12:0/1232190650 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd53c035070 con 0x5586555390b0 2023-10-05T19:40:21.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.819+0000 7fd542b94700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540ee0 con 0x55865553d850 2023-10-05T19:40:21.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.819+0000 7fd542393700 1 -- 172.21.15.12:0/1232190650 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd52c02d040 con 0x55865553d850 2023-10-05T19:40:21.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.819+0000 7fd542393700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540ee0 con 0x5586555347a0 2023-10-05T19:40:21.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.819+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd534014040 con 0x5586555347a0 2023-10-05T19:40:21.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.819+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540ee0 con 0x558655501d10 2023-10-05T19:40:21.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.819+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd534014040 con 0x558655501d10 2023-10-05T19:40:21.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.819+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540ee0 con 0x5586555347a0 2023-10-05T19:40:21.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.823+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd534014040 con 0x5586555347a0 2023-10-05T19:40:21.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.823+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540ee0 con 0x5586555390b0 2023-10-05T19:40:21.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.823+0000 7fd542b94700 1 -- 172.21.15.12:0/1232190650 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd53c035070 con 0x5586555390b0 2023-10-05T19:40:21.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.823+0000 7fd542b94700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540ee0 con 0x55865553d850 2023-10-05T19:40:21.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.823+0000 7fd542393700 1 -- 172.21.15.12:0/1232190650 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd52c02d040 con 0x55865553d850 2023-10-05T19:40:21.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.823+0000 7fd542393700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540ee0 con 0x55865553d850 2023-10-05T19:40:21.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.823+0000 7fd542393700 1 -- 172.21.15.12:0/1232190650 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd52c02d040 con 0x55865553d850 2023-10-05T19:40:21.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.823+0000 7fd542393700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540ee0 con 0x558655501d10 2023-10-05T19:40:21.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.823+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd534014040 con 0x558655501d10 2023-10-05T19:40:21.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.823+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540ee0 con 0x55865553d850 2023-10-05T19:40:21.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.827+0000 7fd542393700 1 -- 172.21.15.12:0/1232190650 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd52c02d040 con 0x55865553d850 2023-10-05T19:40:21.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.827+0000 7fd542393700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540ee0 con 0x55865553d850 2023-10-05T19:40:21.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.827+0000 7fd542393700 1 -- 172.21.15.12:0/1232190650 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd52c02d040 con 0x55865553d850 2023-10-05T19:40:21.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.827+0000 7fd542393700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540ee0 con 0x5586555347a0 2023-10-05T19:40:21.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.827+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd534014040 con 0x5586555347a0 2023-10-05T19:40:21.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.827+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540ee0 con 0x5586555347a0 2023-10-05T19:40:21.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.827+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd534014040 con 0x5586555347a0 2023-10-05T19:40:21.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.827+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540ee0 con 0x558655501d10 2023-10-05T19:40:21.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.831+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd534014040 con 0x558655501d10 2023-10-05T19:40:21.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.831+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540ee0 con 0x55865553d850 2023-10-05T19:40:21.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.831+0000 7fd542393700 1 -- 172.21.15.12:0/1232190650 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd52c02d040 con 0x55865553d850 2023-10-05T19:40:21.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.831+0000 7fd542393700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540ee0 con 0x5586555390b0 2023-10-05T19:40:21.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.831+0000 7fd542b94700 1 -- 172.21.15.12:0/1232190650 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd53c035070 con 0x5586555390b0 2023-10-05T19:40:21.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.831+0000 7fd542b94700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540ee0 con 0x5586555347a0 2023-10-05T19:40:21.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.831+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd534014040 con 0x5586555347a0 2023-10-05T19:40:21.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.831+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540ee0 con 0x558655501d10 2023-10-05T19:40:21.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.831+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd534014040 con 0x558655501d10 2023-10-05T19:40:21.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.831+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540ee0 con 0x55865553d850 2023-10-05T19:40:21.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.835+0000 7fd542393700 1 -- 172.21.15.12:0/1232190650 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd52c02d040 con 0x55865553d850 2023-10-05T19:40:21.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.835+0000 7fd542393700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540ee0 con 0x558655501d10 2023-10-05T19:40:21.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.835+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd534014040 con 0x558655501d10 2023-10-05T19:40:21.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.835+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540ee0 con 0x5586555390b0 2023-10-05T19:40:21.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.835+0000 7fd542b94700 1 -- 172.21.15.12:0/1232190650 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd53c035070 con 0x5586555390b0 2023-10-05T19:40:21.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.835+0000 7fd542b94700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540ee0 con 0x55865553d850 2023-10-05T19:40:21.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.835+0000 7fd542393700 1 -- 172.21.15.12:0/1232190650 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd52c02d040 con 0x55865553d850 2023-10-05T19:40:21.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.835+0000 7fd542393700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540ee0 con 0x558655501d10 2023-10-05T19:40:21.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.839+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd534014040 con 0x558655501d10 2023-10-05T19:40:21.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.839+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540ee0 con 0x5586555347a0 2023-10-05T19:40:21.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.839+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fd534014040 con 0x5586555347a0 2023-10-05T19:40:21.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.839+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655540ee0 con 0x5586555390b0 2023-10-05T19:40:21.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.839+0000 7fd542b94700 1 -- 172.21.15.12:0/1232190650 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fd53c035070 con 0x5586555390b0 2023-10-05T19:40:21.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.839+0000 7fd54505fe40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:21.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.839+0000 7fd54505fe40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:21.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.839+0000 7fd54505fe40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:21.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.839+0000 7fd54505fe40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:21.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.839+0000 7fd54505fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.839+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55865553cac0 con 0x5586555347a0 2023-10-05T19:40:21.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.839+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fd534014040 con 0x5586555347a0 2023-10-05T19:40:21.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.839+0000 7fd54505fe40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:21.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.839+0000 7fd54505fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.839+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655541830 con 0x5586555390b0 2023-10-05T19:40:21.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.843+0000 7fd542b94700 1 -- 172.21.15.12:0/1232190650 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fd53c035070 con 0x5586555390b0 2023-10-05T19:40:21.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.843+0000 7fd54505fe40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:21.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.843+0000 7fd54505fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.843+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655542120 con 0x5586555347a0 2023-10-05T19:40:21.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.843+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fd534014040 con 0x5586555347a0 2023-10-05T19:40:21.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.843+0000 7fd54505fe40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:21.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.843+0000 7fd54505fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.843+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655542120 con 0x55865553d850 2023-10-05T19:40:21.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.843+0000 7fd542393700 1 -- 172.21.15.12:0/1232190650 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fd52c02d040 con 0x55865553d850 2023-10-05T19:40:21.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.843+0000 7fd54505fe40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:21.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.843+0000 7fd54505fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.843+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655542890 con 0x558655501d10 2023-10-05T19:40:21.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.843+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7fd534014040 con 0x558655501d10 2023-10-05T19:40:21.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.843+0000 7fd54505fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:21.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.843+0000 7fd54505fe40 20 zone default found 2023-10-05T19:40:21.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.843+0000 7fd54505fe40 4 Realm: () 2023-10-05T19:40:21.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.843+0000 7fd54505fe40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:21.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.843+0000 7fd54505fe40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:21.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.843+0000 7fd54505fe40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:21.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.843+0000 7fd54505fe40 20 zonegroup default 2023-10-05T19:40:21.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.843+0000 7fd54505fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.843+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655542560 con 0x558655501d10 2023-10-05T19:40:21.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.847+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fd534014040 con 0x558655501d10 2023-10-05T19:40:21.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.847+0000 7fd54505fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:21.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.847+0000 7fd54505fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.847+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655543ee0 con 0x55865553d850 2023-10-05T19:40:21.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.847+0000 7fd542393700 1 -- 172.21.15.12:0/1232190650 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fd52c02d040 con 0x55865553d850 2023-10-05T19:40:21.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.847+0000 7fd54505fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:21.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.847+0000 7fd54505fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.847+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655544290 con 0x55865553d850 2023-10-05T19:40:21.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.847+0000 7fd542393700 1 -- 172.21.15.12:0/1232190650 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fd52c03c040 con 0x55865553d850 2023-10-05T19:40:21.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.847+0000 7fd54505fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:21.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.847+0000 7fd54505fe40 20 started sync module instance, tier type = 2023-10-05T19:40:21.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.847+0000 7fd54505fe40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:21.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.847+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655543a90 con 0x5586555390b0 2023-10-05T19:40:21.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.851+0000 7fd542b94700 1 -- 172.21.15.12:0/1232190650 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'72 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd53c035070 con 0x5586555390b0 2023-10-05T19:40:21.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.851+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94035445560736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655544b30 con 0x5586555390b0 2023-10-05T19:40:21.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.851+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5586555453d0 con 0x5586555347a0 2023-10-05T19:40:21.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.851+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'72 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd534014040 con 0x5586555347a0 2023-10-05T19:40:21.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.851+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94035445567200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655545820 con 0x5586555347a0 2023-10-05T19:40:21.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.851+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655546270 con 0x55865553d850 2023-10-05T19:40:21.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.855+0000 7fd542b94700 1 -- 172.21.15.12:0/1232190650 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94035445560736] v28'73 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd53c035070 con 0x5586555390b0 2023-10-05T19:40:21.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.855+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94035445567200] v28'73 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd534014040 con 0x5586555347a0 2023-10-05T19:40:21.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.855+0000 7fd542393700 1 -- 172.21.15.12:0/1232190650 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'72 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd52c03c040 con 0x55865553d850 2023-10-05T19:40:21.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.855+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94035445564992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655545820 con 0x55865553d850 2023-10-05T19:40:21.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.855+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655546b10 con 0x5586555390b0 2023-10-05T19:40:21.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.859+0000 7fd542b94700 1 -- 172.21.15.12:0/1232190650 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'72 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd53c035070 con 0x5586555390b0 2023-10-05T19:40:21.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.859+0000 7fd542393700 1 -- 172.21.15.12:0/1232190650 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94035445564992] v28'73 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd52c03c040 con 0x55865553d850 2023-10-05T19:40:21.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.859+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94035445573152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5586555473f0 con 0x5586555390b0 2023-10-05T19:40:21.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.859+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655547a80 con 0x5586555347a0 2023-10-05T19:40:21.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.859+0000 7fd542b94700 1 -- 172.21.15.12:0/1232190650 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94035445573152] v28'73 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd53c035070 con 0x5586555390b0 2023-10-05T19:40:21.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.863+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'72 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd534014040 con 0x5586555347a0 2023-10-05T19:40:21.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.863+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94035445568304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655547a80 con 0x5586555347a0 2023-10-05T19:40:21.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.863+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655548490 con 0x558655501d10 2023-10-05T19:40:21.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.863+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94035445568304] v28'73 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd534014040 con 0x5586555347a0 2023-10-05T19:40:21.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.863+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'72 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd534014040 con 0x558655501d10 2023-10-05T19:40:21.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.863+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94035445575360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655548490 con 0x558655501d10 2023-10-05T19:40:21.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.863+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655548ea0 con 0x558655501d10 2023-10-05T19:40:21.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.867+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(95 notify.5 [watch watch cookie 94035445575360] v28'73 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd534014040 con 0x558655501d10 2023-10-05T19:40:21.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.867+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(96 notify.6 [create] v28'72 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd534014040 con 0x558655501d10 2023-10-05T19:40:21.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.867+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94035445580240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655548ea0 con 0x558655501d10 2023-10-05T19:40:21.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.867+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5586555498b0 con 0x558655501d10 2023-10-05T19:40:21.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.871+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(98 notify.7 [create] v28'72 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd534014040 con 0x558655501d10 2023-10-05T19:40:21.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.871+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94035445584832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558655549d00 con 0x558655501d10 2023-10-05T19:40:21.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.871+0000 7fd54505fe40 20 add_watcher() i=0 2023-10-05T19:40:21.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.871+0000 7fd54505fe40 20 add_watcher() i=1 2023-10-05T19:40:21.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.871+0000 7fd54505fe40 20 add_watcher() i=2 2023-10-05T19:40:21.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.871+0000 7fd54505fe40 20 add_watcher() i=3 2023-10-05T19:40:21.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.871+0000 7fd54505fe40 20 add_watcher() i=4 2023-10-05T19:40:21.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.871+0000 7fd54505fe40 20 add_watcher() i=5 2023-10-05T19:40:21.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.871+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(97 notify.6 [watch watch cookie 94035445580240] v28'73 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd534014040 con 0x558655501d10 2023-10-05T19:40:21.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.871+0000 7fd54505fe40 20 add_watcher() i=6 2023-10-05T19:40:21.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.871+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94035445584832] v28'73 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd534014040 con 0x558655501d10 2023-10-05T19:40:21.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.875+0000 7fd54505fe40 20 add_watcher() i=7 2023-10-05T19:40:21.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.875+0000 7fd54505fe40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:21.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.875+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55865554cd30 con 0x5586555347a0 2023-10-05T19:40:21.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.875+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fd534014040 con 0x5586555347a0 2023-10-05T19:40:21.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.875+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94035445585936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55865554a150 con 0x5586555347a0 2023-10-05T19:40:21.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.875+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94035445585936] v28'82 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fd534014040 con 0x5586555347a0 2023-10-05T19:40:21.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.875+0000 7fd48ffff700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:21.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.879+0000 7fd54505fe40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:21.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.879+0000 7fd54505fe40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:21.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.879+0000 7fd54505fe40 1 Processor -- start 2023-10-05T19:40:21.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.879+0000 7fd54505fe40 1 -- start start 2023-10-05T19:40:21.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.879+0000 7fd54505fe40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558655557190 0x558655557560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.879+0000 7fd54505fe40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x558655557aa0 con 0x558655557190 2023-10-05T19:40:21.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.879+0000 7fd542393700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558655557190 0x558655557560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.879+0000 7fd542393700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558655557190 0x558655557560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37392/0 (socket says 172.21.15.12:37392) 2023-10-05T19:40:21.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.879+0000 7fd542393700 1 -- 172.21.15.12:0/1165877895 learned_addr learned my addr 172.21.15.12:0/1165877895 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:21.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.879+0000 7fd542393700 1 -- 172.21.15.12:0/1165877895 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558655557be0 con 0x558655557190 2023-10-05T19:40:21.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.879+0000 7fd542393700 1 --2- 172.21.15.12:0/1165877895 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558655557190 0x558655557560 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd53c011b60 tx=0x7fd52c027040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edb3e70cc53ff512 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.879+0000 7fd48f7fe700 1 -- 172.21.15.12:0/1165877895 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd52c039040 con 0x558655557190 2023-10-05T19:40:21.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.879+0000 7fd48f7fe700 1 -- 172.21.15.12:0/1165877895 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd52c03a040 con 0x558655557190 2023-10-05T19:40:21.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.879+0000 7fd48f7fe700 1 -- 172.21.15.12:0/1165877895 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd52c02a080 con 0x558655557190 2023-10-05T19:40:21.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.879+0000 7fd54505fe40 1 -- 172.21.15.12:0/1165877895 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558655557190 msgr2=0x558655557560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.879+0000 7fd54505fe40 1 --2- 172.21.15.12:0/1165877895 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558655557190 0x558655557560 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd53c011b60 tx=0x7fd52c027040 comp rx=0 tx=0).stop 2023-10-05T19:40:21.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.879+0000 7fd54505fe40 1 -- 172.21.15.12:0/1165877895 shutdown_connections 2023-10-05T19:40:21.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.879+0000 7fd54505fe40 1 --2- 172.21.15.12:0/1165877895 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558655557190 0x558655557560 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.879+0000 7fd54505fe40 1 -- 172.21.15.12:0/1165877895 >> 172.21.15.12:0/1165877895 conn(0x558655552550 msgr2=0x5586555506a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:21.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.883+0000 7fd54505fe40 1 -- 172.21.15.12:0/1165877895 shutdown_connections 2023-10-05T19:40:21.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.883+0000 7fd54505fe40 1 -- 172.21.15.12:0/1165877895 wait complete. 2023-10-05T19:40:21.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.883+0000 7fd54505fe40 1 Processor -- start 2023-10-05T19:40:21.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.883+0000 7fd54505fe40 1 -- start start 2023-10-05T19:40:21.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.883+0000 7fd54505fe40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558655557190 0x55865555cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.883+0000 7fd54505fe40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55865555dd70 con 0x558655557190 2023-10-05T19:40:21.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.883+0000 7fd542393700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558655557190 0x55865555cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.883+0000 7fd542393700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558655557190 0x55865555cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37406/0 (socket says 172.21.15.12:37406) 2023-10-05T19:40:21.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.883+0000 7fd542393700 1 -- 172.21.15.12:0/4156430770 learned_addr learned my addr 172.21.15.12:0/4156430770 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:21.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.883+0000 7fd542393700 1 -- 172.21.15.12:0/4156430770 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd530001e10 con 0x558655557190 2023-10-05T19:40:21.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.883+0000 7fd542393700 1 --2- 172.21.15.12:0/4156430770 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558655557190 0x55865555cd70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd52c042040 tx=0x7fd52c03f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.883+0000 7fd48dffb700 1 -- 172.21.15.12:0/4156430770 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd52c039040 con 0x558655557190 2023-10-05T19:40:21.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.883+0000 7fd48dffb700 1 -- 172.21.15.12:0/4156430770 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd52c02b040 con 0x558655557190 2023-10-05T19:40:21.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.883+0000 7fd48dffb700 1 -- 172.21.15.12:0/4156430770 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd52c01a040 con 0x558655557190 2023-10-05T19:40:21.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.883+0000 7fd54505fe40 1 -- 172.21.15.12:0/4156430770 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55865555d370 con 0x558655557190 2023-10-05T19:40:21.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.883+0000 7fd54505fe40 1 -- 172.21.15.12:0/4156430770 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55865555d750 con 0x558655557190 2023-10-05T19:40:21.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.887+0000 7fd48dffb700 1 -- 172.21.15.12:0/4156430770 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fd52c04b730 con 0x558655557190 2023-10-05T19:40:21.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.887+0000 7fd48dffb700 1 --2- 172.21.15.12:0/4156430770 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd480030c50 0x7fd480033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.887+0000 7fd48dffb700 1 -- 172.21.15.12:0/4156430770 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7fd52c028040 con 0x558655557190 2023-10-05T19:40:21.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.887+0000 7fd543395700 1 --2- 172.21.15.12:0/4156430770 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd480030c50 0x7fd480033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.887+0000 7fd54505fe40 5 note: GC not initialized 2023-10-05T19:40:21.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.887+0000 7fd543395700 1 --2- 172.21.15.12:0/4156430770 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd480030c50 0x7fd480033100 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x5586555473c0 tx=0x7fd53400a9b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.887+0000 7fd47effd700 20 reqs_thread_entry: start 2023-10-05T19:40:21.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.891+0000 7fd47cff9700 20 reqs_thread_entry: start 2023-10-05T19:40:21.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.891+0000 7fd54505fe40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:21.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.891+0000 7fd54505fe40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:21.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.891+0000 7fd476ffd700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:52 2023 2023-10-05T19:40:21.899 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:21.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.891+0000 7fd4767fc700 20 reqs_thread_entry: start 2023-10-05T19:40:21.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.891+0000 7fd54505fe40 20 get_system_obj_state: rctx=0x7ffc9fb1ddb8 obj=default.rgw.meta:users.uid:foo state=0x5586553dc0b0 s->prefetch_data=0 2023-10-05T19:40:21.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.891+0000 7fd54505fe40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2023-10-05T19:40:21.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.891+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5586555c5580 con 0x5586555347a0 2023-10-05T19:40:21.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.891+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(102 foo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7fd534014040 con 0x5586555347a0 2023-10-05T19:40:21.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.891+0000 7fd54505fe40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x16 2023-10-05T19:40:21.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.891+0000 7fd54505fe40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:21.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.891+0000 7fd54505fe40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:21.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.891+0000 7fd54505fe40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:21.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.891+0000 7fd54505fe40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:21.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.891+0000 7fd54505fe40 10 cache get: name=default.rgw.meta+users.uid+foo : type miss (requested=0x13, cached=0x16) 2023-10-05T19:40:21.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.891+0000 7fd54505fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.891+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5586555c8010 con 0x5586555347a0 2023-10-05T19:40:21.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.891+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(103 foo [call,call out=48b,read 0~621 out=621b,getxattrs out=75b] v0'0 uv8 ondisk = 0) v8 ==== 273+0+744 (crc 0 0 0) 0x7fd534014040 con 0x5586555347a0 2023-10-05T19:40:21.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.891+0000 7fd54505fe40 20 rados_obj.operate() r=0 bl.length=621 2023-10-05T19:40:21.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.891+0000 7fd54505fe40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2023-10-05T19:40:21.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.891+0000 7fd54505fe40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:21.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.891+0000 7fd54505fe40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:21.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.891+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:104 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.set in=58b,writefull 0~539 in=539b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5586555c4aa0 con 0x5586555347a0 2023-10-05T19:40:21.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.895+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(104 foo [delete,create,call,writefull 0~539] v28'9 uv9 ondisk = 0) v8 ==== 273+0+0 (crc 0 0 0) 0x7fd534014040 con 0x5586555347a0 2023-10-05T19:40:21.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.895+0000 7fd54505fe40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2023-10-05T19:40:21.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.895+0000 7fd54505fe40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:21.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.895+0000 7fd54505fe40 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2023-10-05T19:40:21.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.895+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:105 7.d 7:b4812045:::notify.4:head [notify cookie 94035446089040 in=727b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5586555c9020 con 0x5586555347a0 2023-10-05T19:40:21.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.895+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== watch-notify(notify (1) cookie 94035445568304 notify 120259084296 ret 0) v3 ==== 757+0+0 (crc 0 0 0) 0x7fd534033430 con 0x5586555347a0 2023-10-05T19:40:21.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.895+0000 7fd540b90700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084296 cookie 94035445568304 notifier 4369 bl.length()=715 2023-10-05T19:40:21.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.895+0000 7fd540b90700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2023-10-05T19:40:21.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.895+0000 7fd540b90700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:21.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.895+0000 7fd540b90700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:106 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fd528003d90 con 0x5586555347a0 2023-10-05T19:40:21.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.895+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(105 notify.4 [notify cookie 94035446089040 out=8b] v0'0 uv72 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fd534014040 con 0x5586555347a0 2023-10-05T19:40:21.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.895+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== watch-notify(notify_complete (2) cookie 94035446089040 notify 120259084296 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7fd5340336b0 con 0x5586555347a0 2023-10-05T19:40:21.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.899+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.1 v2:172.21.15.12:6808/29933 39 ==== osd_op_reply(106 notify.4 [notify-ack] v0'0 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd534014040 con 0x5586555347a0 2023-10-05T19:40:21.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.899+0000 7fd54505fe40 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:users.swift:foo:foo1, ofs0, ns] 2023-10-05T19:40:21.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.899+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:107 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94035446089040 in=171b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5586555c8b30 con 0x5586555390b0 2023-10-05T19:40:21.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.899+0000 7fd542b94700 1 -- 172.21.15.12:0/1232190650 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== watch-notify(notify (1) cookie 94035445560736 notify 120259084292 ret 0) v3 ==== 201+0+0 (crc 0 0 0) 0x55865552b6a0 con 0x5586555390b0 2023-10-05T19:40:21.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.899+0000 7fd53bfff700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084292 cookie 94035445560736 notifier 4369 bl.length()=159 2023-10-05T19:40:21.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.899+0000 7fd53bfff700 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:108 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fd4940032d0 con 0x5586555390b0 2023-10-05T19:40:21.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.899+0000 7fd542b94700 1 -- 172.21.15.12:0/1232190650 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(107 notify.0 [notify cookie 94035446089040 out=8b] v0'0 uv72 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fd53c035070 con 0x5586555390b0 2023-10-05T19:40:21.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.899+0000 7fd542b94700 1 -- 172.21.15.12:0/1232190650 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(108 notify.0 [notify-ack] v0'0 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd53c035070 con 0x5586555390b0 2023-10-05T19:40:21.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.899+0000 7fd542b94700 1 -- 172.21.15.12:0/1232190650 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== watch-notify(notify_complete (2) cookie 94035446089040 notify 120259084292 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55865552b2c0 con 0x5586555390b0 2023-10-05T19:40:21.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.899+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:109 8.8 8:17d68d04:users.swift::foo%3afoo1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5586555c4550 con 0x5586555347a0 2023-10-05T19:40:21.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.903+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.1 v2:172.21.15.12:6808/29933 40 ==== osd_op_reply(109 foo:foo1 [delete] v28'2 uv2 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd534014040 con 0x5586555347a0 2023-10-05T19:40:21.910 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:40:21.910 INFO:teuthology.orchestra.run.smithi012.stdout: "user_id": "foo", 2023-10-05T19:40:21.910 INFO:teuthology.orchestra.run.smithi012.stdout: "display_name": "Foo", 2023-10-05T19:40:21.910 INFO:teuthology.orchestra.run.smithi012.stdout: "email": "foo@foo.com", 2023-10-05T19:40:21.910 INFO:teuthology.orchestra.run.smithi012.stdout: "suspended": 0, 2023-10-05T19:40:21.911 INFO:teuthology.orchestra.run.smithi012.stdout: "max_buckets": 4, 2023-10-05T19:40:21.911 INFO:teuthology.orchestra.run.smithi012.stdout: "subusers": [ 2023-10-05T19:40:21.911 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:21.911 INFO:teuthology.orchestra.run.smithi012.stdout: "id": "foo:foo1", 2023-10-05T19:40:21.911 INFO:teuthology.orchestra.run.smithi012.stdout: "permissions": "full-control" 2023-10-05T19:40:21.911 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:21.911 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:21.911 INFO:teuthology.orchestra.run.smithi012.stdout: "id": "foo:foo2", 2023-10-05T19:40:21.911 INFO:teuthology.orchestra.run.smithi012.stdout: "permissions": "" 2023-10-05T19:40:21.911 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:21.911 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:21.911 INFO:teuthology.orchestra.run.smithi012.stdout: "keys": [ 2023-10-05T19:40:21.911 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:21.912 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:40:21.912 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2023-10-05T19:40:21.912 INFO:teuthology.orchestra.run.smithi012.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2023-10-05T19:40:21.912 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:21.912 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:21.912 INFO:teuthology.orchestra.run.smithi012.stdout: "swift_keys": [ 2023-10-05T19:40:21.912 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:21.912 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo:foo2", 2023-10-05T19:40:21.912 INFO:teuthology.orchestra.run.smithi012.stdout: "secret_key": "ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy" 2023-10-05T19:40:21.912 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:21.912 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:21.912 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": [], 2023-10-05T19:40:21.913 INFO:teuthology.orchestra.run.smithi012.stdout: "op_mask": "read, write, delete", 2023-10-05T19:40:21.913 INFO:teuthology.orchestra.run.smithi012.stdout: "default_placement": "", 2023-10-05T19:40:21.913 INFO:teuthology.orchestra.run.smithi012.stdout: "default_storage_class": "", 2023-10-05T19:40:21.913 INFO:teuthology.orchestra.run.smithi012.stdout: "placement_tags": [], 2023-10-05T19:40:21.913 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket_quota": { 2023-10-05T19:40:21.913 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:21.913 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:21.913 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:21.913 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:21.913 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:21.913 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:21.913 INFO:teuthology.orchestra.run.smithi012.stdout: "user_quota": { 2023-10-05T19:40:21.914 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:21.914 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:21.914 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:21.914 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:21.914 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:21.914 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:21.914 INFO:teuthology.orchestra.run.smithi012.stdout: "temp_url_keys": [], 2023-10-05T19:40:21.914 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "rgw", 2023-10-05T19:40:21.914 INFO:teuthology.orchestra.run.smithi012.stdout: "mfa_ids": [] 2023-10-05T19:40:21.914 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:40:21.914 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:40:21.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.903+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:110 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94035445560736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5586555174e0 con 0x5586555390b0 2023-10-05T19:40:21.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.903+0000 7fd542b94700 1 -- 172.21.15.12:0/1232190650 <== osd.2 v2:172.21.15.12:6800/29934 23 ==== osd_op_reply(110 notify.0 [watch unwatch cookie 94035445560736] v28'74 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd53c035070 con 0x5586555390b0 2023-10-05T19:40:21.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.903+0000 7fd54505fe40 20 remove_watcher() i=0 2023-10-05T19:40:21.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.903+0000 7fd54505fe40 2 removed watcher, disabling cache 2023-10-05T19:40:21.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.903+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:111 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94035445567200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5586555174e0 con 0x5586555347a0 2023-10-05T19:40:21.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.907+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.1 v2:172.21.15.12:6808/29933 41 ==== osd_op_reply(111 notify.1 [watch unwatch cookie 94035445567200] v28'74 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd534014040 con 0x5586555347a0 2023-10-05T19:40:21.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.907+0000 7fd54505fe40 20 remove_watcher() i=1 2023-10-05T19:40:21.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.907+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:112 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94035445564992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5586555174e0 con 0x55865553d850 2023-10-05T19:40:21.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.911+0000 7fd542393700 1 -- 172.21.15.12:0/1232190650 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(112 notify.2 [watch unwatch cookie 94035445564992] v28'74 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd52c03c040 con 0x55865553d850 2023-10-05T19:40:21.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.911+0000 7fd54505fe40 20 remove_watcher() i=2 2023-10-05T19:40:21.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.911+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:113 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94035445573152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5586555174e0 con 0x5586555390b0 2023-10-05T19:40:21.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.911+0000 7fd542b94700 1 -- 172.21.15.12:0/1232190650 <== osd.2 v2:172.21.15.12:6800/29934 24 ==== osd_op_reply(113 notify.3 [watch unwatch cookie 94035445573152] v28'74 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd53c035070 con 0x5586555390b0 2023-10-05T19:40:21.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.911+0000 7fd54505fe40 20 remove_watcher() i=3 2023-10-05T19:40:21.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.911+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:114 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94035445568304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5586555174e0 con 0x5586555347a0 2023-10-05T19:40:21.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.915+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.1 v2:172.21.15.12:6808/29933 42 ==== osd_op_reply(114 notify.4 [watch unwatch cookie 94035445568304] v28'74 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd534014040 con 0x5586555347a0 2023-10-05T19:40:21.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.915+0000 7fd54505fe40 20 remove_watcher() i=4 2023-10-05T19:40:21.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.915+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:115 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94035445575360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5586555453d0 con 0x558655501d10 2023-10-05T19:40:21.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.919+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(115 notify.5 [watch unwatch cookie 94035445575360] v28'74 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd534014040 con 0x558655501d10 2023-10-05T19:40:21.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.919+0000 7fd54505fe40 20 remove_watcher() i=5 2023-10-05T19:40:21.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.919+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:116 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94035445580240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5586555453d0 con 0x558655501d10 2023-10-05T19:40:21.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.919+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(116 notify.6 [watch unwatch cookie 94035445580240] v28'74 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd534014040 con 0x558655501d10 2023-10-05T19:40:21.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.919+0000 7fd54505fe40 20 remove_watcher() i=6 2023-10-05T19:40:21.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.919+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:117 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94035445584832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5586555453d0 con 0x558655501d10 2023-10-05T19:40:21.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.923+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(117 notify.7 [watch unwatch cookie 94035445584832] v28'74 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd534014040 con 0x558655501d10 2023-10-05T19:40:21.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.923+0000 7fd54505fe40 20 remove_watcher() i=7 2023-10-05T19:40:21.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.927+0000 7fd48ffff700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:21.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.927+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:118 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94035445585936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5586555453d0 con 0x5586555347a0 2023-10-05T19:40:21.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.927+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 <== osd.1 v2:172.21.15.12:6808/29933 43 ==== osd_op_reply(118 data_loggenerations_metadata [watch unwatch cookie 94035445585936] v28'83 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fd534014040 con 0x5586555347a0 2023-10-05T19:40:21.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.927+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x558655501d10 msgr2=0x7fd53c0460b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.927+0000 7fd54505fe40 1 --2- 172.21.15.12:0/1232190650 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x558655501d10 0x7fd53c0460b0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.927+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5586555347a0 msgr2=0x558655536b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.927+0000 7fd54505fe40 1 --2- 172.21.15.12:0/1232190650 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5586555347a0 0x558655536b80 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.927+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5586555390b0 msgr2=0x55865553b490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.927+0000 7fd54505fe40 1 --2- 172.21.15.12:0/1232190650 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5586555390b0 0x55865553b490 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.927+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55865553d850 msgr2=0x55865553fc60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.927+0000 7fd54505fe40 1 --2- 172.21.15.12:0/1232190650 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55865553d850 0x55865553fc60 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.927+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd524030c00 msgr2=0x7fd5240330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.927+0000 7fd54505fe40 1 --2- 172.21.15.12:0/1232190650 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd524030c00 0x7fd5240330b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd52c0044f0 tx=0x7fd52c006690 comp rx=0 tx=0).stop 2023-10-05T19:40:21.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.927+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558655502620 msgr2=0x55865552ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.927+0000 7fd54505fe40 1 --2- 172.21.15.12:0/1232190650 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558655502620 0x55865552ad80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd53c00afd0 tx=0x7fd53c01c000 comp rx=0 tx=0).stop 2023-10-05T19:40:21.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.927+0000 7fd543395700 1 -- 172.21.15.12:0/1232190650 reap_dead start 2023-10-05T19:40:21.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.931+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 shutdown_connections 2023-10-05T19:40:21.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.931+0000 7fd54505fe40 1 --2- 172.21.15.12:0/1232190650 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558655502620 0x55865552ad80 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.931+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 >> 172.21.15.12:0/1232190650 conn(0x5586554fd730 msgr2=0x55865551b280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:21.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.931+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 shutdown_connections 2023-10-05T19:40:21.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.931+0000 7fd54505fe40 1 -- 172.21.15.12:0/1232190650 wait complete. 2023-10-05T19:40:21.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.931+0000 7fd54505fe40 1 -- 172.21.15.12:0/4156430770 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd480030c50 msgr2=0x7fd480033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.931+0000 7fd54505fe40 1 --2- 172.21.15.12:0/4156430770 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd480030c50 0x7fd480033100 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x5586555473c0 tx=0x7fd53400a9b0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.931+0000 7fd54505fe40 1 -- 172.21.15.12:0/4156430770 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558655557190 msgr2=0x55865555cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.931+0000 7fd54505fe40 1 --2- 172.21.15.12:0/4156430770 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558655557190 0x55865555cd70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd52c042040 tx=0x7fd52c03f000 comp rx=0 tx=0).stop 2023-10-05T19:40:21.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.931+0000 7fd54505fe40 1 -- 172.21.15.12:0/4156430770 shutdown_connections 2023-10-05T19:40:21.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.931+0000 7fd54505fe40 1 --2- 172.21.15.12:0/4156430770 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd480030c50 0x7fd480033100 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.931+0000 7fd54505fe40 1 --2- 172.21.15.12:0/4156430770 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558655557190 0x55865555cd70 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.931+0000 7fd54505fe40 1 -- 172.21.15.12:0/4156430770 >> 172.21.15.12:0/4156430770 conn(0x558655552550 msgr2=0x5586555505f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:21.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.931+0000 7fd54505fe40 1 -- 172.21.15.12:0/4156430770 shutdown_connections 2023-10-05T19:40:21.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.931+0000 7fd54505fe40 1 -- 172.21.15.12:0/4156430770 wait complete. 2023-10-05T19:40:21.945 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}, {'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:40:21.945 INFO:tasks.util.rgw:rgwadmin: client.0 : ['subuser', 'rm', '--subuser', 'foo:foo1'] 2023-10-05T19:40:21.946 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'subuser', 'rm', '--subuser', 'foo:foo1'] 2023-10-05T19:40:21.946 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph subuser rm --subuser foo:foo1 2023-10-05T19:40:21.985 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:21.986 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:21.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.979+0000 7fe66253ee40 1 Processor -- start 2023-10-05T19:40:21.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.979+0000 7fe66253ee40 1 -- start start 2023-10-05T19:40:21.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.979+0000 7fe66253ee40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x558f2a64b910 0x558f2a6d9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.979+0000 7fe66253ee40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x558f2a623bd0 con 0x558f2a64cc20 2023-10-05T19:40:21.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.979+0000 7fe66253ee40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x558f2a5e0ee0 con 0x558f2a64b910 2023-10-05T19:40:21.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.979+0000 7fe65f872700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x558f2a64b910 0x558f2a6d9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.979+0000 7fe65f872700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x558f2a64b910 0x558f2a6d9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37416/0 (socket says 172.21.15.12:37416) 2023-10-05T19:40:21.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.979+0000 7fe65f872700 1 -- 172.21.15.12:0/2759608790 learned_addr learned my addr 172.21.15.12:0/2759608790 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:21.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.979+0000 7fe660073700 1 --1- 172.21.15.12:0/2759608790 >> v1:172.21.15.8:6789/0 conn(0x558f2a64cc20 0x558f2a64b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:47350/0 (socket says 172.21.15.12:47350) 2023-10-05T19:40:21.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.979+0000 7fe65f872700 1 -- 172.21.15.12:0/2759608790 >> v1:172.21.15.8:6789/0 conn(0x558f2a64cc20 legacy=0x558f2a64b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.979+0000 7fe65f872700 1 -- 172.21.15.12:0/2759608790 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f2a64d0b0 con 0x558f2a64b910 2023-10-05T19:40:21.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.979+0000 7fe65f872700 1 --2- 172.21.15.12:0/2759608790 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558f2a64b910 0x558f2a6d9b40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe64800a5c0 tx=0x7fe648005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d2174f38a86d59b server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.979+0000 7fe65f071700 1 -- 172.21.15.12:0/2759608790 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe64800e510 con 0x558f2a64b910 2023-10-05T19:40:21.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.979+0000 7fe65f071700 1 -- 172.21.15.12:0/2759608790 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe64800e670 con 0x558f2a64b910 2023-10-05T19:40:21.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.979+0000 7fe65f071700 1 -- 172.21.15.12:0/2759608790 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe64800e920 con 0x558f2a64b910 2023-10-05T19:40:21.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.979+0000 7fe66253ee40 1 -- 172.21.15.12:0/2759608790 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558f2a64b910 msgr2=0x558f2a6d9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.979+0000 7fe66253ee40 1 --2- 172.21.15.12:0/2759608790 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558f2a64b910 0x558f2a6d9b40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe64800a5c0 tx=0x7fe648005e00 comp rx=0 tx=0).stop 2023-10-05T19:40:21.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.979+0000 7fe66253ee40 1 -- 172.21.15.12:0/2759608790 shutdown_connections 2023-10-05T19:40:21.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.979+0000 7fe66253ee40 1 --2- 172.21.15.12:0/2759608790 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558f2a64b910 0x558f2a6d9b40 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.979+0000 7fe66253ee40 1 -- 172.21.15.12:0/2759608790 >> 172.21.15.12:0/2759608790 conn(0x558f2a6d2850 msgr2=0x558f2a6d4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:21.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.979+0000 7fe66253ee40 1 -- 172.21.15.12:0/2759608790 shutdown_connections 2023-10-05T19:40:21.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.979+0000 7fe66253ee40 1 -- 172.21.15.12:0/2759608790 wait complete. 2023-10-05T19:40:21.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.983+0000 7fe66253ee40 1 Processor -- start 2023-10-05T19:40:21.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.983+0000 7fe66253ee40 1 -- start start 2023-10-05T19:40:21.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.983+0000 7fe66253ee40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558f2a64b910 0x558f2a6fc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.983+0000 7fe66253ee40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f2a5a4e30 con 0x558f2a64b910 2023-10-05T19:40:21.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.983+0000 7fe660073700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558f2a64b910 0x558f2a6fc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.983+0000 7fe660073700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558f2a64b910 0x558f2a6fc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37420/0 (socket says 172.21.15.12:37420) 2023-10-05T19:40:21.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.983+0000 7fe660073700 1 -- 172.21.15.12:0/4140128822 learned_addr learned my addr 172.21.15.12:0/4140128822 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:21.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.983+0000 7fe660073700 1 -- 172.21.15.12:0/4140128822 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe64800a180 con 0x558f2a64b910 2023-10-05T19:40:21.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.983+0000 7fe660073700 1 --2- 172.21.15.12:0/4140128822 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558f2a64b910 0x558f2a6fc500 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe650001cf0 tx=0x7fe6500018a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3148736252372ac server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.983+0000 7fe65e06f700 1 -- 172.21.15.12:0/4140128822 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe650014070 con 0x558f2a64b910 2023-10-05T19:40:21.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.983+0000 7fe65e06f700 1 -- 172.21.15.12:0/4140128822 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe65000be10 con 0x558f2a64b910 2023-10-05T19:40:21.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.983+0000 7fe65e06f700 1 -- 172.21.15.12:0/4140128822 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe65000f590 con 0x558f2a64b910 2023-10-05T19:40:21.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.983+0000 7fe66253ee40 1 -- 172.21.15.12:0/4140128822 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558f2a64b910 msgr2=0x558f2a6fc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:21.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.983+0000 7fe66253ee40 1 --2- 172.21.15.12:0/4140128822 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558f2a64b910 0x558f2a6fc500 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe650001cf0 tx=0x7fe6500018a0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.983+0000 7fe66253ee40 1 -- 172.21.15.12:0/4140128822 shutdown_connections 2023-10-05T19:40:21.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.983+0000 7fe66253ee40 1 --2- 172.21.15.12:0/4140128822 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558f2a64b910 0x558f2a6fc500 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:21.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.983+0000 7fe66253ee40 1 -- 172.21.15.12:0/4140128822 >> 172.21.15.12:0/4140128822 conn(0x558f2a6d2850 msgr2=0x558f2a6f80c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:21.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.983+0000 7fe66253ee40 1 -- 172.21.15.12:0/4140128822 shutdown_connections 2023-10-05T19:40:21.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.983+0000 7fe66253ee40 1 -- 172.21.15.12:0/4140128822 wait complete. 2023-10-05T19:40:21.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.983+0000 7fe66253ee40 1 Processor -- start 2023-10-05T19:40:21.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.987+0000 7fe66253ee40 1 -- start start 2023-10-05T19:40:21.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.987+0000 7fe66253ee40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558f2a64b910 0x558f2a7032d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.987+0000 7fe66253ee40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f2a577ed0 con 0x558f2a64b910 2023-10-05T19:40:21.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.987+0000 7fe660073700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558f2a64b910 0x558f2a7032d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.987+0000 7fe660073700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558f2a64b910 0x558f2a7032d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37426/0 (socket says 172.21.15.12:37426) 2023-10-05T19:40:21.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.987+0000 7fe660073700 1 -- 172.21.15.12:0/830148928 learned_addr learned my addr 172.21.15.12:0/830148928 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:21.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.987+0000 7fe660073700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f2a623bd0 con 0x558f2a64b910 2023-10-05T19:40:21.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.987+0000 7fe660073700 1 --2- 172.21.15.12:0/830148928 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558f2a64b910 0x558f2a7032d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe6500018e0 tx=0x7fe650016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:21.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.987+0000 7fe65c86c700 1 -- 172.21.15.12:0/830148928 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe65001c670 con 0x558f2a64b910 2023-10-05T19:40:21.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.987+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558f2a703810 con 0x558f2a64b910 2023-10-05T19:40:21.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.987+0000 7fe65c86c700 1 -- 172.21.15.12:0/830148928 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe65001c7f0 con 0x558f2a64b910 2023-10-05T19:40:21.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.987+0000 7fe65c86c700 1 -- 172.21.15.12:0/830148928 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe65001cae0 con 0x558f2a64b910 2023-10-05T19:40:21.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.987+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558f2a703c50 con 0x558f2a64b910 2023-10-05T19:40:21.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.991+0000 7fe65c86c700 1 -- 172.21.15.12:0/830148928 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fe650023020 con 0x558f2a64b910 2023-10-05T19:40:21.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.991+0000 7fe65c86c700 1 --2- 172.21.15.12:0/830148928 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe644030c00 0x7fe6440330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:21.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.991+0000 7fe65c86c700 1 -- 172.21.15.12:0/830148928 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7fe650008030 con 0x558f2a64b910 2023-10-05T19:40:21.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.991+0000 7fe66253ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:21.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.991+0000 7fe65f872700 1 --2- 172.21.15.12:0/830148928 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe644030c00 0x7fe6440330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:21.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.991+0000 7fe66253ee40 1 --2- 172.21.15.12:0/830148928 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x558f2a64cc20 0x558f2a70edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.991+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a70f470 con 0x558f2a64cc20 2023-10-05T19:40:22.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.991+0000 7fe660874700 1 --2- 172.21.15.12:0/830148928 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x558f2a64cc20 0x558f2a70edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.991+0000 7fe65f872700 1 --2- 172.21.15.12:0/830148928 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe644030c00 0x7fe6440330b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe648000f80 tx=0x7fe648013000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.991+0000 7fe660874700 1 --2- 172.21.15.12:0/830148928 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x558f2a64cc20 0x558f2a70edc0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.991+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe65800a500 con 0x558f2a64cc20 2023-10-05T19:40:22.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.991+0000 7fe66253ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:22.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.991+0000 7fe66253ee40 20 realm 2023-10-05T19:40:22.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.991+0000 7fe66253ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.991+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a70c420 con 0x558f2a64cc20 2023-10-05T19:40:22.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.991+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe65800a500 con 0x558f2a64cc20 2023-10-05T19:40:22.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.991+0000 7fe66253ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:22.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.991+0000 7fe66253ee40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:22.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.991+0000 7fe66253ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.991+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a70ff50 con 0x558f2a64cc20 2023-10-05T19:40:22.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.995+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe65800a500 con 0x558f2a64cc20 2023-10-05T19:40:22.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.995+0000 7fe66253ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:22.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.995+0000 7fe66253ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.995+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a70fb60 con 0x558f2a64cc20 2023-10-05T19:40:22.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.995+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fe65800a500 con 0x558f2a64cc20 2023-10-05T19:40:22.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.995+0000 7fe66253ee40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:22.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.995+0000 7fe66253ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.995+0000 7fe66253ee40 1 --2- 172.21.15.12:0/830148928 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x558f2a7112f0 0x558f2a7136d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.995+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a713c10 con 0x558f2a7112f0 2023-10-05T19:40:22.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.995+0000 7fe660073700 1 --2- 172.21.15.12:0/830148928 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x558f2a7112f0 0x558f2a7136d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.995+0000 7fe660073700 1 --2- 172.21.15.12:0/830148928 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x558f2a7112f0 0x558f2a7136d0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.995+0000 7fe660073700 1 -- 172.21.15.12:0/830148928 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fe650016070 con 0x558f2a7112f0 2023-10-05T19:40:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.995+0000 7fe66253ee40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.995+0000 7fe66253ee40 20 searching for the correct realm 2023-10-05T19:40:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.995+0000 7fe66253ee40 1 --2- 172.21.15.12:0/830148928 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x558f2a715a90 0x558f2a717ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.995+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a718550 con 0x558f2a715a90 2023-10-05T19:40:22.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.995+0000 7fe65f872700 1 --2- 172.21.15.12:0/830148928 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x558f2a715a90 0x558f2a717ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.999+0000 7fe65f872700 1 --2- 172.21.15.12:0/830148928 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x558f2a715a90 0x558f2a717ea0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.999+0000 7fe65f872700 1 -- 172.21.15.12:0/830148928 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fe648017050 con 0x558f2a715a90 2023-10-05T19:40:22.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.999+0000 7fe65f872700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a718550 con 0x558f2a7112f0 2023-10-05T19:40:22.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.999+0000 7fe660073700 1 -- 172.21.15.12:0/830148928 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe650016070 con 0x558f2a7112f0 2023-10-05T19:40:22.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.999+0000 7fe660073700 1 --2- 172.21.15.12:0/830148928 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fe65001a070 0x7fe6500422e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:21.999+0000 7fe660073700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a718550 con 0x7fe65001a070 2023-10-05T19:40:22.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.003+0000 7fe660874700 1 --2- 172.21.15.12:0/830148928 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fe65001a070 0x7fe6500422e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.003+0000 7fe660874700 1 --2- 172.21.15.12:0/830148928 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fe65001a070 0x7fe6500422e0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.003+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe658014040 con 0x7fe65001a070 2023-10-05T19:40:22.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.003+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a718550 con 0x558f2a64cc20 2023-10-05T19:40:22.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.003+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe658014040 con 0x558f2a64cc20 2023-10-05T19:40:22.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.003+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a718550 con 0x558f2a64cc20 2023-10-05T19:40:22.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.003+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe658014040 con 0x558f2a64cc20 2023-10-05T19:40:22.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.003+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a718550 con 0x7fe65001a070 2023-10-05T19:40:22.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.007+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe658014040 con 0x7fe65001a070 2023-10-05T19:40:22.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.007+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a718550 con 0x7fe65001a070 2023-10-05T19:40:22.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.007+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe658014040 con 0x7fe65001a070 2023-10-05T19:40:22.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.007+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a718550 con 0x558f2a7112f0 2023-10-05T19:40:22.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.007+0000 7fe660073700 1 -- 172.21.15.12:0/830148928 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe650016070 con 0x558f2a7112f0 2023-10-05T19:40:22.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.007+0000 7fe660073700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a718550 con 0x558f2a715a90 2023-10-05T19:40:22.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.007+0000 7fe65f872700 1 -- 172.21.15.12:0/830148928 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe648017050 con 0x558f2a715a90 2023-10-05T19:40:22.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.007+0000 7fe65f872700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a718550 con 0x558f2a64cc20 2023-10-05T19:40:22.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.007+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe658014040 con 0x558f2a64cc20 2023-10-05T19:40:22.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.007+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a718550 con 0x7fe65001a070 2023-10-05T19:40:22.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.011+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe658014040 con 0x7fe65001a070 2023-10-05T19:40:22.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.011+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a718550 con 0x558f2a64cc20 2023-10-05T19:40:22.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.011+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe658014040 con 0x558f2a64cc20 2023-10-05T19:40:22.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.011+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a718550 con 0x558f2a7112f0 2023-10-05T19:40:22.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.011+0000 7fe660073700 1 -- 172.21.15.12:0/830148928 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe650016070 con 0x558f2a7112f0 2023-10-05T19:40:22.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.011+0000 7fe660073700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a718550 con 0x558f2a715a90 2023-10-05T19:40:22.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.011+0000 7fe65f872700 1 -- 172.21.15.12:0/830148928 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe648017050 con 0x558f2a715a90 2023-10-05T19:40:22.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.011+0000 7fe65f872700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a718550 con 0x558f2a715a90 2023-10-05T19:40:22.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.011+0000 7fe65f872700 1 -- 172.21.15.12:0/830148928 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe648017050 con 0x558f2a715a90 2023-10-05T19:40:22.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.011+0000 7fe65f872700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a718550 con 0x7fe65001a070 2023-10-05T19:40:22.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.015+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe658014040 con 0x7fe65001a070 2023-10-05T19:40:22.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.015+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a718550 con 0x558f2a715a90 2023-10-05T19:40:22.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.015+0000 7fe65f872700 1 -- 172.21.15.12:0/830148928 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe648017050 con 0x558f2a715a90 2023-10-05T19:40:22.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.015+0000 7fe65f872700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a718550 con 0x558f2a715a90 2023-10-05T19:40:22.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.015+0000 7fe65f872700 1 -- 172.21.15.12:0/830148928 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe648017050 con 0x558f2a715a90 2023-10-05T19:40:22.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.015+0000 7fe65f872700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a718550 con 0x558f2a64cc20 2023-10-05T19:40:22.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.015+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe658014040 con 0x558f2a64cc20 2023-10-05T19:40:22.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.015+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a718550 con 0x558f2a64cc20 2023-10-05T19:40:22.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.015+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe658014040 con 0x558f2a64cc20 2023-10-05T19:40:22.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.015+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a718550 con 0x7fe65001a070 2023-10-05T19:40:22.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.015+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe658014040 con 0x7fe65001a070 2023-10-05T19:40:22.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.019+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a718550 con 0x558f2a715a90 2023-10-05T19:40:22.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.019+0000 7fe65f872700 1 -- 172.21.15.12:0/830148928 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe648017050 con 0x558f2a715a90 2023-10-05T19:40:22.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.019+0000 7fe65f872700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a718550 con 0x558f2a7112f0 2023-10-05T19:40:22.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.019+0000 7fe660073700 1 -- 172.21.15.12:0/830148928 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe650016070 con 0x558f2a7112f0 2023-10-05T19:40:22.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.019+0000 7fe660073700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a718550 con 0x558f2a64cc20 2023-10-05T19:40:22.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.019+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe658014040 con 0x558f2a64cc20 2023-10-05T19:40:22.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.019+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a718550 con 0x7fe65001a070 2023-10-05T19:40:22.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.019+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe658014040 con 0x7fe65001a070 2023-10-05T19:40:22.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.019+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a718550 con 0x558f2a715a90 2023-10-05T19:40:22.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.023+0000 7fe65f872700 1 -- 172.21.15.12:0/830148928 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe648017050 con 0x558f2a715a90 2023-10-05T19:40:22.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.023+0000 7fe65f872700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a718550 con 0x7fe65001a070 2023-10-05T19:40:22.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.023+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe658014040 con 0x7fe65001a070 2023-10-05T19:40:22.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.023+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a718550 con 0x558f2a7112f0 2023-10-05T19:40:22.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.023+0000 7fe660073700 1 -- 172.21.15.12:0/830148928 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe650016070 con 0x558f2a7112f0 2023-10-05T19:40:22.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.023+0000 7fe660073700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a718550 con 0x558f2a715a90 2023-10-05T19:40:22.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.023+0000 7fe65f872700 1 -- 172.21.15.12:0/830148928 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe648017050 con 0x558f2a715a90 2023-10-05T19:40:22.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.023+0000 7fe65f872700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a718550 con 0x7fe65001a070 2023-10-05T19:40:22.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.023+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe658014040 con 0x7fe65001a070 2023-10-05T19:40:22.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.023+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a718550 con 0x558f2a64cc20 2023-10-05T19:40:22.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.027+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fe658014040 con 0x558f2a64cc20 2023-10-05T19:40:22.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.027+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a718550 con 0x558f2a7112f0 2023-10-05T19:40:22.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.027+0000 7fe660073700 1 -- 172.21.15.12:0/830148928 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fe650016070 con 0x558f2a7112f0 2023-10-05T19:40:22.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.027+0000 7fe66253ee40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:22.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.027+0000 7fe66253ee40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:22.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.027+0000 7fe66253ee40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:22.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.027+0000 7fe66253ee40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:22.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.027+0000 7fe66253ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.027+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a714810 con 0x558f2a64cc20 2023-10-05T19:40:22.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.027+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe658014040 con 0x558f2a64cc20 2023-10-05T19:40:22.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.027+0000 7fe66253ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:22.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.027+0000 7fe66253ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.027+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a714440 con 0x558f2a64cc20 2023-10-05T19:40:22.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.027+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fe658014040 con 0x558f2a64cc20 2023-10-05T19:40:22.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.027+0000 7fe66253ee40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:22.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.027+0000 7fe66253ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.027+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a7189c0 con 0x558f2a715a90 2023-10-05T19:40:22.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.027+0000 7fe65f872700 1 -- 172.21.15.12:0/830148928 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fe648017050 con 0x558f2a715a90 2023-10-05T19:40:22.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.027+0000 7fe66253ee40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:22.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.027+0000 7fe66253ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.027+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a718d70 con 0x558f2a715a90 2023-10-05T19:40:22.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.031+0000 7fe65f872700 1 -- 172.21.15.12:0/830148928 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7fe648017050 con 0x558f2a715a90 2023-10-05T19:40:22.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.031+0000 7fe66253ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:22.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.031+0000 7fe66253ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.031+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a719120 con 0x558f2a64cc20 2023-10-05T19:40:22.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.031+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe658014040 con 0x558f2a64cc20 2023-10-05T19:40:22.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.031+0000 7fe66253ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:22.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.031+0000 7fe66253ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.031+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a714ca0 con 0x558f2a64cc20 2023-10-05T19:40:22.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.031+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fe658014040 con 0x558f2a64cc20 2023-10-05T19:40:22.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.031+0000 7fe66253ee40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:22.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.031+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a714c70 con 0x558f2a715a90 2023-10-05T19:40:22.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.035+0000 7fe65f872700 1 -- 172.21.15.12:0/830148928 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fe648017050 con 0x558f2a715a90 2023-10-05T19:40:22.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.035+0000 7fe65f872700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a714c70 con 0x558f2a7112f0 2023-10-05T19:40:22.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.035+0000 7fe660073700 1 -- 172.21.15.12:0/830148928 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe650016070 con 0x558f2a7112f0 2023-10-05T19:40:22.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.035+0000 7fe660073700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a714c70 con 0x7fe65001a070 2023-10-05T19:40:22.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.035+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe658014040 con 0x7fe65001a070 2023-10-05T19:40:22.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.035+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a719120 con 0x558f2a64cc20 2023-10-05T19:40:22.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.035+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe658014040 con 0x558f2a64cc20 2023-10-05T19:40:22.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.035+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a719120 con 0x558f2a64cc20 2023-10-05T19:40:22.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.039+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe658014040 con 0x558f2a64cc20 2023-10-05T19:40:22.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.039+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a719120 con 0x7fe65001a070 2023-10-05T19:40:22.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.039+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe658014040 con 0x7fe65001a070 2023-10-05T19:40:22.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.039+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a719120 con 0x7fe65001a070 2023-10-05T19:40:22.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.039+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe658014040 con 0x7fe65001a070 2023-10-05T19:40:22.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.039+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a719120 con 0x558f2a7112f0 2023-10-05T19:40:22.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.039+0000 7fe660073700 1 -- 172.21.15.12:0/830148928 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe650016070 con 0x558f2a7112f0 2023-10-05T19:40:22.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.039+0000 7fe660073700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a719120 con 0x558f2a715a90 2023-10-05T19:40:22.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.039+0000 7fe65f872700 1 -- 172.21.15.12:0/830148928 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe648017050 con 0x558f2a715a90 2023-10-05T19:40:22.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.039+0000 7fe65f872700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a719120 con 0x558f2a64cc20 2023-10-05T19:40:22.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.043+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe658014040 con 0x558f2a64cc20 2023-10-05T19:40:22.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.043+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a719120 con 0x7fe65001a070 2023-10-05T19:40:22.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.043+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe658014040 con 0x7fe65001a070 2023-10-05T19:40:22.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.043+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a719120 con 0x558f2a64cc20 2023-10-05T19:40:22.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.043+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe658014040 con 0x558f2a64cc20 2023-10-05T19:40:22.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.043+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a719120 con 0x558f2a7112f0 2023-10-05T19:40:22.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.043+0000 7fe660073700 1 -- 172.21.15.12:0/830148928 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe650016070 con 0x558f2a7112f0 2023-10-05T19:40:22.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.043+0000 7fe660073700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a719120 con 0x558f2a715a90 2023-10-05T19:40:22.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.047+0000 7fe65f872700 1 -- 172.21.15.12:0/830148928 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe648017050 con 0x558f2a715a90 2023-10-05T19:40:22.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.047+0000 7fe65f872700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a719120 con 0x558f2a715a90 2023-10-05T19:40:22.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.047+0000 7fe65f872700 1 -- 172.21.15.12:0/830148928 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe648017050 con 0x558f2a715a90 2023-10-05T19:40:22.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.047+0000 7fe65f872700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a719120 con 0x7fe65001a070 2023-10-05T19:40:22.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.047+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe658014040 con 0x7fe65001a070 2023-10-05T19:40:22.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.047+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a719120 con 0x558f2a715a90 2023-10-05T19:40:22.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.047+0000 7fe65f872700 1 -- 172.21.15.12:0/830148928 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe648017050 con 0x558f2a715a90 2023-10-05T19:40:22.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.047+0000 7fe65f872700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a719120 con 0x558f2a715a90 2023-10-05T19:40:22.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.047+0000 7fe65f872700 1 -- 172.21.15.12:0/830148928 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe648017050 con 0x558f2a715a90 2023-10-05T19:40:22.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.047+0000 7fe65f872700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a719120 con 0x558f2a64cc20 2023-10-05T19:40:22.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.051+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe658014040 con 0x558f2a64cc20 2023-10-05T19:40:22.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.051+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a719120 con 0x558f2a64cc20 2023-10-05T19:40:22.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.051+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe658014040 con 0x558f2a64cc20 2023-10-05T19:40:22.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.051+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a719120 con 0x7fe65001a070 2023-10-05T19:40:22.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.051+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe658014040 con 0x7fe65001a070 2023-10-05T19:40:22.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.051+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a719120 con 0x558f2a715a90 2023-10-05T19:40:22.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.051+0000 7fe65f872700 1 -- 172.21.15.12:0/830148928 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe648017050 con 0x558f2a715a90 2023-10-05T19:40:22.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.051+0000 7fe65f872700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a719120 con 0x558f2a7112f0 2023-10-05T19:40:22.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.055+0000 7fe660073700 1 -- 172.21.15.12:0/830148928 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe650016070 con 0x558f2a7112f0 2023-10-05T19:40:22.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.055+0000 7fe660073700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a719120 con 0x558f2a64cc20 2023-10-05T19:40:22.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.055+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe658014040 con 0x558f2a64cc20 2023-10-05T19:40:22.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.055+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a719120 con 0x7fe65001a070 2023-10-05T19:40:22.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.055+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe658014040 con 0x7fe65001a070 2023-10-05T19:40:22.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.055+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a719120 con 0x558f2a715a90 2023-10-05T19:40:22.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.055+0000 7fe65f872700 1 -- 172.21.15.12:0/830148928 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe648017050 con 0x558f2a715a90 2023-10-05T19:40:22.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.055+0000 7fe65f872700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a719120 con 0x7fe65001a070 2023-10-05T19:40:22.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.059+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe658014040 con 0x7fe65001a070 2023-10-05T19:40:22.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.059+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a719120 con 0x558f2a7112f0 2023-10-05T19:40:22.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.059+0000 7fe660073700 1 -- 172.21.15.12:0/830148928 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe650016070 con 0x558f2a7112f0 2023-10-05T19:40:22.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.059+0000 7fe660073700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a719120 con 0x558f2a715a90 2023-10-05T19:40:22.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.059+0000 7fe65f872700 1 -- 172.21.15.12:0/830148928 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe648017050 con 0x558f2a715a90 2023-10-05T19:40:22.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.059+0000 7fe65f872700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a719120 con 0x7fe65001a070 2023-10-05T19:40:22.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.059+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe658014040 con 0x7fe65001a070 2023-10-05T19:40:22.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.059+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a719120 con 0x558f2a64cc20 2023-10-05T19:40:22.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.063+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fe658014040 con 0x558f2a64cc20 2023-10-05T19:40:22.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.063+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a719120 con 0x558f2a7112f0 2023-10-05T19:40:22.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.063+0000 7fe660073700 1 -- 172.21.15.12:0/830148928 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fe650016070 con 0x558f2a7112f0 2023-10-05T19:40:22.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.063+0000 7fe66253ee40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:22.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.063+0000 7fe66253ee40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:22.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.063+0000 7fe66253ee40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:22.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.063+0000 7fe66253ee40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:22.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.063+0000 7fe66253ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.063+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a714c70 con 0x558f2a64cc20 2023-10-05T19:40:22.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.063+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fe658014040 con 0x558f2a64cc20 2023-10-05T19:40:22.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.063+0000 7fe66253ee40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:22.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.063+0000 7fe66253ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.063+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a719a10 con 0x558f2a7112f0 2023-10-05T19:40:22.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.063+0000 7fe660073700 1 -- 172.21.15.12:0/830148928 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fe650016070 con 0x558f2a7112f0 2023-10-05T19:40:22.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.063+0000 7fe66253ee40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:22.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.063+0000 7fe66253ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.063+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a71a300 con 0x558f2a64cc20 2023-10-05T19:40:22.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.063+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fe658014040 con 0x558f2a64cc20 2023-10-05T19:40:22.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.063+0000 7fe66253ee40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:22.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.063+0000 7fe66253ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.063+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a71a300 con 0x558f2a715a90 2023-10-05T19:40:22.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.067+0000 7fe65f872700 1 -- 172.21.15.12:0/830148928 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fe648017050 con 0x558f2a715a90 2023-10-05T19:40:22.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.067+0000 7fe66253ee40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:22.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.067+0000 7fe66253ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.067+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a71aa70 con 0x7fe65001a070 2023-10-05T19:40:22.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.067+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7fe658014040 con 0x7fe65001a070 2023-10-05T19:40:22.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.067+0000 7fe66253ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:22.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.067+0000 7fe66253ee40 20 zone default found 2023-10-05T19:40:22.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.067+0000 7fe66253ee40 4 Realm: () 2023-10-05T19:40:22.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.067+0000 7fe66253ee40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:22.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.067+0000 7fe66253ee40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:22.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.067+0000 7fe66253ee40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:22.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.067+0000 7fe66253ee40 20 zonegroup default 2023-10-05T19:40:22.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.067+0000 7fe66253ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.067+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a71aa20 con 0x7fe65001a070 2023-10-05T19:40:22.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.067+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fe658014040 con 0x7fe65001a070 2023-10-05T19:40:22.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.067+0000 7fe66253ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:22.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.067+0000 7fe66253ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.067+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a71c3a0 con 0x558f2a715a90 2023-10-05T19:40:22.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.067+0000 7fe65f872700 1 -- 172.21.15.12:0/830148928 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fe648017050 con 0x558f2a715a90 2023-10-05T19:40:22.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.067+0000 7fe66253ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:22.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.067+0000 7fe66253ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.067+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a71c750 con 0x558f2a715a90 2023-10-05T19:40:22.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.071+0000 7fe65f872700 1 -- 172.21.15.12:0/830148928 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fe648017050 con 0x558f2a715a90 2023-10-05T19:40:22.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.071+0000 7fe66253ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:22.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.071+0000 7fe66253ee40 20 started sync module instance, tier type = 2023-10-05T19:40:22.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.071+0000 7fe66253ee40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:22.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.071+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a71ba60 con 0x558f2a7112f0 2023-10-05T19:40:22.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.071+0000 7fe660073700 1 -- 172.21.15.12:0/830148928 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'75 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe650016070 con 0x558f2a7112f0 2023-10-05T19:40:22.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.071+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94073380773392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a71cb00 con 0x558f2a7112f0 2023-10-05T19:40:22.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.071+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a71d3a0 con 0x558f2a64cc20 2023-10-05T19:40:22.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.075+0000 7fe660073700 1 -- 172.21.15.12:0/830148928 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94073380773392] v28'76 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe650016070 con 0x558f2a7112f0 2023-10-05T19:40:22.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.075+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'75 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe658014040 con 0x558f2a64cc20 2023-10-05T19:40:22.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.075+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94073380781664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a71d3a0 con 0x558f2a64cc20 2023-10-05T19:40:22.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.075+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a71def0 con 0x558f2a715a90 2023-10-05T19:40:22.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.079+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94073380781664] v28'76 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe658014040 con 0x558f2a64cc20 2023-10-05T19:40:22.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.079+0000 7fe65f872700 1 -- 172.21.15.12:0/830148928 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'75 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe648017050 con 0x558f2a715a90 2023-10-05T19:40:22.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.079+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94073380785840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a71db20 con 0x558f2a715a90 2023-10-05T19:40:22.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.079+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a71ecd0 con 0x558f2a7112f0 2023-10-05T19:40:22.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.083+0000 7fe660073700 1 -- 172.21.15.12:0/830148928 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'75 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe650016070 con 0x558f2a7112f0 2023-10-05T19:40:22.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.083+0000 7fe65f872700 1 -- 172.21.15.12:0/830148928 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94073380785840] v28'76 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe648017050 con 0x558f2a715a90 2023-10-05T19:40:22.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.083+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94073380794496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a71e900 con 0x558f2a7112f0 2023-10-05T19:40:22.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.083+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a71e2f0 con 0x558f2a64cc20 2023-10-05T19:40:22.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.083+0000 7fe660073700 1 -- 172.21.15.12:0/830148928 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94073380794496] v28'76 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe650016070 con 0x558f2a7112f0 2023-10-05T19:40:22.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.087+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'75 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe658014040 con 0x558f2a64cc20 2023-10-05T19:40:22.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.087+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94073380795600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a71fc70 con 0x558f2a64cc20 2023-10-05T19:40:22.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.087+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a720680 con 0x7fe65001a070 2023-10-05T19:40:22.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.087+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94073380795600] v28'76 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe658014040 con 0x558f2a64cc20 2023-10-05T19:40:22.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.087+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'75 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe658014040 con 0x7fe65001a070 2023-10-05T19:40:22.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.087+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94073380791024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a720680 con 0x7fe65001a070 2023-10-05T19:40:22.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.087+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a7210e0 con 0x7fe65001a070 2023-10-05T19:40:22.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.091+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(95 notify.5 [watch watch cookie 94073380791024] v28'76 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe658014040 con 0x7fe65001a070 2023-10-05T19:40:22.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.091+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(96 notify.6 [create] v28'75 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe658014040 con 0x7fe65001a070 2023-10-05T19:40:22.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.091+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94073380799424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a7210e0 con 0x7fe65001a070 2023-10-05T19:40:22.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.091+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a721af0 con 0x7fe65001a070 2023-10-05T19:40:22.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.091+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(98 notify.7 [create] v28'75 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe658014040 con 0x7fe65001a070 2023-10-05T19:40:22.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.091+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(97 notify.6 [watch watch cookie 94073380799424] v28'76 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe658014040 con 0x7fe65001a070 2023-10-05T19:40:22.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.091+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94073380804096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a721f40 con 0x7fe65001a070 2023-10-05T19:40:22.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.091+0000 7fe66253ee40 20 add_watcher() i=0 2023-10-05T19:40:22.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.091+0000 7fe66253ee40 20 add_watcher() i=1 2023-10-05T19:40:22.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.091+0000 7fe66253ee40 20 add_watcher() i=2 2023-10-05T19:40:22.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.091+0000 7fe66253ee40 20 add_watcher() i=3 2023-10-05T19:40:22.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.091+0000 7fe66253ee40 20 add_watcher() i=4 2023-10-05T19:40:22.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.091+0000 7fe66253ee40 20 add_watcher() i=5 2023-10-05T19:40:22.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.091+0000 7fe66253ee40 20 add_watcher() i=6 2023-10-05T19:40:22.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.095+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94073380804096] v28'76 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe658014040 con 0x7fe65001a070 2023-10-05T19:40:22.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.095+0000 7fe66253ee40 20 add_watcher() i=7 2023-10-05T19:40:22.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.095+0000 7fe66253ee40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:22.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.095+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a724f70 con 0x558f2a64cc20 2023-10-05T19:40:22.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.095+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fe658014040 con 0x558f2a64cc20 2023-10-05T19:40:22.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.095+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94073380805200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a722390 con 0x558f2a64cc20 2023-10-05T19:40:22.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.099+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94073380805200] v28'84 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fe658014040 con 0x558f2a64cc20 2023-10-05T19:40:22.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.099+0000 7fe630ff1700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:22.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.099+0000 7fe66253ee40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:22.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.099+0000 7fe66253ee40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:22.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.099+0000 7fe66253ee40 1 Processor -- start 2023-10-05T19:40:22.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.099+0000 7fe66253ee40 1 -- start start 2023-10-05T19:40:22.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.099+0000 7fe66253ee40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558f2a72f410 0x558f2a72f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.099+0000 7fe66253ee40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f2a72fd20 con 0x558f2a72f410 2023-10-05T19:40:22.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.099+0000 7fe65f872700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558f2a72f410 0x558f2a72f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.099+0000 7fe65f872700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558f2a72f410 0x558f2a72f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37428/0 (socket says 172.21.15.12:37428) 2023-10-05T19:40:22.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.099+0000 7fe65f872700 1 -- 172.21.15.12:0/3554618492 learned_addr learned my addr 172.21.15.12:0/3554618492 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:22.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.099+0000 7fe65f872700 1 -- 172.21.15.12:0/3554618492 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f2a72fe60 con 0x558f2a72f410 2023-10-05T19:40:22.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.099+0000 7fe65f872700 1 --2- 172.21.15.12:0/3554618492 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558f2a72f410 0x558f2a72f7e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x558f2a64d080 tx=0x7fe648010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ef72b16ae86252 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.103+0000 7fe5abfff700 1 -- 172.21.15.12:0/3554618492 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe648010c10 con 0x558f2a72f410 2023-10-05T19:40:22.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.103+0000 7fe5abfff700 1 -- 172.21.15.12:0/3554618492 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe648010d90 con 0x558f2a72f410 2023-10-05T19:40:22.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.103+0000 7fe5abfff700 1 -- 172.21.15.12:0/3554618492 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe64801c430 con 0x558f2a72f410 2023-10-05T19:40:22.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.103+0000 7fe66253ee40 1 -- 172.21.15.12:0/3554618492 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558f2a72f410 msgr2=0x558f2a72f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.103+0000 7fe66253ee40 1 --2- 172.21.15.12:0/3554618492 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558f2a72f410 0x558f2a72f7e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x558f2a64d080 tx=0x7fe648010000 comp rx=0 tx=0).stop 2023-10-05T19:40:22.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.103+0000 7fe66253ee40 1 -- 172.21.15.12:0/3554618492 shutdown_connections 2023-10-05T19:40:22.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.103+0000 7fe66253ee40 1 --2- 172.21.15.12:0/3554618492 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558f2a72f410 0x558f2a72f7e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:22.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.103+0000 7fe66253ee40 1 -- 172.21.15.12:0/3554618492 >> 172.21.15.12:0/3554618492 conn(0x558f2a72a7d0 msgr2=0x558f2a728920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:22.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.103+0000 7fe66253ee40 1 -- 172.21.15.12:0/3554618492 shutdown_connections 2023-10-05T19:40:22.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.103+0000 7fe66253ee40 1 -- 172.21.15.12:0/3554618492 wait complete. 2023-10-05T19:40:22.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.103+0000 7fe66253ee40 1 Processor -- start 2023-10-05T19:40:22.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.103+0000 7fe66253ee40 1 -- start start 2023-10-05T19:40:22.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.103+0000 7fe66253ee40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558f2a72f410 0x558f2a734eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.103+0000 7fe66253ee40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f2a735eb0 con 0x558f2a72f410 2023-10-05T19:40:22.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.103+0000 7fe65f872700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558f2a72f410 0x558f2a734eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.103+0000 7fe65f872700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558f2a72f410 0x558f2a734eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37438/0 (socket says 172.21.15.12:37438) 2023-10-05T19:40:22.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.103+0000 7fe65f872700 1 -- 172.21.15.12:0/1323761749 learned_addr learned my addr 172.21.15.12:0/1323761749 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:22.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.103+0000 7fe65f872700 1 -- 172.21.15.12:0/1323761749 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f2a64d0b0 con 0x558f2a72f410 2023-10-05T19:40:22.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.103+0000 7fe65f872700 1 --2- 172.21.15.12:0/1323761749 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558f2a72f410 0x558f2a734eb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x558f2a729490 tx=0x7fe648036000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.107+0000 7fe5aa7fc700 1 -- 172.21.15.12:0/1323761749 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe64802fcf0 con 0x558f2a72f410 2023-10-05T19:40:22.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.107+0000 7fe5aa7fc700 1 -- 172.21.15.12:0/1323761749 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe64802fe90 con 0x558f2a72f410 2023-10-05T19:40:22.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.107+0000 7fe66253ee40 1 -- 172.21.15.12:0/1323761749 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558f2a7354b0 con 0x558f2a72f410 2023-10-05T19:40:22.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.107+0000 7fe5aa7fc700 1 -- 172.21.15.12:0/1323761749 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe64803f410 con 0x558f2a72f410 2023-10-05T19:40:22.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.107+0000 7fe66253ee40 1 -- 172.21.15.12:0/1323761749 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558f2a7358f0 con 0x558f2a72f410 2023-10-05T19:40:22.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.107+0000 7fe5aa7fc700 1 -- 172.21.15.12:0/1323761749 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fe648047070 con 0x558f2a72f410 2023-10-05T19:40:22.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.107+0000 7fe5aa7fc700 1 --2- 172.21.15.12:0/1323761749 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe5a0030c00 0x7fe5a00330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.107+0000 7fe5aa7fc700 1 -- 172.21.15.12:0/1323761749 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7fe648053b00 con 0x558f2a72f410 2023-10-05T19:40:22.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.107+0000 7fe66253ee40 5 note: GC not initialized 2023-10-05T19:40:22.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.107+0000 7fe660874700 1 --2- 172.21.15.12:0/1323761749 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe5a0030c00 0x7fe5a00330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.107+0000 7fe660874700 1 --2- 172.21.15.12:0/1323761749 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe5a0030c00 0x7fe5a00330b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x558f2a720e40 tx=0x7fe65800ab60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.111+0000 7fe59d7fa700 20 reqs_thread_entry: start 2023-10-05T19:40:22.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.111+0000 7fe66253ee40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:22.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.111+0000 7fe59f7fe700 20 reqs_thread_entry: start 2023-10-05T19:40:22.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.111+0000 7fe66253ee40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:22.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.111+0000 7fe58b7fe700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:52 2023 2023-10-05T19:40:22.120 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:22.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.111+0000 7fe58affd700 20 reqs_thread_entry: start 2023-10-05T19:40:22.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.111+0000 7fe66253ee40 20 get_system_obj_state: rctx=0x7fffd5e213a8 obj=default.rgw.meta:users.uid:foo state=0x558f2a5b00b0 s->prefetch_data=0 2023-10-05T19:40:22.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.111+0000 7fe66253ee40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2023-10-05T19:40:22.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.111+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a79d660 con 0x558f2a64cc20 2023-10-05T19:40:22.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.111+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(102 foo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7fe658014040 con 0x558f2a64cc20 2023-10-05T19:40:22.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.111+0000 7fe66253ee40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x16 2023-10-05T19:40:22.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.111+0000 7fe66253ee40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:22.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.111+0000 7fe66253ee40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:22.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.111+0000 7fe66253ee40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:22.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.111+0000 7fe66253ee40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:22.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.111+0000 7fe66253ee40 10 cache get: name=default.rgw.meta+users.uid+foo : type miss (requested=0x13, cached=0x16) 2023-10-05T19:40:22.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.111+0000 7fe66253ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.111+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a7a00f0 con 0x558f2a64cc20 2023-10-05T19:40:22.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.111+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(103 foo [call,call out=48b,read 0~539 out=539b,getxattrs out=75b] v0'0 uv9 ondisk = 0) v8 ==== 273+0+662 (crc 0 0 0) 0x7fe658014040 con 0x558f2a64cc20 2023-10-05T19:40:22.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.111+0000 7fe66253ee40 20 rados_obj.operate() r=0 bl.length=539 2023-10-05T19:40:22.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.111+0000 7fe66253ee40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2023-10-05T19:40:22.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.111+0000 7fe66253ee40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:22.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.111+0000 7fe66253ee40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:22.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.115+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:104 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.set in=58b,writefull 0~513 in=513b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a7a1370 con 0x558f2a64cc20 2023-10-05T19:40:22.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.115+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(104 foo [delete,create,call,writefull 0~513] v28'10 uv10 ondisk = 0) v8 ==== 273+0+0 (crc 0 0 0) 0x7fe658014040 con 0x558f2a64cc20 2023-10-05T19:40:22.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.115+0000 7fe66253ee40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2023-10-05T19:40:22.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.115+0000 7fe66253ee40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:22.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.115+0000 7fe66253ee40 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2023-10-05T19:40:22.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.115+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:105 7.d 7:b4812045:::notify.4:head [notify cookie 94073381326464 in=701b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a7a00f0 con 0x558f2a64cc20 2023-10-05T19:40:22.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.119+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== watch-notify(notify (1) cookie 94073380795600 notify 120259084297 ret 0) v3 ==== 731+0+0 (crc 0 0 0) 0x7fe658041450 con 0x558f2a64cc20 2023-10-05T19:40:22.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.119+0000 7fe65e06f700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084297 cookie 94073380795600 notifier 4375 bl.length()=689 2023-10-05T19:40:22.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.119+0000 7fe65e06f700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2023-10-05T19:40:22.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.119+0000 7fe65e06f700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:22.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.119+0000 7fe65e06f700 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:106 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fe638003d90 con 0x558f2a64cc20 2023-10-05T19:40:22.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.119+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(105 notify.4 [notify cookie 94073381326464 out=8b] v0'0 uv75 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fe658014040 con 0x558f2a64cc20 2023-10-05T19:40:22.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.119+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== osd_op_reply(106 notify.4 [notify-ack] v0'0 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe658014040 con 0x558f2a64cc20 2023-10-05T19:40:22.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.119+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.1 v2:172.21.15.12:6808/29933 39 ==== watch-notify(notify_complete (2) cookie 94073381326464 notify 120259084297 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7fe658026440 con 0x558f2a64cc20 2023-10-05T19:40:22.128 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:40:22.128 INFO:teuthology.orchestra.run.smithi012.stdout: "user_id": "foo", 2023-10-05T19:40:22.128 INFO:teuthology.orchestra.run.smithi012.stdout: "display_name": "Foo", 2023-10-05T19:40:22.128 INFO:teuthology.orchestra.run.smithi012.stdout: "email": "foo@foo.com", 2023-10-05T19:40:22.128 INFO:teuthology.orchestra.run.smithi012.stdout: "suspended": 0, 2023-10-05T19:40:22.128 INFO:teuthology.orchestra.run.smithi012.stdout: "max_buckets": 4, 2023-10-05T19:40:22.128 INFO:teuthology.orchestra.run.smithi012.stdout: "subusers": [ 2023-10-05T19:40:22.128 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:22.128 INFO:teuthology.orchestra.run.smithi012.stdout: "id": "foo:foo2", 2023-10-05T19:40:22.128 INFO:teuthology.orchestra.run.smithi012.stdout: "permissions": "" 2023-10-05T19:40:22.129 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:22.129 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:22.129 INFO:teuthology.orchestra.run.smithi012.stdout: "keys": [ 2023-10-05T19:40:22.129 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:22.129 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:40:22.129 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2023-10-05T19:40:22.129 INFO:teuthology.orchestra.run.smithi012.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2023-10-05T19:40:22.129 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:22.129 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:22.129 INFO:teuthology.orchestra.run.smithi012.stdout: "swift_keys": [ 2023-10-05T19:40:22.129 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:22.129 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo:foo2", 2023-10-05T19:40:22.130 INFO:teuthology.orchestra.run.smithi012.stdout: "secret_key": "ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy" 2023-10-05T19:40:22.130 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:22.130 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:22.130 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": [], 2023-10-05T19:40:22.130 INFO:teuthology.orchestra.run.smithi012.stdout: "op_mask": "read, write, delete", 2023-10-05T19:40:22.130 INFO:teuthology.orchestra.run.smithi012.stdout: "default_placement": "", 2023-10-05T19:40:22.130 INFO:teuthology.orchestra.run.smithi012.stdout: "default_storage_class": "", 2023-10-05T19:40:22.130 INFO:teuthology.orchestra.run.smithi012.stdout: "placement_tags": [], 2023-10-05T19:40:22.130 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket_quota": { 2023-10-05T19:40:22.130 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:22.130 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:22.130 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:22.131 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:22.131 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:22.131 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:22.131 INFO:teuthology.orchestra.run.smithi012.stdout: "user_quota": { 2023-10-05T19:40:22.131 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:22.131 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:22.131 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:22.131 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:22.131 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:22.131 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:22.131 INFO:teuthology.orchestra.run.smithi012.stdout: "temp_url_keys": [], 2023-10-05T19:40:22.131 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "rgw", 2023-10-05T19:40:22.132 INFO:teuthology.orchestra.run.smithi012.stdout: "mfa_ids": [] 2023-10-05T19:40:22.132 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:40:22.132 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:40:22.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.119+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:107 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94073380773392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a6f2da0 con 0x558f2a7112f0 2023-10-05T19:40:22.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.123+0000 7fe660073700 1 -- 172.21.15.12:0/830148928 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(107 notify.0 [watch unwatch cookie 94073380773392] v28'77 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe650016070 con 0x558f2a7112f0 2023-10-05T19:40:22.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.123+0000 7fe66253ee40 20 remove_watcher() i=0 2023-10-05T19:40:22.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.123+0000 7fe66253ee40 2 removed watcher, disabling cache 2023-10-05T19:40:22.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.123+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:108 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94073380781664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a6f2da0 con 0x558f2a64cc20 2023-10-05T19:40:22.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.123+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.1 v2:172.21.15.12:6808/29933 40 ==== osd_op_reply(108 notify.1 [watch unwatch cookie 94073380781664] v28'77 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe658014040 con 0x558f2a64cc20 2023-10-05T19:40:22.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.123+0000 7fe66253ee40 20 remove_watcher() i=1 2023-10-05T19:40:22.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.123+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:109 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94073380785840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a6f2da0 con 0x558f2a715a90 2023-10-05T19:40:22.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.127+0000 7fe65f872700 1 -- 172.21.15.12:0/830148928 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(109 notify.2 [watch unwatch cookie 94073380785840] v28'77 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe648017050 con 0x558f2a715a90 2023-10-05T19:40:22.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.127+0000 7fe66253ee40 20 remove_watcher() i=2 2023-10-05T19:40:22.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.127+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:110 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94073380794496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a6f2da0 con 0x558f2a7112f0 2023-10-05T19:40:22.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.131+0000 7fe660073700 1 -- 172.21.15.12:0/830148928 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(110 notify.3 [watch unwatch cookie 94073380794496] v28'77 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe650016070 con 0x558f2a7112f0 2023-10-05T19:40:22.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.131+0000 7fe66253ee40 20 remove_watcher() i=3 2023-10-05T19:40:22.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.131+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:111 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94073380795600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a6f2da0 con 0x558f2a64cc20 2023-10-05T19:40:22.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.131+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.1 v2:172.21.15.12:6808/29933 41 ==== osd_op_reply(111 notify.4 [watch unwatch cookie 94073380795600] v28'77 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe658014040 con 0x558f2a64cc20 2023-10-05T19:40:22.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.131+0000 7fe66253ee40 20 remove_watcher() i=4 2023-10-05T19:40:22.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.131+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:112 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94073380791024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a6f2da0 con 0x7fe65001a070 2023-10-05T19:40:22.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.135+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(112 notify.5 [watch unwatch cookie 94073380791024] v28'77 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe658014040 con 0x7fe65001a070 2023-10-05T19:40:22.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.135+0000 7fe66253ee40 20 remove_watcher() i=5 2023-10-05T19:40:22.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.135+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:113 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94073380799424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a6f2da0 con 0x7fe65001a070 2023-10-05T19:40:22.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.139+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(113 notify.6 [watch unwatch cookie 94073380799424] v28'77 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe658014040 con 0x7fe65001a070 2023-10-05T19:40:22.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.139+0000 7fe66253ee40 20 remove_watcher() i=6 2023-10-05T19:40:22.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.139+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:114 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94073380804096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a6f2da0 con 0x7fe65001a070 2023-10-05T19:40:22.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.139+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(114 notify.7 [watch unwatch cookie 94073380804096] v28'77 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe658014040 con 0x7fe65001a070 2023-10-05T19:40:22.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.139+0000 7fe66253ee40 20 remove_watcher() i=7 2023-10-05T19:40:22.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.143+0000 7fe630ff1700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:22.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.143+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:115 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94073380805200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558f2a6f2da0 con 0x558f2a64cc20 2023-10-05T19:40:22.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.147+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 <== osd.1 v2:172.21.15.12:6808/29933 42 ==== osd_op_reply(115 data_loggenerations_metadata [watch unwatch cookie 94073380805200] v28'85 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fe658014040 con 0x558f2a64cc20 2023-10-05T19:40:22.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.147+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fe65001a070 msgr2=0x7fe6500422e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.147+0000 7fe66253ee40 1 --2- 172.21.15.12:0/830148928 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fe65001a070 0x7fe6500422e0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:22.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.147+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x558f2a64cc20 msgr2=0x558f2a70edc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.147+0000 7fe66253ee40 1 --2- 172.21.15.12:0/830148928 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x558f2a64cc20 0x558f2a70edc0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:22.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.147+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x558f2a7112f0 msgr2=0x558f2a7136d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.147+0000 7fe66253ee40 1 --2- 172.21.15.12:0/830148928 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x558f2a7112f0 0x558f2a7136d0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:22.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.147+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x558f2a715a90 msgr2=0x558f2a717ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.147+0000 7fe66253ee40 1 --2- 172.21.15.12:0/830148928 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x558f2a715a90 0x558f2a717ea0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:22.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.147+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe644030c00 msgr2=0x7fe6440330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.147+0000 7fe66253ee40 1 --2- 172.21.15.12:0/830148928 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe644030c00 0x7fe6440330b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe648000f80 tx=0x7fe648013000 comp rx=0 tx=0).stop 2023-10-05T19:40:22.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.147+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558f2a64b910 msgr2=0x558f2a7032d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.147+0000 7fe660874700 1 -- 172.21.15.12:0/830148928 reap_dead start 2023-10-05T19:40:22.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.147+0000 7fe66253ee40 1 --2- 172.21.15.12:0/830148928 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558f2a64b910 0x558f2a7032d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe6500018e0 tx=0x7fe650016000 comp rx=0 tx=0).stop 2023-10-05T19:40:22.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.147+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 shutdown_connections 2023-10-05T19:40:22.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.147+0000 7fe66253ee40 1 --2- 172.21.15.12:0/830148928 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558f2a64b910 0x558f2a7032d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:22.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.147+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 >> 172.21.15.12:0/830148928 conn(0x558f2a6d2850 msgr2=0x558f2a6f6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:22.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.147+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 shutdown_connections 2023-10-05T19:40:22.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.147+0000 7fe66253ee40 1 -- 172.21.15.12:0/830148928 wait complete. 2023-10-05T19:40:22.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.147+0000 7fe66253ee40 1 -- 172.21.15.12:0/1323761749 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe5a0030c00 msgr2=0x7fe5a00330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.147+0000 7fe66253ee40 1 --2- 172.21.15.12:0/1323761749 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe5a0030c00 0x7fe5a00330b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x558f2a720e40 tx=0x7fe65800ab60 comp rx=0 tx=0).stop 2023-10-05T19:40:22.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.147+0000 7fe66253ee40 1 -- 172.21.15.12:0/1323761749 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558f2a72f410 msgr2=0x558f2a734eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.147+0000 7fe66253ee40 1 --2- 172.21.15.12:0/1323761749 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558f2a72f410 0x558f2a734eb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x558f2a729490 tx=0x7fe648036000 comp rx=0 tx=0).stop 2023-10-05T19:40:22.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.147+0000 7fe66253ee40 1 -- 172.21.15.12:0/1323761749 shutdown_connections 2023-10-05T19:40:22.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.147+0000 7fe66253ee40 1 --2- 172.21.15.12:0/1323761749 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe5a0030c00 0x7fe5a00330b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:22.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.147+0000 7fe66253ee40 1 --2- 172.21.15.12:0/1323761749 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558f2a72f410 0x558f2a734eb0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:22.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.147+0000 7fe66253ee40 1 -- 172.21.15.12:0/1323761749 >> 172.21.15.12:0/1323761749 conn(0x558f2a72a7d0 msgr2=0x558f2a728870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:22.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.147+0000 7fe66253ee40 1 -- 172.21.15.12:0/1323761749 shutdown_connections 2023-10-05T19:40:22.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.147+0000 7fe66253ee40 1 -- 172.21.15.12:0/1323761749 wait complete. 2023-10-05T19:40:22.162 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:40:22.162 INFO:tasks.util.rgw:rgwadmin: client.0 : ['subuser', 'rm', '--subuser', 'foo:foo2', '--key-type', 'swift', '--purge-keys'] 2023-10-05T19:40:22.162 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'subuser', 'rm', '--subuser', 'foo:foo2', '--key-type', 'swift', '--purge-keys'] 2023-10-05T19:40:22.162 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph subuser rm --subuser foo:foo2 --key-type swift --purge-keys 2023-10-05T19:40:22.250 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:22.250 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:22.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.243+0000 7feb0e2a9e40 1 Processor -- start 2023-10-05T19:40:22.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.243+0000 7feb0e2a9e40 1 -- start start 2023-10-05T19:40:22.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.243+0000 7feb0e2a9e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55fe69523fb0 0x55fe69524380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.243+0000 7feb0e2a9e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55fe6946ebd0 con 0x55fe695248c0 2023-10-05T19:40:22.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.243+0000 7feb0e2a9e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x55fe6942bee0 con 0x55fe69523fb0 2023-10-05T19:40:22.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.243+0000 7feb0bdde700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55fe69523fb0 0x55fe69524380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.243+0000 7feb0b5dd700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x55fe695248c0 0x55fe69525360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:47360/0 (socket says 172.21.15.12:47360) 2023-10-05T19:40:22.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.243+0000 7feb0b5dd700 1 -- 172.21.15.12:0/954463141 learned_addr learned my addr 172.21.15.12:0/954463141 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:22.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.243+0000 7feb0addc700 1 -- 172.21.15.12:0/954463141 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3040273172 0 0) 0x55fe6946ebd0 con 0x55fe695248c0 2023-10-05T19:40:22.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.243+0000 7feb0addc700 1 -- 172.21.15.12:0/954463141 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feaec001e10 con 0x55fe695248c0 2023-10-05T19:40:22.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.243+0000 7feb0bdde700 1 -- 172.21.15.12:0/954463141 >> v1:172.21.15.8:6789/0 conn(0x55fe695248c0 legacy=0x55fe69525360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.243+0000 7feb0bdde700 1 -- 172.21.15.12:0/954463141 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe69525a70 con 0x55fe69523fb0 2023-10-05T19:40:22.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.243+0000 7feb0bdde700 1 --2- 172.21.15.12:0/954463141 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fe69523fb0 0x55fe69524380 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7feafc00a5d0 tx=0x7feafc005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5cf90aca50bd53a server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.243+0000 7feb0addc700 1 -- 172.21.15.12:0/954463141 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feafc007610 con 0x55fe69523fb0 2023-10-05T19:40:22.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.243+0000 7feb0addc700 1 -- 172.21.15.12:0/954463141 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feafc007770 con 0x55fe69523fb0 2023-10-05T19:40:22.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.243+0000 7feb0addc700 1 -- 172.21.15.12:0/954463141 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feafc007a40 con 0x55fe69523fb0 2023-10-05T19:40:22.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.247+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/954463141 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fe69523fb0 msgr2=0x55fe69524380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.247+0000 7feb0e2a9e40 1 --2- 172.21.15.12:0/954463141 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fe69523fb0 0x55fe69524380 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7feafc00a5d0 tx=0x7feafc005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:22.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.247+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/954463141 shutdown_connections 2023-10-05T19:40:22.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.247+0000 7feb0e2a9e40 1 --2- 172.21.15.12:0/954463141 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fe69523fb0 0x55fe69524380 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:22.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.247+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/954463141 >> 172.21.15.12:0/954463141 conn(0x55fe69497800 msgr2=0x55fe69497c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:22.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.247+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/954463141 shutdown_connections 2023-10-05T19:40:22.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.247+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/954463141 wait complete. 2023-10-05T19:40:22.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.247+0000 7feb0e2a9e40 1 Processor -- start 2023-10-05T19:40:22.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.247+0000 7feb0e2a9e40 1 -- start start 2023-10-05T19:40:22.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.247+0000 7feb0e2a9e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fe69523fb0 0x55fe69542f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.247+0000 7feb0e2a9e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe693efe30 con 0x55fe69523fb0 2023-10-05T19:40:22.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.247+0000 7feb0bdde700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fe69523fb0 0x55fe69542f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.247+0000 7feb0bdde700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fe69523fb0 0x55fe69542f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37468/0 (socket says 172.21.15.12:37468) 2023-10-05T19:40:22.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.247+0000 7feb0bdde700 1 -- 172.21.15.12:0/2249288572 learned_addr learned my addr 172.21.15.12:0/2249288572 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:22.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.247+0000 7feb0bdde700 1 -- 172.21.15.12:0/2249288572 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feafc005cf0 con 0x55fe69523fb0 2023-10-05T19:40:22.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.247+0000 7feb0bdde700 1 --2- 172.21.15.12:0/2249288572 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fe69523fb0 0x55fe69542f30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7feafc000c00 tx=0x7feafc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19de553dbfe3ba1c server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.247+0000 7feb09dda700 1 -- 172.21.15.12:0/2249288572 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feafc0078e0 con 0x55fe69523fb0 2023-10-05T19:40:22.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.247+0000 7feb09dda700 1 -- 172.21.15.12:0/2249288572 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feafc007a40 con 0x55fe69523fb0 2023-10-05T19:40:22.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.247+0000 7feb09dda700 1 -- 172.21.15.12:0/2249288572 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feafc007bd0 con 0x55fe69523fb0 2023-10-05T19:40:22.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.251+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/2249288572 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fe69523fb0 msgr2=0x55fe69542f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.251+0000 7feb0e2a9e40 1 --2- 172.21.15.12:0/2249288572 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fe69523fb0 0x55fe69542f30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7feafc000c00 tx=0x7feafc014000 comp rx=0 tx=0).stop 2023-10-05T19:40:22.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.251+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/2249288572 shutdown_connections 2023-10-05T19:40:22.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.251+0000 7feb0e2a9e40 1 --2- 172.21.15.12:0/2249288572 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fe69523fb0 0x55fe69542f30 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:22.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.251+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/2249288572 >> 172.21.15.12:0/2249288572 conn(0x55fe69497800 msgr2=0x55fe6953ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:22.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.251+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/2249288572 shutdown_connections 2023-10-05T19:40:22.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.251+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/2249288572 wait complete. 2023-10-05T19:40:22.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.251+0000 7feb0e2a9e40 1 Processor -- start 2023-10-05T19:40:22.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.251+0000 7feb0e2a9e40 1 -- start start 2023-10-05T19:40:22.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.251+0000 7feb0e2a9e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fe69523fb0 0x55fe69549c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.251+0000 7feb0e2a9e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe693c2ed0 con 0x55fe69523fb0 2023-10-05T19:40:22.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.251+0000 7feb0bdde700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fe69523fb0 0x55fe69549c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.251+0000 7feb0bdde700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fe69523fb0 0x55fe69549c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37470/0 (socket says 172.21.15.12:37470) 2023-10-05T19:40:22.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.251+0000 7feb0bdde700 1 -- 172.21.15.12:0/4008125634 learned_addr learned my addr 172.21.15.12:0/4008125634 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:22.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.251+0000 7feb0bdde700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe69525a70 con 0x55fe69523fb0 2023-10-05T19:40:22.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.251+0000 7feb0bdde700 1 --2- 172.21.15.12:0/4008125634 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fe69523fb0 0x55fe69549c30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7feafc01f040 tx=0x7feafc01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.251+0000 7feb03fff700 1 -- 172.21.15.12:0/4008125634 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feafc007e50 con 0x55fe69523fb0 2023-10-05T19:40:22.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.251+0000 7feb03fff700 1 -- 172.21.15.12:0/4008125634 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feafc029070 con 0x55fe69523fb0 2023-10-05T19:40:22.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.251+0000 7feb03fff700 1 -- 172.21.15.12:0/4008125634 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feafc030480 con 0x55fe69523fb0 2023-10-05T19:40:22.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.251+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55fe6954a170 con 0x55fe69523fb0 2023-10-05T19:40:22.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.251+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe6954a5b0 con 0x55fe69523fb0 2023-10-05T19:40:22.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.255+0000 7feb03fff700 1 -- 172.21.15.12:0/4008125634 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7feafc012400 con 0x55fe69523fb0 2023-10-05T19:40:22.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.255+0000 7feb03fff700 1 --2- 172.21.15.12:0/4008125634 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7feaf0030bf0 0x7feaf00330a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.255+0000 7feb03fff700 1 -- 172.21.15.12:0/4008125634 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7feafc02b070 con 0x55fe69523fb0 2023-10-05T19:40:22.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.255+0000 7feb0b5dd700 1 --2- 172.21.15.12:0/4008125634 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7feaf0030bf0 0x7feaf00330a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.255+0000 7feb0e2a9e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.255+0000 7feb0e2a9e40 1 --2- 172.21.15.12:0/4008125634 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55fe695248c0 0x55fe69555680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.255+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe69555db0 con 0x55fe695248c0 2023-10-05T19:40:22.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.255+0000 7feb0c5df700 1 --2- 172.21.15.12:0/4008125634 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55fe695248c0 0x55fe69555680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.255+0000 7feb0b5dd700 1 --2- 172.21.15.12:0/4008125634 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7feaf0030bf0 0x7feaf00330a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7feaf40044f0 tx=0x7feaf4006690 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.255+0000 7feb0c5df700 1 --2- 172.21.15.12:0/4008125634 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55fe695248c0 0x55fe69555680 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.255+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7feb0400a500 con 0x55fe695248c0 2023-10-05T19:40:22.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.259+0000 7feb0e2a9e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:22.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.259+0000 7feb0e2a9e40 20 realm 2023-10-05T19:40:22.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.259+0000 7feb0e2a9e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.259+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe69552cb0 con 0x55fe695248c0 2023-10-05T19:40:22.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.259+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7feb0400a500 con 0x55fe695248c0 2023-10-05T19:40:22.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.259+0000 7feb0e2a9e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:22.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.259+0000 7feb0e2a9e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:22.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.259+0000 7feb0e2a9e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.259+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe695568e0 con 0x55fe695248c0 2023-10-05T19:40:22.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.259+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7feb0400a500 con 0x55fe695248c0 2023-10-05T19:40:22.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.259+0000 7feb0e2a9e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:22.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.259+0000 7feb0e2a9e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.259+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe695564f0 con 0x55fe695248c0 2023-10-05T19:40:22.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.259+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7feb0400a500 con 0x55fe695248c0 2023-10-05T19:40:22.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.259+0000 7feb0e2a9e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:22.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.259+0000 7feb0e2a9e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.259+0000 7feb0e2a9e40 1 --2- 172.21.15.12:0/4008125634 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55fe69557c80 0x55fe6955a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.259+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955a5a0 con 0x55fe69557c80 2023-10-05T19:40:22.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.259+0000 7feb0bdde700 1 --2- 172.21.15.12:0/4008125634 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55fe69557c80 0x55fe6955a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.259+0000 7feb0bdde700 1 --2- 172.21.15.12:0/4008125634 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55fe69557c80 0x55fe6955a060 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.259+0000 7feb0bdde700 1 -- 172.21.15.12:0/4008125634 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7feafc01c7f0 con 0x55fe69557c80 2023-10-05T19:40:22.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.259+0000 7feb0e2a9e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:22.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.259+0000 7feb0e2a9e40 20 searching for the correct realm 2023-10-05T19:40:22.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.259+0000 7feb0e2a9e40 1 --2- 172.21.15.12:0/4008125634 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55fe6955c420 0x55fe6955e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.259+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955eee0 con 0x55fe6955c420 2023-10-05T19:40:22.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.259+0000 7feb0b5dd700 1 --2- 172.21.15.12:0/4008125634 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55fe6955c420 0x55fe6955e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.263+0000 7feb0b5dd700 1 --2- 172.21.15.12:0/4008125634 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55fe6955c420 0x55fe6955e830 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.263+0000 7feb0b5dd700 1 -- 172.21.15.12:0/4008125634 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7feaf4004520 con 0x55fe6955c420 2023-10-05T19:40:22.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.263+0000 7feb0b5dd700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955eee0 con 0x55fe69557c80 2023-10-05T19:40:22.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.267+0000 7feb0bdde700 1 -- 172.21.15.12:0/4008125634 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feafc01c7f0 con 0x55fe69557c80 2023-10-05T19:40:22.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.267+0000 7feb0bdde700 1 --2- 172.21.15.12:0/4008125634 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7feafc007030 0x7feafc00c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.267+0000 7feb0bdde700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955eee0 con 0x7feafc007030 2023-10-05T19:40:22.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.267+0000 7feb0c5df700 1 --2- 172.21.15.12:0/4008125634 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7feafc007030 0x7feafc00c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.267+0000 7feb0c5df700 1 --2- 172.21.15.12:0/4008125634 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7feafc007030 0x7feafc00c9c0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.267+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feb04014040 con 0x7feafc007030 2023-10-05T19:40:22.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.267+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955eee0 con 0x55fe695248c0 2023-10-05T19:40:22.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.267+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feb04014040 con 0x55fe695248c0 2023-10-05T19:40:22.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.267+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955eee0 con 0x55fe695248c0 2023-10-05T19:40:22.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.267+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feb04014040 con 0x55fe695248c0 2023-10-05T19:40:22.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.271+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955eee0 con 0x7feafc007030 2023-10-05T19:40:22.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.271+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feb04014040 con 0x7feafc007030 2023-10-05T19:40:22.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.271+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955eee0 con 0x7feafc007030 2023-10-05T19:40:22.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.271+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feb04014040 con 0x7feafc007030 2023-10-05T19:40:22.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.271+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955eee0 con 0x55fe69557c80 2023-10-05T19:40:22.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.271+0000 7feb0bdde700 1 -- 172.21.15.12:0/4008125634 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feafc01c7f0 con 0x55fe69557c80 2023-10-05T19:40:22.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.271+0000 7feb0bdde700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955eee0 con 0x55fe6955c420 2023-10-05T19:40:22.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.271+0000 7feb0b5dd700 1 -- 172.21.15.12:0/4008125634 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feaf4004520 con 0x55fe6955c420 2023-10-05T19:40:22.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.271+0000 7feb0b5dd700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955eee0 con 0x55fe695248c0 2023-10-05T19:40:22.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.271+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feb04014040 con 0x55fe695248c0 2023-10-05T19:40:22.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.271+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955eee0 con 0x7feafc007030 2023-10-05T19:40:22.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.275+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feb04014040 con 0x7feafc007030 2023-10-05T19:40:22.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.275+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955eee0 con 0x55fe695248c0 2023-10-05T19:40:22.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.275+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feb04014040 con 0x55fe695248c0 2023-10-05T19:40:22.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.275+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955eee0 con 0x55fe69557c80 2023-10-05T19:40:22.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.275+0000 7feb0bdde700 1 -- 172.21.15.12:0/4008125634 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feafc01c7f0 con 0x55fe69557c80 2023-10-05T19:40:22.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.275+0000 7feb0bdde700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955eee0 con 0x55fe6955c420 2023-10-05T19:40:22.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.275+0000 7feb0b5dd700 1 -- 172.21.15.12:0/4008125634 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feaf4004520 con 0x55fe6955c420 2023-10-05T19:40:22.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.275+0000 7feb0b5dd700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955eee0 con 0x55fe6955c420 2023-10-05T19:40:22.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.279+0000 7feb0b5dd700 1 -- 172.21.15.12:0/4008125634 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feaf4004520 con 0x55fe6955c420 2023-10-05T19:40:22.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.279+0000 7feb0b5dd700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955eee0 con 0x7feafc007030 2023-10-05T19:40:22.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.279+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feb04014040 con 0x7feafc007030 2023-10-05T19:40:22.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.279+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955eee0 con 0x55fe6955c420 2023-10-05T19:40:22.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.279+0000 7feb0b5dd700 1 -- 172.21.15.12:0/4008125634 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feaf4004520 con 0x55fe6955c420 2023-10-05T19:40:22.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.279+0000 7feb0b5dd700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955eee0 con 0x55fe6955c420 2023-10-05T19:40:22.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.279+0000 7feb0b5dd700 1 -- 172.21.15.12:0/4008125634 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feaf4004520 con 0x55fe6955c420 2023-10-05T19:40:22.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.279+0000 7feb0b5dd700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955eee0 con 0x55fe695248c0 2023-10-05T19:40:22.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.279+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feb04014040 con 0x55fe695248c0 2023-10-05T19:40:22.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.279+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955eee0 con 0x55fe695248c0 2023-10-05T19:40:22.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.283+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feb04014040 con 0x55fe695248c0 2023-10-05T19:40:22.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.283+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955eee0 con 0x7feafc007030 2023-10-05T19:40:22.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.283+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feb04014040 con 0x7feafc007030 2023-10-05T19:40:22.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.283+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955eee0 con 0x55fe6955c420 2023-10-05T19:40:22.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.283+0000 7feb0b5dd700 1 -- 172.21.15.12:0/4008125634 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feaf4004520 con 0x55fe6955c420 2023-10-05T19:40:22.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.283+0000 7feb0b5dd700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955eee0 con 0x55fe69557c80 2023-10-05T19:40:22.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.283+0000 7feb0bdde700 1 -- 172.21.15.12:0/4008125634 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feafc01c7f0 con 0x55fe69557c80 2023-10-05T19:40:22.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.283+0000 7feb0bdde700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955eee0 con 0x55fe695248c0 2023-10-05T19:40:22.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.287+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feb04014040 con 0x55fe695248c0 2023-10-05T19:40:22.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.287+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955eee0 con 0x7feafc007030 2023-10-05T19:40:22.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.287+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feb04014040 con 0x7feafc007030 2023-10-05T19:40:22.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.287+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955eee0 con 0x55fe6955c420 2023-10-05T19:40:22.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.287+0000 7feb0b5dd700 1 -- 172.21.15.12:0/4008125634 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feaf4004520 con 0x55fe6955c420 2023-10-05T19:40:22.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.287+0000 7feb0b5dd700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955eee0 con 0x7feafc007030 2023-10-05T19:40:22.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.287+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feb04014040 con 0x7feafc007030 2023-10-05T19:40:22.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.287+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955eee0 con 0x55fe69557c80 2023-10-05T19:40:22.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.291+0000 7feb0bdde700 1 -- 172.21.15.12:0/4008125634 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feafc01c7f0 con 0x55fe69557c80 2023-10-05T19:40:22.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.291+0000 7feb0bdde700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955eee0 con 0x55fe6955c420 2023-10-05T19:40:22.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.291+0000 7feb0b5dd700 1 -- 172.21.15.12:0/4008125634 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feaf4004520 con 0x55fe6955c420 2023-10-05T19:40:22.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.291+0000 7feb0b5dd700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955eee0 con 0x7feafc007030 2023-10-05T19:40:22.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.291+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feb04014040 con 0x7feafc007030 2023-10-05T19:40:22.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.291+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955eee0 con 0x55fe695248c0 2023-10-05T19:40:22.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.291+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7feb04014040 con 0x55fe695248c0 2023-10-05T19:40:22.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.291+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955eee0 con 0x55fe69557c80 2023-10-05T19:40:22.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.291+0000 7feb0bdde700 1 -- 172.21.15.12:0/4008125634 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7feafc01c7f0 con 0x55fe69557c80 2023-10-05T19:40:22.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.291+0000 7feb0e2a9e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:22.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.295+0000 7feb0e2a9e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:22.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.295+0000 7feb0e2a9e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:22.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.295+0000 7feb0e2a9e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:22.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.295+0000 7feb0e2a9e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.295+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955b1a0 con 0x55fe695248c0 2023-10-05T19:40:22.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.295+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7feb04014040 con 0x55fe695248c0 2023-10-05T19:40:22.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.295+0000 7feb0e2a9e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:22.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.295+0000 7feb0e2a9e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.295+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955add0 con 0x55fe695248c0 2023-10-05T19:40:22.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.295+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7feb04014040 con 0x55fe695248c0 2023-10-05T19:40:22.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.295+0000 7feb0e2a9e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:22.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.295+0000 7feb0e2a9e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.295+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955f350 con 0x55fe6955c420 2023-10-05T19:40:22.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.295+0000 7feb0b5dd700 1 -- 172.21.15.12:0/4008125634 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7feaf4004520 con 0x55fe6955c420 2023-10-05T19:40:22.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.295+0000 7feb0e2a9e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:22.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.295+0000 7feb0e2a9e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.295+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955f700 con 0x55fe6955c420 2023-10-05T19:40:22.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.295+0000 7feb0b5dd700 1 -- 172.21.15.12:0/4008125634 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7feaf4004520 con 0x55fe6955c420 2023-10-05T19:40:22.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.295+0000 7feb0e2a9e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:22.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.295+0000 7feb0e2a9e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.295+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955fab0 con 0x55fe695248c0 2023-10-05T19:40:22.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.295+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7feb04014040 con 0x55fe695248c0 2023-10-05T19:40:22.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.295+0000 7feb0e2a9e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:22.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.295+0000 7feb0e2a9e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.295+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955b690 con 0x55fe695248c0 2023-10-05T19:40:22.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.299+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7feb04014040 con 0x55fe695248c0 2023-10-05T19:40:22.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.299+0000 7feb0e2a9e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:22.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.299+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955b660 con 0x55fe6955c420 2023-10-05T19:40:22.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.299+0000 7feb0b5dd700 1 -- 172.21.15.12:0/4008125634 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7feaf4004520 con 0x55fe6955c420 2023-10-05T19:40:22.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.299+0000 7feb0b5dd700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955b660 con 0x55fe69557c80 2023-10-05T19:40:22.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.303+0000 7feb0bdde700 1 -- 172.21.15.12:0/4008125634 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feafc04f040 con 0x55fe69557c80 2023-10-05T19:40:22.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.303+0000 7feb0bdde700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955b660 con 0x7feafc007030 2023-10-05T19:40:22.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.303+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feb04014040 con 0x7feafc007030 2023-10-05T19:40:22.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.303+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955fab0 con 0x55fe695248c0 2023-10-05T19:40:22.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.303+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feb04014040 con 0x55fe695248c0 2023-10-05T19:40:22.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.303+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955fab0 con 0x55fe695248c0 2023-10-05T19:40:22.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.303+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feb04014040 con 0x55fe695248c0 2023-10-05T19:40:22.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.303+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955fab0 con 0x7feafc007030 2023-10-05T19:40:22.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.303+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feb04014040 con 0x7feafc007030 2023-10-05T19:40:22.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.303+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955fab0 con 0x7feafc007030 2023-10-05T19:40:22.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.307+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feb04014040 con 0x7feafc007030 2023-10-05T19:40:22.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.307+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955fab0 con 0x55fe69557c80 2023-10-05T19:40:22.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.307+0000 7feb0bdde700 1 -- 172.21.15.12:0/4008125634 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feafc04f040 con 0x55fe69557c80 2023-10-05T19:40:22.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.307+0000 7feb0bdde700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955fab0 con 0x55fe6955c420 2023-10-05T19:40:22.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.307+0000 7feb0b5dd700 1 -- 172.21.15.12:0/4008125634 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feaf402d040 con 0x55fe6955c420 2023-10-05T19:40:22.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.307+0000 7feb0b5dd700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955fab0 con 0x55fe695248c0 2023-10-05T19:40:22.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.307+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feb04014040 con 0x55fe695248c0 2023-10-05T19:40:22.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.307+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955fab0 con 0x7feafc007030 2023-10-05T19:40:22.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.311+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feb04014040 con 0x7feafc007030 2023-10-05T19:40:22.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.311+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955fab0 con 0x55fe695248c0 2023-10-05T19:40:22.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.311+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feb04014040 con 0x55fe695248c0 2023-10-05T19:40:22.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.311+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955fab0 con 0x55fe69557c80 2023-10-05T19:40:22.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.311+0000 7feb0bdde700 1 -- 172.21.15.12:0/4008125634 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feafc04f040 con 0x55fe69557c80 2023-10-05T19:40:22.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.311+0000 7feb0bdde700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955fab0 con 0x55fe6955c420 2023-10-05T19:40:22.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.311+0000 7feb0b5dd700 1 -- 172.21.15.12:0/4008125634 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feaf402d040 con 0x55fe6955c420 2023-10-05T19:40:22.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.311+0000 7feb0b5dd700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955fab0 con 0x55fe6955c420 2023-10-05T19:40:22.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.315+0000 7feb0b5dd700 1 -- 172.21.15.12:0/4008125634 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feaf402d040 con 0x55fe6955c420 2023-10-05T19:40:22.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.315+0000 7feb0b5dd700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955fab0 con 0x7feafc007030 2023-10-05T19:40:22.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.315+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feb0402e040 con 0x7feafc007030 2023-10-05T19:40:22.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.315+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955fab0 con 0x55fe6955c420 2023-10-05T19:40:22.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.315+0000 7feb0b5dd700 1 -- 172.21.15.12:0/4008125634 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feaf402d040 con 0x55fe6955c420 2023-10-05T19:40:22.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.315+0000 7feb0b5dd700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955fab0 con 0x55fe6955c420 2023-10-05T19:40:22.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.315+0000 7feb0b5dd700 1 -- 172.21.15.12:0/4008125634 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feaf402d040 con 0x55fe6955c420 2023-10-05T19:40:22.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.315+0000 7feb0b5dd700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955fab0 con 0x55fe695248c0 2023-10-05T19:40:22.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.319+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feb0402e040 con 0x55fe695248c0 2023-10-05T19:40:22.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.319+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955fab0 con 0x55fe695248c0 2023-10-05T19:40:22.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.319+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feb0402e040 con 0x55fe695248c0 2023-10-05T19:40:22.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.319+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955fab0 con 0x7feafc007030 2023-10-05T19:40:22.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.319+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feb0402e040 con 0x7feafc007030 2023-10-05T19:40:22.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.319+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955fab0 con 0x55fe6955c420 2023-10-05T19:40:22.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.319+0000 7feb0b5dd700 1 -- 172.21.15.12:0/4008125634 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feaf402d040 con 0x55fe6955c420 2023-10-05T19:40:22.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.319+0000 7feb0b5dd700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955fab0 con 0x55fe69557c80 2023-10-05T19:40:22.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.323+0000 7feb0bdde700 1 -- 172.21.15.12:0/4008125634 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feafc04f040 con 0x55fe69557c80 2023-10-05T19:40:22.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.323+0000 7feb0bdde700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7feafc053040 con 0x55fe695248c0 2023-10-05T19:40:22.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.323+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feb0402e040 con 0x55fe695248c0 2023-10-05T19:40:22.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.323+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7feafc053040 con 0x7feafc007030 2023-10-05T19:40:22.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.323+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feb0402e040 con 0x7feafc007030 2023-10-05T19:40:22.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.323+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7feafc053040 con 0x55fe6955c420 2023-10-05T19:40:22.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.323+0000 7feb0b5dd700 1 -- 172.21.15.12:0/4008125634 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feaf402d040 con 0x55fe6955c420 2023-10-05T19:40:22.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.323+0000 7feb0b5dd700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7feafc053040 con 0x7feafc007030 2023-10-05T19:40:22.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.323+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feb0402e040 con 0x7feafc007030 2023-10-05T19:40:22.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.323+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7feafc053040 con 0x55fe69557c80 2023-10-05T19:40:22.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.327+0000 7feb0bdde700 1 -- 172.21.15.12:0/4008125634 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feafc04f040 con 0x55fe69557c80 2023-10-05T19:40:22.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.327+0000 7feb0bdde700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7feafc053040 con 0x55fe6955c420 2023-10-05T19:40:22.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.327+0000 7feb0b5dd700 1 -- 172.21.15.12:0/4008125634 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feaf402d040 con 0x55fe6955c420 2023-10-05T19:40:22.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.327+0000 7feb0b5dd700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7feafc053040 con 0x7feafc007030 2023-10-05T19:40:22.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.327+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feb0402e040 con 0x7feafc007030 2023-10-05T19:40:22.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.327+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7feafc053040 con 0x55fe695248c0 2023-10-05T19:40:22.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.327+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7feb0402e040 con 0x55fe695248c0 2023-10-05T19:40:22.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.327+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7feafc053040 con 0x55fe69557c80 2023-10-05T19:40:22.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.331+0000 7feb0bdde700 1 -- 172.21.15.12:0/4008125634 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7feafc04f040 con 0x55fe69557c80 2023-10-05T19:40:22.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.331+0000 7feb0e2a9e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:22.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.331+0000 7feb0e2a9e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:22.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.331+0000 7feb0e2a9e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:22.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.331+0000 7feb0e2a9e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:22.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.331+0000 7feb0e2a9e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.331+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6955b660 con 0x55fe695248c0 2023-10-05T19:40:22.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.331+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7feb0402e040 con 0x55fe695248c0 2023-10-05T19:40:22.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.331+0000 7feb0e2a9e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:22.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.331+0000 7feb0e2a9e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.331+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe695603f0 con 0x55fe69557c80 2023-10-05T19:40:22.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.331+0000 7feb0bdde700 1 -- 172.21.15.12:0/4008125634 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7feafc04f040 con 0x55fe69557c80 2023-10-05T19:40:22.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.331+0000 7feb0e2a9e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:22.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.331+0000 7feb0e2a9e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.331+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe69560c90 con 0x55fe695248c0 2023-10-05T19:40:22.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.331+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7feb0402e040 con 0x55fe695248c0 2023-10-05T19:40:22.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.331+0000 7feb0e2a9e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:22.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.331+0000 7feb0e2a9e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.331+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe69560c90 con 0x55fe6955c420 2023-10-05T19:40:22.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.335+0000 7feb0b5dd700 1 -- 172.21.15.12:0/4008125634 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7feaf402d040 con 0x55fe6955c420 2023-10-05T19:40:22.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.335+0000 7feb0e2a9e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:22.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.335+0000 7feb0e2a9e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.335+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe69561530 con 0x7feafc007030 2023-10-05T19:40:22.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.335+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7feb0402e040 con 0x7feafc007030 2023-10-05T19:40:22.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.335+0000 7feb0e2a9e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:22.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.335+0000 7feb0e2a9e40 20 zone default found 2023-10-05T19:40:22.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.335+0000 7feb0e2a9e40 4 Realm: () 2023-10-05T19:40:22.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.335+0000 7feb0e2a9e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:22.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.335+0000 7feb0e2a9e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:22.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.335+0000 7feb0e2a9e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:22.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.335+0000 7feb0e2a9e40 20 zonegroup default 2023-10-05T19:40:22.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.335+0000 7feb0e2a9e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.335+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe69561530 con 0x7feafc007030 2023-10-05T19:40:22.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.335+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7feb0402e040 con 0x7feafc007030 2023-10-05T19:40:22.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.335+0000 7feb0e2a9e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:22.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.335+0000 7feb0e2a9e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.335+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe69562eb0 con 0x55fe6955c420 2023-10-05T19:40:22.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.335+0000 7feb0b5dd700 1 -- 172.21.15.12:0/4008125634 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7feaf402d040 con 0x55fe6955c420 2023-10-05T19:40:22.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.335+0000 7feb0e2a9e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:22.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.335+0000 7feb0e2a9e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.335+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe69563260 con 0x55fe6955c420 2023-10-05T19:40:22.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.335+0000 7feb0b5dd700 1 -- 172.21.15.12:0/4008125634 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7feaf403c040 con 0x55fe6955c420 2023-10-05T19:40:22.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.335+0000 7feb0e2a9e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:22.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.335+0000 7feb0e2a9e40 20 started sync module instance, tier type = 2023-10-05T19:40:22.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.335+0000 7feb0e2a9e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:22.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.335+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe69562570 con 0x55fe69557c80 2023-10-05T19:40:22.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.339+0000 7feb0bdde700 1 -- 172.21.15.12:0/4008125634 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'78 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feafc04f040 con 0x55fe69557c80 2023-10-05T19:40:22.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.339+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94551177306240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe695629c0 con 0x55fe69557c80 2023-10-05T19:40:22.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.339+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe69563b00 con 0x55fe695248c0 2023-10-05T19:40:22.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.343+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'78 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feb0402e040 con 0x55fe695248c0 2023-10-05T19:40:22.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.343+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94551177311760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe695629c0 con 0x55fe695248c0 2023-10-05T19:40:22.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.343+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe69564730 con 0x55fe6955c420 2023-10-05T19:40:22.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.343+0000 7feb0bdde700 1 -- 172.21.15.12:0/4008125634 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94551177306240] v28'79 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feafc04f040 con 0x55fe69557c80 2023-10-05T19:40:22.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.347+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94551177311760] v28'79 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feb0402e040 con 0x55fe695248c0 2023-10-05T19:40:22.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.347+0000 7feb0b5dd700 1 -- 172.21.15.12:0/4008125634 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'78 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feaf403c040 con 0x55fe6955c420 2023-10-05T19:40:22.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.347+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94551177312864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe695629c0 con 0x55fe6955c420 2023-10-05T19:40:22.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.347+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe69565020 con 0x55fe69557c80 2023-10-05T19:40:22.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.351+0000 7feb0b5dd700 1 -- 172.21.15.12:0/4008125634 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94551177312864] v28'79 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feaf403c040 con 0x55fe6955c420 2023-10-05T19:40:22.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.351+0000 7feb0bdde700 1 -- 172.21.15.12:0/4008125634 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'78 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feafc04f040 con 0x55fe69557c80 2023-10-05T19:40:22.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.351+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94551177307344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe69565020 con 0x55fe69557c80 2023-10-05T19:40:22.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.351+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe69565a70 con 0x55fe695248c0 2023-10-05T19:40:22.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.351+0000 7feb0bdde700 1 -- 172.21.15.12:0/4008125634 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94551177307344] v28'79 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feafc04f040 con 0x55fe69557c80 2023-10-05T19:40:22.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.351+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'78 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feb0402e040 con 0x55fe695248c0 2023-10-05T19:40:22.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.351+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94551177319744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe69565020 con 0x55fe695248c0 2023-10-05T19:40:22.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.351+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe695664d0 con 0x7feafc007030 2023-10-05T19:40:22.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.355+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94551177319744] v28'79 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feb0402e040 con 0x55fe695248c0 2023-10-05T19:40:22.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.355+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'78 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feb0402e040 con 0x7feafc007030 2023-10-05T19:40:22.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.355+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94551177317440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe695664d0 con 0x7feafc007030 2023-10-05T19:40:22.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.355+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe69566db0 con 0x7feafc007030 2023-10-05T19:40:22.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.359+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(95 notify.5 [watch watch cookie 94551177317440] v28'79 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feb0402e040 con 0x7feafc007030 2023-10-05T19:40:22.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.359+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(96 notify.6 [create] v28'78 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feb0402e040 con 0x7feafc007030 2023-10-05T19:40:22.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.359+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94551177322464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe695669a0 con 0x7feafc007030 2023-10-05T19:40:22.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.359+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe695677c0 con 0x7feafc007030 2023-10-05T19:40:22.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.359+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(97 notify.6 [watch watch cookie 94551177322464] v28'79 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feb0402e040 con 0x7feafc007030 2023-10-05T19:40:22.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.359+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(98 notify.7 [create] v28'78 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feb0402e040 con 0x7feafc007030 2023-10-05T19:40:22.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.359+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94551177323568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe69567780 con 0x7feafc007030 2023-10-05T19:40:22.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.359+0000 7feb0e2a9e40 20 add_watcher() i=0 2023-10-05T19:40:22.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.359+0000 7feb0e2a9e40 20 add_watcher() i=1 2023-10-05T19:40:22.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.359+0000 7feb0e2a9e40 20 add_watcher() i=2 2023-10-05T19:40:22.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.359+0000 7feb0e2a9e40 20 add_watcher() i=3 2023-10-05T19:40:22.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.359+0000 7feb0e2a9e40 20 add_watcher() i=4 2023-10-05T19:40:22.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.359+0000 7feb0e2a9e40 20 add_watcher() i=5 2023-10-05T19:40:22.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.359+0000 7feb0e2a9e40 20 add_watcher() i=6 2023-10-05T19:40:22.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.363+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94551177323568] v28'79 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feb0402e040 con 0x7feafc007030 2023-10-05T19:40:22.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.363+0000 7feb0e2a9e40 20 add_watcher() i=7 2023-10-05T19:40:22.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.363+0000 7feb0e2a9e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:22.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.363+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe6956ab90 con 0x55fe695248c0 2023-10-05T19:40:22.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.363+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7feb0402e040 con 0x55fe695248c0 2023-10-05T19:40:22.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.363+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94551177327712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe69568050 con 0x55fe695248c0 2023-10-05T19:40:22.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.367+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94551177327712] v28'86 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7feb0402e040 con 0x55fe695248c0 2023-10-05T19:40:22.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.367+0000 7fea57fff700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:22.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.367+0000 7feb0e2a9e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:22.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.367+0000 7feb0e2a9e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:22.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.367+0000 7feb0e2a9e40 1 Processor -- start 2023-10-05T19:40:22.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.367+0000 7feb0e2a9e40 1 -- start start 2023-10-05T19:40:22.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.367+0000 7feb0e2a9e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fe69575260 0x55fe69575630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.367+0000 7feb0e2a9e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe69575b70 con 0x55fe69575260 2023-10-05T19:40:22.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.367+0000 7feb0b5dd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fe69575260 0x55fe69575630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.367+0000 7feb0b5dd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fe69575260 0x55fe69575630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37476/0 (socket says 172.21.15.12:37476) 2023-10-05T19:40:22.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.367+0000 7feb0b5dd700 1 -- 172.21.15.12:0/2167837420 learned_addr learned my addr 172.21.15.12:0/2167837420 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:22.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.367+0000 7feb0b5dd700 1 -- 172.21.15.12:0/2167837420 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe69575cb0 con 0x55fe69575260 2023-10-05T19:40:22.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.367+0000 7feb0b5dd700 1 --2- 172.21.15.12:0/2167837420 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fe69575260 0x55fe69575630 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7feb040058f0 tx=0x7feaf4027040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf31a65b0cb4490 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.367+0000 7fea577fe700 1 -- 172.21.15.12:0/2167837420 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feaf4039040 con 0x55fe69575260 2023-10-05T19:40:22.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.367+0000 7fea577fe700 1 -- 172.21.15.12:0/2167837420 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf403a040 con 0x55fe69575260 2023-10-05T19:40:22.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.367+0000 7fea577fe700 1 -- 172.21.15.12:0/2167837420 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feaf402a080 con 0x55fe69575260 2023-10-05T19:40:22.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.371+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/2167837420 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fe69575260 msgr2=0x55fe69575630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.371+0000 7feb0e2a9e40 1 --2- 172.21.15.12:0/2167837420 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fe69575260 0x55fe69575630 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7feb040058f0 tx=0x7feaf4027040 comp rx=0 tx=0).stop 2023-10-05T19:40:22.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.371+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/2167837420 shutdown_connections 2023-10-05T19:40:22.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.371+0000 7feb0e2a9e40 1 --2- 172.21.15.12:0/2167837420 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fe69575260 0x55fe69575630 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:22.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.371+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/2167837420 >> 172.21.15.12:0/2167837420 conn(0x55fe69570620 msgr2=0x55fe6956e770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:22.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.371+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/2167837420 shutdown_connections 2023-10-05T19:40:22.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.371+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/2167837420 wait complete. 2023-10-05T19:40:22.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.371+0000 7feb0e2a9e40 1 Processor -- start 2023-10-05T19:40:22.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.371+0000 7feb0e2a9e40 1 -- start start 2023-10-05T19:40:22.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.371+0000 7feb0e2a9e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fe69575260 0x55fe6957ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.371+0000 7feb0e2a9e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe6957be00 con 0x55fe69575260 2023-10-05T19:40:22.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.371+0000 7feb0b5dd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fe69575260 0x55fe6957ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.371+0000 7feb0b5dd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fe69575260 0x55fe6957ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:37492/0 (socket says 172.21.15.12:37492) 2023-10-05T19:40:22.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.371+0000 7feb0b5dd700 1 -- 172.21.15.12:0/4067100010 learned_addr learned my addr 172.21.15.12:0/4067100010 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:22.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.371+0000 7feb0b5dd700 1 -- 172.21.15.12:0/4067100010 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaec001e10 con 0x55fe69575260 2023-10-05T19:40:22.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.371+0000 7feb0b5dd700 1 --2- 172.21.15.12:0/4067100010 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fe69575260 0x55fe6957ae00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7feaf4042040 tx=0x7feaf403f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.371+0000 7fea55ffb700 1 -- 172.21.15.12:0/4067100010 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feaf4039040 con 0x55fe69575260 2023-10-05T19:40:22.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.371+0000 7fea55ffb700 1 -- 172.21.15.12:0/4067100010 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf402b040 con 0x55fe69575260 2023-10-05T19:40:22.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.371+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4067100010 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55fe6957b400 con 0x55fe69575260 2023-10-05T19:40:22.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.371+0000 7fea55ffb700 1 -- 172.21.15.12:0/4067100010 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feaf401a040 con 0x55fe69575260 2023-10-05T19:40:22.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.375+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4067100010 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe6957b7e0 con 0x55fe69575260 2023-10-05T19:40:22.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.375+0000 7fea55ffb700 1 -- 172.21.15.12:0/4067100010 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7feaf404b7b0 con 0x55fe69575260 2023-10-05T19:40:22.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.375+0000 7fea55ffb700 1 --2- 172.21.15.12:0/4067100010 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fea48030c00 0x7fea480330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.375+0000 7fea55ffb700 1 -- 172.21.15.12:0/4067100010 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7feaf4028040 con 0x55fe69575260 2023-10-05T19:40:22.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.375+0000 7feb0c5df700 1 --2- 172.21.15.12:0/4067100010 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fea48030c00 0x7fea480330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.375+0000 7feb0e2a9e40 5 note: GC not initialized 2023-10-05T19:40:22.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.375+0000 7feb0c5df700 1 --2- 172.21.15.12:0/4067100010 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fea48030c00 0x7fea480330b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x55fe69567a60 tx=0x7feb04005500 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.379+0000 7fea4effd700 20 reqs_thread_entry: start 2023-10-05T19:40:22.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.379+0000 7feb0e2a9e40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:22.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.379+0000 7fea4cff9700 20 reqs_thread_entry: start 2023-10-05T19:40:22.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.379+0000 7feb0e2a9e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:22.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.379+0000 7fea3affd700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:52 2023 2023-10-05T19:40:22.388 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:22.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.379+0000 7fea3a7fc700 20 reqs_thread_entry: start 2023-10-05T19:40:22.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.379+0000 7feb0e2a9e40 20 get_system_obj_state: rctx=0x7fff139360d8 obj=default.rgw.meta:users.uid:foo state=0x55fe693fb0b0 s->prefetch_data=0 2023-10-05T19:40:22.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.379+0000 7feb0e2a9e40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2023-10-05T19:40:22.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.379+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe695e35d0 con 0x55fe695248c0 2023-10-05T19:40:22.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.379+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(102 foo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7feb0402e040 con 0x55fe695248c0 2023-10-05T19:40:22.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.379+0000 7feb0e2a9e40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x16 2023-10-05T19:40:22.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.379+0000 7feb0e2a9e40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:22.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.379+0000 7feb0e2a9e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:22.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.379+0000 7feb0e2a9e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:22.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.379+0000 7feb0e2a9e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:22.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.379+0000 7feb0e2a9e40 10 cache get: name=default.rgw.meta+users.uid+foo : type miss (requested=0x13, cached=0x16) 2023-10-05T19:40:22.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.379+0000 7feb0e2a9e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.379+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe695e6060 con 0x55fe695248c0 2023-10-05T19:40:22.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.383+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(103 foo [call,call out=48b,read 0~513 out=513b,getxattrs out=75b] v0'0 uv10 ondisk = 0) v8 ==== 273+0+636 (crc 0 0 0) 0x7feb0402e040 con 0x55fe695248c0 2023-10-05T19:40:22.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.383+0000 7feb0e2a9e40 20 rados_obj.operate() r=0 bl.length=513 2023-10-05T19:40:22.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.383+0000 7feb0e2a9e40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2023-10-05T19:40:22.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.383+0000 7feb0e2a9e40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:22.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.383+0000 7feb0e2a9e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:22.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.383+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:104 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.set in=58b,writefull 0~357 in=357b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe695e2660 con 0x55fe695248c0 2023-10-05T19:40:22.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.383+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(104 foo [delete,create,call,writefull 0~357] v28'11 uv11 ondisk = 0) v8 ==== 273+0+0 (crc 0 0 0) 0x7feb0402e040 con 0x55fe695248c0 2023-10-05T19:40:22.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.383+0000 7feb0e2a9e40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2023-10-05T19:40:22.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.383+0000 7feb0e2a9e40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:22.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.383+0000 7feb0e2a9e40 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2023-10-05T19:40:22.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.383+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:105 7.d 7:b4812045:::notify.4:head [notify cookie 94551177830768 in=545b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe695e2660 con 0x55fe695248c0 2023-10-05T19:40:22.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.387+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== watch-notify(notify (1) cookie 94551177319744 notify 120259084298 ret 0) v3 ==== 575+0+0 (crc 0 0 0) 0x7feb04040b60 con 0x55fe695248c0 2023-10-05T19:40:22.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.387+0000 7feb095d9700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084298 cookie 94551177319744 notifier 4381 bl.length()=533 2023-10-05T19:40:22.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.387+0000 7feb095d9700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2023-10-05T19:40:22.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.387+0000 7feb095d9700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:22.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.387+0000 7feb095d9700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:106 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7feae4003d90 con 0x55fe695248c0 2023-10-05T19:40:22.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.387+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(105 notify.4 [notify cookie 94551177830768 out=8b] v0'0 uv78 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7feb0402e040 con 0x55fe695248c0 2023-10-05T19:40:22.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.387+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== osd_op_reply(106 notify.4 [notify-ack] v0'0 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feb0402e040 con 0x55fe695248c0 2023-10-05T19:40:22.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.387+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.1 v2:172.21.15.12:6808/29933 39 ==== watch-notify(notify_complete (2) cookie 94551177830768 notify 120259084298 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7feb04040cf0 con 0x55fe695248c0 2023-10-05T19:40:22.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.387+0000 7feb0e2a9e40 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:users.swift:foo:foo2, ofs0, ns] 2023-10-05T19:40:22.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.387+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:107 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94551177830768 in=171b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe695e25c0 con 0x55fe69557c80 2023-10-05T19:40:22.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.387+0000 7feb0bdde700 1 -- 172.21.15.12:0/4008125634 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== watch-notify(notify (1) cookie 94551177306240 notify 120259084293 ret 0) v3 ==== 201+0+0 (crc 0 0 0) 0x7feafc046040 con 0x55fe69557c80 2023-10-05T19:40:22.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.387+0000 7feb09dda700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084293 cookie 94551177306240 notifier 4381 bl.length()=159 2023-10-05T19:40:22.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.387+0000 7feb09dda700 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:108 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fea5c003290 con 0x55fe69557c80 2023-10-05T19:40:22.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.387+0000 7feb0bdde700 1 -- 172.21.15.12:0/4008125634 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(107 notify.0 [notify cookie 94551177830768 out=8b] v0'0 uv78 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7feafc04f040 con 0x55fe69557c80 2023-10-05T19:40:22.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.387+0000 7feb0bdde700 1 -- 172.21.15.12:0/4008125634 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== watch-notify(notify_complete (2) cookie 94551177830768 notify 120259084293 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55fe6954a5b0 con 0x55fe69557c80 2023-10-05T19:40:22.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.387+0000 7feb0bdde700 1 -- 172.21.15.12:0/4008125634 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(108 notify.0 [notify-ack] v0'0 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feafc04f040 con 0x55fe69557c80 2023-10-05T19:40:22.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.387+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:109 8.2 8:45671577:users.swift::foo%3afoo2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe695e4b60 con 0x55fe69557c80 2023-10-05T19:40:22.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.391+0000 7feb0bdde700 1 -- 172.21.15.12:0/4008125634 <== osd.2 v2:172.21.15.12:6800/29934 23 ==== osd_op_reply(109 foo:foo2 [delete] v28'2 uv2 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feafc04f040 con 0x55fe69557c80 2023-10-05T19:40:22.400 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:40:22.400 INFO:teuthology.orchestra.run.smithi012.stdout: "user_id": "foo", 2023-10-05T19:40:22.400 INFO:teuthology.orchestra.run.smithi012.stdout: "display_name": "Foo", 2023-10-05T19:40:22.401 INFO:teuthology.orchestra.run.smithi012.stdout: "email": "foo@foo.com", 2023-10-05T19:40:22.401 INFO:teuthology.orchestra.run.smithi012.stdout: "suspended": 0, 2023-10-05T19:40:22.401 INFO:teuthology.orchestra.run.smithi012.stdout: "max_buckets": 4, 2023-10-05T19:40:22.401 INFO:teuthology.orchestra.run.smithi012.stdout: "subusers": [], 2023-10-05T19:40:22.401 INFO:teuthology.orchestra.run.smithi012.stdout: "keys": [ 2023-10-05T19:40:22.401 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:22.401 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:40:22.401 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2023-10-05T19:40:22.401 INFO:teuthology.orchestra.run.smithi012.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2023-10-05T19:40:22.401 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:22.401 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:22.402 INFO:teuthology.orchestra.run.smithi012.stdout: "swift_keys": [], 2023-10-05T19:40:22.402 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": [], 2023-10-05T19:40:22.402 INFO:teuthology.orchestra.run.smithi012.stdout: "op_mask": "read, write, delete", 2023-10-05T19:40:22.402 INFO:teuthology.orchestra.run.smithi012.stdout: "default_placement": "", 2023-10-05T19:40:22.402 INFO:teuthology.orchestra.run.smithi012.stdout: "default_storage_class": "", 2023-10-05T19:40:22.402 INFO:teuthology.orchestra.run.smithi012.stdout: "placement_tags": [], 2023-10-05T19:40:22.402 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket_quota": { 2023-10-05T19:40:22.402 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:22.402 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:22.402 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:22.402 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:22.402 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:22.403 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:22.403 INFO:teuthology.orchestra.run.smithi012.stdout: "user_quota": { 2023-10-05T19:40:22.403 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:22.403 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:22.403 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:22.403 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:22.403 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:22.403 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:22.403 INFO:teuthology.orchestra.run.smithi012.stdout: "temp_url_keys": [], 2023-10-05T19:40:22.403 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "rgw", 2023-10-05T19:40:22.403 INFO:teuthology.orchestra.run.smithi012.stdout: "mfa_ids": [] 2023-10-05T19:40:22.403 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:40:22.404 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:40:22.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.391+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:110 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94551177306240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe695397d0 con 0x55fe69557c80 2023-10-05T19:40:22.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.395+0000 7feb0bdde700 1 -- 172.21.15.12:0/4008125634 <== osd.2 v2:172.21.15.12:6800/29934 24 ==== osd_op_reply(110 notify.0 [watch unwatch cookie 94551177306240] v28'80 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feafc04f040 con 0x55fe69557c80 2023-10-05T19:40:22.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.395+0000 7feb0e2a9e40 20 remove_watcher() i=0 2023-10-05T19:40:22.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.395+0000 7feb0e2a9e40 2 removed watcher, disabling cache 2023-10-05T19:40:22.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.395+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:111 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94551177311760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe695397d0 con 0x55fe695248c0 2023-10-05T19:40:22.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.399+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.1 v2:172.21.15.12:6808/29933 40 ==== osd_op_reply(111 notify.1 [watch unwatch cookie 94551177311760] v28'80 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feb0402e040 con 0x55fe695248c0 2023-10-05T19:40:22.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.399+0000 7feb0e2a9e40 20 remove_watcher() i=1 2023-10-05T19:40:22.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.399+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:112 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94551177312864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe695397d0 con 0x55fe6955c420 2023-10-05T19:40:22.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.399+0000 7feb0b5dd700 1 -- 172.21.15.12:0/4008125634 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(112 notify.2 [watch unwatch cookie 94551177312864] v28'80 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feaf403c040 con 0x55fe6955c420 2023-10-05T19:40:22.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.399+0000 7feb0e2a9e40 20 remove_watcher() i=2 2023-10-05T19:40:22.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.399+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:113 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94551177307344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe695397d0 con 0x55fe69557c80 2023-10-05T19:40:22.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.403+0000 7feb0bdde700 1 -- 172.21.15.12:0/4008125634 <== osd.2 v2:172.21.15.12:6800/29934 25 ==== osd_op_reply(113 notify.3 [watch unwatch cookie 94551177307344] v28'80 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feafc04f040 con 0x55fe69557c80 2023-10-05T19:40:22.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.403+0000 7feb0e2a9e40 20 remove_watcher() i=3 2023-10-05T19:40:22.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.403+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:114 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94551177319744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe69562570 con 0x55fe695248c0 2023-10-05T19:40:22.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.407+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.1 v2:172.21.15.12:6808/29933 41 ==== osd_op_reply(114 notify.4 [watch unwatch cookie 94551177319744] v28'80 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feb0402e040 con 0x55fe695248c0 2023-10-05T19:40:22.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.407+0000 7feb0e2a9e40 20 remove_watcher() i=4 2023-10-05T19:40:22.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.407+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:115 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94551177317440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe695e2660 con 0x7feafc007030 2023-10-05T19:40:22.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.407+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(115 notify.5 [watch unwatch cookie 94551177317440] v28'80 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feb0402e040 con 0x7feafc007030 2023-10-05T19:40:22.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.411+0000 7feb0e2a9e40 20 remove_watcher() i=5 2023-10-05T19:40:22.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.411+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:116 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94551177322464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe695e2660 con 0x7feafc007030 2023-10-05T19:40:22.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.411+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(116 notify.6 [watch unwatch cookie 94551177322464] v28'80 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feb0402e040 con 0x7feafc007030 2023-10-05T19:40:22.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.411+0000 7feb0e2a9e40 20 remove_watcher() i=6 2023-10-05T19:40:22.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.411+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:117 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94551177323568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe695e2660 con 0x7feafc007030 2023-10-05T19:40:22.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.415+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(117 notify.7 [watch unwatch cookie 94551177323568] v28'80 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feb0402e040 con 0x7feafc007030 2023-10-05T19:40:22.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.415+0000 7feb0e2a9e40 20 remove_watcher() i=7 2023-10-05T19:40:22.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.419+0000 7fea57fff700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:22.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.419+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:118 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94551177327712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55fe695e2660 con 0x55fe695248c0 2023-10-05T19:40:22.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.419+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 <== osd.1 v2:172.21.15.12:6808/29933 42 ==== osd_op_reply(118 data_loggenerations_metadata [watch unwatch cookie 94551177327712] v28'87 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7feb0402e040 con 0x55fe695248c0 2023-10-05T19:40:22.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.419+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7feafc007030 msgr2=0x7feafc00c9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.419+0000 7feb0e2a9e40 1 --2- 172.21.15.12:0/4008125634 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7feafc007030 0x7feafc00c9c0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:22.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.419+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55fe695248c0 msgr2=0x55fe69555680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.419+0000 7feb0e2a9e40 1 --2- 172.21.15.12:0/4008125634 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55fe695248c0 0x55fe69555680 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:22.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.419+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55fe69557c80 msgr2=0x55fe6955a060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.419+0000 7feb0e2a9e40 1 --2- 172.21.15.12:0/4008125634 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55fe69557c80 0x55fe6955a060 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:22.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.419+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55fe6955c420 msgr2=0x55fe6955e830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.419+0000 7feb0e2a9e40 1 --2- 172.21.15.12:0/4008125634 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55fe6955c420 0x55fe6955e830 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:22.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.419+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7feaf0030bf0 msgr2=0x7feaf00330a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.419+0000 7feb0e2a9e40 1 --2- 172.21.15.12:0/4008125634 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7feaf0030bf0 0x7feaf00330a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7feaf40044f0 tx=0x7feaf4006690 comp rx=0 tx=0).stop 2023-10-05T19:40:22.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.419+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fe69523fb0 msgr2=0x55fe69549c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.419+0000 7feb0e2a9e40 1 --2- 172.21.15.12:0/4008125634 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fe69523fb0 0x55fe69549c30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7feafc01f040 tx=0x7feafc01c000 comp rx=0 tx=0).stop 2023-10-05T19:40:22.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.419+0000 7feb0c5df700 1 -- 172.21.15.12:0/4008125634 reap_dead start 2023-10-05T19:40:22.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.423+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 shutdown_connections 2023-10-05T19:40:22.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.423+0000 7feb0e2a9e40 1 --2- 172.21.15.12:0/4008125634 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fe69523fb0 0x55fe69549c30 secure :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7feafc01f040 tx=0x7feafc01c000 comp rx=0 tx=0).stop 2023-10-05T19:40:22.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.423+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 >> 172.21.15.12:0/4008125634 conn(0x55fe69497800 msgr2=0x55fe6953d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:22.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.423+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 shutdown_connections 2023-10-05T19:40:22.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.423+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4008125634 wait complete. 2023-10-05T19:40:22.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.423+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4067100010 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fea48030c00 msgr2=0x7fea480330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.423+0000 7feb0e2a9e40 1 --2- 172.21.15.12:0/4067100010 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fea48030c00 0x7fea480330b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x55fe69567a60 tx=0x7feb04005500 comp rx=0 tx=0).stop 2023-10-05T19:40:22.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.423+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4067100010 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fe69575260 msgr2=0x55fe6957ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.423+0000 7feb0e2a9e40 1 --2- 172.21.15.12:0/4067100010 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fe69575260 0x55fe6957ae00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7feaf4042040 tx=0x7feaf403f000 comp rx=0 tx=0).stop 2023-10-05T19:40:22.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.423+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4067100010 shutdown_connections 2023-10-05T19:40:22.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.423+0000 7feb0e2a9e40 1 --2- 172.21.15.12:0/4067100010 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fea48030c00 0x7fea480330b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:22.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.423+0000 7feb0e2a9e40 1 --2- 172.21.15.12:0/4067100010 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fe69575260 0x55fe6957ae00 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:22.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.423+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4067100010 >> 172.21.15.12:0/4067100010 conn(0x55fe69570620 msgr2=0x55fe6956e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:22.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.423+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4067100010 shutdown_connections 2023-10-05T19:40:22.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.423+0000 7feb0e2a9e40 1 -- 172.21.15.12:0/4067100010 wait complete. 2023-10-05T19:40:22.437 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:40:22.438 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'stats', '--uid', 'foo'] 2023-10-05T19:40:22.438 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'stats', '--uid', 'foo'] 2023-10-05T19:40:22.438 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket stats --uid foo 2023-10-05T19:40:22.478 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:22.478 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:22.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.471+0000 7f1ba4a89e40 1 Processor -- start 2023-10-05T19:40:22.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.471+0000 7f1ba4a89e40 1 -- start start 2023-10-05T19:40:22.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.471+0000 7f1ba4a89e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x561bca48b7e0 0x561bca48a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.471+0000 7f1ba4a89e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x561bca3d7bd0 con 0x561bca48b410 2023-10-05T19:40:22.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.471+0000 7f1ba4a89e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x561bca394ee0 con 0x561bca48b7e0 2023-10-05T19:40:22.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.471+0000 7f1ba1dbd700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x561bca48b7e0 0x561bca48a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.471+0000 7f1ba1dbd700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x561bca48b7e0 0x561bca48a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57372/0 (socket says 172.21.15.12:57372) 2023-10-05T19:40:22.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.471+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3179418621 learned_addr learned my addr 172.21.15.12:0/3179418621 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:22.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.471+0000 7f1ba25be700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x561bca48b410 0x561bca489fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:35450/0 (socket says 172.21.15.12:35450) 2023-10-05T19:40:22.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.471+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3179418621 >> v1:172.21.15.8:6789/0 conn(0x561bca48b410 legacy=0x561bca489fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.471+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3179418621 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bca48acd0 con 0x561bca48b7e0 2023-10-05T19:40:22.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.471+0000 7f1ba1dbd700 1 --2- 172.21.15.12:0/3179418621 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561bca48b7e0 0x561bca48a6d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00a5c0 tx=0x7f1b8c005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1b89c225746938e server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.475+0000 7f1ba15bc700 1 -- 172.21.15.12:0/3179418621 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b8c00e510 con 0x561bca48b7e0 2023-10-05T19:40:22.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.475+0000 7f1ba15bc700 1 -- 172.21.15.12:0/3179418621 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b8c00e670 con 0x561bca48b7e0 2023-10-05T19:40:22.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.475+0000 7f1ba15bc700 1 -- 172.21.15.12:0/3179418621 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b8c00e920 con 0x561bca48b7e0 2023-10-05T19:40:22.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.475+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3179418621 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561bca48b7e0 msgr2=0x561bca48a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.475+0000 7f1ba4a89e40 1 --2- 172.21.15.12:0/3179418621 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561bca48b7e0 0x561bca48a6d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00a5c0 tx=0x7f1b8c005e00 comp rx=0 tx=0).stop 2023-10-05T19:40:22.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.475+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3179418621 shutdown_connections 2023-10-05T19:40:22.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.475+0000 7f1ba4a89e40 1 --2- 172.21.15.12:0/3179418621 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561bca48b7e0 0x561bca48a6d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:22.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.475+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3179418621 >> 172.21.15.12:0/3179418621 conn(0x561bca485720 msgr2=0x561bca487b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:22.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.475+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3179418621 shutdown_connections 2023-10-05T19:40:22.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.475+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3179418621 wait complete. 2023-10-05T19:40:22.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.475+0000 7f1ba4a89e40 1 Processor -- start 2023-10-05T19:40:22.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.475+0000 7f1ba4a89e40 1 -- start start 2023-10-05T19:40:22.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.475+0000 7f1ba4a89e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561bca48b410 0x561bca4ac120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.475+0000 7f1ba4a89e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bca358e30 con 0x561bca48b410 2023-10-05T19:40:22.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.475+0000 7f1ba25be700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561bca48b410 0x561bca4ac120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.475+0000 7f1ba25be700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561bca48b410 0x561bca4ac120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57384/0 (socket says 172.21.15.12:57384) 2023-10-05T19:40:22.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.475+0000 7f1ba25be700 1 -- 172.21.15.12:0/3105968681 learned_addr learned my addr 172.21.15.12:0/3105968681 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:22.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.475+0000 7f1ba25be700 1 -- 172.21.15.12:0/3105968681 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b8c00a180 con 0x561bca48b410 2023-10-05T19:40:22.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.475+0000 7f1ba25be700 1 --2- 172.21.15.12:0/3105968681 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561bca48b410 0x561bca4ac120 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1b94001cb0 tx=0x7f1b94001860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec2772eff33bcdbf server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.475+0000 7f1b9bfff700 1 -- 172.21.15.12:0/3105968681 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b94014070 con 0x561bca48b410 2023-10-05T19:40:22.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.475+0000 7f1b9bfff700 1 -- 172.21.15.12:0/3105968681 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b9400bd40 con 0x561bca48b410 2023-10-05T19:40:22.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.475+0000 7f1b9bfff700 1 -- 172.21.15.12:0/3105968681 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b9400f450 con 0x561bca48b410 2023-10-05T19:40:22.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.479+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3105968681 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561bca48b410 msgr2=0x561bca4ac120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.479+0000 7f1ba4a89e40 1 --2- 172.21.15.12:0/3105968681 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561bca48b410 0x561bca4ac120 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1b94001cb0 tx=0x7f1b94001860 comp rx=0 tx=0).stop 2023-10-05T19:40:22.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.479+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3105968681 shutdown_connections 2023-10-05T19:40:22.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.479+0000 7f1ba4a89e40 1 --2- 172.21.15.12:0/3105968681 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561bca48b410 0x561bca4ac120 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:22.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.479+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3105968681 >> 172.21.15.12:0/3105968681 conn(0x561bca485720 msgr2=0x561bca4a7ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:22.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.479+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3105968681 shutdown_connections 2023-10-05T19:40:22.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.479+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3105968681 wait complete. 2023-10-05T19:40:22.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.479+0000 7f1ba4a89e40 1 Processor -- start 2023-10-05T19:40:22.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.479+0000 7f1ba4a89e40 1 -- start start 2023-10-05T19:40:22.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.479+0000 7f1ba4a89e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561bca48b410 0x561bca4b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.479+0000 7f1ba4a89e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bca32bed0 con 0x561bca48b410 2023-10-05T19:40:22.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.479+0000 7f1ba25be700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561bca48b410 0x561bca4b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.479+0000 7f1ba25be700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561bca48b410 0x561bca4b2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57396/0 (socket says 172.21.15.12:57396) 2023-10-05T19:40:22.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.479+0000 7f1ba25be700 1 -- 172.21.15.12:0/3485427888 learned_addr learned my addr 172.21.15.12:0/3485427888 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:22.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.483+0000 7f1ba25be700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bca3d7bd0 con 0x561bca48b410 2023-10-05T19:40:22.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.483+0000 7f1ba25be700 1 --2- 172.21.15.12:0/3485427888 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561bca48b410 0x561bca4b2ef0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1b94000c00 tx=0x7f1b94016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.483+0000 7f1b9a7fc700 1 -- 172.21.15.12:0/3485427888 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b9400beb0 con 0x561bca48b410 2023-10-05T19:40:22.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.483+0000 7f1b9a7fc700 1 -- 172.21.15.12:0/3485427888 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b9400f450 con 0x561bca48b410 2023-10-05T19:40:22.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.483+0000 7f1b9a7fc700 1 -- 172.21.15.12:0/3485427888 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b9401cc80 con 0x561bca48b410 2023-10-05T19:40:22.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.483+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561bca4b3430 con 0x561bca48b410 2023-10-05T19:40:22.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.483+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561bca4b3870 con 0x561bca48b410 2023-10-05T19:40:22.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.483+0000 7f1b9a7fc700 1 -- 172.21.15.12:0/3485427888 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f1b94023020 con 0x561bca48b410 2023-10-05T19:40:22.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.483+0000 7f1b9a7fc700 1 --2- 172.21.15.12:0/3485427888 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1b84030c00 0x7f1b840330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.483+0000 7f1b9a7fc700 1 -- 172.21.15.12:0/3485427888 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f1b94008030 con 0x561bca48b410 2023-10-05T19:40:22.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.483+0000 7f1ba4a89e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.483+0000 7f1ba4a89e40 1 --2- 172.21.15.12:0/3485427888 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x561bca48b7e0 0x561bca4be9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.483+0000 7f1ba1dbd700 1 --2- 172.21.15.12:0/3485427888 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1b84030c00 0x7f1b840330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.483+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4bf060 con 0x561bca48b7e0 2023-10-05T19:40:22.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.483+0000 7f1ba2dbf700 1 --2- 172.21.15.12:0/3485427888 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x561bca48b7e0 0x561bca4be9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.487+0000 7f1ba1dbd700 1 --2- 172.21.15.12:0/3485427888 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1b84030c00 0x7f1b840330b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c005e40 tx=0x7f1b8c013000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.487+0000 7f1ba2dbf700 1 --2- 172.21.15.12:0/3485427888 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x561bca48b7e0 0x561bca4be9b0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.487+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1b9c00a500 con 0x561bca48b7e0 2023-10-05T19:40:22.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.487+0000 7f1ba4a89e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:22.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.487+0000 7f1ba4a89e40 20 realm 2023-10-05T19:40:22.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.487+0000 7f1ba4a89e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.487+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4bc010 con 0x561bca48b7e0 2023-10-05T19:40:22.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.487+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1b9c00a500 con 0x561bca48b7e0 2023-10-05T19:40:22.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.487+0000 7f1ba4a89e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:22.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.487+0000 7f1ba4a89e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:22.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.487+0000 7f1ba4a89e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.487+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4bfb40 con 0x561bca48b7e0 2023-10-05T19:40:22.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.487+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1b9c00a500 con 0x561bca48b7e0 2023-10-05T19:40:22.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.487+0000 7f1ba4a89e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:22.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.487+0000 7f1ba4a89e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.487+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4bf720 con 0x561bca48b7e0 2023-10-05T19:40:22.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.487+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f1b9c00a500 con 0x561bca48b7e0 2023-10-05T19:40:22.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.487+0000 7f1ba4a89e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:22.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.487+0000 7f1ba4a89e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.487+0000 7f1ba4a89e40 1 --2- 172.21.15.12:0/3485427888 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x561bca4c0ad0 0x561bca4c2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.487+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c3620 con 0x561bca4c0ad0 2023-10-05T19:40:22.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.487+0000 7f1ba25be700 1 --2- 172.21.15.12:0/3485427888 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x561bca4c0ad0 0x561bca4c2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.491+0000 7f1ba25be700 1 --2- 172.21.15.12:0/3485427888 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x561bca4c0ad0 0x561bca4c2eb0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.491+0000 7f1ba25be700 1 -- 172.21.15.12:0/3485427888 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f1b94016070 con 0x561bca4c0ad0 2023-10-05T19:40:22.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.491+0000 7f1ba4a89e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:22.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.491+0000 7f1ba4a89e40 20 searching for the correct realm 2023-10-05T19:40:22.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.491+0000 7f1ba4a89e40 1 --2- 172.21.15.12:0/3485427888 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x561bca4c5450 0x561bca4c7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.491+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c7d70 con 0x561bca4c5450 2023-10-05T19:40:22.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.491+0000 7f1ba1dbd700 1 --2- 172.21.15.12:0/3485427888 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x561bca4c5450 0x561bca4c7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.491+0000 7f1ba1dbd700 1 --2- 172.21.15.12:0/3485427888 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x561bca4c5450 0x561bca4c7830 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.491+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3485427888 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f1b8c017050 con 0x561bca4c5450 2023-10-05T19:40:22.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.491+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c7d70 con 0x561bca4c0ad0 2023-10-05T19:40:22.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.495+0000 7f1ba25be700 1 -- 172.21.15.12:0/3485427888 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b94016070 con 0x561bca4c0ad0 2023-10-05T19:40:22.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.495+0000 7f1ba25be700 1 --2- 172.21.15.12:0/3485427888 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f1b9401a070 0x7f1b940253d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.495+0000 7f1ba25be700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c7d70 con 0x7f1b9401a070 2023-10-05T19:40:22.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.495+0000 7f1ba2dbf700 1 --2- 172.21.15.12:0/3485427888 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f1b9401a070 0x7f1b940253d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.495+0000 7f1ba2dbf700 1 --2- 172.21.15.12:0/3485427888 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f1b9401a070 0x7f1b940253d0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.495+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b9c014040 con 0x7f1b9401a070 2023-10-05T19:40:22.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.495+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c7d70 con 0x561bca48b7e0 2023-10-05T19:40:22.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.495+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b9c014040 con 0x561bca48b7e0 2023-10-05T19:40:22.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.495+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c7d70 con 0x561bca48b7e0 2023-10-05T19:40:22.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.499+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b9c014040 con 0x561bca48b7e0 2023-10-05T19:40:22.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.499+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c7d70 con 0x7f1b9401a070 2023-10-05T19:40:22.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.499+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b9c014040 con 0x7f1b9401a070 2023-10-05T19:40:22.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.499+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c7d70 con 0x7f1b9401a070 2023-10-05T19:40:22.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.499+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b9c014040 con 0x7f1b9401a070 2023-10-05T19:40:22.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.499+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c7d70 con 0x561bca4c0ad0 2023-10-05T19:40:22.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.499+0000 7f1ba25be700 1 -- 172.21.15.12:0/3485427888 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b94016070 con 0x561bca4c0ad0 2023-10-05T19:40:22.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.499+0000 7f1ba25be700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c7d70 con 0x561bca4c5450 2023-10-05T19:40:22.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.499+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3485427888 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b8c017050 con 0x561bca4c5450 2023-10-05T19:40:22.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.499+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c7d70 con 0x561bca48b7e0 2023-10-05T19:40:22.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.499+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b9c014040 con 0x561bca48b7e0 2023-10-05T19:40:22.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.499+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c7d70 con 0x7f1b9401a070 2023-10-05T19:40:22.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.503+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b9c014040 con 0x7f1b9401a070 2023-10-05T19:40:22.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.503+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c7d70 con 0x561bca48b7e0 2023-10-05T19:40:22.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.503+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b9c014040 con 0x561bca48b7e0 2023-10-05T19:40:22.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.503+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c7d70 con 0x561bca4c0ad0 2023-10-05T19:40:22.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.503+0000 7f1ba25be700 1 -- 172.21.15.12:0/3485427888 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b94016070 con 0x561bca4c0ad0 2023-10-05T19:40:22.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.503+0000 7f1ba25be700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c7d70 con 0x561bca4c5450 2023-10-05T19:40:22.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.503+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3485427888 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b8c017050 con 0x561bca4c5450 2023-10-05T19:40:22.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.503+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c7d70 con 0x561bca4c5450 2023-10-05T19:40:22.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.503+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3485427888 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b8c017050 con 0x561bca4c5450 2023-10-05T19:40:22.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.503+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c7d70 con 0x7f1b9401a070 2023-10-05T19:40:22.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.507+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b9c014040 con 0x7f1b9401a070 2023-10-05T19:40:22.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.507+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c7d70 con 0x561bca4c5450 2023-10-05T19:40:22.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.507+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3485427888 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b8c017050 con 0x561bca4c5450 2023-10-05T19:40:22.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.507+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c7d70 con 0x561bca4c5450 2023-10-05T19:40:22.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.507+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3485427888 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b8c017050 con 0x561bca4c5450 2023-10-05T19:40:22.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.507+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c7d70 con 0x561bca48b7e0 2023-10-05T19:40:22.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.507+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b9c014040 con 0x561bca48b7e0 2023-10-05T19:40:22.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.507+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c7d70 con 0x561bca48b7e0 2023-10-05T19:40:22.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.507+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b9c014040 con 0x561bca48b7e0 2023-10-05T19:40:22.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.507+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c7d70 con 0x7f1b9401a070 2023-10-05T19:40:22.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.511+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b9c014040 con 0x7f1b9401a070 2023-10-05T19:40:22.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.511+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c7d70 con 0x561bca4c5450 2023-10-05T19:40:22.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.511+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3485427888 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b8c017050 con 0x561bca4c5450 2023-10-05T19:40:22.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.511+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c7d70 con 0x561bca4c0ad0 2023-10-05T19:40:22.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.511+0000 7f1ba25be700 1 -- 172.21.15.12:0/3485427888 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b94016070 con 0x561bca4c0ad0 2023-10-05T19:40:22.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.511+0000 7f1ba25be700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c7d70 con 0x561bca48b7e0 2023-10-05T19:40:22.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.511+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b9c014040 con 0x561bca48b7e0 2023-10-05T19:40:22.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.511+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c7d70 con 0x7f1b9401a070 2023-10-05T19:40:22.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.511+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b9c014040 con 0x7f1b9401a070 2023-10-05T19:40:22.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.511+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c7d70 con 0x561bca4c5450 2023-10-05T19:40:22.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.515+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3485427888 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b8c017050 con 0x561bca4c5450 2023-10-05T19:40:22.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.515+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c7d70 con 0x7f1b9401a070 2023-10-05T19:40:22.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.515+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b9c014040 con 0x7f1b9401a070 2023-10-05T19:40:22.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.515+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c7d70 con 0x561bca4c0ad0 2023-10-05T19:40:22.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.515+0000 7f1ba25be700 1 -- 172.21.15.12:0/3485427888 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b94016070 con 0x561bca4c0ad0 2023-10-05T19:40:22.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.515+0000 7f1ba25be700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c7d70 con 0x561bca4c5450 2023-10-05T19:40:22.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.515+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3485427888 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b8c017050 con 0x561bca4c5450 2023-10-05T19:40:22.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.515+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c7d70 con 0x7f1b9401a070 2023-10-05T19:40:22.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.519+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b9c014040 con 0x7f1b9401a070 2023-10-05T19:40:22.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.519+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c7d70 con 0x561bca48b7e0 2023-10-05T19:40:22.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.519+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f1b9c014040 con 0x561bca48b7e0 2023-10-05T19:40:22.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.519+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c7d70 con 0x561bca4c0ad0 2023-10-05T19:40:22.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.519+0000 7f1ba25be700 1 -- 172.21.15.12:0/3485427888 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f1b94016070 con 0x561bca4c0ad0 2023-10-05T19:40:22.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.519+0000 7f1ba4a89e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:22.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.519+0000 7f1ba4a89e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:22.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.519+0000 7f1ba4a89e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:22.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.519+0000 7f1ba4a89e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:22.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.519+0000 7f1ba4a89e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.519+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c41a0 con 0x561bca48b7e0 2023-10-05T19:40:22.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.519+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1b9c014040 con 0x561bca48b7e0 2023-10-05T19:40:22.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.519+0000 7f1ba4a89e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:22.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.519+0000 7f1ba4a89e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.519+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c3d70 con 0x561bca48b7e0 2023-10-05T19:40:22.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.519+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f1b9c014040 con 0x561bca48b7e0 2023-10-05T19:40:22.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.519+0000 7f1ba4a89e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:22.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.519+0000 7f1ba4a89e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.519+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c4ad0 con 0x561bca4c5450 2023-10-05T19:40:22.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.523+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3485427888 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f1b8c017050 con 0x561bca4c5450 2023-10-05T19:40:22.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.523+0000 7f1ba4a89e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:22.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.523+0000 7f1ba4a89e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.523+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c86a0 con 0x561bca4c5450 2023-10-05T19:40:22.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.523+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3485427888 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f1b8c017050 con 0x561bca4c5450 2023-10-05T19:40:22.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.523+0000 7f1ba4a89e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:22.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.523+0000 7f1ba4a89e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.523+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c8a50 con 0x561bca48b7e0 2023-10-05T19:40:22.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.523+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1b9c014040 con 0x561bca48b7e0 2023-10-05T19:40:22.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.523+0000 7f1ba4a89e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:22.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.523+0000 7f1ba4a89e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.523+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c8e00 con 0x561bca48b7e0 2023-10-05T19:40:22.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.523+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f1b9c014040 con 0x561bca48b7e0 2023-10-05T19:40:22.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.523+0000 7f1ba4a89e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:22.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.523+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c9e10 con 0x561bca4c5450 2023-10-05T19:40:22.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.527+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3485427888 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f1b8c017050 con 0x561bca4c5450 2023-10-05T19:40:22.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.527+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c9e10 con 0x561bca4c0ad0 2023-10-05T19:40:22.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.527+0000 7f1ba25be700 1 -- 172.21.15.12:0/3485427888 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b94016070 con 0x561bca4c0ad0 2023-10-05T19:40:22.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.527+0000 7f1ba25be700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c9e10 con 0x7f1b9401a070 2023-10-05T19:40:22.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.527+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b9c014040 con 0x7f1b9401a070 2023-10-05T19:40:22.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.527+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c8a50 con 0x561bca48b7e0 2023-10-05T19:40:22.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.527+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b9c014040 con 0x561bca48b7e0 2023-10-05T19:40:22.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.527+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c8a50 con 0x561bca48b7e0 2023-10-05T19:40:22.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.531+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b9c014040 con 0x561bca48b7e0 2023-10-05T19:40:22.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.531+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c8a50 con 0x7f1b9401a070 2023-10-05T19:40:22.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.531+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b9c014040 con 0x7f1b9401a070 2023-10-05T19:40:22.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.531+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c8a50 con 0x7f1b9401a070 2023-10-05T19:40:22.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.531+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b9c014040 con 0x7f1b9401a070 2023-10-05T19:40:22.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.531+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c8a50 con 0x561bca4c0ad0 2023-10-05T19:40:22.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.531+0000 7f1ba25be700 1 -- 172.21.15.12:0/3485427888 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b94016070 con 0x561bca4c0ad0 2023-10-05T19:40:22.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.531+0000 7f1ba25be700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c8a50 con 0x561bca4c5450 2023-10-05T19:40:22.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.531+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3485427888 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b8c017050 con 0x561bca4c5450 2023-10-05T19:40:22.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.531+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c8a50 con 0x561bca48b7e0 2023-10-05T19:40:22.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.535+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b9c014040 con 0x561bca48b7e0 2023-10-05T19:40:22.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.535+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c8a50 con 0x7f1b9401a070 2023-10-05T19:40:22.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.535+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b9c014040 con 0x7f1b9401a070 2023-10-05T19:40:22.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.535+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c8a50 con 0x561bca48b7e0 2023-10-05T19:40:22.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.535+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b9c014040 con 0x561bca48b7e0 2023-10-05T19:40:22.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.535+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c8a50 con 0x561bca4c0ad0 2023-10-05T19:40:22.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.535+0000 7f1ba25be700 1 -- 172.21.15.12:0/3485427888 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b94016070 con 0x561bca4c0ad0 2023-10-05T19:40:22.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.535+0000 7f1ba25be700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c8a50 con 0x561bca4c5450 2023-10-05T19:40:22.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.535+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3485427888 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b8c017050 con 0x561bca4c5450 2023-10-05T19:40:22.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.535+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c8a50 con 0x561bca4c5450 2023-10-05T19:40:22.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.539+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3485427888 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b8c017050 con 0x561bca4c5450 2023-10-05T19:40:22.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.539+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c8a50 con 0x7f1b9401a070 2023-10-05T19:40:22.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.539+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b9c014040 con 0x7f1b9401a070 2023-10-05T19:40:22.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.539+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c8a50 con 0x561bca4c5450 2023-10-05T19:40:22.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.539+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3485427888 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b8c017050 con 0x561bca4c5450 2023-10-05T19:40:22.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.539+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c8a50 con 0x561bca4c5450 2023-10-05T19:40:22.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.539+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3485427888 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b8c017050 con 0x561bca4c5450 2023-10-05T19:40:22.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.539+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c8a50 con 0x561bca48b7e0 2023-10-05T19:40:22.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.539+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b9c014040 con 0x561bca48b7e0 2023-10-05T19:40:22.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.539+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c8a50 con 0x561bca48b7e0 2023-10-05T19:40:22.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.543+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b9c014040 con 0x561bca48b7e0 2023-10-05T19:40:22.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.543+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c8a50 con 0x7f1b9401a070 2023-10-05T19:40:22.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.543+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b9c014040 con 0x7f1b9401a070 2023-10-05T19:40:22.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.543+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c8a50 con 0x561bca4c5450 2023-10-05T19:40:22.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.543+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3485427888 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b8c017050 con 0x561bca4c5450 2023-10-05T19:40:22.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.543+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c8a50 con 0x561bca4c0ad0 2023-10-05T19:40:22.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.543+0000 7f1ba25be700 1 -- 172.21.15.12:0/3485427888 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b94016070 con 0x561bca4c0ad0 2023-10-05T19:40:22.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.543+0000 7f1ba25be700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c8a50 con 0x561bca48b7e0 2023-10-05T19:40:22.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.547+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b9c014040 con 0x561bca48b7e0 2023-10-05T19:40:22.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.547+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c8a50 con 0x7f1b9401a070 2023-10-05T19:40:22.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.547+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b9c014040 con 0x7f1b9401a070 2023-10-05T19:40:22.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.547+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c8a50 con 0x561bca4c5450 2023-10-05T19:40:22.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.547+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3485427888 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b8c017050 con 0x561bca4c5450 2023-10-05T19:40:22.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.547+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c8a50 con 0x7f1b9401a070 2023-10-05T19:40:22.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.547+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b9c014040 con 0x7f1b9401a070 2023-10-05T19:40:22.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.547+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c8a50 con 0x561bca4c0ad0 2023-10-05T19:40:22.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.547+0000 7f1ba25be700 1 -- 172.21.15.12:0/3485427888 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b94016070 con 0x561bca4c0ad0 2023-10-05T19:40:22.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.547+0000 7f1ba25be700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c8a50 con 0x561bca4c5450 2023-10-05T19:40:22.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.551+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3485427888 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b8c017050 con 0x561bca4c5450 2023-10-05T19:40:22.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.551+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c8a50 con 0x7f1b9401a070 2023-10-05T19:40:22.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.551+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1b9c014040 con 0x7f1b9401a070 2023-10-05T19:40:22.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.551+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c8a50 con 0x561bca48b7e0 2023-10-05T19:40:22.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.551+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f1b9c014040 con 0x561bca48b7e0 2023-10-05T19:40:22.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.551+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c8a50 con 0x561bca4c0ad0 2023-10-05T19:40:22.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.551+0000 7f1ba25be700 1 -- 172.21.15.12:0/3485427888 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f1b94016070 con 0x561bca4c0ad0 2023-10-05T19:40:22.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.551+0000 7f1ba4a89e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:22.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.551+0000 7f1ba4a89e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:22.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.551+0000 7f1ba4a89e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:22.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.551+0000 7f1ba4a89e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:22.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.551+0000 7f1ba4a89e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.551+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c8e00 con 0x561bca48b7e0 2023-10-05T19:40:22.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.555+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f1b9c014040 con 0x561bca48b7e0 2023-10-05T19:40:22.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.555+0000 7f1ba4a89e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:22.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.555+0000 7f1ba4a89e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.555+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c91b0 con 0x561bca4c0ad0 2023-10-05T19:40:22.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.555+0000 7f1ba25be700 1 -- 172.21.15.12:0/3485427888 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f1b94016070 con 0x561bca4c0ad0 2023-10-05T19:40:22.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.555+0000 7f1ba4a89e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:22.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.555+0000 7f1ba4a89e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.555+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c8270 con 0x561bca48b7e0 2023-10-05T19:40:22.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.555+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f1b9c014040 con 0x561bca48b7e0 2023-10-05T19:40:22.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.555+0000 7f1ba4a89e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:22.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.555+0000 7f1ba4a89e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.555+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c8240 con 0x561bca4c5450 2023-10-05T19:40:22.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.555+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3485427888 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f1b8c017050 con 0x561bca4c5450 2023-10-05T19:40:22.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.555+0000 7f1ba4a89e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:22.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.555+0000 7f1ba4a89e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.555+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c9ca0 con 0x7f1b9401a070 2023-10-05T19:40:22.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.555+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f1b9c014040 con 0x7f1b9401a070 2023-10-05T19:40:22.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.555+0000 7f1ba4a89e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:22.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.555+0000 7f1ba4a89e40 20 zone default found 2023-10-05T19:40:22.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.555+0000 7f1ba4a89e40 4 Realm: () 2023-10-05T19:40:22.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.555+0000 7f1ba4a89e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:22.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.555+0000 7f1ba4a89e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:22.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.555+0000 7f1ba4a89e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:22.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.555+0000 7f1ba4a89e40 20 zonegroup default 2023-10-05T19:40:22.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.555+0000 7f1ba4a89e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.555+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4c9ca0 con 0x7f1b9401a070 2023-10-05T19:40:22.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.559+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f1b9c014040 con 0x7f1b9401a070 2023-10-05T19:40:22.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.559+0000 7f1ba4a89e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:22.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.559+0000 7f1ba4a89e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.559+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4cb620 con 0x561bca4c5450 2023-10-05T19:40:22.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.559+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3485427888 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f1b8c017050 con 0x561bca4c5450 2023-10-05T19:40:22.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.559+0000 7f1ba4a89e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:22.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.559+0000 7f1ba4a89e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.559+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4cb9d0 con 0x561bca4c5450 2023-10-05T19:40:22.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.559+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3485427888 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f1b8c017050 con 0x561bca4c5450 2023-10-05T19:40:22.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.559+0000 7f1ba4a89e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:22.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.559+0000 7f1ba4a89e40 20 started sync module instance, tier type = 2023-10-05T19:40:22.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.559+0000 7f1ba4a89e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:22.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.559+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4cad30 con 0x561bca4c0ad0 2023-10-05T19:40:22.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.563+0000 7f1ba25be700 1 -- 172.21.15.12:0/3485427888 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'81 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1b94016070 con 0x561bca4c0ad0 2023-10-05T19:40:22.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.563+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94677358129456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4ca960 con 0x561bca4c0ad0 2023-10-05T19:40:22.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.563+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4cc900 con 0x561bca48b7e0 2023-10-05T19:40:22.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.563+0000 7f1ba25be700 1 -- 172.21.15.12:0/3485427888 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94677358129456] v28'82 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1b94016070 con 0x561bca4c0ad0 2023-10-05T19:40:22.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.567+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'81 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1b9c014040 con 0x561bca48b7e0 2023-10-05T19:40:22.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.567+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94677358132688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4ccd00 con 0x561bca48b7e0 2023-10-05T19:40:22.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.567+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4cd5a0 con 0x561bca4c5450 2023-10-05T19:40:22.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.567+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3485427888 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'81 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1b8c017050 con 0x561bca4c5450 2023-10-05T19:40:22.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.567+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94677358136864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4cde90 con 0x561bca4c5450 2023-10-05T19:40:22.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.567+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4ce8a0 con 0x561bca4c0ad0 2023-10-05T19:40:22.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.567+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94677358132688] v28'82 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1b9c014040 con 0x561bca48b7e0 2023-10-05T19:40:22.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.571+0000 7f1ba25be700 1 -- 172.21.15.12:0/3485427888 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'81 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1b94016070 con 0x561bca4c0ad0 2023-10-05T19:40:22.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.571+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94677358134224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4ce4d0 con 0x561bca4c0ad0 2023-10-05T19:40:22.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.571+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4cf810 con 0x561bca48b7e0 2023-10-05T19:40:22.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.571+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3485427888 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94677358136864] v28'82 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1b8c017050 con 0x561bca4c5450 2023-10-05T19:40:22.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.571+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'81 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1b9c014040 con 0x561bca48b7e0 2023-10-05T19:40:22.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.571+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94677358139808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4ce4d0 con 0x561bca48b7e0 2023-10-05T19:40:22.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.575+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4d0270 con 0x7f1b9401a070 2023-10-05T19:40:22.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.575+0000 7f1ba25be700 1 -- 172.21.15.12:0/3485427888 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94677358134224] v28'82 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1b94016070 con 0x561bca4c0ad0 2023-10-05T19:40:22.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.575+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94677358139808] v28'82 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1b9c014040 con 0x561bca48b7e0 2023-10-05T19:40:22.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.575+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'81 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1b9c014040 con 0x7f1b9401a070 2023-10-05T19:40:22.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.575+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94677358144592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4cfd80 con 0x7f1b9401a070 2023-10-05T19:40:22.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.575+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4d0620 con 0x7f1b9401a070 2023-10-05T19:40:22.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.579+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(96 notify.6 [create] v28'81 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1b9c014040 con 0x7f1b9401a070 2023-10-05T19:40:22.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.579+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94677358149552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4d0930 con 0x7f1b9401a070 2023-10-05T19:40:22.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.579+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4d14c0 con 0x7f1b9401a070 2023-10-05T19:40:22.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.579+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(95 notify.5 [watch watch cookie 94677358144592] v28'82 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1b9c014040 con 0x7f1b9401a070 2023-10-05T19:40:22.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.583+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(98 notify.7 [create] v28'81 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1b9c014040 con 0x7f1b9401a070 2023-10-05T19:40:22.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.583+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94677358146272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4d1480 con 0x7f1b9401a070 2023-10-05T19:40:22.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.583+0000 7f1ba4a89e40 20 add_watcher() i=0 2023-10-05T19:40:22.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.583+0000 7f1ba4a89e40 20 add_watcher() i=1 2023-10-05T19:40:22.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.583+0000 7f1ba4a89e40 20 add_watcher() i=2 2023-10-05T19:40:22.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.583+0000 7f1ba4a89e40 20 add_watcher() i=3 2023-10-05T19:40:22.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.583+0000 7f1ba4a89e40 20 add_watcher() i=4 2023-10-05T19:40:22.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.583+0000 7f1ba4a89e40 20 add_watcher() i=5 2023-10-05T19:40:22.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.583+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(97 notify.6 [watch watch cookie 94677358149552] v28'82 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1b9c014040 con 0x7f1b9401a070 2023-10-05T19:40:22.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.583+0000 7f1ba4a89e40 20 add_watcher() i=6 2023-10-05T19:40:22.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.583+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94677358146272] v28'82 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1b9c014040 con 0x7f1b9401a070 2023-10-05T19:40:22.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.583+0000 7f1ba4a89e40 20 add_watcher() i=7 2023-10-05T19:40:22.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.583+0000 7f1ba4a89e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:22.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.583+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4d44b0 con 0x561bca48b7e0 2023-10-05T19:40:22.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.583+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f1b9c014040 con 0x561bca48b7e0 2023-10-05T19:40:22.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.583+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94677358153616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca4d18d0 con 0x561bca48b7e0 2023-10-05T19:40:22.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.587+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94677358153616] v28'88 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f1b9c014040 con 0x561bca48b7e0 2023-10-05T19:40:22.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.587+0000 7f1b73fdf700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:22.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.587+0000 7f1ba4a89e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:22.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.587+0000 7f1ba4a89e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:22.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.587+0000 7f1ba4a89e40 1 Processor -- start 2023-10-05T19:40:22.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.587+0000 7f1ba4a89e40 1 -- start start 2023-10-05T19:40:22.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.587+0000 7f1ba4a89e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561bca4de5e0 0x561bca4de9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.587+0000 7f1ba4a89e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bca4deef0 con 0x561bca4de5e0 2023-10-05T19:40:22.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.587+0000 7f1ba1dbd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561bca4de5e0 0x561bca4de9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.587+0000 7f1ba1dbd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561bca4de5e0 0x561bca4de9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57412/0 (socket says 172.21.15.12:57412) 2023-10-05T19:40:22.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.587+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3744655215 learned_addr learned my addr 172.21.15.12:0/3744655215 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:22.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.587+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3744655215 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bca4df030 con 0x561bca4de5e0 2023-10-05T19:40:22.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.587+0000 7f1ba1dbd700 1 --2- 172.21.15.12:0/3744655215 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561bca4de5e0 0x561bca4de9b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x561bca48aca0 tx=0x7f1b8c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9688e7bd922b5b3b server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.591+0000 7f1b737de700 1 -- 172.21.15.12:0/3744655215 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b8c010c50 con 0x561bca4de5e0 2023-10-05T19:40:22.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.591+0000 7f1b737de700 1 -- 172.21.15.12:0/3744655215 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b8c010db0 con 0x561bca4de5e0 2023-10-05T19:40:22.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.591+0000 7f1b737de700 1 -- 172.21.15.12:0/3744655215 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b8c01c430 con 0x561bca4de5e0 2023-10-05T19:40:22.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.591+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3744655215 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561bca4de5e0 msgr2=0x561bca4de9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.591+0000 7f1ba4a89e40 1 --2- 172.21.15.12:0/3744655215 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561bca4de5e0 0x561bca4de9b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x561bca48aca0 tx=0x7f1b8c010000 comp rx=0 tx=0).stop 2023-10-05T19:40:22.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.591+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3744655215 shutdown_connections 2023-10-05T19:40:22.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.591+0000 7f1ba4a89e40 1 --2- 172.21.15.12:0/3744655215 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561bca4de5e0 0x561bca4de9b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:22.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.591+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3744655215 >> 172.21.15.12:0/3744655215 conn(0x561bca4d99a0 msgr2=0x561bca4d7b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:22.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.591+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3744655215 shutdown_connections 2023-10-05T19:40:22.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.591+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3744655215 wait complete. 2023-10-05T19:40:22.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.591+0000 7f1ba4a89e40 1 Processor -- start 2023-10-05T19:40:22.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.591+0000 7f1ba4a89e40 1 -- start start 2023-10-05T19:40:22.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.591+0000 7f1ba4a89e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561bca4de5e0 0x561bca4e4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.591+0000 7f1ba4a89e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bca4e5050 con 0x561bca4de5e0 2023-10-05T19:40:22.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.591+0000 7f1ba1dbd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561bca4de5e0 0x561bca4e4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.591+0000 7f1ba1dbd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561bca4de5e0 0x561bca4e4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57428/0 (socket says 172.21.15.12:57428) 2023-10-05T19:40:22.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.591+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3357409715 learned_addr learned my addr 172.21.15.12:0/3357409715 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:22.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.591+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3357409715 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bca48acd0 con 0x561bca4de5e0 2023-10-05T19:40:22.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.595+0000 7f1ba1dbd700 1 --2- 172.21.15.12:0/3357409715 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561bca4de5e0 0x561bca4e4050 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00efd0 tx=0x7f1b8c033000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.595+0000 7f1b71fdb700 1 -- 172.21.15.12:0/3357409715 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b8c027400 con 0x561bca4de5e0 2023-10-05T19:40:22.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.595+0000 7f1b71fdb700 1 -- 172.21.15.12:0/3357409715 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b8c027560 con 0x561bca4de5e0 2023-10-05T19:40:22.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.595+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3357409715 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561bca4e4650 con 0x561bca4de5e0 2023-10-05T19:40:22.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.595+0000 7f1b71fdb700 1 -- 172.21.15.12:0/3357409715 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b8c027710 con 0x561bca4de5e0 2023-10-05T19:40:22.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.595+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3357409715 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561bca4e4a90 con 0x561bca4de5e0 2023-10-05T19:40:22.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.595+0000 7f1b71fdb700 1 -- 172.21.15.12:0/3357409715 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f1b8c049030 con 0x561bca4de5e0 2023-10-05T19:40:22.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.595+0000 7f1b71fdb700 1 --2- 172.21.15.12:0/3357409715 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1ae8030c50 0x7f1ae8033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.595+0000 7f1b71fdb700 1 -- 172.21.15.12:0/3357409715 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f1b8c047070 con 0x561bca4de5e0 2023-10-05T19:40:22.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.595+0000 7f1ba2dbf700 1 --2- 172.21.15.12:0/3357409715 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1ae8030c50 0x7f1ae8033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.595+0000 7f1ba4a89e40 5 note: GC not initialized 2023-10-05T19:40:22.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.595+0000 7f1ba2dbf700 1 --2- 172.21.15.12:0/3357409715 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1ae8030c50 0x7f1ae8033100 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x561bca4d0b30 tx=0x7f1b9c00ab40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.599+0000 7f1ae2ffd700 20 reqs_thread_entry: start 2023-10-05T19:40:22.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.599+0000 7f1ae0ff9700 20 reqs_thread_entry: start 2023-10-05T19:40:22.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.599+0000 7f1ba4a89e40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:22.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.599+0000 7f1ba4a89e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:22.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.599+0000 7f1ad2ffd700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:53 2023 2023-10-05T19:40:22.608 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:22.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.599+0000 7f1ba4a89e40 20 get_system_obj_state: rctx=0x7ffc643572b8 obj=default.rgw.meta:users.uid:foo state=0x561bca3640b0 s->prefetch_data=0 2023-10-05T19:40:22.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.599+0000 7f1ad27fc700 20 reqs_thread_entry: start 2023-10-05T19:40:22.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.599+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca54c920 con 0x561bca48b7e0 2023-10-05T19:40:22.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.599+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(102 foo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7f1b9c014040 con 0x561bca48b7e0 2023-10-05T19:40:22.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.599+0000 7f1ba4a89e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:22.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.599+0000 7f1ba4a89e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:22.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.599+0000 7f1ba4a89e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.599+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca54c8f0 con 0x561bca48b7e0 2023-10-05T19:40:22.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.599+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(103 foo [call,call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv11 ondisk = 0) v8 ==== 273+0+480 (crc 0 0 0) 0x7f1b9c014040 con 0x561bca48b7e0 2023-10-05T19:40:22.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.599+0000 7f1ba4a89e40 20 rados_obj.operate() r=0 bl.length=357 2023-10-05T19:40:22.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.599+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:104 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca54c8f0 con 0x561bca4c0ad0 2023-10-05T19:40:22.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.603+0000 7f1ba25be700 1 -- 172.21.15.12:0/3485427888 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(104 foo.buckets [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 155+0+0 (crc 0 0 0) 0x7f1b94016070 con 0x561bca4c0ad0 2023-10-05T19:40:22.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.603+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:105 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94677358129456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca54c8f0 con 0x561bca4c0ad0 2023-10-05T19:40:22.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.603+0000 7f1ba25be700 1 -- 172.21.15.12:0/3485427888 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(105 notify.0 [watch unwatch cookie 94677358129456] v28'83 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1b94016070 con 0x561bca4c0ad0 2023-10-05T19:40:22.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.603+0000 7f1ba4a89e40 20 remove_watcher() i=0 2023-10-05T19:40:22.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.607+0000 7f1ba4a89e40 2 removed watcher, disabling cache 2023-10-05T19:40:22.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.607+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:106 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94677358132688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca54c8f0 con 0x561bca48b7e0 2023-10-05T19:40:22.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.607+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(106 notify.1 [watch unwatch cookie 94677358132688] v28'83 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1b9c014040 con 0x561bca48b7e0 2023-10-05T19:40:22.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.607+0000 7f1ba4a89e40 20 remove_watcher() i=1 2023-10-05T19:40:22.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.607+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:107 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94677358136864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca54c8f0 con 0x561bca4c5450 2023-10-05T19:40:22.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.611+0000 7f1ba1dbd700 1 -- 172.21.15.12:0/3485427888 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(107 notify.2 [watch unwatch cookie 94677358136864] v28'83 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1b8c017050 con 0x561bca4c5450 2023-10-05T19:40:22.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.611+0000 7f1ba4a89e40 20 remove_watcher() i=2 2023-10-05T19:40:22.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.611+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:108 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94677358134224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca54c8f0 con 0x561bca4c0ad0 2023-10-05T19:40:22.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.611+0000 7f1ba25be700 1 -- 172.21.15.12:0/3485427888 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(108 notify.3 [watch unwatch cookie 94677358134224] v28'83 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1b94016070 con 0x561bca4c0ad0 2023-10-05T19:40:22.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.611+0000 7f1ba4a89e40 20 remove_watcher() i=3 2023-10-05T19:40:22.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.611+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:109 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94677358139808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca54c8f0 con 0x561bca48b7e0 2023-10-05T19:40:22.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.615+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(109 notify.4 [watch unwatch cookie 94677358139808] v28'83 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1b9c014040 con 0x561bca48b7e0 2023-10-05T19:40:22.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.615+0000 7f1ba4a89e40 20 remove_watcher() i=4 2023-10-05T19:40:22.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.615+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:110 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94677358144592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca54c8f0 con 0x7f1b9401a070 2023-10-05T19:40:22.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.619+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(110 notify.5 [watch unwatch cookie 94677358144592] v28'83 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1b9c014040 con 0x7f1b9401a070 2023-10-05T19:40:22.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.619+0000 7f1ba4a89e40 20 remove_watcher() i=5 2023-10-05T19:40:22.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.619+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:111 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94677358149552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca54c8f0 con 0x7f1b9401a070 2023-10-05T19:40:22.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.619+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(111 notify.6 [watch unwatch cookie 94677358149552] v28'83 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1b9c014040 con 0x7f1b9401a070 2023-10-05T19:40:22.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.619+0000 7f1ba4a89e40 20 remove_watcher() i=6 2023-10-05T19:40:22.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.619+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:112 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94677358146272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca54c8f0 con 0x7f1b9401a070 2023-10-05T19:40:22.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.623+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(112 notify.7 [watch unwatch cookie 94677358146272] v28'83 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1b9c014040 con 0x7f1b9401a070 2023-10-05T19:40:22.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.623+0000 7f1ba4a89e40 20 remove_watcher() i=7 2023-10-05T19:40:22.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.623+0000 7f1b73fdf700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:22.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.627+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:113 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94677358153616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561bca50bcc0 con 0x561bca48b7e0 2023-10-05T19:40:22.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.627+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(113 data_loggenerations_metadata [watch unwatch cookie 94677358153616] v28'89 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f1b9c014040 con 0x561bca48b7e0 2023-10-05T19:40:22.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.627+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f1b9401a070 msgr2=0x7f1b940253d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.627+0000 7f1ba4a89e40 1 --2- 172.21.15.12:0/3485427888 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f1b9401a070 0x7f1b940253d0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:22.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.627+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x561bca48b7e0 msgr2=0x561bca4be9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.627+0000 7f1ba4a89e40 1 --2- 172.21.15.12:0/3485427888 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x561bca48b7e0 0x561bca4be9b0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:22.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.627+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x561bca4c0ad0 msgr2=0x561bca4c2eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.627+0000 7f1ba4a89e40 1 --2- 172.21.15.12:0/3485427888 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x561bca4c0ad0 0x561bca4c2eb0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:22.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.627+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x561bca4c5450 msgr2=0x561bca4c7830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.627+0000 7f1ba4a89e40 1 --2- 172.21.15.12:0/3485427888 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x561bca4c5450 0x561bca4c7830 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:22.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.627+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1b84030c00 msgr2=0x7f1b840330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.627+0000 7f1ba4a89e40 1 --2- 172.21.15.12:0/3485427888 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1b84030c00 0x7f1b840330b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c005e40 tx=0x7f1b8c013000 comp rx=0 tx=0).stop 2023-10-05T19:40:22.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.627+0000 7f1ba2dbf700 1 -- 172.21.15.12:0/3485427888 reap_dead start 2023-10-05T19:40:22.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.627+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561bca48b410 msgr2=0x561bca4b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.627+0000 7f1ba4a89e40 1 --2- 172.21.15.12:0/3485427888 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561bca48b410 0x561bca4b2ef0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1b94000c00 tx=0x7f1b94016000 comp rx=0 tx=0).stop 2023-10-05T19:40:22.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.627+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 shutdown_connections 2023-10-05T19:40:22.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.627+0000 7f1ba4a89e40 1 --2- 172.21.15.12:0/3485427888 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561bca48b410 0x561bca4b2ef0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:22.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.627+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 >> 172.21.15.12:0/3485427888 conn(0x561bca485720 msgr2=0x561bca4a6760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:22.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.631+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 shutdown_connections 2023-10-05T19:40:22.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.631+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3485427888 wait complete. 2023-10-05T19:40:22.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.631+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3357409715 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1ae8030c50 msgr2=0x7f1ae8033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.631+0000 7f1ba4a89e40 1 --2- 172.21.15.12:0/3357409715 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1ae8030c50 0x7f1ae8033100 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x561bca4d0b30 tx=0x7f1b9c00ab40 comp rx=0 tx=0).stop 2023-10-05T19:40:22.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.631+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3357409715 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561bca4de5e0 msgr2=0x561bca4e4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.631+0000 7f1ba4a89e40 1 --2- 172.21.15.12:0/3357409715 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561bca4de5e0 0x561bca4e4050 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00efd0 tx=0x7f1b8c033000 comp rx=0 tx=0).stop 2023-10-05T19:40:22.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.631+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3357409715 shutdown_connections 2023-10-05T19:40:22.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.631+0000 7f1ba4a89e40 1 --2- 172.21.15.12:0/3357409715 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1ae8030c50 0x7f1ae8033100 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:22.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.631+0000 7f1ba4a89e40 1 --2- 172.21.15.12:0/3357409715 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561bca4de5e0 0x561bca4e4050 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:22.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.631+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3357409715 >> 172.21.15.12:0/3357409715 conn(0x561bca4d99a0 msgr2=0x561bca4d7aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:22.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.631+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3357409715 shutdown_connections 2023-10-05T19:40:22.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.631+0000 7f1ba4a89e40 1 -- 172.21.15.12:0/3357409715 wait complete. 2023-10-05T19:40:22.642 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2023-10-05T19:40:22.645 DEBUG:tasks.util.rgw: json result: [] 2023-10-05T19:40:22.645 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'list', '--uid', 'foo'] 2023-10-05T19:40:22.645 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'list', '--uid', 'foo'] 2023-10-05T19:40:22.645 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket list --uid foo 2023-10-05T19:40:22.679 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:22.679 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:22.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.671+0000 7f116a3b7e40 1 Processor -- start 2023-10-05T19:40:22.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.671+0000 7f116a3b7e40 1 -- start start 2023-10-05T19:40:22.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.671+0000 7f116a3b7e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x5572ea8153d0 0x5572ea8a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.671+0000 7f116a3b7e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5572ea7edbd0 con 0x5572ea8a4110 2023-10-05T19:40:22.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.671+0000 7f116a3b7e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x5572ea7aaee0 con 0x5572ea8153d0 2023-10-05T19:40:22.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.671+0000 7f1167eec700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x5572ea8153d0 0x5572ea8a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.671+0000 7f11676eb700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x5572ea8a4110 0x5572ea8a64f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:35460/0 (socket says 172.21.15.12:35460) 2023-10-05T19:40:22.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.671+0000 7f11676eb700 1 -- 172.21.15.12:0/668355925 learned_addr learned my addr 172.21.15.12:0/668355925 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:22.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.671+0000 7f1167eec700 1 -- 172.21.15.12:0/668355925 >> v1:172.21.15.8:6789/0 conn(0x5572ea8a4110 legacy=0x5572ea8a64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.671+0000 7f1167eec700 1 -- 172.21.15.12:0/668355925 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572ea8a8c20 con 0x5572ea8153d0 2023-10-05T19:40:22.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.675+0000 7f1167eec700 1 --2- 172.21.15.12:0/668355925 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5572ea8153d0 0x5572ea8a3bd0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f115800a5d0 tx=0x7f1158005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d142e1cc36e4101b server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.675+0000 7f1166eea700 1 -- 172.21.15.12:0/668355925 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11580086a0 con 0x5572ea8153d0 2023-10-05T19:40:22.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.675+0000 7f1166eea700 1 -- 172.21.15.12:0/668355925 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1158008800 con 0x5572ea8153d0 2023-10-05T19:40:22.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.675+0000 7f1166eea700 1 -- 172.21.15.12:0/668355925 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1158008ad0 con 0x5572ea8153d0 2023-10-05T19:40:22.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.675+0000 7f116a3b7e40 1 -- 172.21.15.12:0/668355925 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5572ea8153d0 msgr2=0x5572ea8a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.675+0000 7f116a3b7e40 1 --2- 172.21.15.12:0/668355925 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5572ea8153d0 0x5572ea8a3bd0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f115800a5d0 tx=0x7f1158005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:22.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.675+0000 7f116a3b7e40 1 -- 172.21.15.12:0/668355925 shutdown_connections 2023-10-05T19:40:22.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.675+0000 7f116a3b7e40 1 --2- 172.21.15.12:0/668355925 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5572ea8153d0 0x5572ea8a3bd0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:22.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.675+0000 7f116a3b7e40 1 -- 172.21.15.12:0/668355925 >> 172.21.15.12:0/668355925 conn(0x5572ea89b720 msgr2=0x5572ea89db30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:22.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.675+0000 7f116a3b7e40 1 -- 172.21.15.12:0/668355925 shutdown_connections 2023-10-05T19:40:22.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.675+0000 7f116a3b7e40 1 -- 172.21.15.12:0/668355925 wait complete. 2023-10-05T19:40:22.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.675+0000 7f116a3b7e40 1 Processor -- start 2023-10-05T19:40:22.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.675+0000 7f116a3b7e40 1 -- start start 2023-10-05T19:40:22.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.675+0000 7f116a3b7e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5572ea8153d0 0x5572ea8bfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.675+0000 7f116a3b7e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572ea76ee30 con 0x5572ea8153d0 2023-10-05T19:40:22.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.675+0000 7f1167eec700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5572ea8153d0 0x5572ea8bfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.675+0000 7f1167eec700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5572ea8153d0 0x5572ea8bfe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57444/0 (socket says 172.21.15.12:57444) 2023-10-05T19:40:22.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.675+0000 7f1167eec700 1 -- 172.21.15.12:0/2636498486 learned_addr learned my addr 172.21.15.12:0/2636498486 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:22.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.675+0000 7f1167eec700 1 -- 172.21.15.12:0/2636498486 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1158005cf0 con 0x5572ea8153d0 2023-10-05T19:40:22.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.675+0000 7f1167eec700 1 --2- 172.21.15.12:0/2636498486 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5572ea8153d0 0x5572ea8bfe90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1158005a90 tx=0x7f1158015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e50c813d9bcd500 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.679+0000 7f1165ee8700 1 -- 172.21.15.12:0/2636498486 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1158008a40 con 0x5572ea8153d0 2023-10-05T19:40:22.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.679+0000 7f1165ee8700 1 -- 172.21.15.12:0/2636498486 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1158008bc0 con 0x5572ea8153d0 2023-10-05T19:40:22.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.679+0000 7f1165ee8700 1 -- 172.21.15.12:0/2636498486 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1158012450 con 0x5572ea8153d0 2023-10-05T19:40:22.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.679+0000 7f116a3b7e40 1 -- 172.21.15.12:0/2636498486 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5572ea8153d0 msgr2=0x5572ea8bfe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.679+0000 7f116a3b7e40 1 --2- 172.21.15.12:0/2636498486 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5572ea8153d0 0x5572ea8bfe90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1158005a90 tx=0x7f1158015000 comp rx=0 tx=0).stop 2023-10-05T19:40:22.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.679+0000 7f116a3b7e40 1 -- 172.21.15.12:0/2636498486 shutdown_connections 2023-10-05T19:40:22.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.679+0000 7f116a3b7e40 1 --2- 172.21.15.12:0/2636498486 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5572ea8153d0 0x5572ea8bfe90 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:22.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.679+0000 7f116a3b7e40 1 -- 172.21.15.12:0/2636498486 >> 172.21.15.12:0/2636498486 conn(0x5572ea89b720 msgr2=0x5572ea8bba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:22.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.679+0000 7f116a3b7e40 1 -- 172.21.15.12:0/2636498486 shutdown_connections 2023-10-05T19:40:22.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.679+0000 7f116a3b7e40 1 -- 172.21.15.12:0/2636498486 wait complete. 2023-10-05T19:40:22.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.679+0000 7f116a3b7e40 1 Processor -- start 2023-10-05T19:40:22.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.679+0000 7f116a3b7e40 1 -- start start 2023-10-05T19:40:22.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.679+0000 7f116a3b7e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5572ea8153d0 0x5572ea8a1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.679+0000 7f116a3b7e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572ea741ed0 con 0x5572ea8153d0 2023-10-05T19:40:22.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.679+0000 7f1167eec700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5572ea8153d0 0x5572ea8a1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.679+0000 7f1167eec700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5572ea8153d0 0x5572ea8a1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57458/0 (socket says 172.21.15.12:57458) 2023-10-05T19:40:22.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.679+0000 7f1167eec700 1 -- 172.21.15.12:0/3584340816 learned_addr learned my addr 172.21.15.12:0/3584340816 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:22.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.683+0000 7f1167eec700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572ea8a8c20 con 0x5572ea8153d0 2023-10-05T19:40:22.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.683+0000 7f1167eec700 1 --2- 172.21.15.12:0/3584340816 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5572ea8153d0 0x5572ea8a1360 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f115800c010 tx=0x7f115801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.683+0000 7f114ffff700 1 -- 172.21.15.12:0/3584340816 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1158008e50 con 0x5572ea8153d0 2023-10-05T19:40:22.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.683+0000 7f114ffff700 1 -- 172.21.15.12:0/3584340816 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1158025660 con 0x5572ea8153d0 2023-10-05T19:40:22.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.683+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5572ea8a18a0 con 0x5572ea8153d0 2023-10-05T19:40:22.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.683+0000 7f114ffff700 1 -- 172.21.15.12:0/3584340816 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1158025950 con 0x5572ea8153d0 2023-10-05T19:40:22.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.683+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5572ea89fc60 con 0x5572ea8153d0 2023-10-05T19:40:22.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.683+0000 7f114ffff700 1 -- 172.21.15.12:0/3584340816 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f1158019070 con 0x5572ea8153d0 2023-10-05T19:40:22.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.683+0000 7f116a3b7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.683+0000 7f114ffff700 1 --2- 172.21.15.12:0/3584340816 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1154030c00 0x7f11540330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.687+0000 7f114ffff700 1 -- 172.21.15.12:0/3584340816 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f1158046610 con 0x5572ea8153d0 2023-10-05T19:40:22.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.687+0000 7f11676eb700 1 --2- 172.21.15.12:0/3584340816 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1154030c00 0x7f11540330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.687+0000 7f116a3b7e40 1 --2- 172.21.15.12:0/3584340816 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5572ea8a4110 0x5572ea8d47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.687+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8d4e20 con 0x5572ea8a4110 2023-10-05T19:40:22.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.687+0000 7f11686ed700 1 --2- 172.21.15.12:0/3584340816 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5572ea8a4110 0x5572ea8d47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.687+0000 7f11676eb700 1 --2- 172.21.15.12:0/3584340816 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1154030c00 0x7f11540330b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f115c001df0 tx=0x7f115c001310 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.687+0000 7f11686ed700 1 --2- 172.21.15.12:0/3584340816 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5572ea8a4110 0x5572ea8d47b0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.687+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f116000a5a0 con 0x5572ea8a4110 2023-10-05T19:40:22.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.687+0000 7f116a3b7e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:22.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.687+0000 7f116a3b7e40 20 realm 2023-10-05T19:40:22.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.687+0000 7f116a3b7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.687+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8d1e10 con 0x5572ea8a4110 2023-10-05T19:40:22.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.687+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f116000a5a0 con 0x5572ea8a4110 2023-10-05T19:40:22.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.687+0000 7f116a3b7e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:22.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.687+0000 7f116a3b7e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:22.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.687+0000 7f116a3b7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.687+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8d5900 con 0x5572ea8a4110 2023-10-05T19:40:22.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.687+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f116000a5a0 con 0x5572ea8a4110 2023-10-05T19:40:22.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.687+0000 7f116a3b7e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:22.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.691+0000 7f116a3b7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.691+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8d54e0 con 0x5572ea8a4110 2023-10-05T19:40:22.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.691+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f116000a5a0 con 0x5572ea8a4110 2023-10-05T19:40:22.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.691+0000 7f116a3b7e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:22.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.691+0000 7f116a3b7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.691+0000 7f116a3b7e40 1 --2- 172.21.15.12:0/3584340816 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5572ea8d6890 0x5572ea8d8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.691+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8d93e0 con 0x5572ea8d6890 2023-10-05T19:40:22.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.691+0000 7f1167eec700 1 --2- 172.21.15.12:0/3584340816 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5572ea8d6890 0x5572ea8d8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.691+0000 7f1167eec700 1 --2- 172.21.15.12:0/3584340816 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5572ea8d6890 0x5572ea8d8c70 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.691+0000 7f1167eec700 1 -- 172.21.15.12:0/3584340816 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f1158011570 con 0x5572ea8d6890 2023-10-05T19:40:22.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.691+0000 7f116a3b7e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:22.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.691+0000 7f116a3b7e40 20 searching for the correct realm 2023-10-05T19:40:22.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.691+0000 7f116a3b7e40 1 --2- 172.21.15.12:0/3584340816 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5572ea8db210 0x5572ea8dd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.691+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8ddb30 con 0x5572ea8db210 2023-10-05T19:40:22.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.691+0000 7f11676eb700 1 --2- 172.21.15.12:0/3584340816 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5572ea8db210 0x5572ea8dd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.695+0000 7f11676eb700 1 --2- 172.21.15.12:0/3584340816 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5572ea8db210 0x5572ea8dd5f0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.695+0000 7f11676eb700 1 -- 172.21.15.12:0/3584340816 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f115c006070 con 0x5572ea8db210 2023-10-05T19:40:22.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.695+0000 7f11676eb700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8ddb30 con 0x5572ea8d6890 2023-10-05T19:40:22.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.695+0000 7f1167eec700 1 -- 172.21.15.12:0/3584340816 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1158011570 con 0x5572ea8d6890 2023-10-05T19:40:22.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.695+0000 7f1167eec700 1 --2- 172.21.15.12:0/3584340816 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f1158007030 0x7f1158011830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.695+0000 7f1167eec700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8ddb30 con 0x7f1158007030 2023-10-05T19:40:22.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.699+0000 7f11686ed700 1 --2- 172.21.15.12:0/3584340816 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f1158007030 0x7f1158011830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.699+0000 7f11686ed700 1 --2- 172.21.15.12:0/3584340816 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f1158007030 0x7f1158011830 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.699+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f116000a5a0 con 0x7f1158007030 2023-10-05T19:40:22.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.699+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8ddb30 con 0x5572ea8a4110 2023-10-05T19:40:22.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.699+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f116000a5a0 con 0x5572ea8a4110 2023-10-05T19:40:22.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.699+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8ddb30 con 0x5572ea8a4110 2023-10-05T19:40:22.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.699+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f116000a5a0 con 0x5572ea8a4110 2023-10-05T19:40:22.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.699+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8ddb30 con 0x7f1158007030 2023-10-05T19:40:22.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.699+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f116000a5a0 con 0x7f1158007030 2023-10-05T19:40:22.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.699+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8ddb30 con 0x7f1158007030 2023-10-05T19:40:22.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.703+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f116000a5a0 con 0x7f1158007030 2023-10-05T19:40:22.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.703+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8ddb30 con 0x5572ea8d6890 2023-10-05T19:40:22.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.703+0000 7f1167eec700 1 -- 172.21.15.12:0/3584340816 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1158011570 con 0x5572ea8d6890 2023-10-05T19:40:22.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.703+0000 7f1167eec700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8ddb30 con 0x5572ea8db210 2023-10-05T19:40:22.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.703+0000 7f11676eb700 1 -- 172.21.15.12:0/3584340816 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f115c006070 con 0x5572ea8db210 2023-10-05T19:40:22.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.703+0000 7f11676eb700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8ddb30 con 0x5572ea8a4110 2023-10-05T19:40:22.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.703+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f116000a5a0 con 0x5572ea8a4110 2023-10-05T19:40:22.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.703+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8ddb30 con 0x7f1158007030 2023-10-05T19:40:22.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.703+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f116000a5a0 con 0x7f1158007030 2023-10-05T19:40:22.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.703+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8ddb30 con 0x5572ea8a4110 2023-10-05T19:40:22.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.703+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f116000a5a0 con 0x5572ea8a4110 2023-10-05T19:40:22.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.703+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8ddb30 con 0x5572ea8d6890 2023-10-05T19:40:22.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.707+0000 7f1167eec700 1 -- 172.21.15.12:0/3584340816 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1158011570 con 0x5572ea8d6890 2023-10-05T19:40:22.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.707+0000 7f1167eec700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8ddb30 con 0x5572ea8db210 2023-10-05T19:40:22.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.707+0000 7f11676eb700 1 -- 172.21.15.12:0/3584340816 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f115c006070 con 0x5572ea8db210 2023-10-05T19:40:22.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.707+0000 7f11676eb700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8ddb30 con 0x5572ea8db210 2023-10-05T19:40:22.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.707+0000 7f11676eb700 1 -- 172.21.15.12:0/3584340816 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f115c006070 con 0x5572ea8db210 2023-10-05T19:40:22.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.707+0000 7f11676eb700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8ddb30 con 0x7f1158007030 2023-10-05T19:40:22.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.707+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f116000a5a0 con 0x7f1158007030 2023-10-05T19:40:22.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.707+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8ddb30 con 0x5572ea8db210 2023-10-05T19:40:22.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.707+0000 7f11676eb700 1 -- 172.21.15.12:0/3584340816 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f115c006070 con 0x5572ea8db210 2023-10-05T19:40:22.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.707+0000 7f11676eb700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8ddb30 con 0x5572ea8db210 2023-10-05T19:40:22.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.707+0000 7f11676eb700 1 -- 172.21.15.12:0/3584340816 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f115c006070 con 0x5572ea8db210 2023-10-05T19:40:22.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.707+0000 7f11676eb700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8ddb30 con 0x5572ea8a4110 2023-10-05T19:40:22.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.707+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f116000a5a0 con 0x5572ea8a4110 2023-10-05T19:40:22.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.711+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8ddb30 con 0x5572ea8a4110 2023-10-05T19:40:22.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.711+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f116000a5a0 con 0x5572ea8a4110 2023-10-05T19:40:22.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.711+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8ddb30 con 0x7f1158007030 2023-10-05T19:40:22.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.711+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f116000a5a0 con 0x7f1158007030 2023-10-05T19:40:22.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.711+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8ddb30 con 0x5572ea8db210 2023-10-05T19:40:22.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.711+0000 7f11676eb700 1 -- 172.21.15.12:0/3584340816 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f115c006070 con 0x5572ea8db210 2023-10-05T19:40:22.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.711+0000 7f11676eb700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8ddb30 con 0x5572ea8d6890 2023-10-05T19:40:22.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.711+0000 7f1167eec700 1 -- 172.21.15.12:0/3584340816 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1158011570 con 0x5572ea8d6890 2023-10-05T19:40:22.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.711+0000 7f1167eec700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8ddb30 con 0x5572ea8a4110 2023-10-05T19:40:22.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.711+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f116000a5a0 con 0x5572ea8a4110 2023-10-05T19:40:22.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.711+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8ddb30 con 0x7f1158007030 2023-10-05T19:40:22.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.715+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f116000a5a0 con 0x7f1158007030 2023-10-05T19:40:22.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.715+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8ddb30 con 0x5572ea8db210 2023-10-05T19:40:22.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.715+0000 7f11676eb700 1 -- 172.21.15.12:0/3584340816 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f115c006070 con 0x5572ea8db210 2023-10-05T19:40:22.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.715+0000 7f11676eb700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8ddb30 con 0x7f1158007030 2023-10-05T19:40:22.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.715+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f116000a5a0 con 0x7f1158007030 2023-10-05T19:40:22.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.715+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8ddb30 con 0x5572ea8d6890 2023-10-05T19:40:22.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.715+0000 7f1167eec700 1 -- 172.21.15.12:0/3584340816 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1158011570 con 0x5572ea8d6890 2023-10-05T19:40:22.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.715+0000 7f1167eec700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8ddb30 con 0x5572ea8db210 2023-10-05T19:40:22.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.715+0000 7f11676eb700 1 -- 172.21.15.12:0/3584340816 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f115c006070 con 0x5572ea8db210 2023-10-05T19:40:22.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.715+0000 7f11676eb700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8ddb30 con 0x7f1158007030 2023-10-05T19:40:22.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.719+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f116000a5a0 con 0x7f1158007030 2023-10-05T19:40:22.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.719+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8ddb30 con 0x5572ea8a4110 2023-10-05T19:40:22.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.719+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f116000a5a0 con 0x5572ea8a4110 2023-10-05T19:40:22.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.719+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8ddb30 con 0x5572ea8d6890 2023-10-05T19:40:22.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.719+0000 7f1167eec700 1 -- 172.21.15.12:0/3584340816 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f1158011570 con 0x5572ea8d6890 2023-10-05T19:40:22.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.719+0000 7f116a3b7e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:22.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.719+0000 7f116a3b7e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:22.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.719+0000 7f116a3b7e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:22.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.719+0000 7f116a3b7e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:22.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.719+0000 7f116a3b7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.719+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8d9fc0 con 0x5572ea8a4110 2023-10-05T19:40:22.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.719+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f116000a5a0 con 0x5572ea8a4110 2023-10-05T19:40:22.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.719+0000 7f116a3b7e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:22.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.719+0000 7f116a3b7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.719+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8da910 con 0x5572ea8a4110 2023-10-05T19:40:22.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.719+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f116000a5a0 con 0x5572ea8a4110 2023-10-05T19:40:22.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.719+0000 7f116a3b7e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:22.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.719+0000 7f116a3b7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.719+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8d9b80 con 0x5572ea8db210 2023-10-05T19:40:22.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.723+0000 7f11676eb700 1 -- 172.21.15.12:0/3584340816 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f115c006070 con 0x5572ea8db210 2023-10-05T19:40:22.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.723+0000 7f116a3b7e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:22.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.723+0000 7f116a3b7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.723+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8de3d0 con 0x5572ea8db210 2023-10-05T19:40:22.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.723+0000 7f11676eb700 1 -- 172.21.15.12:0/3584340816 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f115c006070 con 0x5572ea8db210 2023-10-05T19:40:22.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.723+0000 7f116a3b7e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:22.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.723+0000 7f116a3b7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.723+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8de780 con 0x5572ea8a4110 2023-10-05T19:40:22.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.723+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f116000a5a0 con 0x5572ea8a4110 2023-10-05T19:40:22.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.723+0000 7f116a3b7e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:22.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.723+0000 7f116a3b7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.723+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8deb30 con 0x5572ea8a4110 2023-10-05T19:40:22.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.723+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f116000a5a0 con 0x5572ea8a4110 2023-10-05T19:40:22.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.723+0000 7f116a3b7e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:22.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.723+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8dfb40 con 0x5572ea8db210 2023-10-05T19:40:22.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.727+0000 7f11676eb700 1 -- 172.21.15.12:0/3584340816 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f115c006070 con 0x5572ea8db210 2023-10-05T19:40:22.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.727+0000 7f11676eb700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8dfb40 con 0x5572ea8d6890 2023-10-05T19:40:22.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.727+0000 7f1167eec700 1 -- 172.21.15.12:0/3584340816 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1158011570 con 0x5572ea8d6890 2023-10-05T19:40:22.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.727+0000 7f1167eec700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8dfb40 con 0x7f1158007030 2023-10-05T19:40:22.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.727+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f116000a5a0 con 0x7f1158007030 2023-10-05T19:40:22.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.727+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8de780 con 0x5572ea8a4110 2023-10-05T19:40:22.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.727+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f116000a5a0 con 0x5572ea8a4110 2023-10-05T19:40:22.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.727+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8de780 con 0x5572ea8a4110 2023-10-05T19:40:22.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.731+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f116000a5a0 con 0x5572ea8a4110 2023-10-05T19:40:22.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.731+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8de780 con 0x7f1158007030 2023-10-05T19:40:22.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.731+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f116000a5a0 con 0x7f1158007030 2023-10-05T19:40:22.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.731+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8de780 con 0x7f1158007030 2023-10-05T19:40:22.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.731+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f116000a5a0 con 0x7f1158007030 2023-10-05T19:40:22.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.731+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8de780 con 0x5572ea8d6890 2023-10-05T19:40:22.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.731+0000 7f1167eec700 1 -- 172.21.15.12:0/3584340816 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1158011570 con 0x5572ea8d6890 2023-10-05T19:40:22.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.731+0000 7f1167eec700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8de780 con 0x5572ea8db210 2023-10-05T19:40:22.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.731+0000 7f11676eb700 1 -- 172.21.15.12:0/3584340816 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f115c006070 con 0x5572ea8db210 2023-10-05T19:40:22.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.731+0000 7f11676eb700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8de780 con 0x5572ea8a4110 2023-10-05T19:40:22.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.735+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f116000a5a0 con 0x5572ea8a4110 2023-10-05T19:40:22.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.735+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8de780 con 0x7f1158007030 2023-10-05T19:40:22.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.735+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f116000a5a0 con 0x7f1158007030 2023-10-05T19:40:22.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.735+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8de780 con 0x5572ea8a4110 2023-10-05T19:40:22.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.735+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f116000a5a0 con 0x5572ea8a4110 2023-10-05T19:40:22.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.735+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8de780 con 0x5572ea8d6890 2023-10-05T19:40:22.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.735+0000 7f1167eec700 1 -- 172.21.15.12:0/3584340816 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1158011570 con 0x5572ea8d6890 2023-10-05T19:40:22.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.735+0000 7f1167eec700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8de780 con 0x5572ea8db210 2023-10-05T19:40:22.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.735+0000 7f11676eb700 1 -- 172.21.15.12:0/3584340816 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f115c006070 con 0x5572ea8db210 2023-10-05T19:40:22.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.735+0000 7f11676eb700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8de780 con 0x5572ea8db210 2023-10-05T19:40:22.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.739+0000 7f11676eb700 1 -- 172.21.15.12:0/3584340816 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f115c006070 con 0x5572ea8db210 2023-10-05T19:40:22.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.739+0000 7f11676eb700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8de780 con 0x7f1158007030 2023-10-05T19:40:22.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.739+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f116000a5a0 con 0x7f1158007030 2023-10-05T19:40:22.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.739+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8de780 con 0x5572ea8db210 2023-10-05T19:40:22.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.739+0000 7f11676eb700 1 -- 172.21.15.12:0/3584340816 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f115c006070 con 0x5572ea8db210 2023-10-05T19:40:22.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.739+0000 7f11676eb700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8de780 con 0x5572ea8db210 2023-10-05T19:40:22.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.739+0000 7f11676eb700 1 -- 172.21.15.12:0/3584340816 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f115c006070 con 0x5572ea8db210 2023-10-05T19:40:22.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.739+0000 7f11676eb700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8de780 con 0x5572ea8a4110 2023-10-05T19:40:22.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.739+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f116000a5a0 con 0x5572ea8a4110 2023-10-05T19:40:22.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.739+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8de780 con 0x5572ea8a4110 2023-10-05T19:40:22.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.743+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f116000a5a0 con 0x5572ea8a4110 2023-10-05T19:40:22.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.743+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8de780 con 0x7f1158007030 2023-10-05T19:40:22.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.743+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f116000a5a0 con 0x7f1158007030 2023-10-05T19:40:22.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.743+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8de780 con 0x5572ea8db210 2023-10-05T19:40:22.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.743+0000 7f11676eb700 1 -- 172.21.15.12:0/3584340816 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f115c006070 con 0x5572ea8db210 2023-10-05T19:40:22.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.743+0000 7f11676eb700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8de780 con 0x5572ea8d6890 2023-10-05T19:40:22.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.743+0000 7f1167eec700 1 -- 172.21.15.12:0/3584340816 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1158011570 con 0x5572ea8d6890 2023-10-05T19:40:22.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.743+0000 7f1167eec700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8de780 con 0x5572ea8a4110 2023-10-05T19:40:22.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.743+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f116000a5a0 con 0x5572ea8a4110 2023-10-05T19:40:22.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.743+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8de780 con 0x7f1158007030 2023-10-05T19:40:22.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.747+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f116000a5a0 con 0x7f1158007030 2023-10-05T19:40:22.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.747+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8de780 con 0x5572ea8db210 2023-10-05T19:40:22.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.747+0000 7f11676eb700 1 -- 172.21.15.12:0/3584340816 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f115c006070 con 0x5572ea8db210 2023-10-05T19:40:22.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.747+0000 7f11676eb700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8de780 con 0x7f1158007030 2023-10-05T19:40:22.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.747+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f116000a5a0 con 0x7f1158007030 2023-10-05T19:40:22.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.747+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8de780 con 0x5572ea8d6890 2023-10-05T19:40:22.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.747+0000 7f1167eec700 1 -- 172.21.15.12:0/3584340816 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1158011570 con 0x5572ea8d6890 2023-10-05T19:40:22.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.747+0000 7f1167eec700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8de780 con 0x5572ea8db210 2023-10-05T19:40:22.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.747+0000 7f11676eb700 1 -- 172.21.15.12:0/3584340816 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f115c006070 con 0x5572ea8db210 2023-10-05T19:40:22.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.747+0000 7f11676eb700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8de780 con 0x7f1158007030 2023-10-05T19:40:22.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.751+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f116000a5a0 con 0x7f1158007030 2023-10-05T19:40:22.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.751+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8de780 con 0x5572ea8a4110 2023-10-05T19:40:22.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.751+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f116000a5a0 con 0x5572ea8a4110 2023-10-05T19:40:22.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.751+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8de780 con 0x5572ea8d6890 2023-10-05T19:40:22.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.751+0000 7f1167eec700 1 -- 172.21.15.12:0/3584340816 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f1158011570 con 0x5572ea8d6890 2023-10-05T19:40:22.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.751+0000 7f116a3b7e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:22.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.751+0000 7f116a3b7e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:22.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.751+0000 7f116a3b7e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:22.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.751+0000 7f116a3b7e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:22.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.751+0000 7f116a3b7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.751+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8deb30 con 0x5572ea8a4110 2023-10-05T19:40:22.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.751+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f116000a5a0 con 0x5572ea8a4110 2023-10-05T19:40:22.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.751+0000 7f116a3b7e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:22.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.751+0000 7f116a3b7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.751+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8deee0 con 0x5572ea8d6890 2023-10-05T19:40:22.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.755+0000 7f1167eec700 1 -- 172.21.15.12:0/3584340816 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f1158011570 con 0x5572ea8d6890 2023-10-05T19:40:22.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.755+0000 7f116a3b7e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:22.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.755+0000 7f116a3b7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.755+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8df780 con 0x5572ea8a4110 2023-10-05T19:40:22.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.755+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f116000a5a0 con 0x5572ea8a4110 2023-10-05T19:40:22.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.755+0000 7f116a3b7e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:22.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.755+0000 7f116a3b7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.755+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8df780 con 0x5572ea8db210 2023-10-05T19:40:22.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.755+0000 7f11676eb700 1 -- 172.21.15.12:0/3584340816 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f115c006070 con 0x5572ea8db210 2023-10-05T19:40:22.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.755+0000 7f116a3b7e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:22.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.755+0000 7f116a3b7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.755+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8e0020 con 0x7f1158007030 2023-10-05T19:40:22.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.755+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f116000a5a0 con 0x7f1158007030 2023-10-05T19:40:22.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.755+0000 7f116a3b7e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:22.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.755+0000 7f116a3b7e40 20 zone default found 2023-10-05T19:40:22.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.755+0000 7f116a3b7e40 4 Realm: () 2023-10-05T19:40:22.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.755+0000 7f116a3b7e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:22.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.755+0000 7f116a3b7e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:22.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.755+0000 7f116a3b7e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:22.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.755+0000 7f116a3b7e40 20 zonegroup default 2023-10-05T19:40:22.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.755+0000 7f116a3b7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.755+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8e0020 con 0x7f1158007030 2023-10-05T19:40:22.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.759+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f116000a5a0 con 0x7f1158007030 2023-10-05T19:40:22.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.759+0000 7f116a3b7e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:22.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.759+0000 7f116a3b7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.759+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8e19a0 con 0x5572ea8db210 2023-10-05T19:40:22.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.759+0000 7f11676eb700 1 -- 172.21.15.12:0/3584340816 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f115c006070 con 0x5572ea8db210 2023-10-05T19:40:22.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.759+0000 7f116a3b7e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:22.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.759+0000 7f116a3b7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.759+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8e1d50 con 0x5572ea8db210 2023-10-05T19:40:22.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.759+0000 7f11676eb700 1 -- 172.21.15.12:0/3584340816 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f115c006070 con 0x5572ea8db210 2023-10-05T19:40:22.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.759+0000 7f116a3b7e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:22.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.759+0000 7f116a3b7e40 20 started sync module instance, tier type = 2023-10-05T19:40:22.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.759+0000 7f116a3b7e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:22.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.759+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8e1060 con 0x5572ea8d6890 2023-10-05T19:40:22.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.763+0000 7f1167eec700 1 -- 172.21.15.12:0/3584340816 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'84 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1158011570 con 0x5572ea8d6890 2023-10-05T19:40:22.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.763+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93952049810288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8e2100 con 0x5572ea8d6890 2023-10-05T19:40:22.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.763+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8e29a0 con 0x5572ea8a4110 2023-10-05T19:40:22.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.767+0000 7f1167eec700 1 -- 172.21.15.12:0/3584340816 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 93952049810288] v28'85 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1158011570 con 0x5572ea8d6890 2023-10-05T19:40:22.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.767+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'84 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f116000a5a0 con 0x5572ea8a4110 2023-10-05T19:40:22.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.767+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93952049812576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8e29a0 con 0x5572ea8a4110 2023-10-05T19:40:22.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.767+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8e33f0 con 0x5572ea8db210 2023-10-05T19:40:22.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.767+0000 7f11676eb700 1 -- 172.21.15.12:0/3584340816 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'84 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f115c006070 con 0x5572ea8db210 2023-10-05T19:40:22.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.767+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93952049819328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8e3ce0 con 0x5572ea8db210 2023-10-05T19:40:22.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.767+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8e46f0 con 0x5572ea8d6890 2023-10-05T19:40:22.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.771+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 93952049812576] v28'85 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f116000a5a0 con 0x5572ea8a4110 2023-10-05T19:40:22.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.771+0000 7f11676eb700 1 -- 172.21.15.12:0/3584340816 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 93952049819328] v28'85 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f115c006070 con 0x5572ea8db210 2023-10-05T19:40:22.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.771+0000 7f1167eec700 1 -- 172.21.15.12:0/3584340816 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'84 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1158011570 con 0x5572ea8d6890 2023-10-05T19:40:22.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.771+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 93952049824256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8e3910 con 0x5572ea8d6890 2023-10-05T19:40:22.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.771+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8e5170 con 0x5572ea8a4110 2023-10-05T19:40:22.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.775+0000 7f1167eec700 1 -- 172.21.15.12:0/3584340816 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 93952049824256] v28'85 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1158011570 con 0x5572ea8d6890 2023-10-05T19:40:22.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.775+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'84 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f116000a5a0 con 0x5572ea8a4110 2023-10-05T19:40:22.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.775+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 93952049816752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8e5130 con 0x5572ea8a4110 2023-10-05T19:40:22.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.775+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8e5b40 con 0x7f1158007030 2023-10-05T19:40:22.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.779+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'84 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f116000a5a0 con 0x7f1158007030 2023-10-05T19:40:22.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.779+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 93952049829456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8e5f90 con 0x7f1158007030 2023-10-05T19:40:22.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.779+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8e69a0 con 0x7f1158007030 2023-10-05T19:40:22.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.779+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 93952049816752] v28'85 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f116000a5a0 con 0x5572ea8a4110 2023-10-05T19:40:22.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.779+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(95 notify.5 [watch watch cookie 93952049829456] v28'85 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f116000a5a0 con 0x7f1158007030 2023-10-05T19:40:22.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.779+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(96 notify.6 [create] v28'84 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f116000a5a0 con 0x7f1158007030 2023-10-05T19:40:22.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.779+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93952049830560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8e5ef0 con 0x7f1158007030 2023-10-05T19:40:22.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.779+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8e6e90 con 0x7f1158007030 2023-10-05T19:40:22.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.783+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(98 notify.7 [create] v28'84 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f116000a5a0 con 0x7f1158007030 2023-10-05T19:40:22.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.783+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(97 notify.6 [watch watch cookie 93952049830560] v28'85 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f116000a5a0 con 0x7f1158007030 2023-10-05T19:40:22.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.783+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93952049834400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8e6df0 con 0x7f1158007030 2023-10-05T19:40:22.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.783+0000 7f116a3b7e40 20 add_watcher() i=0 2023-10-05T19:40:22.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.783+0000 7f116a3b7e40 20 add_watcher() i=1 2023-10-05T19:40:22.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.783+0000 7f116a3b7e40 20 add_watcher() i=2 2023-10-05T19:40:22.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.783+0000 7f116a3b7e40 20 add_watcher() i=3 2023-10-05T19:40:22.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.783+0000 7f116a3b7e40 20 add_watcher() i=4 2023-10-05T19:40:22.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.783+0000 7f116a3b7e40 20 add_watcher() i=5 2023-10-05T19:40:22.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.783+0000 7f116a3b7e40 20 add_watcher() i=6 2023-10-05T19:40:22.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.787+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 93952049834400] v28'85 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f116000a5a0 con 0x7f1158007030 2023-10-05T19:40:22.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.787+0000 7f116a3b7e40 20 add_watcher() i=7 2023-10-05T19:40:22.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.787+0000 7f116a3b7e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:22.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.787+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8ea310 con 0x5572ea8a4110 2023-10-05T19:40:22.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.787+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f116000a5a0 con 0x5572ea8a4110 2023-10-05T19:40:22.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.787+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93952049835504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8e7730 con 0x5572ea8a4110 2023-10-05T19:40:22.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.791+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 93952049835504] v28'90 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f116000a5a0 con 0x5572ea8a4110 2023-10-05T19:40:22.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.791+0000 7f11397e2700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:22.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.791+0000 7f116a3b7e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:22.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.791+0000 7f116a3b7e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:22.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.791+0000 7f116a3b7e40 1 Processor -- start 2023-10-05T19:40:22.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.791+0000 7f116a3b7e40 1 -- start start 2023-10-05T19:40:22.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.791+0000 7f116a3b7e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5572ea8f4500 0x5572ea8f48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.791+0000 7f116a3b7e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572ea8f4e10 con 0x5572ea8f4500 2023-10-05T19:40:22.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.791+0000 7f11676eb700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5572ea8f4500 0x5572ea8f48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.791+0000 7f11676eb700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5572ea8f4500 0x5572ea8f48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57460/0 (socket says 172.21.15.12:57460) 2023-10-05T19:40:22.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.791+0000 7f11676eb700 1 -- 172.21.15.12:0/1620554934 learned_addr learned my addr 172.21.15.12:0/1620554934 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:22.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.791+0000 7f11676eb700 1 -- 172.21.15.12:0/1620554934 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572ea8f4f50 con 0x5572ea8f4500 2023-10-05T19:40:22.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.791+0000 7f11676eb700 1 --2- 172.21.15.12:0/1620554934 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5572ea8f4500 0x5572ea8f48d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x5572ea8d99e0 tx=0x7f115c01e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f8d9f58977d09ef server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.791+0000 7f1138fe1700 1 -- 172.21.15.12:0/1620554934 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f115c013040 con 0x5572ea8f4500 2023-10-05T19:40:22.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.791+0000 7f1138fe1700 1 -- 172.21.15.12:0/1620554934 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f115c022040 con 0x5572ea8f4500 2023-10-05T19:40:22.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.791+0000 7f1138fe1700 1 -- 172.21.15.12:0/1620554934 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f115c00e920 con 0x5572ea8f4500 2023-10-05T19:40:22.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.795+0000 7f116a3b7e40 1 -- 172.21.15.12:0/1620554934 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5572ea8f4500 msgr2=0x5572ea8f48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.795+0000 7f116a3b7e40 1 --2- 172.21.15.12:0/1620554934 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5572ea8f4500 0x5572ea8f48d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x5572ea8d99e0 tx=0x7f115c01e040 comp rx=0 tx=0).stop 2023-10-05T19:40:22.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.795+0000 7f116a3b7e40 1 -- 172.21.15.12:0/1620554934 shutdown_connections 2023-10-05T19:40:22.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.795+0000 7f116a3b7e40 1 --2- 172.21.15.12:0/1620554934 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5572ea8f4500 0x5572ea8f48d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:22.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.795+0000 7f116a3b7e40 1 -- 172.21.15.12:0/1620554934 >> 172.21.15.12:0/1620554934 conn(0x5572ea8ef8c0 msgr2=0x5572ea8eda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:22.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.795+0000 7f116a3b7e40 1 -- 172.21.15.12:0/1620554934 shutdown_connections 2023-10-05T19:40:22.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.795+0000 7f116a3b7e40 1 -- 172.21.15.12:0/1620554934 wait complete. 2023-10-05T19:40:22.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.795+0000 7f116a3b7e40 1 Processor -- start 2023-10-05T19:40:22.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.795+0000 7f116a3b7e40 1 -- start start 2023-10-05T19:40:22.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.795+0000 7f116a3b7e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5572ea8f4500 0x5572ea8fa050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.795+0000 7f116a3b7e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572ea8fb050 con 0x5572ea8f4500 2023-10-05T19:40:22.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.795+0000 7f11676eb700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5572ea8f4500 0x5572ea8fa050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.795+0000 7f11676eb700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5572ea8f4500 0x5572ea8fa050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57476/0 (socket says 172.21.15.12:57476) 2023-10-05T19:40:22.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.795+0000 7f11676eb700 1 -- 172.21.15.12:0/990733636 learned_addr learned my addr 172.21.15.12:0/990733636 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:22.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.795+0000 7f11676eb700 1 -- 172.21.15.12:0/990733636 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f115c034040 con 0x5572ea8f4500 2023-10-05T19:40:22.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.795+0000 7f11676eb700 1 --2- 172.21.15.12:0/990733636 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5572ea8f4500 0x5572ea8fa050 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f115c049040 tx=0x7f115c03d030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.799+0000 7f10b2ffd700 1 -- 172.21.15.12:0/990733636 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f115c02c640 con 0x5572ea8f4500 2023-10-05T19:40:22.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.799+0000 7f10b2ffd700 1 -- 172.21.15.12:0/990733636 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f115c02c7a0 con 0x5572ea8f4500 2023-10-05T19:40:22.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.799+0000 7f116a3b7e40 1 -- 172.21.15.12:0/990733636 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5572ea8fa650 con 0x5572ea8f4500 2023-10-05T19:40:22.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.799+0000 7f10b2ffd700 1 -- 172.21.15.12:0/990733636 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f115c02c930 con 0x5572ea8f4500 2023-10-05T19:40:22.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.799+0000 7f116a3b7e40 1 -- 172.21.15.12:0/990733636 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5572ea8faa90 con 0x5572ea8f4500 2023-10-05T19:40:22.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.799+0000 7f10b2ffd700 1 -- 172.21.15.12:0/990733636 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f115c02ca90 con 0x5572ea8f4500 2023-10-05T19:40:22.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.799+0000 7f10b2ffd700 1 --2- 172.21.15.12:0/990733636 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f10a8030c50 0x7f10a8033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.799+0000 7f10b2ffd700 1 -- 172.21.15.12:0/990733636 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f115c049120 con 0x5572ea8f4500 2023-10-05T19:40:22.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.799+0000 7f11686ed700 1 --2- 172.21.15.12:0/990733636 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f10a8030c50 0x7f10a8033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.799+0000 7f116a3b7e40 5 note: GC not initialized 2023-10-05T19:40:22.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.799+0000 7f11686ed700 1 --2- 172.21.15.12:0/990733636 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f10a8030c50 0x7f10a8033100 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x5572ea8e6790 tx=0x7f116000ae20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.803+0000 7f10a7fff700 20 reqs_thread_entry: start 2023-10-05T19:40:22.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.803+0000 7f116a3b7e40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:22.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.803+0000 7f10a5ffb700 20 reqs_thread_entry: start 2023-10-05T19:40:22.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.803+0000 7f116a3b7e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:22.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.803+0000 7f1093fff700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:52 2023 2023-10-05T19:40:22.813 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:22.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.803+0000 7f10937fe700 20 reqs_thread_entry: start 2023-10-05T19:40:22.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.803+0000 7f116a3b7e40 20 get_system_obj_state: rctx=0x7fff87db3f68 obj=default.rgw.meta:users.uid:foo state=0x5572ea77a0b0 s->prefetch_data=0 2023-10-05T19:40:22.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.803+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea9628a0 con 0x5572ea8a4110 2023-10-05T19:40:22.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.803+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(102 foo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7f1160040070 con 0x5572ea8a4110 2023-10-05T19:40:22.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.803+0000 7f116a3b7e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:22.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.803+0000 7f116a3b7e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:22.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.803+0000 7f116a3b7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.803+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea962860 con 0x5572ea8a4110 2023-10-05T19:40:22.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.807+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(103 foo [call,call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv11 ondisk = 0) v8 ==== 273+0+480 (crc 0 0 0) 0x7f1160040070 con 0x5572ea8a4110 2023-10-05T19:40:22.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.807+0000 7f116a3b7e40 20 rados_obj.operate() r=0 bl.length=357 2023-10-05T19:40:22.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.807+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:104 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea962860 con 0x5572ea8d6890 2023-10-05T19:40:22.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.807+0000 7f1167eec700 1 -- 172.21.15.12:0/3584340816 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(104 foo.buckets [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 155+0+0 (crc 0 0 0) 0x7f1158011570 con 0x5572ea8d6890 2023-10-05T19:40:22.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.807+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:105 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93952049810288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea962860 con 0x5572ea8d6890 2023-10-05T19:40:22.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.811+0000 7f1167eec700 1 -- 172.21.15.12:0/3584340816 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(105 notify.0 [watch unwatch cookie 93952049810288] v28'86 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1158011570 con 0x5572ea8d6890 2023-10-05T19:40:22.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.811+0000 7f116a3b7e40 20 remove_watcher() i=0 2023-10-05T19:40:22.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.811+0000 7f116a3b7e40 2 removed watcher, disabling cache 2023-10-05T19:40:22.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.811+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:106 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93952049812576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea962860 con 0x5572ea8a4110 2023-10-05T19:40:22.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.811+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(106 notify.1 [watch unwatch cookie 93952049812576] v28'86 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1160040070 con 0x5572ea8a4110 2023-10-05T19:40:22.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.811+0000 7f116a3b7e40 20 remove_watcher() i=1 2023-10-05T19:40:22.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.811+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:107 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93952049819328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea962860 con 0x5572ea8db210 2023-10-05T19:40:22.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.815+0000 7f11676eb700 1 -- 172.21.15.12:0/3584340816 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(107 notify.2 [watch unwatch cookie 93952049819328] v28'86 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f115c054020 con 0x5572ea8db210 2023-10-05T19:40:22.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.815+0000 7f116a3b7e40 20 remove_watcher() i=2 2023-10-05T19:40:22.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.815+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:108 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93952049824256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea962860 con 0x5572ea8d6890 2023-10-05T19:40:22.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.819+0000 7f1167eec700 1 -- 172.21.15.12:0/3584340816 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(108 notify.3 [watch unwatch cookie 93952049824256] v28'86 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1158011570 con 0x5572ea8d6890 2023-10-05T19:40:22.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.819+0000 7f116a3b7e40 20 remove_watcher() i=3 2023-10-05T19:40:22.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.819+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:109 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93952049816752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea962860 con 0x5572ea8a4110 2023-10-05T19:40:22.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.823+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(109 notify.4 [watch unwatch cookie 93952049816752] v28'86 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1160040070 con 0x5572ea8a4110 2023-10-05T19:40:22.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.823+0000 7f116a3b7e40 20 remove_watcher() i=4 2023-10-05T19:40:22.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.823+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:110 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93952049829456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea962860 con 0x7f1158007030 2023-10-05T19:40:22.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.823+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(110 notify.5 [watch unwatch cookie 93952049829456] v28'86 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1160040070 con 0x7f1158007030 2023-10-05T19:40:22.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.823+0000 7f116a3b7e40 20 remove_watcher() i=5 2023-10-05T19:40:22.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.823+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:111 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93952049830560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea962860 con 0x7f1158007030 2023-10-05T19:40:22.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.827+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(111 notify.6 [watch unwatch cookie 93952049830560] v28'86 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1160040070 con 0x7f1158007030 2023-10-05T19:40:22.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.827+0000 7f116a3b7e40 20 remove_watcher() i=6 2023-10-05T19:40:22.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.827+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:112 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93952049834400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea8e5b40 con 0x7f1158007030 2023-10-05T19:40:22.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.831+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(112 notify.7 [watch unwatch cookie 93952049834400] v28'86 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1160040070 con 0x7f1158007030 2023-10-05T19:40:22.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.831+0000 7f116a3b7e40 20 remove_watcher() i=7 2023-10-05T19:40:22.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.831+0000 7f11397e2700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:22.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.831+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:113 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93952049835504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5572ea921c20 con 0x5572ea8a4110 2023-10-05T19:40:22.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.835+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(113 data_loggenerations_metadata [watch unwatch cookie 93952049835504] v28'91 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f1160040070 con 0x5572ea8a4110 2023-10-05T19:40:22.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.835+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f1158007030 msgr2=0x7f1158011830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.835+0000 7f116a3b7e40 1 --2- 172.21.15.12:0/3584340816 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f1158007030 0x7f1158011830 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:22.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.835+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5572ea8a4110 msgr2=0x5572ea8d47b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.835+0000 7f116a3b7e40 1 --2- 172.21.15.12:0/3584340816 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5572ea8a4110 0x5572ea8d47b0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:22.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.835+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5572ea8d6890 msgr2=0x5572ea8d8c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.835+0000 7f116a3b7e40 1 --2- 172.21.15.12:0/3584340816 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5572ea8d6890 0x5572ea8d8c70 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:22.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.835+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5572ea8db210 msgr2=0x5572ea8dd5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.835+0000 7f116a3b7e40 1 --2- 172.21.15.12:0/3584340816 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5572ea8db210 0x5572ea8dd5f0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:22.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.835+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1154030c00 msgr2=0x7f11540330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.835+0000 7f116a3b7e40 1 --2- 172.21.15.12:0/3584340816 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1154030c00 0x7f11540330b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f115c001df0 tx=0x7f115c001310 comp rx=0 tx=0).stop 2023-10-05T19:40:22.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.835+0000 7f11686ed700 1 -- 172.21.15.12:0/3584340816 reap_dead start 2023-10-05T19:40:22.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.835+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5572ea8153d0 msgr2=0x5572ea8a1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.835+0000 7f116a3b7e40 1 --2- 172.21.15.12:0/3584340816 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5572ea8153d0 0x5572ea8a1360 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f115800c010 tx=0x7f115801c000 comp rx=0 tx=0).stop 2023-10-05T19:40:22.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.835+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 shutdown_connections 2023-10-05T19:40:22.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.835+0000 7f116a3b7e40 1 --2- 172.21.15.12:0/3584340816 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5572ea8153d0 0x5572ea8a1360 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:22.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.835+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 >> 172.21.15.12:0/3584340816 conn(0x5572ea89b720 msgr2=0x5572ea8ba4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:22.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.835+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 shutdown_connections 2023-10-05T19:40:22.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.835+0000 7f116a3b7e40 1 -- 172.21.15.12:0/3584340816 wait complete. 2023-10-05T19:40:22.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.839+0000 7f116a3b7e40 1 -- 172.21.15.12:0/990733636 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f10a8030c50 msgr2=0x7f10a8033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.839+0000 7f116a3b7e40 1 --2- 172.21.15.12:0/990733636 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f10a8030c50 0x7f10a8033100 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x5572ea8e6790 tx=0x7f116000ae20 comp rx=0 tx=0).stop 2023-10-05T19:40:22.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.839+0000 7f116a3b7e40 1 -- 172.21.15.12:0/990733636 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5572ea8f4500 msgr2=0x5572ea8fa050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.839+0000 7f116a3b7e40 1 --2- 172.21.15.12:0/990733636 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5572ea8f4500 0x5572ea8fa050 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f115c049040 tx=0x7f115c03d030 comp rx=0 tx=0).stop 2023-10-05T19:40:22.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.839+0000 7f116a3b7e40 1 -- 172.21.15.12:0/990733636 shutdown_connections 2023-10-05T19:40:22.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.839+0000 7f116a3b7e40 1 --2- 172.21.15.12:0/990733636 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f10a8030c50 0x7f10a8033100 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:22.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.839+0000 7f116a3b7e40 1 --2- 172.21.15.12:0/990733636 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5572ea8f4500 0x5572ea8fa050 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:22.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.839+0000 7f116a3b7e40 1 -- 172.21.15.12:0/990733636 >> 172.21.15.12:0/990733636 conn(0x5572ea8ef8c0 msgr2=0x5572ea8ed9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:22.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.839+0000 7f116a3b7e40 1 -- 172.21.15.12:0/990733636 shutdown_connections 2023-10-05T19:40:22.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.839+0000 7f116a3b7e40 1 -- 172.21.15.12:0/990733636 wait complete. 2023-10-05T19:40:22.849 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2023-10-05T19:40:22.853 DEBUG:tasks.util.rgw: json result: [] 2023-10-05T19:40:22.881 INFO:tasks.radosgw_admin:RL: create_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2023-10-05T19:40:22.881 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'list', '--uid', 'foo'] 2023-10-05T19:40:22.881 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'list', '--uid', 'foo'] 2023-10-05T19:40:22.882 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket list --uid foo 2023-10-05T19:40:22.932 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:22.932 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:22.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.923+0000 7f187dab4e40 1 Processor -- start 2023-10-05T19:40:22.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.923+0000 7f187dab4e40 1 -- start start 2023-10-05T19:40:22.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.923+0000 7f187dab4e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55b0ecf305f0 0x55b0ecf329d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.923+0000 7f187dab4e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55b0ece76bd0 con 0x55b0ecf2c010 2023-10-05T19:40:22.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.923+0000 7f187dab4e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x55b0ece33ee0 con 0x55b0ecf305f0 2023-10-05T19:40:22.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.923+0000 7f187ade8700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55b0ecf305f0 0x55b0ecf329d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.923+0000 7f187b5e9700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x55b0ecf2c010 0x55b0ecf2c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:35466/0 (socket says 172.21.15.12:35466) 2023-10-05T19:40:22.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.923+0000 7f187b5e9700 1 -- 172.21.15.12:0/3315430223 learned_addr learned my addr 172.21.15.12:0/3315430223 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:22.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.927+0000 7f187a5e7700 1 -- 172.21.15.12:0/3315430223 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 722847186 0 0) 0x55b0ece76bd0 con 0x55b0ecf2c010 2023-10-05T19:40:22.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.927+0000 7f187a5e7700 1 -- 172.21.15.12:0/3315430223 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1868001e10 con 0x55b0ecf2c010 2023-10-05T19:40:22.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.927+0000 7f187ade8700 1 -- 172.21.15.12:0/3315430223 >> v1:172.21.15.8:6789/0 conn(0x55b0ecf2c010 legacy=0x55b0ecf2c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.927+0000 7f187ade8700 1 -- 172.21.15.12:0/3315430223 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0ecf2cc20 con 0x55b0ecf305f0 2023-10-05T19:40:22.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.927+0000 7f187ade8700 1 --2- 172.21.15.12:0/3315430223 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b0ecf305f0 0x55b0ecf329d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f186400a520 tx=0x7f1864005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7cb6ac3787254da server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.927+0000 7f187a5e7700 1 -- 172.21.15.12:0/3315430223 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f186400f590 con 0x55b0ecf305f0 2023-10-05T19:40:22.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.927+0000 7f187a5e7700 1 -- 172.21.15.12:0/3315430223 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f186400f6f0 con 0x55b0ecf305f0 2023-10-05T19:40:22.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.927+0000 7f187a5e7700 1 -- 172.21.15.12:0/3315430223 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f186400f860 con 0x55b0ecf305f0 2023-10-05T19:40:22.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.927+0000 7f187dab4e40 1 -- 172.21.15.12:0/3315430223 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b0ecf305f0 msgr2=0x55b0ecf329d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.927+0000 7f187dab4e40 1 --2- 172.21.15.12:0/3315430223 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b0ecf305f0 0x55b0ecf329d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f186400a520 tx=0x7f1864005d40 comp rx=0 tx=0).stop 2023-10-05T19:40:22.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.927+0000 7f187dab4e40 1 -- 172.21.15.12:0/3315430223 shutdown_connections 2023-10-05T19:40:22.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.927+0000 7f187dab4e40 1 --2- 172.21.15.12:0/3315430223 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b0ecf305f0 0x55b0ecf329d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:22.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.927+0000 7f187dab4e40 1 -- 172.21.15.12:0/3315430223 >> 172.21.15.12:0/3315430223 conn(0x55b0ece9f860 msgr2=0x55b0ece9fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:22.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.927+0000 7f187dab4e40 1 -- 172.21.15.12:0/3315430223 shutdown_connections 2023-10-05T19:40:22.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.927+0000 7f187dab4e40 1 -- 172.21.15.12:0/3315430223 wait complete. 2023-10-05T19:40:22.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.927+0000 7f187dab4e40 1 Processor -- start 2023-10-05T19:40:22.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.927+0000 7f187dab4e40 1 -- start start 2023-10-05T19:40:22.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.927+0000 7f187dab4e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b0ecf2c010 0x55b0ecf4af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.927+0000 7f187dab4e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0ecdf7e30 con 0x55b0ecf2c010 2023-10-05T19:40:22.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.927+0000 7f187b5e9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b0ecf2c010 0x55b0ecf4af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.927+0000 7f187b5e9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b0ecf2c010 0x55b0ecf4af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57494/0 (socket says 172.21.15.12:57494) 2023-10-05T19:40:22.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.927+0000 7f187b5e9700 1 -- 172.21.15.12:0/3860750402 learned_addr learned my addr 172.21.15.12:0/3860750402 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:22.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.931+0000 7f187b5e9700 1 -- 172.21.15.12:0/3860750402 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f186400a0e0 con 0x55b0ecf2c010 2023-10-05T19:40:22.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.931+0000 7f187b5e9700 1 --2- 172.21.15.12:0/3860750402 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b0ecf2c010 0x55b0ecf4af90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1874001c30 tx=0x7f1874001810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7105ff203d4b1d13 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.931+0000 7f18795e5700 1 -- 172.21.15.12:0/3860750402 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f187400bbf0 con 0x55b0ecf2c010 2023-10-05T19:40:22.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.931+0000 7f18795e5700 1 -- 172.21.15.12:0/3860750402 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f187400bd50 con 0x55b0ecf2c010 2023-10-05T19:40:22.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.931+0000 7f18795e5700 1 -- 172.21.15.12:0/3860750402 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f187400e470 con 0x55b0ecf2c010 2023-10-05T19:40:22.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.931+0000 7f187dab4e40 1 -- 172.21.15.12:0/3860750402 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b0ecf2c010 msgr2=0x55b0ecf4af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:22.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.931+0000 7f187dab4e40 1 --2- 172.21.15.12:0/3860750402 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b0ecf2c010 0x55b0ecf4af90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1874001c30 tx=0x7f1874001810 comp rx=0 tx=0).stop 2023-10-05T19:40:22.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.931+0000 7f187dab4e40 1 -- 172.21.15.12:0/3860750402 shutdown_connections 2023-10-05T19:40:22.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.931+0000 7f187dab4e40 1 --2- 172.21.15.12:0/3860750402 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b0ecf2c010 0x55b0ecf4af90 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:22.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.931+0000 7f187dab4e40 1 -- 172.21.15.12:0/3860750402 >> 172.21.15.12:0/3860750402 conn(0x55b0ece9f860 msgr2=0x55b0ecf46b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:22.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.931+0000 7f187dab4e40 1 -- 172.21.15.12:0/3860750402 shutdown_connections 2023-10-05T19:40:22.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.931+0000 7f187dab4e40 1 -- 172.21.15.12:0/3860750402 wait complete. 2023-10-05T19:40:22.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.931+0000 7f187dab4e40 1 Processor -- start 2023-10-05T19:40:22.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.931+0000 7f187dab4e40 1 -- start start 2023-10-05T19:40:22.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.931+0000 7f187dab4e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b0ecf2c010 0x55b0ecf51d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.931+0000 7f187dab4e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0ecdcaed0 con 0x55b0ecf2c010 2023-10-05T19:40:22.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.931+0000 7f187b5e9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b0ecf2c010 0x55b0ecf51d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.931+0000 7f187b5e9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b0ecf2c010 0x55b0ecf51d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57504/0 (socket says 172.21.15.12:57504) 2023-10-05T19:40:22.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.931+0000 7f187b5e9700 1 -- 172.21.15.12:0/704651447 learned_addr learned my addr 172.21.15.12:0/704651447 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:22.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.935+0000 7f187b5e9700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1868001e10 con 0x55b0ecf2c010 2023-10-05T19:40:22.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.935+0000 7f187b5e9700 1 --2- 172.21.15.12:0/704651447 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b0ecf2c010 0x55b0ecf51d70 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f187400b960 tx=0x7f1874014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.935+0000 7f18737fe700 1 -- 172.21.15.12:0/704651447 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f187401f070 con 0x55b0ecf2c010 2023-10-05T19:40:22.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.935+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b0ecf522b0 con 0x55b0ecf2c010 2023-10-05T19:40:22.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.935+0000 7f18737fe700 1 -- 172.21.15.12:0/704651447 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f187401a2e0 con 0x55b0ecf2c010 2023-10-05T19:40:22.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.935+0000 7f18737fe700 1 -- 172.21.15.12:0/704651447 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f187400bec0 con 0x55b0ecf2c010 2023-10-05T19:40:22.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.935+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b0ecf526f0 con 0x55b0ecf2c010 2023-10-05T19:40:22.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.935+0000 7f18737fe700 1 -- 172.21.15.12:0/704651447 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f187401aa10 con 0x55b0ecf2c010 2023-10-05T19:40:22.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.935+0000 7f18737fe700 1 --2- 172.21.15.12:0/704651447 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f185c030bf0 0x7f185c0330a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.935+0000 7f187dab4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.935+0000 7f18737fe700 1 -- 172.21.15.12:0/704651447 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f1874044f30 con 0x55b0ecf2c010 2023-10-05T19:40:22.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.935+0000 7f187dab4e40 1 --2- 172.21.15.12:0/704651447 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55b0ecf305f0 0x55b0ecf5d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.935+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf5dea0 con 0x55b0ecf305f0 2023-10-05T19:40:22.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.935+0000 7f187ade8700 1 --2- 172.21.15.12:0/704651447 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f185c030bf0 0x7f185c0330a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.935+0000 7f187bdea700 1 --2- 172.21.15.12:0/704651447 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55b0ecf305f0 0x55b0ecf5d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.935+0000 7f187bdea700 1 --2- 172.21.15.12:0/704651447 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55b0ecf305f0 0x55b0ecf5d890 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.939+0000 7f187ade8700 1 --2- 172.21.15.12:0/704651447 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f185c030bf0 0x7f185c0330a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1864000f80 tx=0x7f1864014000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.939+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f186c00a500 con 0x55b0ecf305f0 2023-10-05T19:40:22.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.939+0000 7f187dab4e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:22.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.939+0000 7f187dab4e40 20 realm 2023-10-05T19:40:22.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.939+0000 7f187dab4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.939+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf5aef0 con 0x55b0ecf305f0 2023-10-05T19:40:22.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.939+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f186c00a500 con 0x55b0ecf305f0 2023-10-05T19:40:22.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.939+0000 7f187dab4e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:22.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.939+0000 7f187dab4e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:22.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.939+0000 7f187dab4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.939+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf5e980 con 0x55b0ecf305f0 2023-10-05T19:40:22.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.939+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f186c00a500 con 0x55b0ecf305f0 2023-10-05T19:40:22.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.939+0000 7f187dab4e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:22.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.939+0000 7f187dab4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.939+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf5e560 con 0x55b0ecf305f0 2023-10-05T19:40:22.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.939+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f186c00a500 con 0x55b0ecf305f0 2023-10-05T19:40:22.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.939+0000 7f187dab4e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:22.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.939+0000 7f187dab4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.939+0000 7f187dab4e40 1 --2- 172.21.15.12:0/704651447 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55b0ecf5f910 0x55b0ecf61cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.939+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf62460 con 0x55b0ecf5f910 2023-10-05T19:40:22.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.939+0000 7f187b5e9700 1 --2- 172.21.15.12:0/704651447 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55b0ecf5f910 0x55b0ecf61cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.939+0000 7f187b5e9700 1 --2- 172.21.15.12:0/704651447 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55b0ecf5f910 0x55b0ecf61cf0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.939+0000 7f187b5e9700 1 -- 172.21.15.12:0/704651447 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f1874014070 con 0x55b0ecf5f910 2023-10-05T19:40:22.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.939+0000 7f187dab4e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:22.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.939+0000 7f187dab4e40 20 searching for the correct realm 2023-10-05T19:40:22.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.939+0000 7f187dab4e40 1 --2- 172.21.15.12:0/704651447 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55b0ecf64290 0x55b0ecf66670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.939+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf66bb0 con 0x55b0ecf64290 2023-10-05T19:40:22.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.943+0000 7f187ade8700 1 --2- 172.21.15.12:0/704651447 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55b0ecf64290 0x55b0ecf66670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.943+0000 7f187ade8700 1 --2- 172.21.15.12:0/704651447 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55b0ecf64290 0x55b0ecf66670 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.943+0000 7f187ade8700 1 -- 172.21.15.12:0/704651447 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f186400fb90 con 0x55b0ecf64290 2023-10-05T19:40:22.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.943+0000 7f187ade8700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf66bb0 con 0x55b0ecf5f910 2023-10-05T19:40:22.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.943+0000 7f187b5e9700 1 -- 172.21.15.12:0/704651447 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1874014070 con 0x55b0ecf5f910 2023-10-05T19:40:22.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.943+0000 7f187b5e9700 1 --2- 172.21.15.12:0/704651447 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f1874016c10 0x7f18740142c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:22.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.943+0000 7f187b5e9700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf66bb0 con 0x7f1874016c10 2023-10-05T19:40:22.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.943+0000 7f187bdea700 1 --2- 172.21.15.12:0/704651447 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f1874016c10 0x7f18740142c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:22.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.947+0000 7f187bdea700 1 --2- 172.21.15.12:0/704651447 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f1874016c10 0x7f18740142c0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:22.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.947+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186c014040 con 0x7f1874016c10 2023-10-05T19:40:22.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.947+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf66bb0 con 0x55b0ecf305f0 2023-10-05T19:40:22.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.947+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186c014040 con 0x55b0ecf305f0 2023-10-05T19:40:22.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.947+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf66bb0 con 0x55b0ecf305f0 2023-10-05T19:40:22.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.947+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186c014040 con 0x55b0ecf305f0 2023-10-05T19:40:22.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.947+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf66bb0 con 0x7f1874016c10 2023-10-05T19:40:22.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.947+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186c014040 con 0x7f1874016c10 2023-10-05T19:40:22.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.947+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf66bb0 con 0x7f1874016c10 2023-10-05T19:40:22.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.951+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186c014040 con 0x7f1874016c10 2023-10-05T19:40:22.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.951+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf66bb0 con 0x55b0ecf5f910 2023-10-05T19:40:22.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.951+0000 7f187b5e9700 1 -- 172.21.15.12:0/704651447 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1874014070 con 0x55b0ecf5f910 2023-10-05T19:40:22.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.951+0000 7f187b5e9700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf66bb0 con 0x55b0ecf64290 2023-10-05T19:40:22.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.951+0000 7f187ade8700 1 -- 172.21.15.12:0/704651447 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186400fb90 con 0x55b0ecf64290 2023-10-05T19:40:22.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.951+0000 7f187ade8700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf66bb0 con 0x55b0ecf305f0 2023-10-05T19:40:22.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.951+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186c014040 con 0x55b0ecf305f0 2023-10-05T19:40:22.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.951+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf66bb0 con 0x7f1874016c10 2023-10-05T19:40:22.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.951+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186c014040 con 0x7f1874016c10 2023-10-05T19:40:22.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.951+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf66bb0 con 0x55b0ecf305f0 2023-10-05T19:40:22.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.955+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186c014040 con 0x55b0ecf305f0 2023-10-05T19:40:22.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.955+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf66bb0 con 0x55b0ecf5f910 2023-10-05T19:40:22.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.955+0000 7f187b5e9700 1 -- 172.21.15.12:0/704651447 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1874014070 con 0x55b0ecf5f910 2023-10-05T19:40:22.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.955+0000 7f187b5e9700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf66bb0 con 0x55b0ecf64290 2023-10-05T19:40:22.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.955+0000 7f187ade8700 1 -- 172.21.15.12:0/704651447 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186400fb90 con 0x55b0ecf64290 2023-10-05T19:40:22.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.955+0000 7f187ade8700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf66bb0 con 0x55b0ecf64290 2023-10-05T19:40:22.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.955+0000 7f187ade8700 1 -- 172.21.15.12:0/704651447 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186400fb90 con 0x55b0ecf64290 2023-10-05T19:40:22.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.955+0000 7f187ade8700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf66bb0 con 0x7f1874016c10 2023-10-05T19:40:22.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.955+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186c014040 con 0x7f1874016c10 2023-10-05T19:40:22.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.955+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf66bb0 con 0x55b0ecf64290 2023-10-05T19:40:22.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.959+0000 7f187ade8700 1 -- 172.21.15.12:0/704651447 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186400fb90 con 0x55b0ecf64290 2023-10-05T19:40:22.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.959+0000 7f187ade8700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf66bb0 con 0x55b0ecf64290 2023-10-05T19:40:22.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.959+0000 7f187ade8700 1 -- 172.21.15.12:0/704651447 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186400fb90 con 0x55b0ecf64290 2023-10-05T19:40:22.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.959+0000 7f187ade8700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf66bb0 con 0x55b0ecf305f0 2023-10-05T19:40:22.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.959+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186c014040 con 0x55b0ecf305f0 2023-10-05T19:40:22.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.959+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf66bb0 con 0x55b0ecf305f0 2023-10-05T19:40:22.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.959+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186c014040 con 0x55b0ecf305f0 2023-10-05T19:40:22.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.959+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf66bb0 con 0x7f1874016c10 2023-10-05T19:40:22.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.959+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186c014040 con 0x7f1874016c10 2023-10-05T19:40:22.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.959+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf66bb0 con 0x55b0ecf64290 2023-10-05T19:40:22.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.963+0000 7f187ade8700 1 -- 172.21.15.12:0/704651447 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186400fb90 con 0x55b0ecf64290 2023-10-05T19:40:22.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.963+0000 7f187ade8700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf66bb0 con 0x55b0ecf5f910 2023-10-05T19:40:22.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.963+0000 7f187b5e9700 1 -- 172.21.15.12:0/704651447 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1874014070 con 0x55b0ecf5f910 2023-10-05T19:40:22.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.963+0000 7f187b5e9700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf66bb0 con 0x55b0ecf305f0 2023-10-05T19:40:22.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.963+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186c014040 con 0x55b0ecf305f0 2023-10-05T19:40:22.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.963+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf66bb0 con 0x7f1874016c10 2023-10-05T19:40:22.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.963+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186c014040 con 0x7f1874016c10 2023-10-05T19:40:22.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.963+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf66bb0 con 0x55b0ecf64290 2023-10-05T19:40:22.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.963+0000 7f187ade8700 1 -- 172.21.15.12:0/704651447 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186400fb90 con 0x55b0ecf64290 2023-10-05T19:40:22.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.963+0000 7f187ade8700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf66bb0 con 0x7f1874016c10 2023-10-05T19:40:22.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.967+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186c014040 con 0x7f1874016c10 2023-10-05T19:40:22.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.967+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf66bb0 con 0x55b0ecf5f910 2023-10-05T19:40:22.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.967+0000 7f187b5e9700 1 -- 172.21.15.12:0/704651447 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1874014070 con 0x55b0ecf5f910 2023-10-05T19:40:22.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.967+0000 7f187b5e9700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf66bb0 con 0x55b0ecf64290 2023-10-05T19:40:22.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.967+0000 7f187ade8700 1 -- 172.21.15.12:0/704651447 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186400fb90 con 0x55b0ecf64290 2023-10-05T19:40:22.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.967+0000 7f187ade8700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf66bb0 con 0x7f1874016c10 2023-10-05T19:40:22.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.967+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186c014040 con 0x7f1874016c10 2023-10-05T19:40:22.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.967+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf66bb0 con 0x55b0ecf305f0 2023-10-05T19:40:22.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.971+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f186c014040 con 0x55b0ecf305f0 2023-10-05T19:40:22.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.971+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf66bb0 con 0x55b0ecf5f910 2023-10-05T19:40:22.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.971+0000 7f187b5e9700 1 -- 172.21.15.12:0/704651447 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f1874014070 con 0x55b0ecf5f910 2023-10-05T19:40:22.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.971+0000 7f187dab4e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:22.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.971+0000 7f187dab4e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:22.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.971+0000 7f187dab4e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:22.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.971+0000 7f187dab4e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:22.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.971+0000 7f187dab4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.971+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf63010 con 0x55b0ecf305f0 2023-10-05T19:40:22.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.971+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f186c014040 con 0x55b0ecf305f0 2023-10-05T19:40:22.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.971+0000 7f187dab4e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:22.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.971+0000 7f187dab4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.971+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf63960 con 0x55b0ecf305f0 2023-10-05T19:40:22.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.971+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f186c014040 con 0x55b0ecf305f0 2023-10-05T19:40:22.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.971+0000 7f187dab4e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:22.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.971+0000 7f187dab4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.971+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf62bd0 con 0x55b0ecf64290 2023-10-05T19:40:22.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.971+0000 7f187ade8700 1 -- 172.21.15.12:0/704651447 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f186400fb90 con 0x55b0ecf64290 2023-10-05T19:40:22.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.971+0000 7f187dab4e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:22.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.971+0000 7f187dab4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.971+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf674e0 con 0x55b0ecf64290 2023-10-05T19:40:22.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.971+0000 7f187ade8700 1 -- 172.21.15.12:0/704651447 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f186400fb90 con 0x55b0ecf64290 2023-10-05T19:40:22.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.971+0000 7f187dab4e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:22.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.971+0000 7f187dab4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.971+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf67890 con 0x55b0ecf305f0 2023-10-05T19:40:22.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.975+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f186c014040 con 0x55b0ecf305f0 2023-10-05T19:40:22.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.975+0000 7f187dab4e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:22.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.975+0000 7f187dab4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:22.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.975+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf67c40 con 0x55b0ecf305f0 2023-10-05T19:40:22.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.975+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f186c014040 con 0x55b0ecf305f0 2023-10-05T19:40:22.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.975+0000 7f187dab4e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:22.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.975+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf68c50 con 0x55b0ecf64290 2023-10-05T19:40:22.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.975+0000 7f187ade8700 1 -- 172.21.15.12:0/704651447 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f186400fb90 con 0x55b0ecf64290 2023-10-05T19:40:22.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.975+0000 7f187ade8700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf68c50 con 0x55b0ecf5f910 2023-10-05T19:40:22.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.979+0000 7f187b5e9700 1 -- 172.21.15.12:0/704651447 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1874014070 con 0x55b0ecf5f910 2023-10-05T19:40:22.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.979+0000 7f187b5e9700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf68c50 con 0x7f1874016c10 2023-10-05T19:40:22.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.979+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186c014040 con 0x7f1874016c10 2023-10-05T19:40:22.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.979+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf67890 con 0x55b0ecf305f0 2023-10-05T19:40:22.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.979+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186c014040 con 0x55b0ecf305f0 2023-10-05T19:40:22.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.979+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf67890 con 0x55b0ecf305f0 2023-10-05T19:40:22.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.979+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186c014040 con 0x55b0ecf305f0 2023-10-05T19:40:22.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.979+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf67890 con 0x7f1874016c10 2023-10-05T19:40:22.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.983+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186c014040 con 0x7f1874016c10 2023-10-05T19:40:22.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.983+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf67890 con 0x7f1874016c10 2023-10-05T19:40:22.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.983+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186c014040 con 0x7f1874016c10 2023-10-05T19:40:22.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.983+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf67890 con 0x55b0ecf5f910 2023-10-05T19:40:22.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.983+0000 7f187b5e9700 1 -- 172.21.15.12:0/704651447 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1874014070 con 0x55b0ecf5f910 2023-10-05T19:40:22.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.983+0000 7f187b5e9700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf67890 con 0x55b0ecf64290 2023-10-05T19:40:22.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.983+0000 7f187ade8700 1 -- 172.21.15.12:0/704651447 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186400fb90 con 0x55b0ecf64290 2023-10-05T19:40:22.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.983+0000 7f187ade8700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf67890 con 0x55b0ecf305f0 2023-10-05T19:40:22.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.983+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186c014040 con 0x55b0ecf305f0 2023-10-05T19:40:22.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.983+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf67890 con 0x7f1874016c10 2023-10-05T19:40:22.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.987+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186c014040 con 0x7f1874016c10 2023-10-05T19:40:22.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.987+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf67890 con 0x55b0ecf305f0 2023-10-05T19:40:22.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.987+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186c014040 con 0x55b0ecf305f0 2023-10-05T19:40:22.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.987+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf67890 con 0x55b0ecf5f910 2023-10-05T19:40:22.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.987+0000 7f187b5e9700 1 -- 172.21.15.12:0/704651447 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1874014070 con 0x55b0ecf5f910 2023-10-05T19:40:22.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.987+0000 7f187b5e9700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf67890 con 0x55b0ecf64290 2023-10-05T19:40:22.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.987+0000 7f187ade8700 1 -- 172.21.15.12:0/704651447 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186400fb90 con 0x55b0ecf64290 2023-10-05T19:40:22.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.987+0000 7f187ade8700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf67890 con 0x55b0ecf64290 2023-10-05T19:40:22.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.991+0000 7f187ade8700 1 -- 172.21.15.12:0/704651447 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186400fb90 con 0x55b0ecf64290 2023-10-05T19:40:22.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.991+0000 7f187ade8700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf67890 con 0x7f1874016c10 2023-10-05T19:40:22.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.991+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186c014040 con 0x7f1874016c10 2023-10-05T19:40:22.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.991+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf67890 con 0x55b0ecf64290 2023-10-05T19:40:22.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.991+0000 7f187ade8700 1 -- 172.21.15.12:0/704651447 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186400fb90 con 0x55b0ecf64290 2023-10-05T19:40:23.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.991+0000 7f187ade8700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf67890 con 0x55b0ecf64290 2023-10-05T19:40:23.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.991+0000 7f187ade8700 1 -- 172.21.15.12:0/704651447 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186400fb90 con 0x55b0ecf64290 2023-10-05T19:40:23.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.991+0000 7f187ade8700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf67890 con 0x55b0ecf305f0 2023-10-05T19:40:23.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.991+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186c014040 con 0x55b0ecf305f0 2023-10-05T19:40:23.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.991+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf67890 con 0x55b0ecf305f0 2023-10-05T19:40:23.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.995+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186c014040 con 0x55b0ecf305f0 2023-10-05T19:40:23.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.995+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf67890 con 0x7f1874016c10 2023-10-05T19:40:23.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.995+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186c014040 con 0x7f1874016c10 2023-10-05T19:40:23.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.995+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf67890 con 0x55b0ecf64290 2023-10-05T19:40:23.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.995+0000 7f187ade8700 1 -- 172.21.15.12:0/704651447 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186400fb90 con 0x55b0ecf64290 2023-10-05T19:40:23.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.995+0000 7f187ade8700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf67890 con 0x55b0ecf5f910 2023-10-05T19:40:23.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.995+0000 7f187b5e9700 1 -- 172.21.15.12:0/704651447 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1874014070 con 0x55b0ecf5f910 2023-10-05T19:40:23.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.995+0000 7f187b5e9700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf67890 con 0x55b0ecf305f0 2023-10-05T19:40:23.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.999+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186c014040 con 0x55b0ecf305f0 2023-10-05T19:40:23.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.999+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf67890 con 0x7f1874016c10 2023-10-05T19:40:23.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.999+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186c014040 con 0x7f1874016c10 2023-10-05T19:40:23.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.999+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf67890 con 0x55b0ecf64290 2023-10-05T19:40:23.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.999+0000 7f187ade8700 1 -- 172.21.15.12:0/704651447 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186400fb90 con 0x55b0ecf64290 2023-10-05T19:40:23.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.999+0000 7f187ade8700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf67890 con 0x7f1874016c10 2023-10-05T19:40:23.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.999+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186c014040 con 0x7f1874016c10 2023-10-05T19:40:23.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:22.999+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf67890 con 0x55b0ecf5f910 2023-10-05T19:40:23.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.003+0000 7f187b5e9700 1 -- 172.21.15.12:0/704651447 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1874014070 con 0x55b0ecf5f910 2023-10-05T19:40:23.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.003+0000 7f187b5e9700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf67890 con 0x55b0ecf64290 2023-10-05T19:40:23.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.003+0000 7f187ade8700 1 -- 172.21.15.12:0/704651447 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186400fb90 con 0x55b0ecf64290 2023-10-05T19:40:23.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.003+0000 7f187ade8700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf67890 con 0x7f1874016c10 2023-10-05T19:40:23.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.003+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f186c014040 con 0x7f1874016c10 2023-10-05T19:40:23.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.003+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf67890 con 0x55b0ecf305f0 2023-10-05T19:40:23.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.003+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f186c014040 con 0x55b0ecf305f0 2023-10-05T19:40:23.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.003+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf67890 con 0x55b0ecf5f910 2023-10-05T19:40:23.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.003+0000 7f187b5e9700 1 -- 172.21.15.12:0/704651447 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f1874014070 con 0x55b0ecf5f910 2023-10-05T19:40:23.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.007+0000 7f187dab4e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:23.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.007+0000 7f187dab4e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:23.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.007+0000 7f187dab4e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:23.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.007+0000 7f187dab4e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:23.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.007+0000 7f187dab4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.007+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf67c40 con 0x55b0ecf305f0 2023-10-05T19:40:23.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.007+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f186c014040 con 0x55b0ecf305f0 2023-10-05T19:40:23.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.007+0000 7f187dab4e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:23.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.007+0000 7f187dab4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.007+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf67ff0 con 0x55b0ecf5f910 2023-10-05T19:40:23.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.007+0000 7f187b5e9700 1 -- 172.21.15.12:0/704651447 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f1874014070 con 0x55b0ecf5f910 2023-10-05T19:40:23.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.007+0000 7f187dab4e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:23.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.007+0000 7f187dab4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.007+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf67110 con 0x55b0ecf305f0 2023-10-05T19:40:23.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.007+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f186c014040 con 0x55b0ecf305f0 2023-10-05T19:40:23.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.007+0000 7f187dab4e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:23.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.007+0000 7f187dab4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.007+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf67120 con 0x55b0ecf64290 2023-10-05T19:40:23.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.007+0000 7f187ade8700 1 -- 172.21.15.12:0/704651447 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f186400fb90 con 0x55b0ecf64290 2023-10-05T19:40:23.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.007+0000 7f187dab4e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:23.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.007+0000 7f187dab4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.007+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf68e90 con 0x7f1874016c10 2023-10-05T19:40:23.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.011+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f186c014040 con 0x7f1874016c10 2023-10-05T19:40:23.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.011+0000 7f187dab4e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:23.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.011+0000 7f187dab4e40 20 zone default found 2023-10-05T19:40:23.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.011+0000 7f187dab4e40 4 Realm: () 2023-10-05T19:40:23.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.011+0000 7f187dab4e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:23.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.011+0000 7f187dab4e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:23.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.011+0000 7f187dab4e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:23.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.011+0000 7f187dab4e40 20 zonegroup default 2023-10-05T19:40:23.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.011+0000 7f187dab4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.011+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf68e90 con 0x7f1874016c10 2023-10-05T19:40:23.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.011+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f186c014040 con 0x7f1874016c10 2023-10-05T19:40:23.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.011+0000 7f187dab4e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:23.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.011+0000 7f187dab4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.011+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf6a810 con 0x55b0ecf64290 2023-10-05T19:40:23.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.011+0000 7f187ade8700 1 -- 172.21.15.12:0/704651447 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f186400fb90 con 0x55b0ecf64290 2023-10-05T19:40:23.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.011+0000 7f187dab4e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:23.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.011+0000 7f187dab4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.011+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf6abc0 con 0x55b0ecf64290 2023-10-05T19:40:23.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.011+0000 7f187ade8700 1 -- 172.21.15.12:0/704651447 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f186400fb90 con 0x55b0ecf64290 2023-10-05T19:40:23.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.011+0000 7f187dab4e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:23.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.011+0000 7f187dab4e40 20 started sync module instance, tier type = 2023-10-05T19:40:23.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.011+0000 7f187dab4e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:23.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.011+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf69ed0 con 0x55b0ecf5f910 2023-10-05T19:40:23.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.015+0000 7f187b5e9700 1 -- 172.21.15.12:0/704651447 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'87 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1874014070 con 0x55b0ecf5f910 2023-10-05T19:40:23.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.015+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94218378189280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf6af70 con 0x55b0ecf5f910 2023-10-05T19:40:23.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.015+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf6b810 con 0x55b0ecf305f0 2023-10-05T19:40:23.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.019+0000 7f187b5e9700 1 -- 172.21.15.12:0/704651447 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94218378189280] v28'88 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1874014070 con 0x55b0ecf5f910 2023-10-05T19:40:23.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.019+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'87 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f186c014040 con 0x55b0ecf305f0 2023-10-05T19:40:23.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.019+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94218378195744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf6a3f0 con 0x55b0ecf305f0 2023-10-05T19:40:23.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.019+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf6c380 con 0x55b0ecf64290 2023-10-05T19:40:23.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.023+0000 7f187ade8700 1 -- 172.21.15.12:0/704651447 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'87 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f186400fb90 con 0x55b0ecf64290 2023-10-05T19:40:23.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.023+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94218378198672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf6cc20 con 0x55b0ecf64290 2023-10-05T19:40:23.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.023+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf6d630 con 0x55b0ecf5f910 2023-10-05T19:40:23.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.023+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94218378195744] v28'88 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f186c014040 con 0x55b0ecf305f0 2023-10-05T19:40:23.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.023+0000 7f187b5e9700 1 -- 172.21.15.12:0/704651447 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'87 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1874014070 con 0x55b0ecf5f910 2023-10-05T19:40:23.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.023+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94218378196848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf6d260 con 0x55b0ecf5f910 2023-10-05T19:40:23.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.023+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf6e5e0 con 0x55b0ecf305f0 2023-10-05T19:40:23.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.023+0000 7f187ade8700 1 -- 172.21.15.12:0/704651447 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94218378198672] v28'88 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f186400fb90 con 0x55b0ecf64290 2023-10-05T19:40:23.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.027+0000 7f187b5e9700 1 -- 172.21.15.12:0/704651447 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94218378196848] v28'88 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1874014070 con 0x55b0ecf5f910 2023-10-05T19:40:23.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.027+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'87 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f186c014040 con 0x55b0ecf305f0 2023-10-05T19:40:23.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.027+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94218378205744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf6e0f0 con 0x55b0ecf305f0 2023-10-05T19:40:23.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.027+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf6eed0 con 0x7f1874016c10 2023-10-05T19:40:23.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.031+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'87 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f186c014040 con 0x7f1874016c10 2023-10-05T19:40:23.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.031+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94218378209104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf6f090 con 0x7f1874016c10 2023-10-05T19:40:23.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.031+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf6f970 con 0x7f1874016c10 2023-10-05T19:40:23.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.031+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94218378205744] v28'88 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f186c014040 con 0x55b0ecf305f0 2023-10-05T19:40:23.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.031+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(95 notify.5 [watch watch cookie 94218378209104] v28'88 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f186c014040 con 0x7f1874016c10 2023-10-05T19:40:23.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.035+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(96 notify.6 [create] v28'87 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f186c014040 con 0x7f1874016c10 2023-10-05T19:40:23.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.035+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94218378200880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf6f090 con 0x7f1874016c10 2023-10-05T19:40:23.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.035+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf6f090 con 0x7f1874016c10 2023-10-05T19:40:23.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.035+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(97 notify.6 [watch watch cookie 94218378200880] v28'88 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f186c014040 con 0x7f1874016c10 2023-10-05T19:40:23.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.035+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(98 notify.7 [create] v28'87 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f186c014040 con 0x7f1874016c10 2023-10-05T19:40:23.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.035+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94218378210208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf6eff0 con 0x7f1874016c10 2023-10-05T19:40:23.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.035+0000 7f187dab4e40 20 add_watcher() i=0 2023-10-05T19:40:23.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.035+0000 7f187dab4e40 20 add_watcher() i=1 2023-10-05T19:40:23.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.035+0000 7f187dab4e40 20 add_watcher() i=2 2023-10-05T19:40:23.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.035+0000 7f187dab4e40 20 add_watcher() i=3 2023-10-05T19:40:23.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.035+0000 7f187dab4e40 20 add_watcher() i=4 2023-10-05T19:40:23.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.035+0000 7f187dab4e40 20 add_watcher() i=5 2023-10-05T19:40:23.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.035+0000 7f187dab4e40 20 add_watcher() i=6 2023-10-05T19:40:23.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.039+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94218378210208] v28'88 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f186c014040 con 0x7f1874016c10 2023-10-05T19:40:23.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.039+0000 7f187dab4e40 20 add_watcher() i=7 2023-10-05T19:40:23.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.039+0000 7f187dab4e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:23.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.039+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf730f0 con 0x55b0ecf305f0 2023-10-05T19:40:23.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.039+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f186c014040 con 0x55b0ecf305f0 2023-10-05T19:40:23.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.039+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94218378214304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf705b0 con 0x55b0ecf305f0 2023-10-05T19:40:23.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.043+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94218378214304] v28'92 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f186c014040 con 0x55b0ecf305f0 2023-10-05T19:40:23.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.043+0000 7f184cfe1700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:23.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.043+0000 7f187dab4e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:23.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.043+0000 7f187dab4e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:23.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.043+0000 7f187dab4e40 1 Processor -- start 2023-10-05T19:40:23.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.043+0000 7f187dab4e40 1 -- start start 2023-10-05T19:40:23.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.043+0000 7f187dab4e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b0ecf7d470 0x55b0ecf7d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:23.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.043+0000 7f187dab4e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0ecf7dd80 con 0x55b0ecf7d470 2023-10-05T19:40:23.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.043+0000 7f187ade8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b0ecf7d470 0x55b0ecf7d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:23.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.043+0000 7f187ade8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b0ecf7d470 0x55b0ecf7d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57508/0 (socket says 172.21.15.12:57508) 2023-10-05T19:40:23.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.043+0000 7f187ade8700 1 -- 172.21.15.12:0/3854750505 learned_addr learned my addr 172.21.15.12:0/3854750505 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:23.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.043+0000 7f187ade8700 1 -- 172.21.15.12:0/3854750505 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0ecf7dec0 con 0x55b0ecf7d470 2023-10-05T19:40:23.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.043+0000 7f187ade8700 1 --2- 172.21.15.12:0/3854750505 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b0ecf7d470 0x55b0ecf7d840 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x55b0ecf2d090 tx=0x7f1864011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adcd375bc0a8926a server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:23.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.043+0000 7f17c7fff700 1 -- 172.21.15.12:0/3854750505 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f186401cc10 con 0x55b0ecf7d470 2023-10-05T19:40:23.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.043+0000 7f17c7fff700 1 -- 172.21.15.12:0/3854750505 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f186401cd90 con 0x55b0ecf7d470 2023-10-05T19:40:23.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.043+0000 7f17c7fff700 1 -- 172.21.15.12:0/3854750505 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1864044440 con 0x55b0ecf7d470 2023-10-05T19:40:23.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.047+0000 7f187dab4e40 1 -- 172.21.15.12:0/3854750505 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b0ecf7d470 msgr2=0x55b0ecf7d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:23.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.047+0000 7f187dab4e40 1 --2- 172.21.15.12:0/3854750505 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b0ecf7d470 0x55b0ecf7d840 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x55b0ecf2d090 tx=0x7f1864011000 comp rx=0 tx=0).stop 2023-10-05T19:40:23.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.047+0000 7f187dab4e40 1 -- 172.21.15.12:0/3854750505 shutdown_connections 2023-10-05T19:40:23.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.047+0000 7f187dab4e40 1 --2- 172.21.15.12:0/3854750505 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b0ecf7d470 0x55b0ecf7d840 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:23.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.047+0000 7f187dab4e40 1 -- 172.21.15.12:0/3854750505 >> 172.21.15.12:0/3854750505 conn(0x55b0ecf78830 msgr2=0x55b0ecf769e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:23.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.047+0000 7f187dab4e40 1 -- 172.21.15.12:0/3854750505 shutdown_connections 2023-10-05T19:40:23.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.047+0000 7f187dab4e40 1 -- 172.21.15.12:0/3854750505 wait complete. 2023-10-05T19:40:23.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.047+0000 7f187dab4e40 1 Processor -- start 2023-10-05T19:40:23.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.047+0000 7f187dab4e40 1 -- start start 2023-10-05T19:40:23.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.047+0000 7f187dab4e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b0ecf7d470 0x55b0ecf82f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:23.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.047+0000 7f187dab4e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0ecf83f40 con 0x55b0ecf7d470 2023-10-05T19:40:23.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.047+0000 7f187ade8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b0ecf7d470 0x55b0ecf82f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:23.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.047+0000 7f187ade8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b0ecf7d470 0x55b0ecf82f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57524/0 (socket says 172.21.15.12:57524) 2023-10-05T19:40:23.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.047+0000 7f187ade8700 1 -- 172.21.15.12:0/1623366266 learned_addr learned my addr 172.21.15.12:0/1623366266 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:23.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.047+0000 7f187ade8700 1 -- 172.21.15.12:0/1623366266 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f186402e040 con 0x55b0ecf7d470 2023-10-05T19:40:23.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.051+0000 7f187ade8700 1 --2- 172.21.15.12:0/1623366266 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b0ecf7d470 0x55b0ecf82f40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x55b0ecf77550 tx=0x7f1864044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:23.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.051+0000 7f17c67fc700 1 -- 172.21.15.12:0/1623366266 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1864044b60 con 0x55b0ecf7d470 2023-10-05T19:40:23.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.051+0000 7f17c67fc700 1 -- 172.21.15.12:0/1623366266 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1864044ce0 con 0x55b0ecf7d470 2023-10-05T19:40:23.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.051+0000 7f187dab4e40 1 -- 172.21.15.12:0/1623366266 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b0ecf83540 con 0x55b0ecf7d470 2023-10-05T19:40:23.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.051+0000 7f17c67fc700 1 -- 172.21.15.12:0/1623366266 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1864057460 con 0x55b0ecf7d470 2023-10-05T19:40:23.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.051+0000 7f187dab4e40 1 -- 172.21.15.12:0/1623366266 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b0ecf839e0 con 0x55b0ecf7d470 2023-10-05T19:40:23.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.051+0000 7f17c67fc700 1 -- 172.21.15.12:0/1623366266 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f1864010420 con 0x55b0ecf7d470 2023-10-05T19:40:23.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.051+0000 7f17c67fc700 1 --2- 172.21.15.12:0/1623366266 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f17bc030c00 0x7f17bc0330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:23.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.051+0000 7f17c67fc700 1 -- 172.21.15.12:0/1623366266 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f1864058c80 con 0x55b0ecf7d470 2023-10-05T19:40:23.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.051+0000 7f187dab4e40 5 note: GC not initialized 2023-10-05T19:40:23.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.051+0000 7f187bdea700 1 --2- 172.21.15.12:0/1623366266 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f17bc030c00 0x7f17bc0330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:23.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.055+0000 7f187bdea700 1 --2- 172.21.15.12:0/1623366266 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f17bc030c00 0x7f17bc0330b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x55b0ecf6f220 tx=0x7f186c00a9b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:23.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.055+0000 7f17bb7fe700 20 reqs_thread_entry: start 2023-10-05T19:40:23.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.055+0000 7f17b97fa700 20 reqs_thread_entry: start 2023-10-05T19:40:23.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.055+0000 7f187dab4e40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:23.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.055+0000 7f187dab4e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:23.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.055+0000 7f17a77fe700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:53 2023 2023-10-05T19:40:23.065 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:23.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.055+0000 7f17a6ffd700 20 reqs_thread_entry: start 2023-10-05T19:40:23.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.055+0000 7f187dab4e40 20 get_system_obj_state: rctx=0x7fff1c2f7508 obj=default.rgw.meta:users.uid:foo state=0x55b0ece030b0 s->prefetch_data=0 2023-10-05T19:40:23.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.055+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecfeb770 con 0x55b0ecf305f0 2023-10-05T19:40:23.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.055+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(102 foo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7f186c014040 con 0x55b0ecf305f0 2023-10-05T19:40:23.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.055+0000 7f187dab4e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:23.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.055+0000 7f187dab4e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:23.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.055+0000 7f187dab4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.055+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecfeb730 con 0x55b0ecf305f0 2023-10-05T19:40:23.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.059+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(103 foo [call,call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv11 ondisk = 0) v8 ==== 273+0+480 (crc 0 0 0) 0x7f186c014040 con 0x55b0ecf305f0 2023-10-05T19:40:23.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.059+0000 7f187dab4e40 20 rados_obj.operate() r=0 bl.length=357 2023-10-05T19:40:23.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.059+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:104 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecfeb730 con 0x55b0ecf5f910 2023-10-05T19:40:23.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.059+0000 7f187b5e9700 1 -- 172.21.15.12:0/704651447 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(104 foo.buckets [call out=183b] v0'0 uv2 ondisk = 0) v8 ==== 155+0+183 (crc 0 0 0) 0x7f1874014070 con 0x55b0ecf5f910 2023-10-05T19:40:23.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.059+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:105 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94218378189280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecfeb730 con 0x55b0ecf5f910 2023-10-05T19:40:23.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.063+0000 7f187b5e9700 1 -- 172.21.15.12:0/704651447 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(105 notify.0 [watch unwatch cookie 94218378189280] v28'89 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1874014070 con 0x55b0ecf5f910 2023-10-05T19:40:23.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.063+0000 7f187dab4e40 20 remove_watcher() i=0 2023-10-05T19:40:23.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.063+0000 7f187dab4e40 2 removed watcher, disabling cache 2023-10-05T19:40:23.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.063+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:106 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94218378195744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecfeb730 con 0x55b0ecf305f0 2023-10-05T19:40:23.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.063+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(106 notify.1 [watch unwatch cookie 94218378195744] v28'89 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f186c014040 con 0x55b0ecf305f0 2023-10-05T19:40:23.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.063+0000 7f187dab4e40 20 remove_watcher() i=1 2023-10-05T19:40:23.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.063+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:107 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94218378198672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecfeb730 con 0x55b0ecf64290 2023-10-05T19:40:23.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.067+0000 7f187ade8700 1 -- 172.21.15.12:0/704651447 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(107 notify.2 [watch unwatch cookie 94218378198672] v28'89 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f186404a020 con 0x55b0ecf64290 2023-10-05T19:40:23.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.067+0000 7f187dab4e40 20 remove_watcher() i=2 2023-10-05T19:40:23.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.067+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:108 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94218378196848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecfeb730 con 0x55b0ecf5f910 2023-10-05T19:40:23.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.071+0000 7f187b5e9700 1 -- 172.21.15.12:0/704651447 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(108 notify.3 [watch unwatch cookie 94218378196848] v28'89 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1874014070 con 0x55b0ecf5f910 2023-10-05T19:40:23.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.071+0000 7f187dab4e40 20 remove_watcher() i=3 2023-10-05T19:40:23.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.071+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:109 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94218378205744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecfeb730 con 0x55b0ecf305f0 2023-10-05T19:40:23.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.071+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(109 notify.4 [watch unwatch cookie 94218378205744] v28'89 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f186c014040 con 0x55b0ecf305f0 2023-10-05T19:40:23.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.071+0000 7f187dab4e40 20 remove_watcher() i=4 2023-10-05T19:40:23.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.071+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:110 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94218378209104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecfeb730 con 0x7f1874016c10 2023-10-05T19:40:23.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.075+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(110 notify.5 [watch unwatch cookie 94218378209104] v28'89 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f186c014040 con 0x7f1874016c10 2023-10-05T19:40:23.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.075+0000 7f187dab4e40 20 remove_watcher() i=5 2023-10-05T19:40:23.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.075+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:111 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94218378200880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecfeb730 con 0x7f1874016c10 2023-10-05T19:40:23.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.079+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(111 notify.6 [watch unwatch cookie 94218378200880] v28'89 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f186c014040 con 0x7f1874016c10 2023-10-05T19:40:23.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.079+0000 7f187dab4e40 20 remove_watcher() i=6 2023-10-05T19:40:23.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.079+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:112 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94218378210208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecfeb730 con 0x7f1874016c10 2023-10-05T19:40:23.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.079+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(112 notify.7 [watch unwatch cookie 94218378210208] v28'89 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f186c014040 con 0x7f1874016c10 2023-10-05T19:40:23.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.079+0000 7f187dab4e40 20 remove_watcher() i=7 2023-10-05T19:40:23.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.083+0000 7f184cfe1700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:23.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.083+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:113 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94218378214304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b0ecf6ec40 con 0x55b0ecf305f0 2023-10-05T19:40:23.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.087+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(113 data_loggenerations_metadata [watch unwatch cookie 94218378214304] v28'93 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f186c014040 con 0x55b0ecf305f0 2023-10-05T19:40:23.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.087+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f1874016c10 msgr2=0x7f18740142c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:23.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.087+0000 7f187dab4e40 1 --2- 172.21.15.12:0/704651447 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f1874016c10 0x7f18740142c0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:23.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.087+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55b0ecf305f0 msgr2=0x55b0ecf5d890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:23.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.087+0000 7f187dab4e40 1 --2- 172.21.15.12:0/704651447 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55b0ecf305f0 0x55b0ecf5d890 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:23.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.087+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55b0ecf5f910 msgr2=0x55b0ecf61cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:23.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.087+0000 7f187dab4e40 1 --2- 172.21.15.12:0/704651447 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55b0ecf5f910 0x55b0ecf61cf0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:23.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.087+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55b0ecf64290 msgr2=0x55b0ecf66670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:23.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.087+0000 7f187dab4e40 1 --2- 172.21.15.12:0/704651447 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55b0ecf64290 0x55b0ecf66670 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:23.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.087+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f185c030bf0 msgr2=0x7f185c0330a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:23.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.087+0000 7f187dab4e40 1 --2- 172.21.15.12:0/704651447 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f185c030bf0 0x7f185c0330a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1864000f80 tx=0x7f1864014000 comp rx=0 tx=0).stop 2023-10-05T19:40:23.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.087+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b0ecf2c010 msgr2=0x55b0ecf51d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:23.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.087+0000 7f187bdea700 1 -- 172.21.15.12:0/704651447 reap_dead start 2023-10-05T19:40:23.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.087+0000 7f187dab4e40 1 --2- 172.21.15.12:0/704651447 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b0ecf2c010 0x55b0ecf51d70 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f187400b960 tx=0x7f1874014000 comp rx=0 tx=0).stop 2023-10-05T19:40:23.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.087+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 shutdown_connections 2023-10-05T19:40:23.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.087+0000 7f187dab4e40 1 --2- 172.21.15.12:0/704651447 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b0ecf2c010 0x55b0ecf51d70 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:23.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.087+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 >> 172.21.15.12:0/704651447 conn(0x55b0ece9f860 msgr2=0x55b0ecf455d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:23.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.087+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 shutdown_connections 2023-10-05T19:40:23.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.087+0000 7f187dab4e40 1 -- 172.21.15.12:0/704651447 wait complete. 2023-10-05T19:40:23.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.087+0000 7f187dab4e40 1 -- 172.21.15.12:0/1623366266 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f17bc030c00 msgr2=0x7f17bc0330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:23.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.087+0000 7f187dab4e40 1 --2- 172.21.15.12:0/1623366266 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f17bc030c00 0x7f17bc0330b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x55b0ecf6f220 tx=0x7f186c00a9b0 comp rx=0 tx=0).stop 2023-10-05T19:40:23.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.087+0000 7f187dab4e40 1 -- 172.21.15.12:0/1623366266 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b0ecf7d470 msgr2=0x55b0ecf82f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:23.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.087+0000 7f187dab4e40 1 --2- 172.21.15.12:0/1623366266 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b0ecf7d470 0x55b0ecf82f40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x55b0ecf77550 tx=0x7f1864044000 comp rx=0 tx=0).stop 2023-10-05T19:40:23.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.087+0000 7f187dab4e40 1 -- 172.21.15.12:0/1623366266 shutdown_connections 2023-10-05T19:40:23.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.087+0000 7f187dab4e40 1 --2- 172.21.15.12:0/1623366266 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f17bc030c00 0x7f17bc0330b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:23.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.087+0000 7f187dab4e40 1 --2- 172.21.15.12:0/1623366266 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b0ecf7d470 0x55b0ecf82f40 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:23.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.087+0000 7f187dab4e40 1 -- 172.21.15.12:0/1623366266 >> 172.21.15.12:0/1623366266 conn(0x55b0ecf78830 msgr2=0x55b0ecf76930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:23.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.087+0000 7f187dab4e40 1 -- 172.21.15.12:0/1623366266 shutdown_connections 2023-10-05T19:40:23.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.087+0000 7f187dab4e40 1 -- 172.21.15.12:0/1623366266 wait complete. 2023-10-05T19:40:23.098 INFO:teuthology.orchestra.run.smithi012.stdout:[ 2023-10-05T19:40:23.099 INFO:teuthology.orchestra.run.smithi012.stdout: "myfoo" 2023-10-05T19:40:23.099 INFO:teuthology.orchestra.run.smithi012.stdout:] 2023-10-05T19:40:23.102 DEBUG:tasks.util.rgw: json result: ['myfoo'] 2023-10-05T19:40:23.105 INFO:tasks.radosgw_admin:RL: list_buckets foo bytes_out=0 bytes_in=0 failed=False 2023-10-05T19:40:23.106 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'list'] 2023-10-05T19:40:23.106 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'list'] 2023-10-05T19:40:23.106 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket list 2023-10-05T19:40:23.142 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:23.142 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:23.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.135+0000 7f0968391e40 1 Processor -- start 2023-10-05T19:40:23.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.135+0000 7f0968391e40 1 -- start start 2023-10-05T19:40:23.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.135+0000 7f0968391e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x561c7987be00 0x561c7987c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:23.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.135+0000 7f0968391e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x561c797c8c00 con 0x561c7987c710 2023-10-05T19:40:23.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.135+0000 7f0968391e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x561c79785ee0 con 0x561c7987be00 2023-10-05T19:40:23.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.135+0000 7f0965ec6700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x561c7987be00 0x561c7987c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:23.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.135+0000 7f09656c5700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x561c7987c710 0x561c7987d1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:35468/0 (socket says 172.21.15.12:35468) 2023-10-05T19:40:23.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.135+0000 7f09656c5700 1 -- 172.21.15.12:0/2515824928 learned_addr learned my addr 172.21.15.12:0/2515824928 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:23.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.135+0000 7f0965ec6700 1 -- 172.21.15.12:0/2515824928 >> v1:172.21.15.8:6789/0 conn(0x561c7987c710 legacy=0x561c7987d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:23.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.135+0000 7f0965ec6700 1 -- 172.21.15.12:0/2515824928 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c7987d8c0 con 0x561c7987be00 2023-10-05T19:40:23.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.135+0000 7f0965ec6700 1 --2- 172.21.15.12:0/2515824928 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c7987be00 0x561c7987c1d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f095800a5d0 tx=0x7f0958005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f115dc17dffd203 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:23.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.135+0000 7f0964ec4700 1 -- 172.21.15.12:0/2515824928 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09580086a0 con 0x561c7987be00 2023-10-05T19:40:23.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.135+0000 7f0964ec4700 1 -- 172.21.15.12:0/2515824928 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0958008800 con 0x561c7987be00 2023-10-05T19:40:23.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.135+0000 7f0964ec4700 1 -- 172.21.15.12:0/2515824928 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0958008ad0 con 0x561c7987be00 2023-10-05T19:40:23.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.139+0000 7f0968391e40 1 -- 172.21.15.12:0/2515824928 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c7987be00 msgr2=0x561c7987c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:23.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.139+0000 7f0968391e40 1 --2- 172.21.15.12:0/2515824928 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c7987be00 0x561c7987c1d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f095800a5d0 tx=0x7f0958005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:23.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.139+0000 7f0968391e40 1 -- 172.21.15.12:0/2515824928 shutdown_connections 2023-10-05T19:40:23.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.139+0000 7f0968391e40 1 --2- 172.21.15.12:0/2515824928 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c7987be00 0x561c7987c1d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:23.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.139+0000 7f0968391e40 1 -- 172.21.15.12:0/2515824928 >> 172.21.15.12:0/2515824928 conn(0x561c79877820 msgr2=0x561c79879c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:23.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.139+0000 7f0968391e40 1 -- 172.21.15.12:0/2515824928 shutdown_connections 2023-10-05T19:40:23.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.139+0000 7f0968391e40 1 -- 172.21.15.12:0/2515824928 wait complete. 2023-10-05T19:40:23.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.139+0000 7f0968391e40 1 Processor -- start 2023-10-05T19:40:23.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.139+0000 7f0968391e40 1 -- start start 2023-10-05T19:40:23.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.139+0000 7f0968391e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c7987be00 0x561c7989ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:23.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.139+0000 7f0968391e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c79749e30 con 0x561c7987be00 2023-10-05T19:40:23.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.139+0000 7f0965ec6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c7987be00 0x561c7989ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:23.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.139+0000 7f0965ec6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c7987be00 0x561c7989ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57538/0 (socket says 172.21.15.12:57538) 2023-10-05T19:40:23.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.139+0000 7f0965ec6700 1 -- 172.21.15.12:0/3713561358 learned_addr learned my addr 172.21.15.12:0/3713561358 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:23.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.139+0000 7f0965ec6700 1 -- 172.21.15.12:0/3713561358 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0958005cf0 con 0x561c7987be00 2023-10-05T19:40:23.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.139+0000 7f0965ec6700 1 --2- 172.21.15.12:0/3713561358 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c7987be00 0x561c7989ae30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f095800a520 tx=0x7f0958015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=124632b9b43fea84 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:23.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.139+0000 7f095f7fe700 1 -- 172.21.15.12:0/3713561358 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0958008970 con 0x561c7987be00 2023-10-05T19:40:23.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.139+0000 7f095f7fe700 1 -- 172.21.15.12:0/3713561358 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0958008ad0 con 0x561c7987be00 2023-10-05T19:40:23.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.139+0000 7f095f7fe700 1 -- 172.21.15.12:0/3713561358 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0958008dc0 con 0x561c7987be00 2023-10-05T19:40:23.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.143+0000 7f0968391e40 1 -- 172.21.15.12:0/3713561358 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c7987be00 msgr2=0x561c7989ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:23.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.143+0000 7f0968391e40 1 --2- 172.21.15.12:0/3713561358 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c7987be00 0x561c7989ae30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f095800a520 tx=0x7f0958015000 comp rx=0 tx=0).stop 2023-10-05T19:40:23.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.143+0000 7f0968391e40 1 -- 172.21.15.12:0/3713561358 shutdown_connections 2023-10-05T19:40:23.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.143+0000 7f0968391e40 1 --2- 172.21.15.12:0/3713561358 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c7987be00 0x561c7989ae30 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:23.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.143+0000 7f0968391e40 1 -- 172.21.15.12:0/3713561358 >> 172.21.15.12:0/3713561358 conn(0x561c79877820 msgr2=0x561c798969f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:23.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.143+0000 7f0968391e40 1 -- 172.21.15.12:0/3713561358 shutdown_connections 2023-10-05T19:40:23.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.143+0000 7f0968391e40 1 -- 172.21.15.12:0/3713561358 wait complete. 2023-10-05T19:40:23.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.143+0000 7f0968391e40 1 Processor -- start 2023-10-05T19:40:23.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.143+0000 7f0968391e40 1 -- start start 2023-10-05T19:40:23.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.143+0000 7f0968391e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c7987be00 0x561c797f1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:23.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.143+0000 7f0968391e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7971ced0 con 0x561c7987be00 2023-10-05T19:40:23.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.143+0000 7f0965ec6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c7987be00 0x561c797f1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:23.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.143+0000 7f0965ec6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c7987be00 0x561c797f1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57552/0 (socket says 172.21.15.12:57552) 2023-10-05T19:40:23.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.143+0000 7f0965ec6700 1 -- 172.21.15.12:0/2390960790 learned_addr learned my addr 172.21.15.12:0/2390960790 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:23.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.143+0000 7f0965ec6700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c7987d8c0 con 0x561c7987be00 2023-10-05T19:40:23.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.143+0000 7f0965ec6700 1 --2- 172.21.15.12:0/2390960790 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c7987be00 0x561c797f1ac0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0958005540 tx=0x7f0958003050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:23.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.143+0000 7f095dffb700 1 -- 172.21.15.12:0/2390960790 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0958012bf0 con 0x561c7987be00 2023-10-05T19:40:23.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.143+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561c797f2000 con 0x561c7987be00 2023-10-05T19:40:23.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.143+0000 7f095dffb700 1 -- 172.21.15.12:0/2390960790 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f095802a070 con 0x561c7987be00 2023-10-05T19:40:23.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.143+0000 7f095dffb700 1 -- 172.21.15.12:0/2390960790 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0958012e90 con 0x561c7987be00 2023-10-05T19:40:23.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.143+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561c797f03c0 con 0x561c7987be00 2023-10-05T19:40:23.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.147+0000 7f095dffb700 1 -- 172.21.15.12:0/2390960790 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f0958019070 con 0x561c7987be00 2023-10-05T19:40:23.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.147+0000 7f095dffb700 1 --2- 172.21.15.12:0/2390960790 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f0948030c00 0x7f09480330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:23.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.147+0000 7f095dffb700 1 -- 172.21.15.12:0/2390960790 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f095802c070 con 0x561c7987be00 2023-10-05T19:40:23.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.147+0000 7f09656c5700 1 --2- 172.21.15.12:0/2390960790 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f0948030c00 0x7f09480330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:23.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.147+0000 7f0968391e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.147+0000 7f0968391e40 1 --2- 172.21.15.12:0/2390960790 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x561c7987c710 0x561c798af6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:23.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.147+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798afde0 con 0x561c7987c710 2023-10-05T19:40:23.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.147+0000 7f09666c7700 1 --2- 172.21.15.12:0/2390960790 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x561c7987c710 0x561c798af6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:23.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.147+0000 7f09656c5700 1 --2- 172.21.15.12:0/2390960790 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f0948030c00 0x7f09480330b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0950001e90 tx=0x7f09500013b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:23.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.147+0000 7f09666c7700 1 --2- 172.21.15.12:0/2390960790 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x561c7987c710 0x561c798af6b0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:23.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.147+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f096000a5a0 con 0x561c7987c710 2023-10-05T19:40:23.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.147+0000 7f0968391e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:23.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.147+0000 7f0968391e40 20 realm 2023-10-05T19:40:23.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.147+0000 7f0968391e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.147+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798acd10 con 0x561c7987c710 2023-10-05T19:40:23.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.147+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f096000a5a0 con 0x561c7987c710 2023-10-05T19:40:23.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.147+0000 7f0968391e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:23.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.147+0000 7f0968391e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:23.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.147+0000 7f0968391e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.147+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b08c0 con 0x561c7987c710 2023-10-05T19:40:23.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.151+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f096000a5a0 con 0x561c7987c710 2023-10-05T19:40:23.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.151+0000 7f0968391e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:23.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.151+0000 7f0968391e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.151+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b04a0 con 0x561c7987c710 2023-10-05T19:40:23.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.151+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f096000a5a0 con 0x561c7987c710 2023-10-05T19:40:23.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.151+0000 7f0968391e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:23.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.151+0000 7f0968391e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.151+0000 7f0968391e40 1 --2- 172.21.15.12:0/2390960790 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x561c798b1850 0x561c798b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:23.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.151+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b43a0 con 0x561c798b1850 2023-10-05T19:40:23.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.151+0000 7f0965ec6700 1 --2- 172.21.15.12:0/2390960790 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x561c798b1850 0x561c798b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:23.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.151+0000 7f0965ec6700 1 --2- 172.21.15.12:0/2390960790 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x561c798b1850 0x561c798b3c30 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:23.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.151+0000 7f0965ec6700 1 -- 172.21.15.12:0/2390960790 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f0958022020 con 0x561c798b1850 2023-10-05T19:40:23.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.151+0000 7f0968391e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:23.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.151+0000 7f0968391e40 20 searching for the correct realm 2023-10-05T19:40:23.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.151+0000 7f0968391e40 1 --2- 172.21.15.12:0/2390960790 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x561c798b61d0 0x561c798b85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:23.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.151+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b8af0 con 0x561c798b61d0 2023-10-05T19:40:23.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.151+0000 7f09656c5700 1 --2- 172.21.15.12:0/2390960790 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x561c798b61d0 0x561c798b85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:23.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.151+0000 7f09656c5700 1 --2- 172.21.15.12:0/2390960790 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x561c798b61d0 0x561c798b85b0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:23.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.155+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f0950006070 con 0x561c798b61d0 2023-10-05T19:40:23.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.155+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b8af0 con 0x561c798b1850 2023-10-05T19:40:23.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.155+0000 7f0965ec6700 1 -- 172.21.15.12:0/2390960790 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0958022020 con 0x561c798b1850 2023-10-05T19:40:23.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.155+0000 7f0965ec6700 1 --2- 172.21.15.12:0/2390960790 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f0958007030 0x7f0958001620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:23.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.155+0000 7f0965ec6700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b8af0 con 0x7f0958007030 2023-10-05T19:40:23.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.155+0000 7f09666c7700 1 --2- 172.21.15.12:0/2390960790 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f0958007030 0x7f0958001620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:23.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.155+0000 7f09666c7700 1 --2- 172.21.15.12:0/2390960790 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f0958007030 0x7f0958001620 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:23.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.159+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f096000a5a0 con 0x7f0958007030 2023-10-05T19:40:23.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.159+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b8af0 con 0x561c7987c710 2023-10-05T19:40:23.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.159+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f096000a5a0 con 0x561c7987c710 2023-10-05T19:40:23.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.159+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b8af0 con 0x561c7987c710 2023-10-05T19:40:23.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.159+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f096000a5a0 con 0x561c7987c710 2023-10-05T19:40:23.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.159+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b8af0 con 0x7f0958007030 2023-10-05T19:40:23.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.159+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f096000a5a0 con 0x7f0958007030 2023-10-05T19:40:23.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.159+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b8af0 con 0x7f0958007030 2023-10-05T19:40:23.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.159+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f096000a5a0 con 0x7f0958007030 2023-10-05T19:40:23.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.159+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b8af0 con 0x561c798b1850 2023-10-05T19:40:23.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.163+0000 7f0965ec6700 1 -- 172.21.15.12:0/2390960790 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0958022020 con 0x561c798b1850 2023-10-05T19:40:23.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.163+0000 7f0965ec6700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b8af0 con 0x561c798b61d0 2023-10-05T19:40:23.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.163+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0950006070 con 0x561c798b61d0 2023-10-05T19:40:23.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.163+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b8af0 con 0x561c7987c710 2023-10-05T19:40:23.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.163+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f096000a5a0 con 0x561c7987c710 2023-10-05T19:40:23.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.163+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b8af0 con 0x7f0958007030 2023-10-05T19:40:23.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.163+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f096000a5a0 con 0x7f0958007030 2023-10-05T19:40:23.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.163+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b8af0 con 0x561c7987c710 2023-10-05T19:40:23.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.163+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f096000a5a0 con 0x561c7987c710 2023-10-05T19:40:23.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.163+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b8af0 con 0x561c798b1850 2023-10-05T19:40:23.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.167+0000 7f0965ec6700 1 -- 172.21.15.12:0/2390960790 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0958022020 con 0x561c798b1850 2023-10-05T19:40:23.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.167+0000 7f0965ec6700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b8af0 con 0x561c798b61d0 2023-10-05T19:40:23.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.167+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0950006070 con 0x561c798b61d0 2023-10-05T19:40:23.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.167+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b8af0 con 0x561c798b61d0 2023-10-05T19:40:23.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.167+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0950006070 con 0x561c798b61d0 2023-10-05T19:40:23.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.167+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b8af0 con 0x7f0958007030 2023-10-05T19:40:23.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.167+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f096000a5a0 con 0x7f0958007030 2023-10-05T19:40:23.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.167+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b8af0 con 0x561c798b61d0 2023-10-05T19:40:23.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.167+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0950006070 con 0x561c798b61d0 2023-10-05T19:40:23.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.167+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b8af0 con 0x561c798b61d0 2023-10-05T19:40:23.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.167+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0950006070 con 0x561c798b61d0 2023-10-05T19:40:23.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.167+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b8af0 con 0x561c7987c710 2023-10-05T19:40:23.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.171+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f096000a5a0 con 0x561c7987c710 2023-10-05T19:40:23.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.171+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b8af0 con 0x561c7987c710 2023-10-05T19:40:23.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.171+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f096000a5a0 con 0x561c7987c710 2023-10-05T19:40:23.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.171+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b8af0 con 0x7f0958007030 2023-10-05T19:40:23.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.171+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f096000a5a0 con 0x7f0958007030 2023-10-05T19:40:23.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.171+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b8af0 con 0x561c798b61d0 2023-10-05T19:40:23.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.171+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0950006070 con 0x561c798b61d0 2023-10-05T19:40:23.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.171+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b8af0 con 0x561c798b1850 2023-10-05T19:40:23.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.171+0000 7f0965ec6700 1 -- 172.21.15.12:0/2390960790 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0958022020 con 0x561c798b1850 2023-10-05T19:40:23.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.171+0000 7f0965ec6700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b8af0 con 0x561c7987c710 2023-10-05T19:40:23.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.175+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f096000a5a0 con 0x561c7987c710 2023-10-05T19:40:23.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.175+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b8af0 con 0x7f0958007030 2023-10-05T19:40:23.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.175+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f096000a5a0 con 0x7f0958007030 2023-10-05T19:40:23.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.175+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b8af0 con 0x561c798b61d0 2023-10-05T19:40:23.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.175+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0950006070 con 0x561c798b61d0 2023-10-05T19:40:23.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.175+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b8af0 con 0x7f0958007030 2023-10-05T19:40:23.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.175+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f096000a5a0 con 0x7f0958007030 2023-10-05T19:40:23.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.175+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b8af0 con 0x561c798b1850 2023-10-05T19:40:23.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.179+0000 7f0965ec6700 1 -- 172.21.15.12:0/2390960790 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0958022020 con 0x561c798b1850 2023-10-05T19:40:23.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.179+0000 7f0965ec6700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b8af0 con 0x561c798b61d0 2023-10-05T19:40:23.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.179+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0950006070 con 0x561c798b61d0 2023-10-05T19:40:23.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.179+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b8af0 con 0x7f0958007030 2023-10-05T19:40:23.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.179+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f096000a5a0 con 0x7f0958007030 2023-10-05T19:40:23.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.179+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b8af0 con 0x561c7987c710 2023-10-05T19:40:23.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.179+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f096000a5a0 con 0x561c7987c710 2023-10-05T19:40:23.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.179+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b8af0 con 0x561c798b1850 2023-10-05T19:40:23.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.179+0000 7f0965ec6700 1 -- 172.21.15.12:0/2390960790 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f0958022020 con 0x561c798b1850 2023-10-05T19:40:23.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.179+0000 7f0968391e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:23.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.179+0000 7f0968391e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:23.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.179+0000 7f0968391e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:23.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.179+0000 7f0968391e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:23.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.179+0000 7f0968391e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.179+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b4f80 con 0x561c7987c710 2023-10-05T19:40:23.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.183+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f096000a5a0 con 0x561c7987c710 2023-10-05T19:40:23.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.183+0000 7f0968391e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:23.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.183+0000 7f0968391e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.183+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b58d0 con 0x561c7987c710 2023-10-05T19:40:23.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.183+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f096000a5a0 con 0x561c7987c710 2023-10-05T19:40:23.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.183+0000 7f0968391e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:23.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.183+0000 7f0968391e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.183+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b4b40 con 0x561c798b61d0 2023-10-05T19:40:23.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.183+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f0950006070 con 0x561c798b61d0 2023-10-05T19:40:23.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.183+0000 7f0968391e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:23.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.183+0000 7f0968391e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.183+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b9420 con 0x561c798b61d0 2023-10-05T19:40:23.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.183+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f0950006070 con 0x561c798b61d0 2023-10-05T19:40:23.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.183+0000 7f0968391e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:23.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.183+0000 7f0968391e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.183+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b97d0 con 0x561c7987c710 2023-10-05T19:40:23.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.187+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f096000a5a0 con 0x561c7987c710 2023-10-05T19:40:23.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.187+0000 7f0968391e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:23.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.187+0000 7f0968391e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.187+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b9b80 con 0x561c7987c710 2023-10-05T19:40:23.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.187+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f096000a5a0 con 0x561c7987c710 2023-10-05T19:40:23.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.187+0000 7f0968391e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:23.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.187+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798bab90 con 0x561c798b61d0 2023-10-05T19:40:23.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.187+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f0950006070 con 0x561c798b61d0 2023-10-05T19:40:23.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.187+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798bab90 con 0x561c798b1850 2023-10-05T19:40:23.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.191+0000 7f0965ec6700 1 -- 172.21.15.12:0/2390960790 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0958022020 con 0x561c798b1850 2023-10-05T19:40:23.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.191+0000 7f0965ec6700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798bab90 con 0x7f0958007030 2023-10-05T19:40:23.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.191+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f096000a5a0 con 0x7f0958007030 2023-10-05T19:40:23.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.191+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b97d0 con 0x561c7987c710 2023-10-05T19:40:23.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.191+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f096000a5a0 con 0x561c7987c710 2023-10-05T19:40:23.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.191+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b97d0 con 0x561c7987c710 2023-10-05T19:40:23.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.191+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f096000a5a0 con 0x561c7987c710 2023-10-05T19:40:23.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.191+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b97d0 con 0x7f0958007030 2023-10-05T19:40:23.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.195+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f096000a5a0 con 0x7f0958007030 2023-10-05T19:40:23.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.195+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b97d0 con 0x7f0958007030 2023-10-05T19:40:23.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.195+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f096000a5a0 con 0x7f0958007030 2023-10-05T19:40:23.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.195+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b97d0 con 0x561c798b1850 2023-10-05T19:40:23.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.195+0000 7f0965ec6700 1 -- 172.21.15.12:0/2390960790 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0958022020 con 0x561c798b1850 2023-10-05T19:40:23.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.195+0000 7f0965ec6700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b97d0 con 0x561c798b61d0 2023-10-05T19:40:23.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.195+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0950006070 con 0x561c798b61d0 2023-10-05T19:40:23.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.195+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b97d0 con 0x561c7987c710 2023-10-05T19:40:23.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.195+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f096000a5a0 con 0x561c7987c710 2023-10-05T19:40:23.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.195+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b97d0 con 0x7f0958007030 2023-10-05T19:40:23.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.199+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f096000a5a0 con 0x7f0958007030 2023-10-05T19:40:23.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.199+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b97d0 con 0x561c7987c710 2023-10-05T19:40:23.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.199+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f096000a5a0 con 0x561c7987c710 2023-10-05T19:40:23.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.199+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b97d0 con 0x561c798b1850 2023-10-05T19:40:23.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.199+0000 7f0965ec6700 1 -- 172.21.15.12:0/2390960790 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0958022020 con 0x561c798b1850 2023-10-05T19:40:23.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.199+0000 7f0965ec6700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b97d0 con 0x561c798b61d0 2023-10-05T19:40:23.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.199+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0950006070 con 0x561c798b61d0 2023-10-05T19:40:23.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.199+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b97d0 con 0x561c798b61d0 2023-10-05T19:40:23.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.203+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0950006070 con 0x561c798b61d0 2023-10-05T19:40:23.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.203+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b97d0 con 0x7f0958007030 2023-10-05T19:40:23.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.203+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f096000a5a0 con 0x7f0958007030 2023-10-05T19:40:23.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.203+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b97d0 con 0x561c798b61d0 2023-10-05T19:40:23.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.203+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0950006070 con 0x561c798b61d0 2023-10-05T19:40:23.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.203+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b97d0 con 0x561c798b61d0 2023-10-05T19:40:23.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.203+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0950006070 con 0x561c798b61d0 2023-10-05T19:40:23.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.203+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b97d0 con 0x561c7987c710 2023-10-05T19:40:23.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.203+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f096000a5a0 con 0x561c7987c710 2023-10-05T19:40:23.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.203+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b97d0 con 0x561c7987c710 2023-10-05T19:40:23.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.207+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f096000a5a0 con 0x561c7987c710 2023-10-05T19:40:23.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.207+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b97d0 con 0x7f0958007030 2023-10-05T19:40:23.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.207+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f096000a5a0 con 0x7f0958007030 2023-10-05T19:40:23.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.207+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b97d0 con 0x561c798b61d0 2023-10-05T19:40:23.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.207+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0950006070 con 0x561c798b61d0 2023-10-05T19:40:23.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.207+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b97d0 con 0x561c798b1850 2023-10-05T19:40:23.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.207+0000 7f0965ec6700 1 -- 172.21.15.12:0/2390960790 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0958022020 con 0x561c798b1850 2023-10-05T19:40:23.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.207+0000 7f0965ec6700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b97d0 con 0x561c7987c710 2023-10-05T19:40:23.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.207+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f096000a5a0 con 0x561c7987c710 2023-10-05T19:40:23.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.211+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b97d0 con 0x7f0958007030 2023-10-05T19:40:23.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.211+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f096000a5a0 con 0x7f0958007030 2023-10-05T19:40:23.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.211+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b97d0 con 0x561c798b61d0 2023-10-05T19:40:23.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.211+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0950006070 con 0x561c798b61d0 2023-10-05T19:40:23.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.211+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b97d0 con 0x7f0958007030 2023-10-05T19:40:23.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.211+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f096000a5a0 con 0x7f0958007030 2023-10-05T19:40:23.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.211+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b97d0 con 0x561c798b1850 2023-10-05T19:40:23.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.211+0000 7f0965ec6700 1 -- 172.21.15.12:0/2390960790 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0958022020 con 0x561c798b1850 2023-10-05T19:40:23.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.215+0000 7f0965ec6700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b97d0 con 0x561c798b61d0 2023-10-05T19:40:23.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.215+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0950006070 con 0x561c798b61d0 2023-10-05T19:40:23.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.215+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b97d0 con 0x7f0958007030 2023-10-05T19:40:23.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.215+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f096000a5a0 con 0x7f0958007030 2023-10-05T19:40:23.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.215+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b97d0 con 0x561c7987c710 2023-10-05T19:40:23.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.215+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f096000a5a0 con 0x561c7987c710 2023-10-05T19:40:23.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.215+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b97d0 con 0x561c798b1850 2023-10-05T19:40:23.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.215+0000 7f0965ec6700 1 -- 172.21.15.12:0/2390960790 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f0958022020 con 0x561c798b1850 2023-10-05T19:40:23.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.215+0000 7f0968391e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:23.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.215+0000 7f0968391e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:23.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.215+0000 7f0968391e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:23.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.215+0000 7f0968391e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:23.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.215+0000 7f0968391e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.215+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b9b80 con 0x561c7987c710 2023-10-05T19:40:23.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.219+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f096000a5a0 con 0x561c7987c710 2023-10-05T19:40:23.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.219+0000 7f0968391e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:23.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.219+0000 7f0968391e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.219+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b9f30 con 0x561c798b1850 2023-10-05T19:40:23.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.219+0000 7f0965ec6700 1 -- 172.21.15.12:0/2390960790 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f0958022020 con 0x561c798b1850 2023-10-05T19:40:23.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.219+0000 7f0968391e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:23.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.219+0000 7f0968391e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.219+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798ba7d0 con 0x561c7987c710 2023-10-05T19:40:23.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.219+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f096000a5a0 con 0x561c7987c710 2023-10-05T19:40:23.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.219+0000 7f0968391e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:23.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.219+0000 7f0968391e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.219+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798ba7d0 con 0x561c798b61d0 2023-10-05T19:40:23.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.219+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f0950006070 con 0x561c798b61d0 2023-10-05T19:40:23.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.219+0000 7f0968391e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:23.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.219+0000 7f0968391e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.219+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798ba360 con 0x7f0958007030 2023-10-05T19:40:23.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.223+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f096000a5a0 con 0x7f0958007030 2023-10-05T19:40:23.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.223+0000 7f0968391e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:23.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.223+0000 7f0968391e40 20 zone default found 2023-10-05T19:40:23.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.223+0000 7f0968391e40 4 Realm: () 2023-10-05T19:40:23.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.223+0000 7f0968391e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:23.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.223+0000 7f0968391e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:23.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.223+0000 7f0968391e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:23.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.223+0000 7f0968391e40 20 zonegroup default 2023-10-05T19:40:23.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.223+0000 7f0968391e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.223+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798ba360 con 0x7f0958007030 2023-10-05T19:40:23.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.223+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f096000a5a0 con 0x7f0958007030 2023-10-05T19:40:23.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.223+0000 7f0968391e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:23.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.223+0000 7f0968391e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.223+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798ba360 con 0x561c798b61d0 2023-10-05T19:40:23.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.223+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f0950006070 con 0x561c798b61d0 2023-10-05T19:40:23.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.223+0000 7f0968391e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:23.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.223+0000 7f0968391e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.223+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798bc9f0 con 0x561c798b61d0 2023-10-05T19:40:23.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.223+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f0950006070 con 0x561c798b61d0 2023-10-05T19:40:23.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.223+0000 7f0968391e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:23.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.223+0000 7f0968391e40 20 started sync module instance, tier type = 2023-10-05T19:40:23.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.223+0000 7f0968391e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:23.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.223+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798bc0b0 con 0x561c798b1850 2023-10-05T19:40:23.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.227+0000 7f0965ec6700 1 -- 172.21.15.12:0/2390960790 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'90 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0958022020 con 0x561c798b1850 2023-10-05T19:40:23.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.227+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94680298276016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798bbce0 con 0x561c798b1850 2023-10-05T19:40:23.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.227+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798bd780 con 0x561c7987c710 2023-10-05T19:40:23.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.231+0000 7f0965ec6700 1 -- 172.21.15.12:0/2390960790 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94680298276016] v28'91 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0958022020 con 0x561c798b1850 2023-10-05T19:40:23.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.231+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'90 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f096000a5a0 con 0x561c7987c710 2023-10-05T19:40:23.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.231+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94680298278304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798bd6e0 con 0x561c7987c710 2023-10-05T19:40:23.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.231+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798bdfc0 con 0x561c798b61d0 2023-10-05T19:40:23.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.235+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'90 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0950006070 con 0x561c798b61d0 2023-10-05T19:40:23.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.235+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94680298280960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798be8b0 con 0x561c798b61d0 2023-10-05T19:40:23.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.235+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798bf2c0 con 0x561c798b1850 2023-10-05T19:40:23.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.235+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94680298278304] v28'91 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f096000a5a0 con 0x561c7987c710 2023-10-05T19:40:23.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.235+0000 7f0965ec6700 1 -- 172.21.15.12:0/2390960790 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'90 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0958022020 con 0x561c798b1850 2023-10-05T19:40:23.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.235+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94680298286544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798bd310 con 0x561c798b1850 2023-10-05T19:40:23.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.235+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798bfea0 con 0x561c7987c710 2023-10-05T19:40:23.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.235+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94680298280960] v28'91 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0950006070 con 0x561c798b61d0 2023-10-05T19:40:23.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.239+0000 7f0965ec6700 1 -- 172.21.15.12:0/2390960790 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94680298286544] v28'91 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0958022020 con 0x561c798b1850 2023-10-05T19:40:23.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.239+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'90 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f096000a5a0 con 0x561c7987c710 2023-10-05T19:40:23.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.239+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94680298283968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798bd310 con 0x561c7987c710 2023-10-05T19:40:23.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.239+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798bf390 con 0x7f0958007030 2023-10-05T19:40:23.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.243+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'90 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f096000a5a0 con 0x7f0958007030 2023-10-05T19:40:23.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.243+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94680298291648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798bf390 con 0x7f0958007030 2023-10-05T19:40:23.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.243+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798c1200 con 0x7f0958007030 2023-10-05T19:40:23.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.243+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94680298283968] v28'91 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f096000a5a0 con 0x561c7987c710 2023-10-05T19:40:23.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.243+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(95 notify.5 [watch watch cookie 94680298291648] v28'91 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f096000a5a0 con 0x7f0958007030 2023-10-05T19:40:23.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.243+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(96 notify.6 [create] v28'90 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f096000a5a0 con 0x7f0958007030 2023-10-05T19:40:23.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.243+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94680298292752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798c0cd0 con 0x7f0958007030 2023-10-05T19:40:23.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.243+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798c16e0 con 0x7f0958007030 2023-10-05T19:40:23.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.247+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(97 notify.6 [watch watch cookie 94680298292752] v28'91 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f096000a5a0 con 0x7f0958007030 2023-10-05T19:40:23.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.247+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(98 notify.7 [create] v28'90 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f096000a5a0 con 0x7f0958007030 2023-10-05T19:40:23.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.247+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94680298289520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798c16e0 con 0x7f0958007030 2023-10-05T19:40:23.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.247+0000 7f0968391e40 20 add_watcher() i=0 2023-10-05T19:40:23.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.247+0000 7f0968391e40 20 add_watcher() i=1 2023-10-05T19:40:23.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.247+0000 7f0968391e40 20 add_watcher() i=2 2023-10-05T19:40:23.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.247+0000 7f0968391e40 20 add_watcher() i=3 2023-10-05T19:40:23.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.247+0000 7f0968391e40 20 add_watcher() i=4 2023-10-05T19:40:23.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.247+0000 7f0968391e40 20 add_watcher() i=5 2023-10-05T19:40:23.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.247+0000 7f0968391e40 20 add_watcher() i=6 2023-10-05T19:40:23.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.247+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94680298289520] v28'91 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f096000a5a0 con 0x7f0958007030 2023-10-05T19:40:23.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.247+0000 7f0968391e40 20 add_watcher() i=7 2023-10-05T19:40:23.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.247+0000 7f0968391e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:23.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.247+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798c4880 con 0x561c7987c710 2023-10-05T19:40:23.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.251+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f096000a5a0 con 0x561c7987c710 2023-10-05T19:40:23.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.251+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94680298296160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798c1e00 con 0x561c7987c710 2023-10-05T19:40:23.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.251+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94680298296160] v28'94 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f096000a5a0 con 0x561c7987c710 2023-10-05T19:40:23.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.251+0000 7f0936fe5700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:23.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.251+0000 7f0968391e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:23.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.251+0000 7f0968391e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:23.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.251+0000 7f0968391e40 1 Processor -- start 2023-10-05T19:40:23.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.251+0000 7f0968391e40 1 -- start start 2023-10-05T19:40:23.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.255+0000 7f0968391e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c798ced30 0x561c798cf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:23.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.255+0000 7f0968391e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c798cf640 con 0x561c798ced30 2023-10-05T19:40:23.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.255+0000 7f09656c5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c798ced30 0x561c798cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:23.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.255+0000 7f09656c5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c798ced30 0x561c798cf100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57554/0 (socket says 172.21.15.12:57554) 2023-10-05T19:40:23.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.255+0000 7f09656c5700 1 -- 172.21.15.12:0/1705969437 learned_addr learned my addr 172.21.15.12:0/1705969437 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:23.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.255+0000 7f09656c5700 1 -- 172.21.15.12:0/1705969437 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c798cf780 con 0x561c798ced30 2023-10-05T19:40:23.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.255+0000 7f09656c5700 1 --2- 172.21.15.12:0/1705969437 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c798ced30 0x561c798cf100 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x561c798b4790 tx=0x7f095001e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f9bff3976b29532 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:23.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.255+0000 7f09367e4700 1 -- 172.21.15.12:0/1705969437 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0950013040 con 0x561c798ced30 2023-10-05T19:40:23.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.255+0000 7f09367e4700 1 -- 172.21.15.12:0/1705969437 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0950022040 con 0x561c798ced30 2023-10-05T19:40:23.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.255+0000 7f09367e4700 1 -- 172.21.15.12:0/1705969437 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f095000ea70 con 0x561c798ced30 2023-10-05T19:40:23.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.255+0000 7f0968391e40 1 -- 172.21.15.12:0/1705969437 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c798ced30 msgr2=0x561c798cf100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:23.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.255+0000 7f0968391e40 1 --2- 172.21.15.12:0/1705969437 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c798ced30 0x561c798cf100 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x561c798b4790 tx=0x7f095001e040 comp rx=0 tx=0).stop 2023-10-05T19:40:23.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.255+0000 7f0968391e40 1 -- 172.21.15.12:0/1705969437 shutdown_connections 2023-10-05T19:40:23.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.255+0000 7f0968391e40 1 --2- 172.21.15.12:0/1705969437 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c798ced30 0x561c798cf100 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:23.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.255+0000 7f0968391e40 1 -- 172.21.15.12:0/1705969437 >> 172.21.15.12:0/1705969437 conn(0x561c798ca0f0 msgr2=0x561c798c82a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:23.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.255+0000 7f0968391e40 1 -- 172.21.15.12:0/1705969437 shutdown_connections 2023-10-05T19:40:23.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.255+0000 7f0968391e40 1 -- 172.21.15.12:0/1705969437 wait complete. 2023-10-05T19:40:23.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.259+0000 7f0968391e40 1 Processor -- start 2023-10-05T19:40:23.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.259+0000 7f0968391e40 1 -- start start 2023-10-05T19:40:23.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.259+0000 7f0968391e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c798ced30 0x561c798d4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:23.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.259+0000 7f0968391e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c798d5850 con 0x561c798ced30 2023-10-05T19:40:23.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.259+0000 7f09656c5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c798ced30 0x561c798d4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:23.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.259+0000 7f09656c5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c798ced30 0x561c798d4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57570/0 (socket says 172.21.15.12:57570) 2023-10-05T19:40:23.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.259+0000 7f09656c5700 1 -- 172.21.15.12:0/1262835917 learned_addr learned my addr 172.21.15.12:0/1262835917 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:23.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.259+0000 7f09656c5700 1 -- 172.21.15.12:0/1262835917 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0950034040 con 0x561c798ced30 2023-10-05T19:40:23.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.259+0000 7f09656c5700 1 --2- 172.21.15.12:0/1262835917 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c798ced30 0x561c798d4850 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f095000bed0 tx=0x7f095003d030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:23.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.259+0000 7f0934fe1700 1 -- 172.21.15.12:0/1262835917 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f095002c640 con 0x561c798ced30 2023-10-05T19:40:23.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.259+0000 7f0934fe1700 1 -- 172.21.15.12:0/1262835917 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f095002c7a0 con 0x561c798ced30 2023-10-05T19:40:23.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.259+0000 7f0968391e40 1 -- 172.21.15.12:0/1262835917 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561c798d4e50 con 0x561c798ced30 2023-10-05T19:40:23.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.259+0000 7f0934fe1700 1 -- 172.21.15.12:0/1262835917 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f095002c990 con 0x561c798ced30 2023-10-05T19:40:23.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.259+0000 7f0968391e40 1 -- 172.21.15.12:0/1262835917 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561c798d5290 con 0x561c798ced30 2023-10-05T19:40:23.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.263+0000 7f0934fe1700 1 -- 172.21.15.12:0/1262835917 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f095004d070 con 0x561c798ced30 2023-10-05T19:40:23.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.263+0000 7f0934fe1700 1 --2- 172.21.15.12:0/1262835917 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f08ac030c50 0x7f08ac033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:23.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.263+0000 7f0934fe1700 1 -- 172.21.15.12:0/1262835917 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f0950050ae0 con 0x561c798ced30 2023-10-05T19:40:23.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.263+0000 7f09666c7700 1 --2- 172.21.15.12:0/1262835917 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f08ac030c50 0x7f08ac033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:23.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.263+0000 7f0968391e40 5 note: GC not initialized 2023-10-05T19:40:23.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.263+0000 7f09666c7700 1 --2- 172.21.15.12:0/1262835917 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f08ac030c50 0x7f08ac033100 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x561c798c1d60 tx=0x7f096000ae20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:23.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.263+0000 7f08a5ffb700 20 reqs_thread_entry: start 2023-10-05T19:40:23.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.263+0000 7f08977fe700 20 reqs_thread_entry: start 2023-10-05T19:40:23.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.263+0000 7f0968391e40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:23.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.267+0000 7f0968391e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:23.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.267+0000 7f08957fa700 20 reqs_thread_entry: start 2023-10-05T19:40:23.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.267+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:102 8.0 8:00000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c7993ca50 con 0x561c798b61d0 2023-10-05T19:40:23.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.267+0000 7f0895ffb700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:53 2023 2023-10-05T19:40:23.275 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:23.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.267+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(102 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0950006070 con 0x561c798b61d0 2023-10-05T19:40:23.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.267+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:103 8.10 8:08000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798be8b0 con 0x561c798b61d0 2023-10-05T19:40:23.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.267+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 <== osd.3 v2:172.21.15.12:6816/29951 27 ==== osd_op_reply(103 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0950006070 con 0x561c798b61d0 2023-10-05T19:40:23.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.267+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:104 8.8 8:10000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798be8b0 con 0x561c7987c710 2023-10-05T19:40:23.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.271+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(104 [pgnls start_epoch 28 out=49b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0960040070 con 0x561c7987c710 2023-10-05T19:40:23.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.271+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:105 8.18 8:18000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b9b80 con 0x561c7987c710 2023-10-05T19:40:23.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.271+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(105 [pgnls start_epoch 28 out=49b] v28'11 uv11 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0960040070 con 0x561c7987c710 2023-10-05T19:40:23.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.271+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:106 8.4 8:20000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b9b80 con 0x561c798b1850 2023-10-05T19:40:23.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.271+0000 7f0965ec6700 1 -- 172.21.15.12:0/2390960790 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(106 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0958022020 con 0x561c798b1850 2023-10-05T19:40:23.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.271+0000 7f0965ec6700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:107 8.14 8:28000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798bc0b0 con 0x561c798b61d0 2023-10-05T19:40:23.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.271+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 <== osd.3 v2:172.21.15.12:6816/29951 28 ==== osd_op_reply(107 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0950006070 con 0x561c798b61d0 2023-10-05T19:40:23.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.271+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:108 8.c 8:30000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798bc0b0 con 0x561c7987c710 2023-10-05T19:40:23.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.271+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(108 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0960040070 con 0x561c7987c710 2023-10-05T19:40:23.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.271+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:109 8.1c 8:38000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798bc0b0 con 0x561c798b61d0 2023-10-05T19:40:23.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.271+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 <== osd.3 v2:172.21.15.12:6816/29951 29 ==== osd_op_reply(109 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f095005f040 con 0x561c798b61d0 2023-10-05T19:40:23.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.271+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:110 8.2 8:40000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798bc0b0 con 0x561c798b1850 2023-10-05T19:40:23.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.275+0000 7f0965ec6700 1 -- 172.21.15.12:0/2390960790 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(110 [pgnls start_epoch 28 out=49b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0958022020 con 0x561c798b1850 2023-10-05T19:40:23.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.275+0000 7f0965ec6700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:111 8.12 8:48000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798bc0b0 con 0x561c798b61d0 2023-10-05T19:40:23.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.275+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 <== osd.3 v2:172.21.15.12:6816/29951 30 ==== osd_op_reply(111 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f095005f040 con 0x561c798b61d0 2023-10-05T19:40:23.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.275+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:112 8.a 8:50000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798bc0b0 con 0x561c7987c710 2023-10-05T19:40:23.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.275+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(112 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0960040070 con 0x561c7987c710 2023-10-05T19:40:23.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.275+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:113 8.1a 8:58000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798bc0b0 con 0x7f0958007030 2023-10-05T19:40:23.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.275+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(113 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0960040070 con 0x7f0958007030 2023-10-05T19:40:23.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.275+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:114 8.6 8:60000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798bc0b0 con 0x7f0958007030 2023-10-05T19:40:23.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.279+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(114 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0960040070 con 0x7f0958007030 2023-10-05T19:40:23.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.279+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:115 8.16 8:68000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798bc0b0 con 0x561c798b61d0 2023-10-05T19:40:23.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.279+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 <== osd.3 v2:172.21.15.12:6816/29951 31 ==== osd_op_reply(115 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f095005f040 con 0x561c798b61d0 2023-10-05T19:40:23.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.279+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:116 8.e 8:70000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798bc0b0 con 0x7f0958007030 2023-10-05T19:40:23.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.279+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(116 [pgnls start_epoch 28 out=127b] v28'1 uv1 ondisk = 1) v8 ==== 144+0+127 (crc 0 0 0) 0x7f0960040070 con 0x7f0958007030 2023-10-05T19:40:23.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.279+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:117 8.1e 8:78000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f0960051040 con 0x7f0958007030 2023-10-05T19:40:23.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.279+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.0 v2:172.21.15.8:6800/27563 30 ==== osd_op_reply(117 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0960040070 con 0x7f0958007030 2023-10-05T19:40:23.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.279+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:118 8.1 8:80000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f0960051040 con 0x7f0958007030 2023-10-05T19:40:23.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.283+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.0 v2:172.21.15.8:6800/27563 31 ==== osd_op_reply(118 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0960040070 con 0x7f0958007030 2023-10-05T19:40:23.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.283+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:119 8.11 8:88000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f0960051040 con 0x561c798b61d0 2023-10-05T19:40:23.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.283+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 <== osd.3 v2:172.21.15.12:6816/29951 32 ==== osd_op_reply(119 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f095005f040 con 0x561c798b61d0 2023-10-05T19:40:23.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.283+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:120 8.9 8:90000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f0960051040 con 0x561c798b1850 2023-10-05T19:40:23.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.283+0000 7f0965ec6700 1 -- 172.21.15.12:0/2390960790 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(120 [pgnls start_epoch 28 out=70b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+70 (crc 0 0 0) 0x7f0958022020 con 0x561c798b1850 2023-10-05T19:40:23.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.283+0000 7f0965ec6700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:121 8.19 8:98000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f0960051040 con 0x561c7987c710 2023-10-05T19:40:23.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.283+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(121 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0960040070 con 0x561c7987c710 2023-10-05T19:40:23.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.283+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:122 8.5 8:a0000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f0960051040 con 0x561c7987c710 2023-10-05T19:40:23.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.283+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== osd_op_reply(122 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0960040070 con 0x561c7987c710 2023-10-05T19:40:23.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.283+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:123 8.15 8:a8000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f0960051040 con 0x561c798b61d0 2023-10-05T19:40:23.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.287+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 <== osd.3 v2:172.21.15.12:6816/29951 33 ==== osd_op_reply(123 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f095005f040 con 0x561c798b61d0 2023-10-05T19:40:23.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.287+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:124 8.d 8:b0000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f0960051040 con 0x561c798b1850 2023-10-05T19:40:23.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.287+0000 7f0965ec6700 1 -- 172.21.15.12:0/2390960790 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(124 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0958022020 con 0x561c798b1850 2023-10-05T19:40:23.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.287+0000 7f0965ec6700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:125 8.1d 8:b8000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f0960051040 con 0x7f0958007030 2023-10-05T19:40:23.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.287+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.0 v2:172.21.15.8:6800/27563 32 ==== osd_op_reply(125 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0960040070 con 0x7f0958007030 2023-10-05T19:40:23.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.287+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:126 8.3 8:c0000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f0960051040 con 0x561c7987c710 2023-10-05T19:40:23.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.287+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.1 v2:172.21.15.12:6808/29933 39 ==== osd_op_reply(126 [pgnls start_epoch 28 out=49b] v28'1 uv1 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0960040070 con 0x561c7987c710 2023-10-05T19:40:23.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.287+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:127 8.13 8:c8000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f0960051040 con 0x7f0958007030 2023-10-05T19:40:23.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.291+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.0 v2:172.21.15.8:6800/27563 33 ==== osd_op_reply(127 [pgnls start_epoch 28 out=49b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0960040070 con 0x7f0958007030 2023-10-05T19:40:23.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.291+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:128 8.b 8:d0000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f0960051040 con 0x561c798b61d0 2023-10-05T19:40:23.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.291+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 <== osd.3 v2:172.21.15.12:6816/29951 34 ==== osd_op_reply(128 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f095005f040 con 0x561c798b61d0 2023-10-05T19:40:23.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.291+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:129 8.1b 8:d8000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f0960051040 con 0x561c798b1850 2023-10-05T19:40:23.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.291+0000 7f0965ec6700 1 -- 172.21.15.12:0/2390960790 <== osd.2 v2:172.21.15.12:6800/29934 23 ==== osd_op_reply(129 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0958022020 con 0x561c798b1850 2023-10-05T19:40:23.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.291+0000 7f0965ec6700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:130 8.7 8:e0000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f0960051040 con 0x7f0958007030 2023-10-05T19:40:23.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.291+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.0 v2:172.21.15.8:6800/27563 34 ==== osd_op_reply(130 [pgnls start_epoch 28 out=49b] v28'1 uv1 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0960040070 con 0x7f0958007030 2023-10-05T19:40:23.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.291+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:131 8.17 8:e8000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f0960051040 con 0x561c7987c710 2023-10-05T19:40:23.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.291+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.1 v2:172.21.15.12:6808/29933 40 ==== osd_op_reply(131 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0960040070 con 0x561c7987c710 2023-10-05T19:40:23.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.291+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:132 8.f 8:f0000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f0960051040 con 0x561c798b61d0 2023-10-05T19:40:23.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.295+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 <== osd.3 v2:172.21.15.12:6816/29951 35 ==== osd_op_reply(132 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f095005f040 con 0x561c798b61d0 2023-10-05T19:40:23.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.295+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:133 8.1f 8:f8000000:root:::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f0960051040 con 0x561c798b1850 2023-10-05T19:40:23.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.295+0000 7f0965ec6700 1 -- 172.21.15.12:0/2390960790 <== osd.2 v2:172.21.15.12:6800/29934 24 ==== osd_op_reply(133 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0958022020 con 0x561c798b1850 2023-10-05T19:40:23.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.295+0000 7f0968391e40 20 RGWRados::pool_iterate: got .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 2023-10-05T19:40:23.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.295+0000 7f0968391e40 20 RGWRados::pool_iterate: got myfoo 2023-10-05T19:40:23.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.295+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:134 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94680298276016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b9b80 con 0x561c798b1850 2023-10-05T19:40:23.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.299+0000 7f0965ec6700 1 -- 172.21.15.12:0/2390960790 <== osd.2 v2:172.21.15.12:6800/29934 25 ==== osd_op_reply(134 notify.0 [watch unwatch cookie 94680298276016] v28'92 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0958022020 con 0x561c798b1850 2023-10-05T19:40:23.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.299+0000 7f0968391e40 20 remove_watcher() i=0 2023-10-05T19:40:23.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.299+0000 7f0968391e40 2 removed watcher, disabling cache 2023-10-05T19:40:23.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.299+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:135 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94680298278304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798bc0b0 con 0x561c7987c710 2023-10-05T19:40:23.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.303+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.1 v2:172.21.15.12:6808/29933 41 ==== osd_op_reply(135 notify.1 [watch unwatch cookie 94680298278304] v28'92 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0960040070 con 0x561c7987c710 2023-10-05T19:40:23.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.303+0000 7f0968391e40 20 remove_watcher() i=1 2023-10-05T19:40:23.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.303+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:136 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94680298280960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798b9b80 con 0x561c798b61d0 2023-10-05T19:40:23.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.307+0000 7f09656c5700 1 -- 172.21.15.12:0/2390960790 <== osd.3 v2:172.21.15.12:6816/29951 36 ==== osd_op_reply(136 notify.2 [watch unwatch cookie 94680298280960] v28'92 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f095005f040 con 0x561c798b61d0 2023-10-05T19:40:23.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.307+0000 7f0968391e40 20 remove_watcher() i=2 2023-10-05T19:40:23.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.307+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:137 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94680298286544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798bcda0 con 0x561c798b1850 2023-10-05T19:40:23.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.307+0000 7f0965ec6700 1 -- 172.21.15.12:0/2390960790 <== osd.2 v2:172.21.15.12:6800/29934 26 ==== osd_op_reply(137 notify.3 [watch unwatch cookie 94680298286544] v28'92 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0958022020 con 0x561c798b1850 2023-10-05T19:40:23.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.307+0000 7f0968391e40 20 remove_watcher() i=3 2023-10-05T19:40:23.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.307+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:138 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94680298283968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798bedd0 con 0x561c7987c710 2023-10-05T19:40:23.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.311+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.1 v2:172.21.15.12:6808/29933 42 ==== osd_op_reply(138 notify.4 [watch unwatch cookie 94680298283968] v28'92 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0960040070 con 0x561c7987c710 2023-10-05T19:40:23.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.311+0000 7f0968391e40 20 remove_watcher() i=4 2023-10-05T19:40:23.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.311+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:139 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94680298291648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798bcda0 con 0x7f0958007030 2023-10-05T19:40:23.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.315+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.0 v2:172.21.15.8:6800/27563 35 ==== osd_op_reply(139 notify.5 [watch unwatch cookie 94680298291648] v28'92 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0960040070 con 0x7f0958007030 2023-10-05T19:40:23.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.315+0000 7f0968391e40 20 remove_watcher() i=5 2023-10-05T19:40:23.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.315+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:140 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94680298292752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798be3c0 con 0x7f0958007030 2023-10-05T19:40:23.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.315+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.0 v2:172.21.15.8:6800/27563 36 ==== osd_op_reply(140 notify.6 [watch unwatch cookie 94680298292752] v28'92 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0960040070 con 0x7f0958007030 2023-10-05T19:40:23.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.315+0000 7f0968391e40 20 remove_watcher() i=6 2023-10-05T19:40:23.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.315+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:141 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94680298289520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798c06b0 con 0x7f0958007030 2023-10-05T19:40:23.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.319+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.0 v2:172.21.15.8:6800/27563 37 ==== osd_op_reply(141 notify.7 [watch unwatch cookie 94680298289520] v28'92 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0960040070 con 0x7f0958007030 2023-10-05T19:40:23.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.319+0000 7f0968391e40 20 remove_watcher() i=7 2023-10-05T19:40:23.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.323+0000 7f0936fe5700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:23.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.323+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:142 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94680298296160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c798bedd0 con 0x561c7987c710 2023-10-05T19:40:23.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.323+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 <== osd.1 v2:172.21.15.12:6808/29933 43 ==== osd_op_reply(142 data_loggenerations_metadata [watch unwatch cookie 94680298296160] v28'95 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f0960040070 con 0x561c7987c710 2023-10-05T19:40:23.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.323+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f0958007030 msgr2=0x7f0958001620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:23.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.327+0000 7f0968391e40 1 --2- 172.21.15.12:0/2390960790 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f0958007030 0x7f0958001620 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:23.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.327+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x561c7987c710 msgr2=0x561c798af6b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:23.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.327+0000 7f0968391e40 1 --2- 172.21.15.12:0/2390960790 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x561c7987c710 0x561c798af6b0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:23.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.327+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x561c798b1850 msgr2=0x561c798b3c30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:23.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.327+0000 7f0968391e40 1 --2- 172.21.15.12:0/2390960790 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x561c798b1850 0x561c798b3c30 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:23.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.327+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x561c798b61d0 msgr2=0x561c798b85b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:23.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.327+0000 7f0968391e40 1 --2- 172.21.15.12:0/2390960790 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x561c798b61d0 0x561c798b85b0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:23.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.327+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f0948030c00 msgr2=0x7f09480330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:23.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.327+0000 7f0968391e40 1 --2- 172.21.15.12:0/2390960790 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f0948030c00 0x7f09480330b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0950001e90 tx=0x7f09500013b0 comp rx=0 tx=0).stop 2023-10-05T19:40:23.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.327+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c7987be00 msgr2=0x561c797f1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:23.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.327+0000 7f0968391e40 1 --2- 172.21.15.12:0/2390960790 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c7987be00 0x561c797f1ac0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0958005540 tx=0x7f0958003050 comp rx=0 tx=0).stop 2023-10-05T19:40:23.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.327+0000 7f09666c7700 1 -- 172.21.15.12:0/2390960790 reap_dead start 2023-10-05T19:40:23.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.327+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 shutdown_connections 2023-10-05T19:40:23.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.327+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 >> 172.21.15.12:0/2390960790 conn(0x561c79877820 msgr2=0x561c79895470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:23.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.327+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 shutdown_connections 2023-10-05T19:40:23.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.327+0000 7f0968391e40 1 -- 172.21.15.12:0/2390960790 wait complete. 2023-10-05T19:40:23.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.327+0000 7f0968391e40 1 -- 172.21.15.12:0/1262835917 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f08ac030c50 msgr2=0x7f08ac033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:23.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.327+0000 7f0968391e40 1 --2- 172.21.15.12:0/1262835917 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f08ac030c50 0x7f08ac033100 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x561c798c1d60 tx=0x7f096000ae20 comp rx=0 tx=0).stop 2023-10-05T19:40:23.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.327+0000 7f0968391e40 1 -- 172.21.15.12:0/1262835917 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c798ced30 msgr2=0x561c798d4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:23.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.327+0000 7f0968391e40 1 --2- 172.21.15.12:0/1262835917 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c798ced30 0x561c798d4850 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f095000bed0 tx=0x7f095003d030 comp rx=0 tx=0).stop 2023-10-05T19:40:23.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.327+0000 7f0968391e40 1 -- 172.21.15.12:0/1262835917 shutdown_connections 2023-10-05T19:40:23.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.327+0000 7f0968391e40 1 --2- 172.21.15.12:0/1262835917 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f08ac030c50 0x7f08ac033100 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:23.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.327+0000 7f0968391e40 1 --2- 172.21.15.12:0/1262835917 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c798ced30 0x561c798d4850 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:23.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.327+0000 7f0968391e40 1 -- 172.21.15.12:0/1262835917 >> 172.21.15.12:0/1262835917 conn(0x561c798ca0f0 msgr2=0x561c798c81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:23.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.327+0000 7f0968391e40 1 -- 172.21.15.12:0/1262835917 shutdown_connections 2023-10-05T19:40:23.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.327+0000 7f0968391e40 1 -- 172.21.15.12:0/1262835917 wait complete. 2023-10-05T19:40:23.339 INFO:teuthology.orchestra.run.smithi012.stdout:[ 2023-10-05T19:40:23.339 INFO:teuthology.orchestra.run.smithi012.stdout: "myfoo" 2023-10-05T19:40:23.339 INFO:teuthology.orchestra.run.smithi012.stdout:] 2023-10-05T19:40:23.342 DEBUG:tasks.util.rgw: json result: ['myfoo'] 2023-10-05T19:40:23.364 INFO:tasks.radosgw_admin:RL: create_bucket myfoo2 foo bytes_out=0 bytes_in=0 failed=False 2023-10-05T19:40:23.383 INFO:tasks.radosgw_admin:RL: create_bucket myfoo3 foo bytes_out=0 bytes_in=0 failed=False 2023-10-05T19:40:23.402 INFO:tasks.radosgw_admin:RL: create_bucket myfoo4 foo bytes_out=0 bytes_in=0 failed=False 2023-10-05T19:40:23.446 INFO:tasks.radosgw_admin:RL: create_bucket myfoo5 foo bytes_out=0 bytes_in=218 failed=True 2023-10-05T19:40:23.529 INFO:tasks.radosgw_admin:RL: delete_bucket myfoo2 foo bytes_out=0 bytes_in=0 failed=False 2023-10-05T19:40:23.606 INFO:tasks.radosgw_admin:RL: delete_bucket myfoo3 foo bytes_out=0 bytes_in=0 failed=False 2023-10-05T19:40:23.680 INFO:tasks.radosgw_admin:RL: delete_bucket myfoo4 foo bytes_out=0 bytes_in=0 failed=False 2023-10-05T19:40:23.680 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'stats', '--bucket', 'myfoo'] 2023-10-05T19:40:23.681 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'stats', '--bucket', 'myfoo'] 2023-10-05T19:40:23.681 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket stats --bucket myfoo 2023-10-05T19:40:23.718 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:23.718 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:23.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.711+0000 7f331dd2ae40 1 Processor -- start 2023-10-05T19:40:23.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.711+0000 7f331dd2ae40 1 -- start start 2023-10-05T19:40:23.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.711+0000 7f331dd2ae40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55c05bba37e0 0x55c05bba26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:23.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.711+0000 7f331dd2ae40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55c05baefbd0 con 0x55c05bba3410 2023-10-05T19:40:23.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.711+0000 7f331dd2ae40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x55c05baacee0 con 0x55c05bba37e0 2023-10-05T19:40:23.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.711+0000 7f331b05e700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55c05bba37e0 0x55c05bba26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:23.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.711+0000 7f331b85f700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x55c05bba3410 0x55c05bba1fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:35476/0 (socket says 172.21.15.12:35476) 2023-10-05T19:40:23.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.711+0000 7f331b05e700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55c05bba37e0 0x55c05bba26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57580/0 (socket says 172.21.15.12:57580) 2023-10-05T19:40:23.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.711+0000 7f331b85f700 1 -- 172.21.15.12:0/3440381717 learned_addr learned my addr 172.21.15.12:0/3440381717 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:23.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.711+0000 7f331b05e700 1 -- 172.21.15.12:0/3440381717 >> v1:172.21.15.8:6789/0 conn(0x55c05bba3410 legacy=0x55c05bba1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:23.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.711+0000 7f331b05e700 1 -- 172.21.15.12:0/3440381717 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c05bba2cd0 con 0x55c05bba37e0 2023-10-05T19:40:23.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.711+0000 7f331b05e700 1 --2- 172.21.15.12:0/3440381717 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c05bba37e0 0x55c05bba26d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f330400a5d0 tx=0x7f3304005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41cc10ebf1a6b0a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:23.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.711+0000 7f331a85d700 1 -- 172.21.15.12:0/3440381717 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f33040086a0 con 0x55c05bba37e0 2023-10-05T19:40:23.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.711+0000 7f331a85d700 1 -- 172.21.15.12:0/3440381717 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3304008800 con 0x55c05bba37e0 2023-10-05T19:40:23.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.711+0000 7f331a85d700 1 -- 172.21.15.12:0/3440381717 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3304008ad0 con 0x55c05bba37e0 2023-10-05T19:40:23.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.715+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3440381717 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c05bba37e0 msgr2=0x55c05bba26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:23.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.715+0000 7f331dd2ae40 1 --2- 172.21.15.12:0/3440381717 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c05bba37e0 0x55c05bba26d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f330400a5d0 tx=0x7f3304005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:23.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.715+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3440381717 shutdown_connections 2023-10-05T19:40:23.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.715+0000 7f331dd2ae40 1 --2- 172.21.15.12:0/3440381717 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c05bba37e0 0x55c05bba26d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:23.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.715+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3440381717 >> 172.21.15.12:0/3440381717 conn(0x55c05bb9d720 msgr2=0x55c05bb9fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:23.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.715+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3440381717 shutdown_connections 2023-10-05T19:40:23.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.715+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3440381717 wait complete. 2023-10-05T19:40:23.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.715+0000 7f331dd2ae40 1 Processor -- start 2023-10-05T19:40:23.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.715+0000 7f331dd2ae40 1 -- start start 2023-10-05T19:40:23.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.715+0000 7f331dd2ae40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c05bba3410 0x55c05bbc4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:23.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.715+0000 7f331dd2ae40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c05ba70e30 con 0x55c05bba3410 2023-10-05T19:40:23.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.715+0000 7f331b85f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c05bba3410 0x55c05bbc4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:23.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.715+0000 7f331b85f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c05bba3410 0x55c05bbc4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57596/0 (socket says 172.21.15.12:57596) 2023-10-05T19:40:23.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.715+0000 7f331b85f700 1 -- 172.21.15.12:0/1509346104 learned_addr learned my addr 172.21.15.12:0/1509346104 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:23.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.715+0000 7f331b85f700 1 -- 172.21.15.12:0/1509346104 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3304005cf0 con 0x55c05bba3410 2023-10-05T19:40:23.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.715+0000 7f331b85f700 1 --2- 172.21.15.12:0/1509346104 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c05bba3410 0x55c05bbc4160 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f330c0014a0 tx=0x7f330c006b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77c69ec242a92ed4 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:23.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.715+0000 7f331985b700 1 -- 172.21.15.12:0/1509346104 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f330c006110 con 0x55c05bba3410 2023-10-05T19:40:23.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.715+0000 7f331985b700 1 -- 172.21.15.12:0/1509346104 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f330c006270 con 0x55c05bba3410 2023-10-05T19:40:23.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.715+0000 7f331985b700 1 -- 172.21.15.12:0/1509346104 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f330c004520 con 0x55c05bba3410 2023-10-05T19:40:23.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.719+0000 7f331dd2ae40 1 -- 172.21.15.12:0/1509346104 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c05bba3410 msgr2=0x55c05bbc4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:23.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.719+0000 7f331dd2ae40 1 --2- 172.21.15.12:0/1509346104 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c05bba3410 0x55c05bbc4160 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f330c0014a0 tx=0x7f330c006b90 comp rx=0 tx=0).stop 2023-10-05T19:40:23.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.719+0000 7f331dd2ae40 1 -- 172.21.15.12:0/1509346104 shutdown_connections 2023-10-05T19:40:23.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.719+0000 7f331dd2ae40 1 --2- 172.21.15.12:0/1509346104 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c05bba3410 0x55c05bbc4160 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:23.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.719+0000 7f331dd2ae40 1 -- 172.21.15.12:0/1509346104 >> 172.21.15.12:0/1509346104 conn(0x55c05bb9d720 msgr2=0x55c05bbbfd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:23.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.719+0000 7f331dd2ae40 1 -- 172.21.15.12:0/1509346104 shutdown_connections 2023-10-05T19:40:23.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.719+0000 7f331dd2ae40 1 -- 172.21.15.12:0/1509346104 wait complete. 2023-10-05T19:40:23.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.719+0000 7f331dd2ae40 1 Processor -- start 2023-10-05T19:40:23.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.719+0000 7f331dd2ae40 1 -- start start 2023-10-05T19:40:23.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.719+0000 7f331dd2ae40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c05bba3410 0x55c05bbcaf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:23.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.719+0000 7f331dd2ae40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c05ba43ed0 con 0x55c05bba3410 2023-10-05T19:40:23.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.719+0000 7f331b85f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c05bba3410 0x55c05bbcaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:23.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.719+0000 7f331b85f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c05bba3410 0x55c05bbcaf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57606/0 (socket says 172.21.15.12:57606) 2023-10-05T19:40:23.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.719+0000 7f331b85f700 1 -- 172.21.15.12:0/3566293686 learned_addr learned my addr 172.21.15.12:0/3566293686 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:23.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.719+0000 7f331b85f700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c05baefbd0 con 0x55c05bba3410 2023-10-05T19:40:23.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.719+0000 7f331b85f700 1 --2- 172.21.15.12:0/3566293686 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c05bba3410 0x55c05bbcaf30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f330c006bd0 tx=0x7f330c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:23.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.719+0000 7f3313fff700 1 -- 172.21.15.12:0/3566293686 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f330c013750 con 0x55c05bba3410 2023-10-05T19:40:23.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.719+0000 7f3313fff700 1 -- 172.21.15.12:0/3566293686 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f330c0138b0 con 0x55c05bba3410 2023-10-05T19:40:23.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.719+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c05bbcb470 con 0x55c05bba3410 2023-10-05T19:40:23.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.719+0000 7f3313fff700 1 -- 172.21.15.12:0/3566293686 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f330c013b80 con 0x55c05bba3410 2023-10-05T19:40:23.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.719+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c05bbcb8b0 con 0x55c05bba3410 2023-10-05T19:40:23.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.723+0000 7f331dd2ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.723+0000 7f3313fff700 1 -- 172.21.15.12:0/3566293686 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f330c013d30 con 0x55c05bba3410 2023-10-05T19:40:23.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.723+0000 7f3313fff700 1 --2- 172.21.15.12:0/3566293686 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f32fc030c50 0x7f32fc033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:23.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.723+0000 7f3313fff700 1 -- 172.21.15.12:0/3566293686 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f330c044f40 con 0x55c05bba3410 2023-10-05T19:40:23.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.723+0000 7f331b05e700 1 --2- 172.21.15.12:0/3566293686 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f32fc030c50 0x7f32fc033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:23.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.723+0000 7f331dd2ae40 1 --2- 172.21.15.12:0/3566293686 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55c05bba37e0 0x55c05bbd69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:23.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.723+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbd70f0 con 0x55c05bba37e0 2023-10-05T19:40:23.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.723+0000 7f331c060700 1 --2- 172.21.15.12:0/3566293686 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55c05bba37e0 0x55c05bbd69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:23.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.723+0000 7f331b05e700 1 --2- 172.21.15.12:0/3566293686 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f32fc030c50 0x7f32fc033100 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3304005a90 tx=0x7f3304015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:23.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.723+0000 7f331c060700 1 --2- 172.21.15.12:0/3566293686 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55c05bba37e0 0x55c05bbd69f0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:23.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.727+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f331400a500 con 0x55c05bba37e0 2023-10-05T19:40:23.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.727+0000 7f331dd2ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:23.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.727+0000 7f331dd2ae40 20 realm 2023-10-05T19:40:23.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.727+0000 7f331dd2ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.727+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbd4050 con 0x55c05bba37e0 2023-10-05T19:40:23.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.727+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f331400a500 con 0x55c05bba37e0 2023-10-05T19:40:23.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.727+0000 7f331dd2ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:23.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.727+0000 7f331dd2ae40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:23.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.727+0000 7f331dd2ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.727+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbd7bd0 con 0x55c05bba37e0 2023-10-05T19:40:23.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.727+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f331400a500 con 0x55c05bba37e0 2023-10-05T19:40:23.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.727+0000 7f331dd2ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:23.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.727+0000 7f331dd2ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.727+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbd77b0 con 0x55c05bba37e0 2023-10-05T19:40:23.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.727+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f331400a500 con 0x55c05bba37e0 2023-10-05T19:40:23.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.727+0000 7f331dd2ae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:23.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.727+0000 7f331dd2ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.727+0000 7f331dd2ae40 1 --2- 172.21.15.12:0/3566293686 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55c05bbd8b60 0x55c05bbdaf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:23.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.727+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbdb6b0 con 0x55c05bbd8b60 2023-10-05T19:40:23.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.727+0000 7f331b85f700 1 --2- 172.21.15.12:0/3566293686 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55c05bbd8b60 0x55c05bbdaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:23.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.727+0000 7f331b85f700 1 --2- 172.21.15.12:0/3566293686 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55c05bbd8b60 0x55c05bbdaf40 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:23.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.727+0000 7f331b85f700 1 -- 172.21.15.12:0/3566293686 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f330c024020 con 0x55c05bbd8b60 2023-10-05T19:40:23.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.727+0000 7f331dd2ae40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:23.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.727+0000 7f331dd2ae40 20 searching for the correct realm 2023-10-05T19:40:23.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.727+0000 7f331dd2ae40 1 --2- 172.21.15.12:0/3566293686 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55c05bbdd4e0 0x55c05bbdf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:23.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.727+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbdfe00 con 0x55c05bbdd4e0 2023-10-05T19:40:23.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.731+0000 7f331b05e700 1 --2- 172.21.15.12:0/3566293686 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55c05bbdd4e0 0x55c05bbdf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:23.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.731+0000 7f331b05e700 1 --2- 172.21.15.12:0/3566293686 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55c05bbdd4e0 0x55c05bbdf8c0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:23.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.731+0000 7f331b05e700 1 -- 172.21.15.12:0/3566293686 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f3304008ae0 con 0x55c05bbdd4e0 2023-10-05T19:40:23.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.731+0000 7f331b05e700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbdfe00 con 0x55c05bbd8b60 2023-10-05T19:40:23.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.731+0000 7f331b85f700 1 -- 172.21.15.12:0/3566293686 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f330c024020 con 0x55c05bbd8b60 2023-10-05T19:40:23.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.731+0000 7f331b85f700 1 --2- 172.21.15.12:0/3566293686 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f330c027be0 0x7f330c028460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:23.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.731+0000 7f331b85f700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbdfe00 con 0x7f330c027be0 2023-10-05T19:40:23.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.731+0000 7f331c060700 1 --2- 172.21.15.12:0/3566293686 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f330c027be0 0x7f330c028460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:23.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.735+0000 7f331c060700 1 --2- 172.21.15.12:0/3566293686 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f330c027be0 0x7f330c028460 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:23.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.735+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3314014040 con 0x7f330c027be0 2023-10-05T19:40:23.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.735+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbdfe00 con 0x55c05bba37e0 2023-10-05T19:40:23.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.735+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3314014040 con 0x55c05bba37e0 2023-10-05T19:40:23.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.735+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbdfe00 con 0x55c05bba37e0 2023-10-05T19:40:23.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.735+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3314014040 con 0x55c05bba37e0 2023-10-05T19:40:23.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.735+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbdfe00 con 0x7f330c027be0 2023-10-05T19:40:23.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.735+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3314014040 con 0x7f330c027be0 2023-10-05T19:40:23.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.735+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbdfe00 con 0x7f330c027be0 2023-10-05T19:40:23.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.739+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3314014040 con 0x7f330c027be0 2023-10-05T19:40:23.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.739+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbdfe00 con 0x55c05bbd8b60 2023-10-05T19:40:23.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.739+0000 7f331b85f700 1 -- 172.21.15.12:0/3566293686 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f330c024020 con 0x55c05bbd8b60 2023-10-05T19:40:23.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.739+0000 7f331b85f700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbdfe00 con 0x55c05bbdd4e0 2023-10-05T19:40:23.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.739+0000 7f331b05e700 1 -- 172.21.15.12:0/3566293686 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3304008ae0 con 0x55c05bbdd4e0 2023-10-05T19:40:23.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.739+0000 7f331b05e700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbdfe00 con 0x55c05bba37e0 2023-10-05T19:40:23.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.739+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3314014040 con 0x55c05bba37e0 2023-10-05T19:40:23.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.739+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbdfe00 con 0x7f330c027be0 2023-10-05T19:40:23.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.739+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3314014040 con 0x7f330c027be0 2023-10-05T19:40:23.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.739+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbdfe00 con 0x55c05bba37e0 2023-10-05T19:40:23.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.743+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3314014040 con 0x55c05bba37e0 2023-10-05T19:40:23.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.743+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbdfe00 con 0x55c05bbd8b60 2023-10-05T19:40:23.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.743+0000 7f331b85f700 1 -- 172.21.15.12:0/3566293686 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f330c024020 con 0x55c05bbd8b60 2023-10-05T19:40:23.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.743+0000 7f331b85f700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbdfe00 con 0x55c05bbdd4e0 2023-10-05T19:40:23.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.743+0000 7f331b05e700 1 -- 172.21.15.12:0/3566293686 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3304008ae0 con 0x55c05bbdd4e0 2023-10-05T19:40:23.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.743+0000 7f331b05e700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbdfe00 con 0x55c05bbdd4e0 2023-10-05T19:40:23.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.743+0000 7f331b05e700 1 -- 172.21.15.12:0/3566293686 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3304008ae0 con 0x55c05bbdd4e0 2023-10-05T19:40:23.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.743+0000 7f331b05e700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbdfe00 con 0x7f330c027be0 2023-10-05T19:40:23.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.743+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3314014040 con 0x7f330c027be0 2023-10-05T19:40:23.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.743+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbdfe00 con 0x55c05bbdd4e0 2023-10-05T19:40:23.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.747+0000 7f331b05e700 1 -- 172.21.15.12:0/3566293686 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3304008ae0 con 0x55c05bbdd4e0 2023-10-05T19:40:23.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.747+0000 7f331b05e700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbdfe00 con 0x55c05bbdd4e0 2023-10-05T19:40:23.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.747+0000 7f331b05e700 1 -- 172.21.15.12:0/3566293686 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3304008ae0 con 0x55c05bbdd4e0 2023-10-05T19:40:23.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.747+0000 7f331b05e700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbdfe00 con 0x55c05bba37e0 2023-10-05T19:40:23.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.747+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3314014040 con 0x55c05bba37e0 2023-10-05T19:40:23.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.747+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbdfe00 con 0x55c05bba37e0 2023-10-05T19:40:23.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.747+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3314014040 con 0x55c05bba37e0 2023-10-05T19:40:23.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.747+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbdfe00 con 0x7f330c027be0 2023-10-05T19:40:23.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.747+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3314014040 con 0x7f330c027be0 2023-10-05T19:40:23.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.747+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbdfe00 con 0x55c05bbdd4e0 2023-10-05T19:40:23.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.751+0000 7f331b05e700 1 -- 172.21.15.12:0/3566293686 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3304008ae0 con 0x55c05bbdd4e0 2023-10-05T19:40:23.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.751+0000 7f331b05e700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbdfe00 con 0x55c05bbd8b60 2023-10-05T19:40:23.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.751+0000 7f331b85f700 1 -- 172.21.15.12:0/3566293686 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f330c024020 con 0x55c05bbd8b60 2023-10-05T19:40:23.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.751+0000 7f331b85f700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbdfe00 con 0x55c05bba37e0 2023-10-05T19:40:23.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.751+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3314014040 con 0x55c05bba37e0 2023-10-05T19:40:23.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.751+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbdfe00 con 0x7f330c027be0 2023-10-05T19:40:23.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.751+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3314014040 con 0x7f330c027be0 2023-10-05T19:40:23.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.751+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbdfe00 con 0x55c05bbdd4e0 2023-10-05T19:40:23.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.755+0000 7f331b05e700 1 -- 172.21.15.12:0/3566293686 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3304008ae0 con 0x55c05bbdd4e0 2023-10-05T19:40:23.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.755+0000 7f331b05e700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbdfe00 con 0x7f330c027be0 2023-10-05T19:40:23.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.755+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3314014040 con 0x7f330c027be0 2023-10-05T19:40:23.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.755+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbdfe00 con 0x55c05bbd8b60 2023-10-05T19:40:23.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.755+0000 7f331b85f700 1 -- 172.21.15.12:0/3566293686 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f330c024020 con 0x55c05bbd8b60 2023-10-05T19:40:23.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.755+0000 7f331b85f700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbdfe00 con 0x55c05bbdd4e0 2023-10-05T19:40:23.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.755+0000 7f331b05e700 1 -- 172.21.15.12:0/3566293686 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3304008ae0 con 0x55c05bbdd4e0 2023-10-05T19:40:23.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.755+0000 7f331b05e700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbdfe00 con 0x7f330c027be0 2023-10-05T19:40:23.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.755+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3314014040 con 0x7f330c027be0 2023-10-05T19:40:23.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.755+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbdfe00 con 0x55c05bba37e0 2023-10-05T19:40:23.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.759+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f3314014040 con 0x55c05bba37e0 2023-10-05T19:40:23.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.759+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbdfe00 con 0x55c05bbd8b60 2023-10-05T19:40:23.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.759+0000 7f331b85f700 1 -- 172.21.15.12:0/3566293686 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f330c024020 con 0x55c05bbd8b60 2023-10-05T19:40:23.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.759+0000 7f331dd2ae40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:23.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.759+0000 7f331dd2ae40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:23.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.759+0000 7f331dd2ae40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:23.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.759+0000 7f331dd2ae40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:23.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.759+0000 7f331dd2ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.759+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbdc230 con 0x55c05bba37e0 2023-10-05T19:40:23.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.759+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3314014040 con 0x55c05bba37e0 2023-10-05T19:40:23.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.759+0000 7f331dd2ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:23.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.759+0000 7f331dd2ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.759+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbdbe00 con 0x55c05bba37e0 2023-10-05T19:40:23.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.759+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f3314014040 con 0x55c05bba37e0 2023-10-05T19:40:23.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.759+0000 7f331dd2ae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:23.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.759+0000 7f331dd2ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.759+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbdcb10 con 0x55c05bbdd4e0 2023-10-05T19:40:23.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.759+0000 7f331b05e700 1 -- 172.21.15.12:0/3566293686 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f3304008ae0 con 0x55c05bbdd4e0 2023-10-05T19:40:23.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.759+0000 7f331dd2ae40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:23.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.759+0000 7f331dd2ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.759+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe0730 con 0x55c05bbdd4e0 2023-10-05T19:40:23.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.763+0000 7f331b05e700 1 -- 172.21.15.12:0/3566293686 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f3304008ae0 con 0x55c05bbdd4e0 2023-10-05T19:40:23.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.763+0000 7f331dd2ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:23.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.763+0000 7f331dd2ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.763+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe0ae0 con 0x55c05bba37e0 2023-10-05T19:40:23.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.763+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3314014040 con 0x55c05bba37e0 2023-10-05T19:40:23.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.763+0000 7f331dd2ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:23.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.763+0000 7f331dd2ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.763+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe0e90 con 0x55c05bba37e0 2023-10-05T19:40:23.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.763+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f3314014040 con 0x55c05bba37e0 2023-10-05T19:40:23.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.763+0000 7f331dd2ae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:23.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.763+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe1ea0 con 0x55c05bbdd4e0 2023-10-05T19:40:23.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.767+0000 7f331b05e700 1 -- 172.21.15.12:0/3566293686 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f3304008ae0 con 0x55c05bbdd4e0 2023-10-05T19:40:23.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.767+0000 7f331b05e700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe1ea0 con 0x55c05bbd8b60 2023-10-05T19:40:23.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.767+0000 7f331b85f700 1 -- 172.21.15.12:0/3566293686 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f330c024020 con 0x55c05bbd8b60 2023-10-05T19:40:23.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.767+0000 7f331b85f700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe1ea0 con 0x7f330c027be0 2023-10-05T19:40:23.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.767+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3314014040 con 0x7f330c027be0 2023-10-05T19:40:23.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.767+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe0ae0 con 0x55c05bba37e0 2023-10-05T19:40:23.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.771+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3314014040 con 0x55c05bba37e0 2023-10-05T19:40:23.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.771+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe0ae0 con 0x55c05bba37e0 2023-10-05T19:40:23.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.771+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3314014040 con 0x55c05bba37e0 2023-10-05T19:40:23.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.771+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe0ae0 con 0x7f330c027be0 2023-10-05T19:40:23.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.771+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3314014040 con 0x7f330c027be0 2023-10-05T19:40:23.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.771+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe0ae0 con 0x7f330c027be0 2023-10-05T19:40:23.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.771+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3314014040 con 0x7f330c027be0 2023-10-05T19:40:23.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.771+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe0ae0 con 0x55c05bbd8b60 2023-10-05T19:40:23.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.771+0000 7f331b85f700 1 -- 172.21.15.12:0/3566293686 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f330c024020 con 0x55c05bbd8b60 2023-10-05T19:40:23.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.771+0000 7f331b85f700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe0ae0 con 0x55c05bbdd4e0 2023-10-05T19:40:23.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.775+0000 7f331b05e700 1 -- 172.21.15.12:0/3566293686 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3304008ae0 con 0x55c05bbdd4e0 2023-10-05T19:40:23.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.775+0000 7f331b05e700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe0ae0 con 0x55c05bba37e0 2023-10-05T19:40:23.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.775+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3314014040 con 0x55c05bba37e0 2023-10-05T19:40:23.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.775+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe0ae0 con 0x7f330c027be0 2023-10-05T19:40:23.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.775+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3314014040 con 0x7f330c027be0 2023-10-05T19:40:23.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.775+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe0ae0 con 0x55c05bba37e0 2023-10-05T19:40:23.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.775+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3314014040 con 0x55c05bba37e0 2023-10-05T19:40:23.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.775+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe0ae0 con 0x55c05bbd8b60 2023-10-05T19:40:23.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.775+0000 7f331b85f700 1 -- 172.21.15.12:0/3566293686 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f330c024020 con 0x55c05bbd8b60 2023-10-05T19:40:23.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.779+0000 7f331b85f700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe0ae0 con 0x55c05bbdd4e0 2023-10-05T19:40:23.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.779+0000 7f331b05e700 1 -- 172.21.15.12:0/3566293686 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3304008ae0 con 0x55c05bbdd4e0 2023-10-05T19:40:23.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.779+0000 7f331b05e700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe0ae0 con 0x55c05bbdd4e0 2023-10-05T19:40:23.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.779+0000 7f331b05e700 1 -- 172.21.15.12:0/3566293686 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3304008ae0 con 0x55c05bbdd4e0 2023-10-05T19:40:23.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.779+0000 7f331b05e700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe0ae0 con 0x7f330c027be0 2023-10-05T19:40:23.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.779+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3314014040 con 0x7f330c027be0 2023-10-05T19:40:23.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.779+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe0ae0 con 0x55c05bbdd4e0 2023-10-05T19:40:23.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.779+0000 7f331b05e700 1 -- 172.21.15.12:0/3566293686 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3304008ae0 con 0x55c05bbdd4e0 2023-10-05T19:40:23.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.779+0000 7f331b05e700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe0ae0 con 0x55c05bbdd4e0 2023-10-05T19:40:23.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.783+0000 7f331b05e700 1 -- 172.21.15.12:0/3566293686 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3304008ae0 con 0x55c05bbdd4e0 2023-10-05T19:40:23.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.783+0000 7f331b05e700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe0ae0 con 0x55c05bba37e0 2023-10-05T19:40:23.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.783+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3314014040 con 0x55c05bba37e0 2023-10-05T19:40:23.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.783+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe0ae0 con 0x55c05bba37e0 2023-10-05T19:40:23.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.783+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3314014040 con 0x55c05bba37e0 2023-10-05T19:40:23.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.783+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe0ae0 con 0x7f330c027be0 2023-10-05T19:40:23.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.783+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3314014040 con 0x7f330c027be0 2023-10-05T19:40:23.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.783+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe0ae0 con 0x55c05bbdd4e0 2023-10-05T19:40:23.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.783+0000 7f331b05e700 1 -- 172.21.15.12:0/3566293686 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3304008ae0 con 0x55c05bbdd4e0 2023-10-05T19:40:23.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.783+0000 7f331b05e700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe0ae0 con 0x55c05bbd8b60 2023-10-05T19:40:23.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.787+0000 7f331b85f700 1 -- 172.21.15.12:0/3566293686 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f330c024020 con 0x55c05bbd8b60 2023-10-05T19:40:23.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.787+0000 7f331b85f700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe0ae0 con 0x55c05bba37e0 2023-10-05T19:40:23.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.787+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3314014040 con 0x55c05bba37e0 2023-10-05T19:40:23.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.787+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe0ae0 con 0x7f330c027be0 2023-10-05T19:40:23.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.787+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3314014040 con 0x7f330c027be0 2023-10-05T19:40:23.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.787+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe0ae0 con 0x55c05bbdd4e0 2023-10-05T19:40:23.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.787+0000 7f331b05e700 1 -- 172.21.15.12:0/3566293686 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3304008ae0 con 0x55c05bbdd4e0 2023-10-05T19:40:23.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.787+0000 7f331b05e700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe0ae0 con 0x7f330c027be0 2023-10-05T19:40:23.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.791+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3314014040 con 0x7f330c027be0 2023-10-05T19:40:23.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.791+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe0ae0 con 0x55c05bbd8b60 2023-10-05T19:40:23.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.791+0000 7f331b85f700 1 -- 172.21.15.12:0/3566293686 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f330c024020 con 0x55c05bbd8b60 2023-10-05T19:40:23.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.791+0000 7f331b85f700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe0ae0 con 0x55c05bbdd4e0 2023-10-05T19:40:23.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.791+0000 7f331b05e700 1 -- 172.21.15.12:0/3566293686 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3304008ae0 con 0x55c05bbdd4e0 2023-10-05T19:40:23.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.791+0000 7f331b05e700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe0ae0 con 0x7f330c027be0 2023-10-05T19:40:23.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.791+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3314014040 con 0x7f330c027be0 2023-10-05T19:40:23.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.791+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe0ae0 con 0x55c05bba37e0 2023-10-05T19:40:23.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.791+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f3314014040 con 0x55c05bba37e0 2023-10-05T19:40:23.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.791+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe0ae0 con 0x55c05bbd8b60 2023-10-05T19:40:23.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.795+0000 7f331b85f700 1 -- 172.21.15.12:0/3566293686 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f330c024020 con 0x55c05bbd8b60 2023-10-05T19:40:23.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.795+0000 7f331dd2ae40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:23.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.795+0000 7f331dd2ae40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:23.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.795+0000 7f331dd2ae40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:23.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.795+0000 7f331dd2ae40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:23.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.795+0000 7f331dd2ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.795+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe0e90 con 0x55c05bba37e0 2023-10-05T19:40:23.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.795+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f3314014040 con 0x55c05bba37e0 2023-10-05T19:40:23.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.795+0000 7f331dd2ae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:23.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.795+0000 7f331dd2ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.795+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe1240 con 0x55c05bbd8b60 2023-10-05T19:40:23.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.795+0000 7f331b85f700 1 -- 172.21.15.12:0/3566293686 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f330c024020 con 0x55c05bbd8b60 2023-10-05T19:40:23.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.795+0000 7f331dd2ae40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:23.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.795+0000 7f331dd2ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.795+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe02d0 con 0x55c05bba37e0 2023-10-05T19:40:23.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.795+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f3314014040 con 0x55c05bba37e0 2023-10-05T19:40:23.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.795+0000 7f331dd2ae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:23.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.795+0000 7f331dd2ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.795+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe02a0 con 0x55c05bbdd4e0 2023-10-05T19:40:23.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.799+0000 7f331b05e700 1 -- 172.21.15.12:0/3566293686 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f3304008ae0 con 0x55c05bbdd4e0 2023-10-05T19:40:23.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.799+0000 7f331dd2ae40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:23.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.799+0000 7f331dd2ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.799+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe1d00 con 0x7f330c027be0 2023-10-05T19:40:23.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.799+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f3314014040 con 0x7f330c027be0 2023-10-05T19:40:23.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.799+0000 7f331dd2ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:23.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.799+0000 7f331dd2ae40 20 zone default found 2023-10-05T19:40:23.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.799+0000 7f331dd2ae40 4 Realm: () 2023-10-05T19:40:23.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.799+0000 7f331dd2ae40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:23.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.799+0000 7f331dd2ae40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:23.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.799+0000 7f331dd2ae40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:23.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.799+0000 7f331dd2ae40 20 zonegroup default 2023-10-05T19:40:23.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.799+0000 7f331dd2ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.799+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe1d00 con 0x7f330c027be0 2023-10-05T19:40:23.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.799+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f3314014040 con 0x7f330c027be0 2023-10-05T19:40:23.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.799+0000 7f331dd2ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:23.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.799+0000 7f331dd2ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.799+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe3680 con 0x55c05bbdd4e0 2023-10-05T19:40:23.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.799+0000 7f331b05e700 1 -- 172.21.15.12:0/3566293686 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f3304008ae0 con 0x55c05bbdd4e0 2023-10-05T19:40:23.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.799+0000 7f331dd2ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:23.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.799+0000 7f331dd2ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.803+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe3a30 con 0x55c05bbdd4e0 2023-10-05T19:40:23.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.803+0000 7f331b05e700 1 -- 172.21.15.12:0/3566293686 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f3304008ae0 con 0x55c05bbdd4e0 2023-10-05T19:40:23.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.803+0000 7f331dd2ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:23.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.803+0000 7f331dd2ae40 20 started sync module instance, tier type = 2023-10-05T19:40:23.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.803+0000 7f331dd2ae40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:23.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.803+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe2d90 con 0x55c05bbd8b60 2023-10-05T19:40:23.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.803+0000 7f331b85f700 1 -- 172.21.15.12:0/3566293686 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'93 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f330c024020 con 0x55c05bbd8b60 2023-10-05T19:40:23.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.803+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94284661272976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe29c0 con 0x55c05bbd8b60 2023-10-05T19:40:23.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.807+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe48e0 con 0x55c05bba37e0 2023-10-05T19:40:23.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.807+0000 7f331b85f700 1 -- 172.21.15.12:0/3566293686 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94284661272976] v28'94 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f330c024020 con 0x55c05bbd8b60 2023-10-05T19:40:23.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.807+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'93 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3314014040 con 0x55c05bba37e0 2023-10-05T19:40:23.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.807+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94284661276208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe4ce0 con 0x55c05bba37e0 2023-10-05T19:40:23.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.807+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe5580 con 0x55c05bbdd4e0 2023-10-05T19:40:23.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.811+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94284661276208] v28'94 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3314014040 con 0x55c05bba37e0 2023-10-05T19:40:23.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.811+0000 7f331b05e700 1 -- 172.21.15.12:0/3566293686 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'93 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3304008ae0 con 0x55c05bbdd4e0 2023-10-05T19:40:23.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.811+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94284661279968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe4510 con 0x55c05bbdd4e0 2023-10-05T19:40:23.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.811+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe6360 con 0x55c05bbd8b60 2023-10-05T19:40:23.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.815+0000 7f331b85f700 1 -- 172.21.15.12:0/3566293686 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'93 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f330c024020 con 0x55c05bbd8b60 2023-10-05T19:40:23.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.815+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94284661286672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe5f90 con 0x55c05bbd8b60 2023-10-05T19:40:23.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.815+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe7830 con 0x55c05bba37e0 2023-10-05T19:40:23.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.815+0000 7f331b05e700 1 -- 172.21.15.12:0/3566293686 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94284661279968] v28'94 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3304008ae0 con 0x55c05bbdd4e0 2023-10-05T19:40:23.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.815+0000 7f331b85f700 1 -- 172.21.15.12:0/3566293686 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94284661286672] v28'94 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f330c024020 con 0x55c05bbd8b60 2023-10-05T19:40:23.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.819+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'93 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3314014040 con 0x55c05bba37e0 2023-10-05T19:40:23.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.819+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94284661287776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe5f90 con 0x55c05bba37e0 2023-10-05T19:40:23.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.819+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe7470 con 0x7f330c027be0 2023-10-05T19:40:23.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.819+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94284661287776] v28'94 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3314014040 con 0x55c05bba37e0 2023-10-05T19:40:23.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.819+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'93 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3314014040 con 0x7f330c027be0 2023-10-05T19:40:23.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.819+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94284661283200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe7fe0 con 0x7f330c027be0 2023-10-05T19:40:23.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.819+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe89f0 con 0x7f330c027be0 2023-10-05T19:40:23.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.823+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(96 notify.6 [create] v28'93 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3314014040 con 0x7f330c027be0 2023-10-05T19:40:23.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.823+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94284661294336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe8e40 con 0x7f330c027be0 2023-10-05T19:40:23.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.823+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe9920 con 0x7f330c027be0 2023-10-05T19:40:23.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.823+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(95 notify.5 [watch watch cookie 94284661283200] v28'94 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3314014040 con 0x7f330c027be0 2023-10-05T19:40:23.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.823+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(97 notify.6 [watch watch cookie 94284661294336] v28'94 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3314014040 con 0x7f330c027be0 2023-10-05T19:40:23.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.827+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(98 notify.7 [create] v28'93 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3314014040 con 0x7f330c027be0 2023-10-05T19:40:23.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.827+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94284661291760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe8950 con 0x7f330c027be0 2023-10-05T19:40:23.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.827+0000 7f331dd2ae40 20 add_watcher() i=0 2023-10-05T19:40:23.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.827+0000 7f331dd2ae40 20 add_watcher() i=1 2023-10-05T19:40:23.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.827+0000 7f331dd2ae40 20 add_watcher() i=2 2023-10-05T19:40:23.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.827+0000 7f331dd2ae40 20 add_watcher() i=3 2023-10-05T19:40:23.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.827+0000 7f331dd2ae40 20 add_watcher() i=4 2023-10-05T19:40:23.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.827+0000 7f331dd2ae40 20 add_watcher() i=5 2023-10-05T19:40:23.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.827+0000 7f331dd2ae40 20 add_watcher() i=6 2023-10-05T19:40:23.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.827+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94284661291760] v28'94 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3314014040 con 0x7f330c027be0 2023-10-05T19:40:23.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.827+0000 7f331dd2ae40 20 add_watcher() i=7 2023-10-05T19:40:23.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.827+0000 7f331dd2ae40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:23.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.827+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe8950 con 0x55c05bba37e0 2023-10-05T19:40:23.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.827+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f3314014040 con 0x55c05bba37e0 2023-10-05T19:40:23.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.827+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94284661298160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bbe8950 con 0x55c05bba37e0 2023-10-05T19:40:23.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.831+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94284661298160] v28'97 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f3314014040 con 0x55c05bba37e0 2023-10-05T19:40:23.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.831+0000 7f32ed7e2700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:23.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.831+0000 7f331dd2ae40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:23.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.831+0000 7f331dd2ae40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:23.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.831+0000 7f331dd2ae40 1 Processor -- start 2023-10-05T19:40:23.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.831+0000 7f331dd2ae40 1 -- start start 2023-10-05T19:40:23.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.831+0000 7f331dd2ae40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c05bbf6a00 0x55c05bbf6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:23.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.831+0000 7f331dd2ae40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c05bbf7310 con 0x55c05bbf6a00 2023-10-05T19:40:23.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.831+0000 7f331b05e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c05bbf6a00 0x55c05bbf6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:23.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.831+0000 7f331b05e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c05bbf6a00 0x55c05bbf6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57614/0 (socket says 172.21.15.12:57614) 2023-10-05T19:40:23.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.831+0000 7f331b05e700 1 -- 172.21.15.12:0/1352316355 learned_addr learned my addr 172.21.15.12:0/1352316355 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:23.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.835+0000 7f331b05e700 1 -- 172.21.15.12:0/1352316355 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c05bbf7450 con 0x55c05bbf6a00 2023-10-05T19:40:23.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.835+0000 7f331b05e700 1 --2- 172.21.15.12:0/1352316355 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c05bbf6a00 0x55c05bbf6dd0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x55c05bba2ca0 tx=0x7f3304012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f2f7cb9b08f5889 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:23.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.835+0000 7f32ecfe1700 1 -- 172.21.15.12:0/1352316355 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3304036070 con 0x55c05bbf6a00 2023-10-05T19:40:23.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.835+0000 7f32ecfe1700 1 -- 172.21.15.12:0/1352316355 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33040318c0 con 0x55c05bbf6a00 2023-10-05T19:40:23.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.835+0000 7f32ecfe1700 1 -- 172.21.15.12:0/1352316355 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3304031a90 con 0x55c05bbf6a00 2023-10-05T19:40:23.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.835+0000 7f331dd2ae40 1 -- 172.21.15.12:0/1352316355 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c05bbf6a00 msgr2=0x55c05bbf6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:23.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.835+0000 7f331dd2ae40 1 --2- 172.21.15.12:0/1352316355 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c05bbf6a00 0x55c05bbf6dd0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x55c05bba2ca0 tx=0x7f3304012000 comp rx=0 tx=0).stop 2023-10-05T19:40:23.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.835+0000 7f331dd2ae40 1 -- 172.21.15.12:0/1352316355 shutdown_connections 2023-10-05T19:40:23.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.835+0000 7f331dd2ae40 1 --2- 172.21.15.12:0/1352316355 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c05bbf6a00 0x55c05bbf6dd0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:23.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.835+0000 7f331dd2ae40 1 -- 172.21.15.12:0/1352316355 >> 172.21.15.12:0/1352316355 conn(0x55c05bbf1dc0 msgr2=0x55c05bbeff70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:23.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.835+0000 7f331dd2ae40 1 -- 172.21.15.12:0/1352316355 shutdown_connections 2023-10-05T19:40:23.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.835+0000 7f331dd2ae40 1 -- 172.21.15.12:0/1352316355 wait complete. 2023-10-05T19:40:23.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.835+0000 7f331dd2ae40 1 Processor -- start 2023-10-05T19:40:23.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.835+0000 7f331dd2ae40 1 -- start start 2023-10-05T19:40:23.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.835+0000 7f331dd2ae40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c05bbf6a00 0x55c05bbfc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:23.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.835+0000 7f331dd2ae40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c05bbfd4c0 con 0x55c05bbf6a00 2023-10-05T19:40:23.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.839+0000 7f331b05e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c05bbf6a00 0x55c05bbfc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:23.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.839+0000 7f331b05e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c05bbf6a00 0x55c05bbfc4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57628/0 (socket says 172.21.15.12:57628) 2023-10-05T19:40:23.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.839+0000 7f331b05e700 1 -- 172.21.15.12:0/1503678149 learned_addr learned my addr 172.21.15.12:0/1503678149 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:23.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.839+0000 7f331b05e700 1 -- 172.21.15.12:0/1503678149 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c05bba2cd0 con 0x55c05bbf6a00 2023-10-05T19:40:23.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.839+0000 7f331b05e700 1 --2- 172.21.15.12:0/1503678149 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c05bbf6a00 0x55c05bbfc4c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3304033040 tx=0x7f3304038000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:23.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.839+0000 7f3266ffd700 1 -- 172.21.15.12:0/1503678149 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3304012b90 con 0x55c05bbf6a00 2023-10-05T19:40:23.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.839+0000 7f3266ffd700 1 -- 172.21.15.12:0/1503678149 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3304027b20 con 0x55c05bbf6a00 2023-10-05T19:40:23.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.839+0000 7f3266ffd700 1 -- 172.21.15.12:0/1503678149 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3304027cb0 con 0x55c05bbf6a00 2023-10-05T19:40:23.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.839+0000 7f331dd2ae40 1 -- 172.21.15.12:0/1503678149 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c05bbfcac0 con 0x55c05bbf6a00 2023-10-05T19:40:23.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.839+0000 7f331dd2ae40 1 -- 172.21.15.12:0/1503678149 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c05bbfcea0 con 0x55c05bbf6a00 2023-10-05T19:40:23.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.839+0000 7f3266ffd700 1 -- 172.21.15.12:0/1503678149 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f3304045480 con 0x55c05bbf6a00 2023-10-05T19:40:23.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.839+0000 7f3266ffd700 1 --2- 172.21.15.12:0/1503678149 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f325c030c00 0x7f325c0330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:23.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.839+0000 7f3266ffd700 1 -- 172.21.15.12:0/1503678149 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f330400c0b0 con 0x55c05bbf6a00 2023-10-05T19:40:23.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.839+0000 7f331c060700 1 --2- 172.21.15.12:0/1503678149 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f325c030c00 0x7f325c0330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:23.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.839+0000 7f331dd2ae40 5 note: GC not initialized 2023-10-05T19:40:23.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.843+0000 7f331c060700 1 --2- 172.21.15.12:0/1503678149 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f325c030c00 0x7f325c0330b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x55c05bbe8ed0 tx=0x7f331400ab40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:23.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.843+0000 7f325bfff700 20 reqs_thread_entry: start 2023-10-05T19:40:23.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.843+0000 7f3259ffb700 20 reqs_thread_entry: start 2023-10-05T19:40:23.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.843+0000 7f331dd2ae40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:23.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.843+0000 7f331dd2ae40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:23.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.843+0000 7f3247fff700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:53 2023 2023-10-05T19:40:23.853 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:23.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.843+0000 7f32477fe700 20 reqs_thread_entry: start 2023-10-05T19:40:23.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.843+0000 7f331dd2ae40 20 get_system_obj_state: rctx=0x7ffc54f1cc90 obj=default.rgw.meta:root:myfoo state=0x55c05ba7c0b0 s->prefetch_data=0 2023-10-05T19:40:23.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.843+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:102 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bc654b0 con 0x55c05bbd8b60 2023-10-05T19:40:23.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.843+0000 7f331b85f700 1 -- 172.21.15.12:0/3566293686 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(102 myfoo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 233+0+139 (crc 0 0 0) 0x7f330c024020 con 0x55c05bbd8b60 2023-10-05T19:40:23.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.843+0000 7f331dd2ae40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:23.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.843+0000 7f331dd2ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.843+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:103 8.9 8:91b71841:root::myfoo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bc67a50 con 0x55c05bbd8b60 2023-10-05T19:40:23.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.847+0000 7f331b85f700 1 -- 172.21.15.12:0/3566293686 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(103 myfoo [call,call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) v8 ==== 233+0+213 (crc 0 0 0) 0x7f330c024020 con 0x55c05bbd8b60 2023-10-05T19:40:23.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.847+0000 7f331dd2ae40 20 rados_obj.operate() r=0 bl.length=165 2023-10-05T19:40:23.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.847+0000 7f331dd2ae40 20 get_system_obj_state: rctx=0x7ffc54f1cc90 obj=default.rgw.meta:root:.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 state=0x55c05ba522b0 s->prefetch_data=0 2023-10-05T19:40:23.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.847+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:104 8.e 8:70176bbd:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bc65f20 con 0x7f330c027be0 2023-10-05T19:40:23.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.847+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(104 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 [call out=48b,getxattrs out=218b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 290+0+282 (crc 0 0 0) 0x7f3314014040 con 0x7f330c027be0 2023-10-05T19:40:23.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.847+0000 7f331dd2ae40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:23.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.847+0000 7f331dd2ae40 20 Read xattr: user.rgw.acl 2023-10-05T19:40:23.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.847+0000 7f331dd2ae40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:23.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.847+0000 7f331dd2ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.847+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:105 8.e 8:70176bbd:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bc65a30 con 0x7f330c027be0 2023-10-05T19:40:23.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.847+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(105 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 [call,call out=48b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 332+0+618 (crc 0 0 0) 0x7f3314014040 con 0x7f330c027be0 2023-10-05T19:40:23.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.847+0000 7f331dd2ae40 20 rados_obj.operate() r=0 bl.length=352 2023-10-05T19:40:23.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.847+0000 7f331dd2ae40 20 couldn't put binfo cache entry, might have raced with data changes 2023-10-05T19:40:23.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.847+0000 7f331dd2ae40 20 get_system_obj_state: rctx=0x7ffc54f1cbf0 obj=default.rgw.meta:root:myfoo state=0x55c05ba522b0 s->prefetch_data=0 2023-10-05T19:40:23.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.847+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:106 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bc65a30 con 0x55c05bbd8b60 2023-10-05T19:40:23.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.851+0000 7f331b85f700 1 -- 172.21.15.12:0/3566293686 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(106 myfoo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 233+0+139 (crc 0 0 0) 0x7f330c024020 con 0x55c05bbd8b60 2023-10-05T19:40:23.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.851+0000 7f331dd2ae40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:23.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.851+0000 7f331dd2ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.851+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:107 8.9 8:91b71841:root::myfoo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bc69ff0 con 0x55c05bbd8b60 2023-10-05T19:40:23.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.851+0000 7f331b85f700 1 -- 172.21.15.12:0/3566293686 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(107 myfoo [call,call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) v8 ==== 233+0+213 (crc 0 0 0) 0x7f330c052040 con 0x55c05bbd8b60 2023-10-05T19:40:23.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.851+0000 7f331dd2ae40 20 rados_obj.operate() r=0 bl.length=165 2023-10-05T19:40:23.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.851+0000 7f331dd2ae40 20 get_system_obj_state: rctx=0x7ffc54f1cbf0 obj=default.rgw.meta:root:.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 state=0x55c05ba7c0b0 s->prefetch_data=0 2023-10-05T19:40:23.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.851+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:108 8.e 8:70176bbd:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bc67fd0 con 0x7f330c027be0 2023-10-05T19:40:23.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.851+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(108 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 [call out=48b,getxattrs out=218b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 290+0+282 (crc 0 0 0) 0x7f3314014040 con 0x7f330c027be0 2023-10-05T19:40:23.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.851+0000 7f331dd2ae40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:23.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.851+0000 7f331dd2ae40 20 Read xattr: user.rgw.acl 2023-10-05T19:40:23.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.851+0000 7f331dd2ae40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:23.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.851+0000 7f331dd2ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.851+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:109 8.e 8:70176bbd:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bc6a4e0 con 0x7f330c027be0 2023-10-05T19:40:23.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.851+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.0 v2:172.21.15.8:6800/27563 30 ==== osd_op_reply(109 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 [call,call out=48b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 332+0+618 (crc 0 0 0) 0x7f3314014040 con 0x7f330c027be0 2023-10-05T19:40:23.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.851+0000 7f331dd2ae40 20 rados_obj.operate() r=0 bl.length=352 2023-10-05T19:40:23.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.851+0000 7f331dd2ae40 20 couldn't put binfo cache entry, might have raced with data changes 2023-10-05T19:40:23.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.851+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:110 4.3 4:c350083b:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bc69ff0 con 0x55c05bba37e0 2023-10-05T19:40:23.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.851+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:111 4.1c 4:38d73088:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bc6c8f0 con 0x7f330c027be0 2023-10-05T19:40:23.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.851+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:112 4.a 4:502f3015:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bc6f4a0 con 0x55c05bbd8b60 2023-10-05T19:40:23.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.855+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:113 4.3e 4:7e570c43:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bc71de0 con 0x55c05bbd8b60 2023-10-05T19:40:23.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.855+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:114 4.7 4:e38d8107:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bc747c0 con 0x7f330c027be0 2023-10-05T19:40:23.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.855+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:115 4.1a 4:5add8e75:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bc77240 con 0x55c05bbd8b60 2023-10-05T19:40:23.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.855+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:116 4.2c 4:358e5801:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bc7a150 con 0x55c05bba37e0 2023-10-05T19:40:23.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.855+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:117 4.2 4:4101cfe2:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bc7cf20 con 0x55c05bba37e0 2023-10-05T19:40:23.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.855+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:118 4.30 4:0e5788c0:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bc7fb40 con 0x55c05bbdd4e0 2023-10-05T19:40:23.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.855+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:119 4.19 4:99ccd5a2:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bc82a30 con 0x55c05bbdd4e0 2023-10-05T19:40:23.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.855+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:120 4.2d 4:b73f2ca3:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bc85920 con 0x7f330c027be0 2023-10-05T19:40:23.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.855+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(110 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f3314014040 con 0x55c05bba37e0 2023-10-05T19:40:23.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.855+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.0 v2:172.21.15.8:6800/27563 31 ==== osd_op_reply(111 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f3314014040 con 0x7f330c027be0 2023-10-05T19:40:23.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.855+0000 7f331b85f700 1 -- 172.21.15.12:0/3566293686 <== osd.2 v2:172.21.15.12:6800/29934 23 ==== osd_op_reply(112 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.2 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f330c052040 con 0x55c05bbd8b60 2023-10-05T19:40:23.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.855+0000 7f331b85f700 1 -- 172.21.15.12:0/3566293686 <== osd.2 v2:172.21.15.12:6800/29934 24 ==== osd_op_reply(115 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.5 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f330c052040 con 0x55c05bbd8b60 2023-10-05T19:40:23.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.855+0000 7f331b85f700 1 -- 172.21.15.12:0/3566293686 <== osd.2 v2:172.21.15.12:6800/29934 25 ==== osd_op_reply(113 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.3 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f330c052040 con 0x55c05bbd8b60 2023-10-05T19:40:23.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.855+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.0 v2:172.21.15.8:6800/27563 32 ==== osd_op_reply(114 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f3314014040 con 0x7f330c027be0 2023-10-05T19:40:23.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.855+0000 7f331b05e700 1 -- 172.21.15.12:0/3566293686 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(119 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.9 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f330404b020 con 0x55c05bbdd4e0 2023-10-05T19:40:23.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.855+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.0 v2:172.21.15.8:6800/27563 33 ==== osd_op_reply(120 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7f3314014040 con 0x7f330c027be0 2023-10-05T19:40:23.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.855+0000 7f331b05e700 1 -- 172.21.15.12:0/3566293686 <== osd.3 v2:172.21.15.12:6816/29951 27 ==== osd_op_reply(118 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.8 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f330404b020 con 0x55c05bbdd4e0 2023-10-05T19:40:23.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.855+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(117 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.7 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f3314014040 con 0x55c05bba37e0 2023-10-05T19:40:23.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.855+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(116 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.6 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f3314014040 con 0x55c05bba37e0 2023-10-05T19:40:23.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.855+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:121 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94284661272976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bc6c8f0 con 0x55c05bbd8b60 2023-10-05T19:40:23.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.859+0000 7f331b85f700 1 -- 172.21.15.12:0/3566293686 <== osd.2 v2:172.21.15.12:6800/29934 26 ==== osd_op_reply(121 notify.0 [watch unwatch cookie 94284661272976] v28'95 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f330c052040 con 0x55c05bbd8b60 2023-10-05T19:40:23.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.859+0000 7f331dd2ae40 20 remove_watcher() i=0 2023-10-05T19:40:23.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.859+0000 7f331dd2ae40 2 removed watcher, disabling cache 2023-10-05T19:40:23.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.859+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:122 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94284661276208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bc6c8f0 con 0x55c05bba37e0 2023-10-05T19:40:23.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.863+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(122 notify.1 [watch unwatch cookie 94284661276208] v28'95 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3314014040 con 0x55c05bba37e0 2023-10-05T19:40:23.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.863+0000 7f331dd2ae40 20 remove_watcher() i=1 2023-10-05T19:40:23.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.863+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:123 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94284661279968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bc7a150 con 0x55c05bbdd4e0 2023-10-05T19:40:23.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.863+0000 7f331b05e700 1 -- 172.21.15.12:0/3566293686 <== osd.3 v2:172.21.15.12:6816/29951 28 ==== osd_op_reply(123 notify.2 [watch unwatch cookie 94284661279968] v28'95 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f330404b020 con 0x55c05bbdd4e0 2023-10-05T19:40:23.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.863+0000 7f331dd2ae40 20 remove_watcher() i=2 2023-10-05T19:40:23.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.863+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:124 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94284661286672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bc7a150 con 0x55c05bbd8b60 2023-10-05T19:40:23.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.867+0000 7f331b85f700 1 -- 172.21.15.12:0/3566293686 <== osd.2 v2:172.21.15.12:6800/29934 27 ==== osd_op_reply(124 notify.3 [watch unwatch cookie 94284661286672] v28'95 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f330c052040 con 0x55c05bbd8b60 2023-10-05T19:40:23.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.867+0000 7f331dd2ae40 20 remove_watcher() i=3 2023-10-05T19:40:23.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.867+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:125 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94284661287776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bc7a150 con 0x55c05bba37e0 2023-10-05T19:40:23.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.871+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(125 notify.4 [watch unwatch cookie 94284661287776] v28'95 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3314014040 con 0x55c05bba37e0 2023-10-05T19:40:23.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.871+0000 7f331dd2ae40 20 remove_watcher() i=4 2023-10-05T19:40:23.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.871+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:126 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94284661283200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bc7fac0 con 0x7f330c027be0 2023-10-05T19:40:23.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.871+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.0 v2:172.21.15.8:6800/27563 34 ==== osd_op_reply(126 notify.5 [watch unwatch cookie 94284661283200] v28'95 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3314014040 con 0x7f330c027be0 2023-10-05T19:40:23.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.871+0000 7f331dd2ae40 20 remove_watcher() i=5 2023-10-05T19:40:23.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.871+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:127 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94284661294336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bc7fac0 con 0x7f330c027be0 2023-10-05T19:40:23.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.875+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.0 v2:172.21.15.8:6800/27563 35 ==== osd_op_reply(127 notify.6 [watch unwatch cookie 94284661294336] v28'95 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3314014040 con 0x7f330c027be0 2023-10-05T19:40:23.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.875+0000 7f331dd2ae40 20 remove_watcher() i=6 2023-10-05T19:40:23.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.875+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:128 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94284661291760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bc7fac0 con 0x7f330c027be0 2023-10-05T19:40:23.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.879+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.0 v2:172.21.15.8:6800/27563 36 ==== osd_op_reply(128 notify.7 [watch unwatch cookie 94284661291760] v28'95 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3314014040 con 0x7f330c027be0 2023-10-05T19:40:23.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.879+0000 7f331dd2ae40 20 remove_watcher() i=7 2023-10-05T19:40:23.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.879+0000 7f32ed7e2700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:23.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.879+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:129 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94284661298160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c05bc7fac0 con 0x55c05bba37e0 2023-10-05T19:40:23.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.883+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== osd_op_reply(129 data_loggenerations_metadata [watch unwatch cookie 94284661298160] v28'98 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f3314014040 con 0x55c05bba37e0 2023-10-05T19:40:23.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.883+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f330c027be0 msgr2=0x7f330c028460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:23.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.883+0000 7f331dd2ae40 1 --2- 172.21.15.12:0/3566293686 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f330c027be0 0x7f330c028460 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:23.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.883+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55c05bba37e0 msgr2=0x55c05bbd69f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:23.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.883+0000 7f331dd2ae40 1 --2- 172.21.15.12:0/3566293686 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55c05bba37e0 0x55c05bbd69f0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:23.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.883+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55c05bbd8b60 msgr2=0x55c05bbdaf40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:23.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.883+0000 7f331dd2ae40 1 --2- 172.21.15.12:0/3566293686 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55c05bbd8b60 0x55c05bbdaf40 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:23.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.883+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55c05bbdd4e0 msgr2=0x55c05bbdf8c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:23.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.883+0000 7f331dd2ae40 1 --2- 172.21.15.12:0/3566293686 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55c05bbdd4e0 0x55c05bbdf8c0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:23.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.883+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f32fc030c50 msgr2=0x7f32fc033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:23.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.883+0000 7f331dd2ae40 1 --2- 172.21.15.12:0/3566293686 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f32fc030c50 0x7f32fc033100 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3304005a90 tx=0x7f3304015000 comp rx=0 tx=0).stop 2023-10-05T19:40:23.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.883+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c05bba3410 msgr2=0x55c05bbcaf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:23.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.883+0000 7f331dd2ae40 1 --2- 172.21.15.12:0/3566293686 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c05bba3410 0x55c05bbcaf30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f330c006bd0 tx=0x7f330c014000 comp rx=0 tx=0).stop 2023-10-05T19:40:23.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.883+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 reap_dead start 2023-10-05T19:40:23.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.883+0000 7f331c060700 1 -- 172.21.15.12:0/3566293686 reap_dead start 2023-10-05T19:40:23.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.883+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 shutdown_connections 2023-10-05T19:40:23.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.883+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 >> 172.21.15.12:0/3566293686 conn(0x55c05bb9d720 msgr2=0x55c05bbbe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:23.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.883+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 shutdown_connections 2023-10-05T19:40:23.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.883+0000 7f331dd2ae40 1 -- 172.21.15.12:0/3566293686 wait complete. 2023-10-05T19:40:23.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.883+0000 7f331dd2ae40 1 -- 172.21.15.12:0/1503678149 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f325c030c00 msgr2=0x7f325c0330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:23.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.883+0000 7f331dd2ae40 1 --2- 172.21.15.12:0/1503678149 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f325c030c00 0x7f325c0330b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x55c05bbe8ed0 tx=0x7f331400ab40 comp rx=0 tx=0).stop 2023-10-05T19:40:23.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.883+0000 7f331dd2ae40 1 -- 172.21.15.12:0/1503678149 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c05bbf6a00 msgr2=0x55c05bbfc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:23.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.883+0000 7f331dd2ae40 1 --2- 172.21.15.12:0/1503678149 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c05bbf6a00 0x55c05bbfc4c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3304033040 tx=0x7f3304038000 comp rx=0 tx=0).stop 2023-10-05T19:40:23.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.883+0000 7f331dd2ae40 1 -- 172.21.15.12:0/1503678149 shutdown_connections 2023-10-05T19:40:23.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.883+0000 7f331dd2ae40 1 --2- 172.21.15.12:0/1503678149 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f325c030c00 0x7f325c0330b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:23.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.883+0000 7f331dd2ae40 1 --2- 172.21.15.12:0/1503678149 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c05bbf6a00 0x55c05bbfc4c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:23.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.883+0000 7f331dd2ae40 1 -- 172.21.15.12:0/1503678149 >> 172.21.15.12:0/1503678149 conn(0x55c05bbf1dc0 msgr2=0x55c05bbefec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:23.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.883+0000 7f331dd2ae40 1 -- 172.21.15.12:0/1503678149 shutdown_connections 2023-10-05T19:40:23.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.883+0000 7f331dd2ae40 1 -- 172.21.15.12:0/1503678149 wait complete. 2023-10-05T19:40:23.896 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:40:23.896 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo", 2023-10-05T19:40:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: "num_shards": 11, 2023-10-05T19:40:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: "tenant": "", 2023-10-05T19:40:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: "zonegroup": "1602ab00-ae0f-4921-ad63-b9cfe7bc77db", 2023-10-05T19:40:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: "placement_rule": "default-placement", 2023-10-05T19:40:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: "explicit_placement": { 2023-10-05T19:40:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: "data_pool": "", 2023-10-05T19:40:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: "data_extra_pool": "", 2023-10-05T19:40:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: "index_pool": "" 2023-10-05T19:40:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: "id": "93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1", 2023-10-05T19:40:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: "marker": "93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1", 2023-10-05T19:40:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: "index_type": "Normal", 2023-10-05T19:40:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: "owner": "foo", 2023-10-05T19:40:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: "ver": "0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1", 2023-10-05T19:40:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: "master_ver": "0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0", 2023-10-05T19:40:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: "mtime": "2023-10-05T19:40:22.865997Z", 2023-10-05T19:40:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: "creation_time": "2023-10-05T19:40:22.857926Z", 2023-10-05T19:40:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: "max_marker": "0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#", 2023-10-05T19:40:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: "usage": {}, 2023-10-05T19:40:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket_quota": { 2023-10-05T19:40:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:23.899 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:23.899 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:23.899 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:23.899 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:40:23.900 DEBUG:tasks.util.rgw: json result: {'bucket': 'myfoo', 'num_shards': 11, 'tenant': '', 'zonegroup': '1602ab00-ae0f-4921-ad63-b9cfe7bc77db', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1', 'marker': '93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1', 'index_type': 'Normal', 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2023-10-05T19:40:22.865997Z', 'creation_time': '2023-10-05T19:40:22.857926Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}} 2023-10-05T19:40:23.900 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'stats', '--uid', 'foo'] 2023-10-05T19:40:23.900 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'stats', '--uid', 'foo'] 2023-10-05T19:40:23.900 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket stats --uid foo 2023-10-05T19:40:23.980 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:23.980 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:23.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.971+0000 7f6191049e40 1 Processor -- start 2023-10-05T19:40:23.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.971+0000 7f6191049e40 1 -- start start 2023-10-05T19:40:23.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.971+0000 7f6191049e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x559860f8b300 0x559860f8d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:23.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.971+0000 7f6191049e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x559860ed4bd0 con 0x559860f86d20 2023-10-05T19:40:23.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.971+0000 7f6191049e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x559860e91ee0 con 0x559860f8b300 2023-10-05T19:40:23.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.975+0000 7f618e37d700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x559860f8b300 0x559860f8d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:23.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.975+0000 7f618eb7e700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x559860f86d20 0x559860f870f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:35482/0 (socket says 172.21.15.12:35482) 2023-10-05T19:40:23.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.975+0000 7f618eb7e700 1 -- 172.21.15.12:0/1357781982 learned_addr learned my addr 172.21.15.12:0/1357781982 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:23.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.975+0000 7f618e37d700 1 -- 172.21.15.12:0/1357781982 >> v1:172.21.15.8:6789/0 conn(0x559860f86d20 legacy=0x559860f870f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:23.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.975+0000 7f618db7c700 1 -- 172.21.15.12:0/1357781982 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1090338521 0 0) 0x559860ed4bd0 con 0x559860f86d20 2023-10-05T19:40:23.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.975+0000 7f618e37d700 1 -- 172.21.15.12:0/1357781982 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559860f87930 con 0x559860f8b300 2023-10-05T19:40:23.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.975+0000 7f618e37d700 1 --2- 172.21.15.12:0/1357781982 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559860f8b300 0x559860f8d6e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f617800a5d0 tx=0x7f6178005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb7fa11f225ad130 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:23.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.975+0000 7f618db7c700 1 -- 172.21.15.12:0/1357781982 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f61780086a0 con 0x559860f8b300 2023-10-05T19:40:23.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.975+0000 7f618db7c700 1 -- 172.21.15.12:0/1357781982 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6178008800 con 0x559860f8b300 2023-10-05T19:40:23.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.975+0000 7f618db7c700 1 -- 172.21.15.12:0/1357781982 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6178008ad0 con 0x559860f8b300 2023-10-05T19:40:23.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.975+0000 7f6191049e40 1 -- 172.21.15.12:0/1357781982 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559860f8b300 msgr2=0x559860f8d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:23.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.975+0000 7f6191049e40 1 --2- 172.21.15.12:0/1357781982 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559860f8b300 0x559860f8d6e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f617800a5d0 tx=0x7f6178005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:23.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.975+0000 7f6191049e40 1 -- 172.21.15.12:0/1357781982 shutdown_connections 2023-10-05T19:40:23.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.975+0000 7f6191049e40 1 --2- 172.21.15.12:0/1357781982 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559860f8b300 0x559860f8d6e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:23.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.975+0000 7f6191049e40 1 -- 172.21.15.12:0/1357781982 >> 172.21.15.12:0/1357781982 conn(0x559860f82740 msgr2=0x559860f84b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:23.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.975+0000 7f6191049e40 1 -- 172.21.15.12:0/1357781982 shutdown_connections 2023-10-05T19:40:23.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.975+0000 7f6191049e40 1 -- 172.21.15.12:0/1357781982 wait complete. 2023-10-05T19:40:23.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.975+0000 7f6191049e40 1 Processor -- start 2023-10-05T19:40:23.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.975+0000 7f6191049e40 1 -- start start 2023-10-05T19:40:23.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.975+0000 7f6191049e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559860f86d20 0x559860fa6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:23.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.975+0000 7f6191049e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559860e55e30 con 0x559860f86d20 2023-10-05T19:40:23.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.979+0000 7f618eb7e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559860f86d20 0x559860fa6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:23.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.979+0000 7f618eb7e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559860f86d20 0x559860fa6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57648/0 (socket says 172.21.15.12:57648) 2023-10-05T19:40:23.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.979+0000 7f618eb7e700 1 -- 172.21.15.12:0/3582136444 learned_addr learned my addr 172.21.15.12:0/3582136444 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:23.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.979+0000 7f618eb7e700 1 -- 172.21.15.12:0/3582136444 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6178005cf0 con 0x559860f86d20 2023-10-05T19:40:23.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.979+0000 7f618eb7e700 1 --2- 172.21.15.12:0/3582136444 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559860f86d20 0x559860fa6d80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6180001dc0 tx=0x7f6180001790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb6b58bdbb74006c server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:23.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.979+0000 7f618cb7a700 1 -- 172.21.15.12:0/3582136444 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f618000bce0 con 0x559860f86d20 2023-10-05T19:40:23.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.979+0000 7f618cb7a700 1 -- 172.21.15.12:0/3582136444 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f618000be40 con 0x559860f86d20 2023-10-05T19:40:23.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.979+0000 7f618cb7a700 1 -- 172.21.15.12:0/3582136444 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f618000e5d0 con 0x559860f86d20 2023-10-05T19:40:23.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.979+0000 7f6191049e40 1 -- 172.21.15.12:0/3582136444 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559860f86d20 msgr2=0x559860fa6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:23.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.979+0000 7f6191049e40 1 --2- 172.21.15.12:0/3582136444 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559860f86d20 0x559860fa6d80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6180001dc0 tx=0x7f6180001790 comp rx=0 tx=0).stop 2023-10-05T19:40:23.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.979+0000 7f6191049e40 1 -- 172.21.15.12:0/3582136444 shutdown_connections 2023-10-05T19:40:23.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.979+0000 7f6191049e40 1 --2- 172.21.15.12:0/3582136444 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559860f86d20 0x559860fa6d80 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:23.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.979+0000 7f6191049e40 1 -- 172.21.15.12:0/3582136444 >> 172.21.15.12:0/3582136444 conn(0x559860f82740 msgr2=0x559860fa2940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:23.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.979+0000 7f6191049e40 1 -- 172.21.15.12:0/3582136444 shutdown_connections 2023-10-05T19:40:23.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.979+0000 7f6191049e40 1 -- 172.21.15.12:0/3582136444 wait complete. 2023-10-05T19:40:23.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.979+0000 7f6191049e40 1 Processor -- start 2023-10-05T19:40:23.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.979+0000 7f6191049e40 1 -- start start 2023-10-05T19:40:23.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.979+0000 7f6191049e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559860f86d20 0x559860f90600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:23.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.979+0000 7f6191049e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559860e28ed0 con 0x559860f86d20 2023-10-05T19:40:23.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.979+0000 7f618eb7e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559860f86d20 0x559860f90600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:23.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.979+0000 7f618eb7e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559860f86d20 0x559860f90600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57658/0 (socket says 172.21.15.12:57658) 2023-10-05T19:40:23.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.979+0000 7f618eb7e700 1 -- 172.21.15.12:0/1247925877 learned_addr learned my addr 172.21.15.12:0/1247925877 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:23.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.983+0000 7f618eb7e700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6180001a70 con 0x559860f86d20 2023-10-05T19:40:23.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.983+0000 7f618eb7e700 1 --2- 172.21.15.12:0/1247925877 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559860f86d20 0x559860f90600 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f61800017d0 tx=0x7f6180016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:23.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.983+0000 7f6186ffd700 1 -- 172.21.15.12:0/1247925877 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f618001f070 con 0x559860f86d20 2023-10-05T19:40:23.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.983+0000 7f6186ffd700 1 -- 172.21.15.12:0/1247925877 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f618001a4d0 con 0x559860f86d20 2023-10-05T19:40:23.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.983+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559860f90b40 con 0x559860f86d20 2023-10-05T19:40:23.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.983+0000 7f6186ffd700 1 -- 172.21.15.12:0/1247925877 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f618001a7a0 con 0x559860f86d20 2023-10-05T19:40:23.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.983+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559860f8ef00 con 0x559860f86d20 2023-10-05T19:40:23.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.983+0000 7f6191049e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.983+0000 7f6186ffd700 1 -- 172.21.15.12:0/1247925877 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f618001a900 con 0x559860f86d20 2023-10-05T19:40:23.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.983+0000 7f6186ffd700 1 --2- 172.21.15.12:0/1247925877 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f6174030c50 0x7f6174033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:23.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.987+0000 7f6186ffd700 1 -- 172.21.15.12:0/1247925877 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f6180044fe0 con 0x559860f86d20 2023-10-05T19:40:23.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.987+0000 7f618e37d700 1 --2- 172.21.15.12:0/1247925877 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f6174030c50 0x7f6174033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:23.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.987+0000 7f6191049e40 1 --2- 172.21.15.12:0/1247925877 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x559860f8b300 0x559860fbb880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:23.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.987+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fbbe90 con 0x559860f8b300 2023-10-05T19:40:23.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.987+0000 7f618f37f700 1 --2- 172.21.15.12:0/1247925877 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x559860f8b300 0x559860fbb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:23.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.987+0000 7f618e37d700 1 --2- 172.21.15.12:0/1247925877 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f6174030c50 0x7f6174033100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6178005a90 tx=0x7f6178015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:23.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.987+0000 7f618f37f700 1 --2- 172.21.15.12:0/1247925877 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x559860f8b300 0x559860fbb880 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:23.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.987+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f618800a5a0 con 0x559860f8b300 2023-10-05T19:40:23.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.987+0000 7f6191049e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:23.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.987+0000 7f6191049e40 20 realm 2023-10-05T19:40:23.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.987+0000 7f6191049e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.987+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fb8ee0 con 0x559860f8b300 2023-10-05T19:40:23.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.987+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f618800a5a0 con 0x559860f8b300 2023-10-05T19:40:23.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.987+0000 7f6191049e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:23.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.987+0000 7f6191049e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:23.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.987+0000 7f6191049e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.987+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fbc970 con 0x559860f8b300 2023-10-05T19:40:23.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.987+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f618800a5a0 con 0x559860f8b300 2023-10-05T19:40:23.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.987+0000 7f6191049e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:23.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.987+0000 7f6191049e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.987+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fbc550 con 0x559860f8b300 2023-10-05T19:40:23.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.991+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f618800a5a0 con 0x559860f8b300 2023-10-05T19:40:23.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.991+0000 7f6191049e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:23.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.991+0000 7f6191049e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:23.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.991+0000 7f6191049e40 1 --2- 172.21.15.12:0/1247925877 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x559860fbd900 0x559860fbfce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:23.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.991+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc0450 con 0x559860fbd900 2023-10-05T19:40:23.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.991+0000 7f618eb7e700 1 --2- 172.21.15.12:0/1247925877 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x559860fbd900 0x559860fbfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.991+0000 7f618eb7e700 1 --2- 172.21.15.12:0/1247925877 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x559860fbd900 0x559860fbfce0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.991+0000 7f618eb7e700 1 -- 172.21.15.12:0/1247925877 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f6180014070 con 0x559860fbd900 2023-10-05T19:40:24.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.991+0000 7f6191049e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:24.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.991+0000 7f6191049e40 20 searching for the correct realm 2023-10-05T19:40:24.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.991+0000 7f6191049e40 1 --2- 172.21.15.12:0/1247925877 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x559860fc2280 0x559860fc4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:24.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.991+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc4ba0 con 0x559860fc2280 2023-10-05T19:40:24.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.991+0000 7f618e37d700 1 --2- 172.21.15.12:0/1247925877 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x559860fc2280 0x559860fc4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.991+0000 7f618e37d700 1 --2- 172.21.15.12:0/1247925877 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x559860fc2280 0x559860fc4660 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.995+0000 7f618e37d700 1 -- 172.21.15.12:0/1247925877 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f6178008a80 con 0x559860fc2280 2023-10-05T19:40:24.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.995+0000 7f618e37d700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc4ba0 con 0x559860fbd900 2023-10-05T19:40:24.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.995+0000 7f618eb7e700 1 -- 172.21.15.12:0/1247925877 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6180014070 con 0x559860fbd900 2023-10-05T19:40:24.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.995+0000 7f618eb7e700 1 --2- 172.21.15.12:0/1247925877 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f618000d750 0x7f6180042bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:24.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.995+0000 7f618eb7e700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc4ba0 con 0x7f618000d750 2023-10-05T19:40:24.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.995+0000 7f618f37f700 1 --2- 172.21.15.12:0/1247925877 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f618000d750 0x7f6180042bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.999+0000 7f618f37f700 1 --2- 172.21.15.12:0/1247925877 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f618000d750 0x7f6180042bc0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.999+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f618800a5a0 con 0x7f618000d750 2023-10-05T19:40:24.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.999+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc4ba0 con 0x559860f8b300 2023-10-05T19:40:24.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.999+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f618800a5a0 con 0x559860f8b300 2023-10-05T19:40:24.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.999+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc4ba0 con 0x559860f8b300 2023-10-05T19:40:24.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.999+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f618800a5a0 con 0x559860f8b300 2023-10-05T19:40:24.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.999+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc4ba0 con 0x7f618000d750 2023-10-05T19:40:24.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.999+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f618800a5a0 con 0x7f618000d750 2023-10-05T19:40:24.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:23.999+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc4ba0 con 0x7f618000d750 2023-10-05T19:40:24.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.003+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f618800a5a0 con 0x7f618000d750 2023-10-05T19:40:24.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.003+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc4ba0 con 0x559860fbd900 2023-10-05T19:40:24.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.003+0000 7f618eb7e700 1 -- 172.21.15.12:0/1247925877 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6180036040 con 0x559860fbd900 2023-10-05T19:40:24.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.003+0000 7f618eb7e700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc4ba0 con 0x559860fc2280 2023-10-05T19:40:24.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.003+0000 7f618e37d700 1 -- 172.21.15.12:0/1247925877 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6178008a80 con 0x559860fc2280 2023-10-05T19:40:24.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.003+0000 7f618e37d700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc4ba0 con 0x559860f8b300 2023-10-05T19:40:24.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.003+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f618800a5a0 con 0x559860f8b300 2023-10-05T19:40:24.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.003+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc4ba0 con 0x7f618000d750 2023-10-05T19:40:24.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.003+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f618800a5a0 con 0x7f618000d750 2023-10-05T19:40:24.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.003+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc4ba0 con 0x559860f8b300 2023-10-05T19:40:24.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.007+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f618800a5a0 con 0x559860f8b300 2023-10-05T19:40:24.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.007+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc4ba0 con 0x559860fbd900 2023-10-05T19:40:24.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.007+0000 7f618eb7e700 1 -- 172.21.15.12:0/1247925877 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6180036040 con 0x559860fbd900 2023-10-05T19:40:24.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.007+0000 7f618eb7e700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc4ba0 con 0x559860fc2280 2023-10-05T19:40:24.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.007+0000 7f618e37d700 1 -- 172.21.15.12:0/1247925877 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6178008a80 con 0x559860fc2280 2023-10-05T19:40:24.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.007+0000 7f618e37d700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc4ba0 con 0x559860fc2280 2023-10-05T19:40:24.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.007+0000 7f618e37d700 1 -- 172.21.15.12:0/1247925877 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6178008a80 con 0x559860fc2280 2023-10-05T19:40:24.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.007+0000 7f618e37d700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc4ba0 con 0x7f618000d750 2023-10-05T19:40:24.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.007+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f618800a5a0 con 0x7f618000d750 2023-10-05T19:40:24.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.007+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc4ba0 con 0x559860fc2280 2023-10-05T19:40:24.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.011+0000 7f618e37d700 1 -- 172.21.15.12:0/1247925877 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6178008a80 con 0x559860fc2280 2023-10-05T19:40:24.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.011+0000 7f618e37d700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc4ba0 con 0x559860fc2280 2023-10-05T19:40:24.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.011+0000 7f618e37d700 1 -- 172.21.15.12:0/1247925877 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6178008a80 con 0x559860fc2280 2023-10-05T19:40:24.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.011+0000 7f618e37d700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc4ba0 con 0x559860f8b300 2023-10-05T19:40:24.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.011+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f618800a5a0 con 0x559860f8b300 2023-10-05T19:40:24.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.011+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc4ba0 con 0x559860f8b300 2023-10-05T19:40:24.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.011+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f618800a5a0 con 0x559860f8b300 2023-10-05T19:40:24.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.011+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc4ba0 con 0x7f618000d750 2023-10-05T19:40:24.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.011+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f618800a5a0 con 0x7f618000d750 2023-10-05T19:40:24.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.011+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc4ba0 con 0x559860fc2280 2023-10-05T19:40:24.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.011+0000 7f618e37d700 1 -- 172.21.15.12:0/1247925877 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6178008a80 con 0x559860fc2280 2023-10-05T19:40:24.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.015+0000 7f618e37d700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc4ba0 con 0x559860fbd900 2023-10-05T19:40:24.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.015+0000 7f618eb7e700 1 -- 172.21.15.12:0/1247925877 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6180036040 con 0x559860fbd900 2023-10-05T19:40:24.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.015+0000 7f618eb7e700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc4ba0 con 0x559860f8b300 2023-10-05T19:40:24.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.015+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f618800a5a0 con 0x559860f8b300 2023-10-05T19:40:24.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.015+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc4ba0 con 0x7f618000d750 2023-10-05T19:40:24.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.015+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f618800a5a0 con 0x7f618000d750 2023-10-05T19:40:24.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.015+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc4ba0 con 0x559860fc2280 2023-10-05T19:40:24.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.015+0000 7f618e37d700 1 -- 172.21.15.12:0/1247925877 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6178008a80 con 0x559860fc2280 2023-10-05T19:40:24.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.015+0000 7f618e37d700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc4ba0 con 0x7f618000d750 2023-10-05T19:40:24.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.019+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f618800a5a0 con 0x7f618000d750 2023-10-05T19:40:24.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.019+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc4ba0 con 0x559860fbd900 2023-10-05T19:40:24.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.019+0000 7f618eb7e700 1 -- 172.21.15.12:0/1247925877 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6180036040 con 0x559860fbd900 2023-10-05T19:40:24.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.019+0000 7f618eb7e700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc4ba0 con 0x559860fc2280 2023-10-05T19:40:24.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.019+0000 7f618e37d700 1 -- 172.21.15.12:0/1247925877 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6178008a80 con 0x559860fc2280 2023-10-05T19:40:24.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.019+0000 7f618e37d700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc4ba0 con 0x7f618000d750 2023-10-05T19:40:24.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.019+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f618800a5a0 con 0x7f618000d750 2023-10-05T19:40:24.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.019+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc4ba0 con 0x559860f8b300 2023-10-05T19:40:24.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.019+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f618800a5a0 con 0x559860f8b300 2023-10-05T19:40:24.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.019+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc4ba0 con 0x559860fbd900 2023-10-05T19:40:24.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.023+0000 7f618eb7e700 1 -- 172.21.15.12:0/1247925877 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f6180036040 con 0x559860fbd900 2023-10-05T19:40:24.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.023+0000 7f6191049e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:24.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.023+0000 7f6191049e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:24.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.023+0000 7f6191049e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:24.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.023+0000 7f6191049e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:24.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.023+0000 7f6191049e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.023+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc1030 con 0x559860f8b300 2023-10-05T19:40:24.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.023+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f618800a5a0 con 0x559860f8b300 2023-10-05T19:40:24.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.023+0000 7f6191049e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:24.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.023+0000 7f6191049e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.023+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc1980 con 0x559860f8b300 2023-10-05T19:40:24.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.023+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f618800a5a0 con 0x559860f8b300 2023-10-05T19:40:24.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.023+0000 7f6191049e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:24.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.023+0000 7f6191049e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.023+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc0bf0 con 0x559860fc2280 2023-10-05T19:40:24.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.023+0000 7f618e37d700 1 -- 172.21.15.12:0/1247925877 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f6178008a80 con 0x559860fc2280 2023-10-05T19:40:24.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.023+0000 7f6191049e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:24.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.023+0000 7f6191049e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.023+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc54d0 con 0x559860fc2280 2023-10-05T19:40:24.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.023+0000 7f618e37d700 1 -- 172.21.15.12:0/1247925877 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f6178008a80 con 0x559860fc2280 2023-10-05T19:40:24.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.023+0000 7f6191049e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:24.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.023+0000 7f6191049e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.023+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc5880 con 0x559860f8b300 2023-10-05T19:40:24.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.027+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f618800a5a0 con 0x559860f8b300 2023-10-05T19:40:24.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.027+0000 7f6191049e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:24.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.027+0000 7f6191049e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.027+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc5c30 con 0x559860f8b300 2023-10-05T19:40:24.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.027+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f618800a5a0 con 0x559860f8b300 2023-10-05T19:40:24.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.027+0000 7f6191049e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:24.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.027+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc6c40 con 0x559860fc2280 2023-10-05T19:40:24.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.027+0000 7f618e37d700 1 -- 172.21.15.12:0/1247925877 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f6178008a80 con 0x559860fc2280 2023-10-05T19:40:24.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.027+0000 7f618e37d700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc6c40 con 0x559860fbd900 2023-10-05T19:40:24.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.031+0000 7f618eb7e700 1 -- 172.21.15.12:0/1247925877 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6180036040 con 0x559860fbd900 2023-10-05T19:40:24.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.031+0000 7f618eb7e700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc6c40 con 0x7f618000d750 2023-10-05T19:40:24.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.031+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f618800a5a0 con 0x7f618000d750 2023-10-05T19:40:24.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.031+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc5880 con 0x559860f8b300 2023-10-05T19:40:24.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.031+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f618800a5a0 con 0x559860f8b300 2023-10-05T19:40:24.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.031+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc5880 con 0x559860f8b300 2023-10-05T19:40:24.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.031+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f618800a5a0 con 0x559860f8b300 2023-10-05T19:40:24.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.031+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc5880 con 0x7f618000d750 2023-10-05T19:40:24.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.035+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f618800a5a0 con 0x7f618000d750 2023-10-05T19:40:24.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.035+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc5880 con 0x7f618000d750 2023-10-05T19:40:24.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.035+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f618800a5a0 con 0x7f618000d750 2023-10-05T19:40:24.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.035+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc5880 con 0x559860fbd900 2023-10-05T19:40:24.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.035+0000 7f618eb7e700 1 -- 172.21.15.12:0/1247925877 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6180036040 con 0x559860fbd900 2023-10-05T19:40:24.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.035+0000 7f618eb7e700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc5880 con 0x559860fc2280 2023-10-05T19:40:24.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.035+0000 7f618e37d700 1 -- 172.21.15.12:0/1247925877 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6178008a80 con 0x559860fc2280 2023-10-05T19:40:24.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.035+0000 7f618e37d700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc5880 con 0x559860f8b300 2023-10-05T19:40:24.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.035+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f618800a5a0 con 0x559860f8b300 2023-10-05T19:40:24.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.035+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc5880 con 0x7f618000d750 2023-10-05T19:40:24.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.039+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f618800a5a0 con 0x7f618000d750 2023-10-05T19:40:24.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.039+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc5880 con 0x559860f8b300 2023-10-05T19:40:24.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.039+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f618800a5a0 con 0x559860f8b300 2023-10-05T19:40:24.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.039+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc5880 con 0x559860fbd900 2023-10-05T19:40:24.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.039+0000 7f618eb7e700 1 -- 172.21.15.12:0/1247925877 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6180036040 con 0x559860fbd900 2023-10-05T19:40:24.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.039+0000 7f618eb7e700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6180046040 con 0x559860fc2280 2023-10-05T19:40:24.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.039+0000 7f618e37d700 1 -- 172.21.15.12:0/1247925877 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6178008a80 con 0x559860fc2280 2023-10-05T19:40:24.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.039+0000 7f618e37d700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6180046040 con 0x559860fc2280 2023-10-05T19:40:24.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.039+0000 7f618e37d700 1 -- 172.21.15.12:0/1247925877 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6178008a80 con 0x559860fc2280 2023-10-05T19:40:24.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.039+0000 7f618e37d700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6180046040 con 0x7f618000d750 2023-10-05T19:40:24.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.043+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f618800a5a0 con 0x7f618000d750 2023-10-05T19:40:24.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.043+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6180046040 con 0x559860fc2280 2023-10-05T19:40:24.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.043+0000 7f618e37d700 1 -- 172.21.15.12:0/1247925877 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6178008a80 con 0x559860fc2280 2023-10-05T19:40:24.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.043+0000 7f618e37d700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6180046040 con 0x559860fc2280 2023-10-05T19:40:24.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.043+0000 7f618e37d700 1 -- 172.21.15.12:0/1247925877 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6178008a80 con 0x559860fc2280 2023-10-05T19:40:24.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.043+0000 7f618e37d700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6180046040 con 0x559860f8b300 2023-10-05T19:40:24.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.043+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f618800a5a0 con 0x559860f8b300 2023-10-05T19:40:24.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.043+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6180046040 con 0x559860f8b300 2023-10-05T19:40:24.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.043+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f618800a5a0 con 0x559860f8b300 2023-10-05T19:40:24.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.043+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6180046040 con 0x7f618000d750 2023-10-05T19:40:24.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.047+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f618800a5a0 con 0x7f618000d750 2023-10-05T19:40:24.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.047+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6180046040 con 0x559860fc2280 2023-10-05T19:40:24.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.047+0000 7f618e37d700 1 -- 172.21.15.12:0/1247925877 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6178008a80 con 0x559860fc2280 2023-10-05T19:40:24.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.047+0000 7f618e37d700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6180046040 con 0x559860fbd900 2023-10-05T19:40:24.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.047+0000 7f618eb7e700 1 -- 172.21.15.12:0/1247925877 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6180036040 con 0x559860fbd900 2023-10-05T19:40:24.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.047+0000 7f618eb7e700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6180046040 con 0x559860f8b300 2023-10-05T19:40:24.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.047+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f618800a5a0 con 0x559860f8b300 2023-10-05T19:40:24.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.047+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6180046040 con 0x7f618000d750 2023-10-05T19:40:24.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.047+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f618800a5a0 con 0x7f618000d750 2023-10-05T19:40:24.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.047+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6180046040 con 0x559860fc2280 2023-10-05T19:40:24.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.047+0000 7f618e37d700 1 -- 172.21.15.12:0/1247925877 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6178008a80 con 0x559860fc2280 2023-10-05T19:40:24.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.051+0000 7f618e37d700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6180046040 con 0x7f618000d750 2023-10-05T19:40:24.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.051+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f618800a5a0 con 0x7f618000d750 2023-10-05T19:40:24.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.051+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6180046040 con 0x559860fbd900 2023-10-05T19:40:24.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.051+0000 7f618eb7e700 1 -- 172.21.15.12:0/1247925877 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6180036040 con 0x559860fbd900 2023-10-05T19:40:24.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.051+0000 7f618eb7e700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6180046040 con 0x559860fc2280 2023-10-05T19:40:24.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.051+0000 7f618e37d700 1 -- 172.21.15.12:0/1247925877 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6178008a80 con 0x559860fc2280 2023-10-05T19:40:24.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.051+0000 7f618e37d700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6180046040 con 0x7f618000d750 2023-10-05T19:40:24.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.051+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f618800a5a0 con 0x7f618000d750 2023-10-05T19:40:24.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.051+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6180046040 con 0x559860f8b300 2023-10-05T19:40:24.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.055+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f618800a5a0 con 0x559860f8b300 2023-10-05T19:40:24.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.055+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6180046040 con 0x559860fbd900 2023-10-05T19:40:24.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.055+0000 7f618eb7e700 1 -- 172.21.15.12:0/1247925877 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f6180036040 con 0x559860fbd900 2023-10-05T19:40:24.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.055+0000 7f6191049e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:24.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.055+0000 7f6191049e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:24.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.055+0000 7f6191049e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:24.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.055+0000 7f6191049e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:24.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.055+0000 7f6191049e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.055+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc5c30 con 0x559860f8b300 2023-10-05T19:40:24.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.055+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f618800a5a0 con 0x559860f8b300 2023-10-05T19:40:24.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.055+0000 7f6191049e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:24.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.055+0000 7f6191049e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.055+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc5fe0 con 0x559860fbd900 2023-10-05T19:40:24.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.055+0000 7f618eb7e700 1 -- 172.21.15.12:0/1247925877 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f6180036040 con 0x559860fbd900 2023-10-05T19:40:24.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.055+0000 7f6191049e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:24.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.055+0000 7f6191049e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.055+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc6880 con 0x559860f8b300 2023-10-05T19:40:24.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.055+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f618800a5a0 con 0x559860f8b300 2023-10-05T19:40:24.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.055+0000 7f6191049e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:24.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.055+0000 7f6191049e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.055+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc6880 con 0x559860fc2280 2023-10-05T19:40:24.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.059+0000 7f618e37d700 1 -- 172.21.15.12:0/1247925877 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f6178008a80 con 0x559860fc2280 2023-10-05T19:40:24.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.059+0000 7f6191049e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:24.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.059+0000 7f6191049e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.059+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc6c30 con 0x7f618000d750 2023-10-05T19:40:24.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.059+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f618800a5a0 con 0x7f618000d750 2023-10-05T19:40:24.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.059+0000 7f6191049e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:24.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.059+0000 7f6191049e40 20 zone default found 2023-10-05T19:40:24.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.059+0000 7f6191049e40 4 Realm: () 2023-10-05T19:40:24.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.059+0000 7f6191049e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:24.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.059+0000 7f6191049e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:24.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.059+0000 7f6191049e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:24.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.059+0000 7f6191049e40 20 zonegroup default 2023-10-05T19:40:24.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.059+0000 7f6191049e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.059+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc6c30 con 0x7f618000d750 2023-10-05T19:40:24.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.059+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f618800a5a0 con 0x7f618000d750 2023-10-05T19:40:24.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.059+0000 7f6191049e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:24.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.059+0000 7f6191049e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.059+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc85b0 con 0x559860fc2280 2023-10-05T19:40:24.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.059+0000 7f618e37d700 1 -- 172.21.15.12:0/1247925877 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f6178008a80 con 0x559860fc2280 2023-10-05T19:40:24.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.059+0000 7f6191049e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:24.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.059+0000 7f6191049e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.059+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc8960 con 0x559860fc2280 2023-10-05T19:40:24.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.059+0000 7f618e37d700 1 -- 172.21.15.12:0/1247925877 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f6178008a80 con 0x559860fc2280 2023-10-05T19:40:24.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.059+0000 7f6191049e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:24.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.059+0000 7f6191049e40 20 started sync module instance, tier type = 2023-10-05T19:40:24.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.059+0000 7f6191049e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:24.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.063+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc7c70 con 0x559860fbd900 2023-10-05T19:40:24.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.063+0000 7f618eb7e700 1 -- 172.21.15.12:0/1247925877 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'96 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6180036040 con 0x559860fbd900 2023-10-05T19:40:24.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.063+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94112950550640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc64b0 con 0x559860fbd900 2023-10-05T19:40:24.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.063+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc96f0 con 0x559860f8b300 2023-10-05T19:40:24.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.067+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'96 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f618800a5a0 con 0x559860f8b300 2023-10-05T19:40:24.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.067+0000 7f618eb7e700 1 -- 172.21.15.12:0/1247925877 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94112950550640] v28'97 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6180036040 con 0x559860fbd900 2023-10-05T19:40:24.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.067+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94112950555424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc9fe0 con 0x559860f8b300 2023-10-05T19:40:24.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.067+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fc8d10 con 0x559860fc2280 2023-10-05T19:40:24.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.071+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94112950555424] v28'97 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f618800a5a0 con 0x559860f8b300 2023-10-05T19:40:24.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.071+0000 7f618e37d700 1 -- 172.21.15.12:0/1247925877 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'96 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6178008a80 con 0x559860fc2280 2023-10-05T19:40:24.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.071+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94112950557424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fca430 con 0x559860fc2280 2023-10-05T19:40:24.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.071+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fcafc0 con 0x559860fbd900 2023-10-05T19:40:24.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.075+0000 7f618eb7e700 1 -- 172.21.15.12:0/1247925877 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'96 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6180036040 con 0x559860fbd900 2023-10-05T19:40:24.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.075+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94112950561424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fcb8b0 con 0x559860fbd900 2023-10-05T19:40:24.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.075+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fcc2c0 con 0x559860f8b300 2023-10-05T19:40:24.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.075+0000 7f618e37d700 1 -- 172.21.15.12:0/1247925877 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94112950557424] v28'97 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6178008a80 con 0x559860fc2280 2023-10-05T19:40:24.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.075+0000 7f618eb7e700 1 -- 172.21.15.12:0/1247925877 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94112950561424] v28'97 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6180036040 con 0x559860fbd900 2023-10-05T19:40:24.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.075+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'96 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f618800a5a0 con 0x559860f8b300 2023-10-05T19:40:24.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.075+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94112950563776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fcb810 con 0x559860f8b300 2023-10-05T19:40:24.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.075+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fcc7b0 con 0x7f618000d750 2023-10-05T19:40:24.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.079+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'96 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f618800a5a0 con 0x7f618000d750 2023-10-05T19:40:24.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.079+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94112950567616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fccc00 con 0x7f618000d750 2023-10-05T19:40:24.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.079+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fcd610 con 0x7f618000d750 2023-10-05T19:40:24.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.079+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94112950563776] v28'97 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f618800a5a0 con 0x559860f8b300 2023-10-05T19:40:24.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.083+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(96 notify.6 [create] v28'96 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f618800a5a0 con 0x7f618000d750 2023-10-05T19:40:24.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.083+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94112950566352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fcd610 con 0x7f618000d750 2023-10-05T19:40:24.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.083+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fce020 con 0x7f618000d750 2023-10-05T19:40:24.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.083+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(95 notify.5 [watch watch cookie 94112950567616] v28'97 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f618800a5a0 con 0x7f618000d750 2023-10-05T19:40:24.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.083+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(98 notify.7 [create] v28'96 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f618800a5a0 con 0x7f618000d750 2023-10-05T19:40:24.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.083+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94112950568720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fce020 con 0x7f618000d750 2023-10-05T19:40:24.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.083+0000 7f6191049e40 20 add_watcher() i=0 2023-10-05T19:40:24.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.083+0000 7f6191049e40 20 add_watcher() i=1 2023-10-05T19:40:24.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.083+0000 7f6191049e40 20 add_watcher() i=2 2023-10-05T19:40:24.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.083+0000 7f6191049e40 20 add_watcher() i=3 2023-10-05T19:40:24.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.083+0000 7f6191049e40 20 add_watcher() i=4 2023-10-05T19:40:24.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.083+0000 7f6191049e40 20 add_watcher() i=5 2023-10-05T19:40:24.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.083+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(97 notify.6 [watch watch cookie 94112950566352] v28'97 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f618800a5a0 con 0x7f618000d750 2023-10-05T19:40:24.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.083+0000 7f6191049e40 20 add_watcher() i=6 2023-10-05T19:40:24.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.087+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94112950568720] v28'97 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f618800a5a0 con 0x7f618000d750 2023-10-05T19:40:24.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.087+0000 7f6191049e40 20 add_watcher() i=7 2023-10-05T19:40:24.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.087+0000 7f6191049e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:24.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.087+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fd0b60 con 0x559860f8b300 2023-10-05T19:40:24.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.087+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f618800a5a0 con 0x559860f8b300 2023-10-05T19:40:24.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.087+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94112950571296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559860fce0f0 con 0x559860f8b300 2023-10-05T19:40:24.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.091+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94112950571296] v28'99 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f618800a5a0 con 0x559860f8b300 2023-10-05T19:40:24.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.091+0000 7f615f7ee700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:24.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.091+0000 7f6191049e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:24.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.091+0000 7f6191049e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:24.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.091+0000 7f6191049e40 1 Processor -- start 2023-10-05T19:40:24.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.091+0000 7f6191049e40 1 -- start start 2023-10-05T19:40:24.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.091+0000 7f6191049e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559860fdb090 0x559860fdb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:24.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.091+0000 7f6191049e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559860fdb9a0 con 0x559860fdb090 2023-10-05T19:40:24.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.091+0000 7f618e37d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559860fdb090 0x559860fdb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.091+0000 7f618e37d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559860fdb090 0x559860fdb460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57672/0 (socket says 172.21.15.12:57672) 2023-10-05T19:40:24.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.091+0000 7f618e37d700 1 -- 172.21.15.12:0/1739571672 learned_addr learned my addr 172.21.15.12:0/1739571672 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:24.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.091+0000 7f618e37d700 1 -- 172.21.15.12:0/1739571672 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559860fdbae0 con 0x559860fdb090 2023-10-05T19:40:24.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.091+0000 7f618e37d700 1 --2- 172.21.15.12:0/1739571672 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559860fdb090 0x559860fdb460 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x559860f87da0 tx=0x7f6178012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce555d28535c0b58 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.091+0000 7f615efed700 1 -- 172.21.15.12:0/1739571672 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f61780308b0 con 0x559860fdb090 2023-10-05T19:40:24.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.091+0000 7f615efed700 1 -- 172.21.15.12:0/1739571672 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6178036040 con 0x559860fdb090 2023-10-05T19:40:24.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.091+0000 7f615efed700 1 -- 172.21.15.12:0/1739571672 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6178030a70 con 0x559860fdb090 2023-10-05T19:40:24.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.095+0000 7f6191049e40 1 -- 172.21.15.12:0/1739571672 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559860fdb090 msgr2=0x559860fdb460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.095+0000 7f6191049e40 1 --2- 172.21.15.12:0/1739571672 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559860fdb090 0x559860fdb460 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x559860f87da0 tx=0x7f6178012000 comp rx=0 tx=0).stop 2023-10-05T19:40:24.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.095+0000 7f6191049e40 1 -- 172.21.15.12:0/1739571672 shutdown_connections 2023-10-05T19:40:24.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.095+0000 7f6191049e40 1 --2- 172.21.15.12:0/1739571672 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559860fdb090 0x559860fdb460 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:24.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.095+0000 7f6191049e40 1 -- 172.21.15.12:0/1739571672 >> 172.21.15.12:0/1739571672 conn(0x559860fd6450 msgr2=0x559860fd4600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:24.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.095+0000 7f6191049e40 1 -- 172.21.15.12:0/1739571672 shutdown_connections 2023-10-05T19:40:24.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.095+0000 7f6191049e40 1 -- 172.21.15.12:0/1739571672 wait complete. 2023-10-05T19:40:24.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.095+0000 7f6191049e40 1 Processor -- start 2023-10-05T19:40:24.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.095+0000 7f6191049e40 1 -- start start 2023-10-05T19:40:24.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.095+0000 7f6191049e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559860fdb090 0x559860fe0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:24.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.095+0000 7f6191049e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559860fe1b40 con 0x559860fdb090 2023-10-05T19:40:24.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.095+0000 7f618e37d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559860fdb090 0x559860fe0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.095+0000 7f618e37d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559860fdb090 0x559860fe0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57682/0 (socket says 172.21.15.12:57682) 2023-10-05T19:40:24.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.095+0000 7f618e37d700 1 -- 172.21.15.12:0/3083994181 learned_addr learned my addr 172.21.15.12:0/3083994181 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:24.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.095+0000 7f618e37d700 1 -- 172.21.15.12:0/3083994181 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559860f87930 con 0x559860fdb090 2023-10-05T19:40:24.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.095+0000 7f618e37d700 1 --2- 172.21.15.12:0/3083994181 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559860fdb090 0x559860fe0b40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x559860fd5170 tx=0x7f6178037000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.095+0000 7f615d7ea700 1 -- 172.21.15.12:0/3083994181 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6178027b80 con 0x559860fdb090 2023-10-05T19:40:24.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.095+0000 7f615d7ea700 1 -- 172.21.15.12:0/3083994181 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6178027d00 con 0x559860fdb090 2023-10-05T19:40:24.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.095+0000 7f6191049e40 1 -- 172.21.15.12:0/3083994181 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559860fe1140 con 0x559860fdb090 2023-10-05T19:40:24.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.095+0000 7f615d7ea700 1 -- 172.21.15.12:0/3083994181 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f617804c3f0 con 0x559860fdb090 2023-10-05T19:40:24.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.095+0000 7f6191049e40 1 -- 172.21.15.12:0/3083994181 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559860fe15e0 con 0x559860fdb090 2023-10-05T19:40:24.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.099+0000 7f615d7ea700 1 -- 172.21.15.12:0/3083994181 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f6178055440 con 0x559860fdb090 2023-10-05T19:40:24.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.099+0000 7f615d7ea700 1 --2- 172.21.15.12:0/3083994181 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f60d4030c00 0x7f60d40330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:24.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.099+0000 7f615d7ea700 1 -- 172.21.15.12:0/3083994181 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f617804dd20 con 0x559860fdb090 2023-10-05T19:40:24.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.099+0000 7f618f37f700 1 --2- 172.21.15.12:0/3083994181 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f60d4030c00 0x7f60d40330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.099+0000 7f6191049e40 5 note: GC not initialized 2023-10-05T19:40:24.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.099+0000 7f618f37f700 1 --2- 172.21.15.12:0/3083994181 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f60d4030c00 0x7f60d40330b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x559860fce050 tx=0x7f618800adc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.099+0000 7f60ce7fc700 20 reqs_thread_entry: start 2023-10-05T19:40:24.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.103+0000 7f60bffff700 20 reqs_thread_entry: start 2023-10-05T19:40:24.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.103+0000 7f6191049e40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:24.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.103+0000 7f6191049e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:24.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.103+0000 7f60be7fc700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:54 2023 2023-10-05T19:40:24.111 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:24.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.103+0000 7f60bdffb700 20 reqs_thread_entry: start 2023-10-05T19:40:24.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.103+0000 7f6191049e40 20 get_system_obj_state: rctx=0x7ffd68478348 obj=default.rgw.meta:users.uid:foo state=0x559860e610b0 s->prefetch_data=0 2023-10-05T19:40:24.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.103+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559861049310 con 0x559860f8b300 2023-10-05T19:40:24.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.103+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(102 foo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7f618803e070 con 0x559860f8b300 2023-10-05T19:40:24.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.103+0000 7f6191049e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:24.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.103+0000 7f6191049e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:24.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.103+0000 7f6191049e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.103+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559861049310 con 0x559860f8b300 2023-10-05T19:40:24.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.103+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(103 foo [call,call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv11 ondisk = 0) v8 ==== 273+0+480 (crc 0 0 0) 0x7f618803e070 con 0x559860f8b300 2023-10-05T19:40:24.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.103+0000 7f6191049e40 20 rados_obj.operate() r=0 bl.length=357 2023-10-05T19:40:24.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.103+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:104 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559861049310 con 0x559860fbd900 2023-10-05T19:40:24.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.103+0000 7f618eb7e700 1 -- 172.21.15.12:0/1247925877 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(104 foo.buckets [call out=183b] v0'0 uv14 ondisk = 0) v8 ==== 155+0+183 (crc 0 0 0) 0x7f6180036040 con 0x559860fbd900 2023-10-05T19:40:24.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.103+0000 7f6191049e40 20 get_system_obj_state: rctx=0x7ffd68478200 obj=default.rgw.meta:root:myfoo state=0x559860e610b0 s->prefetch_data=0 2023-10-05T19:40:24.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.103+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:105 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559861049310 con 0x559860fbd900 2023-10-05T19:40:24.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.103+0000 7f618eb7e700 1 -- 172.21.15.12:0/1247925877 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(105 myfoo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 233+0+139 (crc 0 0 0) 0x7f6180036040 con 0x559860fbd900 2023-10-05T19:40:24.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.107+0000 7f6191049e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:24.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.107+0000 7f6191049e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.107+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:106 8.9 8:91b71841:root::myfoo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55986104b0c0 con 0x559860fbd900 2023-10-05T19:40:24.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.107+0000 7f618eb7e700 1 -- 172.21.15.12:0/1247925877 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(106 myfoo [call,call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) v8 ==== 233+0+213 (crc 0 0 0) 0x7f6180036040 con 0x559860fbd900 2023-10-05T19:40:24.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.107+0000 7f6191049e40 20 rados_obj.operate() r=0 bl.length=165 2023-10-05T19:40:24.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.107+0000 7f6191049e40 20 get_system_obj_state: rctx=0x7ffd68478200 obj=default.rgw.meta:root:.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 state=0x559860e372b0 s->prefetch_data=0 2023-10-05T19:40:24.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.107+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:107 8.e 8:70176bbd:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55986104b0c0 con 0x7f618000d750 2023-10-05T19:40:24.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.107+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(107 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 [call out=48b,getxattrs out=218b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 290+0+282 (crc 0 0 0) 0x7f618803e070 con 0x7f618000d750 2023-10-05T19:40:24.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.107+0000 7f6191049e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:24.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.107+0000 7f6191049e40 20 Read xattr: user.rgw.acl 2023-10-05T19:40:24.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.107+0000 7f6191049e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:24.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.107+0000 7f6191049e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.107+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:108 8.e 8:70176bbd:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55986104e030 con 0x7f618000d750 2023-10-05T19:40:24.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.107+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(108 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 [call,call out=48b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 332+0+618 (crc 0 0 0) 0x7f618803e070 con 0x7f618000d750 2023-10-05T19:40:24.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.107+0000 7f6191049e40 20 rados_obj.operate() r=0 bl.length=352 2023-10-05T19:40:24.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.107+0000 7f6191049e40 20 couldn't put binfo cache entry, might have raced with data changes 2023-10-05T19:40:24.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.107+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:109 4.3 4:c350083b:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55986104a730 con 0x559860f8b300 2023-10-05T19:40:24.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.107+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:110 4.1c 4:38d73088:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55986104acb0 con 0x7f618000d750 2023-10-05T19:40:24.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.107+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:111 4.a 4:502f3015:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559861053070 con 0x559860fbd900 2023-10-05T19:40:24.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.107+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:112 4.3e 4:7e570c43:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559861055930 con 0x559860fbd900 2023-10-05T19:40:24.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.107+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:113 4.7 4:e38d8107:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559861058590 con 0x7f618000d750 2023-10-05T19:40:24.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.107+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:114 4.1a 4:5add8e75:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55986105b2f0 con 0x559860fbd900 2023-10-05T19:40:24.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.107+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:115 4.2c 4:358e5801:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55986105dc30 con 0x559860f8b300 2023-10-05T19:40:24.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.107+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:116 4.2 4:4101cfe2:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55986105b2f0 con 0x559860f8b300 2023-10-05T19:40:24.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.111+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:117 4.30 4:0e5788c0:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559861062ec0 con 0x559860fc2280 2023-10-05T19:40:24.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.111+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:118 4.19 4:99ccd5a2:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55986105b2f0 con 0x559860fc2280 2023-10-05T19:40:24.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.111+0000 7f618eb7e700 1 -- 172.21.15.12:0/1247925877 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(111 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.2 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f618004c040 con 0x559860fbd900 2023-10-05T19:40:24.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.111+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(110 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f618803e070 con 0x7f618000d750 2023-10-05T19:40:24.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.111+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.0 v2:172.21.15.8:6800/27563 30 ==== osd_op_reply(113 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f618803e070 con 0x7f618000d750 2023-10-05T19:40:24.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.111+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(109 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f618803e070 con 0x559860f8b300 2023-10-05T19:40:24.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.111+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:119 4.2d 4:b73f2ca3:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559861062ec0 con 0x7f618000d750 2023-10-05T19:40:24.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.111+0000 7f618e37d700 1 -- 172.21.15.12:0/1247925877 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(117 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.8 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f6178040020 con 0x559860fc2280 2023-10-05T19:40:24.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.111+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(116 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.7 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f618803e070 con 0x559860f8b300 2023-10-05T19:40:24.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.111+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(115 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.6 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f618803e070 con 0x559860f8b300 2023-10-05T19:40:24.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.111+0000 7f618eb7e700 1 -- 172.21.15.12:0/1247925877 <== osd.2 v2:172.21.15.12:6800/29934 23 ==== osd_op_reply(112 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.3 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f618004c040 con 0x559860fbd900 2023-10-05T19:40:24.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.111+0000 7f618eb7e700 1 -- 172.21.15.12:0/1247925877 <== osd.2 v2:172.21.15.12:6800/29934 24 ==== osd_op_reply(114 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.5 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f618004c040 con 0x559860fbd900 2023-10-05T19:40:24.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.111+0000 7f618e37d700 1 -- 172.21.15.12:0/1247925877 <== osd.3 v2:172.21.15.12:6816/29951 27 ==== osd_op_reply(118 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.9 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f6178040020 con 0x559860fc2280 2023-10-05T19:40:24.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.111+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.0 v2:172.21.15.8:6800/27563 31 ==== osd_op_reply(119 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7f618803e070 con 0x7f618000d750 2023-10-05T19:40:24.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.111+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:120 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94112950550640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559861062ec0 con 0x559860fbd900 2023-10-05T19:40:24.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.115+0000 7f618eb7e700 1 -- 172.21.15.12:0/1247925877 <== osd.2 v2:172.21.15.12:6800/29934 25 ==== osd_op_reply(120 notify.0 [watch unwatch cookie 94112950550640] v28'98 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f618004c040 con 0x559860fbd900 2023-10-05T19:40:24.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.115+0000 7f6191049e40 20 remove_watcher() i=0 2023-10-05T19:40:24.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.115+0000 7f6191049e40 2 removed watcher, disabling cache 2023-10-05T19:40:24.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.115+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:121 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94112950555424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559861062ec0 con 0x559860f8b300 2023-10-05T19:40:24.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.119+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== osd_op_reply(121 notify.1 [watch unwatch cookie 94112950555424] v28'98 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f618803e070 con 0x559860f8b300 2023-10-05T19:40:24.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.119+0000 7f6191049e40 20 remove_watcher() i=1 2023-10-05T19:40:24.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.119+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:122 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94112950557424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559861062ec0 con 0x559860fc2280 2023-10-05T19:40:24.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.119+0000 7f618e37d700 1 -- 172.21.15.12:0/1247925877 <== osd.3 v2:172.21.15.12:6816/29951 28 ==== osd_op_reply(122 notify.2 [watch unwatch cookie 94112950557424] v28'98 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6178040020 con 0x559860fc2280 2023-10-05T19:40:24.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.119+0000 7f6191049e40 20 remove_watcher() i=2 2023-10-05T19:40:24.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.119+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:123 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94112950561424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559861062ec0 con 0x559860fbd900 2023-10-05T19:40:24.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.123+0000 7f618eb7e700 1 -- 172.21.15.12:0/1247925877 <== osd.2 v2:172.21.15.12:6800/29934 26 ==== osd_op_reply(123 notify.3 [watch unwatch cookie 94112950561424] v28'98 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f618004c040 con 0x559860fbd900 2023-10-05T19:40:24.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.123+0000 7f6191049e40 20 remove_watcher() i=3 2023-10-05T19:40:24.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.123+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:124 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94112950563776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559861062ec0 con 0x559860f8b300 2023-10-05T19:40:24.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.127+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.1 v2:172.21.15.12:6808/29933 39 ==== osd_op_reply(124 notify.4 [watch unwatch cookie 94112950563776] v28'98 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f618803e070 con 0x559860f8b300 2023-10-05T19:40:24.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.127+0000 7f6191049e40 20 remove_watcher() i=4 2023-10-05T19:40:24.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.127+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:125 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94112950567616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559861062ec0 con 0x7f618000d750 2023-10-05T19:40:24.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.131+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.0 v2:172.21.15.8:6800/27563 32 ==== osd_op_reply(125 notify.5 [watch unwatch cookie 94112950567616] v28'98 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f618803e070 con 0x7f618000d750 2023-10-05T19:40:24.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.131+0000 7f6191049e40 20 remove_watcher() i=5 2023-10-05T19:40:24.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.131+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:126 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94112950566352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559861062ec0 con 0x7f618000d750 2023-10-05T19:40:24.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.131+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.0 v2:172.21.15.8:6800/27563 33 ==== osd_op_reply(126 notify.6 [watch unwatch cookie 94112950566352] v28'98 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f618803e070 con 0x7f618000d750 2023-10-05T19:40:24.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.131+0000 7f6191049e40 20 remove_watcher() i=6 2023-10-05T19:40:24.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.131+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:127 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94112950568720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559861062ec0 con 0x7f618000d750 2023-10-05T19:40:24.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.135+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.0 v2:172.21.15.8:6800/27563 34 ==== osd_op_reply(127 notify.7 [watch unwatch cookie 94112950568720] v28'98 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f618803e070 con 0x7f618000d750 2023-10-05T19:40:24.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.135+0000 7f6191049e40 20 remove_watcher() i=7 2023-10-05T19:40:24.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.139+0000 7f615f7ee700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:24.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.139+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:128 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94112950571296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559861062ec0 con 0x559860f8b300 2023-10-05T19:40:24.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.139+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 <== osd.1 v2:172.21.15.12:6808/29933 40 ==== osd_op_reply(128 data_loggenerations_metadata [watch unwatch cookie 94112950571296] v28'100 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f618803e070 con 0x559860f8b300 2023-10-05T19:40:24.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.139+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f618000d750 msgr2=0x7f6180042bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.139+0000 7f6191049e40 1 --2- 172.21.15.12:0/1247925877 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f618000d750 0x7f6180042bc0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:24.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.139+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x559860f8b300 msgr2=0x559860fbb880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.139+0000 7f6191049e40 1 --2- 172.21.15.12:0/1247925877 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x559860f8b300 0x559860fbb880 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:24.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.139+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x559860fbd900 msgr2=0x559860fbfce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.139+0000 7f6191049e40 1 --2- 172.21.15.12:0/1247925877 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x559860fbd900 0x559860fbfce0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:24.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.139+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x559860fc2280 msgr2=0x559860fc4660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.139+0000 7f6191049e40 1 --2- 172.21.15.12:0/1247925877 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x559860fc2280 0x559860fc4660 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:24.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.139+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f6174030c50 msgr2=0x7f6174033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.139+0000 7f6191049e40 1 --2- 172.21.15.12:0/1247925877 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f6174030c50 0x7f6174033100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6178005a90 tx=0x7f6178015000 comp rx=0 tx=0).stop 2023-10-05T19:40:24.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.139+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559860f86d20 msgr2=0x559860f90600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.139+0000 7f6191049e40 1 --2- 172.21.15.12:0/1247925877 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559860f86d20 0x559860f90600 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f61800017d0 tx=0x7f6180016000 comp rx=0 tx=0).stop 2023-10-05T19:40:24.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.139+0000 7f618f37f700 1 -- 172.21.15.12:0/1247925877 reap_dead start 2023-10-05T19:40:24.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.143+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 shutdown_connections 2023-10-05T19:40:24.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.143+0000 7f6191049e40 1 --2- 172.21.15.12:0/1247925877 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559860f86d20 0x559860f90600 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:24.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.143+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 >> 172.21.15.12:0/1247925877 conn(0x559860f82740 msgr2=0x559860fa13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:24.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.143+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 shutdown_connections 2023-10-05T19:40:24.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.143+0000 7f6191049e40 1 -- 172.21.15.12:0/1247925877 wait complete. 2023-10-05T19:40:24.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.143+0000 7f6191049e40 1 -- 172.21.15.12:0/3083994181 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f60d4030c00 msgr2=0x7f60d40330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.143+0000 7f6191049e40 1 --2- 172.21.15.12:0/3083994181 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f60d4030c00 0x7f60d40330b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x559860fce050 tx=0x7f618800adc0 comp rx=0 tx=0).stop 2023-10-05T19:40:24.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.143+0000 7f6191049e40 1 -- 172.21.15.12:0/3083994181 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559860fdb090 msgr2=0x559860fe0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.143+0000 7f6191049e40 1 --2- 172.21.15.12:0/3083994181 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559860fdb090 0x559860fe0b40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x559860fd5170 tx=0x7f6178037000 comp rx=0 tx=0).stop 2023-10-05T19:40:24.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.143+0000 7f6191049e40 1 -- 172.21.15.12:0/3083994181 shutdown_connections 2023-10-05T19:40:24.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.143+0000 7f6191049e40 1 --2- 172.21.15.12:0/3083994181 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f60d4030c00 0x7f60d40330b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:24.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.143+0000 7f6191049e40 1 --2- 172.21.15.12:0/3083994181 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559860fdb090 0x559860fe0b40 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:24.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.143+0000 7f6191049e40 1 -- 172.21.15.12:0/3083994181 >> 172.21.15.12:0/3083994181 conn(0x559860fd6450 msgr2=0x559860fd4550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:24.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.143+0000 7f6191049e40 1 -- 172.21.15.12:0/3083994181 shutdown_connections 2023-10-05T19:40:24.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.143+0000 7f6191049e40 1 -- 172.21.15.12:0/3083994181 wait complete. 2023-10-05T19:40:24.154 INFO:teuthology.orchestra.run.smithi012.stdout:[ 2023-10-05T19:40:24.154 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:24.154 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo", 2023-10-05T19:40:24.154 INFO:teuthology.orchestra.run.smithi012.stdout: "num_shards": 11, 2023-10-05T19:40:24.154 INFO:teuthology.orchestra.run.smithi012.stdout: "tenant": "", 2023-10-05T19:40:24.154 INFO:teuthology.orchestra.run.smithi012.stdout: "zonegroup": "1602ab00-ae0f-4921-ad63-b9cfe7bc77db", 2023-10-05T19:40:24.154 INFO:teuthology.orchestra.run.smithi012.stdout: "placement_rule": "default-placement", 2023-10-05T19:40:24.155 INFO:teuthology.orchestra.run.smithi012.stdout: "explicit_placement": { 2023-10-05T19:40:24.155 INFO:teuthology.orchestra.run.smithi012.stdout: "data_pool": "", 2023-10-05T19:40:24.155 INFO:teuthology.orchestra.run.smithi012.stdout: "data_extra_pool": "", 2023-10-05T19:40:24.155 INFO:teuthology.orchestra.run.smithi012.stdout: "index_pool": "" 2023-10-05T19:40:24.155 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:24.155 INFO:teuthology.orchestra.run.smithi012.stdout: "id": "93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1", 2023-10-05T19:40:24.155 INFO:teuthology.orchestra.run.smithi012.stdout: "marker": "93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1", 2023-10-05T19:40:24.155 INFO:teuthology.orchestra.run.smithi012.stdout: "index_type": "Normal", 2023-10-05T19:40:24.155 INFO:teuthology.orchestra.run.smithi012.stdout: "owner": "foo", 2023-10-05T19:40:24.155 INFO:teuthology.orchestra.run.smithi012.stdout: "ver": "0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1", 2023-10-05T19:40:24.155 INFO:teuthology.orchestra.run.smithi012.stdout: "master_ver": "0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0", 2023-10-05T19:40:24.155 INFO:teuthology.orchestra.run.smithi012.stdout: "mtime": "2023-10-05T19:40:22.865997Z", 2023-10-05T19:40:24.155 INFO:teuthology.orchestra.run.smithi012.stdout: "creation_time": "2023-10-05T19:40:22.857926Z", 2023-10-05T19:40:24.156 INFO:teuthology.orchestra.run.smithi012.stdout: "max_marker": "0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#", 2023-10-05T19:40:24.156 INFO:teuthology.orchestra.run.smithi012.stdout: "usage": {}, 2023-10-05T19:40:24.156 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket_quota": { 2023-10-05T19:40:24.156 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:24.156 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:24.156 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:24.156 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:24.156 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:24.156 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:24.156 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:24.156 INFO:teuthology.orchestra.run.smithi012.stdout:] 2023-10-05T19:40:24.157 DEBUG:tasks.util.rgw: json result: [{'bucket': 'myfoo', 'num_shards': 11, 'tenant': '', 'zonegroup': '1602ab00-ae0f-4921-ad63-b9cfe7bc77db', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1', 'marker': '93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1', 'index_type': 'Normal', 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2023-10-05T19:40:22.865997Z', 'creation_time': '2023-10-05T19:40:22.857926Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}}] 2023-10-05T19:40:24.171 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=3 bytes_in=0 failed=False 2023-10-05T19:40:24.171 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'stats', '--bucket', 'myfoo'] 2023-10-05T19:40:24.171 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'stats', '--bucket', 'myfoo'] 2023-10-05T19:40:24.172 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket stats --bucket myfoo 2023-10-05T19:40:24.211 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:24.211 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:24.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.203+0000 7fbe8e446e40 1 Processor -- start 2023-10-05T19:40:24.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.203+0000 7fbe8e446e40 1 -- start start 2023-10-05T19:40:24.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.203+0000 7fbe8e446e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x562c2224c3d0 0x562c222dabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:24.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.203+0000 7fbe8e446e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x562c22224bd0 con 0x562c222db110 2023-10-05T19:40:24.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.203+0000 7fbe8e446e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x562c221e1ee0 con 0x562c2224c3d0 2023-10-05T19:40:24.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.203+0000 7fbe8bf7b700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x562c2224c3d0 0x562c222dabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.203+0000 7fbe8b77a700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x562c222db110 0x562c222dd4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:35486/0 (socket says 172.21.15.12:35486) 2023-10-05T19:40:24.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.203+0000 7fbe8b77a700 1 -- 172.21.15.12:0/2381924402 learned_addr learned my addr 172.21.15.12:0/2381924402 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:24.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.203+0000 7fbe8bf7b700 1 -- 172.21.15.12:0/2381924402 >> v1:172.21.15.8:6789/0 conn(0x562c222db110 legacy=0x562c222dd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.203+0000 7fbe8bf7b700 1 -- 172.21.15.12:0/2381924402 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c222dfc20 con 0x562c2224c3d0 2023-10-05T19:40:24.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.207+0000 7fbe8bf7b700 1 --2- 172.21.15.12:0/2381924402 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562c2224c3d0 0x562c222dabd0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fbe8400a520 tx=0x7fbe84005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35bed951b2608222 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.207+0000 7fbe8af79700 1 -- 172.21.15.12:0/2381924402 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe8400f590 con 0x562c2224c3d0 2023-10-05T19:40:24.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.207+0000 7fbe8af79700 1 -- 172.21.15.12:0/2381924402 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe8400f6f0 con 0x562c2224c3d0 2023-10-05T19:40:24.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.207+0000 7fbe8af79700 1 -- 172.21.15.12:0/2381924402 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe8400f880 con 0x562c2224c3d0 2023-10-05T19:40:24.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.207+0000 7fbe8e446e40 1 -- 172.21.15.12:0/2381924402 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562c2224c3d0 msgr2=0x562c222dabd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.207+0000 7fbe8e446e40 1 --2- 172.21.15.12:0/2381924402 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562c2224c3d0 0x562c222dabd0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fbe8400a520 tx=0x7fbe84005d40 comp rx=0 tx=0).stop 2023-10-05T19:40:24.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.207+0000 7fbe8e446e40 1 -- 172.21.15.12:0/2381924402 shutdown_connections 2023-10-05T19:40:24.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.207+0000 7fbe8e446e40 1 --2- 172.21.15.12:0/2381924402 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562c2224c3d0 0x562c222dabd0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:24.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.207+0000 7fbe8e446e40 1 -- 172.21.15.12:0/2381924402 >> 172.21.15.12:0/2381924402 conn(0x562c222d2720 msgr2=0x562c222d4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:24.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.207+0000 7fbe8e446e40 1 -- 172.21.15.12:0/2381924402 shutdown_connections 2023-10-05T19:40:24.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.207+0000 7fbe8e446e40 1 -- 172.21.15.12:0/2381924402 wait complete. 2023-10-05T19:40:24.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.207+0000 7fbe8e446e40 1 Processor -- start 2023-10-05T19:40:24.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.211+0000 7fbe8e446e40 1 -- start start 2023-10-05T19:40:24.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.211+0000 7fbe8e446e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562c2224c3d0 0x562c222f6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:24.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.211+0000 7fbe8e446e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c221a5e30 con 0x562c2224c3d0 2023-10-05T19:40:24.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.211+0000 7fbe8bf7b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562c2224c3d0 0x562c222f6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.211+0000 7fbe8bf7b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562c2224c3d0 0x562c222f6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57712/0 (socket says 172.21.15.12:57712) 2023-10-05T19:40:24.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.211+0000 7fbe8bf7b700 1 -- 172.21.15.12:0/3822244925 learned_addr learned my addr 172.21.15.12:0/3822244925 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:24.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.211+0000 7fbe8bf7b700 1 -- 172.21.15.12:0/3822244925 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe8400a0e0 con 0x562c2224c3d0 2023-10-05T19:40:24.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.211+0000 7fbe8bf7b700 1 --2- 172.21.15.12:0/3822244925 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562c2224c3d0 0x562c222f6ee0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fbe8400a470 tx=0x7fbe84014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6baf0f97b5d23d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.211+0000 7fbe89f77700 1 -- 172.21.15.12:0/3822244925 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe8400f860 con 0x562c2224c3d0 2023-10-05T19:40:24.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.211+0000 7fbe89f77700 1 -- 172.21.15.12:0/3822244925 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe8400f9e0 con 0x562c2224c3d0 2023-10-05T19:40:24.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.211+0000 7fbe89f77700 1 -- 172.21.15.12:0/3822244925 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe8400fb90 con 0x562c2224c3d0 2023-10-05T19:40:24.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.211+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3822244925 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562c2224c3d0 msgr2=0x562c222f6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.211+0000 7fbe8e446e40 1 --2- 172.21.15.12:0/3822244925 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562c2224c3d0 0x562c222f6ee0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fbe8400a470 tx=0x7fbe84014000 comp rx=0 tx=0).stop 2023-10-05T19:40:24.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.211+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3822244925 shutdown_connections 2023-10-05T19:40:24.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.211+0000 7fbe8e446e40 1 --2- 172.21.15.12:0/3822244925 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562c2224c3d0 0x562c222f6ee0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:24.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.211+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3822244925 >> 172.21.15.12:0/3822244925 conn(0x562c222d2720 msgr2=0x562c222f2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:24.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.211+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3822244925 shutdown_connections 2023-10-05T19:40:24.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.211+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3822244925 wait complete. 2023-10-05T19:40:24.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.215+0000 7fbe8e446e40 1 Processor -- start 2023-10-05T19:40:24.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.215+0000 7fbe8e446e40 1 -- start start 2023-10-05T19:40:24.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.215+0000 7fbe8e446e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562c222db110 0x562c222fdd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:24.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.215+0000 7fbe8e446e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c22178ed0 con 0x562c222db110 2023-10-05T19:40:24.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.215+0000 7fbe8bf7b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562c222db110 0x562c222fdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.215+0000 7fbe8bf7b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562c222db110 0x562c222fdd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57720/0 (socket says 172.21.15.12:57720) 2023-10-05T19:40:24.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.215+0000 7fbe8bf7b700 1 -- 172.21.15.12:0/3712732475 learned_addr learned my addr 172.21.15.12:0/3712732475 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:24.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.215+0000 7fbe8bf7b700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c222dfc20 con 0x562c222db110 2023-10-05T19:40:24.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.215+0000 7fbe8bf7b700 1 --2- 172.21.15.12:0/3712732475 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562c222db110 0x562c222fdd10 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbe84000f80 tx=0x7fbe8401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.215+0000 7fbe73fff700 1 -- 172.21.15.12:0/3712732475 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe84028070 con 0x562c222db110 2023-10-05T19:40:24.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.215+0000 7fbe73fff700 1 -- 172.21.15.12:0/3712732475 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe84029040 con 0x562c222db110 2023-10-05T19:40:24.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.215+0000 7fbe73fff700 1 -- 172.21.15.12:0/3712732475 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe8402e460 con 0x562c222db110 2023-10-05T19:40:24.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.215+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562c222fe250 con 0x562c222db110 2023-10-05T19:40:24.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.215+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562c222fe630 con 0x562c222db110 2023-10-05T19:40:24.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.219+0000 7fbe73fff700 1 -- 172.21.15.12:0/3712732475 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fbe8402e5c0 con 0x562c222db110 2023-10-05T19:40:24.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.219+0000 7fbe73fff700 1 --2- 172.21.15.12:0/3712732475 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fbe78030ca0 0x7fbe78033150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:24.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.219+0000 7fbe73fff700 1 -- 172.21.15.12:0/3712732475 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7fbe84006000 con 0x562c222db110 2023-10-05T19:40:24.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.219+0000 7fbe8e446e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.219+0000 7fbe8b77a700 1 --2- 172.21.15.12:0/3712732475 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fbe78030ca0 0x7fbe78033150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.219+0000 7fbe8e446e40 1 --2- 172.21.15.12:0/3712732475 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x562c22307690 0x562c22309aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:24.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.219+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c2230a1d0 con 0x562c22307690 2023-10-05T19:40:24.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.219+0000 7fbe8c77c700 1 --2- 172.21.15.12:0/3712732475 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x562c22307690 0x562c22309aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.219+0000 7fbe8b77a700 1 --2- 172.21.15.12:0/3712732475 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fbe78030ca0 0x7fbe78033150 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbe80004170 tx=0x7fbe800012e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.219+0000 7fbe8c77c700 1 --2- 172.21.15.12:0/3712732475 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x562c22307690 0x562c22309aa0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.219+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fbe7c00a500 con 0x562c22307690 2023-10-05T19:40:24.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.219+0000 7fbe8e446e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:24.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.219+0000 7fbe8e446e40 20 realm 2023-10-05T19:40:24.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.219+0000 7fbe8e446e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.219+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22306cb0 con 0x562c22307690 2023-10-05T19:40:24.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.219+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fbe7c00a500 con 0x562c22307690 2023-10-05T19:40:24.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.219+0000 7fbe8e446e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:24.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.219+0000 7fbe8e446e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:24.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.223+0000 7fbe8e446e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.223+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c2230acb0 con 0x562c22307690 2023-10-05T19:40:24.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.223+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fbe7c00a500 con 0x562c22307690 2023-10-05T19:40:24.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.223+0000 7fbe8e446e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:24.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.223+0000 7fbe8e446e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.223+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c2230a890 con 0x562c22307690 2023-10-05T19:40:24.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.223+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fbe7c00a500 con 0x562c22307690 2023-10-05T19:40:24.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.223+0000 7fbe8e446e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:24.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.223+0000 7fbe8e446e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.223+0000 7fbe8e446e40 1 --2- 172.21.15.12:0/3712732475 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x562c2230bc40 0x562c2230e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:24.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.223+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c2230e790 con 0x562c2230bc40 2023-10-05T19:40:24.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.223+0000 7fbe8bf7b700 1 --2- 172.21.15.12:0/3712732475 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x562c2230bc40 0x562c2230e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.223+0000 7fbe8bf7b700 1 --2- 172.21.15.12:0/3712732475 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x562c2230bc40 0x562c2230e020 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.223+0000 7fbe8bf7b700 1 -- 172.21.15.12:0/3712732475 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fbe84034020 con 0x562c2230bc40 2023-10-05T19:40:24.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.223+0000 7fbe8e446e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:24.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.223+0000 7fbe8e446e40 20 searching for the correct realm 2023-10-05T19:40:24.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.223+0000 7fbe8e446e40 1 --2- 172.21.15.12:0/3712732475 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x562c223105c0 0x562c223129a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:24.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.223+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22312ee0 con 0x562c223105c0 2023-10-05T19:40:24.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.223+0000 7fbe8b77a700 1 --2- 172.21.15.12:0/3712732475 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x562c223105c0 0x562c223129a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.223+0000 7fbe8b77a700 1 --2- 172.21.15.12:0/3712732475 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x562c223105c0 0x562c223129a0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.227+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3712732475 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fbe80006070 con 0x562c223105c0 2023-10-05T19:40:24.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.227+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22312ee0 con 0x562c2230bc40 2023-10-05T19:40:24.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.227+0000 7fbe8bf7b700 1 -- 172.21.15.12:0/3712732475 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe84034020 con 0x562c2230bc40 2023-10-05T19:40:24.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.227+0000 7fbe8bf7b700 1 --2- 172.21.15.12:0/3712732475 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x562c2224c3d0 0x7fbe84024400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:24.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.227+0000 7fbe8bf7b700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22312ee0 con 0x562c2224c3d0 2023-10-05T19:40:24.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.227+0000 7fbe8c77c700 1 --2- 172.21.15.12:0/3712732475 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x562c2224c3d0 0x7fbe84024400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.227+0000 7fbe8c77c700 1 --2- 172.21.15.12:0/3712732475 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x562c2224c3d0 0x7fbe84024400 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.231+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe7c014040 con 0x562c2224c3d0 2023-10-05T19:40:24.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.231+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22312ee0 con 0x562c22307690 2023-10-05T19:40:24.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.231+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe7c014040 con 0x562c22307690 2023-10-05T19:40:24.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.231+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22312ee0 con 0x562c22307690 2023-10-05T19:40:24.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.231+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe7c014040 con 0x562c22307690 2023-10-05T19:40:24.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.231+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22312ee0 con 0x562c2224c3d0 2023-10-05T19:40:24.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.231+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe7c014040 con 0x562c2224c3d0 2023-10-05T19:40:24.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.231+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22312ee0 con 0x562c2224c3d0 2023-10-05T19:40:24.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.231+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe7c014040 con 0x562c2224c3d0 2023-10-05T19:40:24.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.231+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22312ee0 con 0x562c2230bc40 2023-10-05T19:40:24.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.235+0000 7fbe8bf7b700 1 -- 172.21.15.12:0/3712732475 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe84034020 con 0x562c2230bc40 2023-10-05T19:40:24.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.235+0000 7fbe8bf7b700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22312ee0 con 0x562c223105c0 2023-10-05T19:40:24.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.235+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3712732475 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe80006070 con 0x562c223105c0 2023-10-05T19:40:24.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.235+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22312ee0 con 0x562c22307690 2023-10-05T19:40:24.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.235+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe7c014040 con 0x562c22307690 2023-10-05T19:40:24.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.235+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22312ee0 con 0x562c2224c3d0 2023-10-05T19:40:24.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.235+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe7c014040 con 0x562c2224c3d0 2023-10-05T19:40:24.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.235+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22312ee0 con 0x562c22307690 2023-10-05T19:40:24.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.235+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe7c014040 con 0x562c22307690 2023-10-05T19:40:24.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.235+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22312ee0 con 0x562c2230bc40 2023-10-05T19:40:24.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.235+0000 7fbe8bf7b700 1 -- 172.21.15.12:0/3712732475 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe84034020 con 0x562c2230bc40 2023-10-05T19:40:24.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.235+0000 7fbe8bf7b700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22312ee0 con 0x562c223105c0 2023-10-05T19:40:24.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.239+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3712732475 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe80006070 con 0x562c223105c0 2023-10-05T19:40:24.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.239+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22312ee0 con 0x562c223105c0 2023-10-05T19:40:24.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.239+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3712732475 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe80006070 con 0x562c223105c0 2023-10-05T19:40:24.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.239+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22312ee0 con 0x562c2224c3d0 2023-10-05T19:40:24.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.239+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe7c014040 con 0x562c2224c3d0 2023-10-05T19:40:24.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.239+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22312ee0 con 0x562c223105c0 2023-10-05T19:40:24.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.239+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3712732475 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe80006070 con 0x562c223105c0 2023-10-05T19:40:24.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.239+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22312ee0 con 0x562c223105c0 2023-10-05T19:40:24.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.239+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3712732475 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe80006070 con 0x562c223105c0 2023-10-05T19:40:24.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.239+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22312ee0 con 0x562c22307690 2023-10-05T19:40:24.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.243+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe7c014040 con 0x562c22307690 2023-10-05T19:40:24.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.243+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22312ee0 con 0x562c22307690 2023-10-05T19:40:24.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.243+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe7c014040 con 0x562c22307690 2023-10-05T19:40:24.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.243+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22312ee0 con 0x562c2224c3d0 2023-10-05T19:40:24.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.243+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe7c014040 con 0x562c2224c3d0 2023-10-05T19:40:24.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.243+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22312ee0 con 0x562c223105c0 2023-10-05T19:40:24.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.243+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3712732475 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe80006070 con 0x562c223105c0 2023-10-05T19:40:24.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.243+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22312ee0 con 0x562c2230bc40 2023-10-05T19:40:24.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.243+0000 7fbe8bf7b700 1 -- 172.21.15.12:0/3712732475 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe84034020 con 0x562c2230bc40 2023-10-05T19:40:24.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.243+0000 7fbe8bf7b700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22312ee0 con 0x562c22307690 2023-10-05T19:40:24.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.247+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe7c014040 con 0x562c22307690 2023-10-05T19:40:24.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.247+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22312ee0 con 0x562c2224c3d0 2023-10-05T19:40:24.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.247+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe7c014040 con 0x562c2224c3d0 2023-10-05T19:40:24.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.247+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22312ee0 con 0x562c223105c0 2023-10-05T19:40:24.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.247+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3712732475 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe80006070 con 0x562c223105c0 2023-10-05T19:40:24.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.247+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22312ee0 con 0x562c2224c3d0 2023-10-05T19:40:24.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.247+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe7c014040 con 0x562c2224c3d0 2023-10-05T19:40:24.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.247+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22312ee0 con 0x562c2230bc40 2023-10-05T19:40:24.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.251+0000 7fbe8bf7b700 1 -- 172.21.15.12:0/3712732475 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe84034020 con 0x562c2230bc40 2023-10-05T19:40:24.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.251+0000 7fbe8bf7b700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22312ee0 con 0x562c223105c0 2023-10-05T19:40:24.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.251+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3712732475 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe80006070 con 0x562c223105c0 2023-10-05T19:40:24.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.251+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22312ee0 con 0x562c2224c3d0 2023-10-05T19:40:24.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.251+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe7c014040 con 0x562c2224c3d0 2023-10-05T19:40:24.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.251+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22312ee0 con 0x562c22307690 2023-10-05T19:40:24.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.251+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fbe7c014040 con 0x562c22307690 2023-10-05T19:40:24.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.251+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22312ee0 con 0x562c2230bc40 2023-10-05T19:40:24.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.255+0000 7fbe8bf7b700 1 -- 172.21.15.12:0/3712732475 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fbe84034020 con 0x562c2230bc40 2023-10-05T19:40:24.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.255+0000 7fbe8e446e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:24.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.255+0000 7fbe8e446e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:24.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.255+0000 7fbe8e446e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:24.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.255+0000 7fbe8e446e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:24.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.255+0000 7fbe8e446e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.255+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c2230f340 con 0x562c22307690 2023-10-05T19:40:24.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.255+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fbe7c014040 con 0x562c22307690 2023-10-05T19:40:24.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.255+0000 7fbe8e446e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:24.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.255+0000 7fbe8e446e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.255+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c2230fc90 con 0x562c22307690 2023-10-05T19:40:24.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.255+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fbe7c014040 con 0x562c22307690 2023-10-05T19:40:24.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.255+0000 7fbe8e446e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:24.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.255+0000 7fbe8e446e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.255+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c2230ef00 con 0x562c223105c0 2023-10-05T19:40:24.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.255+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3712732475 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fbe80006070 con 0x562c223105c0 2023-10-05T19:40:24.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.255+0000 7fbe8e446e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:24.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.255+0000 7fbe8e446e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.255+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22313810 con 0x562c223105c0 2023-10-05T19:40:24.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.255+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3712732475 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7fbe80006070 con 0x562c223105c0 2023-10-05T19:40:24.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.255+0000 7fbe8e446e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:24.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.255+0000 7fbe8e446e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.255+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22313bc0 con 0x562c22307690 2023-10-05T19:40:24.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.255+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fbe7c014040 con 0x562c22307690 2023-10-05T19:40:24.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.255+0000 7fbe8e446e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:24.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.255+0000 7fbe8e446e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.255+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22313f70 con 0x562c22307690 2023-10-05T19:40:24.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.259+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fbe7c014040 con 0x562c22307690 2023-10-05T19:40:24.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.259+0000 7fbe8e446e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:24.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.259+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22314f80 con 0x562c223105c0 2023-10-05T19:40:24.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.259+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3712732475 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fbe80006070 con 0x562c223105c0 2023-10-05T19:40:24.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.259+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22314f80 con 0x562c2230bc40 2023-10-05T19:40:24.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.263+0000 7fbe8bf7b700 1 -- 172.21.15.12:0/3712732475 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe84034020 con 0x562c2230bc40 2023-10-05T19:40:24.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.263+0000 7fbe8bf7b700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22314f80 con 0x562c2224c3d0 2023-10-05T19:40:24.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.263+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe7c014040 con 0x562c2224c3d0 2023-10-05T19:40:24.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.263+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22313bc0 con 0x562c22307690 2023-10-05T19:40:24.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.263+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe7c014040 con 0x562c22307690 2023-10-05T19:40:24.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.263+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22313bc0 con 0x562c22307690 2023-10-05T19:40:24.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.263+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe7c014040 con 0x562c22307690 2023-10-05T19:40:24.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.263+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22313bc0 con 0x562c2224c3d0 2023-10-05T19:40:24.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.267+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe7c014040 con 0x562c2224c3d0 2023-10-05T19:40:24.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.267+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22313bc0 con 0x562c2224c3d0 2023-10-05T19:40:24.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.267+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe7c014040 con 0x562c2224c3d0 2023-10-05T19:40:24.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.267+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22313bc0 con 0x562c2230bc40 2023-10-05T19:40:24.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.267+0000 7fbe8bf7b700 1 -- 172.21.15.12:0/3712732475 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe84034020 con 0x562c2230bc40 2023-10-05T19:40:24.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.267+0000 7fbe8bf7b700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22313bc0 con 0x562c223105c0 2023-10-05T19:40:24.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.267+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3712732475 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe80006070 con 0x562c223105c0 2023-10-05T19:40:24.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.267+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22313bc0 con 0x562c22307690 2023-10-05T19:40:24.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.267+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe7c014040 con 0x562c22307690 2023-10-05T19:40:24.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.267+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22313bc0 con 0x562c2224c3d0 2023-10-05T19:40:24.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.271+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe7c014040 con 0x562c2224c3d0 2023-10-05T19:40:24.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.271+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22313bc0 con 0x562c22307690 2023-10-05T19:40:24.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.271+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe7c014040 con 0x562c22307690 2023-10-05T19:40:24.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.271+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22313bc0 con 0x562c2230bc40 2023-10-05T19:40:24.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.271+0000 7fbe8bf7b700 1 -- 172.21.15.12:0/3712732475 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe84034020 con 0x562c2230bc40 2023-10-05T19:40:24.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.271+0000 7fbe8bf7b700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22313bc0 con 0x562c223105c0 2023-10-05T19:40:24.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.271+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3712732475 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe80006070 con 0x562c223105c0 2023-10-05T19:40:24.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.271+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22313bc0 con 0x562c223105c0 2023-10-05T19:40:24.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.271+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3712732475 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe80006070 con 0x562c223105c0 2023-10-05T19:40:24.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.271+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22313bc0 con 0x562c2224c3d0 2023-10-05T19:40:24.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.275+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe7c014040 con 0x562c2224c3d0 2023-10-05T19:40:24.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.275+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22313bc0 con 0x562c223105c0 2023-10-05T19:40:24.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.275+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3712732475 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe80006070 con 0x562c223105c0 2023-10-05T19:40:24.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.275+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22313bc0 con 0x562c223105c0 2023-10-05T19:40:24.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.275+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3712732475 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe80006070 con 0x562c223105c0 2023-10-05T19:40:24.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.275+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22313bc0 con 0x562c22307690 2023-10-05T19:40:24.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.275+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe7c014040 con 0x562c22307690 2023-10-05T19:40:24.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.275+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22313bc0 con 0x562c22307690 2023-10-05T19:40:24.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.275+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe7c014040 con 0x562c22307690 2023-10-05T19:40:24.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.279+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22313bc0 con 0x562c2224c3d0 2023-10-05T19:40:24.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.279+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe7c014040 con 0x562c2224c3d0 2023-10-05T19:40:24.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.279+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22313bc0 con 0x562c223105c0 2023-10-05T19:40:24.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.279+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3712732475 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe80006070 con 0x562c223105c0 2023-10-05T19:40:24.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.279+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22313bc0 con 0x562c2230bc40 2023-10-05T19:40:24.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.279+0000 7fbe8bf7b700 1 -- 172.21.15.12:0/3712732475 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe84034020 con 0x562c2230bc40 2023-10-05T19:40:24.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.279+0000 7fbe8bf7b700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22313bc0 con 0x562c22307690 2023-10-05T19:40:24.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.279+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe7c014040 con 0x562c22307690 2023-10-05T19:40:24.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.279+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22313bc0 con 0x562c2224c3d0 2023-10-05T19:40:24.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.283+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe7c014040 con 0x562c2224c3d0 2023-10-05T19:40:24.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.283+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22313bc0 con 0x562c223105c0 2023-10-05T19:40:24.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.283+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3712732475 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe80006070 con 0x562c223105c0 2023-10-05T19:40:24.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.283+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22313bc0 con 0x562c2224c3d0 2023-10-05T19:40:24.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.283+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe7c014040 con 0x562c2224c3d0 2023-10-05T19:40:24.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.283+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22313bc0 con 0x562c2230bc40 2023-10-05T19:40:24.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.283+0000 7fbe8bf7b700 1 -- 172.21.15.12:0/3712732475 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe84034020 con 0x562c2230bc40 2023-10-05T19:40:24.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.283+0000 7fbe8bf7b700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22313bc0 con 0x562c223105c0 2023-10-05T19:40:24.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.283+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3712732475 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe80006070 con 0x562c223105c0 2023-10-05T19:40:24.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.287+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22313bc0 con 0x562c2224c3d0 2023-10-05T19:40:24.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.287+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe7c014040 con 0x562c2224c3d0 2023-10-05T19:40:24.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.287+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22313bc0 con 0x562c22307690 2023-10-05T19:40:24.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.287+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fbe7c014040 con 0x562c22307690 2023-10-05T19:40:24.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.287+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22313bc0 con 0x562c2230bc40 2023-10-05T19:40:24.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.287+0000 7fbe8bf7b700 1 -- 172.21.15.12:0/3712732475 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fbe84034020 con 0x562c2230bc40 2023-10-05T19:40:24.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.287+0000 7fbe8e446e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:24.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.287+0000 7fbe8e446e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:24.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.287+0000 7fbe8e446e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:24.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.287+0000 7fbe8e446e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:24.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.287+0000 7fbe8e446e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.287+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22313f70 con 0x562c22307690 2023-10-05T19:40:24.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.287+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fbe7c014040 con 0x562c22307690 2023-10-05T19:40:24.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.287+0000 7fbe8e446e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:24.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.287+0000 7fbe8e446e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.287+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22314320 con 0x562c2230bc40 2023-10-05T19:40:24.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.291+0000 7fbe8bf7b700 1 -- 172.21.15.12:0/3712732475 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fbe84034020 con 0x562c2230bc40 2023-10-05T19:40:24.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.291+0000 7fbe8e446e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:24.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.291+0000 7fbe8e446e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.291+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22314bc0 con 0x562c22307690 2023-10-05T19:40:24.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.291+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fbe7c014040 con 0x562c22307690 2023-10-05T19:40:24.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.291+0000 7fbe8e446e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:24.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.291+0000 7fbe8e446e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.291+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22314bc0 con 0x562c223105c0 2023-10-05T19:40:24.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.291+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3712732475 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fbe80006070 con 0x562c223105c0 2023-10-05T19:40:24.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.291+0000 7fbe8e446e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:24.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.291+0000 7fbe8e446e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.291+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22314780 con 0x562c2224c3d0 2023-10-05T19:40:24.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.291+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7fbe7c014040 con 0x562c2224c3d0 2023-10-05T19:40:24.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.291+0000 7fbe8e446e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:24.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.291+0000 7fbe8e446e40 20 zone default found 2023-10-05T19:40:24.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.291+0000 7fbe8e446e40 4 Realm: () 2023-10-05T19:40:24.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.291+0000 7fbe8e446e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:24.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.291+0000 7fbe8e446e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:24.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.291+0000 7fbe8e446e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:24.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.291+0000 7fbe8e446e40 20 zonegroup default 2023-10-05T19:40:24.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.291+0000 7fbe8e446e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.291+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22314780 con 0x562c2224c3d0 2023-10-05T19:40:24.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.295+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fbe7c014040 con 0x562c2224c3d0 2023-10-05T19:40:24.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.295+0000 7fbe8e446e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:24.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.295+0000 7fbe8e446e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.295+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22314780 con 0x562c223105c0 2023-10-05T19:40:24.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.295+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3712732475 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fbe80006070 con 0x562c223105c0 2023-10-05T19:40:24.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.295+0000 7fbe8e446e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:24.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.295+0000 7fbe8e446e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.295+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22316de0 con 0x562c223105c0 2023-10-05T19:40:24.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.295+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3712732475 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fbe80006070 con 0x562c223105c0 2023-10-05T19:40:24.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.295+0000 7fbe8e446e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:24.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.295+0000 7fbe8e446e40 20 started sync module instance, tier type = 2023-10-05T19:40:24.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.295+0000 7fbe8e446e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:24.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.295+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c223164a0 con 0x562c2230bc40 2023-10-05T19:40:24.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.299+0000 7fbe8bf7b700 1 -- 172.21.15.12:0/3712732475 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'99 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe84034020 con 0x562c2230bc40 2023-10-05T19:40:24.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.299+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94747552213152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c223160d0 con 0x562c2230bc40 2023-10-05T19:40:24.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.299+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22317b70 con 0x562c22307690 2023-10-05T19:40:24.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.303+0000 7fbe8bf7b700 1 -- 172.21.15.12:0/3712732475 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94747552213152] v28'100 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe84034020 con 0x562c2230bc40 2023-10-05T19:40:24.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.303+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'99 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe7c014040 con 0x562c22307690 2023-10-05T19:40:24.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.303+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94747552215440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22317ad0 con 0x562c22307690 2023-10-05T19:40:24.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.303+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22318460 con 0x562c223105c0 2023-10-05T19:40:24.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.303+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94747552215440] v28'100 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe7c014040 con 0x562c22307690 2023-10-05T19:40:24.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.307+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3712732475 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'99 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe80006070 con 0x562c223105c0 2023-10-05T19:40:24.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.307+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94747552216544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22317a30 con 0x562c223105c0 2023-10-05T19:40:24.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.307+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22319240 con 0x562c2230bc40 2023-10-05T19:40:24.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.307+0000 7fbe8bf7b700 1 -- 172.21.15.12:0/3712732475 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'99 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe84034020 con 0x562c2230bc40 2023-10-05T19:40:24.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.307+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3712732475 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94747552216544] v28'100 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe80006070 con 0x562c223105c0 2023-10-05T19:40:24.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.307+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94747552222832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c223197b0 con 0x562c2230bc40 2023-10-05T19:40:24.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.307+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22319d90 con 0x562c22307690 2023-10-05T19:40:24.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.311+0000 7fbe8bf7b700 1 -- 172.21.15.12:0/3712732475 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94747552222832] v28'100 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe84034020 con 0x562c2230bc40 2023-10-05T19:40:24.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.311+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'99 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe7c014040 con 0x562c22307690 2023-10-05T19:40:24.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.311+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94747552223936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22318980 con 0x562c22307690 2023-10-05T19:40:24.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.311+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c2231a770 con 0x562c2224c3d0 2023-10-05T19:40:24.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.315+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'99 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe7c014040 con 0x562c2224c3d0 2023-10-05T19:40:24.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.315+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94747552227968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c2231abc0 con 0x562c2224c3d0 2023-10-05T19:40:24.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.315+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c2231b5d0 con 0x562c2224c3d0 2023-10-05T19:40:24.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.315+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94747552223936] v28'100 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe7c014040 con 0x562c22307690 2023-10-05T19:40:24.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.315+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(95 notify.5 [watch watch cookie 94747552227968] v28'100 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe7c014040 con 0x562c2224c3d0 2023-10-05T19:40:24.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.315+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(96 notify.6 [create] v28'99 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe7c014040 con 0x562c2224c3d0 2023-10-05T19:40:24.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.315+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94747552229072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c2231ab20 con 0x562c2224c3d0 2023-10-05T19:40:24.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.315+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c2231bac0 con 0x562c2224c3d0 2023-10-05T19:40:24.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.319+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(97 notify.6 [watch watch cookie 94747552229072] v28'100 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe7c014040 con 0x562c2224c3d0 2023-10-05T19:40:24.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.319+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(98 notify.7 [create] v28'99 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe7c014040 con 0x562c2224c3d0 2023-10-05T19:40:24.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.319+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94747552231648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c2231ba20 con 0x562c2224c3d0 2023-10-05T19:40:24.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.319+0000 7fbe8e446e40 20 add_watcher() i=0 2023-10-05T19:40:24.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.319+0000 7fbe8e446e40 20 add_watcher() i=1 2023-10-05T19:40:24.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.319+0000 7fbe8e446e40 20 add_watcher() i=2 2023-10-05T19:40:24.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.319+0000 7fbe8e446e40 20 add_watcher() i=3 2023-10-05T19:40:24.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.319+0000 7fbe8e446e40 20 add_watcher() i=4 2023-10-05T19:40:24.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.319+0000 7fbe8e446e40 20 add_watcher() i=5 2023-10-05T19:40:24.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.319+0000 7fbe8e446e40 20 add_watcher() i=6 2023-10-05T19:40:24.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.323+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94747552231648] v28'100 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe7c014040 con 0x562c2224c3d0 2023-10-05T19:40:24.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.323+0000 7fbe8e446e40 20 add_watcher() i=7 2023-10-05T19:40:24.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.323+0000 7fbe8e446e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:24.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.323+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c2231ebc0 con 0x562c22307690 2023-10-05T19:40:24.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.323+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fbe7c014040 con 0x562c22307690 2023-10-05T19:40:24.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.323+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94747552233120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c2231c080 con 0x562c22307690 2023-10-05T19:40:24.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.323+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94747552233120] v28'101 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fbe7c014040 con 0x562c22307690 2023-10-05T19:40:24.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.323+0000 7fbe5d7e2700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:24.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.323+0000 7fbe8e446e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:24.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.323+0000 7fbe8e446e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:24.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.323+0000 7fbe8e446e40 1 Processor -- start 2023-10-05T19:40:24.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.327+0000 7fbe8e446e40 1 -- start start 2023-10-05T19:40:24.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.327+0000 7fbe8e446e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562c22329140 0x562c22329510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:24.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.327+0000 7fbe8e446e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c22329a50 con 0x562c22329140 2023-10-05T19:40:24.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.327+0000 7fbe8b77a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562c22329140 0x562c22329510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.327+0000 7fbe8b77a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562c22329140 0x562c22329510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57724/0 (socket says 172.21.15.12:57724) 2023-10-05T19:40:24.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.327+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3497227058 learned_addr learned my addr 172.21.15.12:0/3497227058 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:24.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.327+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3497227058 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c22329b90 con 0x562c22329140 2023-10-05T19:40:24.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.327+0000 7fbe8b77a700 1 --2- 172.21.15.12:0/3497227058 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562c22329140 0x562c22329510 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x562c2230b3b0 tx=0x7fbe8001e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d07f55eb6de52f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.327+0000 7fbe5cfe1700 1 -- 172.21.15.12:0/3497227058 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe80013040 con 0x562c22329140 2023-10-05T19:40:24.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.327+0000 7fbe5cfe1700 1 -- 172.21.15.12:0/3497227058 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe80022040 con 0x562c22329140 2023-10-05T19:40:24.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.327+0000 7fbe5cfe1700 1 -- 172.21.15.12:0/3497227058 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe8000ea30 con 0x562c22329140 2023-10-05T19:40:24.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.327+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3497227058 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562c22329140 msgr2=0x562c22329510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.327+0000 7fbe8e446e40 1 --2- 172.21.15.12:0/3497227058 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562c22329140 0x562c22329510 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x562c2230b3b0 tx=0x7fbe8001e040 comp rx=0 tx=0).stop 2023-10-05T19:40:24.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.327+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3497227058 shutdown_connections 2023-10-05T19:40:24.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.327+0000 7fbe8e446e40 1 --2- 172.21.15.12:0/3497227058 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562c22329140 0x562c22329510 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:24.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.327+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3497227058 >> 172.21.15.12:0/3497227058 conn(0x562c22324500 msgr2=0x562c223226b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:24.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.327+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3497227058 shutdown_connections 2023-10-05T19:40:24.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.327+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3497227058 wait complete. 2023-10-05T19:40:24.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.331+0000 7fbe8e446e40 1 Processor -- start 2023-10-05T19:40:24.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.331+0000 7fbe8e446e40 1 -- start start 2023-10-05T19:40:24.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.331+0000 7fbe8e446e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562c22329140 0x562c2232ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:24.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.331+0000 7fbe8e446e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c2232fce0 con 0x562c22329140 2023-10-05T19:40:24.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.331+0000 7fbe8b77a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562c22329140 0x562c2232ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.331+0000 7fbe8b77a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562c22329140 0x562c2232ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57740/0 (socket says 172.21.15.12:57740) 2023-10-05T19:40:24.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.331+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3359474967 learned_addr learned my addr 172.21.15.12:0/3359474967 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:24.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.331+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3359474967 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe80034040 con 0x562c22329140 2023-10-05T19:40:24.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.331+0000 7fbe8b77a700 1 --2- 172.21.15.12:0/3359474967 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562c22329140 0x562c2232ece0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fbe80049040 tx=0x7fbe8003d030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.331+0000 7fbdd6ffd700 1 -- 172.21.15.12:0/3359474967 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe8003bea0 con 0x562c22329140 2023-10-05T19:40:24.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.331+0000 7fbdd6ffd700 1 -- 172.21.15.12:0/3359474967 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe8002c640 con 0x562c22329140 2023-10-05T19:40:24.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.331+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3359474967 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562c2232f2e0 con 0x562c22329140 2023-10-05T19:40:24.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.331+0000 7fbdd6ffd700 1 -- 172.21.15.12:0/3359474967 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe8002c7b0 con 0x562c22329140 2023-10-05T19:40:24.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.331+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3359474967 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562c2232f6c0 con 0x562c22329140 2023-10-05T19:40:24.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.335+0000 7fbdd6ffd700 1 -- 172.21.15.12:0/3359474967 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fbe8002c910 con 0x562c22329140 2023-10-05T19:40:24.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.335+0000 7fbdd6ffd700 1 --2- 172.21.15.12:0/3359474967 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fbdcc030c00 0x7fbdcc0330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:24.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.335+0000 7fbdd6ffd700 1 -- 172.21.15.12:0/3359474967 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7fbe80050ac0 con 0x562c22329140 2023-10-05T19:40:24.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.335+0000 7fbe8c77c700 1 --2- 172.21.15.12:0/3359474967 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fbdcc030c00 0x7fbdcc0330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.335+0000 7fbe8e446e40 5 note: GC not initialized 2023-10-05T19:40:24.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.335+0000 7fbe8c77c700 1 --2- 172.21.15.12:0/3359474967 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fbdcc030c00 0x7fbdcc0330b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x562c2231bfe0 tx=0x7fbe7c00ab80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.335+0000 7fbdcbfff700 20 reqs_thread_entry: start 2023-10-05T19:40:24.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.339+0000 7fbe8e446e40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:24.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.339+0000 7fbdc9ffb700 20 reqs_thread_entry: start 2023-10-05T19:40:24.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.339+0000 7fbe8e446e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:24.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.339+0000 7fbdb7fff700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:54 2023 2023-10-05T19:40:24.347 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:24.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.339+0000 7fbdb77fe700 20 reqs_thread_entry: start 2023-10-05T19:40:24.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.339+0000 7fbe8e446e40 20 get_system_obj_state: rctx=0x7ffd7f04d870 obj=default.rgw.meta:root:myfoo state=0x562c221b10b0 s->prefetch_data=0 2023-10-05T19:40:24.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.339+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:102 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22397cf0 con 0x562c2230bc40 2023-10-05T19:40:24.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.339+0000 7fbe8bf7b700 1 -- 172.21.15.12:0/3712732475 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(102 myfoo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 233+0+139 (crc 0 0 0) 0x7fbe84034020 con 0x562c2230bc40 2023-10-05T19:40:24.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.339+0000 7fbe8e446e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:24.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.339+0000 7fbe8e446e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.339+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:103 8.9 8:91b71841:root::myfoo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c2239a290 con 0x562c2230bc40 2023-10-05T19:40:24.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.339+0000 7fbe8bf7b700 1 -- 172.21.15.12:0/3712732475 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(103 myfoo [call,call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) v8 ==== 233+0+213 (crc 0 0 0) 0x7fbe84034020 con 0x562c2230bc40 2023-10-05T19:40:24.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.339+0000 7fbe8e446e40 20 rados_obj.operate() r=0 bl.length=165 2023-10-05T19:40:24.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.339+0000 7fbe8e446e40 20 get_system_obj_state: rctx=0x7ffd7f04d870 obj=default.rgw.meta:root:.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 state=0x562c221872b0 s->prefetch_data=0 2023-10-05T19:40:24.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.339+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:104 8.e 8:70176bbd:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22398760 con 0x562c2224c3d0 2023-10-05T19:40:24.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.339+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(104 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 [call out=48b,getxattrs out=218b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 290+0+282 (crc 0 0 0) 0x7fbe7c014040 con 0x562c2224c3d0 2023-10-05T19:40:24.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.339+0000 7fbe8e446e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:24.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.339+0000 7fbe8e446e40 20 Read xattr: user.rgw.acl 2023-10-05T19:40:24.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.339+0000 7fbe8e446e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:24.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.343+0000 7fbe8e446e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.343+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:105 8.e 8:70176bbd:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22398270 con 0x562c2224c3d0 2023-10-05T19:40:24.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.343+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(105 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 [call,call out=48b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 332+0+618 (crc 0 0 0) 0x7fbe7c014040 con 0x562c2224c3d0 2023-10-05T19:40:24.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.343+0000 7fbe8e446e40 20 rados_obj.operate() r=0 bl.length=352 2023-10-05T19:40:24.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.343+0000 7fbe8e446e40 20 couldn't put binfo cache entry, might have raced with data changes 2023-10-05T19:40:24.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.343+0000 7fbe8e446e40 20 get_system_obj_state: rctx=0x7ffd7f04d7d0 obj=default.rgw.meta:root:myfoo state=0x562c221872b0 s->prefetch_data=0 2023-10-05T19:40:24.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.343+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:106 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22398270 con 0x562c2230bc40 2023-10-05T19:40:24.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.343+0000 7fbe8bf7b700 1 -- 172.21.15.12:0/3712732475 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(106 myfoo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 233+0+139 (crc 0 0 0) 0x7fbe84034020 con 0x562c2230bc40 2023-10-05T19:40:24.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.343+0000 7fbe8e446e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:24.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.343+0000 7fbe8e446e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.343+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:107 8.9 8:91b71841:root::myfoo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c2239c830 con 0x562c2230bc40 2023-10-05T19:40:24.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.343+0000 7fbe8bf7b700 1 -- 172.21.15.12:0/3712732475 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(107 myfoo [call,call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) v8 ==== 233+0+213 (crc 0 0 0) 0x7fbe8405f040 con 0x562c2230bc40 2023-10-05T19:40:24.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.343+0000 7fbe8e446e40 20 rados_obj.operate() r=0 bl.length=165 2023-10-05T19:40:24.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.343+0000 7fbe8e446e40 20 get_system_obj_state: rctx=0x7ffd7f04d7d0 obj=default.rgw.meta:root:.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 state=0x562c221b10b0 s->prefetch_data=0 2023-10-05T19:40:24.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.343+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:108 8.e 8:70176bbd:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c2239a810 con 0x562c2224c3d0 2023-10-05T19:40:24.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.343+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(108 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 [call out=48b,getxattrs out=218b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 290+0+282 (crc 0 0 0) 0x7fbe7c014040 con 0x562c2224c3d0 2023-10-05T19:40:24.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.343+0000 7fbe8e446e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:24.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.343+0000 7fbe8e446e40 20 Read xattr: user.rgw.acl 2023-10-05T19:40:24.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.343+0000 7fbe8e446e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:24.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.343+0000 7fbe8e446e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.343+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:109 8.e 8:70176bbd:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22397140 con 0x562c2224c3d0 2023-10-05T19:40:24.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.347+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.0 v2:172.21.15.8:6800/27563 30 ==== osd_op_reply(109 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 [call,call out=48b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 332+0+618 (crc 0 0 0) 0x7fbe7c014040 con 0x562c2224c3d0 2023-10-05T19:40:24.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.347+0000 7fbe8e446e40 20 rados_obj.operate() r=0 bl.length=352 2023-10-05T19:40:24.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.347+0000 7fbe8e446e40 20 couldn't put binfo cache entry, might have raced with data changes 2023-10-05T19:40:24.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.347+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:110 4.3 4:c350083b:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22399e00 con 0x562c22307690 2023-10-05T19:40:24.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.347+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:111 4.1c 4:38d73088:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c2239f530 con 0x562c2224c3d0 2023-10-05T19:40:24.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.347+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:112 4.a 4:502f3015:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c223a1e40 con 0x562c2230bc40 2023-10-05T19:40:24.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.347+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:113 4.3e 4:7e570c43:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c223a4700 con 0x562c2230bc40 2023-10-05T19:40:24.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.347+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:114 4.7 4:e38d8107:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c223a6fc0 con 0x562c2224c3d0 2023-10-05T19:40:24.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.347+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:115 4.1a 4:5add8e75:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c223a9980 con 0x562c2230bc40 2023-10-05T19:40:24.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.347+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:116 4.2c 4:358e5801:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c223ac6d0 con 0x562c22307690 2023-10-05T19:40:24.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.347+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:117 4.2 4:4101cfe2:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c223aef90 con 0x562c22307690 2023-10-05T19:40:24.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.347+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:118 4.30 4:0e5788c0:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c223b1c30 con 0x562c223105c0 2023-10-05T19:40:24.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.347+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:119 4.19 4:99ccd5a2:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c223b4670 con 0x562c223105c0 2023-10-05T19:40:24.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.347+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:120 4.2d 4:b73f2ca3:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c223b7320 con 0x562c2224c3d0 2023-10-05T19:40:24.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.347+0000 7fbe8bf7b700 1 -- 172.21.15.12:0/3712732475 <== osd.2 v2:172.21.15.12:6800/29934 23 ==== osd_op_reply(113 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.3 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fbe8405f040 con 0x562c2230bc40 2023-10-05T19:40:24.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.347+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.0 v2:172.21.15.8:6800/27563 31 ==== osd_op_reply(111 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fbe7c014040 con 0x562c2224c3d0 2023-10-05T19:40:24.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.347+0000 7fbe8bf7b700 1 -- 172.21.15.12:0/3712732475 <== osd.2 v2:172.21.15.12:6800/29934 24 ==== osd_op_reply(112 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.2 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fbe8405f040 con 0x562c2230bc40 2023-10-05T19:40:24.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.347+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3712732475 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(118 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.8 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fbe80055070 con 0x562c223105c0 2023-10-05T19:40:24.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.347+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.0 v2:172.21.15.8:6800/27563 32 ==== osd_op_reply(114 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fbe7c014040 con 0x562c2224c3d0 2023-10-05T19:40:24.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.347+0000 7fbe8bf7b700 1 -- 172.21.15.12:0/3712732475 <== osd.2 v2:172.21.15.12:6800/29934 25 ==== osd_op_reply(115 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.5 [call out=124b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x7fbe8405f040 con 0x562c2230bc40 2023-10-05T19:40:24.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.347+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3712732475 <== osd.3 v2:172.21.15.12:6816/29951 27 ==== osd_op_reply(119 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.9 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fbe80055070 con 0x562c223105c0 2023-10-05T19:40:24.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.347+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(110 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fbe7c014040 con 0x562c22307690 2023-10-05T19:40:24.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.347+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(116 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.6 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fbe7c014040 con 0x562c22307690 2023-10-05T19:40:24.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.347+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(117 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.7 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fbe7c014040 con 0x562c22307690 2023-10-05T19:40:24.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.347+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.0 v2:172.21.15.8:6800/27563 33 ==== osd_op_reply(120 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7fbe7c014040 con 0x562c2224c3d0 2023-10-05T19:40:24.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.351+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:121 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94747552213152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c222ed7c0 con 0x562c2230bc40 2023-10-05T19:40:24.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.351+0000 7fbe8bf7b700 1 -- 172.21.15.12:0/3712732475 <== osd.2 v2:172.21.15.12:6800/29934 26 ==== osd_op_reply(121 notify.0 [watch unwatch cookie 94747552213152] v28'101 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe8405f040 con 0x562c2230bc40 2023-10-05T19:40:24.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.351+0000 7fbe8e446e40 20 remove_watcher() i=0 2023-10-05T19:40:24.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.351+0000 7fbe8e446e40 2 removed watcher, disabling cache 2023-10-05T19:40:24.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.351+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:122 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94747552215440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c223168a0 con 0x562c22307690 2023-10-05T19:40:24.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.355+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(122 notify.1 [watch unwatch cookie 94747552215440] v28'101 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe7c014040 con 0x562c22307690 2023-10-05T19:40:24.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.355+0000 7fbe8e446e40 20 remove_watcher() i=1 2023-10-05T19:40:24.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.355+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:123 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94747552216544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c223168a0 con 0x562c223105c0 2023-10-05T19:40:24.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.359+0000 7fbe8b77a700 1 -- 172.21.15.12:0/3712732475 <== osd.3 v2:172.21.15.12:6816/29951 28 ==== osd_op_reply(123 notify.2 [watch unwatch cookie 94747552216544] v28'101 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe80055070 con 0x562c223105c0 2023-10-05T19:40:24.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.359+0000 7fbe8e446e40 20 remove_watcher() i=2 2023-10-05T19:40:24.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.359+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:124 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94747552222832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c223168a0 con 0x562c2230bc40 2023-10-05T19:40:24.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.359+0000 7fbe8bf7b700 1 -- 172.21.15.12:0/3712732475 <== osd.2 v2:172.21.15.12:6800/29934 27 ==== osd_op_reply(124 notify.3 [watch unwatch cookie 94747552222832] v28'101 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe84066040 con 0x562c2230bc40 2023-10-05T19:40:24.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.359+0000 7fbe8e446e40 20 remove_watcher() i=3 2023-10-05T19:40:24.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.359+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:125 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94747552223936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c22318e70 con 0x562c22307690 2023-10-05T19:40:24.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.363+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(125 notify.4 [watch unwatch cookie 94747552223936] v28'101 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe7c014040 con 0x562c22307690 2023-10-05T19:40:24.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.363+0000 7fbe8e446e40 20 remove_watcher() i=4 2023-10-05T19:40:24.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.363+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:126 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94747552227968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c223168a0 con 0x562c2224c3d0 2023-10-05T19:40:24.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.367+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.0 v2:172.21.15.8:6800/27563 34 ==== osd_op_reply(126 notify.5 [watch unwatch cookie 94747552227968] v28'101 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe7c014040 con 0x562c2224c3d0 2023-10-05T19:40:24.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.367+0000 7fbe8e446e40 20 remove_watcher() i=5 2023-10-05T19:40:24.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.367+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:127 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94747552229072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c2231a280 con 0x562c2224c3d0 2023-10-05T19:40:24.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.371+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.0 v2:172.21.15.8:6800/27563 35 ==== osd_op_reply(127 notify.6 [watch unwatch cookie 94747552229072] v28'101 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe7c014040 con 0x562c2224c3d0 2023-10-05T19:40:24.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.371+0000 7fbe8e446e40 20 remove_watcher() i=6 2023-10-05T19:40:24.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.371+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:128 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94747552231648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c223168a0 con 0x562c2224c3d0 2023-10-05T19:40:24.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.371+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.0 v2:172.21.15.8:6800/27563 36 ==== osd_op_reply(128 notify.7 [watch unwatch cookie 94747552231648] v28'101 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe7c014040 con 0x562c2224c3d0 2023-10-05T19:40:24.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.371+0000 7fbe8e446e40 20 remove_watcher() i=7 2023-10-05T19:40:24.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.375+0000 7fbe5d7e2700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:24.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.375+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:129 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94747552233120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562c2231b0e0 con 0x562c22307690 2023-10-05T19:40:24.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.379+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== osd_op_reply(129 data_loggenerations_metadata [watch unwatch cookie 94747552233120] v28'102 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fbe7c014040 con 0x562c22307690 2023-10-05T19:40:24.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.379+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x562c2224c3d0 msgr2=0x7fbe84024400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.379+0000 7fbe8e446e40 1 --2- 172.21.15.12:0/3712732475 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x562c2224c3d0 0x7fbe84024400 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:24.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.379+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x562c22307690 msgr2=0x562c22309aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.379+0000 7fbe8e446e40 1 --2- 172.21.15.12:0/3712732475 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x562c22307690 0x562c22309aa0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:24.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.379+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x562c2230bc40 msgr2=0x562c2230e020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.379+0000 7fbe8e446e40 1 --2- 172.21.15.12:0/3712732475 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x562c2230bc40 0x562c2230e020 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:24.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.379+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x562c223105c0 msgr2=0x562c223129a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.379+0000 7fbe8e446e40 1 --2- 172.21.15.12:0/3712732475 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x562c223105c0 0x562c223129a0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:24.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.379+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fbe78030ca0 msgr2=0x7fbe78033150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.379+0000 7fbe8e446e40 1 --2- 172.21.15.12:0/3712732475 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fbe78030ca0 0x7fbe78033150 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbe80004170 tx=0x7fbe800012e0 comp rx=0 tx=0).stop 2023-10-05T19:40:24.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.379+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562c222db110 msgr2=0x562c222fdd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.379+0000 7fbe8e446e40 1 --2- 172.21.15.12:0/3712732475 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562c222db110 0x562c222fdd10 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbe84000f80 tx=0x7fbe8401d000 comp rx=0 tx=0).stop 2023-10-05T19:40:24.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.379+0000 7fbe8c77c700 1 -- 172.21.15.12:0/3712732475 reap_dead start 2023-10-05T19:40:24.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.379+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 shutdown_connections 2023-10-05T19:40:24.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.379+0000 7fbe8e446e40 1 --2- 172.21.15.12:0/3712732475 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562c222db110 0x562c222fdd10 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:24.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.379+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 >> 172.21.15.12:0/3712732475 conn(0x562c222d2720 msgr2=0x562c222f1520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:24.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.379+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 shutdown_connections 2023-10-05T19:40:24.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.379+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3712732475 wait complete. 2023-10-05T19:40:24.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.379+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3359474967 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fbdcc030c00 msgr2=0x7fbdcc0330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.379+0000 7fbe8e446e40 1 --2- 172.21.15.12:0/3359474967 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fbdcc030c00 0x7fbdcc0330b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x562c2231bfe0 tx=0x7fbe7c00ab80 comp rx=0 tx=0).stop 2023-10-05T19:40:24.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.379+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3359474967 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562c22329140 msgr2=0x562c2232ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.379+0000 7fbe8e446e40 1 --2- 172.21.15.12:0/3359474967 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562c22329140 0x562c2232ece0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fbe80049040 tx=0x7fbe8003d030 comp rx=0 tx=0).stop 2023-10-05T19:40:24.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.379+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3359474967 shutdown_connections 2023-10-05T19:40:24.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.379+0000 7fbe8e446e40 1 --2- 172.21.15.12:0/3359474967 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fbdcc030c00 0x7fbdcc0330b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:24.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.379+0000 7fbe8e446e40 1 --2- 172.21.15.12:0/3359474967 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562c22329140 0x562c2232ece0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:24.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.379+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3359474967 >> 172.21.15.12:0/3359474967 conn(0x562c22324500 msgr2=0x562c22322600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:24.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.379+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3359474967 shutdown_connections 2023-10-05T19:40:24.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.379+0000 7fbe8e446e40 1 -- 172.21.15.12:0/3359474967 wait complete. 2023-10-05T19:40:24.391 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:40:24.392 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo", 2023-10-05T19:40:24.392 INFO:teuthology.orchestra.run.smithi012.stdout: "num_shards": 11, 2023-10-05T19:40:24.392 INFO:teuthology.orchestra.run.smithi012.stdout: "tenant": "", 2023-10-05T19:40:24.392 INFO:teuthology.orchestra.run.smithi012.stdout: "zonegroup": "1602ab00-ae0f-4921-ad63-b9cfe7bc77db", 2023-10-05T19:40:24.392 INFO:teuthology.orchestra.run.smithi012.stdout: "placement_rule": "default-placement", 2023-10-05T19:40:24.392 INFO:teuthology.orchestra.run.smithi012.stdout: "explicit_placement": { 2023-10-05T19:40:24.392 INFO:teuthology.orchestra.run.smithi012.stdout: "data_pool": "", 2023-10-05T19:40:24.392 INFO:teuthology.orchestra.run.smithi012.stdout: "data_extra_pool": "", 2023-10-05T19:40:24.392 INFO:teuthology.orchestra.run.smithi012.stdout: "index_pool": "" 2023-10-05T19:40:24.392 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:24.392 INFO:teuthology.orchestra.run.smithi012.stdout: "id": "93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1", 2023-10-05T19:40:24.392 INFO:teuthology.orchestra.run.smithi012.stdout: "marker": "93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1", 2023-10-05T19:40:24.393 INFO:teuthology.orchestra.run.smithi012.stdout: "index_type": "Normal", 2023-10-05T19:40:24.393 INFO:teuthology.orchestra.run.smithi012.stdout: "owner": "foo", 2023-10-05T19:40:24.393 INFO:teuthology.orchestra.run.smithi012.stdout: "ver": "0#1,1#1,2#1,3#1,4#1,5#2,6#1,7#1,8#1,9#1,10#1", 2023-10-05T19:40:24.393 INFO:teuthology.orchestra.run.smithi012.stdout: "master_ver": "0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0", 2023-10-05T19:40:24.393 INFO:teuthology.orchestra.run.smithi012.stdout: "mtime": "2023-10-05T19:40:22.865997Z", 2023-10-05T19:40:24.393 INFO:teuthology.orchestra.run.smithi012.stdout: "creation_time": "2023-10-05T19:40:22.857926Z", 2023-10-05T19:40:24.393 INFO:teuthology.orchestra.run.smithi012.stdout: "max_marker": "0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#", 2023-10-05T19:40:24.393 INFO:teuthology.orchestra.run.smithi012.stdout: "usage": { 2023-10-05T19:40:24.393 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw.main": { 2023-10-05T19:40:24.393 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 3, 2023-10-05T19:40:24.393 INFO:teuthology.orchestra.run.smithi012.stdout: "size_actual": 4096, 2023-10-05T19:40:24.393 INFO:teuthology.orchestra.run.smithi012.stdout: "size_utilized": 3, 2023-10-05T19:40:24.394 INFO:teuthology.orchestra.run.smithi012.stdout: "size_kb": 1, 2023-10-05T19:40:24.394 INFO:teuthology.orchestra.run.smithi012.stdout: "size_kb_actual": 4, 2023-10-05T19:40:24.394 INFO:teuthology.orchestra.run.smithi012.stdout: "size_kb_utilized": 1, 2023-10-05T19:40:24.394 INFO:teuthology.orchestra.run.smithi012.stdout: "num_objects": 1 2023-10-05T19:40:24.394 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:24.394 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:24.394 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket_quota": { 2023-10-05T19:40:24.394 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:24.394 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:24.394 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:24.394 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:24.394 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:24.395 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:24.395 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:40:24.395 DEBUG:tasks.util.rgw: json result: {'bucket': 'myfoo', 'num_shards': 11, 'tenant': '', 'zonegroup': '1602ab00-ae0f-4921-ad63-b9cfe7bc77db', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1', 'marker': '93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1', 'index_type': 'Normal', 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#2,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2023-10-05T19:40:22.865997Z', 'creation_time': '2023-10-05T19:40:22.857926Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {'rgw.main': {'size': 3, 'size_actual': 4096, 'size_utilized': 3, 'size_kb': 1, 'size_kb_actual': 4, 'size_kb_utilized': 1, 'num_objects': 1}}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}} 2023-10-05T19:40:24.396 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'stats', '--uid', 'foo', '--sync-stats'] 2023-10-05T19:40:24.396 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'stats', '--uid', 'foo', '--sync-stats'] 2023-10-05T19:40:24.396 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user stats --uid foo --sync-stats 2023-10-05T19:40:24.481 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:24.481 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:24.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.475+0000 7f10a59b2e40 1 Processor -- start 2023-10-05T19:40:24.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.475+0000 7f10a59b2e40 1 -- start start 2023-10-05T19:40:24.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.475+0000 7f10a59b2e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x5636c7dab020 0x5636c7dab3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:24.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.475+0000 7f10a59b2e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5636c7cf5bd0 con 0x5636c7dab930 2023-10-05T19:40:24.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.475+0000 7f10a59b2e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x5636c7cb2ee0 con 0x5636c7dab020 2023-10-05T19:40:24.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.475+0000 7f10a2ce6700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x5636c7dab930 0x5636c7dac3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:35496/0 (socket says 172.21.15.12:35496) 2023-10-05T19:40:24.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.475+0000 7f10a2ce6700 1 -- 172.21.15.12:0/3745142113 learned_addr learned my addr 172.21.15.12:0/3745142113 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:24.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.475+0000 7f10a34e7700 1 --2- 172.21.15.12:0/3745142113 >> v2:172.21.15.8:3300/0 conn(0x5636c7dab020 0x5636c7dab3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.475+0000 7f10a24e5700 1 -- 172.21.15.12:0/3745142113 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1011533512 0 0) 0x5636c7cf5bd0 con 0x5636c7dab930 2023-10-05T19:40:24.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.475+0000 7f10a24e5700 1 -- 172.21.15.12:0/3745142113 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1090001e10 con 0x5636c7dab930 2023-10-05T19:40:24.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.475+0000 7f10a34e7700 1 -- 172.21.15.12:0/3745142113 >> v1:172.21.15.8:6789/0 conn(0x5636c7dab930 legacy=0x5636c7dac3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.475+0000 7f10a34e7700 1 -- 172.21.15.12:0/3745142113 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636c7dacae0 con 0x5636c7dab020 2023-10-05T19:40:24.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.475+0000 7f10a34e7700 1 --2- 172.21.15.12:0/3745142113 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636c7dab020 0x5636c7dab3f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f109400a5d0 tx=0x7f1094005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=123b9649c32475f server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.475+0000 7f10a24e5700 1 -- 172.21.15.12:0/3745142113 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1094007610 con 0x5636c7dab020 2023-10-05T19:40:24.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.475+0000 7f10a24e5700 1 -- 172.21.15.12:0/3745142113 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1094007770 con 0x5636c7dab020 2023-10-05T19:40:24.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.475+0000 7f10a24e5700 1 -- 172.21.15.12:0/3745142113 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1094007a40 con 0x5636c7dab020 2023-10-05T19:40:24.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.475+0000 7f10a59b2e40 1 -- 172.21.15.12:0/3745142113 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636c7dab020 msgr2=0x5636c7dab3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.475+0000 7f10a59b2e40 1 --2- 172.21.15.12:0/3745142113 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636c7dab020 0x5636c7dab3f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f109400a5d0 tx=0x7f1094005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:24.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.475+0000 7f10a59b2e40 1 -- 172.21.15.12:0/3745142113 shutdown_connections 2023-10-05T19:40:24.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.475+0000 7f10a59b2e40 1 --2- 172.21.15.12:0/3745142113 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636c7dab020 0x5636c7dab3f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:24.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.475+0000 7f10a59b2e40 1 -- 172.21.15.12:0/3745142113 >> 172.21.15.12:0/3745142113 conn(0x5636c7d1e870 msgr2=0x5636c7d1ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:24.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.475+0000 7f10a59b2e40 1 -- 172.21.15.12:0/3745142113 shutdown_connections 2023-10-05T19:40:24.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.475+0000 7f10a59b2e40 1 -- 172.21.15.12:0/3745142113 wait complete. 2023-10-05T19:40:24.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.479+0000 7f10a59b2e40 1 Processor -- start 2023-10-05T19:40:24.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.479+0000 7f10a59b2e40 1 -- start start 2023-10-05T19:40:24.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.479+0000 7f10a59b2e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636c7dab020 0x5636c7dc9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:24.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.479+0000 7f10a59b2e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636c7c76e30 con 0x5636c7dab020 2023-10-05T19:40:24.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.479+0000 7f10a34e7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636c7dab020 0x5636c7dc9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.479+0000 7f10a34e7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636c7dab020 0x5636c7dc9f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57746/0 (socket says 172.21.15.12:57746) 2023-10-05T19:40:24.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.479+0000 7f10a34e7700 1 -- 172.21.15.12:0/2721882971 learned_addr learned my addr 172.21.15.12:0/2721882971 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:24.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.479+0000 7f10a34e7700 1 -- 172.21.15.12:0/2721882971 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1094005cf0 con 0x5636c7dab020 2023-10-05T19:40:24.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.479+0000 7f10a34e7700 1 --2- 172.21.15.12:0/2721882971 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636c7dab020 0x5636c7dc9f50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1094000c00 tx=0x7f1094014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=291589c4e5ba4f63 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.479+0000 7f10a14e3700 1 -- 172.21.15.12:0/2721882971 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f10940078e0 con 0x5636c7dab020 2023-10-05T19:40:24.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.479+0000 7f10a14e3700 1 -- 172.21.15.12:0/2721882971 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1094007a80 con 0x5636c7dab020 2023-10-05T19:40:24.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.479+0000 7f10a14e3700 1 -- 172.21.15.12:0/2721882971 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1094007d70 con 0x5636c7dab020 2023-10-05T19:40:24.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.479+0000 7f10a59b2e40 1 -- 172.21.15.12:0/2721882971 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636c7dab020 msgr2=0x5636c7dc9f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.479+0000 7f10a59b2e40 1 --2- 172.21.15.12:0/2721882971 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636c7dab020 0x5636c7dc9f50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1094000c00 tx=0x7f1094014000 comp rx=0 tx=0).stop 2023-10-05T19:40:24.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.479+0000 7f10a59b2e40 1 -- 172.21.15.12:0/2721882971 shutdown_connections 2023-10-05T19:40:24.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.479+0000 7f10a59b2e40 1 --2- 172.21.15.12:0/2721882971 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636c7dab020 0x5636c7dc9f50 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:24.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.479+0000 7f10a59b2e40 1 -- 172.21.15.12:0/2721882971 >> 172.21.15.12:0/2721882971 conn(0x5636c7d1e870 msgr2=0x5636c7dc5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:24.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.479+0000 7f10a59b2e40 1 -- 172.21.15.12:0/2721882971 shutdown_connections 2023-10-05T19:40:24.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.479+0000 7f10a59b2e40 1 -- 172.21.15.12:0/2721882971 wait complete. 2023-10-05T19:40:24.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.479+0000 7f10a59b2e40 1 Processor -- start 2023-10-05T19:40:24.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.483+0000 7f10a59b2e40 1 -- start start 2023-10-05T19:40:24.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.483+0000 7f10a59b2e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636c7dab020 0x5636c7dd0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:24.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.483+0000 7f10a59b2e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636c7c49ed0 con 0x5636c7dab020 2023-10-05T19:40:24.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.483+0000 7f10a34e7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636c7dab020 0x5636c7dd0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.483+0000 7f10a34e7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636c7dab020 0x5636c7dd0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57760/0 (socket says 172.21.15.12:57760) 2023-10-05T19:40:24.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.483+0000 7f10a34e7700 1 -- 172.21.15.12:0/4274334332 learned_addr learned my addr 172.21.15.12:0/4274334332 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:24.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.483+0000 7f10a34e7700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636c7dacae0 con 0x5636c7dab020 2023-10-05T19:40:24.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.483+0000 7f10a34e7700 1 --2- 172.21.15.12:0/4274334332 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636c7dab020 0x5636c7dd0bb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f109400afd0 tx=0x7f1094003050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.483+0000 7f109b7fe700 1 -- 172.21.15.12:0/4274334332 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f109401d980 con 0x5636c7dab020 2023-10-05T19:40:24.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.483+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5636c7dd10f0 con 0x5636c7dab020 2023-10-05T19:40:24.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.483+0000 7f109b7fe700 1 -- 172.21.15.12:0/4274334332 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109402b070 con 0x5636c7dab020 2023-10-05T19:40:24.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.483+0000 7f109b7fe700 1 -- 172.21.15.12:0/4274334332 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f109401dc80 con 0x5636c7dab020 2023-10-05T19:40:24.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.483+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5636c7dd1530 con 0x5636c7dab020 2023-10-05T19:40:24.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.487+0000 7f109b7fe700 1 -- 172.21.15.12:0/4274334332 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f109402d030 con 0x5636c7dab020 2023-10-05T19:40:24.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.487+0000 7f10a59b2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.487+0000 7f109b7fe700 1 --2- 172.21.15.12:0/4274334332 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1084030ca0 0x7f1084033150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:24.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.487+0000 7f109b7fe700 1 -- 172.21.15.12:0/4274334332 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f1094046510 con 0x5636c7dab020 2023-10-05T19:40:24.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.487+0000 7f10a2ce6700 1 --2- 172.21.15.12:0/4274334332 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1084030ca0 0x7f1084033150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.487+0000 7f10a59b2e40 1 --2- 172.21.15.12:0/4274334332 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5636c7dab930 0x5636c7ddc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:24.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.487+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7ddcc80 con 0x5636c7dab930 2023-10-05T19:40:24.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.487+0000 7f10a3ce8700 1 --2- 172.21.15.12:0/4274334332 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5636c7dab930 0x5636c7ddc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.487+0000 7f10a2ce6700 1 --2- 172.21.15.12:0/4274334332 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1084030ca0 0x7f1084033150 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f108c004170 tx=0x7f108c001400 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.487+0000 7f10a3ce8700 1 --2- 172.21.15.12:0/4274334332 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5636c7dab930 0x5636c7ddc610 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.487+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f109c00a5a0 con 0x5636c7dab930 2023-10-05T19:40:24.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.487+0000 7f10a59b2e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:24.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.487+0000 7f10a59b2e40 20 realm 2023-10-05T19:40:24.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.487+0000 7f10a59b2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.487+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7dd9c70 con 0x5636c7dab930 2023-10-05T19:40:24.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.491+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f109c00a5a0 con 0x5636c7dab930 2023-10-05T19:40:24.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.491+0000 7f10a59b2e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:24.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.491+0000 7f10a59b2e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:24.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.491+0000 7f10a59b2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.491+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7ddd760 con 0x5636c7dab930 2023-10-05T19:40:24.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.491+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f109c00a5a0 con 0x5636c7dab930 2023-10-05T19:40:24.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.491+0000 7f10a59b2e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:24.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.491+0000 7f10a59b2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.491+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7ddd340 con 0x5636c7dab930 2023-10-05T19:40:24.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.491+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f109c00a5a0 con 0x5636c7dab930 2023-10-05T19:40:24.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.491+0000 7f10a59b2e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:24.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.491+0000 7f10a59b2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.491+0000 7f10a59b2e40 1 --2- 172.21.15.12:0/4274334332 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5636c7dde6f0 0x5636c7de0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:24.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.491+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de1240 con 0x5636c7dde6f0 2023-10-05T19:40:24.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.491+0000 7f10a34e7700 1 --2- 172.21.15.12:0/4274334332 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5636c7dde6f0 0x5636c7de0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.491+0000 7f10a34e7700 1 --2- 172.21.15.12:0/4274334332 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5636c7dde6f0 0x5636c7de0ad0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.491+0000 7f10a34e7700 1 -- 172.21.15.12:0/4274334332 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f1094041380 con 0x5636c7dde6f0 2023-10-05T19:40:24.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.491+0000 7f10a59b2e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:24.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.491+0000 7f10a59b2e40 20 searching for the correct realm 2023-10-05T19:40:24.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.491+0000 7f10a59b2e40 1 --2- 172.21.15.12:0/4274334332 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5636c7de3070 0x5636c7de5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:24.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.491+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de5990 con 0x5636c7de3070 2023-10-05T19:40:24.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.491+0000 7f10a2ce6700 1 --2- 172.21.15.12:0/4274334332 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5636c7de3070 0x5636c7de5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.495+0000 7f10a2ce6700 1 --2- 172.21.15.12:0/4274334332 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5636c7de3070 0x5636c7de5450 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.495+0000 7f10a2ce6700 1 -- 172.21.15.12:0/4274334332 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f108c006070 con 0x5636c7de3070 2023-10-05T19:40:24.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.495+0000 7f10a2ce6700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de5990 con 0x5636c7dde6f0 2023-10-05T19:40:24.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.495+0000 7f10a34e7700 1 -- 172.21.15.12:0/4274334332 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1094041380 con 0x5636c7dde6f0 2023-10-05T19:40:24.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.495+0000 7f10a34e7700 1 --2- 172.21.15.12:0/4274334332 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f1094039070 0x7f109400c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:24.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.495+0000 7f10a34e7700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de5990 con 0x7f1094039070 2023-10-05T19:40:24.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.495+0000 7f10a3ce8700 1 --2- 172.21.15.12:0/4274334332 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f1094039070 0x7f109400c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.499+0000 7f10a3ce8700 1 --2- 172.21.15.12:0/4274334332 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f1094039070 0x7f109400c9c0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.499+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f109c00a5a0 con 0x7f1094039070 2023-10-05T19:40:24.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.499+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de5990 con 0x5636c7dab930 2023-10-05T19:40:24.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.499+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f109c00a5a0 con 0x5636c7dab930 2023-10-05T19:40:24.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.499+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de5990 con 0x5636c7dab930 2023-10-05T19:40:24.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.499+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f109c00a5a0 con 0x5636c7dab930 2023-10-05T19:40:24.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.499+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de5990 con 0x7f1094039070 2023-10-05T19:40:24.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.499+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f109c00a5a0 con 0x7f1094039070 2023-10-05T19:40:24.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.499+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de5990 con 0x7f1094039070 2023-10-05T19:40:24.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.503+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f109c00a5a0 con 0x7f1094039070 2023-10-05T19:40:24.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.503+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de5990 con 0x5636c7dde6f0 2023-10-05T19:40:24.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.503+0000 7f10a34e7700 1 -- 172.21.15.12:0/4274334332 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1094041380 con 0x5636c7dde6f0 2023-10-05T19:40:24.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.503+0000 7f10a34e7700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de5990 con 0x5636c7de3070 2023-10-05T19:40:24.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.503+0000 7f10a2ce6700 1 -- 172.21.15.12:0/4274334332 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f108c006070 con 0x5636c7de3070 2023-10-05T19:40:24.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.503+0000 7f10a2ce6700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de5990 con 0x5636c7dab930 2023-10-05T19:40:24.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.503+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f109c00a5a0 con 0x5636c7dab930 2023-10-05T19:40:24.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.503+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de5990 con 0x7f1094039070 2023-10-05T19:40:24.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.507+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f109c00a5a0 con 0x7f1094039070 2023-10-05T19:40:24.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.507+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de5990 con 0x5636c7dab930 2023-10-05T19:40:24.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.507+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f109c00a5a0 con 0x5636c7dab930 2023-10-05T19:40:24.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.507+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de5990 con 0x5636c7dde6f0 2023-10-05T19:40:24.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.507+0000 7f10a34e7700 1 -- 172.21.15.12:0/4274334332 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1094041380 con 0x5636c7dde6f0 2023-10-05T19:40:24.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.507+0000 7f10a34e7700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de5990 con 0x5636c7de3070 2023-10-05T19:40:24.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.507+0000 7f10a2ce6700 1 -- 172.21.15.12:0/4274334332 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f108c006070 con 0x5636c7de3070 2023-10-05T19:40:24.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.507+0000 7f10a2ce6700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de5990 con 0x5636c7de3070 2023-10-05T19:40:24.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.507+0000 7f10a2ce6700 1 -- 172.21.15.12:0/4274334332 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f108c006070 con 0x5636c7de3070 2023-10-05T19:40:24.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.507+0000 7f10a2ce6700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de5990 con 0x7f1094039070 2023-10-05T19:40:24.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.511+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f109c00a5a0 con 0x7f1094039070 2023-10-05T19:40:24.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.511+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de5990 con 0x5636c7de3070 2023-10-05T19:40:24.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.511+0000 7f10a2ce6700 1 -- 172.21.15.12:0/4274334332 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f108c006070 con 0x5636c7de3070 2023-10-05T19:40:24.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.511+0000 7f10a2ce6700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de5990 con 0x5636c7de3070 2023-10-05T19:40:24.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.511+0000 7f10a2ce6700 1 -- 172.21.15.12:0/4274334332 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f108c006070 con 0x5636c7de3070 2023-10-05T19:40:24.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.511+0000 7f10a2ce6700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de5990 con 0x5636c7dab930 2023-10-05T19:40:24.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.511+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f109c00a5a0 con 0x5636c7dab930 2023-10-05T19:40:24.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.511+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de5990 con 0x5636c7dab930 2023-10-05T19:40:24.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.511+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f109c00a5a0 con 0x5636c7dab930 2023-10-05T19:40:24.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.511+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de5990 con 0x7f1094039070 2023-10-05T19:40:24.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.515+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f109c00a5a0 con 0x7f1094039070 2023-10-05T19:40:24.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.515+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de5990 con 0x5636c7de3070 2023-10-05T19:40:24.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.515+0000 7f10a2ce6700 1 -- 172.21.15.12:0/4274334332 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f108c006070 con 0x5636c7de3070 2023-10-05T19:40:24.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.515+0000 7f10a2ce6700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de5990 con 0x5636c7dde6f0 2023-10-05T19:40:24.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.515+0000 7f10a34e7700 1 -- 172.21.15.12:0/4274334332 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1094041380 con 0x5636c7dde6f0 2023-10-05T19:40:24.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.515+0000 7f10a34e7700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de5990 con 0x5636c7dab930 2023-10-05T19:40:24.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.515+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f109c00a5a0 con 0x5636c7dab930 2023-10-05T19:40:24.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.515+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de5990 con 0x7f1094039070 2023-10-05T19:40:24.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.519+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f109c00a5a0 con 0x7f1094039070 2023-10-05T19:40:24.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.519+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de5990 con 0x5636c7de3070 2023-10-05T19:40:24.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.519+0000 7f10a2ce6700 1 -- 172.21.15.12:0/4274334332 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f108c006070 con 0x5636c7de3070 2023-10-05T19:40:24.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.519+0000 7f10a2ce6700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de5990 con 0x7f1094039070 2023-10-05T19:40:24.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.519+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f109c00a5a0 con 0x7f1094039070 2023-10-05T19:40:24.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.519+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de5990 con 0x5636c7dde6f0 2023-10-05T19:40:24.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.519+0000 7f10a34e7700 1 -- 172.21.15.12:0/4274334332 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1094041380 con 0x5636c7dde6f0 2023-10-05T19:40:24.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.519+0000 7f10a34e7700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de5990 con 0x5636c7de3070 2023-10-05T19:40:24.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.523+0000 7f10a2ce6700 1 -- 172.21.15.12:0/4274334332 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f108c006070 con 0x5636c7de3070 2023-10-05T19:40:24.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.523+0000 7f10a2ce6700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de5990 con 0x7f1094039070 2023-10-05T19:40:24.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.523+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f109c00a5a0 con 0x7f1094039070 2023-10-05T19:40:24.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.523+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de5990 con 0x5636c7dab930 2023-10-05T19:40:24.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.523+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f109c00a5a0 con 0x5636c7dab930 2023-10-05T19:40:24.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.523+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de5990 con 0x5636c7dde6f0 2023-10-05T19:40:24.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.523+0000 7f10a34e7700 1 -- 172.21.15.12:0/4274334332 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f1094041380 con 0x5636c7dde6f0 2023-10-05T19:40:24.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.523+0000 7f10a59b2e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:24.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.523+0000 7f10a59b2e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:24.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.523+0000 7f10a59b2e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:24.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.523+0000 7f10a59b2e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:24.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.523+0000 7f10a59b2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.523+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de1e20 con 0x5636c7dab930 2023-10-05T19:40:24.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.527+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f109c00a5a0 con 0x5636c7dab930 2023-10-05T19:40:24.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.527+0000 7f10a59b2e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:24.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.527+0000 7f10a59b2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.527+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de2770 con 0x5636c7dab930 2023-10-05T19:40:24.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.527+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f109c00a5a0 con 0x5636c7dab930 2023-10-05T19:40:24.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.527+0000 7f10a59b2e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:24.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.527+0000 7f10a59b2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.527+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de19e0 con 0x5636c7de3070 2023-10-05T19:40:24.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.527+0000 7f10a2ce6700 1 -- 172.21.15.12:0/4274334332 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f108c006070 con 0x5636c7de3070 2023-10-05T19:40:24.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.527+0000 7f10a59b2e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:24.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.527+0000 7f10a59b2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.527+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de62c0 con 0x5636c7de3070 2023-10-05T19:40:24.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.527+0000 7f10a2ce6700 1 -- 172.21.15.12:0/4274334332 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f108c006070 con 0x5636c7de3070 2023-10-05T19:40:24.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.527+0000 7f10a59b2e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:24.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.527+0000 7f10a59b2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.527+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de6670 con 0x5636c7dab930 2023-10-05T19:40:24.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.527+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f109c00a5a0 con 0x5636c7dab930 2023-10-05T19:40:24.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.527+0000 7f10a59b2e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:24.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.527+0000 7f10a59b2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.527+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de6a20 con 0x5636c7dab930 2023-10-05T19:40:24.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.531+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f109c00a5a0 con 0x5636c7dab930 2023-10-05T19:40:24.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.531+0000 7f10a59b2e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:24.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.531+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de7a30 con 0x5636c7de3070 2023-10-05T19:40:24.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.531+0000 7f10a2ce6700 1 -- 172.21.15.12:0/4274334332 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f108c006070 con 0x5636c7de3070 2023-10-05T19:40:24.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.531+0000 7f10a2ce6700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de7a30 con 0x5636c7dde6f0 2023-10-05T19:40:24.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.535+0000 7f10a34e7700 1 -- 172.21.15.12:0/4274334332 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1094041380 con 0x5636c7dde6f0 2023-10-05T19:40:24.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.535+0000 7f10a34e7700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de7a30 con 0x7f1094039070 2023-10-05T19:40:24.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.535+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f109c00a5a0 con 0x7f1094039070 2023-10-05T19:40:24.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.535+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de6670 con 0x5636c7dab930 2023-10-05T19:40:24.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.535+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f109c00a5a0 con 0x5636c7dab930 2023-10-05T19:40:24.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.535+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de6670 con 0x5636c7dab930 2023-10-05T19:40:24.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.535+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f109c00a5a0 con 0x5636c7dab930 2023-10-05T19:40:24.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.535+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de6670 con 0x7f1094039070 2023-10-05T19:40:24.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.539+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f109c00a5a0 con 0x7f1094039070 2023-10-05T19:40:24.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.539+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de6670 con 0x7f1094039070 2023-10-05T19:40:24.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.539+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f109c00a5a0 con 0x7f1094039070 2023-10-05T19:40:24.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.539+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de6670 con 0x5636c7dde6f0 2023-10-05T19:40:24.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.539+0000 7f10a34e7700 1 -- 172.21.15.12:0/4274334332 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1094041380 con 0x5636c7dde6f0 2023-10-05T19:40:24.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.539+0000 7f10a34e7700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de6670 con 0x5636c7de3070 2023-10-05T19:40:24.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.539+0000 7f10a2ce6700 1 -- 172.21.15.12:0/4274334332 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f108c006070 con 0x5636c7de3070 2023-10-05T19:40:24.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.539+0000 7f10a2ce6700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de6670 con 0x5636c7dab930 2023-10-05T19:40:24.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.543+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f109c00a5a0 con 0x5636c7dab930 2023-10-05T19:40:24.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.543+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de6670 con 0x7f1094039070 2023-10-05T19:40:24.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.543+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f109c00a5a0 con 0x7f1094039070 2023-10-05T19:40:24.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.543+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de6670 con 0x5636c7dab930 2023-10-05T19:40:24.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.543+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f109c00a5a0 con 0x5636c7dab930 2023-10-05T19:40:24.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.543+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de6670 con 0x5636c7dde6f0 2023-10-05T19:40:24.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.543+0000 7f10a34e7700 1 -- 172.21.15.12:0/4274334332 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1094041380 con 0x5636c7dde6f0 2023-10-05T19:40:24.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.543+0000 7f10a34e7700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de6670 con 0x5636c7de3070 2023-10-05T19:40:24.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.543+0000 7f10a2ce6700 1 -- 172.21.15.12:0/4274334332 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f108c006070 con 0x5636c7de3070 2023-10-05T19:40:24.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.543+0000 7f10a2ce6700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de6670 con 0x5636c7de3070 2023-10-05T19:40:24.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.547+0000 7f10a2ce6700 1 -- 172.21.15.12:0/4274334332 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f108c006070 con 0x5636c7de3070 2023-10-05T19:40:24.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.547+0000 7f10a2ce6700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de6670 con 0x7f1094039070 2023-10-05T19:40:24.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.547+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f109c00a5a0 con 0x7f1094039070 2023-10-05T19:40:24.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.547+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de6670 con 0x5636c7de3070 2023-10-05T19:40:24.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.547+0000 7f10a2ce6700 1 -- 172.21.15.12:0/4274334332 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f108c006070 con 0x5636c7de3070 2023-10-05T19:40:24.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.547+0000 7f10a2ce6700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de6670 con 0x5636c7de3070 2023-10-05T19:40:24.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.547+0000 7f10a2ce6700 1 -- 172.21.15.12:0/4274334332 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f108c006070 con 0x5636c7de3070 2023-10-05T19:40:24.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.547+0000 7f10a2ce6700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de6670 con 0x5636c7dab930 2023-10-05T19:40:24.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.547+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f109c00a5a0 con 0x5636c7dab930 2023-10-05T19:40:24.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.547+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de6670 con 0x5636c7dab930 2023-10-05T19:40:24.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.551+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f109c00a5a0 con 0x5636c7dab930 2023-10-05T19:40:24.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.551+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de6670 con 0x7f1094039070 2023-10-05T19:40:24.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.551+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f109c00a5a0 con 0x7f1094039070 2023-10-05T19:40:24.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.551+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de6670 con 0x5636c7de3070 2023-10-05T19:40:24.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.551+0000 7f10a2ce6700 1 -- 172.21.15.12:0/4274334332 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f108c006070 con 0x5636c7de3070 2023-10-05T19:40:24.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.551+0000 7f10a2ce6700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de6670 con 0x5636c7dde6f0 2023-10-05T19:40:24.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.551+0000 7f10a34e7700 1 -- 172.21.15.12:0/4274334332 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1094041380 con 0x5636c7dde6f0 2023-10-05T19:40:24.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.551+0000 7f10a34e7700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de6670 con 0x5636c7dab930 2023-10-05T19:40:24.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.555+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f109c00a5a0 con 0x5636c7dab930 2023-10-05T19:40:24.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.555+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de6670 con 0x7f1094039070 2023-10-05T19:40:24.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.555+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f109c00a5a0 con 0x7f1094039070 2023-10-05T19:40:24.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.555+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de6670 con 0x5636c7de3070 2023-10-05T19:40:24.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.555+0000 7f10a2ce6700 1 -- 172.21.15.12:0/4274334332 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f108c006070 con 0x5636c7de3070 2023-10-05T19:40:24.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.555+0000 7f10a2ce6700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de6670 con 0x7f1094039070 2023-10-05T19:40:24.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.555+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f109c00a5a0 con 0x7f1094039070 2023-10-05T19:40:24.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.555+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de6670 con 0x5636c7dde6f0 2023-10-05T19:40:24.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.555+0000 7f10a34e7700 1 -- 172.21.15.12:0/4274334332 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1094041380 con 0x5636c7dde6f0 2023-10-05T19:40:24.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.555+0000 7f10a34e7700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de6670 con 0x5636c7de3070 2023-10-05T19:40:24.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.559+0000 7f10a2ce6700 1 -- 172.21.15.12:0/4274334332 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f108c006070 con 0x5636c7de3070 2023-10-05T19:40:24.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.559+0000 7f10a2ce6700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de6670 con 0x7f1094039070 2023-10-05T19:40:24.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.559+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f109c00a5a0 con 0x7f1094039070 2023-10-05T19:40:24.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.559+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de6670 con 0x5636c7dab930 2023-10-05T19:40:24.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.559+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f109c00a5a0 con 0x5636c7dab930 2023-10-05T19:40:24.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.559+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de6670 con 0x5636c7dde6f0 2023-10-05T19:40:24.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.559+0000 7f10a34e7700 1 -- 172.21.15.12:0/4274334332 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f1094041380 con 0x5636c7dde6f0 2023-10-05T19:40:24.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.559+0000 7f10a59b2e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:24.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.559+0000 7f10a59b2e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:24.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.559+0000 7f10a59b2e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:24.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.559+0000 7f10a59b2e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:24.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.559+0000 7f10a59b2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.559+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de6a20 con 0x5636c7dab930 2023-10-05T19:40:24.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.563+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f109c00a5a0 con 0x5636c7dab930 2023-10-05T19:40:24.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.563+0000 7f10a59b2e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:24.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.563+0000 7f10a59b2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.563+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de6dd0 con 0x5636c7dde6f0 2023-10-05T19:40:24.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.563+0000 7f10a34e7700 1 -- 172.21.15.12:0/4274334332 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f1094041380 con 0x5636c7dde6f0 2023-10-05T19:40:24.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.563+0000 7f10a59b2e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:24.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.563+0000 7f10a59b2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.563+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de7670 con 0x5636c7dab930 2023-10-05T19:40:24.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.563+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f109c00a5a0 con 0x5636c7dab930 2023-10-05T19:40:24.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.563+0000 7f10a59b2e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:24.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.563+0000 7f10a59b2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.563+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de7670 con 0x5636c7de3070 2023-10-05T19:40:24.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.563+0000 7f10a2ce6700 1 -- 172.21.15.12:0/4274334332 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f108c006070 con 0x5636c7de3070 2023-10-05T19:40:24.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.563+0000 7f10a59b2e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:24.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.563+0000 7f10a59b2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.563+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de7f10 con 0x7f1094039070 2023-10-05T19:40:24.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.567+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f109c00a5a0 con 0x7f1094039070 2023-10-05T19:40:24.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.567+0000 7f10a59b2e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:24.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.567+0000 7f10a59b2e40 20 zone default found 2023-10-05T19:40:24.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.567+0000 7f10a59b2e40 4 Realm: () 2023-10-05T19:40:24.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.567+0000 7f10a59b2e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:24.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.567+0000 7f10a59b2e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:24.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.567+0000 7f10a59b2e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:24.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.567+0000 7f10a59b2e40 20 zonegroup default 2023-10-05T19:40:24.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.567+0000 7f10a59b2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.567+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de7f10 con 0x7f1094039070 2023-10-05T19:40:24.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.567+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f109c00a5a0 con 0x7f1094039070 2023-10-05T19:40:24.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.567+0000 7f10a59b2e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:24.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.567+0000 7f10a59b2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.567+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de9890 con 0x5636c7de3070 2023-10-05T19:40:24.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.567+0000 7f10a2ce6700 1 -- 172.21.15.12:0/4274334332 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f108c006070 con 0x5636c7de3070 2023-10-05T19:40:24.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.567+0000 7f10a59b2e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:24.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.567+0000 7f10a59b2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.567+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de9c40 con 0x5636c7de3070 2023-10-05T19:40:24.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.567+0000 7f10a2ce6700 1 -- 172.21.15.12:0/4274334332 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f108c006070 con 0x5636c7de3070 2023-10-05T19:40:24.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.567+0000 7f10a59b2e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:24.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.567+0000 7f10a59b2e40 20 started sync module instance, tier type = 2023-10-05T19:40:24.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.567+0000 7f10a59b2e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:24.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.567+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de8f50 con 0x5636c7dde6f0 2023-10-05T19:40:24.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.571+0000 7f10a34e7700 1 -- 172.21.15.12:0/4274334332 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'102 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1094041380 con 0x5636c7dde6f0 2023-10-05T19:40:24.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.571+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94793281473120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de9ff0 con 0x5636c7dde6f0 2023-10-05T19:40:24.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.571+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7dea890 con 0x5636c7dab930 2023-10-05T19:40:24.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.575+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'102 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f109c00a5a0 con 0x5636c7dab930 2023-10-05T19:40:24.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.575+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94793281478640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7dea9b0 con 0x5636c7dab930 2023-10-05T19:40:24.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.575+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7deb510 con 0x5636c7de3070 2023-10-05T19:40:24.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.575+0000 7f10a34e7700 1 -- 172.21.15.12:0/4274334332 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94793281473120] v28'103 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1094041380 con 0x5636c7dde6f0 2023-10-05T19:40:24.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.575+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94793281478640] v28'103 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f109c00a5a0 con 0x5636c7dab930 2023-10-05T19:40:24.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.579+0000 7f10a2ce6700 1 -- 172.21.15.12:0/4274334332 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'102 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f108c006070 con 0x5636c7de3070 2023-10-05T19:40:24.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.579+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94793281482720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7dea440 con 0x5636c7de3070 2023-10-05T19:40:24.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.579+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7debe00 con 0x5636c7dde6f0 2023-10-05T19:40:24.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.579+0000 7f10a34e7700 1 -- 172.21.15.12:0/4274334332 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'102 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1094041380 con 0x5636c7dde6f0 2023-10-05T19:40:24.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.579+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94793281485072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7dec250 con 0x5636c7dde6f0 2023-10-05T19:40:24.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.579+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7decd40 con 0x5636c7dab930 2023-10-05T19:40:24.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.579+0000 7f10a2ce6700 1 -- 172.21.15.12:0/4274334332 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94793281482720] v28'103 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f108c006070 con 0x5636c7de3070 2023-10-05T19:40:24.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.583+0000 7f10a34e7700 1 -- 172.21.15.12:0/4274334332 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94793281485072] v28'103 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1094041380 con 0x5636c7dde6f0 2023-10-05T19:40:24.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.583+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'102 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f109c00a5a0 con 0x5636c7dab930 2023-10-05T19:40:24.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.583+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94793281488976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7dec250 con 0x5636c7dab930 2023-10-05T19:40:24.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.583+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7decca0 con 0x7f1094039070 2023-10-05T19:40:24.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.587+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'102 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f109c00a5a0 con 0x7f1094039070 2023-10-05T19:40:24.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.587+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94793281491104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7decca0 con 0x7f1094039070 2023-10-05T19:40:24.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.587+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7dee110 con 0x7f1094039070 2023-10-05T19:40:24.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.587+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94793281488976] v28'103 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f109c00a5a0 con 0x5636c7dab930 2023-10-05T19:40:24.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.587+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(95 notify.5 [watch watch cookie 94793281491104] v28'103 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f109c00a5a0 con 0x7f1094039070 2023-10-05T19:40:24.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.587+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(96 notify.6 [create] v28'102 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f109c00a5a0 con 0x7f1094039070 2023-10-05T19:40:24.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.587+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94793281492208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7dedbe0 con 0x7f1094039070 2023-10-05T19:40:24.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.587+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7deea00 con 0x7f1094039070 2023-10-05T19:40:24.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.591+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(97 notify.6 [watch watch cookie 94793281492208] v28'103 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f109c00a5a0 con 0x7f1094039070 2023-10-05T19:40:24.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.591+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(98 notify.7 [create] v28'102 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f109c00a5a0 con 0x7f1094039070 2023-10-05T19:40:24.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.591+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94793281486176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7dedbe0 con 0x7f1094039070 2023-10-05T19:40:24.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.591+0000 7f10a59b2e40 20 add_watcher() i=0 2023-10-05T19:40:24.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.591+0000 7f10a59b2e40 20 add_watcher() i=1 2023-10-05T19:40:24.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.591+0000 7f10a59b2e40 20 add_watcher() i=2 2023-10-05T19:40:24.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.591+0000 7f10a59b2e40 20 add_watcher() i=3 2023-10-05T19:40:24.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.591+0000 7f10a59b2e40 20 add_watcher() i=4 2023-10-05T19:40:24.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.591+0000 7f10a59b2e40 20 add_watcher() i=5 2023-10-05T19:40:24.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.591+0000 7f10a59b2e40 20 add_watcher() i=6 2023-10-05T19:40:24.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.595+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94793281486176] v28'103 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f109c00a5a0 con 0x7f1094039070 2023-10-05T19:40:24.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.595+0000 7f10a59b2e40 20 add_watcher() i=7 2023-10-05T19:40:24.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.595+0000 7f10a59b2e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:24.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.595+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7df1850 con 0x5636c7dab930 2023-10-05T19:40:24.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.595+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f109c00a5a0 con 0x5636c7dab930 2023-10-05T19:40:24.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.595+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94793281495616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7deed10 con 0x5636c7dab930 2023-10-05T19:40:24.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.595+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94793281495616] v28'103 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f109c00a5a0 con 0x5636c7dab930 2023-10-05T19:40:24.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.595+0000 7f1074fe1700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:24.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.595+0000 7f10a59b2e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:24.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.595+0000 7f10a59b2e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:24.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.599+0000 7f10a59b2e40 1 Processor -- start 2023-10-05T19:40:24.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.599+0000 7f10a59b2e40 1 -- start start 2023-10-05T19:40:24.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.599+0000 7f10a59b2e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636c7dfbc30 0x5636c7dfc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:24.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.599+0000 7f10a59b2e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636c7dfc540 con 0x5636c7dfbc30 2023-10-05T19:40:24.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.599+0000 7f10a2ce6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636c7dfbc30 0x5636c7dfc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.599+0000 7f10a2ce6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636c7dfbc30 0x5636c7dfc000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57764/0 (socket says 172.21.15.12:57764) 2023-10-05T19:40:24.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.599+0000 7f10a2ce6700 1 -- 172.21.15.12:0/166925696 learned_addr learned my addr 172.21.15.12:0/166925696 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:24.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.599+0000 7f10a2ce6700 1 -- 172.21.15.12:0/166925696 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636c7dfc680 con 0x5636c7dfbc30 2023-10-05T19:40:24.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.599+0000 7f10a2ce6700 1 --2- 172.21.15.12:0/166925696 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636c7dfbc30 0x5636c7dfc000 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1094011700 tx=0x7f108c01e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=175f1f68ee9228fa server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.599+0000 7f0feffff700 1 -- 172.21.15.12:0/166925696 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f108c013040 con 0x5636c7dfbc30 2023-10-05T19:40:24.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.599+0000 7f0feffff700 1 -- 172.21.15.12:0/166925696 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f108c022040 con 0x5636c7dfbc30 2023-10-05T19:40:24.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.599+0000 7f0feffff700 1 -- 172.21.15.12:0/166925696 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f108c00ee90 con 0x5636c7dfbc30 2023-10-05T19:40:24.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.599+0000 7f10a59b2e40 1 -- 172.21.15.12:0/166925696 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636c7dfbc30 msgr2=0x5636c7dfc000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.599+0000 7f10a59b2e40 1 --2- 172.21.15.12:0/166925696 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636c7dfbc30 0x5636c7dfc000 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1094011700 tx=0x7f108c01e040 comp rx=0 tx=0).stop 2023-10-05T19:40:24.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.599+0000 7f10a59b2e40 1 -- 172.21.15.12:0/166925696 shutdown_connections 2023-10-05T19:40:24.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.599+0000 7f10a59b2e40 1 --2- 172.21.15.12:0/166925696 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636c7dfbc30 0x5636c7dfc000 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:24.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.599+0000 7f10a59b2e40 1 -- 172.21.15.12:0/166925696 >> 172.21.15.12:0/166925696 conn(0x5636c7df6ff0 msgr2=0x5636c7df51a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:24.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.599+0000 7f10a59b2e40 1 -- 172.21.15.12:0/166925696 shutdown_connections 2023-10-05T19:40:24.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.599+0000 7f10a59b2e40 1 -- 172.21.15.12:0/166925696 wait complete. 2023-10-05T19:40:24.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.603+0000 7f10a59b2e40 1 Processor -- start 2023-10-05T19:40:24.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.603+0000 7f10a59b2e40 1 -- start start 2023-10-05T19:40:24.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.603+0000 7f10a59b2e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636c7dfbc30 0x5636c7e01780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:24.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.603+0000 7f10a59b2e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636c7e02780 con 0x5636c7dfbc30 2023-10-05T19:40:24.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.603+0000 7f10a2ce6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636c7dfbc30 0x5636c7e01780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.603+0000 7f10a2ce6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636c7dfbc30 0x5636c7e01780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57780/0 (socket says 172.21.15.12:57780) 2023-10-05T19:40:24.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.603+0000 7f10a2ce6700 1 -- 172.21.15.12:0/1187783617 learned_addr learned my addr 172.21.15.12:0/1187783617 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:24.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.603+0000 7f10a2ce6700 1 -- 172.21.15.12:0/1187783617 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f108c034040 con 0x5636c7dfbc30 2023-10-05T19:40:24.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.603+0000 7f10a2ce6700 1 --2- 172.21.15.12:0/1187783617 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636c7dfbc30 0x5636c7e01780 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f108c045000 tx=0x7f108c040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.603+0000 7f0fee7fc700 1 -- 172.21.15.12:0/1187783617 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f108c03be90 con 0x5636c7dfbc30 2023-10-05T19:40:24.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.603+0000 7f0fee7fc700 1 -- 172.21.15.12:0/1187783617 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f108c02c640 con 0x5636c7dfbc30 2023-10-05T19:40:24.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.603+0000 7f10a59b2e40 1 -- 172.21.15.12:0/1187783617 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5636c7e01d80 con 0x5636c7dfbc30 2023-10-05T19:40:24.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.603+0000 7f0fee7fc700 1 -- 172.21.15.12:0/1187783617 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f108c02c7f0 con 0x5636c7dfbc30 2023-10-05T19:40:24.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.603+0000 7f10a59b2e40 1 -- 172.21.15.12:0/1187783617 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5636c7e021c0 con 0x5636c7dfbc30 2023-10-05T19:40:24.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.607+0000 7f0fee7fc700 1 -- 172.21.15.12:0/1187783617 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f108c04d070 con 0x5636c7dfbc30 2023-10-05T19:40:24.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.607+0000 7f0fee7fc700 1 --2- 172.21.15.12:0/1187783617 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f0fe4030c50 0x7f0fe4033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:24.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.607+0000 7f0fee7fc700 1 -- 172.21.15.12:0/1187783617 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f108c050ad0 con 0x5636c7dfbc30 2023-10-05T19:40:24.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.607+0000 7f10a3ce8700 1 --2- 172.21.15.12:0/1187783617 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f0fe4030c50 0x7f0fe4033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.607+0000 7f10a59b2e40 5 note: GC not initialized 2023-10-05T19:40:24.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.607+0000 7f10a3ce8700 1 --2- 172.21.15.12:0/1187783617 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f0fe4030c50 0x7f0fe4033100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x5636c7dede60 tx=0x7f109c00ae20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.607+0000 7f0fe37fe700 20 reqs_thread_entry: start 2023-10-05T19:40:24.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.607+0000 7f10a59b2e40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:24.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.607+0000 7f0fe17fa700 20 reqs_thread_entry: start 2023-10-05T19:40:24.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.607+0000 7f10a59b2e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:24.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.607+0000 7f0fcf7fe700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:54 2023 2023-10-05T19:40:24.618 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:24.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.607+0000 7f0fceffd700 20 reqs_thread_entry: start 2023-10-05T19:40:24.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.611+0000 7f10a59b2e40 20 get_system_obj_state: rctx=0x7ffc9d4a77d8 obj=default.rgw.meta:users.uid:foo state=0x5636c7c820b0 s->prefetch_data=0 2023-10-05T19:40:24.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.611+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7e6a090 con 0x5636c7dab930 2023-10-05T19:40:24.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.611+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(102 foo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7f109c00a5a0 con 0x5636c7dab930 2023-10-05T19:40:24.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.611+0000 7f10a59b2e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:24.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.611+0000 7f10a59b2e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:24.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.611+0000 7f10a59b2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.611+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7e6a010 con 0x5636c7dab930 2023-10-05T19:40:24.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.611+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(103 foo [call,call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv11 ondisk = 0) v8 ==== 273+0+480 (crc 0 0 0) 0x7f109c00a5a0 con 0x5636c7dab930 2023-10-05T19:40:24.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.611+0000 7f10a59b2e40 20 rados_obj.operate() r=0 bl.length=357 2023-10-05T19:40:24.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.611+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:104 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7e6a010 con 0x5636c7dde6f0 2023-10-05T19:40:24.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.611+0000 7f10a34e7700 1 -- 172.21.15.12:0/4274334332 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(104 foo.buckets [call out=183b] v0'0 uv14 ondisk = 0) v8 ==== 155+0+183 (crc 0 0 0) 0x7f1094041380 con 0x5636c7dde6f0 2023-10-05T19:40:24.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.611+0000 7f10a59b2e40 20 get_system_obj_state: rctx=0x7ffc9d4a7b30 obj=default.rgw.meta:root:.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 state=0x5636c7c820b0 s->prefetch_data=0 2023-10-05T19:40:24.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.611+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:105 8.e 8:70176bbd:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7e6a010 con 0x7f1094039070 2023-10-05T19:40:24.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.611+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(105 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 [call out=48b,getxattrs out=218b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 290+0+282 (crc 0 0 0) 0x7f109c00a5a0 con 0x7f1094039070 2023-10-05T19:40:24.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.611+0000 7f10a59b2e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:24.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.611+0000 7f10a59b2e40 20 Read xattr: user.rgw.acl 2023-10-05T19:40:24.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.611+0000 7f10a59b2e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:24.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.611+0000 7f10a59b2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.611+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:106 8.e 8:70176bbd:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7e6dda0 con 0x7f1094039070 2023-10-05T19:40:24.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.615+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(106 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 [call,call out=48b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 332+0+618 (crc 0 0 0) 0x7f109c00a5a0 con 0x7f1094039070 2023-10-05T19:40:24.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.615+0000 7f10a59b2e40 20 rados_obj.operate() r=0 bl.length=352 2023-10-05T19:40:24.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.615+0000 7f10a59b2e40 20 couldn't put binfo cache entry, might have raced with data changes 2023-10-05T19:40:24.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.615+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:107 4.3 4:c350083b:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7e6d8c0 con 0x5636c7dab930 2023-10-05T19:40:24.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.615+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:108 4.1c 4:38d73088:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7e6dd80 con 0x7f1094039070 2023-10-05T19:40:24.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.615+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:109 4.a 4:502f3015:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7e730f0 con 0x5636c7dde6f0 2023-10-05T19:40:24.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.615+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:110 4.3e 4:7e570c43:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7e75aa0 con 0x5636c7dde6f0 2023-10-05T19:40:24.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.615+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:111 4.7 4:e38d8107:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7e785b0 con 0x7f1094039070 2023-10-05T19:40:24.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.615+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:112 4.1a 4:5add8e75:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7e730f0 con 0x5636c7dde6f0 2023-10-05T19:40:24.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.615+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:113 4.2c 4:358e5801:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7e785b0 con 0x5636c7dab930 2023-10-05T19:40:24.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.615+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:114 4.2 4:4101cfe2:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7e7f980 con 0x5636c7dab930 2023-10-05T19:40:24.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.615+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:115 4.30 4:0e5788c0:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7e82520 con 0x5636c7de3070 2023-10-05T19:40:24.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.615+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:116 4.19 4:99ccd5a2:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7e84de0 con 0x5636c7de3070 2023-10-05T19:40:24.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.615+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:117 4.2d 4:b73f2ca3:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7e878b0 con 0x7f1094039070 2023-10-05T19:40:24.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.615+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(108 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f109c00a5a0 con 0x7f1094039070 2023-10-05T19:40:24.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.615+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.0 v2:172.21.15.8:6800/27563 30 ==== osd_op_reply(111 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f109c00a5a0 con 0x7f1094039070 2023-10-05T19:40:24.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.615+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.0 v2:172.21.15.8:6800/27563 31 ==== osd_op_reply(117 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7f109c00a5a0 con 0x7f1094039070 2023-10-05T19:40:24.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.615+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(107 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f109c00a5a0 con 0x5636c7dab930 2023-10-05T19:40:24.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.615+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(114 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.7 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f109c00a5a0 con 0x5636c7dab930 2023-10-05T19:40:24.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.615+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(113 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.6 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f109c00a5a0 con 0x5636c7dab930 2023-10-05T19:40:24.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.615+0000 7f10a34e7700 1 -- 172.21.15.12:0/4274334332 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(109 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.2 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f1094041380 con 0x5636c7dde6f0 2023-10-05T19:40:24.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.615+0000 7f10a34e7700 1 -- 172.21.15.12:0/4274334332 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(110 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.3 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f1094041380 con 0x5636c7dde6f0 2023-10-05T19:40:24.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.615+0000 7f10a34e7700 1 -- 172.21.15.12:0/4274334332 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(112 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.5 [call out=124b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x7f1094041380 con 0x5636c7dde6f0 2023-10-05T19:40:24.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.615+0000 7f10a2ce6700 1 -- 172.21.15.12:0/4274334332 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(116 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.9 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f108c055070 con 0x5636c7de3070 2023-10-05T19:40:24.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.615+0000 7f10a2ce6700 1 -- 172.21.15.12:0/4274334332 <== osd.3 v2:172.21.15.12:6816/29951 27 ==== osd_op_reply(115 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.8 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f108c055070 con 0x5636c7de3070 2023-10-05T19:40:24.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.615+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:118 8.9 8:96d98480:users.uid::foo.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7e691e0 con 0x5636c7dde6f0 2023-10-05T19:40:24.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.619+0000 7f10a34e7700 1 -- 172.21.15.12:0/4274334332 <== osd.2 v2:172.21.15.12:6800/29934 23 ==== osd_op_reply(118 foo.buckets [call] v28'15 uv15 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x7f1094041380 con 0x5636c7dde6f0 2023-10-05T19:40:24.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.619+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:119 8.9 8:96d98480:users.uid::foo.buckets:head [call user.complete_stats_sync in=37b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7e878b0 con 0x5636c7dde6f0 2023-10-05T19:40:24.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.623+0000 7f10a34e7700 1 -- 172.21.15.12:0/4274334332 <== osd.2 v2:172.21.15.12:6800/29934 24 ==== osd_op_reply(119 foo.buckets [call] v28'16 uv16 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x7f109405b040 con 0x5636c7dde6f0 2023-10-05T19:40:24.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.623+0000 7f10a59b2e40 20 get_system_obj_state: rctx=0x7ffc9d4a7a18 obj=default.rgw.meta:users.uid:foo state=0x5636c7c820b0 s->prefetch_data=0 2023-10-05T19:40:24.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.623+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:120 8.18 8:1e6ca5ed:users.uid::foo:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7e878b0 con 0x5636c7dab930 2023-10-05T19:40:24.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.623+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== osd_op_reply(120 foo [getxattrs out=75b,stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 189+0+91 (crc 0 0 0) 0x7f109c00a5a0 con 0x5636c7dab930 2023-10-05T19:40:24.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.623+0000 7f10a59b2e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:24.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.623+0000 7f10a59b2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.623+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:121 8.18 8:1e6ca5ed:users.uid::foo:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7e878b0 con 0x5636c7dab930 2023-10-05T19:40:24.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.623+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.1 v2:172.21.15.12:6808/29933 39 ==== osd_op_reply(121 foo [read 0~357 out=357b] v0'0 uv11 ondisk = 0) v8 ==== 147+0+357 (crc 0 0 0) 0x7f109c00a5a0 con 0x5636c7dab930 2023-10-05T19:40:24.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.623+0000 7f10a59b2e40 20 rados_obj.operate() r=0 bl.length=357 2023-10-05T19:40:24.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.623+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:122 8.9 8:96d98480:users.uid::foo.buckets:head [call user.get_header in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7e878b0 con 0x5636c7dde6f0 2023-10-05T19:40:24.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.627+0000 7f10a34e7700 1 -- 172.21.15.12:0/4274334332 <== osd.2 v2:172.21.15.12:6800/29934 25 ==== osd_op_reply(122 foo.buckets [call out=58b] v0'0 uv16 ondisk = 0) v8 ==== 155+0+58 (crc 0 0 0) 0x7f109405b040 con 0x5636c7dde6f0 2023-10-05T19:40:24.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.627+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:123 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94793281473120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7e77f80 con 0x5636c7dde6f0 2023-10-05T19:40:24.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.627+0000 7f10a34e7700 1 -- 172.21.15.12:0/4274334332 <== osd.2 v2:172.21.15.12:6800/29934 26 ==== osd_op_reply(123 notify.0 [watch unwatch cookie 94793281473120] v28'104 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f109405b040 con 0x5636c7dde6f0 2023-10-05T19:40:24.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.627+0000 7f10a59b2e40 20 remove_watcher() i=0 2023-10-05T19:40:24.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.627+0000 7f10a59b2e40 2 removed watcher, disabling cache 2023-10-05T19:40:24.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.627+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:124 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94793281478640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7dc0870 con 0x5636c7dab930 2023-10-05T19:40:24.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.631+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.1 v2:172.21.15.12:6808/29933 40 ==== osd_op_reply(124 notify.1 [watch unwatch cookie 94793281478640] v28'104 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f109c00a5a0 con 0x5636c7dab930 2023-10-05T19:40:24.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.631+0000 7f10a59b2e40 20 remove_watcher() i=1 2023-10-05T19:40:24.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.631+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:125 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94793281482720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de9ff0 con 0x5636c7de3070 2023-10-05T19:40:24.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.635+0000 7f10a2ce6700 1 -- 172.21.15.12:0/4274334332 <== osd.3 v2:172.21.15.12:6816/29951 28 ==== osd_op_reply(125 notify.2 [watch unwatch cookie 94793281482720] v28'104 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f108c055070 con 0x5636c7de3070 2023-10-05T19:40:24.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.635+0000 7f10a59b2e40 20 remove_watcher() i=2 2023-10-05T19:40:24.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.635+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:126 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94793281485072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de9ff0 con 0x5636c7dde6f0 2023-10-05T19:40:24.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.635+0000 7f10a34e7700 1 -- 172.21.15.12:0/4274334332 <== osd.2 v2:172.21.15.12:6800/29934 27 ==== osd_op_reply(126 notify.3 [watch unwatch cookie 94793281485072] v28'104 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f109405b040 con 0x5636c7dde6f0 2023-10-05T19:40:24.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.635+0000 7f10a59b2e40 20 remove_watcher() i=3 2023-10-05T19:40:24.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.635+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:127 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94793281488976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de9ff0 con 0x5636c7dab930 2023-10-05T19:40:24.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.639+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.1 v2:172.21.15.12:6808/29933 41 ==== osd_op_reply(127 notify.4 [watch unwatch cookie 94793281488976] v28'104 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f109c00a5a0 con 0x5636c7dab930 2023-10-05T19:40:24.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.639+0000 7f10a59b2e40 20 remove_watcher() i=4 2023-10-05T19:40:24.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.639+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:128 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94793281491104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7dec850 con 0x7f1094039070 2023-10-05T19:40:24.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.643+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.0 v2:172.21.15.8:6800/27563 32 ==== osd_op_reply(128 notify.5 [watch unwatch cookie 94793281491104] v28'104 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f109c00a5a0 con 0x7f1094039070 2023-10-05T19:40:24.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.643+0000 7f10a59b2e40 20 remove_watcher() i=5 2023-10-05T19:40:24.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.643+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:129 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94793281492208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de9ff0 con 0x7f1094039070 2023-10-05T19:40:24.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.647+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.0 v2:172.21.15.8:6800/27563 33 ==== osd_op_reply(129 notify.6 [watch unwatch cookie 94793281492208] v28'104 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f109c00a5a0 con 0x7f1094039070 2023-10-05T19:40:24.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.647+0000 7f10a59b2e40 20 remove_watcher() i=6 2023-10-05T19:40:24.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.647+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:130 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94793281486176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de9ff0 con 0x7f1094039070 2023-10-05T19:40:24.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.647+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.0 v2:172.21.15.8:6800/27563 34 ==== osd_op_reply(130 notify.7 [watch unwatch cookie 94793281486176] v28'104 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f109c00a5a0 con 0x7f1094039070 2023-10-05T19:40:24.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.647+0000 7f10a59b2e40 20 remove_watcher() i=7 2023-10-05T19:40:24.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.651+0000 7f1074fe1700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:24.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.651+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:131 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94793281495616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5636c7de9ff0 con 0x5636c7dab930 2023-10-05T19:40:24.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.651+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 <== osd.1 v2:172.21.15.12:6808/29933 42 ==== osd_op_reply(131 data_loggenerations_metadata [watch unwatch cookie 94793281495616] v28'104 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f109c00a5a0 con 0x5636c7dab930 2023-10-05T19:40:24.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.651+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f1094039070 msgr2=0x7f109400c9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.651+0000 7f10a59b2e40 1 --2- 172.21.15.12:0/4274334332 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f1094039070 0x7f109400c9c0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:24.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.651+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5636c7dab930 msgr2=0x5636c7ddc610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.651+0000 7f10a59b2e40 1 --2- 172.21.15.12:0/4274334332 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5636c7dab930 0x5636c7ddc610 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:24.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.651+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5636c7dde6f0 msgr2=0x5636c7de0ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.651+0000 7f10a59b2e40 1 --2- 172.21.15.12:0/4274334332 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5636c7dde6f0 0x5636c7de0ad0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:24.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.651+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5636c7de3070 msgr2=0x5636c7de5450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.651+0000 7f10a59b2e40 1 --2- 172.21.15.12:0/4274334332 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5636c7de3070 0x5636c7de5450 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:24.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.651+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1084030ca0 msgr2=0x7f1084033150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.651+0000 7f10a59b2e40 1 --2- 172.21.15.12:0/4274334332 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1084030ca0 0x7f1084033150 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f108c004170 tx=0x7f108c001400 comp rx=0 tx=0).stop 2023-10-05T19:40:24.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.651+0000 7f10a3ce8700 1 -- 172.21.15.12:0/4274334332 reap_dead start 2023-10-05T19:40:24.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.651+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636c7dab020 msgr2=0x5636c7dd0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.651+0000 7f10a59b2e40 1 --2- 172.21.15.12:0/4274334332 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636c7dab020 0x5636c7dd0bb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f109400afd0 tx=0x7f1094003050 comp rx=0 tx=0).stop 2023-10-05T19:40:24.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.651+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 shutdown_connections 2023-10-05T19:40:24.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.651+0000 7f10a59b2e40 1 --2- 172.21.15.12:0/4274334332 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636c7dab020 0x5636c7dd0bb0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:24.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.651+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 >> 172.21.15.12:0/4274334332 conn(0x5636c7d1e870 msgr2=0x5636c7dc4590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:24.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.651+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 shutdown_connections 2023-10-05T19:40:24.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.651+0000 7f10a59b2e40 1 -- 172.21.15.12:0/4274334332 wait complete. 2023-10-05T19:40:24.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.655+0000 7f10a59b2e40 1 -- 172.21.15.12:0/1187783617 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f0fe4030c50 msgr2=0x7f0fe4033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.655+0000 7f10a59b2e40 1 --2- 172.21.15.12:0/1187783617 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f0fe4030c50 0x7f0fe4033100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x5636c7dede60 tx=0x7f109c00ae20 comp rx=0 tx=0).stop 2023-10-05T19:40:24.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.655+0000 7f10a59b2e40 1 -- 172.21.15.12:0/1187783617 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636c7dfbc30 msgr2=0x5636c7e01780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.655+0000 7f10a59b2e40 1 --2- 172.21.15.12:0/1187783617 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636c7dfbc30 0x5636c7e01780 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f108c045000 tx=0x7f108c040000 comp rx=0 tx=0).stop 2023-10-05T19:40:24.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.655+0000 7f10a59b2e40 1 -- 172.21.15.12:0/1187783617 shutdown_connections 2023-10-05T19:40:24.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.655+0000 7f10a59b2e40 1 --2- 172.21.15.12:0/1187783617 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f0fe4030c50 0x7f0fe4033100 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:24.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.655+0000 7f10a59b2e40 1 --2- 172.21.15.12:0/1187783617 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5636c7dfbc30 0x5636c7e01780 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:24.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.655+0000 7f10a59b2e40 1 -- 172.21.15.12:0/1187783617 >> 172.21.15.12:0/1187783617 conn(0x5636c7df6ff0 msgr2=0x5636c7df50f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:24.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.655+0000 7f10a59b2e40 1 -- 172.21.15.12:0/1187783617 shutdown_connections 2023-10-05T19:40:24.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.655+0000 7f10a59b2e40 1 -- 172.21.15.12:0/1187783617 wait complete. 2023-10-05T19:40:24.665 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:40:24.665 INFO:teuthology.orchestra.run.smithi012.stdout: "stats": { 2023-10-05T19:40:24.666 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 3, 2023-10-05T19:40:24.666 INFO:teuthology.orchestra.run.smithi012.stdout: "size_actual": 4096, 2023-10-05T19:40:24.666 INFO:teuthology.orchestra.run.smithi012.stdout: "size_kb": 1, 2023-10-05T19:40:24.666 INFO:teuthology.orchestra.run.smithi012.stdout: "size_kb_actual": 4, 2023-10-05T19:40:24.666 INFO:teuthology.orchestra.run.smithi012.stdout: "num_objects": 1 2023-10-05T19:40:24.666 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:24.666 INFO:teuthology.orchestra.run.smithi012.stdout: "last_stats_sync": "2023-10-05T19:40:24.625504Z", 2023-10-05T19:40:24.666 INFO:teuthology.orchestra.run.smithi012.stdout: "last_stats_update": "2023-10-05T19:40:24.621743Z" 2023-10-05T19:40:24.666 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:40:24.667 DEBUG:tasks.util.rgw: json result: {'stats': {'size': 3, 'size_actual': 4096, 'size_kb': 1, 'size_kb_actual': 4, 'num_objects': 1}, 'last_stats_sync': '2023-10-05T19:40:24.625504Z', 'last_stats_update': '2023-10-05T19:40:24.621743Z'} 2023-10-05T19:40:24.677 INFO:tasks.radosgw_admin:RL: delete_obj myfoo foo bytes_out=0 bytes_in=0 failed=False 2023-10-05T19:40:24.677 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'unlink', '--uid', 'foo', '--bucket', 'myfoo'] 2023-10-05T19:40:24.677 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'unlink', '--uid', 'foo', '--bucket', 'myfoo'] 2023-10-05T19:40:24.677 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket unlink --uid foo --bucket myfoo 2023-10-05T19:40:24.719 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:24.719 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:24.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.711+0000 7f8ed76e5e40 1 Processor -- start 2023-10-05T19:40:24.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.711+0000 7f8ed76e5e40 1 -- start start 2023-10-05T19:40:24.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.711+0000 7f8ed76e5e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x5623b6c917f0 0x5623b6c906e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:24.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.711+0000 7f8ed76e5e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5623b6bddbd0 con 0x5623b6c91420 2023-10-05T19:40:24.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.711+0000 7f8ed76e5e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x5623b6b9aee0 con 0x5623b6c917f0 2023-10-05T19:40:24.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.711+0000 7f8ecca19700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x5623b6c917f0 0x5623b6c906e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.711+0000 7f8ed521a700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x5623b6c91420 0x5623b6c8ffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:35508/0 (socket says 172.21.15.12:35508) 2023-10-05T19:40:24.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.711+0000 7f8ed521a700 1 -- 172.21.15.12:0/3697834068 learned_addr learned my addr 172.21.15.12:0/3697834068 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:24.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.715+0000 7f8ecca19700 1 -- 172.21.15.12:0/3697834068 >> v1:172.21.15.8:6789/0 conn(0x5623b6c91420 legacy=0x5623b6c8ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.715+0000 7f8ed4a19700 1 -- 172.21.15.12:0/3697834068 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3327083344 0 0) 0x5623b6bddbd0 con 0x5623b6c91420 2023-10-05T19:40:24.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.715+0000 7f8ecca19700 1 -- 172.21.15.12:0/3697834068 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623b6c90ce0 con 0x5623b6c917f0 2023-10-05T19:40:24.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.715+0000 7f8ecca19700 1 --2- 172.21.15.12:0/3697834068 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623b6c917f0 0x5623b6c906e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8ec000a5d0 tx=0x7f8ec0005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=954124507a6c1569 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.715+0000 7f8ed4a19700 1 -- 172.21.15.12:0/3697834068 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ec00086a0 con 0x5623b6c917f0 2023-10-05T19:40:24.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.715+0000 7f8ed4a19700 1 -- 172.21.15.12:0/3697834068 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec0008800 con 0x5623b6c917f0 2023-10-05T19:40:24.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.715+0000 7f8ed4a19700 1 -- 172.21.15.12:0/3697834068 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ec0008ad0 con 0x5623b6c917f0 2023-10-05T19:40:24.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.715+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/3697834068 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623b6c917f0 msgr2=0x5623b6c906e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.715+0000 7f8ed76e5e40 1 --2- 172.21.15.12:0/3697834068 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623b6c917f0 0x5623b6c906e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8ec000a5d0 tx=0x7f8ec0005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:24.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.715+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/3697834068 shutdown_connections 2023-10-05T19:40:24.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.715+0000 7f8ed76e5e40 1 --2- 172.21.15.12:0/3697834068 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623b6c917f0 0x5623b6c906e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:24.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.715+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/3697834068 >> 172.21.15.12:0/3697834068 conn(0x5623b6c8b730 msgr2=0x5623b6c8db40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:24.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.715+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/3697834068 shutdown_connections 2023-10-05T19:40:24.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.715+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/3697834068 wait complete. 2023-10-05T19:40:24.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.715+0000 7f8ed76e5e40 1 Processor -- start 2023-10-05T19:40:24.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.715+0000 7f8ed76e5e40 1 -- start start 2023-10-05T19:40:24.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.715+0000 7f8ed76e5e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623b6c91420 0x5623b6cb1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:24.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.715+0000 7f8ed76e5e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623b6b5ee30 con 0x5623b6c91420 2023-10-05T19:40:24.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.715+0000 7f8ed521a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623b6c91420 0x5623b6cb1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.715+0000 7f8ed521a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623b6c91420 0x5623b6cb1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57790/0 (socket says 172.21.15.12:57790) 2023-10-05T19:40:24.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.715+0000 7f8ed521a700 1 -- 172.21.15.12:0/1758482427 learned_addr learned my addr 172.21.15.12:0/1758482427 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:24.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.719+0000 7f8ed521a700 1 -- 172.21.15.12:0/1758482427 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec0005cf0 con 0x5623b6c91420 2023-10-05T19:40:24.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.719+0000 7f8ed521a700 1 --2- 172.21.15.12:0/1758482427 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623b6c91420 0x5623b6cb1fd0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8001dc0 tx=0x7f8ec8001790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb4e13e518ee54c server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.719+0000 7f8ecf7fe700 1 -- 172.21.15.12:0/1758482427 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ec800bce0 con 0x5623b6c91420 2023-10-05T19:40:24.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.719+0000 7f8ecf7fe700 1 -- 172.21.15.12:0/1758482427 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec800be40 con 0x5623b6c91420 2023-10-05T19:40:24.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.719+0000 7f8ecf7fe700 1 -- 172.21.15.12:0/1758482427 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ec800e5d0 con 0x5623b6c91420 2023-10-05T19:40:24.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.719+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/1758482427 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623b6c91420 msgr2=0x5623b6cb1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.719+0000 7f8ed76e5e40 1 --2- 172.21.15.12:0/1758482427 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623b6c91420 0x5623b6cb1fd0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8001dc0 tx=0x7f8ec8001790 comp rx=0 tx=0).stop 2023-10-05T19:40:24.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.719+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/1758482427 shutdown_connections 2023-10-05T19:40:24.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.719+0000 7f8ed76e5e40 1 --2- 172.21.15.12:0/1758482427 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623b6c91420 0x5623b6cb1fd0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:24.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.719+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/1758482427 >> 172.21.15.12:0/1758482427 conn(0x5623b6c8b730 msgr2=0x5623b6cadb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:24.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.719+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/1758482427 shutdown_connections 2023-10-05T19:40:24.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.719+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/1758482427 wait complete. 2023-10-05T19:40:24.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.719+0000 7f8ed76e5e40 1 Processor -- start 2023-10-05T19:40:24.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.719+0000 7f8ed76e5e40 1 -- start start 2023-10-05T19:40:24.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.719+0000 7f8ed76e5e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623b6c91420 0x5623b6cb8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:24.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.719+0000 7f8ed76e5e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623b6b31ed0 con 0x5623b6c91420 2023-10-05T19:40:24.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.719+0000 7f8ed521a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623b6c91420 0x5623b6cb8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.719+0000 7f8ed521a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623b6c91420 0x5623b6cb8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57792/0 (socket says 172.21.15.12:57792) 2023-10-05T19:40:24.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.719+0000 7f8ed521a700 1 -- 172.21.15.12:0/2964906848 learned_addr learned my addr 172.21.15.12:0/2964906848 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:24.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.719+0000 7f8ed521a700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec8001a70 con 0x5623b6c91420 2023-10-05T19:40:24.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.719+0000 7f8ed521a700 1 --2- 172.21.15.12:0/2964906848 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623b6c91420 0x5623b6cb8d60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8ec80017d0 tx=0x7f8ec8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.723+0000 7f8ecdffb700 1 -- 172.21.15.12:0/2964906848 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ec801f070 con 0x5623b6c91420 2023-10-05T19:40:24.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.723+0000 7f8ecdffb700 1 -- 172.21.15.12:0/2964906848 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec801a4d0 con 0x5623b6c91420 2023-10-05T19:40:24.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.723+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5623b6cb92a0 con 0x5623b6c91420 2023-10-05T19:40:24.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.723+0000 7f8ecdffb700 1 -- 172.21.15.12:0/2964906848 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ec801a7a0 con 0x5623b6c91420 2023-10-05T19:40:24.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.723+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5623b6cb96e0 con 0x5623b6c91420 2023-10-05T19:40:24.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.723+0000 7f8ed76e5e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.723+0000 7f8ecdffb700 1 -- 172.21.15.12:0/2964906848 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f8ec801a900 con 0x5623b6c91420 2023-10-05T19:40:24.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.723+0000 7f8ecdffb700 1 --2- 172.21.15.12:0/2964906848 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f8eb8030c50 0x7f8eb8033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:24.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.723+0000 7f8ecdffb700 1 -- 172.21.15.12:0/2964906848 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f8ec8045050 con 0x5623b6c91420 2023-10-05T19:40:24.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.723+0000 7f8ecca19700 1 --2- 172.21.15.12:0/2964906848 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f8eb8030c50 0x7f8eb8033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.723+0000 7f8ed76e5e40 1 --2- 172.21.15.12:0/2964906848 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5623b6c917f0 0x5623b6cc4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:24.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.723+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cc4ed0 con 0x5623b6c917f0 2023-10-05T19:40:24.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.723+0000 7f8ed5a1b700 1 --2- 172.21.15.12:0/2964906848 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5623b6c917f0 0x5623b6cc4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.723+0000 7f8ecca19700 1 --2- 172.21.15.12:0/2964906848 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f8eb8030c50 0x7f8eb8033100 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0005a90 tx=0x7f8ec0015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.723+0000 7f8ed5a1b700 1 --2- 172.21.15.12:0/2964906848 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5623b6c917f0 0x5623b6cc4820 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.723+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8ed000a5a0 con 0x5623b6c917f0 2023-10-05T19:40:24.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.723+0000 7f8ed76e5e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:24.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.723+0000 7f8ed76e5e40 20 realm 2023-10-05T19:40:24.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.723+0000 7f8ed76e5e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.723+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cc1e80 con 0x5623b6c917f0 2023-10-05T19:40:24.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.727+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8ed000a5a0 con 0x5623b6c917f0 2023-10-05T19:40:24.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.727+0000 7f8ed76e5e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:24.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.727+0000 7f8ed76e5e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:24.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.727+0000 7f8ed76e5e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.727+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cc59b0 con 0x5623b6c917f0 2023-10-05T19:40:24.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.727+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8ed000a5a0 con 0x5623b6c917f0 2023-10-05T19:40:24.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.727+0000 7f8ed76e5e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:24.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.727+0000 7f8ed76e5e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.727+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cc55c0 con 0x5623b6c917f0 2023-10-05T19:40:24.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.727+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f8ed000a5a0 con 0x5623b6c917f0 2023-10-05T19:40:24.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.727+0000 7f8ed76e5e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:24.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.727+0000 7f8ed76e5e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.727+0000 7f8ed76e5e40 1 --2- 172.21.15.12:0/2964906848 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5623b6cc6d50 0x5623b6cc9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:24.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.727+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cc9670 con 0x5623b6cc6d50 2023-10-05T19:40:24.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.727+0000 7f8ed521a700 1 --2- 172.21.15.12:0/2964906848 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5623b6cc6d50 0x5623b6cc9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.727+0000 7f8ed521a700 1 --2- 172.21.15.12:0/2964906848 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5623b6cc6d50 0x5623b6cc9130 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.727+0000 7f8ed521a700 1 -- 172.21.15.12:0/2964906848 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f8ec8014070 con 0x5623b6cc6d50 2023-10-05T19:40:24.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.727+0000 7f8ed76e5e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:24.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.727+0000 7f8ed76e5e40 20 searching for the correct realm 2023-10-05T19:40:24.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.727+0000 7f8ed76e5e40 1 --2- 172.21.15.12:0/2964906848 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5623b6ccb4f0 0x5623b6ccd900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:24.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.727+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ccdfb0 con 0x5623b6ccb4f0 2023-10-05T19:40:24.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.727+0000 7f8ecca19700 1 --2- 172.21.15.12:0/2964906848 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5623b6ccb4f0 0x5623b6ccd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.727+0000 7f8ecca19700 1 --2- 172.21.15.12:0/2964906848 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5623b6ccb4f0 0x5623b6ccd900 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.731+0000 7f8ecca19700 1 -- 172.21.15.12:0/2964906848 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f8ec0008ae0 con 0x5623b6ccb4f0 2023-10-05T19:40:24.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.731+0000 7f8ecca19700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ccdfb0 con 0x5623b6cc6d50 2023-10-05T19:40:24.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.731+0000 7f8ed521a700 1 -- 172.21.15.12:0/2964906848 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ec8014070 con 0x5623b6cc6d50 2023-10-05T19:40:24.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.731+0000 7f8ed521a700 1 --2- 172.21.15.12:0/2964906848 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f8ec802d020 0x7f8ec802c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:24.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.731+0000 7f8ed521a700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ccdfb0 con 0x7f8ec802d020 2023-10-05T19:40:24.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.731+0000 7f8ed5a1b700 1 --2- 172.21.15.12:0/2964906848 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f8ec802d020 0x7f8ec802c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.731+0000 7f8ed5a1b700 1 --2- 172.21.15.12:0/2964906848 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f8ec802d020 0x7f8ec802c5d0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.735+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ed000a5a0 con 0x7f8ec802d020 2023-10-05T19:40:24.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.735+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ccdfb0 con 0x5623b6c917f0 2023-10-05T19:40:24.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.735+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ed000a5a0 con 0x5623b6c917f0 2023-10-05T19:40:24.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.735+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ccdfb0 con 0x5623b6c917f0 2023-10-05T19:40:24.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.735+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ed000a5a0 con 0x5623b6c917f0 2023-10-05T19:40:24.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.735+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ccdfb0 con 0x7f8ec802d020 2023-10-05T19:40:24.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.735+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ed000a5a0 con 0x7f8ec802d020 2023-10-05T19:40:24.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.735+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ccdfb0 con 0x7f8ec802d020 2023-10-05T19:40:24.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.735+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ed000a5a0 con 0x7f8ec802d020 2023-10-05T19:40:24.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.735+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ccdfb0 con 0x5623b6cc6d50 2023-10-05T19:40:24.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.735+0000 7f8ed521a700 1 -- 172.21.15.12:0/2964906848 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ec8014070 con 0x5623b6cc6d50 2023-10-05T19:40:24.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.739+0000 7f8ed521a700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ccdfb0 con 0x5623b6ccb4f0 2023-10-05T19:40:24.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.739+0000 7f8ecca19700 1 -- 172.21.15.12:0/2964906848 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ec0008ae0 con 0x5623b6ccb4f0 2023-10-05T19:40:24.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.739+0000 7f8ecca19700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ccdfb0 con 0x5623b6c917f0 2023-10-05T19:40:24.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.739+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ed000a5a0 con 0x5623b6c917f0 2023-10-05T19:40:24.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.739+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ccdfb0 con 0x7f8ec802d020 2023-10-05T19:40:24.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.739+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ed000a5a0 con 0x7f8ec802d020 2023-10-05T19:40:24.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.739+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ccdfb0 con 0x5623b6c917f0 2023-10-05T19:40:24.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.739+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ed000a5a0 con 0x5623b6c917f0 2023-10-05T19:40:24.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.739+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ccdfb0 con 0x5623b6cc6d50 2023-10-05T19:40:24.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.739+0000 7f8ed521a700 1 -- 172.21.15.12:0/2964906848 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ec8014070 con 0x5623b6cc6d50 2023-10-05T19:40:24.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.739+0000 7f8ed521a700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ccdfb0 con 0x5623b6ccb4f0 2023-10-05T19:40:24.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.743+0000 7f8ecca19700 1 -- 172.21.15.12:0/2964906848 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ec0008ae0 con 0x5623b6ccb4f0 2023-10-05T19:40:24.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.743+0000 7f8ecca19700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ccdfb0 con 0x5623b6ccb4f0 2023-10-05T19:40:24.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.743+0000 7f8ecca19700 1 -- 172.21.15.12:0/2964906848 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ec0008ae0 con 0x5623b6ccb4f0 2023-10-05T19:40:24.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.743+0000 7f8ecca19700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ccdfb0 con 0x7f8ec802d020 2023-10-05T19:40:24.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.743+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ed000a5a0 con 0x7f8ec802d020 2023-10-05T19:40:24.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.743+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ccdfb0 con 0x5623b6ccb4f0 2023-10-05T19:40:24.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.743+0000 7f8ecca19700 1 -- 172.21.15.12:0/2964906848 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ec0008ae0 con 0x5623b6ccb4f0 2023-10-05T19:40:24.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.743+0000 7f8ecca19700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ccdfb0 con 0x5623b6ccb4f0 2023-10-05T19:40:24.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.743+0000 7f8ecca19700 1 -- 172.21.15.12:0/2964906848 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ec0008ae0 con 0x5623b6ccb4f0 2023-10-05T19:40:24.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.743+0000 7f8ecca19700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ccdfb0 con 0x5623b6c917f0 2023-10-05T19:40:24.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.747+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ed000a5a0 con 0x5623b6c917f0 2023-10-05T19:40:24.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.747+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ccdfb0 con 0x5623b6c917f0 2023-10-05T19:40:24.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.747+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ed000a5a0 con 0x5623b6c917f0 2023-10-05T19:40:24.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.747+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ccdfb0 con 0x7f8ec802d020 2023-10-05T19:40:24.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.747+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ed000a5a0 con 0x7f8ec802d020 2023-10-05T19:40:24.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.747+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ccdfb0 con 0x5623b6ccb4f0 2023-10-05T19:40:24.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.747+0000 7f8ecca19700 1 -- 172.21.15.12:0/2964906848 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ec0008ae0 con 0x5623b6ccb4f0 2023-10-05T19:40:24.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.747+0000 7f8ecca19700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ccdfb0 con 0x5623b6cc6d50 2023-10-05T19:40:24.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.751+0000 7f8ed521a700 1 -- 172.21.15.12:0/2964906848 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ec8014070 con 0x5623b6cc6d50 2023-10-05T19:40:24.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.751+0000 7f8ed521a700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ccdfb0 con 0x5623b6c917f0 2023-10-05T19:40:24.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.751+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ed000a5a0 con 0x5623b6c917f0 2023-10-05T19:40:24.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.751+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ccdfb0 con 0x7f8ec802d020 2023-10-05T19:40:24.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.751+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ed000a5a0 con 0x7f8ec802d020 2023-10-05T19:40:24.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.751+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ccdfb0 con 0x5623b6ccb4f0 2023-10-05T19:40:24.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.751+0000 7f8ecca19700 1 -- 172.21.15.12:0/2964906848 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ec0008ae0 con 0x5623b6ccb4f0 2023-10-05T19:40:24.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.751+0000 7f8ecca19700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ccdfb0 con 0x7f8ec802d020 2023-10-05T19:40:24.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.751+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ed000a5a0 con 0x7f8ec802d020 2023-10-05T19:40:24.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.751+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ccdfb0 con 0x5623b6cc6d50 2023-10-05T19:40:24.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.755+0000 7f8ed521a700 1 -- 172.21.15.12:0/2964906848 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ec8014070 con 0x5623b6cc6d50 2023-10-05T19:40:24.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.755+0000 7f8ed521a700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ccdfb0 con 0x5623b6ccb4f0 2023-10-05T19:40:24.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.755+0000 7f8ecca19700 1 -- 172.21.15.12:0/2964906848 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ec0008ae0 con 0x5623b6ccb4f0 2023-10-05T19:40:24.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.755+0000 7f8ecca19700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ccdfb0 con 0x7f8ec802d020 2023-10-05T19:40:24.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.755+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ed000a5a0 con 0x7f8ec802d020 2023-10-05T19:40:24.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.755+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ccdfb0 con 0x5623b6c917f0 2023-10-05T19:40:24.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.755+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f8ed000a5a0 con 0x5623b6c917f0 2023-10-05T19:40:24.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.755+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ccdfb0 con 0x5623b6cc6d50 2023-10-05T19:40:24.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.755+0000 7f8ed521a700 1 -- 172.21.15.12:0/2964906848 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f8ec8014070 con 0x5623b6cc6d50 2023-10-05T19:40:24.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.759+0000 7f8ed76e5e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:24.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.759+0000 7f8ed76e5e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:24.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.759+0000 7f8ed76e5e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:24.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.759+0000 7f8ed76e5e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:24.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.759+0000 7f8ed76e5e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.759+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cca270 con 0x5623b6c917f0 2023-10-05T19:40:24.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.759+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8ed000a5a0 con 0x5623b6c917f0 2023-10-05T19:40:24.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.759+0000 7f8ed76e5e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:24.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.759+0000 7f8ed76e5e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.759+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cc9ea0 con 0x5623b6c917f0 2023-10-05T19:40:24.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.759+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f8ed000a5a0 con 0x5623b6c917f0 2023-10-05T19:40:24.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.759+0000 7f8ed76e5e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:24.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.759+0000 7f8ed76e5e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.759+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cce420 con 0x5623b6ccb4f0 2023-10-05T19:40:24.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.759+0000 7f8ecca19700 1 -- 172.21.15.12:0/2964906848 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f8ec0008ae0 con 0x5623b6ccb4f0 2023-10-05T19:40:24.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.759+0000 7f8ed76e5e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:24.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.759+0000 7f8ed76e5e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.759+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cce7d0 con 0x5623b6ccb4f0 2023-10-05T19:40:24.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.759+0000 7f8ecca19700 1 -- 172.21.15.12:0/2964906848 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f8ec0008ae0 con 0x5623b6ccb4f0 2023-10-05T19:40:24.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.759+0000 7f8ed76e5e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:24.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.759+0000 7f8ed76e5e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.759+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cceb80 con 0x5623b6c917f0 2023-10-05T19:40:24.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.759+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8ed000a5a0 con 0x5623b6c917f0 2023-10-05T19:40:24.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.759+0000 7f8ed76e5e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:24.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.759+0000 7f8ed76e5e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.759+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ccef90 con 0x5623b6c917f0 2023-10-05T19:40:24.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.763+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f8ed000a5a0 con 0x5623b6c917f0 2023-10-05T19:40:24.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.763+0000 7f8ed76e5e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:24.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.763+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ccef60 con 0x5623b6ccb4f0 2023-10-05T19:40:24.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.763+0000 7f8ecca19700 1 -- 172.21.15.12:0/2964906848 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f8ec0008ae0 con 0x5623b6ccb4f0 2023-10-05T19:40:24.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.763+0000 7f8ecca19700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ccef60 con 0x5623b6cc6d50 2023-10-05T19:40:24.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.767+0000 7f8ed521a700 1 -- 172.21.15.12:0/2964906848 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ec8014070 con 0x5623b6cc6d50 2023-10-05T19:40:24.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.767+0000 7f8ed521a700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ccef60 con 0x7f8ec802d020 2023-10-05T19:40:24.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.767+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ed000a5a0 con 0x7f8ec802d020 2023-10-05T19:40:24.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.767+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cceb80 con 0x5623b6c917f0 2023-10-05T19:40:24.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.767+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ed000a5a0 con 0x5623b6c917f0 2023-10-05T19:40:24.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.767+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cceb80 con 0x5623b6c917f0 2023-10-05T19:40:24.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.767+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ed000a5a0 con 0x5623b6c917f0 2023-10-05T19:40:24.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.767+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cceb80 con 0x7f8ec802d020 2023-10-05T19:40:24.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.767+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ed000a5a0 con 0x7f8ec802d020 2023-10-05T19:40:24.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.767+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cceb80 con 0x7f8ec802d020 2023-10-05T19:40:24.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.771+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ed000a5a0 con 0x7f8ec802d020 2023-10-05T19:40:24.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.771+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cceb80 con 0x5623b6cc6d50 2023-10-05T19:40:24.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.771+0000 7f8ed521a700 1 -- 172.21.15.12:0/2964906848 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ec8014070 con 0x5623b6cc6d50 2023-10-05T19:40:24.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.771+0000 7f8ed521a700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cceb80 con 0x5623b6ccb4f0 2023-10-05T19:40:24.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.771+0000 7f8ecca19700 1 -- 172.21.15.12:0/2964906848 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ec0008ae0 con 0x5623b6ccb4f0 2023-10-05T19:40:24.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.771+0000 7f8ecca19700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cceb80 con 0x5623b6c917f0 2023-10-05T19:40:24.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.771+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ed000a5a0 con 0x5623b6c917f0 2023-10-05T19:40:24.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.771+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cceb80 con 0x7f8ec802d020 2023-10-05T19:40:24.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.771+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ed000a5a0 con 0x7f8ec802d020 2023-10-05T19:40:24.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.771+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cceb80 con 0x5623b6c917f0 2023-10-05T19:40:24.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.775+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ed000a5a0 con 0x5623b6c917f0 2023-10-05T19:40:24.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.775+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cceb80 con 0x5623b6cc6d50 2023-10-05T19:40:24.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.775+0000 7f8ed521a700 1 -- 172.21.15.12:0/2964906848 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ec8014070 con 0x5623b6cc6d50 2023-10-05T19:40:24.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.775+0000 7f8ed521a700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cceb80 con 0x5623b6ccb4f0 2023-10-05T19:40:24.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.775+0000 7f8ecca19700 1 -- 172.21.15.12:0/2964906848 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ec0008ae0 con 0x5623b6ccb4f0 2023-10-05T19:40:24.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.775+0000 7f8ecca19700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cceb80 con 0x5623b6ccb4f0 2023-10-05T19:40:24.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.775+0000 7f8ecca19700 1 -- 172.21.15.12:0/2964906848 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ec0008ae0 con 0x5623b6ccb4f0 2023-10-05T19:40:24.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.775+0000 7f8ecca19700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cceb80 con 0x7f8ec802d020 2023-10-05T19:40:24.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.775+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ed000a5a0 con 0x7f8ec802d020 2023-10-05T19:40:24.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.775+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cceb80 con 0x5623b6ccb4f0 2023-10-05T19:40:24.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.779+0000 7f8ecca19700 1 -- 172.21.15.12:0/2964906848 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ec0008ae0 con 0x5623b6ccb4f0 2023-10-05T19:40:24.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.779+0000 7f8ecca19700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cceb80 con 0x5623b6ccb4f0 2023-10-05T19:40:24.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.779+0000 7f8ecca19700 1 -- 172.21.15.12:0/2964906848 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ec0008ae0 con 0x5623b6ccb4f0 2023-10-05T19:40:24.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.779+0000 7f8ecca19700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cceb80 con 0x5623b6c917f0 2023-10-05T19:40:24.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.779+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ed000a5a0 con 0x5623b6c917f0 2023-10-05T19:40:24.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.779+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cceb80 con 0x5623b6c917f0 2023-10-05T19:40:24.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.779+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ed000a5a0 con 0x5623b6c917f0 2023-10-05T19:40:24.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.779+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cceb80 con 0x7f8ec802d020 2023-10-05T19:40:24.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.779+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ed000a5a0 con 0x7f8ec802d020 2023-10-05T19:40:24.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.779+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cceb80 con 0x5623b6ccb4f0 2023-10-05T19:40:24.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.783+0000 7f8ecca19700 1 -- 172.21.15.12:0/2964906848 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ec0008ae0 con 0x5623b6ccb4f0 2023-10-05T19:40:24.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.783+0000 7f8ecca19700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cceb80 con 0x5623b6cc6d50 2023-10-05T19:40:24.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.783+0000 7f8ed521a700 1 -- 172.21.15.12:0/2964906848 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ec8014070 con 0x5623b6cc6d50 2023-10-05T19:40:24.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.783+0000 7f8ed521a700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cceb80 con 0x5623b6c917f0 2023-10-05T19:40:24.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.783+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ed000a5a0 con 0x5623b6c917f0 2023-10-05T19:40:24.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.783+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cceb80 con 0x7f8ec802d020 2023-10-05T19:40:24.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.783+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ed000a5a0 con 0x7f8ec802d020 2023-10-05T19:40:24.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.783+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cceb80 con 0x5623b6ccb4f0 2023-10-05T19:40:24.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.787+0000 7f8ecca19700 1 -- 172.21.15.12:0/2964906848 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ec0008ae0 con 0x5623b6ccb4f0 2023-10-05T19:40:24.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.787+0000 7f8ecca19700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cceb80 con 0x7f8ec802d020 2023-10-05T19:40:24.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.787+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ed000a5a0 con 0x7f8ec802d020 2023-10-05T19:40:24.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.787+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cceb80 con 0x5623b6cc6d50 2023-10-05T19:40:24.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.787+0000 7f8ed521a700 1 -- 172.21.15.12:0/2964906848 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ec8014070 con 0x5623b6cc6d50 2023-10-05T19:40:24.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.787+0000 7f8ed521a700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cceb80 con 0x5623b6ccb4f0 2023-10-05T19:40:24.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.787+0000 7f8ecca19700 1 -- 172.21.15.12:0/2964906848 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ec0008ae0 con 0x5623b6ccb4f0 2023-10-05T19:40:24.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.787+0000 7f8ecca19700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cceb80 con 0x7f8ec802d020 2023-10-05T19:40:24.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.787+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8ed000a5a0 con 0x7f8ec802d020 2023-10-05T19:40:24.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.787+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cceb80 con 0x5623b6c917f0 2023-10-05T19:40:24.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.791+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f8ed000a5a0 con 0x5623b6c917f0 2023-10-05T19:40:24.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.791+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cceb80 con 0x5623b6cc6d50 2023-10-05T19:40:24.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.791+0000 7f8ed521a700 1 -- 172.21.15.12:0/2964906848 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f8ec8014070 con 0x5623b6cc6d50 2023-10-05T19:40:24.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.791+0000 7f8ed76e5e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:24.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.791+0000 7f8ed76e5e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:24.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.791+0000 7f8ed76e5e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:24.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.791+0000 7f8ed76e5e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:24.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.791+0000 7f8ed76e5e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.791+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ccef60 con 0x5623b6c917f0 2023-10-05T19:40:24.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.791+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f8ed000a5a0 con 0x5623b6c917f0 2023-10-05T19:40:24.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.791+0000 7f8ed76e5e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:24.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.791+0000 7f8ed76e5e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.791+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ccf3c0 con 0x5623b6cc6d50 2023-10-05T19:40:24.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.791+0000 7f8ed521a700 1 -- 172.21.15.12:0/2964906848 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f8ec8014070 con 0x5623b6cc6d50 2023-10-05T19:40:24.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.791+0000 7f8ed76e5e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:24.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.791+0000 7f8ed76e5e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.791+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ccfcb0 con 0x5623b6c917f0 2023-10-05T19:40:24.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.795+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f8ed000a5a0 con 0x5623b6c917f0 2023-10-05T19:40:24.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.795+0000 7f8ed76e5e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:24.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.795+0000 7f8ed76e5e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.795+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ccfcb0 con 0x5623b6ccb4f0 2023-10-05T19:40:24.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.795+0000 7f8ecca19700 1 -- 172.21.15.12:0/2964906848 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f8ec0008ae0 con 0x5623b6ccb4f0 2023-10-05T19:40:24.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.795+0000 7f8ed76e5e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:24.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.795+0000 7f8ed76e5e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.795+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cd0420 con 0x7f8ec802d020 2023-10-05T19:40:24.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.795+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f8ed000a5a0 con 0x7f8ec802d020 2023-10-05T19:40:24.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.795+0000 7f8ed76e5e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:24.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.795+0000 7f8ed76e5e40 20 zone default found 2023-10-05T19:40:24.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.795+0000 7f8ed76e5e40 4 Realm: () 2023-10-05T19:40:24.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.795+0000 7f8ed76e5e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:24.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.795+0000 7f8ed76e5e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:24.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.795+0000 7f8ed76e5e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:24.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.795+0000 7f8ed76e5e40 20 zonegroup default 2023-10-05T19:40:24.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.795+0000 7f8ed76e5e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.795+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cd0190 con 0x7f8ec802d020 2023-10-05T19:40:24.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.795+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f8ed000a5a0 con 0x7f8ec802d020 2023-10-05T19:40:24.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.795+0000 7f8ed76e5e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:24.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.799+0000 7f8ed76e5e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.799+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cd1b10 con 0x5623b6ccb4f0 2023-10-05T19:40:24.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.799+0000 7f8ecca19700 1 -- 172.21.15.12:0/2964906848 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f8ec0008ae0 con 0x5623b6ccb4f0 2023-10-05T19:40:24.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.799+0000 7f8ed76e5e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:24.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.799+0000 7f8ed76e5e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.799+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cd1ec0 con 0x5623b6ccb4f0 2023-10-05T19:40:24.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.799+0000 7f8ecca19700 1 -- 172.21.15.12:0/2964906848 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f8ec0008ae0 con 0x5623b6ccb4f0 2023-10-05T19:40:24.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.799+0000 7f8ed76e5e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:24.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.799+0000 7f8ed76e5e40 20 started sync module instance, tier type = 2023-10-05T19:40:24.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.799+0000 7f8ed76e5e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:24.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.799+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cd11d0 con 0x5623b6cc6d50 2023-10-05T19:40:24.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.803+0000 7f8ed521a700 1 -- 172.21.15.12:0/2964906848 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'105 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8ec8014070 con 0x5623b6cc6d50 2023-10-05T19:40:24.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.803+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94711390737872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ccf8e0 con 0x5623b6cc6d50 2023-10-05T19:40:24.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.803+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cd2c50 con 0x5623b6c917f0 2023-10-05T19:40:24.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.807+0000 7f8ed521a700 1 -- 172.21.15.12:0/2964906848 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94711390737872] v28'106 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8ec8014070 con 0x5623b6cc6d50 2023-10-05T19:40:24.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.807+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'105 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8ed000a5a0 con 0x5623b6c917f0 2023-10-05T19:40:24.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.807+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94711390741392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cd2cd0 con 0x5623b6c917f0 2023-10-05T19:40:24.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.807+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cd3860 con 0x5623b6ccb4f0 2023-10-05T19:40:24.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.811+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94711390741392] v28'106 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8ed000a5a0 con 0x5623b6c917f0 2023-10-05T19:40:24.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.811+0000 7f8ecca19700 1 -- 172.21.15.12:0/2964906848 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'105 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8ec0008ae0 con 0x5623b6ccb4f0 2023-10-05T19:40:24.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.811+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94711390742496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cd3490 con 0x5623b6ccb4f0 2023-10-05T19:40:24.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.811+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cd4640 con 0x5623b6cc6d50 2023-10-05T19:40:24.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.811+0000 7f8ed521a700 1 -- 172.21.15.12:0/2964906848 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'105 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8ec8014070 con 0x5623b6cc6d50 2023-10-05T19:40:24.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.811+0000 7f8ecca19700 1 -- 172.21.15.12:0/2964906848 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94711390742496] v28'106 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8ec0008ae0 con 0x5623b6ccb4f0 2023-10-05T19:40:24.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.811+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94711390751216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cd4270 con 0x5623b6cc6d50 2023-10-05T19:40:24.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.811+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cd3c60 con 0x5623b6c917f0 2023-10-05T19:40:24.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.815+0000 7f8ed521a700 1 -- 172.21.15.12:0/2964906848 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94711390751216] v28'106 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8ec8014070 con 0x5623b6cc6d50 2023-10-05T19:40:24.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.815+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'105 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8ed000a5a0 con 0x5623b6c917f0 2023-10-05T19:40:24.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.815+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94711390752320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cd4270 con 0x5623b6c917f0 2023-10-05T19:40:24.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.815+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cd5f90 con 0x7f8ec802d020 2023-10-05T19:40:24.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.819+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'105 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8ed000a5a0 con 0x7f8ec802d020 2023-10-05T19:40:24.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.819+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94711390755488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cd63e0 con 0x7f8ec802d020 2023-10-05T19:40:24.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.819+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cd6e50 con 0x7f8ec802d020 2023-10-05T19:40:24.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.819+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94711390752320] v28'106 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8ed000a5a0 con 0x5623b6c917f0 2023-10-05T19:40:24.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.819+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(95 notify.5 [watch watch cookie 94711390755488] v28'106 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8ed000a5a0 con 0x7f8ec802d020 2023-10-05T19:40:24.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.823+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(96 notify.6 [create] v28'105 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8ed000a5a0 con 0x7f8ec802d020 2023-10-05T19:40:24.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.823+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94711390756592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cd6340 con 0x7f8ec802d020 2023-10-05T19:40:24.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.823+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cd7340 con 0x7f8ec802d020 2023-10-05T19:40:24.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.823+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(97 notify.6 [watch watch cookie 94711390756592] v28'106 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8ed000a5a0 con 0x7f8ec802d020 2023-10-05T19:40:24.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.823+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(98 notify.7 [create] v28'105 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8ed000a5a0 con 0x7f8ec802d020 2023-10-05T19:40:24.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.823+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94711390759264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cd72a0 con 0x7f8ec802d020 2023-10-05T19:40:24.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.823+0000 7f8ed76e5e40 20 add_watcher() i=0 2023-10-05T19:40:24.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.823+0000 7f8ed76e5e40 20 add_watcher() i=1 2023-10-05T19:40:24.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.823+0000 7f8ed76e5e40 20 add_watcher() i=2 2023-10-05T19:40:24.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.823+0000 7f8ed76e5e40 20 add_watcher() i=3 2023-10-05T19:40:24.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.823+0000 7f8ed76e5e40 20 add_watcher() i=4 2023-10-05T19:40:24.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.823+0000 7f8ed76e5e40 20 add_watcher() i=5 2023-10-05T19:40:24.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.823+0000 7f8ed76e5e40 20 add_watcher() i=6 2023-10-05T19:40:24.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.827+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94711390759264] v28'106 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8ed000a5a0 con 0x7f8ec802d020 2023-10-05T19:40:24.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.827+0000 7f8ed76e5e40 20 add_watcher() i=7 2023-10-05T19:40:24.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.827+0000 7f8ed76e5e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:24.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.827+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cda4a0 con 0x5623b6c917f0 2023-10-05T19:40:24.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.827+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f8ed000a5a0 con 0x5623b6c917f0 2023-10-05T19:40:24.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.827+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94711390760736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6cd7860 con 0x5623b6c917f0 2023-10-05T19:40:24.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.831+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94711390760736] v28'105 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f8ed000a5a0 con 0x5623b6c917f0 2023-10-05T19:40:24.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.831+0000 7f8ea67e4700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:24.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.831+0000 7f8ed76e5e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:24.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.831+0000 7f8ed76e5e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:24.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.831+0000 7f8ed76e5e40 1 Processor -- start 2023-10-05T19:40:24.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.831+0000 7f8ed76e5e40 1 -- start start 2023-10-05T19:40:24.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.831+0000 7f8ed76e5e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623b6ce4cd0 0x5623b6ce50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:24.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.831+0000 7f8ed76e5e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623b6ce55e0 con 0x5623b6ce4cd0 2023-10-05T19:40:24.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.831+0000 7f8ecca19700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623b6ce4cd0 0x5623b6ce50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.831+0000 7f8ecca19700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623b6ce4cd0 0x5623b6ce50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57804/0 (socket says 172.21.15.12:57804) 2023-10-05T19:40:24.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.831+0000 7f8ecca19700 1 -- 172.21.15.12:0/1872741371 learned_addr learned my addr 172.21.15.12:0/1872741371 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:24.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.831+0000 7f8ecca19700 1 -- 172.21.15.12:0/1872741371 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623b6ce5720 con 0x5623b6ce4cd0 2023-10-05T19:40:24.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.831+0000 7f8ecca19700 1 --2- 172.21.15.12:0/1872741371 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623b6ce4cd0 0x5623b6ce50a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x5623b6c90cb0 tx=0x7f8ec0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d5fd5f1482f2e90 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.831+0000 7f8ea5fe3700 1 -- 172.21.15.12:0/1872741371 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ec0030930 con 0x5623b6ce4cd0 2023-10-05T19:40:24.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.831+0000 7f8ea5fe3700 1 -- 172.21.15.12:0/1872741371 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec0030ad0 con 0x5623b6ce4cd0 2023-10-05T19:40:24.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.831+0000 7f8ea5fe3700 1 -- 172.21.15.12:0/1872741371 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ec0030cc0 con 0x5623b6ce4cd0 2023-10-05T19:40:24.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.835+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/1872741371 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623b6ce4cd0 msgr2=0x5623b6ce50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.835+0000 7f8ed76e5e40 1 --2- 172.21.15.12:0/1872741371 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623b6ce4cd0 0x5623b6ce50a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x5623b6c90cb0 tx=0x7f8ec0012000 comp rx=0 tx=0).stop 2023-10-05T19:40:24.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.835+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/1872741371 shutdown_connections 2023-10-05T19:40:24.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.835+0000 7f8ed76e5e40 1 --2- 172.21.15.12:0/1872741371 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623b6ce4cd0 0x5623b6ce50a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:24.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.835+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/1872741371 >> 172.21.15.12:0/1872741371 conn(0x5623b6ce0090 msgr2=0x5623b6cde1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:24.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.835+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/1872741371 shutdown_connections 2023-10-05T19:40:24.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.835+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/1872741371 wait complete. 2023-10-05T19:40:24.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.835+0000 7f8ed76e5e40 1 Processor -- start 2023-10-05T19:40:24.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.835+0000 7f8ed76e5e40 1 -- start start 2023-10-05T19:40:24.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.835+0000 7f8ed76e5e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623b6ce4cd0 0x5623b6cea7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:24.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.835+0000 7f8ed76e5e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623b6ceb7c0 con 0x5623b6ce4cd0 2023-10-05T19:40:24.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.835+0000 7f8ecca19700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623b6ce4cd0 0x5623b6cea7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.835+0000 7f8ecca19700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623b6ce4cd0 0x5623b6cea7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57812/0 (socket says 172.21.15.12:57812) 2023-10-05T19:40:24.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.835+0000 7f8ecca19700 1 -- 172.21.15.12:0/1871918138 learned_addr learned my addr 172.21.15.12:0/1871918138 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:24.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.835+0000 7f8ecca19700 1 -- 172.21.15.12:0/1871918138 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623b6c90ce0 con 0x5623b6ce4cd0 2023-10-05T19:40:24.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.835+0000 7f8ecca19700 1 --2- 172.21.15.12:0/1871918138 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623b6ce4cd0 0x5623b6cea7c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8ec003b000 tx=0x7f8ec0034000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.835+0000 7f8e17fff700 1 -- 172.21.15.12:0/1871918138 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ec0027400 con 0x5623b6ce4cd0 2023-10-05T19:40:24.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.835+0000 7f8e17fff700 1 -- 172.21.15.12:0/1871918138 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec0027560 con 0x5623b6ce4cd0 2023-10-05T19:40:24.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.835+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/1871918138 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5623b6ceadc0 con 0x5623b6ce4cd0 2023-10-05T19:40:24.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.835+0000 7f8e17fff700 1 -- 172.21.15.12:0/1871918138 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ec00276f0 con 0x5623b6ce4cd0 2023-10-05T19:40:24.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.839+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/1871918138 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5623b6ceb1a0 con 0x5623b6ce4cd0 2023-10-05T19:40:24.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.839+0000 7f8e17fff700 1 -- 172.21.15.12:0/1871918138 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f8ec0027920 con 0x5623b6ce4cd0 2023-10-05T19:40:24.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.839+0000 7f8e17fff700 1 --2- 172.21.15.12:0/1871918138 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f8e1c030c00 0x7f8e1c0330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:24.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.839+0000 7f8e17fff700 1 -- 172.21.15.12:0/1871918138 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f8ec0054ad0 con 0x5623b6ce4cd0 2023-10-05T19:40:24.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.839+0000 7f8ed76e5e40 5 note: GC not initialized 2023-10-05T19:40:24.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.839+0000 7f8ed5a1b700 1 --2- 172.21.15.12:0/1871918138 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f8e1c030c00 0x7f8e1c0330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.839+0000 7f8ed5a1b700 1 --2- 172.21.15.12:0/1871918138 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f8e1c030c00 0x7f8e1c0330b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x5623b6cd6540 tx=0x7f8ed000ae90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.843+0000 7f8e157fa700 20 reqs_thread_entry: start 2023-10-05T19:40:24.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.843+0000 7f8e0effd700 20 reqs_thread_entry: start 2023-10-05T19:40:24.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.843+0000 7f8ed76e5e40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:24.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.843+0000 7f8ed76e5e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:24.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.843+0000 7f8e0d7fa700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:55 2023 2023-10-05T19:40:24.852 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:24.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.843+0000 7f8e0cff9700 20 reqs_thread_entry: start 2023-10-05T19:40:24.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.843+0000 7f8ed76e5e40 20 get_system_obj_state: rctx=0x7fffac638b18 obj=default.rgw.meta:users.uid:foo state=0x5623b6b6a0b0 s->prefetch_data=0 2023-10-05T19:40:24.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.843+0000 7f8ed76e5e40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2023-10-05T19:40:24.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.843+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6d52fc0 con 0x5623b6c917f0 2023-10-05T19:40:24.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.843+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(102 foo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7f8ed000a5a0 con 0x5623b6c917f0 2023-10-05T19:40:24.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.843+0000 7f8ed76e5e40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x16 2023-10-05T19:40:24.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.843+0000 7f8ed76e5e40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:24.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.843+0000 7f8ed76e5e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:24.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.843+0000 7f8ed76e5e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:24.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.843+0000 7f8ed76e5e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:24.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.843+0000 7f8ed76e5e40 10 cache get: name=default.rgw.meta+users.uid+foo : type miss (requested=0x13, cached=0x16) 2023-10-05T19:40:24.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.843+0000 7f8ed76e5e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.843+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6d55a50 con 0x5623b6c917f0 2023-10-05T19:40:24.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.843+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(103 foo [call,call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv11 ondisk = 0) v8 ==== 273+0+480 (crc 0 0 0) 0x7f8ed000a5a0 con 0x5623b6c917f0 2023-10-05T19:40:24.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.843+0000 7f8ed76e5e40 20 rados_obj.operate() r=0 bl.length=357 2023-10-05T19:40:24.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.843+0000 7f8ed76e5e40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2023-10-05T19:40:24.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.843+0000 7f8ed76e5e40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:24.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.843+0000 7f8ed76e5e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:24.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.847+0000 7f8ed76e5e40 20 get_system_obj_state: rctx=0x7fffac638b60 obj=default.rgw.meta:root:myfoo state=0x5623b6b6a0b0 s->prefetch_data=0 2023-10-05T19:40:24.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.847+0000 7f8ed76e5e40 10 cache get: name=default.rgw.meta+root+myfoo : miss 2023-10-05T19:40:24.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.847+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:104 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6d55260 con 0x5623b6cc6d50 2023-10-05T19:40:24.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.847+0000 7f8ed521a700 1 -- 172.21.15.12:0/2964906848 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(104 myfoo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 233+0+139 (crc 0 0 0) 0x7f8ec8014070 con 0x5623b6cc6d50 2023-10-05T19:40:24.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.847+0000 7f8ed76e5e40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x16 2023-10-05T19:40:24.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.847+0000 7f8ed76e5e40 10 adding default.rgw.meta+root+myfoo to cache LRU end 2023-10-05T19:40:24.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.847+0000 7f8ed76e5e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:24.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.847+0000 7f8ed76e5e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:24.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.847+0000 7f8ed76e5e40 10 cache get: name=default.rgw.meta+root+myfoo : type miss (requested=0x11, cached=0x16) 2023-10-05T19:40:24.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.847+0000 7f8ed76e5e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.847+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:105 8.9 8:91b71841:root::myfoo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6d57800 con 0x5623b6cc6d50 2023-10-05T19:40:24.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.847+0000 7f8ed521a700 1 -- 172.21.15.12:0/2964906848 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(105 myfoo [call,call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) v8 ==== 233+0+213 (crc 0 0 0) 0x7f8ec8014070 con 0x5623b6cc6d50 2023-10-05T19:40:24.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.847+0000 7f8ed76e5e40 20 rados_obj.operate() r=0 bl.length=165 2023-10-05T19:40:24.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.847+0000 7f8ed76e5e40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2023-10-05T19:40:24.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.847+0000 7f8ed76e5e40 10 moving default.rgw.meta+root+myfoo to cache LRU end 2023-10-05T19:40:24.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.847+0000 7f8ed76e5e40 20 get_system_obj_state: rctx=0x7fffac638b60 obj=default.rgw.meta:root:.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 state=0x5623b6b402b0 s->prefetch_data=0 2023-10-05T19:40:24.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.847+0000 7f8ed76e5e40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 : miss 2023-10-05T19:40:24.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.847+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:106 8.e 8:70176bbd:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6d56ce0 con 0x7f8ec802d020 2023-10-05T19:40:24.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.847+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(106 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 [call out=48b,getxattrs out=218b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 290+0+282 (crc 0 0 0) 0x7f8ed000a5a0 con 0x7f8ec802d020 2023-10-05T19:40:24.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.847+0000 7f8ed76e5e40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 info.flags=0x16 2023-10-05T19:40:24.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.847+0000 7f8ed76e5e40 10 adding default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 to cache LRU end 2023-10-05T19:40:24.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.847+0000 7f8ed76e5e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:24.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.847+0000 7f8ed76e5e40 10 updating xattr: name=user.rgw.acl bl.length()=123 2023-10-05T19:40:24.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.847+0000 7f8ed76e5e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:24.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.847+0000 7f8ed76e5e40 20 Read xattr: user.rgw.acl 2023-10-05T19:40:24.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.847+0000 7f8ed76e5e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:24.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.847+0000 7f8ed76e5e40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 : type miss (requested=0x13, cached=0x16) 2023-10-05T19:40:24.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.847+0000 7f8ed76e5e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.847+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:107 8.e 8:70176bbd:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6d59da0 con 0x7f8ec802d020 2023-10-05T19:40:24.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.847+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(107 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 [call,call out=48b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 332+0+618 (crc 0 0 0) 0x7f8ed000a5a0 con 0x7f8ec802d020 2023-10-05T19:40:24.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.851+0000 7f8ed76e5e40 20 rados_obj.operate() r=0 bl.length=352 2023-10-05T19:40:24.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.851+0000 7f8ed76e5e40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 info.flags=0x13 2023-10-05T19:40:24.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.851+0000 7f8ed76e5e40 10 moving default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 to cache LRU end 2023-10-05T19:40:24.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.851+0000 7f8ed76e5e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:24.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.851+0000 7f8ed76e5e40 10 updating xattr: name=user.rgw.acl bl.length()=123 2023-10-05T19:40:24.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.851+0000 7f8ed76e5e40 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 2023-10-05T19:40:24.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.851+0000 7f8ed76e5e40 20 get_system_obj_state: rctx=0x7fffac638af8 obj=default.rgw.meta:users.uid:foo state=0x5623b6b402b0 s->prefetch_data=0 2023-10-05T19:40:24.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.851+0000 7f8ed76e5e40 10 cache get: name=default.rgw.meta+users.uid+foo : type miss (requested=0x16, cached=0x13) 2023-10-05T19:40:24.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.851+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:108 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6d5a1e0 con 0x5623b6c917f0 2023-10-05T19:40:24.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.851+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(108 foo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7f8ed000a5a0 con 0x5623b6c917f0 2023-10-05T19:40:24.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.851+0000 7f8ed76e5e40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x16 2023-10-05T19:40:24.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.851+0000 7f8ed76e5e40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:24.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.851+0000 7f8ed76e5e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:24.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.851+0000 7f8ed76e5e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:24.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.851+0000 7f8ed76e5e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:24.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.851+0000 7f8ed76e5e40 10 cache get: name=default.rgw.meta+users.uid+foo : hit (requested=0x13, cached=0x17) 2023-10-05T19:40:24.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.851+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:109 8.9 8:96d98480:users.uid::foo.buckets:head [call user.remove_bucket in=58b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6d5b1f0 con 0x5623b6cc6d50 2023-10-05T19:40:24.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.851+0000 7f8ed521a700 1 -- 172.21.15.12:0/2964906848 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(109 foo.buckets [call] v28'17 uv17 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x7f8ec8014070 con 0x5623b6cc6d50 2023-10-05T19:40:24.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.851+0000 7f8ed76e5e40 20 get_system_obj_state: rctx=0x7fffac638c38 obj=default.rgw.meta:root:myfoo state=0x5623b6b402b0 s->prefetch_data=0 2023-10-05T19:40:24.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.851+0000 7f8ed76e5e40 10 cache get: name=default.rgw.meta+root+myfoo : type miss (requested=0x16, cached=0x13) 2023-10-05T19:40:24.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.851+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:110 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6d5a1e0 con 0x5623b6cc6d50 2023-10-05T19:40:24.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.855+0000 7f8ed521a700 1 -- 172.21.15.12:0/2964906848 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(110 myfoo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 233+0+139 (crc 0 0 0) 0x7f8ec8014070 con 0x5623b6cc6d50 2023-10-05T19:40:24.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.855+0000 7f8ed76e5e40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x16 2023-10-05T19:40:24.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.855+0000 7f8ed76e5e40 10 moving default.rgw.meta+root+myfoo to cache LRU end 2023-10-05T19:40:24.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.855+0000 7f8ed76e5e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:24.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.855+0000 7f8ed76e5e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:24.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.855+0000 7f8ed76e5e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:24.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.855+0000 7f8ed76e5e40 10 cache get: name=default.rgw.meta+root+myfoo : hit (requested=0x13, cached=0x17) 2023-10-05T19:40:24.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.855+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:111 8.9 8:91b71841:root::myfoo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~165 in=165b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6d5eab0 con 0x5623b6cc6d50 2023-10-05T19:40:24.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.855+0000 7f8ed521a700 1 -- 172.21.15.12:0/2964906848 <== osd.2 v2:172.21.15.12:6800/29934 23 ==== osd_op_reply(111 myfoo [delete,create,call,call,writefull 0~165] v28'18 uv18 ondisk = 0) v8 ==== 317+0+0 (crc 0 0 0) 0x7f8ec8014070 con 0x5623b6cc6d50 2023-10-05T19:40:24.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.855+0000 7f8ed76e5e40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x17 2023-10-05T19:40:24.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.855+0000 7f8ed76e5e40 10 moving default.rgw.meta+root+myfoo to cache LRU end 2023-10-05T19:40:24.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.855+0000 7f8ed76e5e40 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2023-10-05T19:40:24.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.855+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:112 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94711391315392 in=350b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6d5b6e0 con 0x7f8ec802d020 2023-10-05T19:40:24.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.859+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== watch-notify(notify (1) cookie 94711390756592 notify 120259084301 ret 0) v3 ==== 380+0+0 (crc 0 0 0) 0x7f8ed002d040 con 0x7f8ec802d020 2023-10-05T19:40:24.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.859+0000 7f8ecf7fe700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084301 cookie 94711390756592 notifier 4435 bl.length()=338 2023-10-05T19:40:24.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.859+0000 7f8ecf7fe700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo info.flags=0x17 2023-10-05T19:40:24.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.859+0000 7f8ecf7fe700 10 rgw watcher librados: moving default.rgw.meta+root+myfoo to cache LRU end 2023-10-05T19:40:24.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.859+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.0 v2:172.21.15.8:6800/27563 30 ==== osd_op_reply(112 notify.6 [notify cookie 94711391315392 out=8b] v0'0 uv105 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f8ed000a5a0 con 0x7f8ec802d020 2023-10-05T19:40:24.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.859+0000 7f8ecf7fe700 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:113 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f8eb4003d90 con 0x7f8ec802d020 2023-10-05T19:40:24.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.859+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.0 v2:172.21.15.8:6800/27563 31 ==== osd_op_reply(113 notify.6 [notify-ack] v0'0 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8ed000a5a0 con 0x7f8ec802d020 2023-10-05T19:40:24.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.859+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.0 v2:172.21.15.8:6800/27563 32 ==== watch-notify(notify_complete (2) cookie 94711391315392 notify 120259084301 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f8ed0037d80 con 0x7f8ec802d020 2023-10-05T19:40:24.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.859+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:114 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94711390737872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ca88f0 con 0x5623b6cc6d50 2023-10-05T19:40:24.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.863+0000 7f8ed521a700 1 -- 172.21.15.12:0/2964906848 <== osd.2 v2:172.21.15.12:6800/29934 24 ==== osd_op_reply(114 notify.0 [watch unwatch cookie 94711390737872] v28'107 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8ec8014070 con 0x5623b6cc6d50 2023-10-05T19:40:24.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.863+0000 7f8ed76e5e40 20 remove_watcher() i=0 2023-10-05T19:40:24.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.863+0000 7f8ed76e5e40 2 removed watcher, disabling cache 2023-10-05T19:40:24.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.863+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:115 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94711390741392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ca88f0 con 0x5623b6c917f0 2023-10-05T19:40:24.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.863+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(115 notify.1 [watch unwatch cookie 94711390741392] v28'107 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8ed000a5a0 con 0x5623b6c917f0 2023-10-05T19:40:24.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.863+0000 7f8ed76e5e40 20 remove_watcher() i=1 2023-10-05T19:40:24.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.863+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:116 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94711390742496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ca88f0 con 0x5623b6ccb4f0 2023-10-05T19:40:24.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.867+0000 7f8ecca19700 1 -- 172.21.15.12:0/2964906848 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(116 notify.2 [watch unwatch cookie 94711390742496] v28'107 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8ec0008ae0 con 0x5623b6ccb4f0 2023-10-05T19:40:24.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.867+0000 7f8ed76e5e40 20 remove_watcher() i=2 2023-10-05T19:40:24.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.867+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:117 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94711390751216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ca88f0 con 0x5623b6cc6d50 2023-10-05T19:40:24.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.871+0000 7f8ed521a700 1 -- 172.21.15.12:0/2964906848 <== osd.2 v2:172.21.15.12:6800/29934 25 ==== osd_op_reply(117 notify.3 [watch unwatch cookie 94711390751216] v28'107 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8ec8014070 con 0x5623b6cc6d50 2023-10-05T19:40:24.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.871+0000 7f8ed76e5e40 20 remove_watcher() i=3 2023-10-05T19:40:24.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.871+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:118 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94711390752320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ca88f0 con 0x5623b6c917f0 2023-10-05T19:40:24.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.871+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(118 notify.4 [watch unwatch cookie 94711390752320] v28'107 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8ed000a5a0 con 0x5623b6c917f0 2023-10-05T19:40:24.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.871+0000 7f8ed76e5e40 20 remove_watcher() i=4 2023-10-05T19:40:24.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.871+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:119 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94711390755488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ca88f0 con 0x7f8ec802d020 2023-10-05T19:40:24.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.875+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.0 v2:172.21.15.8:6800/27563 33 ==== osd_op_reply(119 notify.5 [watch unwatch cookie 94711390755488] v28'107 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8ed000a5a0 con 0x7f8ec802d020 2023-10-05T19:40:24.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.875+0000 7f8ed76e5e40 20 remove_watcher() i=5 2023-10-05T19:40:24.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.875+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:120 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94711390756592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ca88f0 con 0x7f8ec802d020 2023-10-05T19:40:24.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.879+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.0 v2:172.21.15.8:6800/27563 34 ==== osd_op_reply(120 notify.6 [watch unwatch cookie 94711390756592] v28'107 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8ed000a5a0 con 0x7f8ec802d020 2023-10-05T19:40:24.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.879+0000 7f8ed76e5e40 20 remove_watcher() i=6 2023-10-05T19:40:24.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.879+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:121 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94711390759264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ca88f0 con 0x7f8ec802d020 2023-10-05T19:40:24.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.883+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.0 v2:172.21.15.8:6800/27563 35 ==== osd_op_reply(121 notify.7 [watch unwatch cookie 94711390759264] v28'107 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8ed000a5a0 con 0x7f8ec802d020 2023-10-05T19:40:24.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.883+0000 7f8ed76e5e40 20 remove_watcher() i=7 2023-10-05T19:40:24.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.883+0000 7f8ea67e4700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:24.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.883+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:122 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94711390760736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623b6ca88f0 con 0x5623b6c917f0 2023-10-05T19:40:24.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.887+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== osd_op_reply(122 data_loggenerations_metadata [watch unwatch cookie 94711390760736] v28'106 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f8ed000a5a0 con 0x5623b6c917f0 2023-10-05T19:40:24.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.887+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f8ec802d020 msgr2=0x7f8ec802c5d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.887+0000 7f8ed76e5e40 1 --2- 172.21.15.12:0/2964906848 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f8ec802d020 0x7f8ec802c5d0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:24.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.887+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5623b6c917f0 msgr2=0x5623b6cc4820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.887+0000 7f8ed76e5e40 1 --2- 172.21.15.12:0/2964906848 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5623b6c917f0 0x5623b6cc4820 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:24.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.887+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5623b6cc6d50 msgr2=0x5623b6cc9130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.887+0000 7f8ed76e5e40 1 --2- 172.21.15.12:0/2964906848 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5623b6cc6d50 0x5623b6cc9130 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:24.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.887+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5623b6ccb4f0 msgr2=0x5623b6ccd900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.887+0000 7f8ed76e5e40 1 --2- 172.21.15.12:0/2964906848 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5623b6ccb4f0 0x5623b6ccd900 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:24.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.887+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f8eb8030c50 msgr2=0x7f8eb8033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.887+0000 7f8ed76e5e40 1 --2- 172.21.15.12:0/2964906848 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f8eb8030c50 0x7f8eb8033100 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0005a90 tx=0x7f8ec0015000 comp rx=0 tx=0).stop 2023-10-05T19:40:24.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.887+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623b6c91420 msgr2=0x5623b6cb8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.887+0000 7f8ed5a1b700 1 -- 172.21.15.12:0/2964906848 reap_dead start 2023-10-05T19:40:24.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.887+0000 7f8ed76e5e40 1 --2- 172.21.15.12:0/2964906848 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623b6c91420 0x5623b6cb8d60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8ec80017d0 tx=0x7f8ec8016000 comp rx=0 tx=0).stop 2023-10-05T19:40:24.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.887+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 shutdown_connections 2023-10-05T19:40:24.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.887+0000 7f8ed76e5e40 1 --2- 172.21.15.12:0/2964906848 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623b6c91420 0x5623b6cb8d60 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:24.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.887+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 >> 172.21.15.12:0/2964906848 conn(0x5623b6c8b730 msgr2=0x5623b6cac610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:24.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.887+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 shutdown_connections 2023-10-05T19:40:24.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.887+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/2964906848 wait complete. 2023-10-05T19:40:24.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.887+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/1871918138 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f8e1c030c00 msgr2=0x7f8e1c0330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.887+0000 7f8ed76e5e40 1 --2- 172.21.15.12:0/1871918138 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f8e1c030c00 0x7f8e1c0330b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x5623b6cd6540 tx=0x7f8ed000ae90 comp rx=0 tx=0).stop 2023-10-05T19:40:24.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.887+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/1871918138 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623b6ce4cd0 msgr2=0x5623b6cea7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.887+0000 7f8ed76e5e40 1 --2- 172.21.15.12:0/1871918138 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623b6ce4cd0 0x5623b6cea7c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8ec003b000 tx=0x7f8ec0034000 comp rx=0 tx=0).stop 2023-10-05T19:40:24.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.887+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/1871918138 shutdown_connections 2023-10-05T19:40:24.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.887+0000 7f8ed76e5e40 1 --2- 172.21.15.12:0/1871918138 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f8e1c030c00 0x7f8e1c0330b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:24.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.887+0000 7f8ed76e5e40 1 --2- 172.21.15.12:0/1871918138 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623b6ce4cd0 0x5623b6cea7c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:24.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.887+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/1871918138 >> 172.21.15.12:0/1871918138 conn(0x5623b6ce0090 msgr2=0x5623b6cde130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:24.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.887+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/1871918138 shutdown_connections 2023-10-05T19:40:24.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.887+0000 7f8ed76e5e40 1 -- 172.21.15.12:0/1871918138 wait complete. 2023-10-05T19:40:24.903 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '1'] 2023-10-05T19:40:24.903 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '1'] 2023-10-05T19:40:24.903 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid fud --display-name Fud --access-key p5YnriCv1nAtykxBrupQ --secret Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh --max-buckets 1 2023-10-05T19:40:24.952 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:24.952 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:24.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.943+0000 7fd4657fde40 1 Processor -- start 2023-10-05T19:40:24.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.943+0000 7fd4657fde40 1 -- start start 2023-10-05T19:40:24.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.943+0000 7fd4657fde40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x5555dc863e70 0x5555dc864240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:24.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.943+0000 7fd4657fde40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5555dc7b0bd0 con 0x5555dc864780 2023-10-05T19:40:24.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.943+0000 7fd4657fde40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x5555dc76dee0 con 0x5555dc863e70 2023-10-05T19:40:24.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.943+0000 7fd463332700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x5555dc863e70 0x5555dc864240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.943+0000 7fd462b31700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x5555dc864780 0x5555dc865220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:35524/0 (socket says 172.21.15.12:35524) 2023-10-05T19:40:24.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.943+0000 7fd462b31700 1 -- 172.21.15.12:0/2998715670 learned_addr learned my addr 172.21.15.12:0/2998715670 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:24.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.947+0000 7fd462330700 1 -- 172.21.15.12:0/2998715670 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 74866419 0 0) 0x5555dc7b0bd0 con 0x5555dc864780 2023-10-05T19:40:24.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.947+0000 7fd462330700 1 -- 172.21.15.12:0/2998715670 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd444001e10 con 0x5555dc864780 2023-10-05T19:40:24.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.947+0000 7fd463332700 1 -- 172.21.15.12:0/2998715670 >> v1:172.21.15.8:6789/0 conn(0x5555dc864780 legacy=0x5555dc865220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.947+0000 7fd463332700 1 -- 172.21.15.12:0/2998715670 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555dc865930 con 0x5555dc863e70 2023-10-05T19:40:24.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.947+0000 7fd463332700 1 --2- 172.21.15.12:0/2998715670 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5555dc863e70 0x5555dc864240 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd44c00a5d0 tx=0x7fd44c005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=215b0c8dbbbeaeea server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.947+0000 7fd462330700 1 -- 172.21.15.12:0/2998715670 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd44c007610 con 0x5555dc863e70 2023-10-05T19:40:24.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.947+0000 7fd462330700 1 -- 172.21.15.12:0/2998715670 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd44c007770 con 0x5555dc863e70 2023-10-05T19:40:24.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.947+0000 7fd462330700 1 -- 172.21.15.12:0/2998715670 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd44c007a40 con 0x5555dc863e70 2023-10-05T19:40:24.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.947+0000 7fd4657fde40 1 -- 172.21.15.12:0/2998715670 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5555dc863e70 msgr2=0x5555dc864240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.947+0000 7fd4657fde40 1 --2- 172.21.15.12:0/2998715670 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5555dc863e70 0x5555dc864240 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd44c00a5d0 tx=0x7fd44c005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:24.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.947+0000 7fd4657fde40 1 -- 172.21.15.12:0/2998715670 shutdown_connections 2023-10-05T19:40:24.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.947+0000 7fd4657fde40 1 --2- 172.21.15.12:0/2998715670 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5555dc863e70 0x5555dc864240 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:24.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.947+0000 7fd4657fde40 1 -- 172.21.15.12:0/2998715670 >> 172.21.15.12:0/2998715670 conn(0x5555dc85f890 msgr2=0x5555dc861ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:24.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.947+0000 7fd4657fde40 1 -- 172.21.15.12:0/2998715670 shutdown_connections 2023-10-05T19:40:24.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.947+0000 7fd4657fde40 1 -- 172.21.15.12:0/2998715670 wait complete. 2023-10-05T19:40:24.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.947+0000 7fd4657fde40 1 Processor -- start 2023-10-05T19:40:24.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.947+0000 7fd4657fde40 1 -- start start 2023-10-05T19:40:24.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.947+0000 7fd4657fde40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5555dc863e70 0x5555dc882df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:24.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.947+0000 7fd4657fde40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555dc731e30 con 0x5555dc863e70 2023-10-05T19:40:24.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.947+0000 7fd463332700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5555dc863e70 0x5555dc882df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.947+0000 7fd463332700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5555dc863e70 0x5555dc882df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57834/0 (socket says 172.21.15.12:57834) 2023-10-05T19:40:24.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.947+0000 7fd463332700 1 -- 172.21.15.12:0/4154051714 learned_addr learned my addr 172.21.15.12:0/4154051714 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:24.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.951+0000 7fd463332700 1 -- 172.21.15.12:0/4154051714 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd44c005cf0 con 0x5555dc863e70 2023-10-05T19:40:24.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.951+0000 7fd463332700 1 --2- 172.21.15.12:0/4154051714 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5555dc863e70 0x5555dc882df0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd44c000c00 tx=0x7fd44c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4077dc58f86843ce server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.951+0000 7fd46132e700 1 -- 172.21.15.12:0/4154051714 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd44c0078e0 con 0x5555dc863e70 2023-10-05T19:40:24.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.951+0000 7fd46132e700 1 -- 172.21.15.12:0/4154051714 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd44c007a40 con 0x5555dc863e70 2023-10-05T19:40:24.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.951+0000 7fd46132e700 1 -- 172.21.15.12:0/4154051714 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd44c007bd0 con 0x5555dc863e70 2023-10-05T19:40:24.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.951+0000 7fd4657fde40 1 -- 172.21.15.12:0/4154051714 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5555dc863e70 msgr2=0x5555dc882df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:24.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.951+0000 7fd4657fde40 1 --2- 172.21.15.12:0/4154051714 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5555dc863e70 0x5555dc882df0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd44c000c00 tx=0x7fd44c014000 comp rx=0 tx=0).stop 2023-10-05T19:40:24.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.951+0000 7fd4657fde40 1 -- 172.21.15.12:0/4154051714 shutdown_connections 2023-10-05T19:40:24.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.951+0000 7fd4657fde40 1 --2- 172.21.15.12:0/4154051714 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5555dc863e70 0x5555dc882df0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:24.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.951+0000 7fd4657fde40 1 -- 172.21.15.12:0/4154051714 >> 172.21.15.12:0/4154051714 conn(0x5555dc85f890 msgr2=0x5555dc87e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:24.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.951+0000 7fd4657fde40 1 -- 172.21.15.12:0/4154051714 shutdown_connections 2023-10-05T19:40:24.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.951+0000 7fd4657fde40 1 -- 172.21.15.12:0/4154051714 wait complete. 2023-10-05T19:40:24.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.951+0000 7fd4657fde40 1 Processor -- start 2023-10-05T19:40:24.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.951+0000 7fd4657fde40 1 -- start start 2023-10-05T19:40:24.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.951+0000 7fd4657fde40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5555dc863e70 0x5555dc889ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:24.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.951+0000 7fd4657fde40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555dc704ed0 con 0x5555dc863e70 2023-10-05T19:40:24.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.955+0000 7fd463332700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5555dc863e70 0x5555dc889ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.955+0000 7fd463332700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5555dc863e70 0x5555dc889ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57842/0 (socket says 172.21.15.12:57842) 2023-10-05T19:40:24.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.955+0000 7fd463332700 1 -- 172.21.15.12:0/3231928650 learned_addr learned my addr 172.21.15.12:0/3231928650 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:24.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.955+0000 7fd463332700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555dc865930 con 0x5555dc863e70 2023-10-05T19:40:24.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.955+0000 7fd463332700 1 --2- 172.21.15.12:0/3231928650 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5555dc863e70 0x5555dc889ab0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd44c01f040 tx=0x7fd44c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.955+0000 7fd45b7fe700 1 -- 172.21.15.12:0/3231928650 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd44c007e50 con 0x5555dc863e70 2023-10-05T19:40:24.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.955+0000 7fd45b7fe700 1 -- 172.21.15.12:0/3231928650 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd44c029070 con 0x5555dc863e70 2023-10-05T19:40:24.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.955+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5555dc889ff0 con 0x5555dc863e70 2023-10-05T19:40:24.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.955+0000 7fd45b7fe700 1 -- 172.21.15.12:0/3231928650 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd44c030480 con 0x5555dc863e70 2023-10-05T19:40:24.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.955+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5555dc88a430 con 0x5555dc863e70 2023-10-05T19:40:24.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.955+0000 7fd45b7fe700 1 -- 172.21.15.12:0/3231928650 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fd44c012400 con 0x5555dc863e70 2023-10-05T19:40:24.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.959+0000 7fd45b7fe700 1 --2- 172.21.15.12:0/3231928650 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd448030c40 0x7fd4480330f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:24.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.959+0000 7fd45b7fe700 1 -- 172.21.15.12:0/3231928650 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7fd44c02b070 con 0x5555dc863e70 2023-10-05T19:40:24.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.959+0000 7fd4657fde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.959+0000 7fd462b31700 1 --2- 172.21.15.12:0/3231928650 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd448030c40 0x7fd4480330f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.959+0000 7fd4657fde40 1 --2- 172.21.15.12:0/3231928650 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5555dc864780 0x5555dc895570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:24.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.959+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc895be0 con 0x5555dc864780 2023-10-05T19:40:24.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.959+0000 7fd463b33700 1 --2- 172.21.15.12:0/3231928650 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5555dc864780 0x5555dc895570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.959+0000 7fd462b31700 1 --2- 172.21.15.12:0/3231928650 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd448030c40 0x7fd4480330f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd45c0044f0 tx=0x7fd45c006690 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.959+0000 7fd463b33700 1 --2- 172.21.15.12:0/3231928650 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5555dc864780 0x5555dc895570 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.959+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd45400a500 con 0x5555dc864780 2023-10-05T19:40:24.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.959+0000 7fd4657fde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:24.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.959+0000 7fd4657fde40 20 realm 2023-10-05T19:40:24.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.959+0000 7fd4657fde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.959+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc892bd0 con 0x5555dc864780 2023-10-05T19:40:24.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.959+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd45400a500 con 0x5555dc864780 2023-10-05T19:40:24.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.959+0000 7fd4657fde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:24.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.959+0000 7fd4657fde40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:24.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.959+0000 7fd4657fde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.959+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc8966c0 con 0x5555dc864780 2023-10-05T19:40:24.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.959+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd45400a500 con 0x5555dc864780 2023-10-05T19:40:24.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.959+0000 7fd4657fde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:24.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.959+0000 7fd4657fde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.959+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc8962d0 con 0x5555dc864780 2023-10-05T19:40:24.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.959+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fd45400a500 con 0x5555dc864780 2023-10-05T19:40:24.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.959+0000 7fd4657fde40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:24.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.959+0000 7fd4657fde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.959+0000 7fd4657fde40 1 --2- 172.21.15.12:0/3231928650 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5555dc897a60 0x5555dc899e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:24.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.959+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89a380 con 0x5555dc897a60 2023-10-05T19:40:24.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.959+0000 7fd463332700 1 --2- 172.21.15.12:0/3231928650 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5555dc897a60 0x5555dc899e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.963+0000 7fd463332700 1 --2- 172.21.15.12:0/3231928650 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5555dc897a60 0x5555dc899e40 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.963+0000 7fd463332700 1 -- 172.21.15.12:0/3231928650 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fd44c01c7f0 con 0x5555dc897a60 2023-10-05T19:40:24.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.963+0000 7fd4657fde40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:24.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.963+0000 7fd4657fde40 20 searching for the correct realm 2023-10-05T19:40:24.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.963+0000 7fd4657fde40 1 --2- 172.21.15.12:0/3231928650 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5555dc89c170 0x5555dc89e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:24.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.963+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89ec00 con 0x5555dc89c170 2023-10-05T19:40:24.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.963+0000 7fd462b31700 1 --2- 172.21.15.12:0/3231928650 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5555dc89c170 0x5555dc89e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.963+0000 7fd462b31700 1 --2- 172.21.15.12:0/3231928650 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5555dc89c170 0x5555dc89e550 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.963+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fd45c004520 con 0x5555dc89c170 2023-10-05T19:40:24.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.963+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89ec00 con 0x5555dc897a60 2023-10-05T19:40:24.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.967+0000 7fd463332700 1 -- 172.21.15.12:0/3231928650 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd44c01c7f0 con 0x5555dc897a60 2023-10-05T19:40:24.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.967+0000 7fd463332700 1 --2- 172.21.15.12:0/3231928650 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fd44c007030 0x7fd44c00c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:24.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.967+0000 7fd463332700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89ec00 con 0x7fd44c007030 2023-10-05T19:40:24.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.967+0000 7fd463b33700 1 --2- 172.21.15.12:0/3231928650 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fd44c007030 0x7fd44c00c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:24.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.967+0000 7fd463b33700 1 --2- 172.21.15.12:0/3231928650 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fd44c007030 0x7fd44c00c9c0 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:24.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.967+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd454014040 con 0x7fd44c007030 2023-10-05T19:40:24.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.967+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89ec00 con 0x5555dc864780 2023-10-05T19:40:24.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.967+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd454014040 con 0x5555dc864780 2023-10-05T19:40:24.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.967+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89ec00 con 0x5555dc864780 2023-10-05T19:40:24.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.967+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd454014040 con 0x5555dc864780 2023-10-05T19:40:24.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.967+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89ec00 con 0x7fd44c007030 2023-10-05T19:40:24.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.971+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd454014040 con 0x7fd44c007030 2023-10-05T19:40:24.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.971+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89ec00 con 0x7fd44c007030 2023-10-05T19:40:24.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.971+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd454014040 con 0x7fd44c007030 2023-10-05T19:40:24.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.971+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89ec00 con 0x5555dc897a60 2023-10-05T19:40:24.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.971+0000 7fd463332700 1 -- 172.21.15.12:0/3231928650 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd44c01c7f0 con 0x5555dc897a60 2023-10-05T19:40:24.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.971+0000 7fd463332700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89ec00 con 0x5555dc89c170 2023-10-05T19:40:24.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.971+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd45c004520 con 0x5555dc89c170 2023-10-05T19:40:24.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.971+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89ec00 con 0x5555dc864780 2023-10-05T19:40:24.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.971+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd454014040 con 0x5555dc864780 2023-10-05T19:40:24.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.971+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89ec00 con 0x7fd44c007030 2023-10-05T19:40:24.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.975+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd454014040 con 0x7fd44c007030 2023-10-05T19:40:24.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.975+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89ec00 con 0x5555dc864780 2023-10-05T19:40:24.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.975+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd454014040 con 0x5555dc864780 2023-10-05T19:40:24.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.975+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89ec00 con 0x5555dc897a60 2023-10-05T19:40:24.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.975+0000 7fd463332700 1 -- 172.21.15.12:0/3231928650 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd44c01c7f0 con 0x5555dc897a60 2023-10-05T19:40:24.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.975+0000 7fd463332700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89ec00 con 0x5555dc89c170 2023-10-05T19:40:24.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.975+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd45c004520 con 0x5555dc89c170 2023-10-05T19:40:24.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.975+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89ec00 con 0x5555dc89c170 2023-10-05T19:40:24.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.975+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd45c004520 con 0x5555dc89c170 2023-10-05T19:40:24.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.975+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89ec00 con 0x7fd44c007030 2023-10-05T19:40:24.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.975+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd454014040 con 0x7fd44c007030 2023-10-05T19:40:24.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.975+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89ec00 con 0x5555dc89c170 2023-10-05T19:40:24.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.979+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd45c004520 con 0x5555dc89c170 2023-10-05T19:40:24.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.979+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89ec00 con 0x5555dc89c170 2023-10-05T19:40:24.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.979+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd45c004520 con 0x5555dc89c170 2023-10-05T19:40:24.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.979+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89ec00 con 0x5555dc864780 2023-10-05T19:40:24.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.979+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd454014040 con 0x5555dc864780 2023-10-05T19:40:24.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.979+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89ec00 con 0x5555dc864780 2023-10-05T19:40:24.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.979+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd454014040 con 0x5555dc864780 2023-10-05T19:40:24.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.979+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89ec00 con 0x7fd44c007030 2023-10-05T19:40:24.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.979+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd454014040 con 0x7fd44c007030 2023-10-05T19:40:24.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.979+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89ec00 con 0x5555dc89c170 2023-10-05T19:40:24.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.979+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd45c004520 con 0x5555dc89c170 2023-10-05T19:40:24.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.979+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89ec00 con 0x5555dc897a60 2023-10-05T19:40:24.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.983+0000 7fd463332700 1 -- 172.21.15.12:0/3231928650 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd44c01c7f0 con 0x5555dc897a60 2023-10-05T19:40:24.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.983+0000 7fd463332700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89ec00 con 0x5555dc864780 2023-10-05T19:40:24.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.983+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd454014040 con 0x5555dc864780 2023-10-05T19:40:24.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.983+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89ec00 con 0x7fd44c007030 2023-10-05T19:40:24.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.983+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd454014040 con 0x7fd44c007030 2023-10-05T19:40:24.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.983+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89ec00 con 0x5555dc89c170 2023-10-05T19:40:24.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.983+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd45c004520 con 0x5555dc89c170 2023-10-05T19:40:24.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.983+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89ec00 con 0x7fd44c007030 2023-10-05T19:40:24.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.983+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd454014040 con 0x7fd44c007030 2023-10-05T19:40:24.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.983+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89ec00 con 0x5555dc897a60 2023-10-05T19:40:24.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.987+0000 7fd463332700 1 -- 172.21.15.12:0/3231928650 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd44c01c7f0 con 0x5555dc897a60 2023-10-05T19:40:24.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.987+0000 7fd463332700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89ec00 con 0x5555dc89c170 2023-10-05T19:40:24.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.987+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd45c004520 con 0x5555dc89c170 2023-10-05T19:40:24.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.987+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89ec00 con 0x7fd44c007030 2023-10-05T19:40:24.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.987+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd454014040 con 0x7fd44c007030 2023-10-05T19:40:24.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.987+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89ec00 con 0x5555dc864780 2023-10-05T19:40:24.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.987+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fd454014040 con 0x5555dc864780 2023-10-05T19:40:24.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.987+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89ec00 con 0x5555dc897a60 2023-10-05T19:40:24.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.991+0000 7fd463332700 1 -- 172.21.15.12:0/3231928650 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fd44c01c7f0 con 0x5555dc897a60 2023-10-05T19:40:24.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.991+0000 7fd4657fde40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:24.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.991+0000 7fd4657fde40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:24.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.991+0000 7fd4657fde40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:24.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.991+0000 7fd4657fde40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:24.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.991+0000 7fd4657fde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:24.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.991+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89aef0 con 0x5555dc864780 2023-10-05T19:40:24.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.991+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd454014040 con 0x5555dc864780 2023-10-05T19:40:25.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.991+0000 7fd4657fde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:25.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.991+0000 7fd4657fde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.991+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89ab20 con 0x5555dc864780 2023-10-05T19:40:25.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.991+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fd454014040 con 0x5555dc864780 2023-10-05T19:40:25.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.991+0000 7fd4657fde40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:25.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.991+0000 7fd4657fde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.991+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89f070 con 0x5555dc89c170 2023-10-05T19:40:25.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.991+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fd45c004520 con 0x5555dc89c170 2023-10-05T19:40:25.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.991+0000 7fd4657fde40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:25.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.991+0000 7fd4657fde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.991+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89f420 con 0x5555dc89c170 2023-10-05T19:40:25.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.991+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7fd45c004520 con 0x5555dc89c170 2023-10-05T19:40:25.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.991+0000 7fd4657fde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:25.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.991+0000 7fd4657fde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.991+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89f7d0 con 0x5555dc864780 2023-10-05T19:40:25.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.991+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd454014040 con 0x5555dc864780 2023-10-05T19:40:25.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.991+0000 7fd4657fde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:25.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.991+0000 7fd4657fde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.991+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89b380 con 0x5555dc864780 2023-10-05T19:40:25.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.995+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fd454014040 con 0x5555dc864780 2023-10-05T19:40:25.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.995+0000 7fd4657fde40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:25.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.995+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89b350 con 0x5555dc89c170 2023-10-05T19:40:25.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.995+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fd45c004520 con 0x5555dc89c170 2023-10-05T19:40:25.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.995+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89b350 con 0x5555dc897a60 2023-10-05T19:40:25.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.995+0000 7fd463332700 1 -- 172.21.15.12:0/3231928650 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd44c051040 con 0x5555dc897a60 2023-10-05T19:40:25.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.995+0000 7fd463332700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89b350 con 0x7fd44c007030 2023-10-05T19:40:25.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.999+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd454014040 con 0x7fd44c007030 2023-10-05T19:40:25.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.999+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89f7d0 con 0x5555dc864780 2023-10-05T19:40:25.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.999+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd454014040 con 0x5555dc864780 2023-10-05T19:40:25.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.999+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89f7d0 con 0x5555dc864780 2023-10-05T19:40:25.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.999+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd454014040 con 0x5555dc864780 2023-10-05T19:40:25.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.999+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89f7d0 con 0x7fd44c007030 2023-10-05T19:40:25.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.999+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd454014040 con 0x7fd44c007030 2023-10-05T19:40:25.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:24.999+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89f7d0 con 0x7fd44c007030 2023-10-05T19:40:25.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.003+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd454014040 con 0x7fd44c007030 2023-10-05T19:40:25.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.003+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89f7d0 con 0x5555dc897a60 2023-10-05T19:40:25.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.003+0000 7fd463332700 1 -- 172.21.15.12:0/3231928650 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd44c051040 con 0x5555dc897a60 2023-10-05T19:40:25.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.003+0000 7fd463332700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89f7d0 con 0x5555dc89c170 2023-10-05T19:40:25.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.003+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd45c02d040 con 0x5555dc89c170 2023-10-05T19:40:25.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.003+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89f7d0 con 0x5555dc864780 2023-10-05T19:40:25.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.003+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd454014040 con 0x5555dc864780 2023-10-05T19:40:25.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.003+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89f7d0 con 0x7fd44c007030 2023-10-05T19:40:25.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.003+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd454014040 con 0x7fd44c007030 2023-10-05T19:40:25.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.007+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89f7d0 con 0x5555dc864780 2023-10-05T19:40:25.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.007+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd454014040 con 0x5555dc864780 2023-10-05T19:40:25.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.007+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89f7d0 con 0x5555dc897a60 2023-10-05T19:40:25.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.007+0000 7fd463332700 1 -- 172.21.15.12:0/3231928650 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd44c051040 con 0x5555dc897a60 2023-10-05T19:40:25.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.007+0000 7fd463332700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89f7d0 con 0x5555dc89c170 2023-10-05T19:40:25.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.007+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd45c02d040 con 0x5555dc89c170 2023-10-05T19:40:25.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.007+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89f7d0 con 0x5555dc89c170 2023-10-05T19:40:25.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.007+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd45c02d040 con 0x5555dc89c170 2023-10-05T19:40:25.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.007+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89f7d0 con 0x7fd44c007030 2023-10-05T19:40:25.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.011+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd454014040 con 0x7fd44c007030 2023-10-05T19:40:25.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.011+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89f7d0 con 0x5555dc89c170 2023-10-05T19:40:25.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.011+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd45c02d040 con 0x5555dc89c170 2023-10-05T19:40:25.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.011+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89f7d0 con 0x5555dc89c170 2023-10-05T19:40:25.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.011+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd45c02d040 con 0x5555dc89c170 2023-10-05T19:40:25.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.011+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89f7d0 con 0x5555dc864780 2023-10-05T19:40:25.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.011+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd454014040 con 0x5555dc864780 2023-10-05T19:40:25.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.011+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89f7d0 con 0x5555dc864780 2023-10-05T19:40:25.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.011+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd454014040 con 0x5555dc864780 2023-10-05T19:40:25.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.011+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89f7d0 con 0x7fd44c007030 2023-10-05T19:40:25.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.015+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd454014040 con 0x7fd44c007030 2023-10-05T19:40:25.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.015+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89f7d0 con 0x5555dc89c170 2023-10-05T19:40:25.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.015+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd45c02d040 con 0x5555dc89c170 2023-10-05T19:40:25.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.015+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89f7d0 con 0x5555dc897a60 2023-10-05T19:40:25.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.015+0000 7fd463332700 1 -- 172.21.15.12:0/3231928650 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd44c051040 con 0x5555dc897a60 2023-10-05T19:40:25.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.015+0000 7fd463332700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fd44c055040 con 0x5555dc864780 2023-10-05T19:40:25.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.015+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd454014040 con 0x5555dc864780 2023-10-05T19:40:25.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.015+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fd44c055040 con 0x7fd44c007030 2023-10-05T19:40:25.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.019+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd454014040 con 0x7fd44c007030 2023-10-05T19:40:25.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.019+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fd44c055040 con 0x5555dc89c170 2023-10-05T19:40:25.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.019+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd45c02d040 con 0x5555dc89c170 2023-10-05T19:40:25.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.019+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fd44c055040 con 0x7fd44c007030 2023-10-05T19:40:25.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.019+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd454014040 con 0x7fd44c007030 2023-10-05T19:40:25.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.019+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fd44c055040 con 0x5555dc897a60 2023-10-05T19:40:25.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.019+0000 7fd463332700 1 -- 172.21.15.12:0/3231928650 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd44c051040 con 0x5555dc897a60 2023-10-05T19:40:25.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.019+0000 7fd463332700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fd44c055040 con 0x5555dc89c170 2023-10-05T19:40:25.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.019+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd45c02d040 con 0x5555dc89c170 2023-10-05T19:40:25.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.019+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fd44c055040 con 0x7fd44c007030 2023-10-05T19:40:25.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.023+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd454014040 con 0x7fd44c007030 2023-10-05T19:40:25.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.023+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fd44c055040 con 0x5555dc864780 2023-10-05T19:40:25.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.023+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fd454014040 con 0x5555dc864780 2023-10-05T19:40:25.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.023+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fd44c055040 con 0x5555dc897a60 2023-10-05T19:40:25.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.023+0000 7fd463332700 1 -- 172.21.15.12:0/3231928650 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fd44c051040 con 0x5555dc897a60 2023-10-05T19:40:25.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.023+0000 7fd4657fde40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:25.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.023+0000 7fd4657fde40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:25.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.023+0000 7fd4657fde40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:25.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.023+0000 7fd4657fde40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:25.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.023+0000 7fd4657fde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.023+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc89b350 con 0x5555dc864780 2023-10-05T19:40:25.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.023+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fd454014040 con 0x5555dc864780 2023-10-05T19:40:25.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.023+0000 7fd4657fde40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:25.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.023+0000 7fd4657fde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.023+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc8a00c0 con 0x5555dc897a60 2023-10-05T19:40:25.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.023+0000 7fd463332700 1 -- 172.21.15.12:0/3231928650 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fd44c051040 con 0x5555dc897a60 2023-10-05T19:40:25.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.023+0000 7fd4657fde40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:25.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.023+0000 7fd4657fde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.023+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc8a09b0 con 0x5555dc864780 2023-10-05T19:40:25.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.027+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fd454014040 con 0x5555dc864780 2023-10-05T19:40:25.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.027+0000 7fd4657fde40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:25.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.027+0000 7fd4657fde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.027+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc8a09b0 con 0x5555dc89c170 2023-10-05T19:40:25.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.027+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fd45c02d040 con 0x5555dc89c170 2023-10-05T19:40:25.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.027+0000 7fd4657fde40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:25.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.027+0000 7fd4657fde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.027+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc8a1120 con 0x7fd44c007030 2023-10-05T19:40:25.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.027+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7fd454014040 con 0x7fd44c007030 2023-10-05T19:40:25.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.027+0000 7fd4657fde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:25.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.027+0000 7fd4657fde40 20 zone default found 2023-10-05T19:40:25.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.027+0000 7fd4657fde40 4 Realm: () 2023-10-05T19:40:25.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.027+0000 7fd4657fde40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:25.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.027+0000 7fd4657fde40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:25.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.027+0000 7fd4657fde40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:25.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.027+0000 7fd4657fde40 20 zonegroup default 2023-10-05T19:40:25.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.027+0000 7fd4657fde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.027+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc8a10d0 con 0x7fd44c007030 2023-10-05T19:40:25.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.027+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fd454014040 con 0x7fd44c007030 2023-10-05T19:40:25.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.027+0000 7fd4657fde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:25.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.027+0000 7fd4657fde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.027+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc8a2a50 con 0x5555dc89c170 2023-10-05T19:40:25.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.031+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fd45c02d040 con 0x5555dc89c170 2023-10-05T19:40:25.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.031+0000 7fd4657fde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:25.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.031+0000 7fd4657fde40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.031+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc8a2e00 con 0x5555dc89c170 2023-10-05T19:40:25.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.031+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fd45c03c040 con 0x5555dc89c170 2023-10-05T19:40:25.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.031+0000 7fd4657fde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:25.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.031+0000 7fd4657fde40 20 started sync module instance, tier type = 2023-10-05T19:40:25.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.031+0000 7fd4657fde40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:25.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.031+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc8a2110 con 0x5555dc897a60 2023-10-05T19:40:25.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.035+0000 7fd463332700 1 -- 172.21.15.12:0/3231928650 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'108 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd44c051040 con 0x5555dc897a60 2023-10-05T19:40:25.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.035+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93827260613824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc8a31b0 con 0x5555dc897a60 2023-10-05T19:40:25.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.035+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc8a3a50 con 0x5555dc864780 2023-10-05T19:40:25.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.035+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'108 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd454014040 con 0x5555dc864780 2023-10-05T19:40:25.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.035+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93827260625328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc8a3bb0 con 0x5555dc864780 2023-10-05T19:40:25.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.035+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc8a46e0 con 0x5555dc89c170 2023-10-05T19:40:25.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.035+0000 7fd463332700 1 -- 172.21.15.12:0/3231928650 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 93827260613824] v28'109 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd44c051040 con 0x5555dc897a60 2023-10-05T19:40:25.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.039+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 93827260625328] v28'109 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd454014040 con 0x5555dc864780 2023-10-05T19:40:25.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.039+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'108 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd45c03c040 con 0x5555dc89c170 2023-10-05T19:40:25.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.039+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93827260629424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc8a3640 con 0x5555dc89c170 2023-10-05T19:40:25.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.039+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc8a4fd0 con 0x5555dc897a60 2023-10-05T19:40:25.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.043+0000 7fd463332700 1 -- 172.21.15.12:0/3231928650 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'108 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd44c051040 con 0x5555dc897a60 2023-10-05T19:40:25.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.043+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 93827260631776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc8a5460 con 0x5555dc897a60 2023-10-05T19:40:25.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.043+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc8a6020 con 0x5555dc864780 2023-10-05T19:40:25.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.043+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 93827260629424] v28'109 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd45c03c040 con 0x5555dc89c170 2023-10-05T19:40:25.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.043+0000 7fd463332700 1 -- 172.21.15.12:0/3231928650 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 93827260631776] v28'109 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd44c051040 con 0x5555dc897a60 2023-10-05T19:40:25.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.047+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'108 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd454014040 con 0x5555dc864780 2023-10-05T19:40:25.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.047+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 93827260632880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc8a5af0 con 0x5555dc864780 2023-10-05T19:40:25.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.047+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc8a6910 con 0x7fd44c007030 2023-10-05T19:40:25.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.047+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'108 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd454014040 con 0x7fd44c007030 2023-10-05T19:40:25.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.047+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 93827260637568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc8a6ac0 con 0x7fd44c007030 2023-10-05T19:40:25.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.047+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc8a74d0 con 0x7fd44c007030 2023-10-05T19:40:25.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.047+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 93827260632880] v28'109 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd454014040 con 0x5555dc864780 2023-10-05T19:40:25.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.051+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(95 notify.5 [watch watch cookie 93827260637568] v28'109 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd454014040 con 0x7fd44c007030 2023-10-05T19:40:25.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.051+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(96 notify.6 [create] v28'108 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd454014040 con 0x7fd44c007030 2023-10-05T19:40:25.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.051+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93827260638672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc8a26a0 con 0x7fd44c007030 2023-10-05T19:40:25.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.051+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc8a79c0 con 0x7fd44c007030 2023-10-05T19:40:25.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.055+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(98 notify.7 [create] v28'108 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd454014040 con 0x7fd44c007030 2023-10-05T19:40:25.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.055+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93827260642512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc8a7e10 con 0x7fd44c007030 2023-10-05T19:40:25.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.055+0000 7fd4657fde40 20 add_watcher() i=0 2023-10-05T19:40:25.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.055+0000 7fd4657fde40 20 add_watcher() i=1 2023-10-05T19:40:25.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.055+0000 7fd4657fde40 20 add_watcher() i=2 2023-10-05T19:40:25.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.055+0000 7fd4657fde40 20 add_watcher() i=3 2023-10-05T19:40:25.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.055+0000 7fd4657fde40 20 add_watcher() i=4 2023-10-05T19:40:25.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.055+0000 7fd4657fde40 20 add_watcher() i=5 2023-10-05T19:40:25.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.055+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(97 notify.6 [watch watch cookie 93827260638672] v28'109 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd454014040 con 0x7fd44c007030 2023-10-05T19:40:25.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.055+0000 7fd4657fde40 20 add_watcher() i=6 2023-10-05T19:40:25.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.055+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 93827260642512] v28'109 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd454014040 con 0x7fd44c007030 2023-10-05T19:40:25.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.055+0000 7fd4657fde40 20 add_watcher() i=7 2023-10-05T19:40:25.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.055+0000 7fd4657fde40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:25.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.055+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc8aae40 con 0x5555dc864780 2023-10-05T19:40:25.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.055+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fd454014040 con 0x5555dc864780 2023-10-05T19:40:25.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.055+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93827260643616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc8a8260 con 0x5555dc864780 2023-10-05T19:40:25.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.059+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 93827260643616] v28'107 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fd454014040 con 0x5555dc864780 2023-10-05T19:40:25.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.059+0000 7fd433fef700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:25.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.059+0000 7fd4657fde40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:25.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.059+0000 7fd4657fde40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:25.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.059+0000 7fd4657fde40 1 Processor -- start 2023-10-05T19:40:25.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.059+0000 7fd4657fde40 1 -- start start 2023-10-05T19:40:25.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.059+0000 7fd4657fde40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5555dc8b5310 0x5555dc8b56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:25.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.059+0000 7fd4657fde40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555dc8b5c20 con 0x5555dc8b5310 2023-10-05T19:40:25.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.059+0000 7fd462b31700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5555dc8b5310 0x5555dc8b56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:25.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.059+0000 7fd462b31700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5555dc8b5310 0x5555dc8b56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57854/0 (socket says 172.21.15.12:57854) 2023-10-05T19:40:25.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.059+0000 7fd462b31700 1 -- 172.21.15.12:0/1300025440 learned_addr learned my addr 172.21.15.12:0/1300025440 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:25.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.063+0000 7fd462b31700 1 -- 172.21.15.12:0/1300025440 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555dc8b5d60 con 0x5555dc8b5310 2023-10-05T19:40:25.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.063+0000 7fd462b31700 1 --2- 172.21.15.12:0/1300025440 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5555dc8b5310 0x5555dc8b56e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x5555dc8974a0 tx=0x7fd45c027040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1e6b4805b106042 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:25.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.063+0000 7fd4337ee700 1 -- 172.21.15.12:0/1300025440 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd45c039040 con 0x5555dc8b5310 2023-10-05T19:40:25.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.063+0000 7fd4337ee700 1 -- 172.21.15.12:0/1300025440 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd45c03a040 con 0x5555dc8b5310 2023-10-05T19:40:25.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.063+0000 7fd4337ee700 1 -- 172.21.15.12:0/1300025440 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd45c02a080 con 0x5555dc8b5310 2023-10-05T19:40:25.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.063+0000 7fd4657fde40 1 -- 172.21.15.12:0/1300025440 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5555dc8b5310 msgr2=0x5555dc8b56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.063+0000 7fd4657fde40 1 --2- 172.21.15.12:0/1300025440 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5555dc8b5310 0x5555dc8b56e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x5555dc8974a0 tx=0x7fd45c027040 comp rx=0 tx=0).stop 2023-10-05T19:40:25.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.063+0000 7fd4657fde40 1 -- 172.21.15.12:0/1300025440 shutdown_connections 2023-10-05T19:40:25.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.063+0000 7fd4657fde40 1 --2- 172.21.15.12:0/1300025440 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5555dc8b5310 0x5555dc8b56e0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:25.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.063+0000 7fd4657fde40 1 -- 172.21.15.12:0/1300025440 >> 172.21.15.12:0/1300025440 conn(0x5555dc8b06d0 msgr2=0x5555dc8ae820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:25.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.063+0000 7fd4657fde40 1 -- 172.21.15.12:0/1300025440 shutdown_connections 2023-10-05T19:40:25.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.063+0000 7fd4657fde40 1 -- 172.21.15.12:0/1300025440 wait complete. 2023-10-05T19:40:25.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.063+0000 7fd4657fde40 1 Processor -- start 2023-10-05T19:40:25.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.063+0000 7fd4657fde40 1 -- start start 2023-10-05T19:40:25.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.063+0000 7fd4657fde40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5555dc8b5310 0x5555dc8baeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:25.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.063+0000 7fd4657fde40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555dc8bbeb0 con 0x5555dc8b5310 2023-10-05T19:40:25.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.067+0000 7fd462b31700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5555dc8b5310 0x5555dc8baeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:25.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.067+0000 7fd462b31700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5555dc8b5310 0x5555dc8baeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57862/0 (socket says 172.21.15.12:57862) 2023-10-05T19:40:25.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.067+0000 7fd462b31700 1 -- 172.21.15.12:0/2022348055 learned_addr learned my addr 172.21.15.12:0/2022348055 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:25.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.067+0000 7fd462b31700 1 -- 172.21.15.12:0/2022348055 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd444001e10 con 0x5555dc8b5310 2023-10-05T19:40:25.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.067+0000 7fd462b31700 1 --2- 172.21.15.12:0/2022348055 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5555dc8b5310 0x5555dc8baeb0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd45c0067a0 tx=0x7fd45c03f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:25.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.067+0000 7fd431feb700 1 -- 172.21.15.12:0/2022348055 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd45c039040 con 0x5555dc8b5310 2023-10-05T19:40:25.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.067+0000 7fd431feb700 1 -- 172.21.15.12:0/2022348055 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd45c02b040 con 0x5555dc8b5310 2023-10-05T19:40:25.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.067+0000 7fd4657fde40 1 -- 172.21.15.12:0/2022348055 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5555dc8bb4b0 con 0x5555dc8b5310 2023-10-05T19:40:25.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.067+0000 7fd431feb700 1 -- 172.21.15.12:0/2022348055 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd45c01a040 con 0x5555dc8b5310 2023-10-05T19:40:25.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.067+0000 7fd4657fde40 1 -- 172.21.15.12:0/2022348055 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5555dc8bb8f0 con 0x5555dc8b5310 2023-10-05T19:40:25.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.067+0000 7fd431feb700 1 -- 172.21.15.12:0/2022348055 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fd45c04c970 con 0x5555dc8b5310 2023-10-05T19:40:25.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.067+0000 7fd431feb700 1 --2- 172.21.15.12:0/2022348055 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd3a8030ca0 0x7fd3a8033150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:25.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.071+0000 7fd431feb700 1 -- 172.21.15.12:0/2022348055 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7fd45c028040 con 0x5555dc8b5310 2023-10-05T19:40:25.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.071+0000 7fd463b33700 1 --2- 172.21.15.12:0/2022348055 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd3a8030ca0 0x7fd3a8033150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:25.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.071+0000 7fd4657fde40 5 note: GC not initialized 2023-10-05T19:40:25.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.071+0000 7fd463b33700 1 --2- 172.21.15.12:0/2022348055 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd3a8030ca0 0x7fd3a8033150 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x5555dc8a7250 tx=0x7fd454033830 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:25.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.071+0000 7fd3a2ffd700 20 reqs_thread_entry: start 2023-10-05T19:40:25.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.071+0000 7fd3a0ff9700 20 reqs_thread_entry: start 2023-10-05T19:40:25.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.071+0000 7fd4657fde40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:25.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.071+0000 7fd4657fde40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:25.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.071+0000 7fd392ffd700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:55 2023 2023-10-05T19:40:25.082 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:25.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.071+0000 7fd4657fde40 20 get_system_obj_state: rctx=0x7ffc876fd728 obj=default.rgw.meta:users.uid:fud state=0x5555dc73d0b0 s->prefetch_data=0 2023-10-05T19:40:25.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.071+0000 7fd4657fde40 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2023-10-05T19:40:25.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.071+0000 7fd3927fc700 20 reqs_thread_entry: start 2023-10-05T19:40:25.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.071+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:102 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc9237e0 con 0x7fd44c007030 2023-10-05T19:40:25.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.075+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(102 fud [call,getxattrs,stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x7fd454014040 con 0x7fd44c007030 2023-10-05T19:40:25.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.075+0000 7fd4657fde40 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x0 2023-10-05T19:40:25.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.075+0000 7fd4657fde40 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2023-10-05T19:40:25.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.075+0000 7fd4657fde40 20 get_system_obj_state: rctx=0x7ffc876fd418 obj=default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ state=0x5555dc73d0b0 s->prefetch_data=0 2023-10-05T19:40:25.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.075+0000 7fd4657fde40 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : miss 2023-10-05T19:40:25.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.075+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:103 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc9237e0 con 0x7fd44c007030 2023-10-05T19:40:25.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.075+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(103 p5YnriCv1nAtykxBrupQ [getxattrs,stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 206+0+0 (crc 0 0 0) 0x7fd454014040 con 0x7fd44c007030 2023-10-05T19:40:25.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.075+0000 7fd4657fde40 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x0 2023-10-05T19:40:25.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.075+0000 7fd4657fde40 10 adding default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2023-10-05T19:40:25.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.075+0000 7fd4657fde40 20 get_system_obj_state: rctx=0x7ffc876fccb8 obj=default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ state=0x5555dc73d0b0 s->prefetch_data=0 2023-10-05T19:40:25.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.075+0000 7fd4657fde40 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2023-10-05T19:40:25.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.075+0000 7fd4657fde40 20 get_system_obj_state: rctx=0x7ffc876fd358 obj=default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ state=0x5555dc73d0b0 s->prefetch_data=0 2023-10-05T19:40:25.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.075+0000 7fd4657fde40 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2023-10-05T19:40:25.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.075+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:104 8.e 8:75f48285:users.uid::fud:head [delete,create,call version.set in=58b,writefull 0~346 in=346b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc925d80 con 0x7fd44c007030 2023-10-05T19:40:25.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.079+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(104 fud [delete,create,call,writefull 0~346] v28'2 uv2 ondisk = 0) v8 ==== 273+0+0 (crc 0 0 0) 0x7fd454014040 con 0x7fd44c007030 2023-10-05T19:40:25.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.079+0000 7fd4657fde40 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x17 2023-10-05T19:40:25.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.079+0000 7fd4657fde40 10 moving default.rgw.meta+users.uid+fud to cache LRU end 2023-10-05T19:40:25.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.079+0000 7fd4657fde40 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:fud, ofs0, ns] 2023-10-05T19:40:25.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.079+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:105 7.3 7:c609908c:::notify.5:head [notify cookie 93827261156720 in=534b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc9256b0 con 0x7fd44c007030 2023-10-05T19:40:25.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.079+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.0 v2:172.21.15.8:6800/27563 30 ==== watch-notify(notify (1) cookie 93827260637568 notify 120259084302 ret 0) v3 ==== 564+0+0 (crc 0 0 0) 0x7fd4540335f0 con 0x7fd44c007030 2023-10-05T19:40:25.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.079+0000 7fd46132e700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084302 cookie 93827260637568 notifier 4441 bl.length()=522 2023-10-05T19:40:25.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.079+0000 7fd46132e700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+fud info.flags=0x17 2023-10-05T19:40:25.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.079+0000 7fd46132e700 10 rgw watcher librados: moving default.rgw.meta+users.uid+fud to cache LRU end 2023-10-05T19:40:25.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.079+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.0 v2:172.21.15.8:6800/27563 31 ==== osd_op_reply(105 notify.5 [notify cookie 93827261156720 out=8b] v0'0 uv108 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fd454014040 con 0x7fd44c007030 2023-10-05T19:40:25.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.079+0000 7fd46132e700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:106 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fd43c003d90 con 0x7fd44c007030 2023-10-05T19:40:25.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.079+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.0 v2:172.21.15.8:6800/27563 32 ==== osd_op_reply(106 notify.5 [notify-ack] v0'0 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd454014040 con 0x7fd44c007030 2023-10-05T19:40:25.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.079+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.0 v2:172.21.15.8:6800/27563 33 ==== watch-notify(notify_complete (2) cookie 93827261156720 notify 120259084302 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7fd454031650 con 0x7fd44c007030 2023-10-05T19:40:25.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.079+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:107 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc923d60 con 0x7fd44c007030 2023-10-05T19:40:25.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.083+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.0 v2:172.21.15.8:6800/27563 34 ==== osd_op_reply(107 p5YnriCv1nAtykxBrupQ [delete,create,writefull 0~7] v28'3 uv3 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x7fd454014040 con 0x7fd44c007030 2023-10-05T19:40:25.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.083+0000 7fd4657fde40 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2023-10-05T19:40:25.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.083+0000 7fd4657fde40 10 moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2023-10-05T19:40:25.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.083+0000 7fd4657fde40 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2023-10-05T19:40:25.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.083+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:108 7.15 7:a93a5511:::notify.2:head [notify cookie 93827261160176 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc925310 con 0x5555dc89c170 2023-10-05T19:40:25.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.083+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== watch-notify(notify (1) cookie 93827260629424 notify 120259084290 ret 0) v3 ==== 219+0+0 (crc 0 0 0) 0x5555dc8bb8f0 con 0x5555dc89c170 2023-10-05T19:40:25.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.083+0000 7fd460b2d700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084290 cookie 93827260629424 notifier 4441 bl.length()=177 2023-10-05T19:40:25.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.083+0000 7fd460b2d700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2023-10-05T19:40:25.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.083+0000 7fd460b2d700 10 rgw watcher librados: moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2023-10-05T19:40:25.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.087+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 <== osd.3 v2:172.21.15.12:6816/29951 27 ==== osd_op_reply(108 notify.2 [notify cookie 93827261160176 out=8b] v0'0 uv108 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fd45c03c040 con 0x5555dc89c170 2023-10-05T19:40:25.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.087+0000 7fd460b2d700 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:109 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fd3b0003450 con 0x5555dc89c170 2023-10-05T19:40:25.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.087+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 <== osd.3 v2:172.21.15.12:6816/29951 28 ==== osd_op_reply(109 notify.2 [notify-ack] v0'0 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd45c03c040 con 0x5555dc89c170 2023-10-05T19:40:25.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.087+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 <== osd.3 v2:172.21.15.12:6816/29951 29 ==== watch-notify(notify_complete (2) cookie 93827261160176 notify 120259084290 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5555dc8bb4b0 con 0x5555dc89c170 2023-10-05T19:40:25.095 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:40:25.095 INFO:teuthology.orchestra.run.smithi012.stdout: "user_id": "fud", 2023-10-05T19:40:25.095 INFO:teuthology.orchestra.run.smithi012.stdout: "display_name": "Fud", 2023-10-05T19:40:25.096 INFO:teuthology.orchestra.run.smithi012.stdout: "email": "", 2023-10-05T19:40:25.096 INFO:teuthology.orchestra.run.smithi012.stdout: "suspended": 0, 2023-10-05T19:40:25.096 INFO:teuthology.orchestra.run.smithi012.stdout: "max_buckets": 1, 2023-10-05T19:40:25.096 INFO:teuthology.orchestra.run.smithi012.stdout: "subusers": [], 2023-10-05T19:40:25.096 INFO:teuthology.orchestra.run.smithi012.stdout: "keys": [ 2023-10-05T19:40:25.096 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:25.096 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "fud", 2023-10-05T19:40:25.096 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key": "p5YnriCv1nAtykxBrupQ", 2023-10-05T19:40:25.096 INFO:teuthology.orchestra.run.smithi012.stdout: "secret_key": "Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh" 2023-10-05T19:40:25.096 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:25.096 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:25.096 INFO:teuthology.orchestra.run.smithi012.stdout: "swift_keys": [], 2023-10-05T19:40:25.097 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": [], 2023-10-05T19:40:25.097 INFO:teuthology.orchestra.run.smithi012.stdout: "op_mask": "read, write, delete", 2023-10-05T19:40:25.097 INFO:teuthology.orchestra.run.smithi012.stdout: "default_placement": "", 2023-10-05T19:40:25.097 INFO:teuthology.orchestra.run.smithi012.stdout: "default_storage_class": "", 2023-10-05T19:40:25.097 INFO:teuthology.orchestra.run.smithi012.stdout: "placement_tags": [], 2023-10-05T19:40:25.097 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket_quota": { 2023-10-05T19:40:25.097 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:25.097 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:25.097 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:25.097 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:25.097 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:25.097 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:25.098 INFO:teuthology.orchestra.run.smithi012.stdout: "user_quota": { 2023-10-05T19:40:25.098 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:25.098 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:25.098 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:25.098 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:25.098 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:25.098 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:25.098 INFO:teuthology.orchestra.run.smithi012.stdout: "temp_url_keys": [], 2023-10-05T19:40:25.098 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "rgw", 2023-10-05T19:40:25.098 INFO:teuthology.orchestra.run.smithi012.stdout: "mfa_ids": [] 2023-10-05T19:40:25.098 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:40:25.098 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:40:25.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.087+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:110 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93827260613824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc879650 con 0x5555dc897a60 2023-10-05T19:40:25.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.091+0000 7fd463332700 1 -- 172.21.15.12:0/3231928650 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(110 notify.0 [watch unwatch cookie 93827260613824] v28'110 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd44c051040 con 0x5555dc897a60 2023-10-05T19:40:25.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.091+0000 7fd4657fde40 20 remove_watcher() i=0 2023-10-05T19:40:25.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.091+0000 7fd4657fde40 2 removed watcher, disabling cache 2023-10-05T19:40:25.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.091+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:111 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93827260625328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc8a04c0 con 0x5555dc864780 2023-10-05T19:40:25.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.091+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(111 notify.1 [watch unwatch cookie 93827260625328] v28'110 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd454014040 con 0x5555dc864780 2023-10-05T19:40:25.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.091+0000 7fd4657fde40 20 remove_watcher() i=1 2023-10-05T19:40:25.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.091+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:112 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93827260629424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc8a04c0 con 0x5555dc89c170 2023-10-05T19:40:25.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.095+0000 7fd462b31700 1 -- 172.21.15.12:0/3231928650 <== osd.3 v2:172.21.15.12:6816/29951 30 ==== osd_op_reply(112 notify.2 [watch unwatch cookie 93827260629424] v28'110 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd45c03c040 con 0x5555dc89c170 2023-10-05T19:40:25.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.095+0000 7fd4657fde40 20 remove_watcher() i=2 2023-10-05T19:40:25.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.095+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:113 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93827260631776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc8a41b0 con 0x5555dc897a60 2023-10-05T19:40:25.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.099+0000 7fd463332700 1 -- 172.21.15.12:0/3231928650 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(113 notify.3 [watch unwatch cookie 93827260631776] v28'110 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd44c051040 con 0x5555dc897a60 2023-10-05T19:40:25.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.099+0000 7fd4657fde40 20 remove_watcher() i=3 2023-10-05T19:40:25.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.099+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:114 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93827260632880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc8a4ae0 con 0x5555dc864780 2023-10-05T19:40:25.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.099+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(114 notify.4 [watch unwatch cookie 93827260632880] v28'110 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd454014040 con 0x5555dc864780 2023-10-05T19:40:25.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.099+0000 7fd4657fde40 20 remove_watcher() i=4 2023-10-05T19:40:25.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.099+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:115 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93827260637568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc8a31b0 con 0x7fd44c007030 2023-10-05T19:40:25.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.103+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.0 v2:172.21.15.8:6800/27563 35 ==== osd_op_reply(115 notify.5 [watch unwatch cookie 93827260637568] v28'110 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd454014040 con 0x7fd44c007030 2023-10-05T19:40:25.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.103+0000 7fd4657fde40 20 remove_watcher() i=5 2023-10-05T19:40:25.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.103+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:116 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93827260638672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc8a31b0 con 0x7fd44c007030 2023-10-05T19:40:25.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.107+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.0 v2:172.21.15.8:6800/27563 36 ==== osd_op_reply(116 notify.6 [watch unwatch cookie 93827260638672] v28'110 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd454014040 con 0x7fd44c007030 2023-10-05T19:40:25.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.107+0000 7fd4657fde40 20 remove_watcher() i=6 2023-10-05T19:40:25.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.107+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:117 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93827260642512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc8a31b0 con 0x7fd44c007030 2023-10-05T19:40:25.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.111+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.0 v2:172.21.15.8:6800/27563 37 ==== osd_op_reply(117 notify.7 [watch unwatch cookie 93827260642512] v28'110 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd454014040 con 0x7fd44c007030 2023-10-05T19:40:25.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.111+0000 7fd4657fde40 20 remove_watcher() i=7 2023-10-05T19:40:25.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.111+0000 7fd433fef700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:25.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.111+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:118 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93827260643616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5555dc8a31b0 con 0x5555dc864780 2023-10-05T19:40:25.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.115+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(118 data_loggenerations_metadata [watch unwatch cookie 93827260643616] v28'108 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fd454014040 con 0x5555dc864780 2023-10-05T19:40:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.115+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fd44c007030 msgr2=0x7fd44c00c9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.115+0000 7fd4657fde40 1 --2- 172.21.15.12:0/3231928650 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fd44c007030 0x7fd44c00c9c0 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.115+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5555dc864780 msgr2=0x5555dc895570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.115+0000 7fd4657fde40 1 --2- 172.21.15.12:0/3231928650 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5555dc864780 0x5555dc895570 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.115+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5555dc897a60 msgr2=0x5555dc899e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.115+0000 7fd4657fde40 1 --2- 172.21.15.12:0/3231928650 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5555dc897a60 0x5555dc899e40 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.115+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5555dc89c170 msgr2=0x5555dc89e550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.115+0000 7fd4657fde40 1 --2- 172.21.15.12:0/3231928650 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5555dc89c170 0x5555dc89e550 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.115+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd448030c40 msgr2=0x7fd4480330f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.115+0000 7fd4657fde40 1 --2- 172.21.15.12:0/3231928650 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd448030c40 0x7fd4480330f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd45c0044f0 tx=0x7fd45c006690 comp rx=0 tx=0).stop 2023-10-05T19:40:25.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.115+0000 7fd463b33700 1 -- 172.21.15.12:0/3231928650 reap_dead start 2023-10-05T19:40:25.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.115+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5555dc863e70 msgr2=0x5555dc889ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.115+0000 7fd4657fde40 1 --2- 172.21.15.12:0/3231928650 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5555dc863e70 0x5555dc889ab0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd44c01f040 tx=0x7fd44c01c000 comp rx=0 tx=0).stop 2023-10-05T19:40:25.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.115+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 shutdown_connections 2023-10-05T19:40:25.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.115+0000 7fd4657fde40 1 --2- 172.21.15.12:0/3231928650 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5555dc863e70 0x5555dc889ab0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:25.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.115+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 >> 172.21.15.12:0/3231928650 conn(0x5555dc85f890 msgr2=0x5555dc87d430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:25.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.115+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 shutdown_connections 2023-10-05T19:40:25.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.115+0000 7fd4657fde40 1 -- 172.21.15.12:0/3231928650 wait complete. 2023-10-05T19:40:25.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.115+0000 7fd4657fde40 1 -- 172.21.15.12:0/2022348055 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd3a8030ca0 msgr2=0x7fd3a8033150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.115+0000 7fd4657fde40 1 --2- 172.21.15.12:0/2022348055 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd3a8030ca0 0x7fd3a8033150 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x5555dc8a7250 tx=0x7fd454033830 comp rx=0 tx=0).stop 2023-10-05T19:40:25.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.115+0000 7fd4657fde40 1 -- 172.21.15.12:0/2022348055 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5555dc8b5310 msgr2=0x5555dc8baeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.115+0000 7fd4657fde40 1 --2- 172.21.15.12:0/2022348055 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5555dc8b5310 0x5555dc8baeb0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd45c0067a0 tx=0x7fd45c03f000 comp rx=0 tx=0).stop 2023-10-05T19:40:25.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.115+0000 7fd4657fde40 1 -- 172.21.15.12:0/2022348055 shutdown_connections 2023-10-05T19:40:25.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.115+0000 7fd4657fde40 1 --2- 172.21.15.12:0/2022348055 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd3a8030ca0 0x7fd3a8033150 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:25.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.115+0000 7fd4657fde40 1 --2- 172.21.15.12:0/2022348055 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5555dc8b5310 0x5555dc8baeb0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:25.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.115+0000 7fd4657fde40 1 -- 172.21.15.12:0/2022348055 >> 172.21.15.12:0/2022348055 conn(0x5555dc8b06d0 msgr2=0x5555dc8ae770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:25.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.115+0000 7fd4657fde40 1 -- 172.21.15.12:0/2022348055 shutdown_connections 2023-10-05T19:40:25.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.119+0000 7fd4657fde40 1 -- 172.21.15.12:0/2022348055 wait complete. 2023-10-05T19:40:25.131 DEBUG:tasks.util.rgw: json result: {'user_id': 'fud', 'display_name': 'Fud', 'email': '', 'suspended': 0, 'max_buckets': 1, 'subusers': [], 'keys': [{'user': 'fud', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:40:25.140 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=3 bytes_in=0 failed=False 2023-10-05T19:40:25.149 INFO:tasks.radosgw_admin:RL: delete_obj myfoo foo bytes_out=0 bytes_in=0 failed=False 2023-10-05T19:40:25.149 INFO:tasks.util.rgw:rgwadmin: client.0 : ['metadata', 'get', 'bucket:myfoo'] 2023-10-05T19:40:25.149 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'metadata', 'get', 'bucket:myfoo'] 2023-10-05T19:40:25.149 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph metadata get bucket:myfoo 2023-10-05T19:40:25.185 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:25.185 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:25.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.179+0000 7fcc005d2e40 1 Processor -- start 2023-10-05T19:40:25.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.179+0000 7fcc005d2e40 1 -- start start 2023-10-05T19:40:25.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.179+0000 7fcc005d2e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55c2262de410 0x55c2262dcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:25.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.179+0000 7fcc005d2e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55c22622abd0 con 0x55c2262de7e0 2023-10-05T19:40:25.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.179+0000 7fcc005d2e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x55c2261e7ee0 con 0x55c2262de410 2023-10-05T19:40:25.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.179+0000 7fcbfe107700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55c2262de410 0x55c2262dcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:25.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.179+0000 7fcbf5906700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x55c2262de7e0 0x55c2262dd500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:35534/0 (socket says 172.21.15.12:35534) 2023-10-05T19:40:25.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.179+0000 7fcbfe107700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55c2262de410 0x55c2262dcfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57864/0 (socket says 172.21.15.12:57864) 2023-10-05T19:40:25.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.179+0000 7fcbf5906700 1 -- 172.21.15.12:0/3770798294 learned_addr learned my addr 172.21.15.12:0/3770798294 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:25.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.179+0000 7fcbfe107700 1 -- 172.21.15.12:0/3770798294 >> v1:172.21.15.8:6789/0 conn(0x55c2262de7e0 legacy=0x55c2262dd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.179+0000 7fcbfe107700 1 -- 172.21.15.12:0/3770798294 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2262ddcd0 con 0x55c2262de410 2023-10-05T19:40:25.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.179+0000 7fcbfe107700 1 --2- 172.21.15.12:0/3770798294 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c2262de410 0x55c2262dcfc0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fcbf000a5d0 tx=0x7fcbf0005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13d4202347c5d80 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:25.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.179+0000 7fcbfd906700 1 -- 172.21.15.12:0/3770798294 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcbf00086a0 con 0x55c2262de410 2023-10-05T19:40:25.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.179+0000 7fcbfd906700 1 -- 172.21.15.12:0/3770798294 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbf0008800 con 0x55c2262de410 2023-10-05T19:40:25.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.179+0000 7fcbfd906700 1 -- 172.21.15.12:0/3770798294 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcbf0008ad0 con 0x55c2262de410 2023-10-05T19:40:25.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.179+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3770798294 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c2262de410 msgr2=0x55c2262dcfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.179+0000 7fcc005d2e40 1 --2- 172.21.15.12:0/3770798294 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c2262de410 0x55c2262dcfc0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fcbf000a5d0 tx=0x7fcbf0005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:25.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.179+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3770798294 shutdown_connections 2023-10-05T19:40:25.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.179+0000 7fcc005d2e40 1 --2- 172.21.15.12:0/3770798294 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c2262de410 0x55c2262dcfc0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:25.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.179+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3770798294 >> 172.21.15.12:0/3770798294 conn(0x55c2262d8720 msgr2=0x55c2262dab30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:25.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.179+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3770798294 shutdown_connections 2023-10-05T19:40:25.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.179+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3770798294 wait complete. 2023-10-05T19:40:25.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.183+0000 7fcc005d2e40 1 Processor -- start 2023-10-05T19:40:25.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.183+0000 7fcc005d2e40 1 -- start start 2023-10-05T19:40:25.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.183+0000 7fcc005d2e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c2262de410 0x55c2262ff180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:25.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.183+0000 7fcc005d2e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2261abe30 con 0x55c2262de410 2023-10-05T19:40:25.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.183+0000 7fcbfe107700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c2262de410 0x55c2262ff180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:25.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.183+0000 7fcbfe107700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c2262de410 0x55c2262ff180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57874/0 (socket says 172.21.15.12:57874) 2023-10-05T19:40:25.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.183+0000 7fcbfe107700 1 -- 172.21.15.12:0/3347267916 learned_addr learned my addr 172.21.15.12:0/3347267916 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:25.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.183+0000 7fcbfe107700 1 -- 172.21.15.12:0/3347267916 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf0005cf0 con 0x55c2262de410 2023-10-05T19:40:25.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.183+0000 7fcbfe107700 1 --2- 172.21.15.12:0/3347267916 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c2262de410 0x55c2262ff180 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0000c00 tx=0x7fcbf0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=862d75010c05aa77 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:25.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.183+0000 7fcbfc904700 1 -- 172.21.15.12:0/3347267916 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcbf0008970 con 0x55c2262de410 2023-10-05T19:40:25.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.183+0000 7fcbfc904700 1 -- 172.21.15.12:0/3347267916 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbf0008af0 con 0x55c2262de410 2023-10-05T19:40:25.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.183+0000 7fcbfc904700 1 -- 172.21.15.12:0/3347267916 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcbf0008de0 con 0x55c2262de410 2023-10-05T19:40:25.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.183+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3347267916 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c2262de410 msgr2=0x55c2262ff180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.183+0000 7fcc005d2e40 1 --2- 172.21.15.12:0/3347267916 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c2262de410 0x55c2262ff180 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0000c00 tx=0x7fcbf0015000 comp rx=0 tx=0).stop 2023-10-05T19:40:25.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.183+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3347267916 shutdown_connections 2023-10-05T19:40:25.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.183+0000 7fcc005d2e40 1 --2- 172.21.15.12:0/3347267916 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c2262de410 0x55c2262ff180 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:25.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.183+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3347267916 >> 172.21.15.12:0/3347267916 conn(0x55c2262d8720 msgr2=0x55c2262fad40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:25.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.183+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3347267916 shutdown_connections 2023-10-05T19:40:25.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.183+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3347267916 wait complete. 2023-10-05T19:40:25.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.183+0000 7fcc005d2e40 1 Processor -- start 2023-10-05T19:40:25.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.183+0000 7fcc005d2e40 1 -- start start 2023-10-05T19:40:25.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.183+0000 7fcc005d2e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c2262de410 0x55c226305de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:25.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.183+0000 7fcc005d2e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c22617eed0 con 0x55c2262de410 2023-10-05T19:40:25.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.183+0000 7fcbfe107700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c2262de410 0x55c226305de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:25.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.183+0000 7fcbfe107700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c2262de410 0x55c226305de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57884/0 (socket says 172.21.15.12:57884) 2023-10-05T19:40:25.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.183+0000 7fcbfe107700 1 -- 172.21.15.12:0/3979373450 learned_addr learned my addr 172.21.15.12:0/3979373450 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:25.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.187+0000 7fcbfe107700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2262ddcd0 con 0x55c2262de410 2023-10-05T19:40:25.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.187+0000 7fcbfe107700 1 --2- 172.21.15.12:0/3979373450 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c2262de410 0x55c226305de0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0000c00 tx=0x7fcbf0003050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:25.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.187+0000 7fcbf6ffd700 1 -- 172.21.15.12:0/3979373450 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcbf0012b70 con 0x55c2262de410 2023-10-05T19:40:25.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.187+0000 7fcbf6ffd700 1 -- 172.21.15.12:0/3979373450 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbf002a070 con 0x55c2262de410 2023-10-05T19:40:25.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.187+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c226306320 con 0x55c2262de410 2023-10-05T19:40:25.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.187+0000 7fcbf6ffd700 1 -- 172.21.15.12:0/3979373450 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcbf0012e50 con 0x55c2262de410 2023-10-05T19:40:25.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.187+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2263067c0 con 0x55c2262de410 2023-10-05T19:40:25.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.187+0000 7fcbf6ffd700 1 -- 172.21.15.12:0/3979373450 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fcbf0019070 con 0x55c2262de410 2023-10-05T19:40:25.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.187+0000 7fcbf6ffd700 1 --2- 172.21.15.12:0/3979373450 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fcbe0030b10 0x7fcbe0032fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:25.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.187+0000 7fcbf6ffd700 1 -- 172.21.15.12:0/3979373450 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7fcbf002c070 con 0x55c2262de410 2023-10-05T19:40:25.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.187+0000 7fcbf5906700 1 --2- 172.21.15.12:0/3979373450 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fcbe0030b10 0x7fcbe0032fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:25.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.187+0000 7fcbf5906700 1 --2- 172.21.15.12:0/3979373450 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fcbe0030b10 0x7fcbe0032fc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8004170 tx=0x7fcbe8001380 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:25.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.187+0000 7fcc005d2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.191+0000 7fcc005d2e40 1 --2- 172.21.15.12:0/3979373450 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55c2262de7e0 0x55c226311830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:25.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.191+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c226311f60 con 0x55c2262de7e0 2023-10-05T19:40:25.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.191+0000 7fcbfe908700 1 --2- 172.21.15.12:0/3979373450 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55c2262de7e0 0x55c226311830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:25.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.191+0000 7fcbfe908700 1 --2- 172.21.15.12:0/3979373450 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55c2262de7e0 0x55c226311830 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:25.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.191+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fcbf800a5a0 con 0x55c2262de7e0 2023-10-05T19:40:25.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.191+0000 7fcc005d2e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:25.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.191+0000 7fcc005d2e40 20 realm 2023-10-05T19:40:25.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.191+0000 7fcc005d2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.191+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22630edd0 con 0x55c2262de7e0 2023-10-05T19:40:25.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.191+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fcbf800a5a0 con 0x55c2262de7e0 2023-10-05T19:40:25.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.191+0000 7fcc005d2e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:25.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.191+0000 7fcc005d2e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:25.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.191+0000 7fcc005d2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.191+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c226312a40 con 0x55c2262de7e0 2023-10-05T19:40:25.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.191+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fcbf800a5a0 con 0x55c2262de7e0 2023-10-05T19:40:25.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.191+0000 7fcc005d2e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:25.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.191+0000 7fcc005d2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.191+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c226312620 con 0x55c2262de7e0 2023-10-05T19:40:25.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.191+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fcbf800a5a0 con 0x55c2262de7e0 2023-10-05T19:40:25.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.191+0000 7fcc005d2e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:25.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.191+0000 7fcc005d2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.191+0000 7fcc005d2e40 1 --2- 172.21.15.12:0/3979373450 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55c2263139d0 0x55c226315db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:25.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.191+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c226316520 con 0x55c2263139d0 2023-10-05T19:40:25.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.195+0000 7fcbfe107700 1 --2- 172.21.15.12:0/3979373450 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55c2263139d0 0x55c226315db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:25.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.195+0000 7fcbfe107700 1 --2- 172.21.15.12:0/3979373450 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55c2263139d0 0x55c226315db0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:25.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.195+0000 7fcbfe107700 1 -- 172.21.15.12:0/3979373450 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fcbf0022020 con 0x55c2263139d0 2023-10-05T19:40:25.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.195+0000 7fcc005d2e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:25.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.195+0000 7fcc005d2e40 20 searching for the correct realm 2023-10-05T19:40:25.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.195+0000 7fcc005d2e40 1 --2- 172.21.15.12:0/3979373450 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55c226318350 0x55c22631a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:25.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.195+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631ac70 con 0x55c226318350 2023-10-05T19:40:25.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.195+0000 7fcbf5906700 1 --2- 172.21.15.12:0/3979373450 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55c226318350 0x55c22631a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:25.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.195+0000 7fcbf5906700 1 --2- 172.21.15.12:0/3979373450 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55c226318350 0x55c22631a730 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:25.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.199+0000 7fcbf5906700 1 -- 172.21.15.12:0/3979373450 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fcbe8006070 con 0x55c226318350 2023-10-05T19:40:25.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.199+0000 7fcbf5906700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631ac70 con 0x55c2263139d0 2023-10-05T19:40:25.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.199+0000 7fcbfe107700 1 -- 172.21.15.12:0/3979373450 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbf0022020 con 0x55c2263139d0 2023-10-05T19:40:25.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.199+0000 7fcbfe107700 1 --2- 172.21.15.12:0/3979373450 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fcbf0007030 0x7fcbf00119f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:25.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.199+0000 7fcbfe107700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631ac70 con 0x7fcbf0007030 2023-10-05T19:40:25.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.199+0000 7fcbfe908700 1 --2- 172.21.15.12:0/3979373450 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fcbf0007030 0x7fcbf00119f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:25.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.199+0000 7fcbfe908700 1 --2- 172.21.15.12:0/3979373450 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fcbf0007030 0x7fcbf00119f0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:25.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.199+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbf800a5a0 con 0x7fcbf0007030 2023-10-05T19:40:25.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.199+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631ac70 con 0x55c2262de7e0 2023-10-05T19:40:25.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.199+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbf800a5a0 con 0x55c2262de7e0 2023-10-05T19:40:25.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.199+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631ac70 con 0x55c2262de7e0 2023-10-05T19:40:25.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.203+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbf800a5a0 con 0x55c2262de7e0 2023-10-05T19:40:25.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.203+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631ac70 con 0x7fcbf0007030 2023-10-05T19:40:25.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.203+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbf800a5a0 con 0x7fcbf0007030 2023-10-05T19:40:25.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.203+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631ac70 con 0x7fcbf0007030 2023-10-05T19:40:25.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.203+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbf800a5a0 con 0x7fcbf0007030 2023-10-05T19:40:25.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.203+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631ac70 con 0x55c2263139d0 2023-10-05T19:40:25.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.203+0000 7fcbfe107700 1 -- 172.21.15.12:0/3979373450 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbf0022020 con 0x55c2263139d0 2023-10-05T19:40:25.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.203+0000 7fcbfe107700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631ac70 con 0x55c226318350 2023-10-05T19:40:25.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.203+0000 7fcbf5906700 1 -- 172.21.15.12:0/3979373450 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbe8006070 con 0x55c226318350 2023-10-05T19:40:25.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.203+0000 7fcbf5906700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631ac70 con 0x55c2262de7e0 2023-10-05T19:40:25.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.207+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbf800a5a0 con 0x55c2262de7e0 2023-10-05T19:40:25.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.207+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631ac70 con 0x7fcbf0007030 2023-10-05T19:40:25.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.207+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbf800a5a0 con 0x7fcbf0007030 2023-10-05T19:40:25.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.207+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631ac70 con 0x55c2262de7e0 2023-10-05T19:40:25.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.207+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbf800a5a0 con 0x55c2262de7e0 2023-10-05T19:40:25.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.207+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631ac70 con 0x55c2263139d0 2023-10-05T19:40:25.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.207+0000 7fcbfe107700 1 -- 172.21.15.12:0/3979373450 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbf0022020 con 0x55c2263139d0 2023-10-05T19:40:25.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.207+0000 7fcbfe107700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631ac70 con 0x55c226318350 2023-10-05T19:40:25.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.207+0000 7fcbf5906700 1 -- 172.21.15.12:0/3979373450 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbe8006070 con 0x55c226318350 2023-10-05T19:40:25.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.207+0000 7fcbf5906700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631ac70 con 0x55c226318350 2023-10-05T19:40:25.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.207+0000 7fcbf5906700 1 -- 172.21.15.12:0/3979373450 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbe8006070 con 0x55c226318350 2023-10-05T19:40:25.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.207+0000 7fcbf5906700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631ac70 con 0x7fcbf0007030 2023-10-05T19:40:25.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.211+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbf800a5a0 con 0x7fcbf0007030 2023-10-05T19:40:25.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.211+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631ac70 con 0x55c226318350 2023-10-05T19:40:25.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.211+0000 7fcbf5906700 1 -- 172.21.15.12:0/3979373450 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbe8006070 con 0x55c226318350 2023-10-05T19:40:25.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.211+0000 7fcbf5906700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631ac70 con 0x55c226318350 2023-10-05T19:40:25.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.211+0000 7fcbf5906700 1 -- 172.21.15.12:0/3979373450 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbe8006070 con 0x55c226318350 2023-10-05T19:40:25.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.211+0000 7fcbf5906700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631ac70 con 0x55c2262de7e0 2023-10-05T19:40:25.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.211+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbf800a5a0 con 0x55c2262de7e0 2023-10-05T19:40:25.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.211+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631ac70 con 0x55c2262de7e0 2023-10-05T19:40:25.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.211+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbf800a5a0 con 0x55c2262de7e0 2023-10-05T19:40:25.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.211+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631ac70 con 0x7fcbf0007030 2023-10-05T19:40:25.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.215+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbf800a5a0 con 0x7fcbf0007030 2023-10-05T19:40:25.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.215+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631ac70 con 0x55c226318350 2023-10-05T19:40:25.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.215+0000 7fcbf5906700 1 -- 172.21.15.12:0/3979373450 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbe8006070 con 0x55c226318350 2023-10-05T19:40:25.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.215+0000 7fcbf5906700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631ac70 con 0x55c2263139d0 2023-10-05T19:40:25.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.215+0000 7fcbfe107700 1 -- 172.21.15.12:0/3979373450 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbf0022020 con 0x55c2263139d0 2023-10-05T19:40:25.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.215+0000 7fcbfe107700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631ac70 con 0x55c2262de7e0 2023-10-05T19:40:25.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.215+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbf800a5a0 con 0x55c2262de7e0 2023-10-05T19:40:25.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.215+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631ac70 con 0x7fcbf0007030 2023-10-05T19:40:25.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.219+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbf800a5a0 con 0x7fcbf0007030 2023-10-05T19:40:25.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.219+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631ac70 con 0x55c226318350 2023-10-05T19:40:25.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.219+0000 7fcbf5906700 1 -- 172.21.15.12:0/3979373450 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbe8006070 con 0x55c226318350 2023-10-05T19:40:25.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.219+0000 7fcbf5906700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631ac70 con 0x7fcbf0007030 2023-10-05T19:40:25.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.219+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbf800a5a0 con 0x7fcbf0007030 2023-10-05T19:40:25.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.219+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631ac70 con 0x55c2263139d0 2023-10-05T19:40:25.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.219+0000 7fcbfe107700 1 -- 172.21.15.12:0/3979373450 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbf0022020 con 0x55c2263139d0 2023-10-05T19:40:25.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.219+0000 7fcbfe107700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631ac70 con 0x55c226318350 2023-10-05T19:40:25.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.219+0000 7fcbf5906700 1 -- 172.21.15.12:0/3979373450 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbe8006070 con 0x55c226318350 2023-10-05T19:40:25.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.219+0000 7fcbf5906700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631ac70 con 0x7fcbf0007030 2023-10-05T19:40:25.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.223+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbf800a5a0 con 0x7fcbf0007030 2023-10-05T19:40:25.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.223+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631ac70 con 0x55c2262de7e0 2023-10-05T19:40:25.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.223+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fcbf800a5a0 con 0x55c2262de7e0 2023-10-05T19:40:25.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.223+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631ac70 con 0x55c2263139d0 2023-10-05T19:40:25.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.223+0000 7fcbfe107700 1 -- 172.21.15.12:0/3979373450 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fcbf0022020 con 0x55c2263139d0 2023-10-05T19:40:25.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.223+0000 7fcc005d2e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:25.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.223+0000 7fcc005d2e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:25.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.223+0000 7fcc005d2e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:25.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.223+0000 7fcc005d2e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:25.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.223+0000 7fcc005d2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.223+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c2263170d0 con 0x55c2262de7e0 2023-10-05T19:40:25.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.223+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fcbf800a5a0 con 0x55c2262de7e0 2023-10-05T19:40:25.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.223+0000 7fcc005d2e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:25.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.223+0000 7fcc005d2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.223+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c226317a20 con 0x55c2262de7e0 2023-10-05T19:40:25.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.227+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fcbf800a5a0 con 0x55c2262de7e0 2023-10-05T19:40:25.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.227+0000 7fcc005d2e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:25.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.227+0000 7fcc005d2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.227+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c226316c90 con 0x55c226318350 2023-10-05T19:40:25.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.227+0000 7fcbf5906700 1 -- 172.21.15.12:0/3979373450 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fcbe8006070 con 0x55c226318350 2023-10-05T19:40:25.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.227+0000 7fcc005d2e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:25.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.227+0000 7fcc005d2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.227+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631b5a0 con 0x55c226318350 2023-10-05T19:40:25.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.227+0000 7fcbf5906700 1 -- 172.21.15.12:0/3979373450 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7fcbe8006070 con 0x55c226318350 2023-10-05T19:40:25.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.227+0000 7fcc005d2e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:25.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.227+0000 7fcc005d2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.227+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631b950 con 0x55c2262de7e0 2023-10-05T19:40:25.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.227+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fcbf800a5a0 con 0x55c2262de7e0 2023-10-05T19:40:25.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.227+0000 7fcc005d2e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:25.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.227+0000 7fcc005d2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.227+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631bd00 con 0x55c2262de7e0 2023-10-05T19:40:25.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.227+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fcbf800a5a0 con 0x55c2262de7e0 2023-10-05T19:40:25.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.227+0000 7fcc005d2e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:25.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.227+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631cd10 con 0x55c226318350 2023-10-05T19:40:25.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.231+0000 7fcbf5906700 1 -- 172.21.15.12:0/3979373450 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fcbe8006070 con 0x55c226318350 2023-10-05T19:40:25.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.231+0000 7fcbf5906700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631cd10 con 0x55c2263139d0 2023-10-05T19:40:25.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.231+0000 7fcbfe107700 1 -- 172.21.15.12:0/3979373450 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbf0022020 con 0x55c2263139d0 2023-10-05T19:40:25.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.231+0000 7fcbfe107700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631cd10 con 0x7fcbf0007030 2023-10-05T19:40:25.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.231+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbf800a5a0 con 0x7fcbf0007030 2023-10-05T19:40:25.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.231+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631b950 con 0x55c2262de7e0 2023-10-05T19:40:25.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.235+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbf800a5a0 con 0x55c2262de7e0 2023-10-05T19:40:25.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.235+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631b950 con 0x55c2262de7e0 2023-10-05T19:40:25.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.235+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbf800a5a0 con 0x55c2262de7e0 2023-10-05T19:40:25.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.235+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631b950 con 0x7fcbf0007030 2023-10-05T19:40:25.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.235+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbf800a5a0 con 0x7fcbf0007030 2023-10-05T19:40:25.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.235+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631b950 con 0x7fcbf0007030 2023-10-05T19:40:25.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.235+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbf800a5a0 con 0x7fcbf0007030 2023-10-05T19:40:25.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.235+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631b950 con 0x55c2263139d0 2023-10-05T19:40:25.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.239+0000 7fcbfe107700 1 -- 172.21.15.12:0/3979373450 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbf0022020 con 0x55c2263139d0 2023-10-05T19:40:25.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.239+0000 7fcbfe107700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631b950 con 0x55c226318350 2023-10-05T19:40:25.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.239+0000 7fcbf5906700 1 -- 172.21.15.12:0/3979373450 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbe8006070 con 0x55c226318350 2023-10-05T19:40:25.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.239+0000 7fcbf5906700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631b950 con 0x55c2262de7e0 2023-10-05T19:40:25.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.239+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbf800a5a0 con 0x55c2262de7e0 2023-10-05T19:40:25.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.239+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631b950 con 0x7fcbf0007030 2023-10-05T19:40:25.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.239+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbf800a5a0 con 0x7fcbf0007030 2023-10-05T19:40:25.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.239+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631b950 con 0x55c2262de7e0 2023-10-05T19:40:25.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.239+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbf800a5a0 con 0x55c2262de7e0 2023-10-05T19:40:25.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.239+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631b950 con 0x55c2263139d0 2023-10-05T19:40:25.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.243+0000 7fcbfe107700 1 -- 172.21.15.12:0/3979373450 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbf0022020 con 0x55c2263139d0 2023-10-05T19:40:25.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.243+0000 7fcbfe107700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631b950 con 0x55c226318350 2023-10-05T19:40:25.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.243+0000 7fcbf5906700 1 -- 172.21.15.12:0/3979373450 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbe8006070 con 0x55c226318350 2023-10-05T19:40:25.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.243+0000 7fcbf5906700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631b950 con 0x55c226318350 2023-10-05T19:40:25.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.243+0000 7fcbf5906700 1 -- 172.21.15.12:0/3979373450 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbe8006070 con 0x55c226318350 2023-10-05T19:40:25.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.243+0000 7fcbf5906700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631b950 con 0x7fcbf0007030 2023-10-05T19:40:25.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.243+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbf800a5a0 con 0x7fcbf0007030 2023-10-05T19:40:25.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.243+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631b950 con 0x55c226318350 2023-10-05T19:40:25.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.247+0000 7fcbf5906700 1 -- 172.21.15.12:0/3979373450 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbe8006070 con 0x55c226318350 2023-10-05T19:40:25.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.247+0000 7fcbf5906700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631b950 con 0x55c226318350 2023-10-05T19:40:25.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.247+0000 7fcbf5906700 1 -- 172.21.15.12:0/3979373450 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbe8006070 con 0x55c226318350 2023-10-05T19:40:25.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.247+0000 7fcbf5906700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631b950 con 0x55c2262de7e0 2023-10-05T19:40:25.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.247+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbf800a5a0 con 0x55c2262de7e0 2023-10-05T19:40:25.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.247+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631b950 con 0x55c2262de7e0 2023-10-05T19:40:25.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.247+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbf800a5a0 con 0x55c2262de7e0 2023-10-05T19:40:25.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.247+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631b950 con 0x7fcbf0007030 2023-10-05T19:40:25.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.247+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbf800a5a0 con 0x7fcbf0007030 2023-10-05T19:40:25.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.247+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631b950 con 0x55c226318350 2023-10-05T19:40:25.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.251+0000 7fcbf5906700 1 -- 172.21.15.12:0/3979373450 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbe8006070 con 0x55c226318350 2023-10-05T19:40:25.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.251+0000 7fcbf5906700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631b950 con 0x55c2263139d0 2023-10-05T19:40:25.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.251+0000 7fcbfe107700 1 -- 172.21.15.12:0/3979373450 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbf0022020 con 0x55c2263139d0 2023-10-05T19:40:25.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.251+0000 7fcbfe107700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631b950 con 0x55c2262de7e0 2023-10-05T19:40:25.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.251+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbf800a5a0 con 0x55c2262de7e0 2023-10-05T19:40:25.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.251+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631b950 con 0x7fcbf0007030 2023-10-05T19:40:25.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.251+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbf800a5a0 con 0x7fcbf0007030 2023-10-05T19:40:25.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.251+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631b950 con 0x55c226318350 2023-10-05T19:40:25.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.255+0000 7fcbf5906700 1 -- 172.21.15.12:0/3979373450 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbe8006070 con 0x55c226318350 2023-10-05T19:40:25.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.255+0000 7fcbf5906700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631b950 con 0x7fcbf0007030 2023-10-05T19:40:25.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.255+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbf800a5a0 con 0x7fcbf0007030 2023-10-05T19:40:25.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.255+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631b950 con 0x55c2263139d0 2023-10-05T19:40:25.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.255+0000 7fcbfe107700 1 -- 172.21.15.12:0/3979373450 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbf0022020 con 0x55c2263139d0 2023-10-05T19:40:25.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.255+0000 7fcbfe107700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631b950 con 0x55c226318350 2023-10-05T19:40:25.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.255+0000 7fcbf5906700 1 -- 172.21.15.12:0/3979373450 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbe8006070 con 0x55c226318350 2023-10-05T19:40:25.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.255+0000 7fcbf5906700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631b950 con 0x7fcbf0007030 2023-10-05T19:40:25.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.259+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcbf800a5a0 con 0x7fcbf0007030 2023-10-05T19:40:25.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.259+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631b950 con 0x55c2262de7e0 2023-10-05T19:40:25.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.259+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fcbf800a5a0 con 0x55c2262de7e0 2023-10-05T19:40:25.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.259+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631b950 con 0x55c2263139d0 2023-10-05T19:40:25.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.259+0000 7fcbfe107700 1 -- 172.21.15.12:0/3979373450 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fcbf0022020 con 0x55c2263139d0 2023-10-05T19:40:25.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.259+0000 7fcc005d2e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:25.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.259+0000 7fcc005d2e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:25.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.259+0000 7fcc005d2e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:25.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.259+0000 7fcc005d2e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:25.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.259+0000 7fcc005d2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.259+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631bd00 con 0x55c2262de7e0 2023-10-05T19:40:25.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.259+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fcbf800a5a0 con 0x55c2262de7e0 2023-10-05T19:40:25.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.259+0000 7fcc005d2e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:25.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.259+0000 7fcc005d2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.259+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631c0b0 con 0x55c2263139d0 2023-10-05T19:40:25.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.263+0000 7fcbfe107700 1 -- 172.21.15.12:0/3979373450 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fcbf0022020 con 0x55c2263139d0 2023-10-05T19:40:25.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.263+0000 7fcc005d2e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:25.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.263+0000 7fcc005d2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.263+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631c950 con 0x55c2262de7e0 2023-10-05T19:40:25.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.263+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fcbf800a5a0 con 0x55c2262de7e0 2023-10-05T19:40:25.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.263+0000 7fcc005d2e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:25.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.263+0000 7fcc005d2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.263+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631c950 con 0x55c226318350 2023-10-05T19:40:25.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.263+0000 7fcbf5906700 1 -- 172.21.15.12:0/3979373450 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fcbe8006070 con 0x55c226318350 2023-10-05T19:40:25.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.263+0000 7fcc005d2e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:25.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.263+0000 7fcc005d2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.263+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631c490 con 0x7fcbf0007030 2023-10-05T19:40:25.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.263+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7fcbf800a5a0 con 0x7fcbf0007030 2023-10-05T19:40:25.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.263+0000 7fcc005d2e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:25.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.263+0000 7fcc005d2e40 20 zone default found 2023-10-05T19:40:25.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.263+0000 7fcc005d2e40 4 Realm: () 2023-10-05T19:40:25.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.263+0000 7fcc005d2e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:25.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.263+0000 7fcc005d2e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:25.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.263+0000 7fcc005d2e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:25.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.263+0000 7fcc005d2e40 20 zonegroup default 2023-10-05T19:40:25.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.263+0000 7fcc005d2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.263+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631c490 con 0x7fcbf0007030 2023-10-05T19:40:25.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.267+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fcbf800a5a0 con 0x7fcbf0007030 2023-10-05T19:40:25.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.267+0000 7fcc005d2e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:25.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.267+0000 7fcc005d2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.267+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631c490 con 0x55c226318350 2023-10-05T19:40:25.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.267+0000 7fcbf5906700 1 -- 172.21.15.12:0/3979373450 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fcbe8006070 con 0x55c226318350 2023-10-05T19:40:25.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.267+0000 7fcc005d2e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:25.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.267+0000 7fcc005d2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.267+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631eb70 con 0x55c226318350 2023-10-05T19:40:25.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.267+0000 7fcbf5906700 1 -- 172.21.15.12:0/3979373450 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fcbe8006070 con 0x55c226318350 2023-10-05T19:40:25.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.267+0000 7fcc005d2e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:25.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.267+0000 7fcc005d2e40 20 started sync module instance, tier type = 2023-10-05T19:40:25.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.267+0000 7fcc005d2e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:25.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.267+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631e230 con 0x55c2263139d0 2023-10-05T19:40:25.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.271+0000 7fcbfe107700 1 -- 172.21.15.12:0/3979373450 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'111 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fcbf0022020 con 0x55c2263139d0 2023-10-05T19:40:25.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.271+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94292352820832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631de60 con 0x55c2263139d0 2023-10-05T19:40:25.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.271+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631f900 con 0x55c2262de7e0 2023-10-05T19:40:25.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.275+0000 7fcbfe107700 1 -- 172.21.15.12:0/3979373450 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94292352820832] v28'112 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fcbf0022020 con 0x55c2263139d0 2023-10-05T19:40:25.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.275+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'111 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fcbf800a5a0 con 0x55c2262de7e0 2023-10-05T19:40:25.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.275+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94292352823072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631f860 con 0x55c2262de7e0 2023-10-05T19:40:25.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.275+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c226320100 con 0x55c226318350 2023-10-05T19:40:25.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.275+0000 7fcbf5906700 1 -- 172.21.15.12:0/3979373450 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'111 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fcbe8006070 con 0x55c226318350 2023-10-05T19:40:25.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.275+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94292352825728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c2263209a0 con 0x55c226318350 2023-10-05T19:40:25.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.275+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c226321280 con 0x55c2263139d0 2023-10-05T19:40:25.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.279+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94292352823072] v28'112 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fcbf800a5a0 con 0x55c2262de7e0 2023-10-05T19:40:25.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.279+0000 7fcbfe107700 1 -- 172.21.15.12:0/3979373450 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'111 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fcbf0022020 con 0x55c2263139d0 2023-10-05T19:40:25.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.279+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94292352824176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c226321680 con 0x55c2263139d0 2023-10-05T19:40:25.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.279+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c226321f20 con 0x55c2262de7e0 2023-10-05T19:40:25.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.279+0000 7fcbf5906700 1 -- 172.21.15.12:0/3979373450 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94292352825728] v28'112 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fcbe8006070 con 0x55c226318350 2023-10-05T19:40:25.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.283+0000 7fcbfe107700 1 -- 172.21.15.12:0/3979373450 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94292352824176] v28'112 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fcbf0022020 con 0x55c2263139d0 2023-10-05T19:40:25.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.283+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'111 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fcbf800a5a0 con 0x55c2262de7e0 2023-10-05T19:40:25.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.283+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94292352830800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c226321b70 con 0x55c2262de7e0 2023-10-05T19:40:25.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.283+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c226322410 con 0x7fcbf0007030 2023-10-05T19:40:25.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.283+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'111 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fcbf800a5a0 con 0x7fcbf0007030 2023-10-05T19:40:25.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.283+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94292352834688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c226322640 con 0x7fcbf0007030 2023-10-05T19:40:25.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.283+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c226323200 con 0x7fcbf0007030 2023-10-05T19:40:25.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.283+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94292352830800] v28'112 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fcbf800a5a0 con 0x55c2262de7e0 2023-10-05T19:40:25.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.287+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(96 notify.6 [create] v28'111 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fcbf800a5a0 con 0x7fcbf0007030 2023-10-05T19:40:25.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.287+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(95 notify.5 [watch watch cookie 94292352834688] v28'112 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fcbf800a5a0 con 0x7fcbf0007030 2023-10-05T19:40:25.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.287+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94292352833152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c2263231c0 con 0x7fcbf0007030 2023-10-05T19:40:25.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.287+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c2263236e0 con 0x7fcbf0007030 2023-10-05T19:40:25.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.291+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(97 notify.6 [watch watch cookie 94292352833152] v28'112 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fcbf800a5a0 con 0x7fcbf0007030 2023-10-05T19:40:25.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.291+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(98 notify.7 [create] v28'111 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fcbf800a5a0 con 0x7fcbf0007030 2023-10-05T19:40:25.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.291+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94292352838864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c226323120 con 0x7fcbf0007030 2023-10-05T19:40:25.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.291+0000 7fcc005d2e40 20 add_watcher() i=0 2023-10-05T19:40:25.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.291+0000 7fcc005d2e40 20 add_watcher() i=1 2023-10-05T19:40:25.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.291+0000 7fcc005d2e40 20 add_watcher() i=2 2023-10-05T19:40:25.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.291+0000 7fcc005d2e40 20 add_watcher() i=3 2023-10-05T19:40:25.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.291+0000 7fcc005d2e40 20 add_watcher() i=4 2023-10-05T19:40:25.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.291+0000 7fcc005d2e40 20 add_watcher() i=5 2023-10-05T19:40:25.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.291+0000 7fcc005d2e40 20 add_watcher() i=6 2023-10-05T19:40:25.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.291+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94292352838864] v28'112 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fcbf800a5a0 con 0x7fcbf0007030 2023-10-05T19:40:25.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.291+0000 7fcc005d2e40 20 add_watcher() i=7 2023-10-05T19:40:25.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.291+0000 7fcc005d2e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:25.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.291+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c226326c00 con 0x55c2262de7e0 2023-10-05T19:40:25.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.291+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fcbf800a5a0 con 0x55c2262de7e0 2023-10-05T19:40:25.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.291+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94292352841440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c226324020 con 0x55c2262de7e0 2023-10-05T19:40:25.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.295+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94292352841440] v28'109 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fcbf800a5a0 con 0x55c2262de7e0 2023-10-05T19:40:25.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.295+0000 7fcbcffdf700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:25.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.295+0000 7fcc005d2e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:25.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.295+0000 7fcc005d2e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:25.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.295+0000 7fcc005d2e40 1 Processor -- start 2023-10-05T19:40:25.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.295+0000 7fcc005d2e40 1 -- start start 2023-10-05T19:40:25.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.295+0000 7fcc005d2e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c226330dd0 0x55c2263311a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:25.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.295+0000 7fcc005d2e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2263316e0 con 0x55c226330dd0 2023-10-05T19:40:25.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.299+0000 7fcbf5906700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c226330dd0 0x55c2263311a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:25.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.299+0000 7fcbf5906700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c226330dd0 0x55c2263311a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57894/0 (socket says 172.21.15.12:57894) 2023-10-05T19:40:25.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.299+0000 7fcbf5906700 1 -- 172.21.15.12:0/1357785808 learned_addr learned my addr 172.21.15.12:0/1357785808 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:25.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.299+0000 7fcbf5906700 1 -- 172.21.15.12:0/1357785808 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c226331820 con 0x55c226330dd0 2023-10-05T19:40:25.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.299+0000 7fcbf5906700 1 --2- 172.21.15.12:0/1357785808 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c226330dd0 0x55c2263311a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8000c00 tx=0x7fcbe801e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=610e28b84cef98fc server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:25.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.299+0000 7fcbcf7de700 1 -- 172.21.15.12:0/1357785808 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcbe8013040 con 0x55c226330dd0 2023-10-05T19:40:25.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.299+0000 7fcbcf7de700 1 -- 172.21.15.12:0/1357785808 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe8022040 con 0x55c226330dd0 2023-10-05T19:40:25.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.299+0000 7fcbcf7de700 1 -- 172.21.15.12:0/1357785808 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcbe800e950 con 0x55c226330dd0 2023-10-05T19:40:25.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.299+0000 7fcc005d2e40 1 -- 172.21.15.12:0/1357785808 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c226330dd0 msgr2=0x55c2263311a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.299+0000 7fcc005d2e40 1 --2- 172.21.15.12:0/1357785808 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c226330dd0 0x55c2263311a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8000c00 tx=0x7fcbe801e040 comp rx=0 tx=0).stop 2023-10-05T19:40:25.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.299+0000 7fcc005d2e40 1 -- 172.21.15.12:0/1357785808 shutdown_connections 2023-10-05T19:40:25.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.299+0000 7fcc005d2e40 1 --2- 172.21.15.12:0/1357785808 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c226330dd0 0x55c2263311a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:25.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.299+0000 7fcc005d2e40 1 -- 172.21.15.12:0/1357785808 >> 172.21.15.12:0/1357785808 conn(0x55c22632c190 msgr2=0x55c22632a340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:25.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.299+0000 7fcc005d2e40 1 -- 172.21.15.12:0/1357785808 shutdown_connections 2023-10-05T19:40:25.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.299+0000 7fcc005d2e40 1 -- 172.21.15.12:0/1357785808 wait complete. 2023-10-05T19:40:25.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.299+0000 7fcc005d2e40 1 Processor -- start 2023-10-05T19:40:25.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.299+0000 7fcc005d2e40 1 -- start start 2023-10-05T19:40:25.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.299+0000 7fcc005d2e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c226330dd0 0x55c226336960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:25.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.299+0000 7fcc005d2e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c226337960 con 0x55c226330dd0 2023-10-05T19:40:25.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.303+0000 7fcbf5906700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c226330dd0 0x55c226336960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:25.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.303+0000 7fcbf5906700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c226330dd0 0x55c226336960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57910/0 (socket says 172.21.15.12:57910) 2023-10-05T19:40:25.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.303+0000 7fcbf5906700 1 -- 172.21.15.12:0/3150076531 learned_addr learned my addr 172.21.15.12:0/3150076531 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:25.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.303+0000 7fcbf5906700 1 -- 172.21.15.12:0/3150076531 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbe8034040 con 0x55c226330dd0 2023-10-05T19:40:25.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.303+0000 7fcbf5906700 1 --2- 172.21.15.12:0/3150076531 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c226330dd0 0x55c226336960 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8045000 tx=0x7fcbe803d030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:25.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.303+0000 7fcbcdfdb700 1 -- 172.21.15.12:0/3150076531 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcbe803ce30 con 0x55c226330dd0 2023-10-05T19:40:25.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.303+0000 7fcbcdfdb700 1 -- 172.21.15.12:0/3150076531 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe802c640 con 0x55c226330dd0 2023-10-05T19:40:25.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.303+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3150076531 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c226336f60 con 0x55c226330dd0 2023-10-05T19:40:25.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.303+0000 7fcbcdfdb700 1 -- 172.21.15.12:0/3150076531 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcbe802c7f0 con 0x55c226330dd0 2023-10-05T19:40:25.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.303+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3150076531 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2263373a0 con 0x55c226330dd0 2023-10-05T19:40:25.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.303+0000 7fcbcdfdb700 1 -- 172.21.15.12:0/3150076531 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fcbe804d070 con 0x55c226330dd0 2023-10-05T19:40:25.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.303+0000 7fcbcdfdb700 1 --2- 172.21.15.12:0/3150076531 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fcb44030ca0 0x7fcb44033150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:25.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.303+0000 7fcbcdfdb700 1 -- 172.21.15.12:0/3150076531 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7fcbe8059b00 con 0x55c226330dd0 2023-10-05T19:40:25.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.303+0000 7fcbfe908700 1 --2- 172.21.15.12:0/3150076531 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fcb44030ca0 0x7fcb44033150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:25.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.307+0000 7fcc005d2e40 5 note: GC not initialized 2023-10-05T19:40:25.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.307+0000 7fcbfe908700 1 --2- 172.21.15.12:0/3150076531 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fcb44030ca0 0x7fcb44033150 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x55c226323280 tx=0x7fcbf800ae20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:25.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.307+0000 7fcb3effd700 20 reqs_thread_entry: start 2023-10-05T19:40:25.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.307+0000 7fcc005d2e40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:25.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.307+0000 7fcb3cff9700 20 reqs_thread_entry: start 2023-10-05T19:40:25.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.307+0000 7fcc005d2e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:25.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.307+0000 7fcb2effd700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:55 2023 2023-10-05T19:40:25.317 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:25.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.307+0000 7fcb2e7fc700 20 reqs_thread_entry: start 2023-10-05T19:40:25.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.307+0000 7fcc005d2e40 20 get_system_obj_state: rctx=0x7ffdffac3a08 obj=default.rgw.meta:root:myfoo state=0x55c2261b70b0 s->prefetch_data=0 2023-10-05T19:40:25.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.307+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:102 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22639ecb0 con 0x55c2263139d0 2023-10-05T19:40:25.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.311+0000 7fcbfe107700 1 -- 172.21.15.12:0/3979373450 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(102 myfoo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv18 ondisk = 0) v8 ==== 233+0+139 (crc 0 0 0) 0x7fcbf0022020 con 0x55c2263139d0 2023-10-05T19:40:25.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.311+0000 7fcc005d2e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:25.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.311+0000 7fcc005d2e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:25.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.311+0000 7fcc005d2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.311+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:103 8.9 8:91b71841:root::myfoo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22639ec60 con 0x55c2263139d0 2023-10-05T19:40:25.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.311+0000 7fcbfe107700 1 -- 172.21.15.12:0/3979373450 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(103 myfoo [call,call out=48b,read 0~165 out=165b,getxattrs out=75b] v0'0 uv18 ondisk = 0) v8 ==== 275+0+288 (crc 0 0 0) 0x7fcbf0022020 con 0x55c2263139d0 2023-10-05T19:40:25.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.311+0000 7fcc005d2e40 20 rados_obj.operate() r=0 bl.length=165 2023-10-05T19:40:25.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.311+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:104 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94292352820832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c2262f5a30 con 0x55c2263139d0 2023-10-05T19:40:25.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.315+0000 7fcbfe107700 1 -- 172.21.15.12:0/3979373450 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(104 notify.0 [watch unwatch cookie 94292352820832] v28'113 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fcbf0022020 con 0x55c2263139d0 2023-10-05T19:40:25.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.315+0000 7fcc005d2e40 20 remove_watcher() i=0 2023-10-05T19:40:25.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.315+0000 7fcc005d2e40 2 removed watcher, disabling cache 2023-10-05T19:40:25.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.315+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:105 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94292352823072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631e660 con 0x55c2262de7e0 2023-10-05T19:40:25.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.315+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(105 notify.1 [watch unwatch cookie 94292352823072] v28'113 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fcbf803e070 con 0x55c2262de7e0 2023-10-05T19:40:25.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.315+0000 7fcc005d2e40 20 remove_watcher() i=1 2023-10-05T19:40:25.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.315+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:106 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94292352825728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631e660 con 0x55c226318350 2023-10-05T19:40:25.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.319+0000 7fcbf5906700 1 -- 172.21.15.12:0/3979373450 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(106 notify.2 [watch unwatch cookie 94292352825728] v28'113 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fcbe8006070 con 0x55c226318350 2023-10-05T19:40:25.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.319+0000 7fcc005d2e40 20 remove_watcher() i=2 2023-10-05T19:40:25.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.319+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:107 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94292352824176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631e660 con 0x55c2263139d0 2023-10-05T19:40:25.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.323+0000 7fcbfe107700 1 -- 172.21.15.12:0/3979373450 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(107 notify.3 [watch unwatch cookie 94292352824176] v28'113 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fcbf0022020 con 0x55c2263139d0 2023-10-05T19:40:25.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.323+0000 7fcc005d2e40 20 remove_watcher() i=3 2023-10-05T19:40:25.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.323+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:108 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94292352830800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631f410 con 0x55c2262de7e0 2023-10-05T19:40:25.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.323+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(108 notify.4 [watch unwatch cookie 94292352830800] v28'113 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fcbf803e070 con 0x55c2262de7e0 2023-10-05T19:40:25.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.323+0000 7fcc005d2e40 20 remove_watcher() i=4 2023-10-05T19:40:25.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.323+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:109 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94292352834688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631f410 con 0x7fcbf0007030 2023-10-05T19:40:25.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.327+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(109 notify.5 [watch unwatch cookie 94292352834688] v28'113 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fcbf803e070 con 0x7fcbf0007030 2023-10-05T19:40:25.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.327+0000 7fcc005d2e40 20 remove_watcher() i=5 2023-10-05T19:40:25.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.327+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:110 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94292352833152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631f410 con 0x7fcbf0007030 2023-10-05T19:40:25.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.331+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(110 notify.6 [watch unwatch cookie 94292352833152] v28'113 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fcbf803e070 con 0x7fcbf0007030 2023-10-05T19:40:25.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.331+0000 7fcc005d2e40 20 remove_watcher() i=6 2023-10-05T19:40:25.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.331+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:111 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94292352838864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c22631f410 con 0x7fcbf0007030 2023-10-05T19:40:25.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.335+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(111 notify.7 [watch unwatch cookie 94292352838864] v28'113 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fcbf803e070 con 0x7fcbf0007030 2023-10-05T19:40:25.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.335+0000 7fcc005d2e40 20 remove_watcher() i=7 2023-10-05T19:40:25.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.335+0000 7fcbcffdf700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:25.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.335+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:112 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94292352841440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c226322cd0 con 0x55c2262de7e0 2023-10-05T19:40:25.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.339+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(112 data_loggenerations_metadata [watch unwatch cookie 94292352841440] v28'110 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fcbf803e070 con 0x55c2262de7e0 2023-10-05T19:40:25.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.339+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fcbf0007030 msgr2=0x7fcbf00119f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.339+0000 7fcc005d2e40 1 --2- 172.21.15.12:0/3979373450 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fcbf0007030 0x7fcbf00119f0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:25.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.339+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55c2262de7e0 msgr2=0x55c226311830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.339+0000 7fcc005d2e40 1 --2- 172.21.15.12:0/3979373450 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55c2262de7e0 0x55c226311830 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:25.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.339+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55c2263139d0 msgr2=0x55c226315db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.339+0000 7fcc005d2e40 1 --2- 172.21.15.12:0/3979373450 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55c2263139d0 0x55c226315db0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:25.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.339+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55c226318350 msgr2=0x55c22631a730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.339+0000 7fcc005d2e40 1 --2- 172.21.15.12:0/3979373450 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55c226318350 0x55c22631a730 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:25.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.339+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fcbe0030b10 msgr2=0x7fcbe0032fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.339+0000 7fcc005d2e40 1 --2- 172.21.15.12:0/3979373450 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fcbe0030b10 0x7fcbe0032fc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8004170 tx=0x7fcbe8001380 comp rx=0 tx=0).stop 2023-10-05T19:40:25.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.339+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c2262de410 msgr2=0x55c226305de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.339+0000 7fcc005d2e40 1 --2- 172.21.15.12:0/3979373450 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c2262de410 0x55c226305de0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0000c00 tx=0x7fcbf0003050 comp rx=0 tx=0).stop 2023-10-05T19:40:25.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.339+0000 7fcbfe908700 1 -- 172.21.15.12:0/3979373450 reap_dead start 2023-10-05T19:40:25.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.339+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 shutdown_connections 2023-10-05T19:40:25.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.339+0000 7fcc005d2e40 1 --2- 172.21.15.12:0/3979373450 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c2262de410 0x55c226305de0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:25.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.339+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 >> 172.21.15.12:0/3979373450 conn(0x55c2262d8720 msgr2=0x55c2262f97c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:25.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.339+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 shutdown_connections 2023-10-05T19:40:25.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.339+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3979373450 wait complete. 2023-10-05T19:40:25.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.343+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3150076531 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fcb44030ca0 msgr2=0x7fcb44033150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.343+0000 7fcc005d2e40 1 --2- 172.21.15.12:0/3150076531 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fcb44030ca0 0x7fcb44033150 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x55c226323280 tx=0x7fcbf800ae20 comp rx=0 tx=0).stop 2023-10-05T19:40:25.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.343+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3150076531 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c226330dd0 msgr2=0x55c226336960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.343+0000 7fcc005d2e40 1 --2- 172.21.15.12:0/3150076531 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c226330dd0 0x55c226336960 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8045000 tx=0x7fcbe803d030 comp rx=0 tx=0).stop 2023-10-05T19:40:25.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.343+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3150076531 shutdown_connections 2023-10-05T19:40:25.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.343+0000 7fcc005d2e40 1 --2- 172.21.15.12:0/3150076531 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fcb44030ca0 0x7fcb44033150 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:25.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.343+0000 7fcc005d2e40 1 --2- 172.21.15.12:0/3150076531 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c226330dd0 0x55c226336960 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:25.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.343+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3150076531 >> 172.21.15.12:0/3150076531 conn(0x55c22632c190 msgr2=0x55c22632a290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:25.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.343+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3150076531 shutdown_connections 2023-10-05T19:40:25.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.343+0000 7fcc005d2e40 1 -- 172.21.15.12:0/3150076531 wait complete. 2023-10-05T19:40:25.353 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:40:25.353 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "bucket:myfoo", 2023-10-05T19:40:25.353 INFO:teuthology.orchestra.run.smithi012.stdout: "ver": { 2023-10-05T19:40:25.353 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "_4M-KAz5vblL6UbhYbNgE0ST", 2023-10-05T19:40:25.353 INFO:teuthology.orchestra.run.smithi012.stdout: "ver": 2 2023-10-05T19:40:25.354 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:25.354 INFO:teuthology.orchestra.run.smithi012.stdout: "mtime": "2023-10-05T19:40:24.859387Z", 2023-10-05T19:40:25.354 INFO:teuthology.orchestra.run.smithi012.stdout: "data": { 2023-10-05T19:40:25.354 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": { 2023-10-05T19:40:25.354 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "myfoo", 2023-10-05T19:40:25.354 INFO:teuthology.orchestra.run.smithi012.stdout: "marker": "93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1", 2023-10-05T19:40:25.354 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket_id": "93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1", 2023-10-05T19:40:25.354 INFO:teuthology.orchestra.run.smithi012.stdout: "tenant": "", 2023-10-05T19:40:25.354 INFO:teuthology.orchestra.run.smithi012.stdout: "explicit_placement": { 2023-10-05T19:40:25.354 INFO:teuthology.orchestra.run.smithi012.stdout: "data_pool": "", 2023-10-05T19:40:25.354 INFO:teuthology.orchestra.run.smithi012.stdout: "data_extra_pool": "", 2023-10-05T19:40:25.355 INFO:teuthology.orchestra.run.smithi012.stdout: "index_pool": "" 2023-10-05T19:40:25.355 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:25.355 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:25.355 INFO:teuthology.orchestra.run.smithi012.stdout: "owner": "foo", 2023-10-05T19:40:25.355 INFO:teuthology.orchestra.run.smithi012.stdout: "creation_time": "2023-10-05T19:40:22.857926Z", 2023-10-05T19:40:25.355 INFO:teuthology.orchestra.run.smithi012.stdout: "linked": "false", 2023-10-05T19:40:25.355 INFO:teuthology.orchestra.run.smithi012.stdout: "has_bucket_info": "false" 2023-10-05T19:40:25.355 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:25.355 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:40:25.356 DEBUG:tasks.util.rgw: json result: {'key': 'bucket:myfoo', 'ver': {'tag': '_4M-KAz5vblL6UbhYbNgE0ST', 'ver': 2}, 'mtime': '2023-10-05T19:40:24.859387Z', 'data': {'bucket': {'name': 'myfoo', 'marker': '93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1', 'bucket_id': '93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1', 'tenant': '', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}}, 'owner': 'foo', 'creation_time': '2023-10-05T19:40:22.857926Z', 'linked': 'false', 'has_bucket_info': 'false'}} 2023-10-05T19:40:25.356 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'link', '--uid', 'fud', '--bucket', 'myfoo', '--bucket-id', '93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1'] 2023-10-05T19:40:25.356 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'link', '--uid', 'fud', '--bucket', 'myfoo', '--bucket-id', '93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1'] 2023-10-05T19:40:25.357 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket link --uid fud --bucket myfoo --bucket-id 93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 2023-10-05T19:40:25.441 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:25.441 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:25.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.435+0000 7f1a844aae40 1 Processor -- start 2023-10-05T19:40:25.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.435+0000 7f1a844aae40 1 -- start start 2023-10-05T19:40:25.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.435+0000 7f1a844aae40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x558818337bc0 0x5588183361a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:25.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.435+0000 7f1a844aae40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55881830ebd0 con 0x5588183366e0 2023-10-05T19:40:25.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.435+0000 7f1a844aae40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x5588182cbee0 con 0x558818337bc0 2023-10-05T19:40:25.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.435+0000 7f1a81fdf700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x558818337bc0 0x5588183361a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:25.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.435+0000 7f1a817de700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x5588183366e0 0x5588183ce6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:35542/0 (socket says 172.21.15.12:35542) 2023-10-05T19:40:25.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.435+0000 7f1a817de700 1 -- 172.21.15.12:0/2258708003 learned_addr learned my addr 172.21.15.12:0/2258708003 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:25.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.435+0000 7f1a80fdd700 1 -- 172.21.15.12:0/2258708003 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2586986158 0 0) 0x55881830ebd0 con 0x5588183366e0 2023-10-05T19:40:25.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.435+0000 7f1a80fdd700 1 -- 172.21.15.12:0/2258708003 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a6c001e10 con 0x5588183366e0 2023-10-05T19:40:25.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.435+0000 7f1a81fdf700 1 -- 172.21.15.12:0/2258708003 >> v1:172.21.15.8:6789/0 conn(0x5588183366e0 legacy=0x5588183ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.435+0000 7f1a81fdf700 1 -- 172.21.15.12:0/2258708003 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558818338050 con 0x558818337bc0 2023-10-05T19:40:25.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.435+0000 7f1a81fdf700 1 --2- 172.21.15.12:0/2258708003 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558818337bc0 0x5588183361a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1a7400a5d0 tx=0x7f1a74005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe7d8c2ecf0a69c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:25.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.435+0000 7f1a80fdd700 1 -- 172.21.15.12:0/2258708003 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a74007610 con 0x558818337bc0 2023-10-05T19:40:25.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.435+0000 7f1a80fdd700 1 -- 172.21.15.12:0/2258708003 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a74007770 con 0x558818337bc0 2023-10-05T19:40:25.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.435+0000 7f1a80fdd700 1 -- 172.21.15.12:0/2258708003 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a74007a40 con 0x558818337bc0 2023-10-05T19:40:25.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.435+0000 7f1a844aae40 1 -- 172.21.15.12:0/2258708003 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558818337bc0 msgr2=0x5588183361a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.435+0000 7f1a844aae40 1 --2- 172.21.15.12:0/2258708003 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558818337bc0 0x5588183361a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1a7400a5d0 tx=0x7f1a74005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:25.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.435+0000 7f1a844aae40 1 -- 172.21.15.12:0/2258708003 shutdown_connections 2023-10-05T19:40:25.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.435+0000 7f1a844aae40 1 --2- 172.21.15.12:0/2258708003 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558818337bc0 0x5588183361a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:25.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.435+0000 7f1a844aae40 1 -- 172.21.15.12:0/2258708003 >> 172.21.15.12:0/2258708003 conn(0x5588183bd7f0 msgr2=0x5588183bfc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:25.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.435+0000 7f1a844aae40 1 -- 172.21.15.12:0/2258708003 shutdown_connections 2023-10-05T19:40:25.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.435+0000 7f1a844aae40 1 -- 172.21.15.12:0/2258708003 wait complete. 2023-10-05T19:40:25.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.439+0000 7f1a844aae40 1 Processor -- start 2023-10-05T19:40:25.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.439+0000 7f1a844aae40 1 -- start start 2023-10-05T19:40:25.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.439+0000 7f1a844aae40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5588183366e0 0x5588183e7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:25.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.439+0000 7f1a844aae40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55881828fe30 con 0x5588183366e0 2023-10-05T19:40:25.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.439+0000 7f1a81fdf700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5588183366e0 0x5588183e7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:25.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.439+0000 7f1a81fdf700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5588183366e0 0x5588183e7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57932/0 (socket says 172.21.15.12:57932) 2023-10-05T19:40:25.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.439+0000 7f1a81fdf700 1 -- 172.21.15.12:0/1804110534 learned_addr learned my addr 172.21.15.12:0/1804110534 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:25.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.439+0000 7f1a81fdf700 1 -- 172.21.15.12:0/1804110534 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a74005cf0 con 0x5588183366e0 2023-10-05T19:40:25.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.439+0000 7f1a81fdf700 1 --2- 172.21.15.12:0/1804110534 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5588183366e0 0x5588183e7400 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1a74000c00 tx=0x7f1a74014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a6e558c59ac5f59 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:25.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.439+0000 7f1a737fe700 1 -- 172.21.15.12:0/1804110534 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a740078e0 con 0x5588183366e0 2023-10-05T19:40:25.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.439+0000 7f1a737fe700 1 -- 172.21.15.12:0/1804110534 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a74007a40 con 0x5588183366e0 2023-10-05T19:40:25.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.439+0000 7f1a737fe700 1 -- 172.21.15.12:0/1804110534 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a74007bd0 con 0x5588183366e0 2023-10-05T19:40:25.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.439+0000 7f1a844aae40 1 -- 172.21.15.12:0/1804110534 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5588183366e0 msgr2=0x5588183e7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.439+0000 7f1a844aae40 1 --2- 172.21.15.12:0/1804110534 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5588183366e0 0x5588183e7400 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1a74000c00 tx=0x7f1a74014000 comp rx=0 tx=0).stop 2023-10-05T19:40:25.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.439+0000 7f1a844aae40 1 -- 172.21.15.12:0/1804110534 shutdown_connections 2023-10-05T19:40:25.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.439+0000 7f1a844aae40 1 --2- 172.21.15.12:0/1804110534 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5588183366e0 0x5588183e7400 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:25.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.439+0000 7f1a844aae40 1 -- 172.21.15.12:0/1804110534 >> 172.21.15.12:0/1804110534 conn(0x5588183bd7f0 msgr2=0x5588183e2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:25.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.439+0000 7f1a844aae40 1 -- 172.21.15.12:0/1804110534 shutdown_connections 2023-10-05T19:40:25.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.439+0000 7f1a844aae40 1 -- 172.21.15.12:0/1804110534 wait complete. 2023-10-05T19:40:25.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.439+0000 7f1a844aae40 1 Processor -- start 2023-10-05T19:40:25.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.439+0000 7f1a844aae40 1 -- start start 2023-10-05T19:40:25.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.443+0000 7f1a844aae40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5588183366e0 0x5588183ee160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:25.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.443+0000 7f1a844aae40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x558818262ed0 con 0x5588183366e0 2023-10-05T19:40:25.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.443+0000 7f1a81fdf700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5588183366e0 0x5588183ee160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:25.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.443+0000 7f1a81fdf700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5588183366e0 0x5588183ee160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57948/0 (socket says 172.21.15.12:57948) 2023-10-05T19:40:25.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.443+0000 7f1a81fdf700 1 -- 172.21.15.12:0/111040976 learned_addr learned my addr 172.21.15.12:0/111040976 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:25.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.443+0000 7f1a81fdf700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558818338050 con 0x5588183366e0 2023-10-05T19:40:25.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.443+0000 7f1a81fdf700 1 --2- 172.21.15.12:0/111040976 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5588183366e0 0x5588183ee160 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1a7401f040 tx=0x7f1a7401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:25.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.443+0000 7f1a71ffb700 1 -- 172.21.15.12:0/111040976 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a740078e0 con 0x5588183366e0 2023-10-05T19:40:25.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.443+0000 7f1a71ffb700 1 -- 172.21.15.12:0/111040976 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a74026650 con 0x5588183366e0 2023-10-05T19:40:25.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.443+0000 7f1a71ffb700 1 -- 172.21.15.12:0/111040976 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a74026820 con 0x5588183366e0 2023-10-05T19:40:25.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.443+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5588183ee6a0 con 0x5588183366e0 2023-10-05T19:40:25.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.443+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5588183eeae0 con 0x5588183366e0 2023-10-05T19:40:25.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.443+0000 7f1a71ffb700 1 -- 172.21.15.12:0/111040976 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f1a740269d0 con 0x5588183366e0 2023-10-05T19:40:25.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.447+0000 7f1a71ffb700 1 --2- 172.21.15.12:0/111040976 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1a64030c40 0x7f1a640330f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:25.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.447+0000 7f1a71ffb700 1 -- 172.21.15.12:0/111040976 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f1a74046b30 con 0x5588183366e0 2023-10-05T19:40:25.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.447+0000 7f1a817de700 1 --2- 172.21.15.12:0/111040976 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1a64030c40 0x7f1a640330f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:25.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.447+0000 7f1a844aae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.447+0000 7f1a844aae40 1 --2- 172.21.15.12:0/111040976 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x558818337bc0 0x5588183f9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:25.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.447+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5588183fa2e0 con 0x558818337bc0 2023-10-05T19:40:25.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.447+0000 7f1a827e0700 1 --2- 172.21.15.12:0/111040976 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x558818337bc0 0x5588183f9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:25.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.447+0000 7f1a817de700 1 --2- 172.21.15.12:0/111040976 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1a64030c40 0x7f1a640330f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1a780044f0 tx=0x7f1a78006690 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:25.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.447+0000 7f1a827e0700 1 --2- 172.21.15.12:0/111040976 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x558818337bc0 0x5588183f9bb0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:25.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.447+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1a7c00a500 con 0x558818337bc0 2023-10-05T19:40:25.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.447+0000 7f1a844aae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:25.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.447+0000 7f1a844aae40 20 realm 2023-10-05T19:40:25.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.447+0000 7f1a844aae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.447+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5588183f71e0 con 0x558818337bc0 2023-10-05T19:40:25.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.447+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1a7c00a500 con 0x558818337bc0 2023-10-05T19:40:25.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.447+0000 7f1a844aae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:25.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.447+0000 7f1a844aae40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:25.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.447+0000 7f1a844aae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.447+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5588183fae10 con 0x558818337bc0 2023-10-05T19:40:25.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.447+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1a7c00a500 con 0x558818337bc0 2023-10-05T19:40:25.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.447+0000 7f1a844aae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:25.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.447+0000 7f1a844aae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.447+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5588183faa20 con 0x558818337bc0 2023-10-05T19:40:25.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.451+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f1a7c00a500 con 0x558818337bc0 2023-10-05T19:40:25.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.451+0000 7f1a844aae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:25.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.451+0000 7f1a844aae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.451+0000 7f1a844aae40 1 --2- 172.21.15.12:0/111040976 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5588183fc1b0 0x5588183fe590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:25.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.451+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5588183fead0 con 0x5588183fc1b0 2023-10-05T19:40:25.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.451+0000 7f1a81fdf700 1 --2- 172.21.15.12:0/111040976 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5588183fc1b0 0x5588183fe590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:25.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.451+0000 7f1a81fdf700 1 --2- 172.21.15.12:0/111040976 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5588183fc1b0 0x5588183fe590 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:25.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.451+0000 7f1a81fdf700 1 -- 172.21.15.12:0/111040976 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f1a74012a10 con 0x5588183fc1b0 2023-10-05T19:40:25.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.451+0000 7f1a844aae40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:25.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.451+0000 7f1a844aae40 20 searching for the correct realm 2023-10-05T19:40:25.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.451+0000 7f1a844aae40 1 --2- 172.21.15.12:0/111040976 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x558818400950 0x558818402d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:25.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.451+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403410 con 0x558818400950 2023-10-05T19:40:25.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.451+0000 7f1a817de700 1 --2- 172.21.15.12:0/111040976 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x558818400950 0x558818402d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:25.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.451+0000 7f1a817de700 1 --2- 172.21.15.12:0/111040976 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x558818400950 0x558818402d60 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:25.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.455+0000 7f1a817de700 1 -- 172.21.15.12:0/111040976 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f1a78004520 con 0x558818400950 2023-10-05T19:40:25.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.455+0000 7f1a817de700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403410 con 0x5588183fc1b0 2023-10-05T19:40:25.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.455+0000 7f1a81fdf700 1 -- 172.21.15.12:0/111040976 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a74012a10 con 0x5588183fc1b0 2023-10-05T19:40:25.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.455+0000 7f1a81fdf700 1 --2- 172.21.15.12:0/111040976 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f1a74007030 0x7f1a7401e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:25.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.455+0000 7f1a81fdf700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403410 con 0x7f1a74007030 2023-10-05T19:40:25.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.455+0000 7f1a827e0700 1 --2- 172.21.15.12:0/111040976 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f1a74007030 0x7f1a7401e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:25.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.455+0000 7f1a827e0700 1 --2- 172.21.15.12:0/111040976 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f1a74007030 0x7f1a7401e8f0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:25.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.455+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a7c014040 con 0x7f1a74007030 2023-10-05T19:40:25.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.455+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403410 con 0x558818337bc0 2023-10-05T19:40:25.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.459+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a7c014040 con 0x558818337bc0 2023-10-05T19:40:25.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.459+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403410 con 0x558818337bc0 2023-10-05T19:40:25.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.459+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a7c014040 con 0x558818337bc0 2023-10-05T19:40:25.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.459+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403410 con 0x7f1a74007030 2023-10-05T19:40:25.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.459+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a7c014040 con 0x7f1a74007030 2023-10-05T19:40:25.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.459+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403410 con 0x7f1a74007030 2023-10-05T19:40:25.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.459+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a7c014040 con 0x7f1a74007030 2023-10-05T19:40:25.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.459+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403410 con 0x5588183fc1b0 2023-10-05T19:40:25.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.459+0000 7f1a81fdf700 1 -- 172.21.15.12:0/111040976 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a74012a10 con 0x5588183fc1b0 2023-10-05T19:40:25.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.459+0000 7f1a81fdf700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403410 con 0x558818400950 2023-10-05T19:40:25.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.463+0000 7f1a817de700 1 -- 172.21.15.12:0/111040976 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a78004520 con 0x558818400950 2023-10-05T19:40:25.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.463+0000 7f1a817de700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403410 con 0x558818337bc0 2023-10-05T19:40:25.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.463+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a7c014040 con 0x558818337bc0 2023-10-05T19:40:25.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.463+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403410 con 0x7f1a74007030 2023-10-05T19:40:25.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.463+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a7c014040 con 0x7f1a74007030 2023-10-05T19:40:25.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.463+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403410 con 0x558818337bc0 2023-10-05T19:40:25.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.463+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a7c014040 con 0x558818337bc0 2023-10-05T19:40:25.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.463+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403410 con 0x5588183fc1b0 2023-10-05T19:40:25.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.463+0000 7f1a81fdf700 1 -- 172.21.15.12:0/111040976 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a74012a10 con 0x5588183fc1b0 2023-10-05T19:40:25.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.463+0000 7f1a81fdf700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403410 con 0x558818400950 2023-10-05T19:40:25.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.463+0000 7f1a817de700 1 -- 172.21.15.12:0/111040976 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a78004520 con 0x558818400950 2023-10-05T19:40:25.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.463+0000 7f1a817de700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403410 con 0x558818400950 2023-10-05T19:40:25.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.467+0000 7f1a817de700 1 -- 172.21.15.12:0/111040976 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a78004520 con 0x558818400950 2023-10-05T19:40:25.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.467+0000 7f1a817de700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403410 con 0x7f1a74007030 2023-10-05T19:40:25.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.467+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a7c014040 con 0x7f1a74007030 2023-10-05T19:40:25.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.467+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403410 con 0x558818400950 2023-10-05T19:40:25.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.467+0000 7f1a817de700 1 -- 172.21.15.12:0/111040976 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a78004520 con 0x558818400950 2023-10-05T19:40:25.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.467+0000 7f1a817de700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403410 con 0x558818400950 2023-10-05T19:40:25.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.467+0000 7f1a817de700 1 -- 172.21.15.12:0/111040976 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a78004520 con 0x558818400950 2023-10-05T19:40:25.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.467+0000 7f1a817de700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403410 con 0x558818337bc0 2023-10-05T19:40:25.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.467+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a7c014040 con 0x558818337bc0 2023-10-05T19:40:25.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.467+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403410 con 0x558818337bc0 2023-10-05T19:40:25.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.471+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a7c014040 con 0x558818337bc0 2023-10-05T19:40:25.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.471+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403410 con 0x7f1a74007030 2023-10-05T19:40:25.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.471+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a7c014040 con 0x7f1a74007030 2023-10-05T19:40:25.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.471+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403410 con 0x558818400950 2023-10-05T19:40:25.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.471+0000 7f1a817de700 1 -- 172.21.15.12:0/111040976 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a78004520 con 0x558818400950 2023-10-05T19:40:25.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.471+0000 7f1a817de700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403410 con 0x5588183fc1b0 2023-10-05T19:40:25.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.471+0000 7f1a81fdf700 1 -- 172.21.15.12:0/111040976 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a74012a10 con 0x5588183fc1b0 2023-10-05T19:40:25.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.471+0000 7f1a81fdf700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403410 con 0x558818337bc0 2023-10-05T19:40:25.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.471+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a7c014040 con 0x558818337bc0 2023-10-05T19:40:25.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.471+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403410 con 0x7f1a74007030 2023-10-05T19:40:25.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.475+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a7c014040 con 0x7f1a74007030 2023-10-05T19:40:25.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.475+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403410 con 0x558818400950 2023-10-05T19:40:25.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.475+0000 7f1a817de700 1 -- 172.21.15.12:0/111040976 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a78004520 con 0x558818400950 2023-10-05T19:40:25.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.475+0000 7f1a817de700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403410 con 0x7f1a74007030 2023-10-05T19:40:25.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.475+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a7c014040 con 0x7f1a74007030 2023-10-05T19:40:25.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.475+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403410 con 0x5588183fc1b0 2023-10-05T19:40:25.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.475+0000 7f1a81fdf700 1 -- 172.21.15.12:0/111040976 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a74012a10 con 0x5588183fc1b0 2023-10-05T19:40:25.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.475+0000 7f1a81fdf700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403410 con 0x558818400950 2023-10-05T19:40:25.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.475+0000 7f1a817de700 1 -- 172.21.15.12:0/111040976 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a78004520 con 0x558818400950 2023-10-05T19:40:25.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.475+0000 7f1a817de700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403410 con 0x7f1a74007030 2023-10-05T19:40:25.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.479+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a7c014040 con 0x7f1a74007030 2023-10-05T19:40:25.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.479+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403410 con 0x558818337bc0 2023-10-05T19:40:25.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.479+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f1a7c014040 con 0x558818337bc0 2023-10-05T19:40:25.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.479+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403410 con 0x5588183fc1b0 2023-10-05T19:40:25.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.479+0000 7f1a81fdf700 1 -- 172.21.15.12:0/111040976 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f1a74012a10 con 0x5588183fc1b0 2023-10-05T19:40:25.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.479+0000 7f1a844aae40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:25.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.479+0000 7f1a844aae40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:25.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.479+0000 7f1a844aae40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:25.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.479+0000 7f1a844aae40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:25.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.479+0000 7f1a844aae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.479+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5588183ff700 con 0x558818337bc0 2023-10-05T19:40:25.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.479+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1a7c014040 con 0x558818337bc0 2023-10-05T19:40:25.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.479+0000 7f1a844aae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:25.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.479+0000 7f1a844aae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.479+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5588183ff330 con 0x558818337bc0 2023-10-05T19:40:25.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.483+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f1a7c014040 con 0x558818337bc0 2023-10-05T19:40:25.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.483+0000 7f1a844aae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:25.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.483+0000 7f1a844aae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.483+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403880 con 0x558818400950 2023-10-05T19:40:25.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.483+0000 7f1a817de700 1 -- 172.21.15.12:0/111040976 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f1a78004520 con 0x558818400950 2023-10-05T19:40:25.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.483+0000 7f1a844aae40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:25.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.483+0000 7f1a844aae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.483+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403c30 con 0x558818400950 2023-10-05T19:40:25.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.483+0000 7f1a817de700 1 -- 172.21.15.12:0/111040976 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f1a78004520 con 0x558818400950 2023-10-05T19:40:25.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.483+0000 7f1a844aae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:25.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.483+0000 7f1a844aae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.483+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403fe0 con 0x558818337bc0 2023-10-05T19:40:25.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.483+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1a7c014040 con 0x558818337bc0 2023-10-05T19:40:25.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.483+0000 7f1a844aae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:25.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.483+0000 7f1a844aae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.483+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5588183ffb90 con 0x558818337bc0 2023-10-05T19:40:25.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.483+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f1a7c014040 con 0x558818337bc0 2023-10-05T19:40:25.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.483+0000 7f1a844aae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:25.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.483+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5588183ffb60 con 0x558818400950 2023-10-05T19:40:25.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.487+0000 7f1a817de700 1 -- 172.21.15.12:0/111040976 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f1a78004520 con 0x558818400950 2023-10-05T19:40:25.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.487+0000 7f1a817de700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5588183ffb60 con 0x5588183fc1b0 2023-10-05T19:40:25.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.487+0000 7f1a81fdf700 1 -- 172.21.15.12:0/111040976 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a74012a10 con 0x5588183fc1b0 2023-10-05T19:40:25.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.487+0000 7f1a81fdf700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5588183ffb60 con 0x7f1a74007030 2023-10-05T19:40:25.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.491+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a7c014040 con 0x7f1a74007030 2023-10-05T19:40:25.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.491+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403fe0 con 0x558818337bc0 2023-10-05T19:40:25.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.491+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a7c014040 con 0x558818337bc0 2023-10-05T19:40:25.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.491+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403fe0 con 0x558818337bc0 2023-10-05T19:40:25.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.491+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a7c014040 con 0x558818337bc0 2023-10-05T19:40:25.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.491+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403fe0 con 0x7f1a74007030 2023-10-05T19:40:25.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.491+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a7c014040 con 0x7f1a74007030 2023-10-05T19:40:25.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.491+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403fe0 con 0x7f1a74007030 2023-10-05T19:40:25.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.495+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a7c014040 con 0x7f1a74007030 2023-10-05T19:40:25.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.495+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403fe0 con 0x5588183fc1b0 2023-10-05T19:40:25.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.495+0000 7f1a81fdf700 1 -- 172.21.15.12:0/111040976 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a74012a10 con 0x5588183fc1b0 2023-10-05T19:40:25.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.495+0000 7f1a81fdf700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403fe0 con 0x558818400950 2023-10-05T19:40:25.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.495+0000 7f1a817de700 1 -- 172.21.15.12:0/111040976 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a7802d040 con 0x558818400950 2023-10-05T19:40:25.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.495+0000 7f1a817de700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403fe0 con 0x558818337bc0 2023-10-05T19:40:25.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.495+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a7c014040 con 0x558818337bc0 2023-10-05T19:40:25.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.495+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403fe0 con 0x7f1a74007030 2023-10-05T19:40:25.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.495+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a7c014040 con 0x7f1a74007030 2023-10-05T19:40:25.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.495+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403fe0 con 0x558818337bc0 2023-10-05T19:40:25.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.499+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a7c014040 con 0x558818337bc0 2023-10-05T19:40:25.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.499+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403fe0 con 0x5588183fc1b0 2023-10-05T19:40:25.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.499+0000 7f1a81fdf700 1 -- 172.21.15.12:0/111040976 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a74012a10 con 0x5588183fc1b0 2023-10-05T19:40:25.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.499+0000 7f1a81fdf700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403fe0 con 0x558818400950 2023-10-05T19:40:25.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.499+0000 7f1a817de700 1 -- 172.21.15.12:0/111040976 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a7802d040 con 0x558818400950 2023-10-05T19:40:25.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.499+0000 7f1a817de700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403fe0 con 0x558818400950 2023-10-05T19:40:25.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.499+0000 7f1a817de700 1 -- 172.21.15.12:0/111040976 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a7802d040 con 0x558818400950 2023-10-05T19:40:25.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.499+0000 7f1a817de700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403fe0 con 0x7f1a74007030 2023-10-05T19:40:25.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.503+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a7c014040 con 0x7f1a74007030 2023-10-05T19:40:25.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.503+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403fe0 con 0x558818400950 2023-10-05T19:40:25.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.503+0000 7f1a817de700 1 -- 172.21.15.12:0/111040976 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a7802d040 con 0x558818400950 2023-10-05T19:40:25.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.503+0000 7f1a817de700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403fe0 con 0x558818400950 2023-10-05T19:40:25.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.503+0000 7f1a817de700 1 -- 172.21.15.12:0/111040976 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a7802d040 con 0x558818400950 2023-10-05T19:40:25.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.503+0000 7f1a817de700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403fe0 con 0x558818337bc0 2023-10-05T19:40:25.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.503+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a7c014040 con 0x558818337bc0 2023-10-05T19:40:25.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.503+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403fe0 con 0x558818337bc0 2023-10-05T19:40:25.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.503+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a7c014040 con 0x558818337bc0 2023-10-05T19:40:25.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.503+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403fe0 con 0x7f1a74007030 2023-10-05T19:40:25.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.507+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a7c014040 con 0x7f1a74007030 2023-10-05T19:40:25.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.507+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403fe0 con 0x558818400950 2023-10-05T19:40:25.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.507+0000 7f1a817de700 1 -- 172.21.15.12:0/111040976 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a7802d040 con 0x558818400950 2023-10-05T19:40:25.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.507+0000 7f1a817de700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403fe0 con 0x5588183fc1b0 2023-10-05T19:40:25.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.507+0000 7f1a81fdf700 1 -- 172.21.15.12:0/111040976 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a74012a10 con 0x5588183fc1b0 2023-10-05T19:40:25.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.507+0000 7f1a81fdf700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403fe0 con 0x558818337bc0 2023-10-05T19:40:25.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.507+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a7c014040 con 0x558818337bc0 2023-10-05T19:40:25.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.507+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403fe0 con 0x7f1a74007030 2023-10-05T19:40:25.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.511+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a7c014040 con 0x7f1a74007030 2023-10-05T19:40:25.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.511+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403fe0 con 0x558818400950 2023-10-05T19:40:25.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.511+0000 7f1a817de700 1 -- 172.21.15.12:0/111040976 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a7802d040 con 0x558818400950 2023-10-05T19:40:25.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.511+0000 7f1a817de700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403fe0 con 0x7f1a74007030 2023-10-05T19:40:25.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.511+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a7c014040 con 0x7f1a74007030 2023-10-05T19:40:25.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.511+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403fe0 con 0x5588183fc1b0 2023-10-05T19:40:25.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.511+0000 7f1a81fdf700 1 -- 172.21.15.12:0/111040976 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a74012a10 con 0x5588183fc1b0 2023-10-05T19:40:25.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.511+0000 7f1a81fdf700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403fe0 con 0x558818400950 2023-10-05T19:40:25.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.511+0000 7f1a817de700 1 -- 172.21.15.12:0/111040976 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a7802d040 con 0x558818400950 2023-10-05T19:40:25.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.511+0000 7f1a817de700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403fe0 con 0x7f1a74007030 2023-10-05T19:40:25.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.515+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1a7c014040 con 0x7f1a74007030 2023-10-05T19:40:25.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.515+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403fe0 con 0x558818337bc0 2023-10-05T19:40:25.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.515+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f1a7c014040 con 0x558818337bc0 2023-10-05T19:40:25.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.515+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818403fe0 con 0x5588183fc1b0 2023-10-05T19:40:25.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.515+0000 7f1a81fdf700 1 -- 172.21.15.12:0/111040976 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f1a74012a10 con 0x5588183fc1b0 2023-10-05T19:40:25.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.515+0000 7f1a844aae40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:25.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.515+0000 7f1a844aae40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:25.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.515+0000 7f1a844aae40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:25.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.515+0000 7f1a844aae40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:25.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.515+0000 7f1a844aae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.515+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5588183ffb60 con 0x558818337bc0 2023-10-05T19:40:25.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.515+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f1a7c014040 con 0x558818337bc0 2023-10-05T19:40:25.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.515+0000 7f1a844aae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:25.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.515+0000 7f1a844aae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.515+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5588184048d0 con 0x5588183fc1b0 2023-10-05T19:40:25.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.519+0000 7f1a81fdf700 1 -- 172.21.15.12:0/111040976 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f1a74012a10 con 0x5588183fc1b0 2023-10-05T19:40:25.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.519+0000 7f1a844aae40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:25.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.519+0000 7f1a844aae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.519+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5588184051c0 con 0x558818337bc0 2023-10-05T19:40:25.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.519+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f1a7c014040 con 0x558818337bc0 2023-10-05T19:40:25.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.519+0000 7f1a844aae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:25.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.519+0000 7f1a844aae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.519+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5588184051c0 con 0x558818400950 2023-10-05T19:40:25.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.519+0000 7f1a817de700 1 -- 172.21.15.12:0/111040976 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f1a7802d040 con 0x558818400950 2023-10-05T19:40:25.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.519+0000 7f1a844aae40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:25.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.519+0000 7f1a844aae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.519+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818405930 con 0x7f1a74007030 2023-10-05T19:40:25.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.519+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f1a7c014040 con 0x7f1a74007030 2023-10-05T19:40:25.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.519+0000 7f1a844aae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:25.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.519+0000 7f1a844aae40 20 zone default found 2023-10-05T19:40:25.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.519+0000 7f1a844aae40 4 Realm: () 2023-10-05T19:40:25.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.519+0000 7f1a844aae40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:25.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.519+0000 7f1a844aae40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:25.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.519+0000 7f1a844aae40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:25.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.519+0000 7f1a844aae40 20 zonegroup default 2023-10-05T19:40:25.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.519+0000 7f1a844aae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.519+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5588184058e0 con 0x7f1a74007030 2023-10-05T19:40:25.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.519+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f1a7c014040 con 0x7f1a74007030 2023-10-05T19:40:25.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.523+0000 7f1a844aae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:25.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.523+0000 7f1a844aae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.523+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818407260 con 0x558818400950 2023-10-05T19:40:25.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.523+0000 7f1a817de700 1 -- 172.21.15.12:0/111040976 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f1a7802d040 con 0x558818400950 2023-10-05T19:40:25.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.523+0000 7f1a844aae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:25.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.523+0000 7f1a844aae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.523+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818407610 con 0x558818400950 2023-10-05T19:40:25.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.523+0000 7f1a817de700 1 -- 172.21.15.12:0/111040976 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f1a7803c040 con 0x558818400950 2023-10-05T19:40:25.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.523+0000 7f1a844aae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:25.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.523+0000 7f1a844aae40 20 started sync module instance, tier type = 2023-10-05T19:40:25.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.523+0000 7f1a844aae40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:25.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.523+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818406920 con 0x5588183fc1b0 2023-10-05T19:40:25.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.527+0000 7f1a81fdf700 1 -- 172.21.15.12:0/111040976 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'114 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1a74012a10 con 0x5588183fc1b0 2023-10-05T19:40:25.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.527+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94043010780368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5588184079c0 con 0x5588183fc1b0 2023-10-05T19:40:25.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.527+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818408260 con 0x558818337bc0 2023-10-05T19:40:25.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.531+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'114 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1a7c014040 con 0x558818337bc0 2023-10-05T19:40:25.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.531+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94043010793104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5588184087d0 con 0x558818337bc0 2023-10-05T19:40:25.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.531+0000 7f1a81fdf700 1 -- 172.21.15.12:0/111040976 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94043010780368] v28'115 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1a74012a10 con 0x5588183fc1b0 2023-10-05T19:40:25.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.531+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818406d20 con 0x558818400950 2023-10-05T19:40:25.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.531+0000 7f1a817de700 1 -- 172.21.15.12:0/111040976 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'114 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1a7803c040 con 0x558818400950 2023-10-05T19:40:25.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.531+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94043010797104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818409770 con 0x558818400950 2023-10-05T19:40:25.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.531+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55881840a010 con 0x5588183fc1b0 2023-10-05T19:40:25.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.535+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94043010793104] v28'115 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1a7c014040 con 0x558818337bc0 2023-10-05T19:40:25.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.535+0000 7f1a81fdf700 1 -- 172.21.15.12:0/111040976 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'114 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1a74012a10 con 0x5588183fc1b0 2023-10-05T19:40:25.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.535+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94043010794208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818409c40 con 0x5588183fc1b0 2023-10-05T19:40:25.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.535+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55881840b0b0 con 0x558818337bc0 2023-10-05T19:40:25.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.535+0000 7f1a817de700 1 -- 172.21.15.12:0/111040976 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94043010797104] v28'115 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1a7803c040 con 0x558818400950 2023-10-05T19:40:25.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.539+0000 7f1a81fdf700 1 -- 172.21.15.12:0/111040976 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94043010794208] v28'115 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1a74012a10 con 0x5588183fc1b0 2023-10-05T19:40:25.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.539+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'114 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1a7c014040 con 0x558818337bc0 2023-10-05T19:40:25.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.539+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94043010802704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818409c40 con 0x558818337bc0 2023-10-05T19:40:25.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.539+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55881840acf0 con 0x7f1a74007030 2023-10-05T19:40:25.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.539+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'114 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1a7c014040 con 0x7f1a74007030 2023-10-05T19:40:25.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.539+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94043010806688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55881840bce0 con 0x7f1a74007030 2023-10-05T19:40:25.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.539+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55881840c7c0 con 0x7f1a74007030 2023-10-05T19:40:25.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.539+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94043010802704] v28'115 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1a7c014040 con 0x558818337bc0 2023-10-05T19:40:25.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.543+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(95 notify.5 [watch watch cookie 94043010806688] v28'115 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1a7c014040 con 0x7f1a74007030 2023-10-05T19:40:25.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.543+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(96 notify.6 [create] v28'114 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1a7c014040 con 0x7f1a74007030 2023-10-05T19:40:25.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.543+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94043010798208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55881840b7f0 con 0x7f1a74007030 2023-10-05T19:40:25.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.543+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55881840cee0 con 0x7f1a74007030 2023-10-05T19:40:25.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.547+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(97 notify.6 [watch watch cookie 94043010798208] v28'115 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1a7c014040 con 0x7f1a74007030 2023-10-05T19:40:25.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.547+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(98 notify.7 [create] v28'114 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1a7c014040 con 0x7f1a74007030 2023-10-05T19:40:25.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.547+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94043010810512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55881840cee0 con 0x7f1a74007030 2023-10-05T19:40:25.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.547+0000 7f1a844aae40 20 add_watcher() i=0 2023-10-05T19:40:25.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.547+0000 7f1a844aae40 20 add_watcher() i=1 2023-10-05T19:40:25.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.547+0000 7f1a844aae40 20 add_watcher() i=2 2023-10-05T19:40:25.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.547+0000 7f1a844aae40 20 add_watcher() i=3 2023-10-05T19:40:25.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.547+0000 7f1a844aae40 20 add_watcher() i=4 2023-10-05T19:40:25.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.547+0000 7f1a844aae40 20 add_watcher() i=5 2023-10-05T19:40:25.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.547+0000 7f1a844aae40 20 add_watcher() i=6 2023-10-05T19:40:25.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.547+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94043010810512] v28'115 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1a7c014040 con 0x7f1a74007030 2023-10-05T19:40:25.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.547+0000 7f1a844aae40 20 add_watcher() i=7 2023-10-05T19:40:25.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.547+0000 7f1a844aae40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:25.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.547+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55881840ff10 con 0x558818337bc0 2023-10-05T19:40:25.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.547+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f1a7c014040 con 0x558818337bc0 2023-10-05T19:40:25.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.547+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94043010812400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55881840d3d0 con 0x558818337bc0 2023-10-05T19:40:25.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.551+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94043010812400] v28'111 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f1a7c014040 con 0x558818337bc0 2023-10-05T19:40:25.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.551+0000 7f1a537de700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:25.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.551+0000 7f1a844aae40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:25.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.551+0000 7f1a844aae40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:25.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.551+0000 7f1a844aae40 1 Processor -- start 2023-10-05T19:40:25.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.551+0000 7f1a844aae40 1 -- start start 2023-10-05T19:40:25.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.551+0000 7f1a844aae40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55881841a630 0x55881841aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:25.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.551+0000 7f1a844aae40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55881841af40 con 0x55881841a630 2023-10-05T19:40:25.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.551+0000 7f1a817de700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55881841a630 0x55881841aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:25.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.551+0000 7f1a817de700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55881841a630 0x55881841aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57952/0 (socket says 172.21.15.12:57952) 2023-10-05T19:40:25.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.551+0000 7f1a817de700 1 -- 172.21.15.12:0/1962153711 learned_addr learned my addr 172.21.15.12:0/1962153711 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:25.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.555+0000 7f1a817de700 1 -- 172.21.15.12:0/1962153711 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55881841b080 con 0x55881841a630 2023-10-05T19:40:25.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.555+0000 7f1a817de700 1 --2- 172.21.15.12:0/1962153711 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55881841a630 0x55881841aa00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1a74005ae0 tx=0x7f1a78027040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90b777fdb771c7c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:25.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.555+0000 7f1a52fdd700 1 -- 172.21.15.12:0/1962153711 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a78039040 con 0x55881841a630 2023-10-05T19:40:25.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.555+0000 7f1a52fdd700 1 -- 172.21.15.12:0/1962153711 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a7803a040 con 0x55881841a630 2023-10-05T19:40:25.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.555+0000 7f1a52fdd700 1 -- 172.21.15.12:0/1962153711 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a7802a080 con 0x55881841a630 2023-10-05T19:40:25.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.555+0000 7f1a844aae40 1 -- 172.21.15.12:0/1962153711 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55881841a630 msgr2=0x55881841aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.555+0000 7f1a844aae40 1 --2- 172.21.15.12:0/1962153711 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55881841a630 0x55881841aa00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1a74005ae0 tx=0x7f1a78027040 comp rx=0 tx=0).stop 2023-10-05T19:40:25.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.555+0000 7f1a844aae40 1 -- 172.21.15.12:0/1962153711 shutdown_connections 2023-10-05T19:40:25.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.555+0000 7f1a844aae40 1 --2- 172.21.15.12:0/1962153711 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55881841a630 0x55881841aa00 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:25.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.555+0000 7f1a844aae40 1 -- 172.21.15.12:0/1962153711 >> 172.21.15.12:0/1962153711 conn(0x5588184159f0 msgr2=0x558818413b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:25.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.555+0000 7f1a844aae40 1 -- 172.21.15.12:0/1962153711 shutdown_connections 2023-10-05T19:40:25.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.555+0000 7f1a844aae40 1 -- 172.21.15.12:0/1962153711 wait complete. 2023-10-05T19:40:25.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.559+0000 7f1a844aae40 1 Processor -- start 2023-10-05T19:40:25.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.559+0000 7f1a844aae40 1 -- start start 2023-10-05T19:40:25.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.559+0000 7f1a844aae40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55881841a630 0x5588184201c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:25.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.559+0000 7f1a844aae40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588184211c0 con 0x55881841a630 2023-10-05T19:40:25.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.559+0000 7f1a817de700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55881841a630 0x5588184201c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:25.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.559+0000 7f1a817de700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55881841a630 0x5588184201c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57958/0 (socket says 172.21.15.12:57958) 2023-10-05T19:40:25.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.559+0000 7f1a817de700 1 -- 172.21.15.12:0/3902790757 learned_addr learned my addr 172.21.15.12:0/3902790757 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:25.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.559+0000 7f1a817de700 1 -- 172.21.15.12:0/3902790757 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a6c001e10 con 0x55881841a630 2023-10-05T19:40:25.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.559+0000 7f1a817de700 1 --2- 172.21.15.12:0/3902790757 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55881841a630 0x5588184201c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1a78004fd0 tx=0x7f1a7803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:25.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.559+0000 7f1a517da700 1 -- 172.21.15.12:0/3902790757 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a78039030 con 0x55881841a630 2023-10-05T19:40:25.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.559+0000 7f1a517da700 1 -- 172.21.15.12:0/3902790757 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a7802b040 con 0x55881841a630 2023-10-05T19:40:25.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.559+0000 7f1a517da700 1 -- 172.21.15.12:0/3902790757 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a7804f040 con 0x55881841a630 2023-10-05T19:40:25.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.559+0000 7f1a844aae40 1 -- 172.21.15.12:0/3902790757 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5588184207c0 con 0x55881841a630 2023-10-05T19:40:25.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.559+0000 7f1a844aae40 1 -- 172.21.15.12:0/3902790757 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558818420ba0 con 0x55881841a630 2023-10-05T19:40:25.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.563+0000 7f1a517da700 1 -- 172.21.15.12:0/3902790757 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f1a7804b730 con 0x55881841a630 2023-10-05T19:40:25.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.563+0000 7f1a517da700 1 --2- 172.21.15.12:0/3902790757 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f19c8030ca0 0x7f19c8033150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:25.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.563+0000 7f1a517da700 1 -- 172.21.15.12:0/3902790757 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f1a7805eab0 con 0x55881841a630 2023-10-05T19:40:25.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.563+0000 7f1a827e0700 1 --2- 172.21.15.12:0/3902790757 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f19c8030ca0 0x7f19c8033150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:25.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.563+0000 7f1a844aae40 5 note: GC not initialized 2023-10-05T19:40:25.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.563+0000 7f1a827e0700 1 --2- 172.21.15.12:0/3902790757 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f19c8030ca0 0x7f19c8033150 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x55881840d330 tx=0x7f1a7c00ab20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:25.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.563+0000 7f19c27fc700 20 reqs_thread_entry: start 2023-10-05T19:40:25.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.563+0000 7f1a844aae40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:25.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.563+0000 7f19b3fff700 20 reqs_thread_entry: start 2023-10-05T19:40:25.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.567+0000 7f1a844aae40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:25.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.567+0000 7f19b27fc700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:55 2023 2023-10-05T19:40:25.574 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:25.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.567+0000 7f19b1ffb700 20 reqs_thread_entry: start 2023-10-05T19:40:25.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.567+0000 7f1a844aae40 20 get_system_obj_state: rctx=0x7ffed8894af8 obj=default.rgw.meta:users.uid:fud state=0x55881829b0b0 s->prefetch_data=0 2023-10-05T19:40:25.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.567+0000 7f1a844aae40 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2023-10-05T19:40:25.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.567+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:102 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818488870 con 0x7f1a74007030 2023-10-05T19:40:25.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.567+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(102 fud [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7f1a7c014040 con 0x7f1a74007030 2023-10-05T19:40:25.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.567+0000 7f1a844aae40 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x16 2023-10-05T19:40:25.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.567+0000 7f1a844aae40 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2023-10-05T19:40:25.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.567+0000 7f1a844aae40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:25.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.567+0000 7f1a844aae40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:25.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.567+0000 7f1a844aae40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:25.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.567+0000 7f1a844aae40 10 cache get: name=default.rgw.meta+users.uid+fud : type miss (requested=0x13, cached=0x16) 2023-10-05T19:40:25.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.567+0000 7f1a844aae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.567+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:103 8.e 8:75f48285:users.uid::fud:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55881848b300 con 0x7f1a74007030 2023-10-05T19:40:25.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.567+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(103 fud [call,call out=48b,read 0~346 out=346b,getxattrs out=75b] v0'0 uv2 ondisk = 0) v8 ==== 273+0+469 (crc 0 0 0) 0x7f1a7c014040 con 0x7f1a74007030 2023-10-05T19:40:25.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.567+0000 7f1a844aae40 20 rados_obj.operate() r=0 bl.length=346 2023-10-05T19:40:25.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.567+0000 7f1a844aae40 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x13 2023-10-05T19:40:25.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.567+0000 7f1a844aae40 10 moving default.rgw.meta+users.uid+fud to cache LRU end 2023-10-05T19:40:25.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.567+0000 7f1a844aae40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:25.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.567+0000 7f1a844aae40 20 get_system_obj_state: rctx=0x7ffed8893ea0 obj=default.rgw.meta:root:myfoo state=0x55881829b0b0 s->prefetch_data=0 2023-10-05T19:40:25.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.567+0000 7f1a844aae40 10 cache get: name=default.rgw.meta+root+myfoo : miss 2023-10-05T19:40:25.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.567+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:104 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55881848ab10 con 0x5588183fc1b0 2023-10-05T19:40:25.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.567+0000 7f1a81fdf700 1 -- 172.21.15.12:0/111040976 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(104 myfoo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv18 ondisk = 0) v8 ==== 233+0+139 (crc 0 0 0) 0x7f1a74012a10 con 0x5588183fc1b0 2023-10-05T19:40:25.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.567+0000 7f1a844aae40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x16 2023-10-05T19:40:25.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.567+0000 7f1a844aae40 10 adding default.rgw.meta+root+myfoo to cache LRU end 2023-10-05T19:40:25.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.567+0000 7f1a844aae40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:25.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.567+0000 7f1a844aae40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:25.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.567+0000 7f1a844aae40 10 cache get: name=default.rgw.meta+root+myfoo : type miss (requested=0x11, cached=0x16) 2023-10-05T19:40:25.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.567+0000 7f1a844aae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.567+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:105 8.9 8:91b71841:root::myfoo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55881848d0b0 con 0x5588183fc1b0 2023-10-05T19:40:25.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.571+0000 7f1a81fdf700 1 -- 172.21.15.12:0/111040976 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(105 myfoo [call,call out=48b,read 0~165 out=165b] v0'0 uv18 ondisk = 0) v8 ==== 233+0+213 (crc 0 0 0) 0x7f1a74012a10 con 0x5588183fc1b0 2023-10-05T19:40:25.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.571+0000 7f1a844aae40 20 rados_obj.operate() r=0 bl.length=165 2023-10-05T19:40:25.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.571+0000 7f1a844aae40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2023-10-05T19:40:25.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.571+0000 7f1a844aae40 10 moving default.rgw.meta+root+myfoo to cache LRU end 2023-10-05T19:40:25.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.571+0000 7f1a844aae40 20 get_system_obj_state: rctx=0x7ffed8893ea0 obj=default.rgw.meta:root:.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 state=0x5588182712b0 s->prefetch_data=0 2023-10-05T19:40:25.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.571+0000 7f1a844aae40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 : miss 2023-10-05T19:40:25.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.571+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:106 8.e 8:70176bbd:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55881848c620 con 0x7f1a74007030 2023-10-05T19:40:25.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.571+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(106 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 [call out=48b,getxattrs out=218b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 290+0+282 (crc 0 0 0) 0x7f1a7c014040 con 0x7f1a74007030 2023-10-05T19:40:25.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.571+0000 7f1a844aae40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 info.flags=0x16 2023-10-05T19:40:25.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.571+0000 7f1a844aae40 10 adding default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 to cache LRU end 2023-10-05T19:40:25.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.571+0000 7f1a844aae40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:25.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.571+0000 7f1a844aae40 10 updating xattr: name=user.rgw.acl bl.length()=123 2023-10-05T19:40:25.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.571+0000 7f1a844aae40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:25.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.571+0000 7f1a844aae40 20 Read xattr: user.rgw.acl 2023-10-05T19:40:25.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.571+0000 7f1a844aae40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:25.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.571+0000 7f1a844aae40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 : type miss (requested=0x13, cached=0x16) 2023-10-05T19:40:25.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.571+0000 7f1a844aae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.571+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:107 8.e 8:70176bbd:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55881848f650 con 0x7f1a74007030 2023-10-05T19:40:25.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.571+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.0 v2:172.21.15.8:6800/27563 30 ==== osd_op_reply(107 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 [call,call out=48b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 332+0+618 (crc 0 0 0) 0x7f1a7c014040 con 0x7f1a74007030 2023-10-05T19:40:25.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.571+0000 7f1a844aae40 20 rados_obj.operate() r=0 bl.length=352 2023-10-05T19:40:25.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.571+0000 7f1a844aae40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 info.flags=0x13 2023-10-05T19:40:25.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.571+0000 7f1a844aae40 10 moving default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 to cache LRU end 2023-10-05T19:40:25.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.571+0000 7f1a844aae40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:25.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.571+0000 7f1a844aae40 10 updating xattr: name=user.rgw.acl bl.length()=123 2023-10-05T19:40:25.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.571+0000 7f1a844aae40 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 2023-10-05T19:40:25.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.571+0000 7f1a844aae40 20 get_system_obj_state: rctx=0x7ffed8893e38 obj=default.rgw.meta:users.uid:fud state=0x5588182712b0 s->prefetch_data=0 2023-10-05T19:40:25.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.571+0000 7f1a844aae40 10 cache get: name=default.rgw.meta+users.uid+fud : type miss (requested=0x16, cached=0x13) 2023-10-05T19:40:25.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.571+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:108 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55881848cc70 con 0x7f1a74007030 2023-10-05T19:40:25.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.575+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.0 v2:172.21.15.8:6800/27563 31 ==== osd_op_reply(108 fud [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7f1a7c014040 con 0x7f1a74007030 2023-10-05T19:40:25.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.575+0000 7f1a844aae40 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x16 2023-10-05T19:40:25.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.575+0000 7f1a844aae40 10 moving default.rgw.meta+users.uid+fud to cache LRU end 2023-10-05T19:40:25.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.575+0000 7f1a844aae40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:25.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.575+0000 7f1a844aae40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:25.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.575+0000 7f1a844aae40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:25.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.575+0000 7f1a844aae40 10 cache get: name=default.rgw.meta+users.uid+fud : hit (requested=0x13, cached=0x17) 2023-10-05T19:40:25.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.575+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:109 8.9 8:96d98480:users.uid::foo.buckets:head [call user.remove_bucket in=58b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5588184918d0 con 0x5588183fc1b0 2023-10-05T19:40:25.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.575+0000 7f1a81fdf700 1 -- 172.21.15.12:0/111040976 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(109 foo.buckets [call] v28'19 uv17 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x7f1a74012a10 con 0x5588183fc1b0 2023-10-05T19:40:25.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.575+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:110 8.e 8:70176bbd:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~352 in=352b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5588184976d0 con 0x7f1a74007030 2023-10-05T19:40:25.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.579+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.0 v2:172.21.15.8:6800/27563 32 ==== osd_op_reply(110 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 [delete,create,call,call,writefull 0~352,setxattr (123)] v28'3 uv3 ondisk = 0) v8 ==== 416+0+0 (crc 0 0 0) 0x7f1a7c014040 con 0x7f1a74007030 2023-10-05T19:40:25.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.579+0000 7f1a844aae40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 info.flags=0x17 2023-10-05T19:40:25.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.579+0000 7f1a844aae40 10 moving default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 to cache LRU end 2023-10-05T19:40:25.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.579+0000 7f1a844aae40 10 updating xattr: name=user.rgw.acl bl.length()=123 2023-10-05T19:40:25.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.579+0000 7f1a844aae40 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1, ofs0, ns] 2023-10-05T19:40:25.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.579+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:111 7.e 7:7759931f:::notify.3:head [notify cookie 94043011380800 in=737b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5588184938a0 con 0x5588183fc1b0 2023-10-05T19:40:25.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.579+0000 7f1a81fdf700 1 -- 172.21.15.12:0/111040976 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== watch-notify(notify (1) cookie 94043010794208 notify 120259084301 ret 0) v3 ==== 767+0+0 (crc 0 0 0) 0x5588183eeae0 con 0x5588183fc1b0 2023-10-05T19:40:25.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.579+0000 7f1a72ffd700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084301 cookie 94043010794208 notifier 4453 bl.length()=725 2023-10-05T19:40:25.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.579+0000 7f1a72ffd700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 info.flags=0x17 2023-10-05T19:40:25.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.579+0000 7f1a72ffd700 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 to cache LRU end 2023-10-05T19:40:25.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.579+0000 7f1a72ffd700 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2023-10-05T19:40:25.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.579+0000 7f1a81fdf700 1 -- 172.21.15.12:0/111040976 <== osd.2 v2:172.21.15.12:6800/29934 23 ==== osd_op_reply(111 notify.3 [notify cookie 94043011380800 out=8b] v0'0 uv114 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f1a74012a10 con 0x5588183fc1b0 2023-10-05T19:40:25.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.579+0000 7f1a72ffd700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:112 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f1a68003d90 con 0x5588183fc1b0 2023-10-05T19:40:25.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.583+0000 7f1a81fdf700 1 -- 172.21.15.12:0/111040976 <== osd.2 v2:172.21.15.12:6800/29934 24 ==== osd_op_reply(112 notify.3 [notify-ack] v0'0 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1a74012a10 con 0x5588183fc1b0 2023-10-05T19:40:25.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.583+0000 7f1a81fdf700 1 -- 172.21.15.12:0/111040976 <== osd.2 v2:172.21.15.12:6800/29934 25 ==== watch-notify(notify_complete (2) cookie 94043011380800 notify 120259084301 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5588183ee6a0 con 0x5588183fc1b0 2023-10-05T19:40:25.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.583+0000 7f1a844aae40 20 handle_bi_update(): bucket=:myfoo[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1]): orig_sources= new_sources= 2023-10-05T19:40:25.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.583+0000 7f1a844aae40 20 handle_bi_update(): bucket=:myfoo[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1]): potential sources added=[] removed=[] 2023-10-05T19:40:25.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.583+0000 7f1a844aae40 20 handle_bi_update(): bucket=:myfoo[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1]): orig_dests= new_dests= 2023-10-05T19:40:25.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.583+0000 7f1a844aae40 20 handle_bi_update(): bucket=:myfoo[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1]): potential dests added=[] removed=[] 2023-10-05T19:40:25.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.583+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:113 8.1 8:810902b1:users.uid::fud.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5588184938a0 con 0x7f1a74007030 2023-10-05T19:40:25.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.587+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.0 v2:172.21.15.8:6800/27563 33 ==== osd_op_reply(113 fud.buckets [call] v28'1 uv1 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x7f1a7c014040 con 0x7f1a74007030 2023-10-05T19:40:25.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.587+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:114 8.9 8:91b71841:root::myfoo:head [delete,create,call version.inc in=38b,writefull 0~165 in=165b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818497600 con 0x5588183fc1b0 2023-10-05T19:40:25.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.587+0000 7f1a81fdf700 1 -- 172.21.15.12:0/111040976 <== osd.2 v2:172.21.15.12:6800/29934 26 ==== osd_op_reply(114 myfoo [delete,create,call,writefull 0~165] v28'20 uv20 ondisk = 0) v8 ==== 275+0+0 (crc 0 0 0) 0x7f1a74012a10 con 0x5588183fc1b0 2023-10-05T19:40:25.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.587+0000 7f1a844aae40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2023-10-05T19:40:25.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.587+0000 7f1a844aae40 10 moving default.rgw.meta+root+myfoo to cache LRU end 2023-10-05T19:40:25.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.587+0000 7f1a844aae40 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2023-10-05T19:40:25.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.587+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:115 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94043011380736 in=326b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818494480 con 0x7f1a74007030 2023-10-05T19:40:25.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.591+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.0 v2:172.21.15.8:6800/27563 34 ==== watch-notify(notify (1) cookie 94043010798208 notify 120259084303 ret 0) v3 ==== 356+0+0 (crc 0 0 0) 0x7f1a7c0330d0 con 0x7f1a74007030 2023-10-05T19:40:25.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.591+0000 7f1a72ffd700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084303 cookie 94043010798208 notifier 4453 bl.length()=314 2023-10-05T19:40:25.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.591+0000 7f1a72ffd700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2023-10-05T19:40:25.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.591+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.0 v2:172.21.15.8:6800/27563 35 ==== osd_op_reply(115 notify.6 [notify cookie 94043011380736 out=8b] v0'0 uv114 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f1a7c014040 con 0x7f1a74007030 2023-10-05T19:40:25.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.591+0000 7f1a72ffd700 10 rgw watcher librados: moving default.rgw.meta+root+myfoo to cache LRU end 2023-10-05T19:40:25.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.591+0000 7f1a72ffd700 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:116 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f1a68003d90 con 0x7f1a74007030 2023-10-05T19:40:25.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.591+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.0 v2:172.21.15.8:6800/27563 36 ==== watch-notify(notify_complete (2) cookie 94043011380736 notify 120259084303 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f1a7c033260 con 0x7f1a74007030 2023-10-05T19:40:25.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.591+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.0 v2:172.21.15.8:6800/27563 37 ==== osd_op_reply(116 notify.6 [notify-ack] v0'0 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1a7c014040 con 0x7f1a74007030 2023-10-05T19:40:25.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.591+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:117 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94043010780368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818494480 con 0x5588183fc1b0 2023-10-05T19:40:25.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.595+0000 7f1a81fdf700 1 -- 172.21.15.12:0/111040976 <== osd.2 v2:172.21.15.12:6800/29934 27 ==== osd_op_reply(117 notify.0 [watch unwatch cookie 94043010780368] v28'116 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1a74012a10 con 0x5588183fc1b0 2023-10-05T19:40:25.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.595+0000 7f1a844aae40 20 remove_watcher() i=0 2023-10-05T19:40:25.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.595+0000 7f1a844aae40 2 removed watcher, disabling cache 2023-10-05T19:40:25.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.595+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:118 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94043010793104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818494480 con 0x558818337bc0 2023-10-05T19:40:25.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.599+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(118 notify.1 [watch unwatch cookie 94043010793104] v28'116 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1a7c014040 con 0x558818337bc0 2023-10-05T19:40:25.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.599+0000 7f1a844aae40 20 remove_watcher() i=1 2023-10-05T19:40:25.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.599+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:119 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94043010797104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818494480 con 0x558818400950 2023-10-05T19:40:25.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.599+0000 7f1a817de700 1 -- 172.21.15.12:0/111040976 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(119 notify.2 [watch unwatch cookie 94043010797104] v28'116 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1a7805a070 con 0x558818400950 2023-10-05T19:40:25.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.599+0000 7f1a844aae40 20 remove_watcher() i=2 2023-10-05T19:40:25.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.599+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:120 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94043010794208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818494480 con 0x5588183fc1b0 2023-10-05T19:40:25.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.603+0000 7f1a81fdf700 1 -- 172.21.15.12:0/111040976 <== osd.2 v2:172.21.15.12:6800/29934 28 ==== osd_op_reply(120 notify.3 [watch unwatch cookie 94043010794208] v28'116 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1a74012a10 con 0x5588183fc1b0 2023-10-05T19:40:25.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.603+0000 7f1a844aae40 20 remove_watcher() i=3 2023-10-05T19:40:25.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.603+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:121 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94043010802704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818494480 con 0x558818337bc0 2023-10-05T19:40:25.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.607+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(121 notify.4 [watch unwatch cookie 94043010802704] v28'116 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1a7c014040 con 0x558818337bc0 2023-10-05T19:40:25.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.607+0000 7f1a844aae40 20 remove_watcher() i=4 2023-10-05T19:40:25.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.607+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:122 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94043010806688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818494480 con 0x7f1a74007030 2023-10-05T19:40:25.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.611+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.0 v2:172.21.15.8:6800/27563 38 ==== osd_op_reply(122 notify.5 [watch unwatch cookie 94043010806688] v28'116 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1a7c014040 con 0x7f1a74007030 2023-10-05T19:40:25.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.611+0000 7f1a844aae40 20 remove_watcher() i=5 2023-10-05T19:40:25.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.611+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:123 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94043010798208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818494480 con 0x7f1a74007030 2023-10-05T19:40:25.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.611+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.0 v2:172.21.15.8:6800/27563 39 ==== osd_op_reply(123 notify.6 [watch unwatch cookie 94043010798208] v28'116 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1a7c014040 con 0x7f1a74007030 2023-10-05T19:40:25.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.611+0000 7f1a844aae40 20 remove_watcher() i=6 2023-10-05T19:40:25.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.611+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:124 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94043010810512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818494480 con 0x7f1a74007030 2023-10-05T19:40:25.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.615+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.0 v2:172.21.15.8:6800/27563 40 ==== osd_op_reply(124 notify.7 [watch unwatch cookie 94043010810512] v28'116 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1a7c014040 con 0x7f1a74007030 2023-10-05T19:40:25.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.615+0000 7f1a844aae40 20 remove_watcher() i=7 2023-10-05T19:40:25.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.619+0000 7f1a537de700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:25.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.619+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:125 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94043010812400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558818494480 con 0x558818337bc0 2023-10-05T19:40:25.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.619+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(125 data_loggenerations_metadata [watch unwatch cookie 94043010812400] v28'112 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f1a7c014040 con 0x558818337bc0 2023-10-05T19:40:25.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.619+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f1a74007030 msgr2=0x7f1a7401e8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.619+0000 7f1a844aae40 1 --2- 172.21.15.12:0/111040976 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f1a74007030 0x7f1a7401e8f0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:25.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.619+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x558818337bc0 msgr2=0x5588183f9bb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.619+0000 7f1a844aae40 1 --2- 172.21.15.12:0/111040976 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x558818337bc0 0x5588183f9bb0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:25.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.619+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5588183fc1b0 msgr2=0x5588183fe590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.619+0000 7f1a844aae40 1 --2- 172.21.15.12:0/111040976 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5588183fc1b0 0x5588183fe590 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:25.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.619+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x558818400950 msgr2=0x558818402d60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.619+0000 7f1a844aae40 1 --2- 172.21.15.12:0/111040976 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x558818400950 0x558818402d60 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:25.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.623+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1a64030c40 msgr2=0x7f1a640330f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.623+0000 7f1a844aae40 1 --2- 172.21.15.12:0/111040976 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1a64030c40 0x7f1a640330f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1a780044f0 tx=0x7f1a78006690 comp rx=0 tx=0).stop 2023-10-05T19:40:25.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.623+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5588183366e0 msgr2=0x5588183ee160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.623+0000 7f1a844aae40 1 --2- 172.21.15.12:0/111040976 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5588183366e0 0x5588183ee160 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1a7401f040 tx=0x7f1a7401c000 comp rx=0 tx=0).stop 2023-10-05T19:40:25.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.623+0000 7f1a827e0700 1 -- 172.21.15.12:0/111040976 reap_dead start 2023-10-05T19:40:25.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.623+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 shutdown_connections 2023-10-05T19:40:25.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.623+0000 7f1a844aae40 1 --2- 172.21.15.12:0/111040976 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5588183366e0 0x5588183ee160 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:25.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.623+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 >> 172.21.15.12:0/111040976 conn(0x5588183bd7f0 msgr2=0x5588183e19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:25.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.623+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 shutdown_connections 2023-10-05T19:40:25.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.623+0000 7f1a844aae40 1 -- 172.21.15.12:0/111040976 wait complete. 2023-10-05T19:40:25.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.623+0000 7f1a844aae40 1 -- 172.21.15.12:0/3902790757 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f19c8030ca0 msgr2=0x7f19c8033150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.623+0000 7f1a844aae40 1 --2- 172.21.15.12:0/3902790757 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f19c8030ca0 0x7f19c8033150 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x55881840d330 tx=0x7f1a7c00ab20 comp rx=0 tx=0).stop 2023-10-05T19:40:25.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.623+0000 7f1a844aae40 1 -- 172.21.15.12:0/3902790757 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55881841a630 msgr2=0x5588184201c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.623+0000 7f1a844aae40 1 --2- 172.21.15.12:0/3902790757 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55881841a630 0x5588184201c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1a78004fd0 tx=0x7f1a7803f000 comp rx=0 tx=0).stop 2023-10-05T19:40:25.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.623+0000 7f1a844aae40 1 -- 172.21.15.12:0/3902790757 shutdown_connections 2023-10-05T19:40:25.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.623+0000 7f1a844aae40 1 --2- 172.21.15.12:0/3902790757 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f19c8030ca0 0x7f19c8033150 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:25.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.623+0000 7f1a844aae40 1 --2- 172.21.15.12:0/3902790757 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55881841a630 0x5588184201c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:25.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.623+0000 7f1a844aae40 1 -- 172.21.15.12:0/3902790757 >> 172.21.15.12:0/3902790757 conn(0x5588184159f0 msgr2=0x558818413a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:25.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.623+0000 7f1a844aae40 1 -- 172.21.15.12:0/3902790757 shutdown_connections 2023-10-05T19:40:25.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.623+0000 7f1a844aae40 1 -- 172.21.15.12:0/3902790757 wait complete. 2023-10-05T19:40:25.637 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'fud'] 2023-10-05T19:40:25.637 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'fud'] 2023-10-05T19:40:25.637 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid fud 2023-10-05T19:40:25.673 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:25.673 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:25.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.667+0000 7f3c8f858e40 1 Processor -- start 2023-10-05T19:40:25.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.667+0000 7f3c8f858e40 1 -- start start 2023-10-05T19:40:25.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.667+0000 7f3c8f858e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x5562f53e4010 0x5562f53e43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:25.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.667+0000 7f3c8f858e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5562f532ebd0 con 0x5562f53e4920 2023-10-05T19:40:25.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.667+0000 7f3c8f858e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x5562f52ebee0 con 0x5562f53e4010 2023-10-05T19:40:25.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.667+0000 7f3c8d38d700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x5562f53e4010 0x5562f53e43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:25.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.667+0000 7f3c8cb8c700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x5562f53e4920 0x5562f53e53c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:35546/0 (socket says 172.21.15.12:35546) 2023-10-05T19:40:25.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.667+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/2043565195 learned_addr learned my addr 172.21.15.12:0/2043565195 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:25.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.667+0000 7f3c8d38d700 1 -- 172.21.15.12:0/2043565195 >> v1:172.21.15.8:6789/0 conn(0x5562f53e4920 legacy=0x5562f53e53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.667+0000 7f3c8d38d700 1 -- 172.21.15.12:0/2043565195 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562f53e5ad0 con 0x5562f53e4010 2023-10-05T19:40:25.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.667+0000 7f3c8d38d700 1 --2- 172.21.15.12:0/2043565195 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5562f53e4010 0x5562f53e43e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3c8000a5d0 tx=0x7f3c80005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ee174acebf7858 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:25.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.667+0000 7f3c7ffff700 1 -- 172.21.15.12:0/2043565195 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c800086a0 con 0x5562f53e4010 2023-10-05T19:40:25.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.667+0000 7f3c7ffff700 1 -- 172.21.15.12:0/2043565195 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c80008800 con 0x5562f53e4010 2023-10-05T19:40:25.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.667+0000 7f3c7ffff700 1 -- 172.21.15.12:0/2043565195 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c80008ad0 con 0x5562f53e4010 2023-10-05T19:40:25.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.667+0000 7f3c8f858e40 1 -- 172.21.15.12:0/2043565195 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5562f53e4010 msgr2=0x5562f53e43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.667+0000 7f3c8f858e40 1 --2- 172.21.15.12:0/2043565195 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5562f53e4010 0x5562f53e43e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3c8000a5d0 tx=0x7f3c80005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:25.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.667+0000 7f3c8f858e40 1 -- 172.21.15.12:0/2043565195 shutdown_connections 2023-10-05T19:40:25.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.667+0000 7f3c8f858e40 1 --2- 172.21.15.12:0/2043565195 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5562f53e4010 0x5562f53e43e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:25.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.667+0000 7f3c8f858e40 1 -- 172.21.15.12:0/2043565195 >> 172.21.15.12:0/2043565195 conn(0x5562f5357860 msgr2=0x5562f5357c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:25.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.667+0000 7f3c8f858e40 1 -- 172.21.15.12:0/2043565195 shutdown_connections 2023-10-05T19:40:25.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.667+0000 7f3c8f858e40 1 -- 172.21.15.12:0/2043565195 wait complete. 2023-10-05T19:40:25.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.671+0000 7f3c8f858e40 1 Processor -- start 2023-10-05T19:40:25.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.671+0000 7f3c8f858e40 1 -- start start 2023-10-05T19:40:25.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.671+0000 7f3c8f858e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5562f53e4010 0x5562f5403100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:25.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.671+0000 7f3c8f858e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562f52afe30 con 0x5562f53e4010 2023-10-05T19:40:25.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.671+0000 7f3c8d38d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5562f53e4010 0x5562f5403100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:25.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.671+0000 7f3c8d38d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5562f53e4010 0x5562f5403100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57982/0 (socket says 172.21.15.12:57982) 2023-10-05T19:40:25.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.671+0000 7f3c8d38d700 1 -- 172.21.15.12:0/2326412555 learned_addr learned my addr 172.21.15.12:0/2326412555 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:25.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.671+0000 7f3c8d38d700 1 -- 172.21.15.12:0/2326412555 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c80005cf0 con 0x5562f53e4010 2023-10-05T19:40:25.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.671+0000 7f3c8d38d700 1 --2- 172.21.15.12:0/2326412555 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5562f53e4010 0x5562f5403100 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3c80000c00 tx=0x7f3c80015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d04c24512f1337 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:25.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.671+0000 7f3c7effd700 1 -- 172.21.15.12:0/2326412555 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c80008970 con 0x5562f53e4010 2023-10-05T19:40:25.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.671+0000 7f3c7effd700 1 -- 172.21.15.12:0/2326412555 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c80008af0 con 0x5562f53e4010 2023-10-05T19:40:25.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.671+0000 7f3c7effd700 1 -- 172.21.15.12:0/2326412555 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c80008de0 con 0x5562f53e4010 2023-10-05T19:40:25.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.671+0000 7f3c8f858e40 1 -- 172.21.15.12:0/2326412555 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5562f53e4010 msgr2=0x5562f5403100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.671+0000 7f3c8f858e40 1 --2- 172.21.15.12:0/2326412555 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5562f53e4010 0x5562f5403100 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3c80000c00 tx=0x7f3c80015000 comp rx=0 tx=0).stop 2023-10-05T19:40:25.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.671+0000 7f3c8f858e40 1 -- 172.21.15.12:0/2326412555 shutdown_connections 2023-10-05T19:40:25.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.671+0000 7f3c8f858e40 1 --2- 172.21.15.12:0/2326412555 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5562f53e4010 0x5562f5403100 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:25.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.671+0000 7f3c8f858e40 1 -- 172.21.15.12:0/2326412555 >> 172.21.15.12:0/2326412555 conn(0x5562f5357860 msgr2=0x5562f53fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:25.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.671+0000 7f3c8f858e40 1 -- 172.21.15.12:0/2326412555 shutdown_connections 2023-10-05T19:40:25.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.671+0000 7f3c8f858e40 1 -- 172.21.15.12:0/2326412555 wait complete. 2023-10-05T19:40:25.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.671+0000 7f3c8f858e40 1 Processor -- start 2023-10-05T19:40:25.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.671+0000 7f3c8f858e40 1 -- start start 2023-10-05T19:40:25.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.671+0000 7f3c8f858e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5562f53e4010 0x5562f5409d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:25.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.675+0000 7f3c8f858e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562f5282ed0 con 0x5562f53e4010 2023-10-05T19:40:25.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.675+0000 7f3c8d38d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5562f53e4010 0x5562f5409d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:25.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.675+0000 7f3c8d38d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5562f53e4010 0x5562f5409d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:57990/0 (socket says 172.21.15.12:57990) 2023-10-05T19:40:25.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.675+0000 7f3c8d38d700 1 -- 172.21.15.12:0/1863343474 learned_addr learned my addr 172.21.15.12:0/1863343474 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:25.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.675+0000 7f3c8d38d700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562f53e5ad0 con 0x5562f53e4010 2023-10-05T19:40:25.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.675+0000 7f3c8d38d700 1 --2- 172.21.15.12:0/1863343474 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5562f53e4010 0x5562f5409d10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3c8000c010 tx=0x7f3c80003050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:25.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.675+0000 7f3c7d7fa700 1 -- 172.21.15.12:0/1863343474 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c80012ba0 con 0x5562f53e4010 2023-10-05T19:40:25.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.675+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5562f540a250 con 0x5562f53e4010 2023-10-05T19:40:25.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.675+0000 7f3c7d7fa700 1 -- 172.21.15.12:0/1863343474 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c8002a070 con 0x5562f53e4010 2023-10-05T19:40:25.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.675+0000 7f3c7d7fa700 1 -- 172.21.15.12:0/1863343474 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c80012ea0 con 0x5562f53e4010 2023-10-05T19:40:25.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.675+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5562f540a6f0 con 0x5562f53e4010 2023-10-05T19:40:25.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.675+0000 7f3c7d7fa700 1 -- 172.21.15.12:0/1863343474 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f3c80019070 con 0x5562f53e4010 2023-10-05T19:40:25.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.675+0000 7f3c8f858e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.675+0000 7f3c7d7fa700 1 --2- 172.21.15.12:0/1863343474 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3c70030bf0 0x7f3c700330a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:25.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.675+0000 7f3c7d7fa700 1 -- 172.21.15.12:0/1863343474 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f3c8002c070 con 0x5562f53e4010 2023-10-05T19:40:25.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.679+0000 7f3c8cb8c700 1 --2- 172.21.15.12:0/1863343474 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3c70030bf0 0x7f3c700330a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:25.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.679+0000 7f3c8f858e40 1 --2- 172.21.15.12:0/1863343474 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5562f53e4920 0x5562f54158d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:25.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.679+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5415ef0 con 0x5562f53e4920 2023-10-05T19:40:25.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.679+0000 7f3c8cb8c700 1 --2- 172.21.15.12:0/1863343474 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3c70030bf0 0x7f3c700330a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3c84004170 tx=0x7f3c84001380 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:25.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.679+0000 7f3c8db8e700 1 --2- 172.21.15.12:0/1863343474 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5562f53e4920 0x5562f54158d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:25.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.679+0000 7f3c8db8e700 1 --2- 172.21.15.12:0/1863343474 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5562f53e4920 0x5562f54158d0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:25.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.679+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3c8800a5a0 con 0x5562f53e4920 2023-10-05T19:40:25.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.679+0000 7f3c8f858e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:25.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.679+0000 7f3c8f858e40 20 realm 2023-10-05T19:40:25.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.679+0000 7f3c8f858e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.679+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5412f30 con 0x5562f53e4920 2023-10-05T19:40:25.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.679+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3c8800a5a0 con 0x5562f53e4920 2023-10-05T19:40:25.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.679+0000 7f3c8f858e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:25.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.679+0000 7f3c8f858e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:25.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.679+0000 7f3c8f858e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.679+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f54169d0 con 0x5562f53e4920 2023-10-05T19:40:25.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.679+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3c8800a5a0 con 0x5562f53e4920 2023-10-05T19:40:25.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.679+0000 7f3c8f858e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:25.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.679+0000 7f3c8f858e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.679+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f54165e0 con 0x5562f53e4920 2023-10-05T19:40:25.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.679+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f3c8800a5a0 con 0x5562f53e4920 2023-10-05T19:40:25.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.679+0000 7f3c8f858e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:25.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.679+0000 7f3c8f858e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.679+0000 7f3c8f858e40 1 --2- 172.21.15.12:0/1863343474 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5562f5417d70 0x5562f541a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:25.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.679+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541a690 con 0x5562f5417d70 2023-10-05T19:40:25.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.683+0000 7f3c8d38d700 1 --2- 172.21.15.12:0/1863343474 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5562f5417d70 0x5562f541a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:25.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.683+0000 7f3c8d38d700 1 --2- 172.21.15.12:0/1863343474 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5562f5417d70 0x5562f541a150 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:25.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.683+0000 7f3c8d38d700 1 -- 172.21.15.12:0/1863343474 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f3c80022020 con 0x5562f5417d70 2023-10-05T19:40:25.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.683+0000 7f3c8f858e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:25.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.683+0000 7f3c8f858e40 20 searching for the correct realm 2023-10-05T19:40:25.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.683+0000 7f3c8f858e40 1 --2- 172.21.15.12:0/1863343474 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5562f541c480 0x5562f541e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:25.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.683+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541ef40 con 0x5562f541c480 2023-10-05T19:40:25.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.683+0000 7f3c8cb8c700 1 --2- 172.21.15.12:0/1863343474 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5562f541c480 0x5562f541e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:25.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.683+0000 7f3c8cb8c700 1 --2- 172.21.15.12:0/1863343474 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5562f541c480 0x5562f541e890 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:25.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.687+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/1863343474 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f3c84006070 con 0x5562f541c480 2023-10-05T19:40:25.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.687+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541ef40 con 0x5562f5417d70 2023-10-05T19:40:25.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.687+0000 7f3c8d38d700 1 -- 172.21.15.12:0/1863343474 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c80022020 con 0x5562f5417d70 2023-10-05T19:40:25.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.687+0000 7f3c8d38d700 1 --2- 172.21.15.12:0/1863343474 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f3c80007030 0x7f3c80031480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:25.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.687+0000 7f3c8d38d700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541ef40 con 0x7f3c80007030 2023-10-05T19:40:25.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.687+0000 7f3c8db8e700 1 --2- 172.21.15.12:0/1863343474 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f3c80007030 0x7f3c80031480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:25.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.687+0000 7f3c8db8e700 1 --2- 172.21.15.12:0/1863343474 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f3c80007030 0x7f3c80031480 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:25.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.687+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c8800a5a0 con 0x7f3c80007030 2023-10-05T19:40:25.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.687+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541ef40 con 0x5562f53e4920 2023-10-05T19:40:25.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.691+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c8800a5a0 con 0x5562f53e4920 2023-10-05T19:40:25.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.691+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541ef40 con 0x5562f53e4920 2023-10-05T19:40:25.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.691+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c8800a5a0 con 0x5562f53e4920 2023-10-05T19:40:25.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.691+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541ef40 con 0x7f3c80007030 2023-10-05T19:40:25.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.691+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c8800a5a0 con 0x7f3c80007030 2023-10-05T19:40:25.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.691+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541ef40 con 0x7f3c80007030 2023-10-05T19:40:25.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.691+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c8800a5a0 con 0x7f3c80007030 2023-10-05T19:40:25.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.691+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541ef40 con 0x5562f5417d70 2023-10-05T19:40:25.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.691+0000 7f3c8d38d700 1 -- 172.21.15.12:0/1863343474 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c80022020 con 0x5562f5417d70 2023-10-05T19:40:25.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.691+0000 7f3c8d38d700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541ef40 con 0x5562f541c480 2023-10-05T19:40:25.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.695+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/1863343474 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c84006070 con 0x5562f541c480 2023-10-05T19:40:25.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.695+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541ef40 con 0x5562f53e4920 2023-10-05T19:40:25.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.695+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c8800a5a0 con 0x5562f53e4920 2023-10-05T19:40:25.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.695+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541ef40 con 0x7f3c80007030 2023-10-05T19:40:25.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.695+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c8800a5a0 con 0x7f3c80007030 2023-10-05T19:40:25.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.695+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541ef40 con 0x5562f53e4920 2023-10-05T19:40:25.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.695+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c8800a5a0 con 0x5562f53e4920 2023-10-05T19:40:25.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.695+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541ef40 con 0x5562f5417d70 2023-10-05T19:40:25.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.695+0000 7f3c8d38d700 1 -- 172.21.15.12:0/1863343474 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c80022020 con 0x5562f5417d70 2023-10-05T19:40:25.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.695+0000 7f3c8d38d700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541ef40 con 0x5562f541c480 2023-10-05T19:40:25.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.699+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/1863343474 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c84006070 con 0x5562f541c480 2023-10-05T19:40:25.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.699+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541ef40 con 0x5562f541c480 2023-10-05T19:40:25.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.699+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/1863343474 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c84006070 con 0x5562f541c480 2023-10-05T19:40:25.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.699+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541ef40 con 0x7f3c80007030 2023-10-05T19:40:25.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.699+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c8800a5a0 con 0x7f3c80007030 2023-10-05T19:40:25.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.699+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541ef40 con 0x5562f541c480 2023-10-05T19:40:25.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.699+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/1863343474 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c84006070 con 0x5562f541c480 2023-10-05T19:40:25.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.699+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541ef40 con 0x5562f541c480 2023-10-05T19:40:25.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.699+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/1863343474 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c84006070 con 0x5562f541c480 2023-10-05T19:40:25.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.699+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541ef40 con 0x5562f53e4920 2023-10-05T19:40:25.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.703+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c8800a5a0 con 0x5562f53e4920 2023-10-05T19:40:25.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.703+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541ef40 con 0x5562f53e4920 2023-10-05T19:40:25.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.703+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c8800a5a0 con 0x5562f53e4920 2023-10-05T19:40:25.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.703+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541ef40 con 0x7f3c80007030 2023-10-05T19:40:25.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.703+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c8800a5a0 con 0x7f3c80007030 2023-10-05T19:40:25.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.703+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541ef40 con 0x5562f541c480 2023-10-05T19:40:25.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.703+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/1863343474 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c84006070 con 0x5562f541c480 2023-10-05T19:40:25.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.703+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541ef40 con 0x5562f5417d70 2023-10-05T19:40:25.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.703+0000 7f3c8d38d700 1 -- 172.21.15.12:0/1863343474 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c80022020 con 0x5562f5417d70 2023-10-05T19:40:25.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.703+0000 7f3c8d38d700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541ef40 con 0x5562f53e4920 2023-10-05T19:40:25.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.707+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c8800a5a0 con 0x5562f53e4920 2023-10-05T19:40:25.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.707+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541ef40 con 0x7f3c80007030 2023-10-05T19:40:25.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.707+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c8800a5a0 con 0x7f3c80007030 2023-10-05T19:40:25.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.707+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541ef40 con 0x5562f541c480 2023-10-05T19:40:25.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.707+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/1863343474 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c84006070 con 0x5562f541c480 2023-10-05T19:40:25.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.707+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541ef40 con 0x7f3c80007030 2023-10-05T19:40:25.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.707+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c8800a5a0 con 0x7f3c80007030 2023-10-05T19:40:25.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.707+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541ef40 con 0x5562f5417d70 2023-10-05T19:40:25.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.711+0000 7f3c8d38d700 1 -- 172.21.15.12:0/1863343474 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c80022020 con 0x5562f5417d70 2023-10-05T19:40:25.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.711+0000 7f3c8d38d700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541ef40 con 0x5562f541c480 2023-10-05T19:40:25.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.711+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/1863343474 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c84006070 con 0x5562f541c480 2023-10-05T19:40:25.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.711+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541ef40 con 0x7f3c80007030 2023-10-05T19:40:25.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.711+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c8800a5a0 con 0x7f3c80007030 2023-10-05T19:40:25.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.711+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541ef40 con 0x5562f53e4920 2023-10-05T19:40:25.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.711+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f3c8800a5a0 con 0x5562f53e4920 2023-10-05T19:40:25.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.711+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541ef40 con 0x5562f5417d70 2023-10-05T19:40:25.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.715+0000 7f3c8d38d700 1 -- 172.21.15.12:0/1863343474 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f3c80022020 con 0x5562f5417d70 2023-10-05T19:40:25.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.715+0000 7f3c8f858e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:25.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.715+0000 7f3c8f858e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:25.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.715+0000 7f3c8f858e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:25.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.715+0000 7f3c8f858e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:25.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.715+0000 7f3c8f858e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.715+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541b230 con 0x5562f53e4920 2023-10-05T19:40:25.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.715+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3c8800a5a0 con 0x5562f53e4920 2023-10-05T19:40:25.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.715+0000 7f3c8f858e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:25.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.715+0000 7f3c8f858e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.715+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541ae60 con 0x5562f53e4920 2023-10-05T19:40:25.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.715+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f3c8800a5a0 con 0x5562f53e4920 2023-10-05T19:40:25.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.715+0000 7f3c8f858e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:25.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.715+0000 7f3c8f858e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.715+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541f3b0 con 0x5562f541c480 2023-10-05T19:40:25.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.715+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/1863343474 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f3c84006070 con 0x5562f541c480 2023-10-05T19:40:25.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.715+0000 7f3c8f858e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:25.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.715+0000 7f3c8f858e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.715+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541f760 con 0x5562f541c480 2023-10-05T19:40:25.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.715+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/1863343474 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f3c84006070 con 0x5562f541c480 2023-10-05T19:40:25.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.715+0000 7f3c8f858e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:25.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.715+0000 7f3c8f858e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.715+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541fb10 con 0x5562f53e4920 2023-10-05T19:40:25.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.719+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3c8800a5a0 con 0x5562f53e4920 2023-10-05T19:40:25.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.719+0000 7f3c8f858e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:25.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.719+0000 7f3c8f858e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.719+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541ff70 con 0x5562f53e4920 2023-10-05T19:40:25.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.719+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f3c8800a5a0 con 0x5562f53e4920 2023-10-05T19:40:25.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.719+0000 7f3c8f858e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:25.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.719+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541ff40 con 0x5562f541c480 2023-10-05T19:40:25.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.719+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/1863343474 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f3c84006070 con 0x5562f541c480 2023-10-05T19:40:25.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.719+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541ff40 con 0x5562f5417d70 2023-10-05T19:40:25.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.723+0000 7f3c8d38d700 1 -- 172.21.15.12:0/1863343474 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c80022020 con 0x5562f5417d70 2023-10-05T19:40:25.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.723+0000 7f3c8d38d700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541ff40 con 0x7f3c80007030 2023-10-05T19:40:25.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.723+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c8800a5a0 con 0x7f3c80007030 2023-10-05T19:40:25.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.723+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541fb10 con 0x5562f53e4920 2023-10-05T19:40:25.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.723+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c8800a5a0 con 0x5562f53e4920 2023-10-05T19:40:25.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.723+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541fb10 con 0x5562f53e4920 2023-10-05T19:40:25.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.723+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c8800a5a0 con 0x5562f53e4920 2023-10-05T19:40:25.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.723+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541fb10 con 0x7f3c80007030 2023-10-05T19:40:25.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.723+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c8800a5a0 con 0x7f3c80007030 2023-10-05T19:40:25.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.723+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541fb10 con 0x7f3c80007030 2023-10-05T19:40:25.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.727+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c8800a5a0 con 0x7f3c80007030 2023-10-05T19:40:25.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.727+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541fb10 con 0x5562f5417d70 2023-10-05T19:40:25.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.727+0000 7f3c8d38d700 1 -- 172.21.15.12:0/1863343474 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c80022020 con 0x5562f5417d70 2023-10-05T19:40:25.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.727+0000 7f3c8d38d700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541fb10 con 0x5562f541c480 2023-10-05T19:40:25.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.727+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/1863343474 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c84006070 con 0x5562f541c480 2023-10-05T19:40:25.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.727+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541fb10 con 0x5562f53e4920 2023-10-05T19:40:25.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.727+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c8800a5a0 con 0x5562f53e4920 2023-10-05T19:40:25.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.727+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541fb10 con 0x7f3c80007030 2023-10-05T19:40:25.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.731+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c8800a5a0 con 0x7f3c80007030 2023-10-05T19:40:25.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.731+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541fb10 con 0x5562f53e4920 2023-10-05T19:40:25.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.731+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c8800a5a0 con 0x5562f53e4920 2023-10-05T19:40:25.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.731+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541fb10 con 0x5562f5417d70 2023-10-05T19:40:25.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.731+0000 7f3c8d38d700 1 -- 172.21.15.12:0/1863343474 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c80022020 con 0x5562f5417d70 2023-10-05T19:40:25.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.731+0000 7f3c8d38d700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541fb10 con 0x5562f541c480 2023-10-05T19:40:25.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.731+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/1863343474 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c84006070 con 0x5562f541c480 2023-10-05T19:40:25.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.731+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541fb10 con 0x5562f541c480 2023-10-05T19:40:25.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.731+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/1863343474 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c84006070 con 0x5562f541c480 2023-10-05T19:40:25.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.731+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541fb10 con 0x7f3c80007030 2023-10-05T19:40:25.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.735+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c8800a5a0 con 0x7f3c80007030 2023-10-05T19:40:25.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.735+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541fb10 con 0x5562f541c480 2023-10-05T19:40:25.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.735+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/1863343474 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c84006070 con 0x5562f541c480 2023-10-05T19:40:25.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.735+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541fb10 con 0x5562f541c480 2023-10-05T19:40:25.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.735+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/1863343474 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c84006070 con 0x5562f541c480 2023-10-05T19:40:25.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.735+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541fb10 con 0x5562f53e4920 2023-10-05T19:40:25.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.735+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c8800a5a0 con 0x5562f53e4920 2023-10-05T19:40:25.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.735+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541fb10 con 0x5562f53e4920 2023-10-05T19:40:25.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.735+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c8800a5a0 con 0x5562f53e4920 2023-10-05T19:40:25.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.735+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541fb10 con 0x7f3c80007030 2023-10-05T19:40:25.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.739+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c8800a5a0 con 0x7f3c80007030 2023-10-05T19:40:25.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.739+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541fb10 con 0x5562f541c480 2023-10-05T19:40:25.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.739+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/1863343474 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c84006070 con 0x5562f541c480 2023-10-05T19:40:25.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.739+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541fb10 con 0x5562f5417d70 2023-10-05T19:40:25.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.739+0000 7f3c8d38d700 1 -- 172.21.15.12:0/1863343474 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c80022020 con 0x5562f5417d70 2023-10-05T19:40:25.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.739+0000 7f3c8d38d700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541fb10 con 0x5562f53e4920 2023-10-05T19:40:25.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.739+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c8800a5a0 con 0x5562f53e4920 2023-10-05T19:40:25.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.739+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541fb10 con 0x7f3c80007030 2023-10-05T19:40:25.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.739+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c8800a5a0 con 0x7f3c80007030 2023-10-05T19:40:25.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.739+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541fb10 con 0x5562f541c480 2023-10-05T19:40:25.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.743+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/1863343474 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c84006070 con 0x5562f541c480 2023-10-05T19:40:25.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.743+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541fb10 con 0x7f3c80007030 2023-10-05T19:40:25.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.743+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c8800a5a0 con 0x7f3c80007030 2023-10-05T19:40:25.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.743+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541fb10 con 0x5562f5417d70 2023-10-05T19:40:25.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.743+0000 7f3c8d38d700 1 -- 172.21.15.12:0/1863343474 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c80022020 con 0x5562f5417d70 2023-10-05T19:40:25.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.743+0000 7f3c8d38d700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541fb10 con 0x5562f541c480 2023-10-05T19:40:25.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.743+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/1863343474 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c84006070 con 0x5562f541c480 2023-10-05T19:40:25.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.743+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541fb10 con 0x7f3c80007030 2023-10-05T19:40:25.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.747+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c8800a5a0 con 0x7f3c80007030 2023-10-05T19:40:25.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.747+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541fb10 con 0x5562f53e4920 2023-10-05T19:40:25.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.747+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f3c8800a5a0 con 0x5562f53e4920 2023-10-05T19:40:25.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.747+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541fb10 con 0x5562f5417d70 2023-10-05T19:40:25.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.747+0000 7f3c8d38d700 1 -- 172.21.15.12:0/1863343474 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f3c80022020 con 0x5562f5417d70 2023-10-05T19:40:25.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.747+0000 7f3c8f858e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:25.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.747+0000 7f3c8f858e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:25.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.747+0000 7f3c8f858e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:25.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.747+0000 7f3c8f858e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:25.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.747+0000 7f3c8f858e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.747+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f541ff40 con 0x5562f53e4920 2023-10-05T19:40:25.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.747+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f3c8800a5a0 con 0x5562f53e4920 2023-10-05T19:40:25.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.747+0000 7f3c8f858e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:25.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.747+0000 7f3c8f858e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.747+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f54203a0 con 0x5562f5417d70 2023-10-05T19:40:25.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.747+0000 7f3c8d38d700 1 -- 172.21.15.12:0/1863343474 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f3c80022020 con 0x5562f5417d70 2023-10-05T19:40:25.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.747+0000 7f3c8f858e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:25.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.751+0000 7f3c8f858e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.751+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5420c90 con 0x5562f53e4920 2023-10-05T19:40:25.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.751+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f3c8800a5a0 con 0x5562f53e4920 2023-10-05T19:40:25.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.751+0000 7f3c8f858e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:25.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.751+0000 7f3c8f858e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.751+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5420c90 con 0x5562f541c480 2023-10-05T19:40:25.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.751+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/1863343474 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f3c84006070 con 0x5562f541c480 2023-10-05T19:40:25.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.751+0000 7f3c8f858e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:25.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.751+0000 7f3c8f858e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.751+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5421400 con 0x7f3c80007030 2023-10-05T19:40:25.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.751+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f3c8800a5a0 con 0x7f3c80007030 2023-10-05T19:40:25.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.751+0000 7f3c8f858e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:25.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.751+0000 7f3c8f858e40 20 zone default found 2023-10-05T19:40:25.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.751+0000 7f3c8f858e40 4 Realm: () 2023-10-05T19:40:25.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.751+0000 7f3c8f858e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:25.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.751+0000 7f3c8f858e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:25.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.751+0000 7f3c8f858e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:25.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.751+0000 7f3c8f858e40 20 zonegroup default 2023-10-05T19:40:25.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.751+0000 7f3c8f858e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.751+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5421140 con 0x7f3c80007030 2023-10-05T19:40:25.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.751+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f3c8800a5a0 con 0x7f3c80007030 2023-10-05T19:40:25.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.751+0000 7f3c8f858e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:25.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.751+0000 7f3c8f858e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.751+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5422ac0 con 0x5562f541c480 2023-10-05T19:40:25.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.755+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/1863343474 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f3c84006070 con 0x5562f541c480 2023-10-05T19:40:25.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.755+0000 7f3c8f858e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:25.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.755+0000 7f3c8f858e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.755+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5422e70 con 0x5562f541c480 2023-10-05T19:40:25.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.755+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/1863343474 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f3c84006070 con 0x5562f541c480 2023-10-05T19:40:25.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.755+0000 7f3c8f858e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:25.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.755+0000 7f3c8f858e40 20 started sync module instance, tier type = 2023-10-05T19:40:25.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.755+0000 7f3c8f858e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:25.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.755+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5422180 con 0x5562f5417d70 2023-10-05T19:40:25.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.759+0000 7f3c8d38d700 1 -- 172.21.15.12:0/1863343474 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'117 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c80022020 con 0x5562f5417d70 2023-10-05T19:40:25.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.759+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93883509908784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f54208c0 con 0x5562f5417d70 2023-10-05T19:40:25.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.759+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5423c00 con 0x5562f53e4920 2023-10-05T19:40:25.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.763+0000 7f3c8d38d700 1 -- 172.21.15.12:0/1863343474 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 93883509908784] v28'118 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c80022020 con 0x5562f5417d70 2023-10-05T19:40:25.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.763+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'117 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c8800a5a0 con 0x5562f53e4920 2023-10-05T19:40:25.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.763+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93883509912096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5423b60 con 0x5562f53e4920 2023-10-05T19:40:25.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.763+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5424570 con 0x5562f541c480 2023-10-05T19:40:25.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.767+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 93883509912096] v28'118 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c8800a5a0 con 0x5562f53e4920 2023-10-05T19:40:25.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.767+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/1863343474 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'117 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c84006070 con 0x5562f541c480 2023-10-05T19:40:25.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.767+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93883509913200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f54241a0 con 0x5562f541c480 2023-10-05T19:40:25.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.767+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5425300 con 0x5562f5417d70 2023-10-05T19:40:25.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.767+0000 7f3c8d38d700 1 -- 172.21.15.12:0/1863343474 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'117 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c80022020 con 0x5562f5417d70 2023-10-05T19:40:25.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.767+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 93883509919248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f54257d0 con 0x5562f5417d70 2023-10-05T19:40:25.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.767+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5426390 con 0x5562f53e4920 2023-10-05T19:40:25.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.767+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/1863343474 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 93883509913200] v28'118 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c84006070 con 0x5562f541c480 2023-10-05T19:40:25.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.771+0000 7f3c8d38d700 1 -- 172.21.15.12:0/1863343474 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 93883509919248] v28'118 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c80022020 con 0x5562f5417d70 2023-10-05T19:40:25.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.771+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'117 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c8800a5a0 con 0x5562f53e4920 2023-10-05T19:40:25.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.771+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 93883509917984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5425e60 con 0x5562f53e4920 2023-10-05T19:40:25.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.771+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5426c80 con 0x7f3c80007030 2023-10-05T19:40:25.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.775+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 93883509917984] v28'118 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c8800a5a0 con 0x5562f53e4920 2023-10-05T19:40:25.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.775+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'117 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c8800a5a0 con 0x7f3c80007030 2023-10-05T19:40:25.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.775+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 93883509920352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f54269b0 con 0x7f3c80007030 2023-10-05T19:40:25.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.775+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5427250 con 0x7f3c80007030 2023-10-05T19:40:25.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.779+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(95 notify.5 [watch watch cookie 93883509920352] v28'118 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c8800a5a0 con 0x7f3c80007030 2023-10-05T19:40:25.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.779+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(96 notify.6 [create] v28'117 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c8800a5a0 con 0x7f3c80007030 2023-10-05T19:40:25.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.779+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93883509925616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5427030 con 0x7f3c80007030 2023-10-05T19:40:25.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.779+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5427a70 con 0x7f3c80007030 2023-10-05T19:40:25.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.779+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(97 notify.6 [watch watch cookie 93883509925616] v28'118 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c8800a5a0 con 0x7f3c80007030 2023-10-05T19:40:25.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.779+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(98 notify.7 [create] v28'117 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c8800a5a0 con 0x7f3c80007030 2023-10-05T19:40:25.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.779+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93883509929904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5427030 con 0x7f3c80007030 2023-10-05T19:40:25.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.779+0000 7f3c8f858e40 20 add_watcher() i=0 2023-10-05T19:40:25.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.779+0000 7f3c8f858e40 20 add_watcher() i=1 2023-10-05T19:40:25.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.779+0000 7f3c8f858e40 20 add_watcher() i=2 2023-10-05T19:40:25.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.779+0000 7f3c8f858e40 20 add_watcher() i=3 2023-10-05T19:40:25.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.779+0000 7f3c8f858e40 20 add_watcher() i=4 2023-10-05T19:40:25.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.779+0000 7f3c8f858e40 20 add_watcher() i=5 2023-10-05T19:40:25.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.779+0000 7f3c8f858e40 20 add_watcher() i=6 2023-10-05T19:40:25.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.783+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 93883509929904] v28'118 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c8800a5a0 con 0x7f3c80007030 2023-10-05T19:40:25.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.783+0000 7f3c8f858e40 20 add_watcher() i=7 2023-10-05T19:40:25.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.783+0000 7f3c8f858e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:25.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.783+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f542aea0 con 0x5562f53e4920 2023-10-05T19:40:25.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.783+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f3c8800a5a0 con 0x5562f53e4920 2023-10-05T19:40:25.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.783+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93883509927184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5428310 con 0x5562f53e4920 2023-10-05T19:40:25.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.787+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 93883509927184] v28'113 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f3c8800a5a0 con 0x5562f53e4920 2023-10-05T19:40:25.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.787+0000 7f3c5e7e4700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:25.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.787+0000 7f3c8f858e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:25.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.787+0000 7f3c8f858e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:25.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.787+0000 7f3c8f858e40 1 Processor -- start 2023-10-05T19:40:25.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.787+0000 7f3c8f858e40 1 -- start start 2023-10-05T19:40:25.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.787+0000 7f3c8f858e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5562f5435520 0x5562f54358f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:25.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.787+0000 7f3c8f858e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562f5435e30 con 0x5562f5435520 2023-10-05T19:40:25.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.787+0000 7f3c8cb8c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5562f5435520 0x5562f54358f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:25.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.787+0000 7f3c8cb8c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5562f5435520 0x5562f54358f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58002/0 (socket says 172.21.15.12:58002) 2023-10-05T19:40:25.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.787+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/3645788771 learned_addr learned my addr 172.21.15.12:0/3645788771 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:25.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.787+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/3645788771 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562f5435f70 con 0x5562f5435520 2023-10-05T19:40:25.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.787+0000 7f3c8cb8c700 1 --2- 172.21.15.12:0/3645788771 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5562f5435520 0x5562f54358f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3c84000c00 tx=0x7f3c8401e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d948526bd1188696 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:25.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.791+0000 7f3c5dfe3700 1 -- 172.21.15.12:0/3645788771 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c84013040 con 0x5562f5435520 2023-10-05T19:40:25.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.791+0000 7f3c5dfe3700 1 -- 172.21.15.12:0/3645788771 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c84022040 con 0x5562f5435520 2023-10-05T19:40:25.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.791+0000 7f3c5dfe3700 1 -- 172.21.15.12:0/3645788771 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c8400e950 con 0x5562f5435520 2023-10-05T19:40:25.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.791+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3645788771 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5562f5435520 msgr2=0x5562f54358f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.791+0000 7f3c8f858e40 1 --2- 172.21.15.12:0/3645788771 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5562f5435520 0x5562f54358f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3c84000c00 tx=0x7f3c8401e040 comp rx=0 tx=0).stop 2023-10-05T19:40:25.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.791+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3645788771 shutdown_connections 2023-10-05T19:40:25.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.791+0000 7f3c8f858e40 1 --2- 172.21.15.12:0/3645788771 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5562f5435520 0x5562f54358f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:25.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.791+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3645788771 >> 172.21.15.12:0/3645788771 conn(0x5562f54308e0 msgr2=0x5562f542ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:25.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.791+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3645788771 shutdown_connections 2023-10-05T19:40:25.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.791+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3645788771 wait complete. 2023-10-05T19:40:25.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.791+0000 7f3c8f858e40 1 Processor -- start 2023-10-05T19:40:25.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.791+0000 7f3c8f858e40 1 -- start start 2023-10-05T19:40:25.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.791+0000 7f3c8f858e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5562f5435520 0x5562f543b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:25.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.791+0000 7f3c8f858e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562f543c100 con 0x5562f5435520 2023-10-05T19:40:25.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.791+0000 7f3c8cb8c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5562f5435520 0x5562f543b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:25.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.791+0000 7f3c8cb8c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5562f5435520 0x5562f543b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58006/0 (socket says 172.21.15.12:58006) 2023-10-05T19:40:25.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.791+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/3505572453 learned_addr learned my addr 172.21.15.12:0/3505572453 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:25.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.795+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/3505572453 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c84034040 con 0x5562f5435520 2023-10-05T19:40:25.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.795+0000 7f3c8cb8c700 1 --2- 172.21.15.12:0/3505572453 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5562f5435520 0x5562f543b100 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3c8400efd0 tx=0x7f3c8403d030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:25.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.795+0000 7f3bcffff700 1 -- 172.21.15.12:0/3505572453 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c84050470 con 0x5562f5435520 2023-10-05T19:40:25.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.795+0000 7f3bcffff700 1 -- 172.21.15.12:0/3505572453 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c840505d0 con 0x5562f5435520 2023-10-05T19:40:25.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.795+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3505572453 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5562f543b700 con 0x5562f5435520 2023-10-05T19:40:25.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.795+0000 7f3bcffff700 1 -- 172.21.15.12:0/3505572453 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c84050780 con 0x5562f5435520 2023-10-05T19:40:25.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.795+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3505572453 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5562f543bb40 con 0x5562f5435520 2023-10-05T19:40:25.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.795+0000 7f3bcffff700 1 -- 172.21.15.12:0/3505572453 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f3c8404d070 con 0x5562f5435520 2023-10-05T19:40:25.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.795+0000 7f3bcffff700 1 --2- 172.21.15.12:0/3505572453 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3bd4030c50 0x7f3bd4033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:25.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.795+0000 7f3bcffff700 1 -- 172.21.15.12:0/3505572453 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f3c8404a170 con 0x5562f5435520 2023-10-05T19:40:25.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.795+0000 7f3c8db8e700 1 --2- 172.21.15.12:0/3505572453 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3bd4030c50 0x7f3bd4033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:25.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.795+0000 7f3c8f858e40 1 --2- 172.21.15.12:0/3505572453 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5562f5445d20 0x5562f5448130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:25.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.795+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3505572453 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5448780 con 0x5562f5445d20 2023-10-05T19:40:25.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.795+0000 7f3c8d38d700 1 --2- 172.21.15.12:0/3505572453 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5562f5445d20 0x5562f5448130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:25.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.799+0000 7f3c8db8e700 1 --2- 172.21.15.12:0/3505572453 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3bd4030c50 0x7f3bd4033100 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x5562f54272e0 tx=0x7f3c8800ae20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:25.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.799+0000 7f3c8d38d700 1 --2- 172.21.15.12:0/3505572453 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5562f5445d20 0x5562f5448130 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:25.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.799+0000 7f3c8d38d700 1 -- 172.21.15.12:0/3505572453 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v28'23 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f3c80022020 con 0x5562f5445d20 2023-10-05T19:40:25.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.799+0000 7f3c8f858e40 1 --2- 172.21.15.12:0/3505572453 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x5562f5449450 0x5562f544b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:25.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.799+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3505572453 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f544bee0 con 0x5562f5449450 2023-10-05T19:40:25.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.799+0000 7f3c8cb8c700 1 --2- 172.21.15.12:0/3505572453 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x5562f5449450 0x5562f544b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:25.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.803+0000 7f3c8cb8c700 1 --2- 172.21.15.12:0/3505572453 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x5562f5449450 0x5562f544b830 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:25.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.803+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/3505572453 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v28'14 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f3c84006070 con 0x5562f5449450 2023-10-05T19:40:25.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.803+0000 7f3c8f858e40 1 --2- 172.21.15.12:0/3505572453 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5562f544cbb0 0x5562f544ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:25.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.803+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3505572453 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f544f6c0 con 0x5562f544cbb0 2023-10-05T19:40:25.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.803+0000 7f3c8db8e700 1 --2- 172.21.15.12:0/3505572453 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5562f544cbb0 0x5562f544ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:25.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.807+0000 7f3c8db8e700 1 --2- 172.21.15.12:0/3505572453 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5562f544cbb0 0x5562f544ef90 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:25.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.807+0000 7f3c8db8e700 1 -- 172.21.15.12:0/3505572453 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v28'14 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f3c8803e070 con 0x5562f544cbb0 2023-10-05T19:40:25.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.807+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3505572453 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f544fd10 con 0x5562f5445d20 2023-10-05T19:40:25.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.811+0000 7f3c8d38d700 1 -- 172.21.15.12:0/3505572453 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v28'114 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f3c80022020 con 0x5562f5445d20 2023-10-05T19:40:25.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.811+0000 7f3c8f858e40 1 --2- 172.21.15.12:0/3505572453 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5562f54509b0 0x5562f5452d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:25.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.811+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3505572453 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f54533f0 con 0x5562f54509b0 2023-10-05T19:40:25.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.811+0000 7f3c8d38d700 1 --2- 172.21.15.12:0/3505572453 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5562f54509b0 0x5562f5452d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:25.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.811+0000 7f3c8d38d700 1 --2- 172.21.15.12:0/3505572453 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5562f54509b0 0x5562f5452d90 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:25.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.815+0000 7f3c8d38d700 1 -- 172.21.15.12:0/3505572453 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v28'32 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f3c80022020 con 0x5562f54509b0 2023-10-05T19:40:25.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.815+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3505572453 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5453a40 con 0x5562f544cbb0 2023-10-05T19:40:25.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.815+0000 7f3c8db8e700 1 -- 172.21.15.12:0/3505572453 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v28'24 uv8 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f3c8803e070 con 0x5562f544cbb0 2023-10-05T19:40:25.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.815+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3505572453 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5454060 con 0x5562f5445d20 2023-10-05T19:40:25.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.819+0000 7f3c8d38d700 1 -- 172.21.15.12:0/3505572453 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v28'16 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f3c80022020 con 0x5562f5445d20 2023-10-05T19:40:25.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.819+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3505572453 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5454060 con 0x5562f5449450 2023-10-05T19:40:25.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.823+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/3505572453 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v28'21 uv8 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f3c84061040 con 0x5562f5449450 2023-10-05T19:40:25.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.823+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3505572453 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5454680 con 0x5562f54509b0 2023-10-05T19:40:25.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.827+0000 7f3c8d38d700 1 -- 172.21.15.12:0/3505572453 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v28'14 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f3c80022020 con 0x5562f54509b0 2023-10-05T19:40:25.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.827+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3505572453 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5454680 con 0x5562f544cbb0 2023-10-05T19:40:25.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.827+0000 7f3c8db8e700 1 -- 172.21.15.12:0/3505572453 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v28'20 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f3c8803e070 con 0x5562f544cbb0 2023-10-05T19:40:25.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.827+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3505572453 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5454b70 con 0x5562f54509b0 2023-10-05T19:40:25.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.831+0000 7f3c8d38d700 1 -- 172.21.15.12:0/3505572453 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v28'33 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3c80022020 con 0x5562f54509b0 2023-10-05T19:40:25.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.831+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3505572453 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5454bb0 con 0x5562f5449450 2023-10-05T19:40:25.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.835+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/3505572453 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v28'16 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3c84061040 con 0x5562f5449450 2023-10-05T19:40:25.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.835+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3505572453 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5455250 con 0x5562f54509b0 2023-10-05T19:40:25.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.835+0000 7f3c8d38d700 1 -- 172.21.15.12:0/3505572453 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v28'14 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3c80022020 con 0x5562f54509b0 2023-10-05T19:40:25.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.839+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3505572453 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5455250 con 0x5562f5449450 2023-10-05T19:40:25.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.839+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/3505572453 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v28'29 uv12 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3c84061040 con 0x5562f5449450 2023-10-05T19:40:25.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.839+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3505572453 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5455250 con 0x5562f5445d20 2023-10-05T19:40:25.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.843+0000 7f3c8d38d700 1 -- 172.21.15.12:0/3505572453 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v28'115 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3c80022020 con 0x5562f5445d20 2023-10-05T19:40:25.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.843+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3505572453 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5455250 con 0x5562f544cbb0 2023-10-05T19:40:25.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.847+0000 7f3c8db8e700 1 -- 172.21.15.12:0/3505572453 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v28'26 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3c8803e070 con 0x5562f544cbb0 2023-10-05T19:40:25.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.847+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3505572453 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5455250 con 0x5562f54509b0 2023-10-05T19:40:25.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.847+0000 7f3c8d38d700 1 -- 172.21.15.12:0/3505572453 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v28'34 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3c80022020 con 0x5562f54509b0 2023-10-05T19:40:25.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.847+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3505572453 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5455250 con 0x5562f544cbb0 2023-10-05T19:40:25.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.851+0000 7f3c8db8e700 1 -- 172.21.15.12:0/3505572453 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v28'14 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3c8803e070 con 0x5562f544cbb0 2023-10-05T19:40:25.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.851+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3505572453 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5455250 con 0x5562f5445d20 2023-10-05T19:40:25.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.851+0000 7f3c8d38d700 1 -- 172.21.15.12:0/3505572453 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v28'27 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3c80022020 con 0x5562f5445d20 2023-10-05T19:40:25.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.851+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3505572453 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5455250 con 0x5562f5445d20 2023-10-05T19:40:25.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.855+0000 7f3c8d38d700 1 -- 172.21.15.12:0/3505572453 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v28'116 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3c80022020 con 0x5562f5445d20 2023-10-05T19:40:25.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.855+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3505572453 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5455680 con 0x5562f544cbb0 2023-10-05T19:40:25.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.859+0000 7f3c8db8e700 1 -- 172.21.15.12:0/3505572453 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v28'27 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3c8803e070 con 0x5562f544cbb0 2023-10-05T19:40:25.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.859+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3505572453 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5455680 con 0x5562f54509b0 2023-10-05T19:40:25.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.859+0000 7f3c8d38d700 1 -- 172.21.15.12:0/3505572453 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v28'24 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3c80022020 con 0x5562f54509b0 2023-10-05T19:40:25.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.859+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3505572453 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5455680 con 0x5562f544cbb0 2023-10-05T19:40:25.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.863+0000 7f3c8db8e700 1 -- 172.21.15.12:0/3505572453 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v28'16 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3c8803e070 con 0x5562f544cbb0 2023-10-05T19:40:25.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.863+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3505572453 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5455680 con 0x5562f544cbb0 2023-10-05T19:40:25.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.863+0000 7f3c8db8e700 1 -- 172.21.15.12:0/3505572453 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v28'22 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3c8803e070 con 0x5562f544cbb0 2023-10-05T19:40:25.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.863+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3505572453 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5455680 con 0x5562f544cbb0 2023-10-05T19:40:25.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.867+0000 7f3c8db8e700 1 -- 172.21.15.12:0/3505572453 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v28'11 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3c8803e070 con 0x5562f544cbb0 2023-10-05T19:40:25.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.867+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3505572453 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5455680 con 0x5562f544cbb0 2023-10-05T19:40:25.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.867+0000 7f3c8db8e700 1 -- 172.21.15.12:0/3505572453 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v28'25 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3c8803e070 con 0x5562f544cbb0 2023-10-05T19:40:25.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.871+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3505572453 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5455680 con 0x5562f5449450 2023-10-05T19:40:25.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.871+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/3505572453 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v28'19 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3c84061040 con 0x5562f5449450 2023-10-05T19:40:25.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.871+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3505572453 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5455680 con 0x5562f544cbb0 2023-10-05T19:40:25.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.875+0000 7f3c8db8e700 1 -- 172.21.15.12:0/3505572453 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v28'14 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3c8803e070 con 0x5562f544cbb0 2023-10-05T19:40:25.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.875+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3505572453 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5455680 con 0x5562f5449450 2023-10-05T19:40:25.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.875+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/3505572453 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v28'22 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3c84061040 con 0x5562f5449450 2023-10-05T19:40:25.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.875+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3505572453 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5455680 con 0x5562f544cbb0 2023-10-05T19:40:25.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.879+0000 7f3c8db8e700 1 -- 172.21.15.12:0/3505572453 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v28'28 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3c8803e070 con 0x5562f544cbb0 2023-10-05T19:40:25.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.879+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3505572453 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5455680 con 0x5562f5449450 2023-10-05T19:40:25.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.883+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/3505572453 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v28'30 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3c84061040 con 0x5562f5449450 2023-10-05T19:40:25.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.883+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3505572453 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5455680 con 0x5562f54509b0 2023-10-05T19:40:25.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.883+0000 7f3c8d38d700 1 -- 172.21.15.12:0/3505572453 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v28'20 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3c80022020 con 0x5562f54509b0 2023-10-05T19:40:25.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.887+0000 7f3bcd7fa700 20 reqs_thread_entry: start 2023-10-05T19:40:25.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.887+0000 7f3bc6ffd700 20 reqs_thread_entry: start 2023-10-05T19:40:25.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.887+0000 7f3c8f858e40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:25.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.887+0000 7f3c8f858e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:25.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.887+0000 7f3bc57fa700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:56 2023 2023-10-05T19:40:25.897 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:25.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.887+0000 7f3c8f858e40 20 get_system_obj_state: rctx=0x7ffd3ded09a8 obj=default.rgw.meta:users.uid:fud state=0x5562f52bb0b0 s->prefetch_data=0 2023-10-05T19:40:25.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.887+0000 7f3bc4ff9700 20 reqs_thread_entry: start 2023-10-05T19:40:25.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.887+0000 7f3c8f858e40 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2023-10-05T19:40:25.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.887+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:102 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f54b3c60 con 0x7f3c80007030 2023-10-05T19:40:25.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.887+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(102 fud [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7f3c8803e070 con 0x7f3c80007030 2023-10-05T19:40:25.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.891+0000 7f3c8f858e40 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x16 2023-10-05T19:40:25.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.891+0000 7f3c8f858e40 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2023-10-05T19:40:25.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.891+0000 7f3c8f858e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:25.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.891+0000 7f3c8f858e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:25.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.891+0000 7f3c8f858e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:25.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.891+0000 7f3c8f858e40 10 cache get: name=default.rgw.meta+users.uid+fud : type miss (requested=0x13, cached=0x16) 2023-10-05T19:40:25.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.891+0000 7f3c8f858e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.891+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:103 8.e 8:75f48285:users.uid::fud:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f54b66f0 con 0x7f3c80007030 2023-10-05T19:40:25.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.891+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(103 fud [call,call out=48b,read 0~346 out=346b,getxattrs out=75b] v0'0 uv2 ondisk = 0) v8 ==== 273+0+469 (crc 0 0 0) 0x7f3c8803e070 con 0x7f3c80007030 2023-10-05T19:40:25.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.891+0000 7f3c8f858e40 20 rados_obj.operate() r=0 bl.length=346 2023-10-05T19:40:25.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.891+0000 7f3c8f858e40 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x13 2023-10-05T19:40:25.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.891+0000 7f3c8f858e40 10 moving default.rgw.meta+users.uid+fud to cache LRU end 2023-10-05T19:40:25.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.891+0000 7f3c8f858e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:25.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.891+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:104 8.1 8:810902b1:users.uid::fud.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f54b66f0 con 0x7f3c80007030 2023-10-05T19:40:25.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.891+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(104 fud.buckets [call out=183b] v0'0 uv1 ondisk = 0) v8 ==== 155+0+183 (crc 0 0 0) 0x7f3c8803e070 con 0x7f3c80007030 2023-10-05T19:40:25.901 INFO:teuthology.orchestra.run.smithi012.stderr:could not remove user: unable to remove user, must specify purge data to remove user with buckets 2023-10-05T19:40:25.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.891+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:105 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93883509908784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5472210 con 0x5562f5417d70 2023-10-05T19:40:25.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.895+0000 7f3c8d38d700 1 -- 172.21.15.12:0/1863343474 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(105 notify.0 [watch unwatch cookie 93883509908784] v28'119 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c80022020 con 0x5562f5417d70 2023-10-05T19:40:25.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.895+0000 7f3c8f858e40 20 remove_watcher() i=0 2023-10-05T19:40:25.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.895+0000 7f3c8f858e40 2 removed watcher, disabling cache 2023-10-05T19:40:25.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.895+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:106 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93883509912096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f53f99e0 con 0x5562f53e4920 2023-10-05T19:40:25.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.899+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(106 notify.1 [watch unwatch cookie 93883509912096] v28'119 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c8803e070 con 0x5562f53e4920 2023-10-05T19:40:25.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.899+0000 7f3c8f858e40 20 remove_watcher() i=1 2023-10-05T19:40:25.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.899+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:107 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93883509913200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f53f99e0 con 0x5562f541c480 2023-10-05T19:40:25.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.899+0000 7f3c8cb8c700 1 -- 172.21.15.12:0/1863343474 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(107 notify.2 [watch unwatch cookie 93883509913200] v28'119 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c84061040 con 0x5562f541c480 2023-10-05T19:40:25.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.899+0000 7f3c8f858e40 20 remove_watcher() i=2 2023-10-05T19:40:25.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.899+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:108 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93883509919248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5423710 con 0x5562f5417d70 2023-10-05T19:40:25.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.903+0000 7f3c8d38d700 1 -- 172.21.15.12:0/1863343474 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(108 notify.3 [watch unwatch cookie 93883509919248] v28'119 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c80022020 con 0x5562f5417d70 2023-10-05T19:40:25.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.903+0000 7f3c8f858e40 20 remove_watcher() i=3 2023-10-05T19:40:25.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.903+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:109 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93883509917984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f53f99e0 con 0x5562f53e4920 2023-10-05T19:40:25.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.907+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(109 notify.4 [watch unwatch cookie 93883509917984] v28'119 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c8803e070 con 0x5562f53e4920 2023-10-05T19:40:25.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.907+0000 7f3c8f858e40 20 remove_watcher() i=4 2023-10-05T19:40:25.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.907+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:110 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93883509920352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f53f99e0 con 0x7f3c80007030 2023-10-05T19:40:25.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.911+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.0 v2:172.21.15.8:6800/27563 30 ==== osd_op_reply(110 notify.5 [watch unwatch cookie 93883509920352] v28'119 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c8803e070 con 0x7f3c80007030 2023-10-05T19:40:25.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.911+0000 7f3c8f858e40 20 remove_watcher() i=5 2023-10-05T19:40:25.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.911+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:111 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93883509925616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5425300 con 0x7f3c80007030 2023-10-05T19:40:25.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.911+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.0 v2:172.21.15.8:6800/27563 31 ==== osd_op_reply(111 notify.6 [watch unwatch cookie 93883509925616] v28'119 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c8803e070 con 0x7f3c80007030 2023-10-05T19:40:25.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.911+0000 7f3c8f858e40 20 remove_watcher() i=6 2023-10-05T19:40:25.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.915+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:112 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93883509929904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f5425300 con 0x7f3c80007030 2023-10-05T19:40:25.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.915+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.0 v2:172.21.15.8:6800/27563 32 ==== osd_op_reply(112 notify.7 [watch unwatch cookie 93883509929904] v28'119 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c8803e070 con 0x7f3c80007030 2023-10-05T19:40:25.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.915+0000 7f3c8f858e40 20 remove_watcher() i=7 2023-10-05T19:40:25.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.919+0000 7f3c5e7e4700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:25.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.919+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:113 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93883509927184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5562f53f99e0 con 0x5562f53e4920 2023-10-05T19:40:25.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.923+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(113 data_loggenerations_metadata [watch unwatch cookie 93883509927184] v28'117 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f3c8803e070 con 0x5562f53e4920 2023-10-05T19:40:25.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.923+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f3c80007030 msgr2=0x7f3c80031480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.923+0000 7f3c8f858e40 1 --2- 172.21.15.12:0/1863343474 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f3c80007030 0x7f3c80031480 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:25.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.923+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5562f53e4920 msgr2=0x5562f54158d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.923+0000 7f3c8f858e40 1 --2- 172.21.15.12:0/1863343474 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5562f53e4920 0x5562f54158d0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:25.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.923+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5562f5417d70 msgr2=0x5562f541a150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.923+0000 7f3c8f858e40 1 --2- 172.21.15.12:0/1863343474 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5562f5417d70 0x5562f541a150 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:25.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.923+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5562f541c480 msgr2=0x5562f541e890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.923+0000 7f3c8f858e40 1 --2- 172.21.15.12:0/1863343474 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5562f541c480 0x5562f541e890 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:25.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.923+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3c70030bf0 msgr2=0x7f3c700330a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.923+0000 7f3c8f858e40 1 --2- 172.21.15.12:0/1863343474 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3c70030bf0 0x7f3c700330a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3c84004170 tx=0x7f3c84001380 comp rx=0 tx=0).stop 2023-10-05T19:40:25.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.923+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5562f53e4010 msgr2=0x5562f5409d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.923+0000 7f3c8db8e700 1 -- 172.21.15.12:0/1863343474 reap_dead start 2023-10-05T19:40:25.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.923+0000 7f3c8f858e40 1 --2- 172.21.15.12:0/1863343474 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5562f53e4010 0x5562f5409d10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3c8000c010 tx=0x7f3c80003050 comp rx=0 tx=0).stop 2023-10-05T19:40:25.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.923+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 shutdown_connections 2023-10-05T19:40:25.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.923+0000 7f3c8f858e40 1 --2- 172.21.15.12:0/1863343474 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5562f53e4010 0x5562f5409d10 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:25.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.923+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 >> 172.21.15.12:0/1863343474 conn(0x5562f5357860 msgr2=0x5562f53fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:25.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.923+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 shutdown_connections 2023-10-05T19:40:25.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.923+0000 7f3c8f858e40 1 -- 172.21.15.12:0/1863343474 wait complete. 2023-10-05T19:40:25.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.923+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3505572453 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x5562f5449450 msgr2=0x5562f544b830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.923+0000 7f3c8f858e40 1 --2- 172.21.15.12:0/3505572453 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x5562f5449450 0x5562f544b830 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:25.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.923+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3505572453 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5562f5445d20 msgr2=0x5562f5448130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.923+0000 7f3c8f858e40 1 --2- 172.21.15.12:0/3505572453 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5562f5445d20 0x5562f5448130 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:25.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.923+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3505572453 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5562f54509b0 msgr2=0x5562f5452d90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.923+0000 7f3c8f858e40 1 --2- 172.21.15.12:0/3505572453 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5562f54509b0 0x5562f5452d90 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:25.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.923+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3505572453 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5562f544cbb0 msgr2=0x5562f544ef90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.923+0000 7f3c8f858e40 1 --2- 172.21.15.12:0/3505572453 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5562f544cbb0 0x5562f544ef90 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:25.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.923+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3505572453 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3bd4030c50 msgr2=0x7f3bd4033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.923+0000 7f3c8f858e40 1 --2- 172.21.15.12:0/3505572453 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3bd4030c50 0x7f3bd4033100 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x5562f54272e0 tx=0x7f3c8800ae20 comp rx=0 tx=0).stop 2023-10-05T19:40:25.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.923+0000 7f3c8d38d700 1 -- 172.21.15.12:0/3505572453 reap_dead start 2023-10-05T19:40:25.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.923+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3505572453 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5562f5435520 msgr2=0x5562f543b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.923+0000 7f3c8f858e40 1 --2- 172.21.15.12:0/3505572453 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5562f5435520 0x5562f543b100 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3c8400efd0 tx=0x7f3c8403d030 comp rx=0 tx=0).stop 2023-10-05T19:40:25.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.923+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3505572453 shutdown_connections 2023-10-05T19:40:25.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.923+0000 7f3c8f858e40 1 --2- 172.21.15.12:0/3505572453 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5562f5435520 0x5562f543b100 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:25.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.923+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3505572453 >> 172.21.15.12:0/3505572453 conn(0x5562f54308e0 msgr2=0x5562f542e980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:25.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.923+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3505572453 shutdown_connections 2023-10-05T19:40:25.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.923+0000 7f3c8f858e40 1 -- 172.21.15.12:0/3505572453 wait complete. 2023-10-05T19:40:25.938 DEBUG:teuthology.orchestra.run:got remote process result: 17 2023-10-05T19:40:25.938 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'unlink', '--uid', 'fud', '--bucket', 'myfoo'] 2023-10-05T19:40:25.938 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'unlink', '--uid', 'fud', '--bucket', 'myfoo'] 2023-10-05T19:40:25.938 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket unlink --uid fud --bucket myfoo 2023-10-05T19:40:25.980 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:25.980 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:25.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.971+0000 7ff348295e40 1 Processor -- start 2023-10-05T19:40:25.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.971+0000 7ff348295e40 1 -- start start 2023-10-05T19:40:25.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.971+0000 7ff348295e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x5653d32ab920 0x5653d333ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:25.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.971+0000 7ff348295e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5653d3283bd0 con 0x5653d32acc30 2023-10-05T19:40:25.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.971+0000 7ff348295e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x5653d3240ee0 con 0x5653d32ab920 2023-10-05T19:40:25.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.971+0000 7ff3455c9700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x5653d32ab920 0x5653d333ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:25.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.971+0000 7ff3455c9700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x5653d32ab920 0x5653d333ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58022/0 (socket says 172.21.15.12:58022) 2023-10-05T19:40:25.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.971+0000 7ff345dca700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x5653d32acc30 0x5653d32ab210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:35552/0 (socket says 172.21.15.12:35552) 2023-10-05T19:40:25.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.971+0000 7ff3455c9700 1 -- 172.21.15.12:0/1155577579 learned_addr learned my addr 172.21.15.12:0/1155577579 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:25.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.975+0000 7ff3455c9700 1 -- 172.21.15.12:0/1155577579 >> v1:172.21.15.8:6789/0 conn(0x5653d32acc30 legacy=0x5653d32ab210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.975+0000 7ff3455c9700 1 -- 172.21.15.12:0/1155577579 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653d32ad0c0 con 0x5653d32ab920 2023-10-05T19:40:25.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.975+0000 7ff3455c9700 1 --2- 172.21.15.12:0/1155577579 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5653d32ab920 0x5653d333ff70 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff33000a5c0 tx=0x7ff330005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a24c849212040b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:25.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.975+0000 7ff344dc8700 1 -- 172.21.15.12:0/1155577579 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff33000e510 con 0x5653d32ab920 2023-10-05T19:40:25.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.975+0000 7ff344dc8700 1 -- 172.21.15.12:0/1155577579 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff33000e670 con 0x5653d32ab920 2023-10-05T19:40:25.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.975+0000 7ff344dc8700 1 -- 172.21.15.12:0/1155577579 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff33000e920 con 0x5653d32ab920 2023-10-05T19:40:25.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.975+0000 7ff348295e40 1 -- 172.21.15.12:0/1155577579 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5653d32ab920 msgr2=0x5653d333ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.975+0000 7ff348295e40 1 --2- 172.21.15.12:0/1155577579 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5653d32ab920 0x5653d333ff70 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff33000a5c0 tx=0x7ff330005e00 comp rx=0 tx=0).stop 2023-10-05T19:40:25.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.975+0000 7ff348295e40 1 -- 172.21.15.12:0/1155577579 shutdown_connections 2023-10-05T19:40:25.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.975+0000 7ff348295e40 1 --2- 172.21.15.12:0/1155577579 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5653d32ab920 0x5653d333ff70 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:25.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.975+0000 7ff348295e40 1 -- 172.21.15.12:0/1155577579 >> 172.21.15.12:0/1155577579 conn(0x5653d3332860 msgr2=0x5653d3334c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:25.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.975+0000 7ff348295e40 1 -- 172.21.15.12:0/1155577579 shutdown_connections 2023-10-05T19:40:25.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.975+0000 7ff348295e40 1 -- 172.21.15.12:0/1155577579 wait complete. 2023-10-05T19:40:25.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.975+0000 7ff348295e40 1 Processor -- start 2023-10-05T19:40:25.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.975+0000 7ff348295e40 1 -- start start 2023-10-05T19:40:25.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.975+0000 7ff348295e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5653d32ab920 0x5653d3358110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:25.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.975+0000 7ff348295e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d3204e30 con 0x5653d32ab920 2023-10-05T19:40:25.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.975+0000 7ff345dca700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5653d32ab920 0x5653d3358110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:25.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.975+0000 7ff345dca700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5653d32ab920 0x5653d3358110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58032/0 (socket says 172.21.15.12:58032) 2023-10-05T19:40:25.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.975+0000 7ff345dca700 1 -- 172.21.15.12:0/2461115356 learned_addr learned my addr 172.21.15.12:0/2461115356 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:25.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.975+0000 7ff345dca700 1 -- 172.21.15.12:0/2461115356 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff33000a180 con 0x5653d32ab920 2023-10-05T19:40:25.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.979+0000 7ff345dca700 1 --2- 172.21.15.12:0/2461115356 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5653d32ab920 0x5653d3358110 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff340001cf0 tx=0x7ff3400018a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f21863a8790c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:25.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.979+0000 7ff33f7fe700 1 -- 172.21.15.12:0/2461115356 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff340014070 con 0x5653d32ab920 2023-10-05T19:40:25.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.979+0000 7ff33f7fe700 1 -- 172.21.15.12:0/2461115356 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff34000be10 con 0x5653d32ab920 2023-10-05T19:40:25.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.979+0000 7ff33f7fe700 1 -- 172.21.15.12:0/2461115356 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff34000f590 con 0x5653d32ab920 2023-10-05T19:40:25.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.979+0000 7ff348295e40 1 -- 172.21.15.12:0/2461115356 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5653d32ab920 msgr2=0x5653d3358110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:25.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.979+0000 7ff348295e40 1 --2- 172.21.15.12:0/2461115356 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5653d32ab920 0x5653d3358110 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff340001cf0 tx=0x7ff3400018a0 comp rx=0 tx=0).stop 2023-10-05T19:40:25.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.979+0000 7ff348295e40 1 -- 172.21.15.12:0/2461115356 shutdown_connections 2023-10-05T19:40:25.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.979+0000 7ff348295e40 1 --2- 172.21.15.12:0/2461115356 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5653d32ab920 0x5653d3358110 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:25.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.979+0000 7ff348295e40 1 -- 172.21.15.12:0/2461115356 >> 172.21.15.12:0/2461115356 conn(0x5653d3332860 msgr2=0x5653d3353cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:25.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.979+0000 7ff348295e40 1 -- 172.21.15.12:0/2461115356 shutdown_connections 2023-10-05T19:40:25.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.979+0000 7ff348295e40 1 -- 172.21.15.12:0/2461115356 wait complete. 2023-10-05T19:40:25.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.979+0000 7ff348295e40 1 Processor -- start 2023-10-05T19:40:25.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.979+0000 7ff348295e40 1 -- start start 2023-10-05T19:40:25.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.979+0000 7ff348295e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5653d32ab920 0x5653d335eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:25.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.979+0000 7ff348295e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d31d7ed0 con 0x5653d32ab920 2023-10-05T19:40:25.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.983+0000 7ff345dca700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5653d32ab920 0x5653d335eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:25.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.983+0000 7ff345dca700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5653d32ab920 0x5653d335eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58042/0 (socket says 172.21.15.12:58042) 2023-10-05T19:40:25.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.983+0000 7ff345dca700 1 -- 172.21.15.12:0/467862307 learned_addr learned my addr 172.21.15.12:0/467862307 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:25.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.983+0000 7ff345dca700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653d3283bd0 con 0x5653d32ab920 2023-10-05T19:40:25.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.983+0000 7ff345dca700 1 --2- 172.21.15.12:0/467862307 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5653d32ab920 0x5653d335eee0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff3400018e0 tx=0x7ff340016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:25.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.983+0000 7ff33dffb700 1 -- 172.21.15.12:0/467862307 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff34001c670 con 0x5653d32ab920 2023-10-05T19:40:25.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.983+0000 7ff33dffb700 1 -- 172.21.15.12:0/467862307 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff34001c7f0 con 0x5653d32ab920 2023-10-05T19:40:25.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.983+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5653d335f420 con 0x5653d32ab920 2023-10-05T19:40:25.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.983+0000 7ff33dffb700 1 -- 172.21.15.12:0/467862307 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff34001cae0 con 0x5653d32ab920 2023-10-05T19:40:25.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.983+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5653d335f860 con 0x5653d32ab920 2023-10-05T19:40:25.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.983+0000 7ff33dffb700 1 -- 172.21.15.12:0/467862307 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7ff340023020 con 0x5653d32ab920 2023-10-05T19:40:25.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.983+0000 7ff348295e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.987+0000 7ff33dffb700 1 --2- 172.21.15.12:0/467862307 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7ff32c030c50 0x7ff32c033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:25.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.987+0000 7ff33dffb700 1 -- 172.21.15.12:0/467862307 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7ff340008030 con 0x5653d32ab920 2023-10-05T19:40:25.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.987+0000 7ff3455c9700 1 --2- 172.21.15.12:0/467862307 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7ff32c030c50 0x7ff32c033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:25.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.987+0000 7ff348295e40 1 --2- 172.21.15.12:0/467862307 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5653d32acc30 0x5653d336a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:25.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.987+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d336b080 con 0x5653d32acc30 2023-10-05T19:40:25.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.987+0000 7ff3465cb700 1 --2- 172.21.15.12:0/467862307 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5653d32acc30 0x5653d336a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:25.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.987+0000 7ff3455c9700 1 --2- 172.21.15.12:0/467862307 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7ff32c030c50 0x7ff32c033100 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff330000f80 tx=0x7ff330013000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:25.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.987+0000 7ff3465cb700 1 --2- 172.21.15.12:0/467862307 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5653d32acc30 0x5653d336a9d0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:25.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.987+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff33800a500 con 0x5653d32acc30 2023-10-05T19:40:25.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.987+0000 7ff348295e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:25.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.987+0000 7ff348295e40 20 realm 2023-10-05T19:40:25.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.987+0000 7ff348295e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.987+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3368030 con 0x5653d32acc30 2023-10-05T19:40:25.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.987+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff33800a500 con 0x5653d32acc30 2023-10-05T19:40:25.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.987+0000 7ff348295e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:25.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.987+0000 7ff348295e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:25.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.987+0000 7ff348295e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.987+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d336bb60 con 0x5653d32acc30 2023-10-05T19:40:25.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.991+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff33800a500 con 0x5653d32acc30 2023-10-05T19:40:25.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.991+0000 7ff348295e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:25.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.991+0000 7ff348295e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:25.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.991+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d336b770 con 0x5653d32acc30 2023-10-05T19:40:26.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.991+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7ff33800a500 con 0x5653d32acc30 2023-10-05T19:40:26.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.991+0000 7ff348295e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:26.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.991+0000 7ff348295e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.991+0000 7ff348295e40 1 --2- 172.21.15.12:0/467862307 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5653d336cf00 0x5653d336f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:26.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.991+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d336f820 con 0x5653d336cf00 2023-10-05T19:40:26.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.991+0000 7ff345dca700 1 --2- 172.21.15.12:0/467862307 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5653d336cf00 0x5653d336f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:26.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.991+0000 7ff345dca700 1 --2- 172.21.15.12:0/467862307 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5653d336cf00 0x5653d336f2e0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:26.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.991+0000 7ff345dca700 1 -- 172.21.15.12:0/467862307 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7ff340016070 con 0x5653d336cf00 2023-10-05T19:40:26.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.991+0000 7ff348295e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:26.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.991+0000 7ff348295e40 20 searching for the correct realm 2023-10-05T19:40:26.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.991+0000 7ff348295e40 1 --2- 172.21.15.12:0/467862307 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5653d33716a0 0x5653d3373ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:26.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.991+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374160 con 0x5653d33716a0 2023-10-05T19:40:26.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.991+0000 7ff3455c9700 1 --2- 172.21.15.12:0/467862307 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5653d33716a0 0x5653d3373ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:26.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.991+0000 7ff3455c9700 1 --2- 172.21.15.12:0/467862307 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5653d33716a0 0x5653d3373ab0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:26.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.995+0000 7ff3455c9700 1 -- 172.21.15.12:0/467862307 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7ff330017050 con 0x5653d33716a0 2023-10-05T19:40:26.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.995+0000 7ff3455c9700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374160 con 0x5653d336cf00 2023-10-05T19:40:26.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.995+0000 7ff345dca700 1 -- 172.21.15.12:0/467862307 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff340016070 con 0x5653d336cf00 2023-10-05T19:40:26.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.995+0000 7ff345dca700 1 --2- 172.21.15.12:0/467862307 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7ff34001a070 0x7ff3400253d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:26.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.995+0000 7ff345dca700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374160 con 0x7ff34001a070 2023-10-05T19:40:26.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.995+0000 7ff3465cb700 1 --2- 172.21.15.12:0/467862307 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7ff34001a070 0x7ff3400253d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:26.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.995+0000 7ff3465cb700 1 --2- 172.21.15.12:0/467862307 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7ff34001a070 0x7ff3400253d0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:26.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.995+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff338014040 con 0x7ff34001a070 2023-10-05T19:40:26.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.995+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374160 con 0x5653d32acc30 2023-10-05T19:40:26.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.999+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff338014040 con 0x5653d32acc30 2023-10-05T19:40:26.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.999+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374160 con 0x5653d32acc30 2023-10-05T19:40:26.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.999+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff338014040 con 0x5653d32acc30 2023-10-05T19:40:26.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.999+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374160 con 0x7ff34001a070 2023-10-05T19:40:26.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.999+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff338014040 con 0x7ff34001a070 2023-10-05T19:40:26.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.999+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374160 con 0x7ff34001a070 2023-10-05T19:40:26.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.999+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff338014040 con 0x7ff34001a070 2023-10-05T19:40:26.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.999+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374160 con 0x5653d336cf00 2023-10-05T19:40:26.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.999+0000 7ff345dca700 1 -- 172.21.15.12:0/467862307 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff340016070 con 0x5653d336cf00 2023-10-05T19:40:26.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:25.999+0000 7ff345dca700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374160 con 0x5653d33716a0 2023-10-05T19:40:26.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.003+0000 7ff3455c9700 1 -- 172.21.15.12:0/467862307 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff330017050 con 0x5653d33716a0 2023-10-05T19:40:26.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.003+0000 7ff3455c9700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374160 con 0x5653d32acc30 2023-10-05T19:40:26.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.003+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff338014040 con 0x5653d32acc30 2023-10-05T19:40:26.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.003+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374160 con 0x7ff34001a070 2023-10-05T19:40:26.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.003+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff338014040 con 0x7ff34001a070 2023-10-05T19:40:26.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.003+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374160 con 0x5653d32acc30 2023-10-05T19:40:26.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.003+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff338014040 con 0x5653d32acc30 2023-10-05T19:40:26.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.003+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374160 con 0x5653d336cf00 2023-10-05T19:40:26.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.003+0000 7ff345dca700 1 -- 172.21.15.12:0/467862307 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff340016070 con 0x5653d336cf00 2023-10-05T19:40:26.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.003+0000 7ff345dca700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374160 con 0x5653d33716a0 2023-10-05T19:40:26.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.007+0000 7ff3455c9700 1 -- 172.21.15.12:0/467862307 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff330017050 con 0x5653d33716a0 2023-10-05T19:40:26.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.007+0000 7ff3455c9700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374160 con 0x5653d33716a0 2023-10-05T19:40:26.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.007+0000 7ff3455c9700 1 -- 172.21.15.12:0/467862307 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff330017050 con 0x5653d33716a0 2023-10-05T19:40:26.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.007+0000 7ff3455c9700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374160 con 0x7ff34001a070 2023-10-05T19:40:26.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.007+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff338014040 con 0x7ff34001a070 2023-10-05T19:40:26.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.007+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374160 con 0x5653d33716a0 2023-10-05T19:40:26.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.007+0000 7ff3455c9700 1 -- 172.21.15.12:0/467862307 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff330017050 con 0x5653d33716a0 2023-10-05T19:40:26.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.007+0000 7ff3455c9700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374160 con 0x5653d33716a0 2023-10-05T19:40:26.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.007+0000 7ff3455c9700 1 -- 172.21.15.12:0/467862307 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff330017050 con 0x5653d33716a0 2023-10-05T19:40:26.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.007+0000 7ff3455c9700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374160 con 0x5653d32acc30 2023-10-05T19:40:26.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.011+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff338014040 con 0x5653d32acc30 2023-10-05T19:40:26.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.011+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374160 con 0x5653d32acc30 2023-10-05T19:40:26.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.011+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff338014040 con 0x5653d32acc30 2023-10-05T19:40:26.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.011+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374160 con 0x7ff34001a070 2023-10-05T19:40:26.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.011+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff338014040 con 0x7ff34001a070 2023-10-05T19:40:26.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.011+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374160 con 0x5653d33716a0 2023-10-05T19:40:26.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.011+0000 7ff3455c9700 1 -- 172.21.15.12:0/467862307 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff330017050 con 0x5653d33716a0 2023-10-05T19:40:26.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.011+0000 7ff3455c9700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374160 con 0x5653d336cf00 2023-10-05T19:40:26.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.011+0000 7ff345dca700 1 -- 172.21.15.12:0/467862307 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff340016070 con 0x5653d336cf00 2023-10-05T19:40:26.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.011+0000 7ff345dca700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374160 con 0x5653d32acc30 2023-10-05T19:40:26.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.015+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff338014040 con 0x5653d32acc30 2023-10-05T19:40:26.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.015+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374160 con 0x7ff34001a070 2023-10-05T19:40:26.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.015+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff338014040 con 0x7ff34001a070 2023-10-05T19:40:26.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.015+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374160 con 0x5653d33716a0 2023-10-05T19:40:26.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.015+0000 7ff3455c9700 1 -- 172.21.15.12:0/467862307 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff330017050 con 0x5653d33716a0 2023-10-05T19:40:26.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.015+0000 7ff3455c9700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374160 con 0x7ff34001a070 2023-10-05T19:40:26.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.015+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff338014040 con 0x7ff34001a070 2023-10-05T19:40:26.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.015+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374160 con 0x5653d336cf00 2023-10-05T19:40:26.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.015+0000 7ff345dca700 1 -- 172.21.15.12:0/467862307 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff340016070 con 0x5653d336cf00 2023-10-05T19:40:26.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.015+0000 7ff345dca700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374160 con 0x5653d33716a0 2023-10-05T19:40:26.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.019+0000 7ff3455c9700 1 -- 172.21.15.12:0/467862307 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff330017050 con 0x5653d33716a0 2023-10-05T19:40:26.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.019+0000 7ff3455c9700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374160 con 0x7ff34001a070 2023-10-05T19:40:26.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.019+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff338014040 con 0x7ff34001a070 2023-10-05T19:40:26.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.019+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374160 con 0x5653d32acc30 2023-10-05T19:40:26.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.019+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7ff338014040 con 0x5653d32acc30 2023-10-05T19:40:26.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.019+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374160 con 0x5653d336cf00 2023-10-05T19:40:26.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.019+0000 7ff345dca700 1 -- 172.21.15.12:0/467862307 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7ff340016070 con 0x5653d336cf00 2023-10-05T19:40:26.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.019+0000 7ff348295e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:26.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.019+0000 7ff348295e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:26.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.019+0000 7ff348295e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:26.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.019+0000 7ff348295e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:26.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.019+0000 7ff348295e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.019+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3370420 con 0x5653d32acc30 2023-10-05T19:40:26.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.023+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff338014040 con 0x5653d32acc30 2023-10-05T19:40:26.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.023+0000 7ff348295e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:26.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.023+0000 7ff348295e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.023+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3370050 con 0x5653d32acc30 2023-10-05T19:40:26.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.023+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7ff338014040 con 0x5653d32acc30 2023-10-05T19:40:26.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.023+0000 7ff348295e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:26.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.023+0000 7ff348295e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.023+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d33745d0 con 0x5653d33716a0 2023-10-05T19:40:26.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.023+0000 7ff3455c9700 1 -- 172.21.15.12:0/467862307 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7ff330017050 con 0x5653d33716a0 2023-10-05T19:40:26.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.023+0000 7ff348295e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:26.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.023+0000 7ff348295e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.023+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374980 con 0x5653d33716a0 2023-10-05T19:40:26.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.023+0000 7ff3455c9700 1 -- 172.21.15.12:0/467862307 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7ff330017050 con 0x5653d33716a0 2023-10-05T19:40:26.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.023+0000 7ff348295e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:26.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.023+0000 7ff348295e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.023+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374d30 con 0x5653d32acc30 2023-10-05T19:40:26.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.023+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff338014040 con 0x5653d32acc30 2023-10-05T19:40:26.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.023+0000 7ff348295e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:26.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.023+0000 7ff348295e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.023+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d33708b0 con 0x5653d32acc30 2023-10-05T19:40:26.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.027+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7ff338014040 con 0x5653d32acc30 2023-10-05T19:40:26.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.027+0000 7ff348295e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:26.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.027+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3370880 con 0x5653d33716a0 2023-10-05T19:40:26.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.027+0000 7ff3455c9700 1 -- 172.21.15.12:0/467862307 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7ff330017050 con 0x5653d33716a0 2023-10-05T19:40:26.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.027+0000 7ff3455c9700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3370880 con 0x5653d336cf00 2023-10-05T19:40:26.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.027+0000 7ff345dca700 1 -- 172.21.15.12:0/467862307 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff340016070 con 0x5653d336cf00 2023-10-05T19:40:26.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.027+0000 7ff345dca700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3370880 con 0x7ff34001a070 2023-10-05T19:40:26.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.031+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff338014040 con 0x7ff34001a070 2023-10-05T19:40:26.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.031+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374d30 con 0x5653d32acc30 2023-10-05T19:40:26.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.031+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff338014040 con 0x5653d32acc30 2023-10-05T19:40:26.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.031+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374d30 con 0x5653d32acc30 2023-10-05T19:40:26.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.031+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff338014040 con 0x5653d32acc30 2023-10-05T19:40:26.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.031+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374d30 con 0x7ff34001a070 2023-10-05T19:40:26.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.031+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff338014040 con 0x7ff34001a070 2023-10-05T19:40:26.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.031+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374d30 con 0x7ff34001a070 2023-10-05T19:40:26.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.031+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff338014040 con 0x7ff34001a070 2023-10-05T19:40:26.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.031+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374d30 con 0x5653d336cf00 2023-10-05T19:40:26.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.035+0000 7ff345dca700 1 -- 172.21.15.12:0/467862307 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff340016070 con 0x5653d336cf00 2023-10-05T19:40:26.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.035+0000 7ff345dca700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374d30 con 0x5653d33716a0 2023-10-05T19:40:26.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.035+0000 7ff3455c9700 1 -- 172.21.15.12:0/467862307 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff330017050 con 0x5653d33716a0 2023-10-05T19:40:26.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.035+0000 7ff3455c9700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374d30 con 0x5653d32acc30 2023-10-05T19:40:26.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.035+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff338014040 con 0x5653d32acc30 2023-10-05T19:40:26.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.035+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374d30 con 0x7ff34001a070 2023-10-05T19:40:26.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.035+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff338014040 con 0x7ff34001a070 2023-10-05T19:40:26.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.035+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374d30 con 0x5653d32acc30 2023-10-05T19:40:26.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.039+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff338014040 con 0x5653d32acc30 2023-10-05T19:40:26.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.039+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374d30 con 0x5653d336cf00 2023-10-05T19:40:26.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.039+0000 7ff345dca700 1 -- 172.21.15.12:0/467862307 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff340016070 con 0x5653d336cf00 2023-10-05T19:40:26.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.039+0000 7ff345dca700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374d30 con 0x5653d33716a0 2023-10-05T19:40:26.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.039+0000 7ff3455c9700 1 -- 172.21.15.12:0/467862307 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff330017050 con 0x5653d33716a0 2023-10-05T19:40:26.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.039+0000 7ff3455c9700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374d30 con 0x5653d33716a0 2023-10-05T19:40:26.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.039+0000 7ff3455c9700 1 -- 172.21.15.12:0/467862307 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff330017050 con 0x5653d33716a0 2023-10-05T19:40:26.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.039+0000 7ff3455c9700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374d30 con 0x7ff34001a070 2023-10-05T19:40:26.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.039+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff338014040 con 0x7ff34001a070 2023-10-05T19:40:26.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.039+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374d30 con 0x5653d33716a0 2023-10-05T19:40:26.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.043+0000 7ff3455c9700 1 -- 172.21.15.12:0/467862307 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff330017050 con 0x5653d33716a0 2023-10-05T19:40:26.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.043+0000 7ff3455c9700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374d30 con 0x5653d33716a0 2023-10-05T19:40:26.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.043+0000 7ff3455c9700 1 -- 172.21.15.12:0/467862307 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff330017050 con 0x5653d33716a0 2023-10-05T19:40:26.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.043+0000 7ff3455c9700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374d30 con 0x5653d32acc30 2023-10-05T19:40:26.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.043+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff338014040 con 0x5653d32acc30 2023-10-05T19:40:26.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.043+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374d30 con 0x5653d32acc30 2023-10-05T19:40:26.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.043+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff338014040 con 0x5653d32acc30 2023-10-05T19:40:26.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.043+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374d30 con 0x7ff34001a070 2023-10-05T19:40:26.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.047+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff338014040 con 0x7ff34001a070 2023-10-05T19:40:26.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.047+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374d30 con 0x5653d33716a0 2023-10-05T19:40:26.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.047+0000 7ff3455c9700 1 -- 172.21.15.12:0/467862307 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff330017050 con 0x5653d33716a0 2023-10-05T19:40:26.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.047+0000 7ff3455c9700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374d30 con 0x5653d336cf00 2023-10-05T19:40:26.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.047+0000 7ff345dca700 1 -- 172.21.15.12:0/467862307 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff340016070 con 0x5653d336cf00 2023-10-05T19:40:26.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.047+0000 7ff345dca700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374d30 con 0x5653d32acc30 2023-10-05T19:40:26.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.047+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff338014040 con 0x5653d32acc30 2023-10-05T19:40:26.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.047+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374d30 con 0x7ff34001a070 2023-10-05T19:40:26.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.051+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff338014040 con 0x7ff34001a070 2023-10-05T19:40:26.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.051+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374d30 con 0x5653d33716a0 2023-10-05T19:40:26.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.051+0000 7ff3455c9700 1 -- 172.21.15.12:0/467862307 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff330017050 con 0x5653d33716a0 2023-10-05T19:40:26.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.051+0000 7ff3455c9700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374d30 con 0x7ff34001a070 2023-10-05T19:40:26.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.051+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff338014040 con 0x7ff34001a070 2023-10-05T19:40:26.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.051+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374d30 con 0x5653d336cf00 2023-10-05T19:40:26.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.051+0000 7ff345dca700 1 -- 172.21.15.12:0/467862307 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff340016070 con 0x5653d336cf00 2023-10-05T19:40:26.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.051+0000 7ff345dca700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374d30 con 0x5653d33716a0 2023-10-05T19:40:26.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.051+0000 7ff3455c9700 1 -- 172.21.15.12:0/467862307 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff330017050 con 0x5653d33716a0 2023-10-05T19:40:26.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.051+0000 7ff3455c9700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374d30 con 0x7ff34001a070 2023-10-05T19:40:26.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.055+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff338014040 con 0x7ff34001a070 2023-10-05T19:40:26.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.055+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374d30 con 0x5653d32acc30 2023-10-05T19:40:26.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.055+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7ff338014040 con 0x5653d32acc30 2023-10-05T19:40:26.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.055+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3374d30 con 0x5653d336cf00 2023-10-05T19:40:26.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.055+0000 7ff345dca700 1 -- 172.21.15.12:0/467862307 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7ff340016070 con 0x5653d336cf00 2023-10-05T19:40:26.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.055+0000 7ff348295e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:26.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.055+0000 7ff348295e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:26.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.055+0000 7ff348295e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:26.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.055+0000 7ff348295e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:26.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.055+0000 7ff348295e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.055+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3370880 con 0x5653d32acc30 2023-10-05T19:40:26.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.055+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7ff338014040 con 0x5653d32acc30 2023-10-05T19:40:26.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.055+0000 7ff348295e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:26.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.055+0000 7ff348295e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.055+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3375620 con 0x5653d336cf00 2023-10-05T19:40:26.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.059+0000 7ff345dca700 1 -- 172.21.15.12:0/467862307 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7ff340016070 con 0x5653d336cf00 2023-10-05T19:40:26.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.059+0000 7ff348295e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:26.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.059+0000 7ff348295e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.059+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3375f10 con 0x5653d32acc30 2023-10-05T19:40:26.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.059+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7ff338014040 con 0x5653d32acc30 2023-10-05T19:40:26.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.059+0000 7ff348295e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:26.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.059+0000 7ff348295e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.059+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3375f10 con 0x5653d33716a0 2023-10-05T19:40:26.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.059+0000 7ff3455c9700 1 -- 172.21.15.12:0/467862307 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7ff330017050 con 0x5653d33716a0 2023-10-05T19:40:26.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.059+0000 7ff348295e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:26.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.059+0000 7ff348295e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.059+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3376680 con 0x7ff34001a070 2023-10-05T19:40:26.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.059+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7ff338014040 con 0x7ff34001a070 2023-10-05T19:40:26.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.059+0000 7ff348295e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:26.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.059+0000 7ff348295e40 20 zone default found 2023-10-05T19:40:26.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.059+0000 7ff348295e40 4 Realm: () 2023-10-05T19:40:26.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.059+0000 7ff348295e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:26.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.059+0000 7ff348295e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:26.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.059+0000 7ff348295e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:26.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.059+0000 7ff348295e40 20 zonegroup default 2023-10-05T19:40:26.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.059+0000 7ff348295e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.059+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3376630 con 0x7ff34001a070 2023-10-05T19:40:26.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.063+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7ff338014040 con 0x7ff34001a070 2023-10-05T19:40:26.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.063+0000 7ff348295e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:26.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.063+0000 7ff348295e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.063+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3377fb0 con 0x5653d33716a0 2023-10-05T19:40:26.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.063+0000 7ff3455c9700 1 -- 172.21.15.12:0/467862307 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7ff330017050 con 0x5653d33716a0 2023-10-05T19:40:26.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.063+0000 7ff348295e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:26.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.063+0000 7ff348295e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.063+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3378360 con 0x5653d33716a0 2023-10-05T19:40:26.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.063+0000 7ff3455c9700 1 -- 172.21.15.12:0/467862307 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7ff330017050 con 0x5653d33716a0 2023-10-05T19:40:26.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.063+0000 7ff348295e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:26.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.063+0000 7ff348295e40 20 started sync module instance, tier type = 2023-10-05T19:40:26.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.063+0000 7ff348295e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:26.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.063+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3377670 con 0x5653d336cf00 2023-10-05T19:40:26.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.067+0000 7ff345dca700 1 -- 172.21.15.12:0/467862307 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'120 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff340016070 con 0x5653d336cf00 2023-10-05T19:40:26.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.067+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94918025894432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3378710 con 0x5653d336cf00 2023-10-05T19:40:26.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.067+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3378fb0 con 0x5653d32acc30 2023-10-05T19:40:26.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.067+0000 7ff345dca700 1 -- 172.21.15.12:0/467862307 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94918025894432] v28'121 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff340016070 con 0x5653d336cf00 2023-10-05T19:40:26.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.067+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'120 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff338014040 con 0x5653d32acc30 2023-10-05T19:40:26.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.071+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94918025906880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3377b90 con 0x5653d32acc30 2023-10-05T19:40:26.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.071+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3379af0 con 0x5653d33716a0 2023-10-05T19:40:26.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.071+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94918025906880] v28'121 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff338014040 con 0x5653d32acc30 2023-10-05T19:40:26.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.071+0000 7ff3455c9700 1 -- 172.21.15.12:0/467862307 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'120 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff330017050 con 0x5653d33716a0 2023-10-05T19:40:26.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.071+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94918025909760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3379ef0 con 0x5653d33716a0 2023-10-05T19:40:26.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.071+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d337a7d0 con 0x5653d336cf00 2023-10-05T19:40:26.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.075+0000 7ff345dca700 1 -- 172.21.15.12:0/467862307 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'120 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff340016070 con 0x5653d336cf00 2023-10-05T19:40:26.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.075+0000 7ff3455c9700 1 -- 172.21.15.12:0/467862307 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94918025909760] v28'121 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff330017050 con 0x5653d33716a0 2023-10-05T19:40:26.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.075+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94918025915344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d337b510 con 0x5653d336cf00 2023-10-05T19:40:26.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.075+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d337b8c0 con 0x5653d32acc30 2023-10-05T19:40:26.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.079+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'120 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff338014040 con 0x5653d32acc30 2023-10-05T19:40:26.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.079+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94918025917712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d337be50 con 0x5653d32acc30 2023-10-05T19:40:26.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.079+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d337c860 con 0x7ff34001a070 2023-10-05T19:40:26.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.079+0000 7ff345dca700 1 -- 172.21.15.12:0/467862307 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94918025915344] v28'121 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff340016070 con 0x5653d336cf00 2023-10-05T19:40:26.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.079+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'120 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff338014040 con 0x7ff34001a070 2023-10-05T19:40:26.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.079+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94918025916448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d337c860 con 0x7ff34001a070 2023-10-05T19:40:26.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.079+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d337d270 con 0x7ff34001a070 2023-10-05T19:40:26.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.079+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94918025917712] v28'121 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff338014040 con 0x5653d32acc30 2023-10-05T19:40:26.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.083+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(95 notify.5 [watch watch cookie 94918025916448] v28'121 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff338014040 con 0x7ff34001a070 2023-10-05T19:40:26.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.083+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(96 notify.6 [create] v28'120 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff338014040 con 0x7ff34001a070 2023-10-05T19:40:26.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.083+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94918025921392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d337ba70 con 0x7ff34001a070 2023-10-05T19:40:26.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.083+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d337d760 con 0x7ff34001a070 2023-10-05T19:40:26.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.087+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(97 notify.6 [watch watch cookie 94918025921392] v28'121 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff338014040 con 0x7ff34001a070 2023-10-05T19:40:26.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.087+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(98 notify.7 [create] v28'120 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff338014040 con 0x7ff34001a070 2023-10-05T19:40:26.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.087+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94918025923968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d337d6c0 con 0x7ff34001a070 2023-10-05T19:40:26.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.087+0000 7ff348295e40 20 add_watcher() i=0 2023-10-05T19:40:26.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.087+0000 7ff348295e40 20 add_watcher() i=1 2023-10-05T19:40:26.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.087+0000 7ff348295e40 20 add_watcher() i=2 2023-10-05T19:40:26.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.087+0000 7ff348295e40 20 add_watcher() i=3 2023-10-05T19:40:26.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.087+0000 7ff348295e40 20 add_watcher() i=4 2023-10-05T19:40:26.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.087+0000 7ff348295e40 20 add_watcher() i=5 2023-10-05T19:40:26.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.087+0000 7ff348295e40 20 add_watcher() i=6 2023-10-05T19:40:26.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.087+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94918025923968] v28'121 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff338014040 con 0x7ff34001a070 2023-10-05T19:40:26.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.087+0000 7ff348295e40 20 add_watcher() i=7 2023-10-05T19:40:26.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.087+0000 7ff348295e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:26.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.087+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3380720 con 0x5653d32acc30 2023-10-05T19:40:26.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.087+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7ff338014040 con 0x5653d32acc30 2023-10-05T19:40:26.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.087+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94918025925072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d337db10 con 0x5653d32acc30 2023-10-05T19:40:26.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.091+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94918025925072] v28'118 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7ff338014040 con 0x5653d32acc30 2023-10-05T19:40:26.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.091+0000 7ff3167ec700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:26.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.091+0000 7ff348295e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:26.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.091+0000 7ff348295e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:26.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.091+0000 7ff348295e40 1 Processor -- start 2023-10-05T19:40:26.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.091+0000 7ff348295e40 1 -- start start 2023-10-05T19:40:26.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.091+0000 7ff348295e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5653d338af90 0x5653d338b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:26.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.091+0000 7ff348295e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d338b8a0 con 0x5653d338af90 2023-10-05T19:40:26.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.091+0000 7ff3455c9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5653d338af90 0x5653d338b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:26.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.091+0000 7ff3455c9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5653d338af90 0x5653d338b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58044/0 (socket says 172.21.15.12:58044) 2023-10-05T19:40:26.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.091+0000 7ff3455c9700 1 -- 172.21.15.12:0/3008146486 learned_addr learned my addr 172.21.15.12:0/3008146486 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:26.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.095+0000 7ff3455c9700 1 -- 172.21.15.12:0/3008146486 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653d338b9e0 con 0x5653d338af90 2023-10-05T19:40:26.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.095+0000 7ff3455c9700 1 --2- 172.21.15.12:0/3008146486 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5653d338af90 0x5653d338b360 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff330003000 tx=0x7ff330010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63c6141c338dd2ae server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:26.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.095+0000 7ff315feb700 1 -- 172.21.15.12:0/3008146486 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff330010c10 con 0x5653d338af90 2023-10-05T19:40:26.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.095+0000 7ff315feb700 1 -- 172.21.15.12:0/3008146486 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff330010d90 con 0x5653d338af90 2023-10-05T19:40:26.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.095+0000 7ff315feb700 1 -- 172.21.15.12:0/3008146486 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff33001c430 con 0x5653d338af90 2023-10-05T19:40:26.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.095+0000 7ff348295e40 1 -- 172.21.15.12:0/3008146486 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5653d338af90 msgr2=0x5653d338b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:26.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.095+0000 7ff348295e40 1 --2- 172.21.15.12:0/3008146486 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5653d338af90 0x5653d338b360 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff330003000 tx=0x7ff330010000 comp rx=0 tx=0).stop 2023-10-05T19:40:26.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.095+0000 7ff348295e40 1 -- 172.21.15.12:0/3008146486 shutdown_connections 2023-10-05T19:40:26.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.095+0000 7ff348295e40 1 --2- 172.21.15.12:0/3008146486 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5653d338af90 0x5653d338b360 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:26.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.095+0000 7ff348295e40 1 -- 172.21.15.12:0/3008146486 >> 172.21.15.12:0/3008146486 conn(0x5653d3386350 msgr2=0x5653d33844a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:26.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.095+0000 7ff348295e40 1 -- 172.21.15.12:0/3008146486 shutdown_connections 2023-10-05T19:40:26.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.095+0000 7ff348295e40 1 -- 172.21.15.12:0/3008146486 wait complete. 2023-10-05T19:40:26.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.095+0000 7ff348295e40 1 Processor -- start 2023-10-05T19:40:26.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.095+0000 7ff348295e40 1 -- start start 2023-10-05T19:40:26.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.095+0000 7ff348295e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5653d338af90 0x5653d3390a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:26.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.095+0000 7ff348295e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d3391a40 con 0x5653d338af90 2023-10-05T19:40:26.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.095+0000 7ff3455c9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5653d338af90 0x5653d3390a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:26.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.099+0000 7ff3455c9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5653d338af90 0x5653d3390a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58056/0 (socket says 172.21.15.12:58056) 2023-10-05T19:40:26.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.099+0000 7ff3455c9700 1 -- 172.21.15.12:0/446319665 learned_addr learned my addr 172.21.15.12:0/446319665 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:26.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.099+0000 7ff3455c9700 1 -- 172.21.15.12:0/446319665 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653d32ad0c0 con 0x5653d338af90 2023-10-05T19:40:26.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.099+0000 7ff3455c9700 1 --2- 172.21.15.12:0/446319665 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5653d338af90 0x5653d3390a40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff330039040 tx=0x7ff330036000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:26.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.099+0000 7ff287fff700 1 -- 172.21.15.12:0/446319665 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff33003fd00 con 0x5653d338af90 2023-10-05T19:40:26.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.099+0000 7ff287fff700 1 -- 172.21.15.12:0/446319665 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff33003fe80 con 0x5653d338af90 2023-10-05T19:40:26.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.099+0000 7ff348295e40 1 -- 172.21.15.12:0/446319665 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5653d3391040 con 0x5653d338af90 2023-10-05T19:40:26.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.099+0000 7ff287fff700 1 -- 172.21.15.12:0/446319665 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff33002f420 con 0x5653d338af90 2023-10-05T19:40:26.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.099+0000 7ff348295e40 1 -- 172.21.15.12:0/446319665 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5653d3391480 con 0x5653d338af90 2023-10-05T19:40:26.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.099+0000 7ff287fff700 1 -- 172.21.15.12:0/446319665 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7ff330047070 con 0x5653d338af90 2023-10-05T19:40:26.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.099+0000 7ff287fff700 1 --2- 172.21.15.12:0/446319665 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7ff28c030c00 0x7ff28c0330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:26.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.099+0000 7ff287fff700 1 -- 172.21.15.12:0/446319665 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7ff330053b00 con 0x5653d338af90 2023-10-05T19:40:26.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.099+0000 7ff348295e40 5 note: GC not initialized 2023-10-05T19:40:26.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.099+0000 7ff3465cb700 1 --2- 172.21.15.12:0/446319665 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7ff28c030c00 0x7ff28c0330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:26.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.103+0000 7ff3465cb700 1 --2- 172.21.15.12:0/446319665 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7ff28c030c00 0x7ff28c0330b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x5653d33807d0 tx=0x7ff33800ab80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:26.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.103+0000 7ff2857fa700 20 reqs_thread_entry: start 2023-10-05T19:40:26.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.103+0000 7ff27effd700 20 reqs_thread_entry: start 2023-10-05T19:40:26.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.103+0000 7ff348295e40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:26.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.103+0000 7ff348295e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:26.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.103+0000 7ff27d7fa700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:56 2023 2023-10-05T19:40:26.112 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:26.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.103+0000 7ff348295e40 20 get_system_obj_state: rctx=0x7ffeaf4ada98 obj=default.rgw.meta:users.uid:fud state=0x5653d32100b0 s->prefetch_data=0 2023-10-05T19:40:26.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.103+0000 7ff348295e40 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2023-10-05T19:40:26.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.103+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:102 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d33f92b0 con 0x7ff34001a070 2023-10-05T19:40:26.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.103+0000 7ff27cff9700 20 reqs_thread_entry: start 2023-10-05T19:40:26.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.103+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(102 fud [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7ff338014040 con 0x7ff34001a070 2023-10-05T19:40:26.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.103+0000 7ff348295e40 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x16 2023-10-05T19:40:26.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.103+0000 7ff348295e40 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2023-10-05T19:40:26.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.103+0000 7ff348295e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:26.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.103+0000 7ff348295e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:26.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.103+0000 7ff348295e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:26.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.103+0000 7ff348295e40 10 cache get: name=default.rgw.meta+users.uid+fud : type miss (requested=0x13, cached=0x16) 2023-10-05T19:40:26.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.103+0000 7ff348295e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.103+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:103 8.e 8:75f48285:users.uid::fud:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d33fbd40 con 0x7ff34001a070 2023-10-05T19:40:26.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.107+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(103 fud [call,call out=48b,read 0~346 out=346b,getxattrs out=75b] v0'0 uv2 ondisk = 0) v8 ==== 273+0+469 (crc 0 0 0) 0x7ff338014040 con 0x7ff34001a070 2023-10-05T19:40:26.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.107+0000 7ff348295e40 20 rados_obj.operate() r=0 bl.length=346 2023-10-05T19:40:26.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.107+0000 7ff348295e40 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x13 2023-10-05T19:40:26.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.107+0000 7ff348295e40 10 moving default.rgw.meta+users.uid+fud to cache LRU end 2023-10-05T19:40:26.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.107+0000 7ff348295e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:26.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.107+0000 7ff348295e40 20 get_system_obj_state: rctx=0x7ffeaf4adae0 obj=default.rgw.meta:root:myfoo state=0x5653d32100b0 s->prefetch_data=0 2023-10-05T19:40:26.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.107+0000 7ff348295e40 10 cache get: name=default.rgw.meta+root+myfoo : miss 2023-10-05T19:40:26.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.107+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:104 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d33fb550 con 0x5653d336cf00 2023-10-05T19:40:26.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.107+0000 7ff345dca700 1 -- 172.21.15.12:0/467862307 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(104 myfoo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv20 ondisk = 0) v8 ==== 233+0+139 (crc 0 0 0) 0x7ff340016070 con 0x5653d336cf00 2023-10-05T19:40:26.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.107+0000 7ff348295e40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x16 2023-10-05T19:40:26.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.107+0000 7ff348295e40 10 adding default.rgw.meta+root+myfoo to cache LRU end 2023-10-05T19:40:26.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.107+0000 7ff348295e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:26.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.107+0000 7ff348295e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:26.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.107+0000 7ff348295e40 10 cache get: name=default.rgw.meta+root+myfoo : type miss (requested=0x11, cached=0x16) 2023-10-05T19:40:26.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.107+0000 7ff348295e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.107+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:105 8.9 8:91b71841:root::myfoo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d33fdaf0 con 0x5653d336cf00 2023-10-05T19:40:26.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.107+0000 7ff345dca700 1 -- 172.21.15.12:0/467862307 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(105 myfoo [call,call out=48b,read 0~165 out=165b] v0'0 uv20 ondisk = 0) v8 ==== 233+0+213 (crc 0 0 0) 0x7ff340016070 con 0x5653d336cf00 2023-10-05T19:40:26.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.107+0000 7ff348295e40 20 rados_obj.operate() r=0 bl.length=165 2023-10-05T19:40:26.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.107+0000 7ff348295e40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2023-10-05T19:40:26.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.107+0000 7ff348295e40 10 moving default.rgw.meta+root+myfoo to cache LRU end 2023-10-05T19:40:26.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.107+0000 7ff348295e40 20 get_system_obj_state: rctx=0x7ffeaf4adae0 obj=default.rgw.meta:root:.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 state=0x5653d31e62b0 s->prefetch_data=0 2023-10-05T19:40:26.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.107+0000 7ff348295e40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 : miss 2023-10-05T19:40:26.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.107+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:106 8.e 8:70176bbd:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d33fcfd0 con 0x7ff34001a070 2023-10-05T19:40:26.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.107+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(106 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 [call out=48b,getxattrs out=218b,stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 290+0+282 (crc 0 0 0) 0x7ff338014040 con 0x7ff34001a070 2023-10-05T19:40:26.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.107+0000 7ff348295e40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 info.flags=0x16 2023-10-05T19:40:26.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.107+0000 7ff348295e40 10 adding default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 to cache LRU end 2023-10-05T19:40:26.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.107+0000 7ff348295e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:26.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.107+0000 7ff348295e40 10 updating xattr: name=user.rgw.acl bl.length()=123 2023-10-05T19:40:26.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.107+0000 7ff348295e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:26.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.107+0000 7ff348295e40 20 Read xattr: user.rgw.acl 2023-10-05T19:40:26.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.107+0000 7ff348295e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:26.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.107+0000 7ff348295e40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 : type miss (requested=0x13, cached=0x16) 2023-10-05T19:40:26.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.107+0000 7ff348295e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.107+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:107 8.e 8:70176bbd:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d33fcc20 con 0x7ff34001a070 2023-10-05T19:40:26.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.111+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.0 v2:172.21.15.8:6800/27563 30 ==== osd_op_reply(107 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 [call,call out=48b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv3 ondisk = 0) v8 ==== 332+0+618 (crc 0 0 0) 0x7ff338014040 con 0x7ff34001a070 2023-10-05T19:40:26.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.111+0000 7ff348295e40 20 rados_obj.operate() r=0 bl.length=352 2023-10-05T19:40:26.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.111+0000 7ff348295e40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 info.flags=0x13 2023-10-05T19:40:26.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.111+0000 7ff348295e40 10 moving default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 to cache LRU end 2023-10-05T19:40:26.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.111+0000 7ff348295e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:26.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.111+0000 7ff348295e40 10 updating xattr: name=user.rgw.acl bl.length()=123 2023-10-05T19:40:26.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.111+0000 7ff348295e40 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 2023-10-05T19:40:26.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.111+0000 7ff348295e40 20 get_system_obj_state: rctx=0x7ffeaf4ada78 obj=default.rgw.meta:users.uid:fud state=0x5653d31e62b0 s->prefetch_data=0 2023-10-05T19:40:26.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.111+0000 7ff348295e40 10 cache get: name=default.rgw.meta+users.uid+fud : type miss (requested=0x16, cached=0x13) 2023-10-05T19:40:26.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.111+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:108 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d33ffdd0 con 0x7ff34001a070 2023-10-05T19:40:26.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.111+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.0 v2:172.21.15.8:6800/27563 31 ==== osd_op_reply(108 fud [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7ff338014040 con 0x7ff34001a070 2023-10-05T19:40:26.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.111+0000 7ff348295e40 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x16 2023-10-05T19:40:26.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.111+0000 7ff348295e40 10 moving default.rgw.meta+users.uid+fud to cache LRU end 2023-10-05T19:40:26.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.111+0000 7ff348295e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:26.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.111+0000 7ff348295e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:26.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.111+0000 7ff348295e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:26.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.111+0000 7ff348295e40 10 cache get: name=default.rgw.meta+users.uid+fud : hit (requested=0x13, cached=0x17) 2023-10-05T19:40:26.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.111+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:109 8.1 8:810902b1:users.uid::fud.buckets:head [call user.remove_bucket in=58b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3400de0 con 0x7ff34001a070 2023-10-05T19:40:26.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.115+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.0 v2:172.21.15.8:6800/27563 32 ==== osd_op_reply(109 fud.buckets [call] v28'2 uv2 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x7ff338014040 con 0x7ff34001a070 2023-10-05T19:40:26.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.115+0000 7ff348295e40 20 get_system_obj_state: rctx=0x7ffeaf4adbb8 obj=default.rgw.meta:root:myfoo state=0x5653d31e62b0 s->prefetch_data=0 2023-10-05T19:40:26.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.115+0000 7ff348295e40 10 cache get: name=default.rgw.meta+root+myfoo : type miss (requested=0x16, cached=0x13) 2023-10-05T19:40:26.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.115+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:110 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d33ffdd0 con 0x5653d336cf00 2023-10-05T19:40:26.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.115+0000 7ff345dca700 1 -- 172.21.15.12:0/467862307 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(110 myfoo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv20 ondisk = 0) v8 ==== 233+0+139 (crc 0 0 0) 0x7ff340016070 con 0x5653d336cf00 2023-10-05T19:40:26.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.115+0000 7ff348295e40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x16 2023-10-05T19:40:26.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.115+0000 7ff348295e40 10 moving default.rgw.meta+root+myfoo to cache LRU end 2023-10-05T19:40:26.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.115+0000 7ff348295e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:26.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.115+0000 7ff348295e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:26.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.115+0000 7ff348295e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:26.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.115+0000 7ff348295e40 10 cache get: name=default.rgw.meta+root+myfoo : hit (requested=0x13, cached=0x17) 2023-10-05T19:40:26.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.115+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:111 8.9 8:91b71841:root::myfoo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~165 in=165b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3404c20 con 0x5653d336cf00 2023-10-05T19:40:26.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.119+0000 7ff345dca700 1 -- 172.21.15.12:0/467862307 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(111 myfoo [delete,create,call,call,writefull 0~165] v28'21 uv21 ondisk = 0) v8 ==== 317+0+0 (crc 0 0 0) 0x7ff34004e040 con 0x5653d336cf00 2023-10-05T19:40:26.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.119+0000 7ff348295e40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x17 2023-10-05T19:40:26.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.119+0000 7ff348295e40 10 moving default.rgw.meta+root+myfoo to cache LRU end 2023-10-05T19:40:26.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.119+0000 7ff348295e40 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2023-10-05T19:40:26.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.119+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:112 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94918026479408 in=350b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3404b80 con 0x7ff34001a070 2023-10-05T19:40:26.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.119+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.0 v2:172.21.15.8:6800/27563 33 ==== watch-notify(notify (1) cookie 94918025921392 notify 120259084304 ret 0) v3 ==== 380+0+0 (crc 0 0 0) 0x7ff3380321e0 con 0x7ff34001a070 2023-10-05T19:40:26.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.119+0000 7ff33f7fe700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084304 cookie 94918025921392 notifier 4465 bl.length()=338 2023-10-05T19:40:26.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.119+0000 7ff33f7fe700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo info.flags=0x17 2023-10-05T19:40:26.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.119+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.0 v2:172.21.15.8:6800/27563 34 ==== osd_op_reply(112 notify.6 [notify cookie 94918026479408 out=8b] v0'0 uv120 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7ff338014040 con 0x7ff34001a070 2023-10-05T19:40:26.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.119+0000 7ff33f7fe700 10 rgw watcher librados: moving default.rgw.meta+root+myfoo to cache LRU end 2023-10-05T19:40:26.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.119+0000 7ff33f7fe700 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:113 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7ff320003d90 con 0x7ff34001a070 2023-10-05T19:40:26.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.119+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.0 v2:172.21.15.8:6800/27563 35 ==== watch-notify(notify_complete (2) cookie 94918026479408 notify 120259084304 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7ff338032430 con 0x7ff34001a070 2023-10-05T19:40:26.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.119+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.0 v2:172.21.15.8:6800/27563 36 ==== osd_op_reply(113 notify.6 [notify-ack] v0'0 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff338014040 con 0x7ff34001a070 2023-10-05T19:40:26.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.123+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:114 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94918025894432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d3404c20 con 0x5653d336cf00 2023-10-05T19:40:26.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.123+0000 7ff345dca700 1 -- 172.21.15.12:0/467862307 <== osd.2 v2:172.21.15.12:6800/29934 23 ==== osd_op_reply(114 notify.0 [watch unwatch cookie 94918025894432] v28'122 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff34004e040 con 0x5653d336cf00 2023-10-05T19:40:26.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.123+0000 7ff348295e40 20 remove_watcher() i=0 2023-10-05T19:40:26.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.123+0000 7ff348295e40 2 removed watcher, disabling cache 2023-10-05T19:40:26.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.123+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:115 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94918025906880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d334e9b0 con 0x5653d32acc30 2023-10-05T19:40:26.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.127+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(115 notify.1 [watch unwatch cookie 94918025906880] v28'122 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff338014040 con 0x5653d32acc30 2023-10-05T19:40:26.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.127+0000 7ff348295e40 20 remove_watcher() i=1 2023-10-05T19:40:26.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.127+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:116 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94918025909760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d334e9b0 con 0x5653d33716a0 2023-10-05T19:40:26.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.131+0000 7ff3455c9700 1 -- 172.21.15.12:0/467862307 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(116 notify.2 [watch unwatch cookie 94918025909760] v28'122 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff330017050 con 0x5653d33716a0 2023-10-05T19:40:26.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.131+0000 7ff348295e40 20 remove_watcher() i=2 2023-10-05T19:40:26.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.131+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:117 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94918025915344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d334e9b0 con 0x5653d336cf00 2023-10-05T19:40:26.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.135+0000 7ff345dca700 1 -- 172.21.15.12:0/467862307 <== osd.2 v2:172.21.15.12:6800/29934 24 ==== osd_op_reply(117 notify.3 [watch unwatch cookie 94918025915344] v28'122 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff34004e040 con 0x5653d336cf00 2023-10-05T19:40:26.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.135+0000 7ff348295e40 20 remove_watcher() i=3 2023-10-05T19:40:26.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.135+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:118 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94918025917712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d334e9b0 con 0x5653d32acc30 2023-10-05T19:40:26.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.135+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(118 notify.4 [watch unwatch cookie 94918025917712] v28'122 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff338014040 con 0x5653d32acc30 2023-10-05T19:40:26.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.135+0000 7ff348295e40 20 remove_watcher() i=4 2023-10-05T19:40:26.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.135+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:119 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94918025916448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d334e9b0 con 0x7ff34001a070 2023-10-05T19:40:26.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.139+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.0 v2:172.21.15.8:6800/27563 37 ==== osd_op_reply(119 notify.5 [watch unwatch cookie 94918025916448] v28'122 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff338014040 con 0x7ff34001a070 2023-10-05T19:40:26.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.139+0000 7ff348295e40 20 remove_watcher() i=5 2023-10-05T19:40:26.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.139+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:120 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94918025921392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d334e9b0 con 0x7ff34001a070 2023-10-05T19:40:26.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.143+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.0 v2:172.21.15.8:6800/27563 38 ==== osd_op_reply(120 notify.6 [watch unwatch cookie 94918025921392] v28'122 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff338014040 con 0x7ff34001a070 2023-10-05T19:40:26.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.143+0000 7ff348295e40 20 remove_watcher() i=6 2023-10-05T19:40:26.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.143+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:121 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94918025923968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d334e9b0 con 0x7ff34001a070 2023-10-05T19:40:26.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.147+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.0 v2:172.21.15.8:6800/27563 39 ==== osd_op_reply(121 notify.7 [watch unwatch cookie 94918025923968] v28'122 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff338014040 con 0x7ff34001a070 2023-10-05T19:40:26.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.147+0000 7ff348295e40 20 remove_watcher() i=7 2023-10-05T19:40:26.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.147+0000 7ff3167ec700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:26.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.147+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:122 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94918025925072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5653d334e9b0 con 0x5653d32acc30 2023-10-05T19:40:26.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.151+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(122 data_loggenerations_metadata [watch unwatch cookie 94918025925072] v28'119 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7ff338014040 con 0x5653d32acc30 2023-10-05T19:40:26.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.151+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7ff34001a070 msgr2=0x7ff3400253d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:26.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.151+0000 7ff348295e40 1 --2- 172.21.15.12:0/467862307 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7ff34001a070 0x7ff3400253d0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:26.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.151+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5653d32acc30 msgr2=0x5653d336a9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:26.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.151+0000 7ff348295e40 1 --2- 172.21.15.12:0/467862307 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5653d32acc30 0x5653d336a9d0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:26.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.151+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5653d336cf00 msgr2=0x5653d336f2e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:26.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.151+0000 7ff348295e40 1 --2- 172.21.15.12:0/467862307 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5653d336cf00 0x5653d336f2e0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:26.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.151+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5653d33716a0 msgr2=0x5653d3373ab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:26.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.151+0000 7ff348295e40 1 --2- 172.21.15.12:0/467862307 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5653d33716a0 0x5653d3373ab0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:26.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.151+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7ff32c030c50 msgr2=0x7ff32c033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:26.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.151+0000 7ff348295e40 1 --2- 172.21.15.12:0/467862307 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7ff32c030c50 0x7ff32c033100 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff330000f80 tx=0x7ff330013000 comp rx=0 tx=0).stop 2023-10-05T19:40:26.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.151+0000 7ff3465cb700 1 -- 172.21.15.12:0/467862307 reap_dead start 2023-10-05T19:40:26.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.151+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5653d32ab920 msgr2=0x5653d335eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:26.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.151+0000 7ff348295e40 1 --2- 172.21.15.12:0/467862307 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5653d32ab920 0x5653d335eee0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff3400018e0 tx=0x7ff340016000 comp rx=0 tx=0).stop 2023-10-05T19:40:26.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.151+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 shutdown_connections 2023-10-05T19:40:26.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.151+0000 7ff348295e40 1 --2- 172.21.15.12:0/467862307 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5653d32ab920 0x5653d335eee0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:26.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.151+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 >> 172.21.15.12:0/467862307 conn(0x5653d3332860 msgr2=0x5653d3352750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:26.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.151+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 shutdown_connections 2023-10-05T19:40:26.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.151+0000 7ff348295e40 1 -- 172.21.15.12:0/467862307 wait complete. 2023-10-05T19:40:26.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.151+0000 7ff348295e40 1 -- 172.21.15.12:0/446319665 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7ff28c030c00 msgr2=0x7ff28c0330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:26.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.151+0000 7ff348295e40 1 --2- 172.21.15.12:0/446319665 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7ff28c030c00 0x7ff28c0330b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x5653d33807d0 tx=0x7ff33800ab80 comp rx=0 tx=0).stop 2023-10-05T19:40:26.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.151+0000 7ff348295e40 1 -- 172.21.15.12:0/446319665 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5653d338af90 msgr2=0x5653d3390a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:26.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.151+0000 7ff348295e40 1 --2- 172.21.15.12:0/446319665 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5653d338af90 0x5653d3390a40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff330039040 tx=0x7ff330036000 comp rx=0 tx=0).stop 2023-10-05T19:40:26.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.151+0000 7ff348295e40 1 -- 172.21.15.12:0/446319665 shutdown_connections 2023-10-05T19:40:26.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.151+0000 7ff348295e40 1 --2- 172.21.15.12:0/446319665 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7ff28c030c00 0x7ff28c0330b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:26.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.151+0000 7ff348295e40 1 --2- 172.21.15.12:0/446319665 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5653d338af90 0x5653d3390a40 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:26.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.151+0000 7ff348295e40 1 -- 172.21.15.12:0/446319665 >> 172.21.15.12:0/446319665 conn(0x5653d3386350 msgr2=0x5653d33843f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:26.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.151+0000 7ff348295e40 1 -- 172.21.15.12:0/446319665 shutdown_connections 2023-10-05T19:40:26.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.151+0000 7ff348295e40 1 -- 172.21.15.12:0/446319665 wait complete. 2023-10-05T19:40:26.168 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'link', '--uid', 'foo', '--bucket', 'myfoo', '--bucket-id', '93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1'] 2023-10-05T19:40:26.168 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'link', '--uid', 'foo', '--bucket', 'myfoo', '--bucket-id', '93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1'] 2023-10-05T19:40:26.168 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket link --uid foo --bucket myfoo --bucket-id 93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 2023-10-05T19:40:26.212 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:26.212 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:26.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.203+0000 7faec9ff6e40 1 Processor -- start 2023-10-05T19:40:26.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.203+0000 7faec9ff6e40 1 -- start start 2023-10-05T19:40:26.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.203+0000 7faec9ff6e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55c62b90fbc0 0x55c62b90e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:26.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.203+0000 7faec9ff6e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55c62b8e6bd0 con 0x55c62b90e6e0 2023-10-05T19:40:26.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.203+0000 7faec9ff6e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x55c62b8a3ee0 con 0x55c62b90fbc0 2023-10-05T19:40:26.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.203+0000 7faec7b2b700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55c62b90fbc0 0x55c62b90e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:26.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.207+0000 7faec732a700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x55c62b90e6e0 0x55c62b9a1ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:35568/0 (socket says 172.21.15.12:35568) 2023-10-05T19:40:26.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.207+0000 7faec7b2b700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55c62b90fbc0 0x55c62b90e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58062/0 (socket says 172.21.15.12:58062) 2023-10-05T19:40:26.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.207+0000 7faec732a700 1 -- 172.21.15.12:0/1435813489 learned_addr learned my addr 172.21.15.12:0/1435813489 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:26.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.207+0000 7faec7b2b700 1 -- 172.21.15.12:0/1435813489 >> v1:172.21.15.8:6789/0 conn(0x55c62b90e6e0 legacy=0x55c62b9a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:26.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.207+0000 7faec7b2b700 1 -- 172.21.15.12:0/1435813489 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c62b910050 con 0x55c62b90fbc0 2023-10-05T19:40:26.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.207+0000 7faec7b2b700 1 --2- 172.21.15.12:0/1435813489 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c62b90fbc0 0x55c62b90e1a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7faeb800a5d0 tx=0x7faeb8005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ba6f25ecc49bf03 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:26.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.207+0000 7faec6b29700 1 -- 172.21.15.12:0/1435813489 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faeb80086a0 con 0x55c62b90fbc0 2023-10-05T19:40:26.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.207+0000 7faec6b29700 1 -- 172.21.15.12:0/1435813489 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb8008800 con 0x55c62b90fbc0 2023-10-05T19:40:26.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.207+0000 7faec6b29700 1 -- 172.21.15.12:0/1435813489 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faeb8008ad0 con 0x55c62b90fbc0 2023-10-05T19:40:26.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.207+0000 7faec9ff6e40 1 -- 172.21.15.12:0/1435813489 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c62b90fbc0 msgr2=0x55c62b90e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:26.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.207+0000 7faec9ff6e40 1 --2- 172.21.15.12:0/1435813489 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c62b90fbc0 0x55c62b90e1a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7faeb800a5d0 tx=0x7faeb8005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:26.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.207+0000 7faec9ff6e40 1 -- 172.21.15.12:0/1435813489 shutdown_connections 2023-10-05T19:40:26.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.207+0000 7faec9ff6e40 1 --2- 172.21.15.12:0/1435813489 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c62b90fbc0 0x55c62b90e1a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:26.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.207+0000 7faec9ff6e40 1 -- 172.21.15.12:0/1435813489 >> 172.21.15.12:0/1435813489 conn(0x55c62b9957f0 msgr2=0x55c62b997c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:26.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.207+0000 7faec9ff6e40 1 -- 172.21.15.12:0/1435813489 shutdown_connections 2023-10-05T19:40:26.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.207+0000 7faec9ff6e40 1 -- 172.21.15.12:0/1435813489 wait complete. 2023-10-05T19:40:26.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.207+0000 7faec9ff6e40 1 Processor -- start 2023-10-05T19:40:26.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.207+0000 7faec9ff6e40 1 -- start start 2023-10-05T19:40:26.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.207+0000 7faec9ff6e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c62b90e6e0 0x55c62b9bb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:26.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.207+0000 7faec9ff6e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c62b867e30 con 0x55c62b90e6e0 2023-10-05T19:40:26.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.207+0000 7faec7b2b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c62b90e6e0 0x55c62b9bb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:26.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.207+0000 7faec7b2b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c62b90e6e0 0x55c62b9bb140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58070/0 (socket says 172.21.15.12:58070) 2023-10-05T19:40:26.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.211+0000 7faec7b2b700 1 -- 172.21.15.12:0/1360946213 learned_addr learned my addr 172.21.15.12:0/1360946213 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:26.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.211+0000 7faec7b2b700 1 -- 172.21.15.12:0/1360946213 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb8005cf0 con 0x55c62b90e6e0 2023-10-05T19:40:26.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.211+0000 7faec7b2b700 1 --2- 172.21.15.12:0/1360946213 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c62b90e6e0 0x55c62b9bb140 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7faeb8005a90 tx=0x7faeb8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd7e9a6d2fd724b server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:26.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.211+0000 7faec5b27700 1 -- 172.21.15.12:0/1360946213 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faeb8008a40 con 0x55c62b90e6e0 2023-10-05T19:40:26.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.211+0000 7faec5b27700 1 -- 172.21.15.12:0/1360946213 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb8008bc0 con 0x55c62b90e6e0 2023-10-05T19:40:26.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.211+0000 7faec5b27700 1 -- 172.21.15.12:0/1360946213 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faeb8012450 con 0x55c62b90e6e0 2023-10-05T19:40:26.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.211+0000 7faec9ff6e40 1 -- 172.21.15.12:0/1360946213 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c62b90e6e0 msgr2=0x55c62b9bb140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:26.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.211+0000 7faec9ff6e40 1 --2- 172.21.15.12:0/1360946213 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c62b90e6e0 0x55c62b9bb140 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7faeb8005a90 tx=0x7faeb8015000 comp rx=0 tx=0).stop 2023-10-05T19:40:26.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.211+0000 7faec9ff6e40 1 -- 172.21.15.12:0/1360946213 shutdown_connections 2023-10-05T19:40:26.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.211+0000 7faec9ff6e40 1 --2- 172.21.15.12:0/1360946213 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c62b90e6e0 0x55c62b9bb140 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:26.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.211+0000 7faec9ff6e40 1 -- 172.21.15.12:0/1360946213 >> 172.21.15.12:0/1360946213 conn(0x55c62b9957f0 msgr2=0x55c62b9b6c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:26.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.211+0000 7faec9ff6e40 1 -- 172.21.15.12:0/1360946213 shutdown_connections 2023-10-05T19:40:26.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.211+0000 7faec9ff6e40 1 -- 172.21.15.12:0/1360946213 wait complete. 2023-10-05T19:40:26.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.211+0000 7faec9ff6e40 1 Processor -- start 2023-10-05T19:40:26.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.211+0000 7faec9ff6e40 1 -- start start 2023-10-05T19:40:26.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.211+0000 7faec9ff6e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c62b90e6e0 0x55c62b9c1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:26.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.211+0000 7faec9ff6e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c62b83aed0 con 0x55c62b90e6e0 2023-10-05T19:40:26.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.211+0000 7faec7b2b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c62b90e6e0 0x55c62b9c1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:26.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.211+0000 7faec7b2b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c62b90e6e0 0x55c62b9c1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58072/0 (socket says 172.21.15.12:58072) 2023-10-05T19:40:26.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.211+0000 7faec7b2b700 1 -- 172.21.15.12:0/4264792383 learned_addr learned my addr 172.21.15.12:0/4264792383 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:26.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.215+0000 7faec7b2b700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c62b910050 con 0x55c62b90e6e0 2023-10-05T19:40:26.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.215+0000 7faec7b2b700 1 --2- 172.21.15.12:0/4264792383 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c62b90e6e0 0x55c62b9c1dc0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7faeb800c010 tx=0x7faeb801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:26.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.215+0000 7faebffff700 1 -- 172.21.15.12:0/4264792383 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faeb8008e50 con 0x55c62b90e6e0 2023-10-05T19:40:26.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.215+0000 7faebffff700 1 -- 172.21.15.12:0/4264792383 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb8012660 con 0x55c62b90e6e0 2023-10-05T19:40:26.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.215+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c62b9c2300 con 0x55c62b90e6e0 2023-10-05T19:40:26.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.215+0000 7faebffff700 1 -- 172.21.15.12:0/4264792383 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faeb8012950 con 0x55c62b90e6e0 2023-10-05T19:40:26.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.215+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c62b9c27a0 con 0x55c62b90e6e0 2023-10-05T19:40:26.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.215+0000 7faec9ff6e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.215+0000 7faebffff700 1 -- 172.21.15.12:0/4264792383 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7faeb8019070 con 0x55c62b90e6e0 2023-10-05T19:40:26.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.215+0000 7faebffff700 1 --2- 172.21.15.12:0/4264792383 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7faea8030c50 0x7faea8033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:26.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.215+0000 7faebffff700 1 -- 172.21.15.12:0/4264792383 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7faeb8045fa0 con 0x55c62b90e6e0 2023-10-05T19:40:26.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.215+0000 7faec732a700 1 --2- 172.21.15.12:0/4264792383 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7faea8030c50 0x7faea8033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:26.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.215+0000 7faec9ff6e40 1 --2- 172.21.15.12:0/4264792383 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55c62b90fbc0 0x55c62b9cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:26.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.215+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9cded0 con 0x55c62b90fbc0 2023-10-05T19:40:26.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.219+0000 7faec832c700 1 --2- 172.21.15.12:0/4264792383 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55c62b90fbc0 0x55c62b9cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:26.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.219+0000 7faec732a700 1 --2- 172.21.15.12:0/4264792383 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7faea8030c50 0x7faea8033100 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7faeb0001e90 tx=0x7faeb00013b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:26.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.219+0000 7faec832c700 1 --2- 172.21.15.12:0/4264792383 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55c62b90fbc0 0x55c62b9cd8b0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:26.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.219+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7faec000a5a0 con 0x55c62b90fbc0 2023-10-05T19:40:26.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.219+0000 7faec9ff6e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:26.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.219+0000 7faec9ff6e40 20 realm 2023-10-05T19:40:26.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.219+0000 7faec9ff6e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.219+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9caf10 con 0x55c62b90fbc0 2023-10-05T19:40:26.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.219+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7faec000a5a0 con 0x55c62b90fbc0 2023-10-05T19:40:26.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.219+0000 7faec9ff6e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:26.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.219+0000 7faec9ff6e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:26.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.219+0000 7faec9ff6e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.219+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9ce9b0 con 0x55c62b90fbc0 2023-10-05T19:40:26.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.219+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7faec000a5a0 con 0x55c62b90fbc0 2023-10-05T19:40:26.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.219+0000 7faec9ff6e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:26.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.219+0000 7faec9ff6e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.219+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9ce5c0 con 0x55c62b90fbc0 2023-10-05T19:40:26.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.219+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7faec000a5a0 con 0x55c62b90fbc0 2023-10-05T19:40:26.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.219+0000 7faec9ff6e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:26.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.219+0000 7faec9ff6e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.219+0000 7faec9ff6e40 1 --2- 172.21.15.12:0/4264792383 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55c62b9cfd50 0x55c62b9d2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:26.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.219+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d2670 con 0x55c62b9cfd50 2023-10-05T19:40:26.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.219+0000 7faec7b2b700 1 --2- 172.21.15.12:0/4264792383 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55c62b9cfd50 0x55c62b9d2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:26.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.223+0000 7faec7b2b700 1 --2- 172.21.15.12:0/4264792383 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55c62b9cfd50 0x55c62b9d2130 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:26.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.223+0000 7faec7b2b700 1 -- 172.21.15.12:0/4264792383 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7faeb8012ab0 con 0x55c62b9cfd50 2023-10-05T19:40:26.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.223+0000 7faec9ff6e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:26.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.223+0000 7faec9ff6e40 20 searching for the correct realm 2023-10-05T19:40:26.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.223+0000 7faec9ff6e40 1 --2- 172.21.15.12:0/4264792383 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55c62b9d44f0 0x55c62b9d6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:26.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.223+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d6fb0 con 0x55c62b9d44f0 2023-10-05T19:40:26.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.223+0000 7faec732a700 1 --2- 172.21.15.12:0/4264792383 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55c62b9d44f0 0x55c62b9d6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:26.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.223+0000 7faec732a700 1 --2- 172.21.15.12:0/4264792383 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55c62b9d44f0 0x55c62b9d6900 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:26.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.223+0000 7faec732a700 1 -- 172.21.15.12:0/4264792383 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7faeb0006070 con 0x55c62b9d44f0 2023-10-05T19:40:26.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.223+0000 7faec732a700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d6fb0 con 0x55c62b9cfd50 2023-10-05T19:40:26.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.227+0000 7faec7b2b700 1 -- 172.21.15.12:0/4264792383 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faeb8012ab0 con 0x55c62b9cfd50 2023-10-05T19:40:26.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.227+0000 7faec7b2b700 1 --2- 172.21.15.12:0/4264792383 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7faeb8007030 0x7faeb8037640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:26.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.227+0000 7faec7b2b700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d6fb0 con 0x7faeb8007030 2023-10-05T19:40:26.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.227+0000 7faec832c700 1 --2- 172.21.15.12:0/4264792383 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7faeb8007030 0x7faeb8037640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:26.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.227+0000 7faec832c700 1 --2- 172.21.15.12:0/4264792383 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7faeb8007030 0x7faeb8037640 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:26.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.227+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faec000a5a0 con 0x7faeb8007030 2023-10-05T19:40:26.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.227+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d6fb0 con 0x55c62b90fbc0 2023-10-05T19:40:26.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.227+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faec000a5a0 con 0x55c62b90fbc0 2023-10-05T19:40:26.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.227+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d6fb0 con 0x55c62b90fbc0 2023-10-05T19:40:26.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.227+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faec000a5a0 con 0x55c62b90fbc0 2023-10-05T19:40:26.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.227+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d6fb0 con 0x7faeb8007030 2023-10-05T19:40:26.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.231+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faec000a5a0 con 0x7faeb8007030 2023-10-05T19:40:26.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.231+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d6fb0 con 0x7faeb8007030 2023-10-05T19:40:26.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.231+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faec000a5a0 con 0x7faeb8007030 2023-10-05T19:40:26.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.231+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d6fb0 con 0x55c62b9cfd50 2023-10-05T19:40:26.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.231+0000 7faec7b2b700 1 -- 172.21.15.12:0/4264792383 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faeb8012ab0 con 0x55c62b9cfd50 2023-10-05T19:40:26.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.231+0000 7faec7b2b700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d6fb0 con 0x55c62b9d44f0 2023-10-05T19:40:26.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.231+0000 7faec732a700 1 -- 172.21.15.12:0/4264792383 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faeb0006070 con 0x55c62b9d44f0 2023-10-05T19:40:26.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.231+0000 7faec732a700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d6fb0 con 0x55c62b90fbc0 2023-10-05T19:40:26.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.231+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faec000a5a0 con 0x55c62b90fbc0 2023-10-05T19:40:26.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.235+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d6fb0 con 0x7faeb8007030 2023-10-05T19:40:26.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.235+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faec000a5a0 con 0x7faeb8007030 2023-10-05T19:40:26.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.235+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d6fb0 con 0x55c62b90fbc0 2023-10-05T19:40:26.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.235+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faec000a5a0 con 0x55c62b90fbc0 2023-10-05T19:40:26.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.235+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d6fb0 con 0x55c62b9cfd50 2023-10-05T19:40:26.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.235+0000 7faec7b2b700 1 -- 172.21.15.12:0/4264792383 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faeb8012ab0 con 0x55c62b9cfd50 2023-10-05T19:40:26.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.235+0000 7faec7b2b700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d6fb0 con 0x55c62b9d44f0 2023-10-05T19:40:26.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.235+0000 7faec732a700 1 -- 172.21.15.12:0/4264792383 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faeb0006070 con 0x55c62b9d44f0 2023-10-05T19:40:26.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.235+0000 7faec732a700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d6fb0 con 0x55c62b9d44f0 2023-10-05T19:40:26.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.239+0000 7faec732a700 1 -- 172.21.15.12:0/4264792383 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faeb0006070 con 0x55c62b9d44f0 2023-10-05T19:40:26.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.239+0000 7faec732a700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d6fb0 con 0x7faeb8007030 2023-10-05T19:40:26.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.239+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faec000a5a0 con 0x7faeb8007030 2023-10-05T19:40:26.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.239+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d6fb0 con 0x55c62b9d44f0 2023-10-05T19:40:26.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.239+0000 7faec732a700 1 -- 172.21.15.12:0/4264792383 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faeb0006070 con 0x55c62b9d44f0 2023-10-05T19:40:26.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.239+0000 7faec732a700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d6fb0 con 0x55c62b9d44f0 2023-10-05T19:40:26.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.239+0000 7faec732a700 1 -- 172.21.15.12:0/4264792383 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faeb0006070 con 0x55c62b9d44f0 2023-10-05T19:40:26.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.239+0000 7faec732a700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d6fb0 con 0x55c62b90fbc0 2023-10-05T19:40:26.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.239+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faec000a5a0 con 0x55c62b90fbc0 2023-10-05T19:40:26.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.239+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d6fb0 con 0x55c62b90fbc0 2023-10-05T19:40:26.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.243+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faec000a5a0 con 0x55c62b90fbc0 2023-10-05T19:40:26.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.243+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d6fb0 con 0x7faeb8007030 2023-10-05T19:40:26.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.243+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faec000a5a0 con 0x7faeb8007030 2023-10-05T19:40:26.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.243+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d6fb0 con 0x55c62b9d44f0 2023-10-05T19:40:26.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.243+0000 7faec732a700 1 -- 172.21.15.12:0/4264792383 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faeb0006070 con 0x55c62b9d44f0 2023-10-05T19:40:26.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.243+0000 7faec732a700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d6fb0 con 0x55c62b9cfd50 2023-10-05T19:40:26.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.243+0000 7faec7b2b700 1 -- 172.21.15.12:0/4264792383 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faeb8012ab0 con 0x55c62b9cfd50 2023-10-05T19:40:26.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.243+0000 7faec7b2b700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d6fb0 con 0x55c62b90fbc0 2023-10-05T19:40:26.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.247+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faec000a5a0 con 0x55c62b90fbc0 2023-10-05T19:40:26.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.247+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d6fb0 con 0x7faeb8007030 2023-10-05T19:40:26.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.247+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faec000a5a0 con 0x7faeb8007030 2023-10-05T19:40:26.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.247+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d6fb0 con 0x55c62b9d44f0 2023-10-05T19:40:26.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.247+0000 7faec732a700 1 -- 172.21.15.12:0/4264792383 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faeb0006070 con 0x55c62b9d44f0 2023-10-05T19:40:26.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.247+0000 7faec732a700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d6fb0 con 0x7faeb8007030 2023-10-05T19:40:26.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.247+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faec000a5a0 con 0x7faeb8007030 2023-10-05T19:40:26.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.247+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d6fb0 con 0x55c62b9cfd50 2023-10-05T19:40:26.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.247+0000 7faec7b2b700 1 -- 172.21.15.12:0/4264792383 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faeb8012ab0 con 0x55c62b9cfd50 2023-10-05T19:40:26.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.247+0000 7faec7b2b700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d6fb0 con 0x55c62b9d44f0 2023-10-05T19:40:26.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.251+0000 7faec732a700 1 -- 172.21.15.12:0/4264792383 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faeb0006070 con 0x55c62b9d44f0 2023-10-05T19:40:26.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.251+0000 7faec732a700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d6fb0 con 0x7faeb8007030 2023-10-05T19:40:26.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.251+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faec000a5a0 con 0x7faeb8007030 2023-10-05T19:40:26.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.251+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d6fb0 con 0x55c62b90fbc0 2023-10-05T19:40:26.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.251+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7faec000a5a0 con 0x55c62b90fbc0 2023-10-05T19:40:26.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.251+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d6fb0 con 0x55c62b9cfd50 2023-10-05T19:40:26.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.251+0000 7faec7b2b700 1 -- 172.21.15.12:0/4264792383 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7faeb8012ab0 con 0x55c62b9cfd50 2023-10-05T19:40:26.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.251+0000 7faec9ff6e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:26.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.251+0000 7faec9ff6e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:26.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.251+0000 7faec9ff6e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:26.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.251+0000 7faec9ff6e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:26.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.251+0000 7faec9ff6e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.251+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d32a0 con 0x55c62b90fbc0 2023-10-05T19:40:26.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.255+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7faec000a5a0 con 0x55c62b90fbc0 2023-10-05T19:40:26.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.255+0000 7faec9ff6e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:26.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.255+0000 7faec9ff6e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.255+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d2ed0 con 0x55c62b90fbc0 2023-10-05T19:40:26.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.255+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7faec000a5a0 con 0x55c62b90fbc0 2023-10-05T19:40:26.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.255+0000 7faec9ff6e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:26.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.255+0000 7faec9ff6e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.255+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d7420 con 0x55c62b9d44f0 2023-10-05T19:40:26.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.255+0000 7faec732a700 1 -- 172.21.15.12:0/4264792383 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7faeb0006070 con 0x55c62b9d44f0 2023-10-05T19:40:26.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.255+0000 7faec9ff6e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:26.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.255+0000 7faec9ff6e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.255+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d77d0 con 0x55c62b9d44f0 2023-10-05T19:40:26.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.255+0000 7faec732a700 1 -- 172.21.15.12:0/4264792383 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7faeb0006070 con 0x55c62b9d44f0 2023-10-05T19:40:26.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.255+0000 7faec9ff6e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:26.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.255+0000 7faec9ff6e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.255+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d7b80 con 0x55c62b90fbc0 2023-10-05T19:40:26.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.255+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7faec000a5a0 con 0x55c62b90fbc0 2023-10-05T19:40:26.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.255+0000 7faec9ff6e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:26.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.255+0000 7faec9ff6e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.255+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d3730 con 0x55c62b90fbc0 2023-10-05T19:40:26.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.259+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7faec000a5a0 con 0x55c62b90fbc0 2023-10-05T19:40:26.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.259+0000 7faec9ff6e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:26.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.259+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d3700 con 0x55c62b9d44f0 2023-10-05T19:40:26.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.259+0000 7faec732a700 1 -- 172.21.15.12:0/4264792383 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7faeb0006070 con 0x55c62b9d44f0 2023-10-05T19:40:26.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.259+0000 7faec732a700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d3700 con 0x55c62b9cfd50 2023-10-05T19:40:26.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.263+0000 7faec7b2b700 1 -- 172.21.15.12:0/4264792383 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faeb8012ab0 con 0x55c62b9cfd50 2023-10-05T19:40:26.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.263+0000 7faec7b2b700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d3700 con 0x7faeb8007030 2023-10-05T19:40:26.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.263+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faec000a5a0 con 0x7faeb8007030 2023-10-05T19:40:26.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.263+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d7b80 con 0x55c62b90fbc0 2023-10-05T19:40:26.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.263+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faec000a5a0 con 0x55c62b90fbc0 2023-10-05T19:40:26.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.263+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d7b80 con 0x55c62b90fbc0 2023-10-05T19:40:26.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.263+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faec000a5a0 con 0x55c62b90fbc0 2023-10-05T19:40:26.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.263+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d7b80 con 0x7faeb8007030 2023-10-05T19:40:26.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.263+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faec000a5a0 con 0x7faeb8007030 2023-10-05T19:40:26.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.263+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d7b80 con 0x7faeb8007030 2023-10-05T19:40:26.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.267+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faec000a5a0 con 0x7faeb8007030 2023-10-05T19:40:26.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.267+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d7b80 con 0x55c62b9cfd50 2023-10-05T19:40:26.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.267+0000 7faec7b2b700 1 -- 172.21.15.12:0/4264792383 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faeb8012ab0 con 0x55c62b9cfd50 2023-10-05T19:40:26.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.267+0000 7faec7b2b700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d7b80 con 0x55c62b9d44f0 2023-10-05T19:40:26.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.267+0000 7faec732a700 1 -- 172.21.15.12:0/4264792383 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faeb0006070 con 0x55c62b9d44f0 2023-10-05T19:40:26.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.267+0000 7faec732a700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d7b80 con 0x55c62b90fbc0 2023-10-05T19:40:26.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.267+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faec000a5a0 con 0x55c62b90fbc0 2023-10-05T19:40:26.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.267+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d7b80 con 0x7faeb8007030 2023-10-05T19:40:26.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.271+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faec000a5a0 con 0x7faeb8007030 2023-10-05T19:40:26.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.271+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d7b80 con 0x55c62b90fbc0 2023-10-05T19:40:26.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.271+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faec000a5a0 con 0x55c62b90fbc0 2023-10-05T19:40:26.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.271+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d7b80 con 0x55c62b9cfd50 2023-10-05T19:40:26.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.271+0000 7faec7b2b700 1 -- 172.21.15.12:0/4264792383 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faeb8012ab0 con 0x55c62b9cfd50 2023-10-05T19:40:26.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.271+0000 7faec7b2b700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d7b80 con 0x55c62b9d44f0 2023-10-05T19:40:26.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.271+0000 7faec732a700 1 -- 172.21.15.12:0/4264792383 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faeb0006070 con 0x55c62b9d44f0 2023-10-05T19:40:26.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.271+0000 7faec732a700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d7b80 con 0x55c62b9d44f0 2023-10-05T19:40:26.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.271+0000 7faec732a700 1 -- 172.21.15.12:0/4264792383 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faeb0006070 con 0x55c62b9d44f0 2023-10-05T19:40:26.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.271+0000 7faec732a700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d7b80 con 0x7faeb8007030 2023-10-05T19:40:26.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.275+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faec000a5a0 con 0x7faeb8007030 2023-10-05T19:40:26.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.275+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d7b80 con 0x55c62b9d44f0 2023-10-05T19:40:26.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.275+0000 7faec732a700 1 -- 172.21.15.12:0/4264792383 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faeb0006070 con 0x55c62b9d44f0 2023-10-05T19:40:26.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.275+0000 7faec732a700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d7b80 con 0x55c62b9d44f0 2023-10-05T19:40:26.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.275+0000 7faec732a700 1 -- 172.21.15.12:0/4264792383 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faeb0006070 con 0x55c62b9d44f0 2023-10-05T19:40:26.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.275+0000 7faec732a700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d7b80 con 0x55c62b90fbc0 2023-10-05T19:40:26.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.275+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faec000a5a0 con 0x55c62b90fbc0 2023-10-05T19:40:26.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.275+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d7b80 con 0x55c62b90fbc0 2023-10-05T19:40:26.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.275+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faec000a5a0 con 0x55c62b90fbc0 2023-10-05T19:40:26.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.275+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d7b80 con 0x7faeb8007030 2023-10-05T19:40:26.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.279+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faec000a5a0 con 0x7faeb8007030 2023-10-05T19:40:26.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.279+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d7b80 con 0x55c62b9d44f0 2023-10-05T19:40:26.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.279+0000 7faec732a700 1 -- 172.21.15.12:0/4264792383 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faeb0006070 con 0x55c62b9d44f0 2023-10-05T19:40:26.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.279+0000 7faec732a700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d7b80 con 0x55c62b9cfd50 2023-10-05T19:40:26.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.279+0000 7faec7b2b700 1 -- 172.21.15.12:0/4264792383 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faeb8012ab0 con 0x55c62b9cfd50 2023-10-05T19:40:26.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.279+0000 7faec7b2b700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d7b80 con 0x55c62b90fbc0 2023-10-05T19:40:26.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.279+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faec000a5a0 con 0x55c62b90fbc0 2023-10-05T19:40:26.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.279+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d7b80 con 0x7faeb8007030 2023-10-05T19:40:26.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.283+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faec000a5a0 con 0x7faeb8007030 2023-10-05T19:40:26.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.283+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d7b80 con 0x55c62b9d44f0 2023-10-05T19:40:26.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.283+0000 7faec732a700 1 -- 172.21.15.12:0/4264792383 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faeb0006070 con 0x55c62b9d44f0 2023-10-05T19:40:26.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.283+0000 7faec732a700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d7b80 con 0x7faeb8007030 2023-10-05T19:40:26.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.283+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faec000a5a0 con 0x7faeb8007030 2023-10-05T19:40:26.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.283+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d7b80 con 0x55c62b9cfd50 2023-10-05T19:40:26.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.283+0000 7faec7b2b700 1 -- 172.21.15.12:0/4264792383 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faeb8012ab0 con 0x55c62b9cfd50 2023-10-05T19:40:26.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.283+0000 7faec7b2b700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d7b80 con 0x55c62b9d44f0 2023-10-05T19:40:26.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.283+0000 7faec732a700 1 -- 172.21.15.12:0/4264792383 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faeb0006070 con 0x55c62b9d44f0 2023-10-05T19:40:26.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.283+0000 7faec732a700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d7b80 con 0x7faeb8007030 2023-10-05T19:40:26.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.287+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faec000a5a0 con 0x7faeb8007030 2023-10-05T19:40:26.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.287+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d7b80 con 0x55c62b90fbc0 2023-10-05T19:40:26.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.287+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7faec000a5a0 con 0x55c62b90fbc0 2023-10-05T19:40:26.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.287+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d7b80 con 0x55c62b9cfd50 2023-10-05T19:40:26.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.287+0000 7faec7b2b700 1 -- 172.21.15.12:0/4264792383 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7faeb8012ab0 con 0x55c62b9cfd50 2023-10-05T19:40:26.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.287+0000 7faec9ff6e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:26.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.287+0000 7faec9ff6e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:26.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.287+0000 7faec9ff6e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:26.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.287+0000 7faec9ff6e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:26.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.287+0000 7faec9ff6e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.287+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d3700 con 0x55c62b90fbc0 2023-10-05T19:40:26.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.287+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7faec000a5a0 con 0x55c62b90fbc0 2023-10-05T19:40:26.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.287+0000 7faec9ff6e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:26.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.287+0000 7faec9ff6e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.287+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d8470 con 0x55c62b9cfd50 2023-10-05T19:40:26.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.291+0000 7faec7b2b700 1 -- 172.21.15.12:0/4264792383 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7faeb8012ab0 con 0x55c62b9cfd50 2023-10-05T19:40:26.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.291+0000 7faec9ff6e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:26.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.291+0000 7faec9ff6e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.291+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d8d60 con 0x55c62b90fbc0 2023-10-05T19:40:26.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.291+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7faec000a5a0 con 0x55c62b90fbc0 2023-10-05T19:40:26.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.291+0000 7faec9ff6e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:26.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.291+0000 7faec9ff6e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.291+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d8d60 con 0x55c62b9d44f0 2023-10-05T19:40:26.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.291+0000 7faec732a700 1 -- 172.21.15.12:0/4264792383 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7faeb0006070 con 0x55c62b9d44f0 2023-10-05T19:40:26.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.291+0000 7faec9ff6e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:26.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.291+0000 7faec9ff6e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.291+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d94d0 con 0x7faeb8007030 2023-10-05T19:40:26.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.291+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7faec000a5a0 con 0x7faeb8007030 2023-10-05T19:40:26.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.291+0000 7faec9ff6e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:26.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.291+0000 7faec9ff6e40 20 zone default found 2023-10-05T19:40:26.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.291+0000 7faec9ff6e40 4 Realm: () 2023-10-05T19:40:26.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.291+0000 7faec9ff6e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:26.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.291+0000 7faec9ff6e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:26.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.291+0000 7faec9ff6e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:26.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.291+0000 7faec9ff6e40 20 zonegroup default 2023-10-05T19:40:26.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.291+0000 7faec9ff6e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.291+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d9210 con 0x7faeb8007030 2023-10-05T19:40:26.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.295+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7faec000a5a0 con 0x7faeb8007030 2023-10-05T19:40:26.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.295+0000 7faec9ff6e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:26.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.295+0000 7faec9ff6e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.295+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9dab90 con 0x55c62b9d44f0 2023-10-05T19:40:26.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.295+0000 7faec732a700 1 -- 172.21.15.12:0/4264792383 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7faeb0006070 con 0x55c62b9d44f0 2023-10-05T19:40:26.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.295+0000 7faec9ff6e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:26.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.295+0000 7faec9ff6e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.295+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9daf40 con 0x55c62b9d44f0 2023-10-05T19:40:26.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.295+0000 7faec732a700 1 -- 172.21.15.12:0/4264792383 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7faeb0006070 con 0x55c62b9d44f0 2023-10-05T19:40:26.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.295+0000 7faec9ff6e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:26.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.295+0000 7faec9ff6e40 20 started sync module instance, tier type = 2023-10-05T19:40:26.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.295+0000 7faec9ff6e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:26.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.295+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9da250 con 0x55c62b9cfd50 2023-10-05T19:40:26.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.299+0000 7faec7b2b700 1 -- 172.21.15.12:0/4264792383 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'123 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faeb8012ab0 con 0x55c62b9cfd50 2023-10-05T19:40:26.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.299+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94309623637504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9d8990 con 0x55c62b9cfd50 2023-10-05T19:40:26.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.299+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9dbd40 con 0x55c62b90fbc0 2023-10-05T19:40:26.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.303+0000 7faec7b2b700 1 -- 172.21.15.12:0/4264792383 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94309623637504] v28'124 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faeb8012ab0 con 0x55c62b9cfd50 2023-10-05T19:40:26.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.303+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'123 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faec000a5a0 con 0x55c62b90fbc0 2023-10-05T19:40:26.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.303+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94309623640816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9dc140 con 0x55c62b90fbc0 2023-10-05T19:40:26.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.303+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9dc9e0 con 0x55c62b9d44f0 2023-10-05T19:40:26.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.303+0000 7faec732a700 1 -- 172.21.15.12:0/4264792383 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'123 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faeb0006070 con 0x55c62b9d44f0 2023-10-05T19:40:26.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.303+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94309623644768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9dd2d0 con 0x55c62b9d44f0 2023-10-05T19:40:26.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.303+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9ddce0 con 0x55c62b9cfd50 2023-10-05T19:40:26.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.307+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94309623640816] v28'124 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faec000a5a0 con 0x55c62b90fbc0 2023-10-05T19:40:26.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.307+0000 7faec7b2b700 1 -- 172.21.15.12:0/4264792383 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'123 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faeb8012ab0 con 0x55c62b9cfd50 2023-10-05T19:40:26.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.307+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94309623642128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9dd910 con 0x55c62b9cfd50 2023-10-05T19:40:26.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.307+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9dec80 con 0x55c62b90fbc0 2023-10-05T19:40:26.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.307+0000 7faec732a700 1 -- 172.21.15.12:0/4264792383 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94309623644768] v28'124 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faeb0006070 con 0x55c62b9d44f0 2023-10-05T19:40:26.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.311+0000 7faec7b2b700 1 -- 172.21.15.12:0/4264792383 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94309623642128] v28'124 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faeb8012ab0 con 0x55c62b9cfd50 2023-10-05T19:40:26.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.311+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'123 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faec000a5a0 con 0x55c62b90fbc0 2023-10-05T19:40:26.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.311+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94309623652496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9de750 con 0x55c62b90fbc0 2023-10-05T19:40:26.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.311+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9de750 con 0x7faeb8007030 2023-10-05T19:40:26.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.311+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94309623652496] v28'124 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faec000a5a0 con 0x55c62b90fbc0 2023-10-05T19:40:26.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.311+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'123 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faec000a5a0 con 0x7faeb8007030 2023-10-05T19:40:26.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.311+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94309623647712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9df160 con 0x7faeb8007030 2023-10-05T19:40:26.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.311+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9dfb70 con 0x7faeb8007030 2023-10-05T19:40:26.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.315+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(95 notify.5 [watch watch cookie 94309623647712] v28'124 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faec000a5a0 con 0x7faeb8007030 2023-10-05T19:40:26.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.315+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(96 notify.6 [create] v28'123 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faec000a5a0 con 0x7faeb8007030 2023-10-05T19:40:26.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.315+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94309623656096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9dfb70 con 0x7faeb8007030 2023-10-05T19:40:26.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.315+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9e0410 con 0x7faeb8007030 2023-10-05T19:40:26.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.319+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(97 notify.6 [watch watch cookie 94309623656096] v28'124 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faec000a5a0 con 0x7faeb8007030 2023-10-05T19:40:26.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.319+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(98 notify.7 [create] v28'123 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faec000a5a0 con 0x7faeb8007030 2023-10-05T19:40:26.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.319+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94309623658672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9e01f0 con 0x7faeb8007030 2023-10-05T19:40:26.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.319+0000 7faec9ff6e40 20 add_watcher() i=0 2023-10-05T19:40:26.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.319+0000 7faec9ff6e40 20 add_watcher() i=1 2023-10-05T19:40:26.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.319+0000 7faec9ff6e40 20 add_watcher() i=2 2023-10-05T19:40:26.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.319+0000 7faec9ff6e40 20 add_watcher() i=3 2023-10-05T19:40:26.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.319+0000 7faec9ff6e40 20 add_watcher() i=4 2023-10-05T19:40:26.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.319+0000 7faec9ff6e40 20 add_watcher() i=5 2023-10-05T19:40:26.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.319+0000 7faec9ff6e40 20 add_watcher() i=6 2023-10-05T19:40:26.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.319+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94309623658672] v28'124 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faec000a5a0 con 0x7faeb8007030 2023-10-05T19:40:26.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.319+0000 7faec9ff6e40 20 add_watcher() i=7 2023-10-05T19:40:26.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.319+0000 7faec9ff6e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:26.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.319+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9e3710 con 0x55c62b90fbc0 2023-10-05T19:40:26.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.323+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7faec000a5a0 con 0x55c62b90fbc0 2023-10-05T19:40:26.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.323+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94309623660240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9e0bd0 con 0x55c62b90fbc0 2023-10-05T19:40:26.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.323+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94309623660240] v28'120 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7faec000a5a0 con 0x55c62b90fbc0 2023-10-05T19:40:26.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.323+0000 7fae997e2700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:26.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.323+0000 7faec9ff6e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:26.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.323+0000 7faec9ff6e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:26.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.323+0000 7faec9ff6e40 1 Processor -- start 2023-10-05T19:40:26.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.323+0000 7faec9ff6e40 1 -- start start 2023-10-05T19:40:26.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.323+0000 7faec9ff6e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c62b9eddd0 0x55c62b9ee1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:26.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.323+0000 7faec9ff6e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c62b9ee6e0 con 0x55c62b9eddd0 2023-10-05T19:40:26.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.327+0000 7faec732a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c62b9eddd0 0x55c62b9ee1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:26.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.327+0000 7faec732a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c62b9eddd0 0x55c62b9ee1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58088/0 (socket says 172.21.15.12:58088) 2023-10-05T19:40:26.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.327+0000 7faec732a700 1 -- 172.21.15.12:0/730101501 learned_addr learned my addr 172.21.15.12:0/730101501 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:26.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.327+0000 7faec732a700 1 -- 172.21.15.12:0/730101501 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c62b9ee820 con 0x55c62b9eddd0 2023-10-05T19:40:26.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.327+0000 7faec732a700 1 --2- 172.21.15.12:0/730101501 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c62b9eddd0 0x55c62b9ee1a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x55c62b9cb340 tx=0x7faeb001e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98b6772e83bda721 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:26.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.327+0000 7fae98fe1700 1 -- 172.21.15.12:0/730101501 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faeb0013040 con 0x55c62b9eddd0 2023-10-05T19:40:26.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.327+0000 7fae98fe1700 1 -- 172.21.15.12:0/730101501 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb0022040 con 0x55c62b9eddd0 2023-10-05T19:40:26.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.327+0000 7fae98fe1700 1 -- 172.21.15.12:0/730101501 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faeb000ee30 con 0x55c62b9eddd0 2023-10-05T19:40:26.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.327+0000 7faec9ff6e40 1 -- 172.21.15.12:0/730101501 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c62b9eddd0 msgr2=0x55c62b9ee1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:26.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.327+0000 7faec9ff6e40 1 --2- 172.21.15.12:0/730101501 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c62b9eddd0 0x55c62b9ee1a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x55c62b9cb340 tx=0x7faeb001e040 comp rx=0 tx=0).stop 2023-10-05T19:40:26.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.327+0000 7faec9ff6e40 1 -- 172.21.15.12:0/730101501 shutdown_connections 2023-10-05T19:40:26.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.327+0000 7faec9ff6e40 1 --2- 172.21.15.12:0/730101501 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c62b9eddd0 0x55c62b9ee1a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:26.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.327+0000 7faec9ff6e40 1 -- 172.21.15.12:0/730101501 >> 172.21.15.12:0/730101501 conn(0x55c62b9e9190 msgr2=0x55c62b9e72e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:26.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.327+0000 7faec9ff6e40 1 -- 172.21.15.12:0/730101501 shutdown_connections 2023-10-05T19:40:26.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.327+0000 7faec9ff6e40 1 -- 172.21.15.12:0/730101501 wait complete. 2023-10-05T19:40:26.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.327+0000 7faec9ff6e40 1 Processor -- start 2023-10-05T19:40:26.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.327+0000 7faec9ff6e40 1 -- start start 2023-10-05T19:40:26.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.327+0000 7faec9ff6e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c62b9eddd0 0x55c62b9f3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:26.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.327+0000 7faec9ff6e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c62b9f4980 con 0x55c62b9eddd0 2023-10-05T19:40:26.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.331+0000 7faec732a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c62b9eddd0 0x55c62b9f3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:26.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.331+0000 7faec732a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c62b9eddd0 0x55c62b9f3980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58092/0 (socket says 172.21.15.12:58092) 2023-10-05T19:40:26.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.331+0000 7faec732a700 1 -- 172.21.15.12:0/1355849037 learned_addr learned my addr 172.21.15.12:0/1355849037 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:26.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.331+0000 7faec732a700 1 -- 172.21.15.12:0/1355849037 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb0034040 con 0x55c62b9eddd0 2023-10-05T19:40:26.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.331+0000 7faec732a700 1 --2- 172.21.15.12:0/1355849037 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c62b9eddd0 0x55c62b9f3980 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7faeb000bed0 tx=0x7faeb003d030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:26.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.331+0000 7fae12ffd700 1 -- 172.21.15.12:0/1355849037 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faeb002c640 con 0x55c62b9eddd0 2023-10-05T19:40:26.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.331+0000 7fae12ffd700 1 -- 172.21.15.12:0/1355849037 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb002c7c0 con 0x55c62b9eddd0 2023-10-05T19:40:26.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.331+0000 7faec9ff6e40 1 -- 172.21.15.12:0/1355849037 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c62b9f3f80 con 0x55c62b9eddd0 2023-10-05T19:40:26.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.331+0000 7fae12ffd700 1 -- 172.21.15.12:0/1355849037 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faeb002c970 con 0x55c62b9eddd0 2023-10-05T19:40:26.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.331+0000 7faec9ff6e40 1 -- 172.21.15.12:0/1355849037 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c62b9f43c0 con 0x55c62b9eddd0 2023-10-05T19:40:26.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.331+0000 7fae12ffd700 1 -- 172.21.15.12:0/1355849037 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7faeb004d070 con 0x55c62b9eddd0 2023-10-05T19:40:26.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.331+0000 7fae12ffd700 1 --2- 172.21.15.12:0/1355849037 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fae08030c00 0x7fae080330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:26.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.331+0000 7fae12ffd700 1 -- 172.21.15.12:0/1355849037 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7faeb0050b20 con 0x55c62b9eddd0 2023-10-05T19:40:26.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.335+0000 7faec832c700 1 --2- 172.21.15.12:0/1355849037 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fae08030c00 0x7fae080330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:26.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.335+0000 7faec9ff6e40 5 note: GC not initialized 2023-10-05T19:40:26.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.335+0000 7faec832c700 1 --2- 172.21.15.12:0/1355849037 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fae08030c00 0x7fae080330b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x55c62b9e0420 tx=0x7faec000ae90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:26.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.335+0000 7fae07fff700 20 reqs_thread_entry: start 2023-10-05T19:40:26.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.335+0000 7fae05ffb700 20 reqs_thread_entry: start 2023-10-05T19:40:26.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.335+0000 7faec9ff6e40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:26.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.335+0000 7faec9ff6e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:26.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.335+0000 7fadf3fff700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:56 2023 2023-10-05T19:40:26.346 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:26.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.335+0000 7faec9ff6e40 20 get_system_obj_state: rctx=0x7fff18e292c8 obj=default.rgw.meta:users.uid:foo state=0x55c62b8730b0 s->prefetch_data=0 2023-10-05T19:40:26.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.335+0000 7faec9ff6e40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2023-10-05T19:40:26.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.339+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62ba5c2d0 con 0x55c62b90fbc0 2023-10-05T19:40:26.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.339+0000 7fadf37fe700 20 reqs_thread_entry: start 2023-10-05T19:40:26.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.339+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(102 foo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7faec000a5a0 con 0x55c62b90fbc0 2023-10-05T19:40:26.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.339+0000 7faec9ff6e40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x16 2023-10-05T19:40:26.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.339+0000 7faec9ff6e40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:26.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.339+0000 7faec9ff6e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:26.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.339+0000 7faec9ff6e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:26.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.339+0000 7faec9ff6e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:26.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.339+0000 7faec9ff6e40 10 cache get: name=default.rgw.meta+users.uid+foo : type miss (requested=0x13, cached=0x16) 2023-10-05T19:40:26.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.339+0000 7faec9ff6e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.339+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62ba5ed60 con 0x55c62b90fbc0 2023-10-05T19:40:26.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.339+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(103 foo [call,call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv11 ondisk = 0) v8 ==== 273+0+480 (crc 0 0 0) 0x7faec000a5a0 con 0x55c62b90fbc0 2023-10-05T19:40:26.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.339+0000 7faec9ff6e40 20 rados_obj.operate() r=0 bl.length=357 2023-10-05T19:40:26.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.339+0000 7faec9ff6e40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2023-10-05T19:40:26.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.339+0000 7faec9ff6e40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:26.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.339+0000 7faec9ff6e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:26.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.339+0000 7faec9ff6e40 20 get_system_obj_state: rctx=0x7fff18e28670 obj=default.rgw.meta:root:myfoo state=0x55c62b8730b0 s->prefetch_data=0 2023-10-05T19:40:26.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.339+0000 7faec9ff6e40 10 cache get: name=default.rgw.meta+root+myfoo : miss 2023-10-05T19:40:26.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.339+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:104 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62ba5e570 con 0x55c62b9cfd50 2023-10-05T19:40:26.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.339+0000 7faec7b2b700 1 -- 172.21.15.12:0/4264792383 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(104 myfoo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv21 ondisk = 0) v8 ==== 233+0+139 (crc 0 0 0) 0x7faeb8012ab0 con 0x55c62b9cfd50 2023-10-05T19:40:26.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.339+0000 7faec9ff6e40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x16 2023-10-05T19:40:26.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.339+0000 7faec9ff6e40 10 adding default.rgw.meta+root+myfoo to cache LRU end 2023-10-05T19:40:26.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.339+0000 7faec9ff6e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:26.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.339+0000 7faec9ff6e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:26.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.339+0000 7faec9ff6e40 10 cache get: name=default.rgw.meta+root+myfoo : type miss (requested=0x11, cached=0x16) 2023-10-05T19:40:26.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.339+0000 7faec9ff6e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.339+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:105 8.9 8:91b71841:root::myfoo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62ba60b10 con 0x55c62b9cfd50 2023-10-05T19:40:26.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.339+0000 7faec7b2b700 1 -- 172.21.15.12:0/4264792383 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(105 myfoo [call,call out=48b,read 0~165 out=165b] v0'0 uv21 ondisk = 0) v8 ==== 233+0+213 (crc 0 0 0) 0x7faeb8012ab0 con 0x55c62b9cfd50 2023-10-05T19:40:26.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.339+0000 7faec9ff6e40 20 rados_obj.operate() r=0 bl.length=165 2023-10-05T19:40:26.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.339+0000 7faec9ff6e40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2023-10-05T19:40:26.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.339+0000 7faec9ff6e40 10 moving default.rgw.meta+root+myfoo to cache LRU end 2023-10-05T19:40:26.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.339+0000 7faec9ff6e40 20 get_system_obj_state: rctx=0x7fff18e28670 obj=default.rgw.meta:root:.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 state=0x55c62b8492b0 s->prefetch_data=0 2023-10-05T19:40:26.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.339+0000 7faec9ff6e40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 : miss 2023-10-05T19:40:26.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.339+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:106 8.e 8:70176bbd:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62ba60160 con 0x7faeb8007030 2023-10-05T19:40:26.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.343+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(106 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 [call out=48b,getxattrs out=218b,stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 290+0+282 (crc 0 0 0) 0x7faec000a5a0 con 0x7faeb8007030 2023-10-05T19:40:26.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.343+0000 7faec9ff6e40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 info.flags=0x16 2023-10-05T19:40:26.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.343+0000 7faec9ff6e40 10 adding default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 to cache LRU end 2023-10-05T19:40:26.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.343+0000 7faec9ff6e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:26.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.343+0000 7faec9ff6e40 10 updating xattr: name=user.rgw.acl bl.length()=123 2023-10-05T19:40:26.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.343+0000 7faec9ff6e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:26.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.343+0000 7faec9ff6e40 20 Read xattr: user.rgw.acl 2023-10-05T19:40:26.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.343+0000 7faec9ff6e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:26.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.343+0000 7faec9ff6e40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 : type miss (requested=0x13, cached=0x16) 2023-10-05T19:40:26.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.343+0000 7faec9ff6e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.343+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:107 8.e 8:70176bbd:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62ba62fb0 con 0x7faeb8007030 2023-10-05T19:40:26.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.343+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(107 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 [call,call out=48b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv3 ondisk = 0) v8 ==== 332+0+618 (crc 0 0 0) 0x7faec000a5a0 con 0x7faeb8007030 2023-10-05T19:40:26.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.343+0000 7faec9ff6e40 20 rados_obj.operate() r=0 bl.length=352 2023-10-05T19:40:26.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.343+0000 7faec9ff6e40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 info.flags=0x13 2023-10-05T19:40:26.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.343+0000 7faec9ff6e40 10 moving default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 to cache LRU end 2023-10-05T19:40:26.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.343+0000 7faec9ff6e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:26.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.343+0000 7faec9ff6e40 10 updating xattr: name=user.rgw.acl bl.length()=123 2023-10-05T19:40:26.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.343+0000 7faec9ff6e40 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 2023-10-05T19:40:26.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.343+0000 7faec9ff6e40 20 get_system_obj_state: rctx=0x7fff18e28608 obj=default.rgw.meta:users.uid:foo state=0x55c62b8492b0 s->prefetch_data=0 2023-10-05T19:40:26.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.343+0000 7faec9ff6e40 10 cache get: name=default.rgw.meta+users.uid+foo : type miss (requested=0x16, cached=0x13) 2023-10-05T19:40:26.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.343+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:108 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62ba635d0 con 0x55c62b90fbc0 2023-10-05T19:40:26.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.343+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(108 foo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7faec000a5a0 con 0x55c62b90fbc0 2023-10-05T19:40:26.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.343+0000 7faec9ff6e40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x16 2023-10-05T19:40:26.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.343+0000 7faec9ff6e40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:26.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.343+0000 7faec9ff6e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:26.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.343+0000 7faec9ff6e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:26.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.343+0000 7faec9ff6e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:26.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.343+0000 7faec9ff6e40 10 cache get: name=default.rgw.meta+users.uid+foo : hit (requested=0x13, cached=0x17) 2023-10-05T19:40:26.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.343+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:109 8.1 8:810902b1:users.uid::fud.buckets:head [call user.remove_bucket in=58b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62ba655f0 con 0x7faeb8007030 2023-10-05T19:40:26.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.347+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(109 fud.buckets [call] v28'3 uv2 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x7faec000a5a0 con 0x7faeb8007030 2023-10-05T19:40:26.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.347+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:110 8.e 8:70176bbd:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~352 in=352b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62ba6b540 con 0x7faeb8007030 2023-10-05T19:40:26.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.351+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.0 v2:172.21.15.8:6800/27563 30 ==== osd_op_reply(110 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 [delete,create,call,call,writefull 0~352,setxattr (123)] v28'4 uv4 ondisk = 0) v8 ==== 416+0+0 (crc 0 0 0) 0x7faec000a5a0 con 0x7faeb8007030 2023-10-05T19:40:26.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.351+0000 7faec9ff6e40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 info.flags=0x17 2023-10-05T19:40:26.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.351+0000 7faec9ff6e40 10 moving default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 to cache LRU end 2023-10-05T19:40:26.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.351+0000 7faec9ff6e40 10 updating xattr: name=user.rgw.acl bl.length()=123 2023-10-05T19:40:26.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.351+0000 7faec9ff6e40 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1, ofs0, ns] 2023-10-05T19:40:26.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.351+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:111 7.e 7:7759931f:::notify.3:head [notify cookie 94309624231088 in=737b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62ba68f50 con 0x55c62b9cfd50 2023-10-05T19:40:26.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.351+0000 7faec7b2b700 1 -- 172.21.15.12:0/4264792383 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== watch-notify(notify (1) cookie 94309623642128 notify 120259084302 ret 0) v3 ==== 767+0+0 (crc 0 0 0) 0x7faeb804e040 con 0x55c62b9cfd50 2023-10-05T19:40:26.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.351+0000 7faec5b27700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084302 cookie 94309623642128 notifier 4471 bl.length()=725 2023-10-05T19:40:26.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.351+0000 7faec5b27700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 info.flags=0x17 2023-10-05T19:40:26.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.351+0000 7faec5b27700 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 to cache LRU end 2023-10-05T19:40:26.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.351+0000 7faec5b27700 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2023-10-05T19:40:26.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.351+0000 7faec7b2b700 1 -- 172.21.15.12:0/4264792383 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(111 notify.3 [notify cookie 94309624231088 out=8b] v0'0 uv123 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7faeb8012ab0 con 0x55c62b9cfd50 2023-10-05T19:40:26.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.351+0000 7faec5b27700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:112 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7faeac003d90 con 0x55c62b9cfd50 2023-10-05T19:40:26.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.351+0000 7faec7b2b700 1 -- 172.21.15.12:0/4264792383 <== osd.2 v2:172.21.15.12:6800/29934 23 ==== osd_op_reply(112 notify.3 [notify-ack] v0'0 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faeb8012ab0 con 0x55c62b9cfd50 2023-10-05T19:40:26.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.351+0000 7faec7b2b700 1 -- 172.21.15.12:0/4264792383 <== osd.2 v2:172.21.15.12:6800/29934 24 ==== watch-notify(notify_complete (2) cookie 94309624231088 notify 120259084302 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55c62b9c27a0 con 0x55c62b9cfd50 2023-10-05T19:40:26.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.351+0000 7faec9ff6e40 20 handle_bi_update(): bucket=:myfoo[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1]): orig_sources= new_sources= 2023-10-05T19:40:26.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.351+0000 7faec9ff6e40 20 handle_bi_update(): bucket=:myfoo[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1]): potential sources added=[] removed=[] 2023-10-05T19:40:26.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.351+0000 7faec9ff6e40 20 handle_bi_update(): bucket=:myfoo[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1]): orig_dests= new_dests= 2023-10-05T19:40:26.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.351+0000 7faec9ff6e40 20 handle_bi_update(): bucket=:myfoo[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1]): potential dests added=[] removed=[] 2023-10-05T19:40:26.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.351+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:113 8.9 8:96d98480:users.uid::foo.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62ba6b470 con 0x55c62b9cfd50 2023-10-05T19:40:26.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.355+0000 7faec7b2b700 1 -- 172.21.15.12:0/4264792383 <== osd.2 v2:172.21.15.12:6800/29934 25 ==== osd_op_reply(113 foo.buckets [call] v28'22 uv22 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x7faeb8012ab0 con 0x55c62b9cfd50 2023-10-05T19:40:26.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.355+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:114 8.9 8:91b71841:root::myfoo:head [delete,create,call version.inc in=38b,writefull 0~165 in=165b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62ba6b470 con 0x55c62b9cfd50 2023-10-05T19:40:26.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.359+0000 7faec7b2b700 1 -- 172.21.15.12:0/4264792383 <== osd.2 v2:172.21.15.12:6800/29934 26 ==== osd_op_reply(114 myfoo [delete,create,call,writefull 0~165] v28'23 uv23 ondisk = 0) v8 ==== 275+0+0 (crc 0 0 0) 0x7faeb8012ab0 con 0x55c62b9cfd50 2023-10-05T19:40:26.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.359+0000 7faec9ff6e40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2023-10-05T19:40:26.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.359+0000 7faec9ff6e40 10 moving default.rgw.meta+root+myfoo to cache LRU end 2023-10-05T19:40:26.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.359+0000 7faec9ff6e40 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2023-10-05T19:40:26.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.359+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:115 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94309624231024 in=326b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62ba6a300 con 0x7faeb8007030 2023-10-05T19:40:26.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.359+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.0 v2:172.21.15.8:6800/27563 31 ==== watch-notify(notify (1) cookie 94309623656096 notify 120259084305 ret 0) v3 ==== 356+0+0 (crc 0 0 0) 0x7faec004a040 con 0x7faeb8007030 2023-10-05T19:40:26.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.359+0000 7faec5b27700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084305 cookie 94309623656096 notifier 4471 bl.length()=314 2023-10-05T19:40:26.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.359+0000 7faec5b27700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2023-10-05T19:40:26.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.359+0000 7faec5b27700 10 rgw watcher librados: moving default.rgw.meta+root+myfoo to cache LRU end 2023-10-05T19:40:26.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.359+0000 7faec5b27700 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:116 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7faeac003d90 con 0x7faeb8007030 2023-10-05T19:40:26.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.359+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.0 v2:172.21.15.8:6800/27563 32 ==== osd_op_reply(115 notify.6 [notify cookie 94309624231024 out=8b] v0'0 uv123 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7faec000a5a0 con 0x7faeb8007030 2023-10-05T19:40:26.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.359+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.0 v2:172.21.15.8:6800/27563 33 ==== osd_op_reply(116 notify.6 [notify-ack] v0'0 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faec000a5a0 con 0x7faeb8007030 2023-10-05T19:40:26.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.363+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.0 v2:172.21.15.8:6800/27563 34 ==== watch-notify(notify_complete (2) cookie 94309624231024 notify 120259084305 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7faec002f040 con 0x7faeb8007030 2023-10-05T19:40:26.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.363+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:117 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94309623637504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9b1890 con 0x55c62b9cfd50 2023-10-05T19:40:26.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.363+0000 7faec7b2b700 1 -- 172.21.15.12:0/4264792383 <== osd.2 v2:172.21.15.12:6800/29934 27 ==== osd_op_reply(117 notify.0 [watch unwatch cookie 94309623637504] v28'125 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faeb8012ab0 con 0x55c62b9cfd50 2023-10-05T19:40:26.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.363+0000 7faec9ff6e40 20 remove_watcher() i=0 2023-10-05T19:40:26.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.363+0000 7faec9ff6e40 2 removed watcher, disabling cache 2023-10-05T19:40:26.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.363+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:118 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94309623640816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9b1890 con 0x55c62b90fbc0 2023-10-05T19:40:26.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.367+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(118 notify.1 [watch unwatch cookie 94309623640816] v28'125 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faec000a5a0 con 0x55c62b90fbc0 2023-10-05T19:40:26.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.367+0000 7faec9ff6e40 20 remove_watcher() i=1 2023-10-05T19:40:26.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.367+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:119 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94309623644768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9b1890 con 0x55c62b9d44f0 2023-10-05T19:40:26.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.371+0000 7faec732a700 1 -- 172.21.15.12:0/4264792383 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(119 notify.2 [watch unwatch cookie 94309623644768] v28'125 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faeb0006070 con 0x55c62b9d44f0 2023-10-05T19:40:26.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.371+0000 7faec9ff6e40 20 remove_watcher() i=2 2023-10-05T19:40:26.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.371+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:120 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94309623642128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9b1890 con 0x55c62b9cfd50 2023-10-05T19:40:26.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.375+0000 7faec7b2b700 1 -- 172.21.15.12:0/4264792383 <== osd.2 v2:172.21.15.12:6800/29934 28 ==== osd_op_reply(120 notify.3 [watch unwatch cookie 94309623642128] v28'125 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faeb8012ab0 con 0x55c62b9cfd50 2023-10-05T19:40:26.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.375+0000 7faec9ff6e40 20 remove_watcher() i=3 2023-10-05T19:40:26.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.375+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:121 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94309623652496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9b1890 con 0x55c62b90fbc0 2023-10-05T19:40:26.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.375+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(121 notify.4 [watch unwatch cookie 94309623652496] v28'125 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faec000a5a0 con 0x55c62b90fbc0 2023-10-05T19:40:26.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.375+0000 7faec9ff6e40 20 remove_watcher() i=4 2023-10-05T19:40:26.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.375+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:122 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94309623647712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9b1890 con 0x7faeb8007030 2023-10-05T19:40:26.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.379+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.0 v2:172.21.15.8:6800/27563 35 ==== osd_op_reply(122 notify.5 [watch unwatch cookie 94309623647712] v28'125 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faec000a5a0 con 0x7faeb8007030 2023-10-05T19:40:26.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.379+0000 7faec9ff6e40 20 remove_watcher() i=5 2023-10-05T19:40:26.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.379+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:123 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94309623656096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9b1890 con 0x7faeb8007030 2023-10-05T19:40:26.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.383+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.0 v2:172.21.15.8:6800/27563 36 ==== osd_op_reply(123 notify.6 [watch unwatch cookie 94309623656096] v28'125 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faec000a5a0 con 0x7faeb8007030 2023-10-05T19:40:26.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.383+0000 7faec9ff6e40 20 remove_watcher() i=6 2023-10-05T19:40:26.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.383+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:124 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94309623658672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9b1890 con 0x7faeb8007030 2023-10-05T19:40:26.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.387+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.0 v2:172.21.15.8:6800/27563 37 ==== osd_op_reply(124 notify.7 [watch unwatch cookie 94309623658672] v28'125 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faec000a5a0 con 0x7faeb8007030 2023-10-05T19:40:26.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.387+0000 7faec9ff6e40 20 remove_watcher() i=7 2023-10-05T19:40:26.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.387+0000 7fae997e2700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:26.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.387+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:125 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94309623660240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55c62b9b1890 con 0x55c62b90fbc0 2023-10-05T19:40:26.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.391+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== osd_op_reply(125 data_loggenerations_metadata [watch unwatch cookie 94309623660240] v28'121 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7faec000a5a0 con 0x55c62b90fbc0 2023-10-05T19:40:26.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.391+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7faeb8007030 msgr2=0x7faeb8037640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:26.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.391+0000 7faec9ff6e40 1 --2- 172.21.15.12:0/4264792383 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7faeb8007030 0x7faeb8037640 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:26.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.391+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55c62b90fbc0 msgr2=0x55c62b9cd8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:26.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.391+0000 7faec9ff6e40 1 --2- 172.21.15.12:0/4264792383 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55c62b90fbc0 0x55c62b9cd8b0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:26.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.391+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55c62b9cfd50 msgr2=0x55c62b9d2130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:26.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.391+0000 7faec9ff6e40 1 --2- 172.21.15.12:0/4264792383 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55c62b9cfd50 0x55c62b9d2130 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:26.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.391+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55c62b9d44f0 msgr2=0x55c62b9d6900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:26.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.391+0000 7faec9ff6e40 1 --2- 172.21.15.12:0/4264792383 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55c62b9d44f0 0x55c62b9d6900 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:26.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.391+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7faea8030c50 msgr2=0x7faea8033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:26.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.391+0000 7faec9ff6e40 1 --2- 172.21.15.12:0/4264792383 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7faea8030c50 0x7faea8033100 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7faeb0001e90 tx=0x7faeb00013b0 comp rx=0 tx=0).stop 2023-10-05T19:40:26.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.391+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c62b90e6e0 msgr2=0x55c62b9c1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:26.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.391+0000 7faec832c700 1 -- 172.21.15.12:0/4264792383 reap_dead start 2023-10-05T19:40:26.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.391+0000 7faec9ff6e40 1 --2- 172.21.15.12:0/4264792383 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c62b90e6e0 0x55c62b9c1dc0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7faeb800c010 tx=0x7faeb801c000 comp rx=0 tx=0).stop 2023-10-05T19:40:26.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.391+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 shutdown_connections 2023-10-05T19:40:26.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.391+0000 7faec9ff6e40 1 --2- 172.21.15.12:0/4264792383 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c62b90e6e0 0x55c62b9c1dc0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:26.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.391+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 >> 172.21.15.12:0/4264792383 conn(0x55c62b9957f0 msgr2=0x55c62b9b5700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:26.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.391+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 shutdown_connections 2023-10-05T19:40:26.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.391+0000 7faec9ff6e40 1 -- 172.21.15.12:0/4264792383 wait complete. 2023-10-05T19:40:26.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.391+0000 7faec9ff6e40 1 -- 172.21.15.12:0/1355849037 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fae08030c00 msgr2=0x7fae080330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:26.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.391+0000 7faec9ff6e40 1 --2- 172.21.15.12:0/1355849037 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fae08030c00 0x7fae080330b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x55c62b9e0420 tx=0x7faec000ae90 comp rx=0 tx=0).stop 2023-10-05T19:40:26.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.391+0000 7faec9ff6e40 1 -- 172.21.15.12:0/1355849037 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c62b9eddd0 msgr2=0x55c62b9f3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:26.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.391+0000 7faec9ff6e40 1 --2- 172.21.15.12:0/1355849037 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c62b9eddd0 0x55c62b9f3980 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7faeb000bed0 tx=0x7faeb003d030 comp rx=0 tx=0).stop 2023-10-05T19:40:26.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.391+0000 7faec9ff6e40 1 -- 172.21.15.12:0/1355849037 shutdown_connections 2023-10-05T19:40:26.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.391+0000 7faec9ff6e40 1 --2- 172.21.15.12:0/1355849037 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fae08030c00 0x7fae080330b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:26.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.391+0000 7faec9ff6e40 1 --2- 172.21.15.12:0/1355849037 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c62b9eddd0 0x55c62b9f3980 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:26.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.391+0000 7faec9ff6e40 1 -- 172.21.15.12:0/1355849037 >> 172.21.15.12:0/1355849037 conn(0x55c62b9e9190 msgr2=0x55c62b9e7230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:26.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.391+0000 7faec9ff6e40 1 -- 172.21.15.12:0/1355849037 shutdown_connections 2023-10-05T19:40:26.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.391+0000 7faec9ff6e40 1 -- 172.21.15.12:0/1355849037 wait complete. 2023-10-05T19:40:26.408 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'fud'] 2023-10-05T19:40:26.408 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'fud'] 2023-10-05T19:40:26.408 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid fud 2023-10-05T19:40:26.447 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:26.448 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:26.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.439+0000 7f545fad0e40 1 Processor -- start 2023-10-05T19:40:26.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.439+0000 7f545fad0e40 1 -- start start 2023-10-05T19:40:26.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.439+0000 7f545fad0e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55b041c4d910 0x55b041ce1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:26.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.439+0000 7f545fad0e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55b041c25bd0 con 0x55b041c4ec20 2023-10-05T19:40:26.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.439+0000 7f545fad0e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x55b041be2ee0 con 0x55b041c4d910 2023-10-05T19:40:26.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.439+0000 7f545ce04700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55b041c4d910 0x55b041ce1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:26.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.439+0000 7f545d605700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x55b041c4ec20 0x55b041c4d200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:35580/0 (socket says 172.21.15.12:35580) 2023-10-05T19:40:26.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.439+0000 7f545ce04700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55b041c4d910 0x55b041ce1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58104/0 (socket says 172.21.15.12:58104) 2023-10-05T19:40:26.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.439+0000 7f545d605700 1 -- 172.21.15.12:0/1913466051 learned_addr learned my addr 172.21.15.12:0/1913466051 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:26.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.443+0000 7f545ce04700 1 -- 172.21.15.12:0/1913466051 >> v1:172.21.15.8:6789/0 conn(0x55b041c4ec20 legacy=0x55b041c4d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:26.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.443+0000 7f5457fff700 1 -- 172.21.15.12:0/1913466051 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 385317320 0 0) 0x55b041c25bd0 con 0x55b041c4ec20 2023-10-05T19:40:26.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.443+0000 7f545ce04700 1 -- 172.21.15.12:0/1913466051 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b041c4f0b0 con 0x55b041c4d910 2023-10-05T19:40:26.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.443+0000 7f545ce04700 1 --2- 172.21.15.12:0/1913466051 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b041c4d910 0x55b041ce1f60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f544800a5d0 tx=0x7f5448005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d283e717d9657d5d server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:26.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.443+0000 7f5457fff700 1 -- 172.21.15.12:0/1913466051 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f54480086a0 con 0x55b041c4d910 2023-10-05T19:40:26.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.443+0000 7f5457fff700 1 -- 172.21.15.12:0/1913466051 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5448008800 con 0x55b041c4d910 2023-10-05T19:40:26.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.443+0000 7f5457fff700 1 -- 172.21.15.12:0/1913466051 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5448008ad0 con 0x55b041c4d910 2023-10-05T19:40:26.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.443+0000 7f545fad0e40 1 -- 172.21.15.12:0/1913466051 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b041c4d910 msgr2=0x55b041ce1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:26.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.443+0000 7f545fad0e40 1 --2- 172.21.15.12:0/1913466051 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b041c4d910 0x55b041ce1f60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f544800a5d0 tx=0x7f5448005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:26.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.443+0000 7f545fad0e40 1 -- 172.21.15.12:0/1913466051 shutdown_connections 2023-10-05T19:40:26.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.443+0000 7f545fad0e40 1 --2- 172.21.15.12:0/1913466051 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b041c4d910 0x55b041ce1f60 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:26.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.443+0000 7f545fad0e40 1 -- 172.21.15.12:0/1913466051 >> 172.21.15.12:0/1913466051 conn(0x55b041cd4850 msgr2=0x55b041cd6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:26.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.443+0000 7f545fad0e40 1 -- 172.21.15.12:0/1913466051 shutdown_connections 2023-10-05T19:40:26.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.443+0000 7f545fad0e40 1 -- 172.21.15.12:0/1913466051 wait complete. 2023-10-05T19:40:26.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.443+0000 7f545fad0e40 1 Processor -- start 2023-10-05T19:40:26.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.443+0000 7f545fad0e40 1 -- start start 2023-10-05T19:40:26.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.443+0000 7f545fad0e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b041c4d910 0x55b041cf9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:26.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.443+0000 7f545fad0e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b041ba6e30 con 0x55b041c4d910 2023-10-05T19:40:26.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.443+0000 7f545d605700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b041c4d910 0x55b041cf9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:26.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.443+0000 7f545d605700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b041c4d910 0x55b041cf9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58112/0 (socket says 172.21.15.12:58112) 2023-10-05T19:40:26.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.443+0000 7f545d605700 1 -- 172.21.15.12:0/948073118 learned_addr learned my addr 172.21.15.12:0/948073118 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:26.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.447+0000 7f545d605700 1 -- 172.21.15.12:0/948073118 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5448005cf0 con 0x55b041c4d910 2023-10-05T19:40:26.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.447+0000 7f545d605700 1 --2- 172.21.15.12:0/948073118 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b041c4d910 0x55b041cf9f70 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5458001630 tx=0x7f54580069b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9826a09b3f5c6710 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:26.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.447+0000 7f5456ffd700 1 -- 172.21.15.12:0/948073118 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5458006320 con 0x55b041c4d910 2023-10-05T19:40:26.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.447+0000 7f5456ffd700 1 -- 172.21.15.12:0/948073118 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54580044c0 con 0x55b041c4d910 2023-10-05T19:40:26.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.447+0000 7f5456ffd700 1 -- 172.21.15.12:0/948073118 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5458004650 con 0x55b041c4d910 2023-10-05T19:40:26.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.447+0000 7f545fad0e40 1 -- 172.21.15.12:0/948073118 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b041c4d910 msgr2=0x55b041cf9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:26.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.447+0000 7f545fad0e40 1 --2- 172.21.15.12:0/948073118 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b041c4d910 0x55b041cf9f70 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5458001630 tx=0x7f54580069b0 comp rx=0 tx=0).stop 2023-10-05T19:40:26.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.447+0000 7f545fad0e40 1 -- 172.21.15.12:0/948073118 shutdown_connections 2023-10-05T19:40:26.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.447+0000 7f545fad0e40 1 --2- 172.21.15.12:0/948073118 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b041c4d910 0x55b041cf9f70 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:26.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.447+0000 7f545fad0e40 1 -- 172.21.15.12:0/948073118 >> 172.21.15.12:0/948073118 conn(0x55b041cd4850 msgr2=0x55b041cf5b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:26.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.447+0000 7f545fad0e40 1 -- 172.21.15.12:0/948073118 shutdown_connections 2023-10-05T19:40:26.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.447+0000 7f545fad0e40 1 -- 172.21.15.12:0/948073118 wait complete. 2023-10-05T19:40:26.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.447+0000 7f545fad0e40 1 Processor -- start 2023-10-05T19:40:26.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.447+0000 7f545fad0e40 1 -- start start 2023-10-05T19:40:26.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.447+0000 7f545fad0e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b041c4d910 0x55b041d00d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:26.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.447+0000 7f545fad0e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b041b79ed0 con 0x55b041c4d910 2023-10-05T19:40:26.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.451+0000 7f545d605700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b041c4d910 0x55b041d00d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:26.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.451+0000 7f545d605700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b041c4d910 0x55b041d00d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58114/0 (socket says 172.21.15.12:58114) 2023-10-05T19:40:26.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.451+0000 7f545d605700 1 -- 172.21.15.12:0/2747982595 learned_addr learned my addr 172.21.15.12:0/2747982595 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:26.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.451+0000 7f545d605700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5458006c90 con 0x55b041c4d910 2023-10-05T19:40:26.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.451+0000 7f545d605700 1 --2- 172.21.15.12:0/2747982595 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b041c4d910 0x55b041d00d00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f54580069f0 tx=0x7f5458014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:26.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.451+0000 7f54557fa700 1 -- 172.21.15.12:0/2747982595 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f54580137a0 con 0x55b041c4d910 2023-10-05T19:40:26.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.451+0000 7f54557fa700 1 -- 172.21.15.12:0/2747982595 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5458013920 con 0x55b041c4d910 2023-10-05T19:40:26.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.451+0000 7f54557fa700 1 -- 172.21.15.12:0/2747982595 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5458013c10 con 0x55b041c4d910 2023-10-05T19:40:26.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.451+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b041d01240 con 0x55b041c4d910 2023-10-05T19:40:26.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.451+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b041d01680 con 0x55b041c4d910 2023-10-05T19:40:26.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.451+0000 7f545fad0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.451+0000 7f54557fa700 1 -- 172.21.15.12:0/2747982595 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f545801a070 con 0x55b041c4d910 2023-10-05T19:40:26.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.455+0000 7f54557fa700 1 --2- 172.21.15.12:0/2747982595 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5440030c50 0x7f5440033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:26.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.455+0000 7f54557fa700 1 -- 172.21.15.12:0/2747982595 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f5458044e40 con 0x55b041c4d910 2023-10-05T19:40:26.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.455+0000 7f545ce04700 1 --2- 172.21.15.12:0/2747982595 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5440030c50 0x7f5440033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:26.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.455+0000 7f545fad0e40 1 --2- 172.21.15.12:0/2747982595 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55b041c4ec20 0x55b041d0c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:26.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.455+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d0cea0 con 0x55b041c4ec20 2023-10-05T19:40:26.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.455+0000 7f545de06700 1 --2- 172.21.15.12:0/2747982595 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55b041c4ec20 0x55b041d0c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:26.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.455+0000 7f545ce04700 1 --2- 172.21.15.12:0/2747982595 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5440030c50 0x7f5440033100 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5448005a90 tx=0x7f5448015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:26.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.455+0000 7f545de06700 1 --2- 172.21.15.12:0/2747982595 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55b041c4ec20 0x55b041d0c7f0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:26.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.455+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f545000a500 con 0x55b041c4ec20 2023-10-05T19:40:26.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.455+0000 7f545fad0e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:26.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.455+0000 7f545fad0e40 20 realm 2023-10-05T19:40:26.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.455+0000 7f545fad0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.455+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d09e50 con 0x55b041c4ec20 2023-10-05T19:40:26.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.455+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f545000a500 con 0x55b041c4ec20 2023-10-05T19:40:26.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.455+0000 7f545fad0e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:26.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.455+0000 7f545fad0e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:26.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.455+0000 7f545fad0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.455+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d0d980 con 0x55b041c4ec20 2023-10-05T19:40:26.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.455+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f545000a500 con 0x55b041c4ec20 2023-10-05T19:40:26.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.455+0000 7f545fad0e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:26.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.455+0000 7f545fad0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.455+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d0d590 con 0x55b041c4ec20 2023-10-05T19:40:26.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.459+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f545000a500 con 0x55b041c4ec20 2023-10-05T19:40:26.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.459+0000 7f545fad0e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:26.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.459+0000 7f545fad0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.459+0000 7f545fad0e40 1 --2- 172.21.15.12:0/2747982595 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55b041d0ed20 0x55b041d11100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:26.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.459+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d11640 con 0x55b041d0ed20 2023-10-05T19:40:26.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.459+0000 7f545d605700 1 --2- 172.21.15.12:0/2747982595 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55b041d0ed20 0x55b041d11100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:26.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.459+0000 7f545d605700 1 --2- 172.21.15.12:0/2747982595 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55b041d0ed20 0x55b041d11100 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:26.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.459+0000 7f545d605700 1 -- 172.21.15.12:0/2747982595 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f5458025020 con 0x55b041d0ed20 2023-10-05T19:40:26.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.459+0000 7f545fad0e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:26.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.459+0000 7f545fad0e40 20 searching for the correct realm 2023-10-05T19:40:26.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.459+0000 7f545fad0e40 1 --2- 172.21.15.12:0/2747982595 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55b041d134c0 0x55b041d158d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:26.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.459+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d15f80 con 0x55b041d134c0 2023-10-05T19:40:26.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.459+0000 7f545ce04700 1 --2- 172.21.15.12:0/2747982595 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55b041d134c0 0x55b041d158d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:26.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.459+0000 7f545ce04700 1 --2- 172.21.15.12:0/2747982595 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55b041d134c0 0x55b041d158d0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:26.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.463+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f5448008ae0 con 0x55b041d134c0 2023-10-05T19:40:26.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.463+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d15f80 con 0x55b041d0ed20 2023-10-05T19:40:26.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.463+0000 7f545d605700 1 -- 172.21.15.12:0/2747982595 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5458025020 con 0x55b041d0ed20 2023-10-05T19:40:26.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.463+0000 7f545d605700 1 --2- 172.21.15.12:0/2747982595 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f54580106e0 0x7f545801e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:26.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.463+0000 7f545d605700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d15f80 con 0x7f54580106e0 2023-10-05T19:40:26.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.463+0000 7f545de06700 1 --2- 172.21.15.12:0/2747982595 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f54580106e0 0x7f545801e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:26.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.463+0000 7f545de06700 1 --2- 172.21.15.12:0/2747982595 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f54580106e0 0x7f545801e590 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:26.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.463+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5450014040 con 0x7f54580106e0 2023-10-05T19:40:26.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.467+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d15f80 con 0x55b041c4ec20 2023-10-05T19:40:26.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.467+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5450014040 con 0x55b041c4ec20 2023-10-05T19:40:26.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.467+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d15f80 con 0x55b041c4ec20 2023-10-05T19:40:26.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.467+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5450014040 con 0x55b041c4ec20 2023-10-05T19:40:26.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.467+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d15f80 con 0x7f54580106e0 2023-10-05T19:40:26.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.467+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5450014040 con 0x7f54580106e0 2023-10-05T19:40:26.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.467+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d15f80 con 0x7f54580106e0 2023-10-05T19:40:26.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.467+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5450014040 con 0x7f54580106e0 2023-10-05T19:40:26.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.467+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d15f80 con 0x55b041d0ed20 2023-10-05T19:40:26.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.467+0000 7f545d605700 1 -- 172.21.15.12:0/2747982595 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5458025020 con 0x55b041d0ed20 2023-10-05T19:40:26.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.467+0000 7f545d605700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d15f80 con 0x55b041d134c0 2023-10-05T19:40:26.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.471+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5448008ae0 con 0x55b041d134c0 2023-10-05T19:40:26.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.471+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d15f80 con 0x55b041c4ec20 2023-10-05T19:40:26.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.471+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5450014040 con 0x55b041c4ec20 2023-10-05T19:40:26.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.471+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d15f80 con 0x7f54580106e0 2023-10-05T19:40:26.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.471+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5450014040 con 0x7f54580106e0 2023-10-05T19:40:26.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.471+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d15f80 con 0x55b041c4ec20 2023-10-05T19:40:26.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.471+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5450014040 con 0x55b041c4ec20 2023-10-05T19:40:26.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.471+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d15f80 con 0x55b041d0ed20 2023-10-05T19:40:26.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.471+0000 7f545d605700 1 -- 172.21.15.12:0/2747982595 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5458025020 con 0x55b041d0ed20 2023-10-05T19:40:26.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.475+0000 7f545d605700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d15f80 con 0x55b041d134c0 2023-10-05T19:40:26.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.475+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5448008ae0 con 0x55b041d134c0 2023-10-05T19:40:26.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.475+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d15f80 con 0x55b041d134c0 2023-10-05T19:40:26.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.475+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5448008ae0 con 0x55b041d134c0 2023-10-05T19:40:26.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.475+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d15f80 con 0x7f54580106e0 2023-10-05T19:40:26.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.475+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5450014040 con 0x7f54580106e0 2023-10-05T19:40:26.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.475+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d15f80 con 0x55b041d134c0 2023-10-05T19:40:26.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.475+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5448008ae0 con 0x55b041d134c0 2023-10-05T19:40:26.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.475+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d15f80 con 0x55b041d134c0 2023-10-05T19:40:26.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.479+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5448008ae0 con 0x55b041d134c0 2023-10-05T19:40:26.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.479+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d15f80 con 0x55b041c4ec20 2023-10-05T19:40:26.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.479+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5450014040 con 0x55b041c4ec20 2023-10-05T19:40:26.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.479+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d15f80 con 0x55b041c4ec20 2023-10-05T19:40:26.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.479+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5450014040 con 0x55b041c4ec20 2023-10-05T19:40:26.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.479+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d15f80 con 0x7f54580106e0 2023-10-05T19:40:26.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.479+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5450014040 con 0x7f54580106e0 2023-10-05T19:40:26.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.479+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d15f80 con 0x55b041d134c0 2023-10-05T19:40:26.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.479+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5448008ae0 con 0x55b041d134c0 2023-10-05T19:40:26.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.479+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d15f80 con 0x55b041d0ed20 2023-10-05T19:40:26.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.483+0000 7f545d605700 1 -- 172.21.15.12:0/2747982595 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5458025020 con 0x55b041d0ed20 2023-10-05T19:40:26.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.483+0000 7f545d605700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d15f80 con 0x55b041c4ec20 2023-10-05T19:40:26.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.483+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5450014040 con 0x55b041c4ec20 2023-10-05T19:40:26.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.483+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d15f80 con 0x7f54580106e0 2023-10-05T19:40:26.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.483+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5450014040 con 0x7f54580106e0 2023-10-05T19:40:26.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.483+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d15f80 con 0x55b041d134c0 2023-10-05T19:40:26.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.483+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5448008ae0 con 0x55b041d134c0 2023-10-05T19:40:26.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.483+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d15f80 con 0x7f54580106e0 2023-10-05T19:40:26.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.487+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5450014040 con 0x7f54580106e0 2023-10-05T19:40:26.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.487+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d15f80 con 0x55b041d0ed20 2023-10-05T19:40:26.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.487+0000 7f545d605700 1 -- 172.21.15.12:0/2747982595 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5458025020 con 0x55b041d0ed20 2023-10-05T19:40:26.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.487+0000 7f545d605700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d15f80 con 0x55b041d134c0 2023-10-05T19:40:26.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.487+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5448008ae0 con 0x55b041d134c0 2023-10-05T19:40:26.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.487+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d15f80 con 0x7f54580106e0 2023-10-05T19:40:26.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.487+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5450014040 con 0x7f54580106e0 2023-10-05T19:40:26.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.487+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d15f80 con 0x55b041c4ec20 2023-10-05T19:40:26.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.491+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f5450014040 con 0x55b041c4ec20 2023-10-05T19:40:26.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.491+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d15f80 con 0x55b041d0ed20 2023-10-05T19:40:26.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.491+0000 7f545d605700 1 -- 172.21.15.12:0/2747982595 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f5458025020 con 0x55b041d0ed20 2023-10-05T19:40:26.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.491+0000 7f545fad0e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:26.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.491+0000 7f545fad0e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:26.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.491+0000 7f545fad0e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:26.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.491+0000 7f545fad0e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:26.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.491+0000 7f545fad0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.491+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d12240 con 0x55b041c4ec20 2023-10-05T19:40:26.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.491+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5450014040 con 0x55b041c4ec20 2023-10-05T19:40:26.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.491+0000 7f545fad0e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:26.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.491+0000 7f545fad0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.491+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d11e70 con 0x55b041c4ec20 2023-10-05T19:40:26.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.491+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f5450014040 con 0x55b041c4ec20 2023-10-05T19:40:26.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.491+0000 7f545fad0e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:26.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.491+0000 7f545fad0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.491+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d163f0 con 0x55b041d134c0 2023-10-05T19:40:26.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.495+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f5448008ae0 con 0x55b041d134c0 2023-10-05T19:40:26.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.495+0000 7f545fad0e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:26.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.495+0000 7f545fad0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.495+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d167a0 con 0x55b041d134c0 2023-10-05T19:40:26.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.495+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f5448008ae0 con 0x55b041d134c0 2023-10-05T19:40:26.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.495+0000 7f545fad0e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:26.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.495+0000 7f545fad0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.495+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d16b50 con 0x55b041c4ec20 2023-10-05T19:40:26.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.495+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5450014040 con 0x55b041c4ec20 2023-10-05T19:40:26.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.495+0000 7f545fad0e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:26.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.495+0000 7f545fad0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.495+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d16f60 con 0x55b041c4ec20 2023-10-05T19:40:26.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.495+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f5450014040 con 0x55b041c4ec20 2023-10-05T19:40:26.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.495+0000 7f545fad0e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:26.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.495+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d16f30 con 0x55b041d134c0 2023-10-05T19:40:26.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.499+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f5448008ae0 con 0x55b041d134c0 2023-10-05T19:40:26.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.499+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d16f30 con 0x55b041d0ed20 2023-10-05T19:40:26.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.499+0000 7f545d605700 1 -- 172.21.15.12:0/2747982595 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5458025020 con 0x55b041d0ed20 2023-10-05T19:40:26.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.499+0000 7f545d605700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d16f30 con 0x7f54580106e0 2023-10-05T19:40:26.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.499+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5450014040 con 0x7f54580106e0 2023-10-05T19:40:26.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.499+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d16b50 con 0x55b041c4ec20 2023-10-05T19:40:26.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.503+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5450014040 con 0x55b041c4ec20 2023-10-05T19:40:26.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.503+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d16b50 con 0x55b041c4ec20 2023-10-05T19:40:26.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.503+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5450014040 con 0x55b041c4ec20 2023-10-05T19:40:26.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.503+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d16b50 con 0x7f54580106e0 2023-10-05T19:40:26.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.503+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5450014040 con 0x7f54580106e0 2023-10-05T19:40:26.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.503+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d16b50 con 0x7f54580106e0 2023-10-05T19:40:26.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.503+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5450014040 con 0x7f54580106e0 2023-10-05T19:40:26.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.503+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d16b50 con 0x55b041d0ed20 2023-10-05T19:40:26.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.503+0000 7f545d605700 1 -- 172.21.15.12:0/2747982595 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5458025020 con 0x55b041d0ed20 2023-10-05T19:40:26.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.503+0000 7f545d605700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d16b50 con 0x55b041d134c0 2023-10-05T19:40:26.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.507+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5448008ae0 con 0x55b041d134c0 2023-10-05T19:40:26.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.507+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d16b50 con 0x55b041c4ec20 2023-10-05T19:40:26.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.507+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5450014040 con 0x55b041c4ec20 2023-10-05T19:40:26.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.507+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d16b50 con 0x7f54580106e0 2023-10-05T19:40:26.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.507+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5450014040 con 0x7f54580106e0 2023-10-05T19:40:26.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.507+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d16b50 con 0x55b041c4ec20 2023-10-05T19:40:26.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.507+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5450014040 con 0x55b041c4ec20 2023-10-05T19:40:26.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.507+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d16b50 con 0x55b041d0ed20 2023-10-05T19:40:26.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.511+0000 7f545d605700 1 -- 172.21.15.12:0/2747982595 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5458025020 con 0x55b041d0ed20 2023-10-05T19:40:26.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.511+0000 7f545d605700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d16b50 con 0x55b041d134c0 2023-10-05T19:40:26.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.511+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5448008ae0 con 0x55b041d134c0 2023-10-05T19:40:26.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.511+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d16b50 con 0x55b041d134c0 2023-10-05T19:40:26.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.511+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5448008ae0 con 0x55b041d134c0 2023-10-05T19:40:26.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.511+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d16b50 con 0x7f54580106e0 2023-10-05T19:40:26.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.511+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5450014040 con 0x7f54580106e0 2023-10-05T19:40:26.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.511+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d16b50 con 0x55b041d134c0 2023-10-05T19:40:26.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.511+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5448008ae0 con 0x55b041d134c0 2023-10-05T19:40:26.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.511+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d16b50 con 0x55b041d134c0 2023-10-05T19:40:26.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.515+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5448008ae0 con 0x55b041d134c0 2023-10-05T19:40:26.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.515+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d16b50 con 0x55b041c4ec20 2023-10-05T19:40:26.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.515+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5450014040 con 0x55b041c4ec20 2023-10-05T19:40:26.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.515+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d16b50 con 0x55b041c4ec20 2023-10-05T19:40:26.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.515+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5450014040 con 0x55b041c4ec20 2023-10-05T19:40:26.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.515+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d16b50 con 0x7f54580106e0 2023-10-05T19:40:26.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.515+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5450014040 con 0x7f54580106e0 2023-10-05T19:40:26.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.515+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d16b50 con 0x55b041d134c0 2023-10-05T19:40:26.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.519+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5448008ae0 con 0x55b041d134c0 2023-10-05T19:40:26.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.519+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d16b50 con 0x55b041d0ed20 2023-10-05T19:40:26.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.519+0000 7f545d605700 1 -- 172.21.15.12:0/2747982595 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5458025020 con 0x55b041d0ed20 2023-10-05T19:40:26.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.519+0000 7f545d605700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d16b50 con 0x55b041c4ec20 2023-10-05T19:40:26.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.519+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5450014040 con 0x55b041c4ec20 2023-10-05T19:40:26.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.519+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d16b50 con 0x7f54580106e0 2023-10-05T19:40:26.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.519+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5450014040 con 0x7f54580106e0 2023-10-05T19:40:26.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.519+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d16b50 con 0x55b041d134c0 2023-10-05T19:40:26.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.523+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5448008ae0 con 0x55b041d134c0 2023-10-05T19:40:26.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.523+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d16b50 con 0x7f54580106e0 2023-10-05T19:40:26.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.523+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5450014040 con 0x7f54580106e0 2023-10-05T19:40:26.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.523+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d16b50 con 0x55b041d0ed20 2023-10-05T19:40:26.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.523+0000 7f545d605700 1 -- 172.21.15.12:0/2747982595 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5458025020 con 0x55b041d0ed20 2023-10-05T19:40:26.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.523+0000 7f545d605700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d16b50 con 0x55b041d134c0 2023-10-05T19:40:26.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.523+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5448008ae0 con 0x55b041d134c0 2023-10-05T19:40:26.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.523+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d16b50 con 0x7f54580106e0 2023-10-05T19:40:26.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.527+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5450014040 con 0x7f54580106e0 2023-10-05T19:40:26.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.527+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d16b50 con 0x55b041c4ec20 2023-10-05T19:40:26.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.527+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f5450014040 con 0x55b041c4ec20 2023-10-05T19:40:26.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.527+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d16b50 con 0x55b041d0ed20 2023-10-05T19:40:26.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.527+0000 7f545d605700 1 -- 172.21.15.12:0/2747982595 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f5458025020 con 0x55b041d0ed20 2023-10-05T19:40:26.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.527+0000 7f545fad0e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:26.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.527+0000 7f545fad0e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:26.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.527+0000 7f545fad0e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:26.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.527+0000 7f545fad0e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:26.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.527+0000 7f545fad0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.527+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d16f30 con 0x55b041c4ec20 2023-10-05T19:40:26.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.527+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f5450014040 con 0x55b041c4ec20 2023-10-05T19:40:26.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.527+0000 7f545fad0e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:26.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.527+0000 7f545fad0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.527+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d17390 con 0x55b041d0ed20 2023-10-05T19:40:26.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.531+0000 7f545d605700 1 -- 172.21.15.12:0/2747982595 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f5458025020 con 0x55b041d0ed20 2023-10-05T19:40:26.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.531+0000 7f545fad0e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:26.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.531+0000 7f545fad0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.531+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d17c80 con 0x55b041c4ec20 2023-10-05T19:40:26.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.531+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f5450014040 con 0x55b041c4ec20 2023-10-05T19:40:26.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.531+0000 7f545fad0e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:26.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.531+0000 7f545fad0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.531+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d17c80 con 0x55b041d134c0 2023-10-05T19:40:26.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.531+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f5448008ae0 con 0x55b041d134c0 2023-10-05T19:40:26.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.531+0000 7f545fad0e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:26.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.531+0000 7f545fad0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.531+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d183f0 con 0x7f54580106e0 2023-10-05T19:40:26.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.531+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f5450014040 con 0x7f54580106e0 2023-10-05T19:40:26.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.531+0000 7f545fad0e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:26.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.531+0000 7f545fad0e40 20 zone default found 2023-10-05T19:40:26.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.531+0000 7f545fad0e40 4 Realm: () 2023-10-05T19:40:26.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.531+0000 7f545fad0e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:26.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.531+0000 7f545fad0e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:26.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.531+0000 7f545fad0e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:26.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.531+0000 7f545fad0e40 20 zonegroup default 2023-10-05T19:40:26.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.531+0000 7f545fad0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.531+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d180c0 con 0x7f54580106e0 2023-10-05T19:40:26.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.531+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f5450014040 con 0x7f54580106e0 2023-10-05T19:40:26.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.531+0000 7f545fad0e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:26.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.531+0000 7f545fad0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.531+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d19a40 con 0x55b041d134c0 2023-10-05T19:40:26.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.535+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f5448008ae0 con 0x55b041d134c0 2023-10-05T19:40:26.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.535+0000 7f545fad0e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:26.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.535+0000 7f545fad0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.535+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d19df0 con 0x55b041d134c0 2023-10-05T19:40:26.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.535+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f5448008ae0 con 0x55b041d134c0 2023-10-05T19:40:26.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.535+0000 7f545fad0e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:26.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.535+0000 7f545fad0e40 20 started sync module instance, tier type = 2023-10-05T19:40:26.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.535+0000 7f545fad0e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:26.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.535+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d195f0 con 0x55b041d0ed20 2023-10-05T19:40:26.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.539+0000 7f545d605700 1 -- 172.21.15.12:0/2747982595 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'126 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5458025020 con 0x55b041d0ed20 2023-10-05T19:40:26.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.539+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94215506858240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d1a690 con 0x55b041d0ed20 2023-10-05T19:40:26.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.539+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d1af30 con 0x55b041c4ec20 2023-10-05T19:40:26.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.539+0000 7f545d605700 1 -- 172.21.15.12:0/2747982595 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94215506858240] v28'127 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5458025020 con 0x55b041d0ed20 2023-10-05T19:40:26.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.539+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'126 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5450014040 con 0x55b041c4ec20 2023-10-05T19:40:26.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.543+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94215506862496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d1af30 con 0x55b041c4ec20 2023-10-05T19:40:26.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.543+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d1b940 con 0x55b041d134c0 2023-10-05T19:40:26.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.543+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'126 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5448008ae0 con 0x55b041d134c0 2023-10-05T19:40:26.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.543+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94215506867280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d1c1e0 con 0x55b041d134c0 2023-10-05T19:40:26.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.543+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d1ca80 con 0x55b041d0ed20 2023-10-05T19:40:26.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.543+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94215506862496] v28'127 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5450014040 con 0x55b041c4ec20 2023-10-05T19:40:26.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.547+0000 7f545d605700 1 -- 172.21.15.12:0/2747982595 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'126 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5458025020 con 0x55b041d0ed20 2023-10-05T19:40:26.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.547+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94215506864704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d1c6b0 con 0x55b041d0ed20 2023-10-05T19:40:26.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.547+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d1dab0 con 0x55b041c4ec20 2023-10-05T19:40:26.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.547+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94215506867280] v28'127 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5448008ae0 con 0x55b041d134c0 2023-10-05T19:40:26.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.547+0000 7f545d605700 1 -- 172.21.15.12:0/2747982595 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94215506864704] v28'127 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5458025020 con 0x55b041d0ed20 2023-10-05T19:40:26.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.551+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'126 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5450014040 con 0x55b041c4ec20 2023-10-05T19:40:26.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.551+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94215506873984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d1d5c0 con 0x55b041c4ec20 2023-10-05T19:40:26.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.551+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d1dfd0 con 0x7f54580106e0 2023-10-05T19:40:26.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.551+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94215506873984] v28'127 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5450014040 con 0x55b041c4ec20 2023-10-05T19:40:26.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.555+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'126 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5450014040 con 0x7f54580106e0 2023-10-05T19:40:26.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.555+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94215506869488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d1dfd0 con 0x7f54580106e0 2023-10-05T19:40:26.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.555+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d1e9e0 con 0x7f54580106e0 2023-10-05T19:40:26.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.555+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(95 notify.5 [watch watch cookie 94215506869488] v28'127 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5450014040 con 0x7f54580106e0 2023-10-05T19:40:26.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.555+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(96 notify.6 [create] v28'126 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5450014040 con 0x7f54580106e0 2023-10-05T19:40:26.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.555+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94215506877712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d1e9e0 con 0x7f54580106e0 2023-10-05T19:40:26.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.555+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d1f3f0 con 0x7f54580106e0 2023-10-05T19:40:26.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.559+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(97 notify.6 [watch watch cookie 94215506877712] v28'127 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5450014040 con 0x7f54580106e0 2023-10-05T19:40:26.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.559+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(98 notify.7 [create] v28'126 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5450014040 con 0x7f54580106e0 2023-10-05T19:40:26.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.559+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94215506880000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d1f3f0 con 0x7f54580106e0 2023-10-05T19:40:26.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.559+0000 7f545fad0e40 20 add_watcher() i=0 2023-10-05T19:40:26.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.559+0000 7f545fad0e40 20 add_watcher() i=1 2023-10-05T19:40:26.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.559+0000 7f545fad0e40 20 add_watcher() i=2 2023-10-05T19:40:26.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.559+0000 7f545fad0e40 20 add_watcher() i=3 2023-10-05T19:40:26.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.559+0000 7f545fad0e40 20 add_watcher() i=4 2023-10-05T19:40:26.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.559+0000 7f545fad0e40 20 add_watcher() i=5 2023-10-05T19:40:26.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.559+0000 7f545fad0e40 20 add_watcher() i=6 2023-10-05T19:40:26.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.563+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94215506880000] v28'127 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5450014040 con 0x7f54580106e0 2023-10-05T19:40:26.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.563+0000 7f545fad0e40 20 add_watcher() i=7 2023-10-05T19:40:26.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.563+0000 7f545fad0e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:26.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.563+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d224c0 con 0x55b041c4ec20 2023-10-05T19:40:26.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.563+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f5450014040 con 0x55b041c4ec20 2023-10-05T19:40:26.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.563+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94215506882304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d1f840 con 0x55b041c4ec20 2023-10-05T19:40:26.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.567+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94215506882304] v28'122 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f5450014040 con 0x55b041c4ec20 2023-10-05T19:40:26.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.567+0000 7f542e7e4700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:26.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.567+0000 7f545fad0e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:26.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.567+0000 7f545fad0e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:26.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.567+0000 7f545fad0e40 1 Processor -- start 2023-10-05T19:40:26.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.567+0000 7f545fad0e40 1 -- start start 2023-10-05T19:40:26.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.567+0000 7f545fad0e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b041d2cce0 0x55b041d2d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:26.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.567+0000 7f545fad0e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b041d2d5f0 con 0x55b041d2cce0 2023-10-05T19:40:26.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.567+0000 7f545ce04700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b041d2cce0 0x55b041d2d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:26.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.567+0000 7f545ce04700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b041d2cce0 0x55b041d2d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58120/0 (socket says 172.21.15.12:58120) 2023-10-05T19:40:26.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.567+0000 7f545ce04700 1 -- 172.21.15.12:0/3143497118 learned_addr learned my addr 172.21.15.12:0/3143497118 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:26.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.567+0000 7f545ce04700 1 -- 172.21.15.12:0/3143497118 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b041d2d730 con 0x55b041d2cce0 2023-10-05T19:40:26.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.567+0000 7f545ce04700 1 --2- 172.21.15.12:0/3143497118 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b041d2cce0 0x55b041d2d0b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x55b041c4f080 tx=0x7f5448012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ea161344f654dee server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:26.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.567+0000 7f542dfe3700 1 -- 172.21.15.12:0/3143497118 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5448030930 con 0x55b041d2cce0 2023-10-05T19:40:26.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.567+0000 7f542dfe3700 1 -- 172.21.15.12:0/3143497118 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5448030ad0 con 0x55b041d2cce0 2023-10-05T19:40:26.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.567+0000 7f542dfe3700 1 -- 172.21.15.12:0/3143497118 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5448030c80 con 0x55b041d2cce0 2023-10-05T19:40:26.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.571+0000 7f545fad0e40 1 -- 172.21.15.12:0/3143497118 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b041d2cce0 msgr2=0x55b041d2d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:26.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.571+0000 7f545fad0e40 1 --2- 172.21.15.12:0/3143497118 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b041d2cce0 0x55b041d2d0b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x55b041c4f080 tx=0x7f5448012000 comp rx=0 tx=0).stop 2023-10-05T19:40:26.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.571+0000 7f545fad0e40 1 -- 172.21.15.12:0/3143497118 shutdown_connections 2023-10-05T19:40:26.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.571+0000 7f545fad0e40 1 --2- 172.21.15.12:0/3143497118 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b041d2cce0 0x55b041d2d0b0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:26.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.571+0000 7f545fad0e40 1 -- 172.21.15.12:0/3143497118 >> 172.21.15.12:0/3143497118 conn(0x55b041d280a0 msgr2=0x55b041d261f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:26.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.571+0000 7f545fad0e40 1 -- 172.21.15.12:0/3143497118 shutdown_connections 2023-10-05T19:40:26.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.571+0000 7f545fad0e40 1 -- 172.21.15.12:0/3143497118 wait complete. 2023-10-05T19:40:26.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.571+0000 7f545fad0e40 1 Processor -- start 2023-10-05T19:40:26.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.571+0000 7f545fad0e40 1 -- start start 2023-10-05T19:40:26.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.571+0000 7f545fad0e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b041d2cce0 0x55b041d327b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:26.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.571+0000 7f545fad0e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b041d337b0 con 0x55b041d2cce0 2023-10-05T19:40:26.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.571+0000 7f545ce04700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b041d2cce0 0x55b041d327b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:26.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.571+0000 7f545ce04700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b041d2cce0 0x55b041d327b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58128/0 (socket says 172.21.15.12:58128) 2023-10-05T19:40:26.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.571+0000 7f545ce04700 1 -- 172.21.15.12:0/1284806226 learned_addr learned my addr 172.21.15.12:0/1284806226 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:26.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.571+0000 7f545ce04700 1 -- 172.21.15.12:0/1284806226 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b041c4f0b0 con 0x55b041d2cce0 2023-10-05T19:40:26.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.571+0000 7f545ce04700 1 --2- 172.21.15.12:0/1284806226 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b041d2cce0 0x55b041d327b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f544803b000 tx=0x7f5448034000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:26.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.575+0000 7f539ffff700 1 -- 172.21.15.12:0/1284806226 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5448027400 con 0x55b041d2cce0 2023-10-05T19:40:26.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.575+0000 7f539ffff700 1 -- 172.21.15.12:0/1284806226 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5448027560 con 0x55b041d2cce0 2023-10-05T19:40:26.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.575+0000 7f545fad0e40 1 -- 172.21.15.12:0/1284806226 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b041d32db0 con 0x55b041d2cce0 2023-10-05T19:40:26.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.575+0000 7f539ffff700 1 -- 172.21.15.12:0/1284806226 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5448027710 con 0x55b041d2cce0 2023-10-05T19:40:26.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.575+0000 7f545fad0e40 1 -- 172.21.15.12:0/1284806226 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b041d33190 con 0x55b041d2cce0 2023-10-05T19:40:26.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.575+0000 7f539ffff700 1 -- 172.21.15.12:0/1284806226 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f5448027870 con 0x55b041d2cce0 2023-10-05T19:40:26.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.575+0000 7f539ffff700 1 --2- 172.21.15.12:0/1284806226 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f53a4030c50 0x7f53a4033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:26.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.575+0000 7f539ffff700 1 -- 172.21.15.12:0/1284806226 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f5448054af0 con 0x55b041d2cce0 2023-10-05T19:40:26.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.575+0000 7f545de06700 1 --2- 172.21.15.12:0/1284806226 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f53a4030c50 0x7f53a4033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:26.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.575+0000 7f545fad0e40 1 --2- 172.21.15.12:0/1284806226 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55b041d3d310 0x55b041d3f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:26.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.575+0000 7f545fad0e40 1 -- 172.21.15.12:0/1284806226 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d3fd70 con 0x55b041d3d310 2023-10-05T19:40:26.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.575+0000 7f545d605700 1 --2- 172.21.15.12:0/1284806226 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55b041d3d310 0x55b041d3f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:26.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.575+0000 7f545de06700 1 --2- 172.21.15.12:0/1284806226 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f53a4030c50 0x7f53a4033100 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x55b041d22540 tx=0x7f545000a9b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:26.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.575+0000 7f545d605700 1 --2- 172.21.15.12:0/1284806226 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55b041d3d310 0x55b041d3f720 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:26.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.579+0000 7f545d605700 1 -- 172.21.15.12:0/1284806226 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v28'24 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f5458025020 con 0x55b041d3d310 2023-10-05T19:40:26.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.579+0000 7f545fad0e40 1 --2- 172.21.15.12:0/1284806226 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55b041d40a40 0x55b041d42e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:26.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.579+0000 7f545fad0e40 1 -- 172.21.15.12:0/1284806226 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d43550 con 0x55b041d40a40 2023-10-05T19:40:26.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.579+0000 7f545ce04700 1 --2- 172.21.15.12:0/1284806226 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55b041d40a40 0x55b041d42e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:26.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.579+0000 7f545ce04700 1 --2- 172.21.15.12:0/1284806226 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55b041d40a40 0x55b041d42e20 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:26.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.583+0000 7f545ce04700 1 -- 172.21.15.12:0/1284806226 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v28'15 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f5448008ae0 con 0x55b041d40a40 2023-10-05T19:40:26.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.583+0000 7f545fad0e40 1 --2- 172.21.15.12:0/1284806226 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55b041d44220 0x55b041d46600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:26.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.583+0000 7f545fad0e40 1 -- 172.21.15.12:0/1284806226 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d46d30 con 0x55b041d44220 2023-10-05T19:40:26.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.583+0000 7f545de06700 1 --2- 172.21.15.12:0/1284806226 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55b041d44220 0x55b041d46600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:26.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.583+0000 7f545de06700 1 --2- 172.21.15.12:0/1284806226 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55b041d44220 0x55b041d46600 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:26.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.587+0000 7f545de06700 1 -- 172.21.15.12:0/1284806226 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v28'15 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f5450014040 con 0x55b041d44220 2023-10-05T19:40:26.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.587+0000 7f545fad0e40 1 -- 172.21.15.12:0/1284806226 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d47380 con 0x55b041d3d310 2023-10-05T19:40:26.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.591+0000 7f545d605700 1 -- 172.21.15.12:0/1284806226 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v28'123 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f5458025020 con 0x55b041d3d310 2023-10-05T19:40:26.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.591+0000 7f545fad0e40 1 --2- 172.21.15.12:0/1284806226 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55b041d48020 0x55b041d4a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:26.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.591+0000 7f545fad0e40 1 -- 172.21.15.12:0/1284806226 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d4aa60 con 0x55b041d48020 2023-10-05T19:40:26.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.591+0000 7f545d605700 1 --2- 172.21.15.12:0/1284806226 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55b041d48020 0x55b041d4a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:26.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.591+0000 7f545d605700 1 --2- 172.21.15.12:0/1284806226 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55b041d48020 0x55b041d4a400 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:26.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.595+0000 7f545d605700 1 -- 172.21.15.12:0/1284806226 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v28'35 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f5458025020 con 0x55b041d48020 2023-10-05T19:40:26.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.595+0000 7f545fad0e40 1 -- 172.21.15.12:0/1284806226 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d4b0b0 con 0x55b041d44220 2023-10-05T19:40:26.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.595+0000 7f545de06700 1 -- 172.21.15.12:0/1284806226 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v28'26 uv8 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f5450014040 con 0x55b041d44220 2023-10-05T19:40:26.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.595+0000 7f545fad0e40 1 -- 172.21.15.12:0/1284806226 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d4b6d0 con 0x55b041d3d310 2023-10-05T19:40:26.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.599+0000 7f545d605700 1 -- 172.21.15.12:0/1284806226 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v28'17 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f5458025020 con 0x55b041d3d310 2023-10-05T19:40:26.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.599+0000 7f545fad0e40 1 -- 172.21.15.12:0/1284806226 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d4b6d0 con 0x55b041d40a40 2023-10-05T19:40:26.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.603+0000 7f545ce04700 1 -- 172.21.15.12:0/1284806226 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v28'23 uv8 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f5448008ae0 con 0x55b041d40a40 2023-10-05T19:40:26.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.603+0000 7f545fad0e40 1 -- 172.21.15.12:0/1284806226 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d4bcf0 con 0x55b041d48020 2023-10-05T19:40:26.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.603+0000 7f545d605700 1 -- 172.21.15.12:0/1284806226 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v28'15 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f5458025020 con 0x55b041d48020 2023-10-05T19:40:26.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.603+0000 7f545fad0e40 1 -- 172.21.15.12:0/1284806226 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d4bcf0 con 0x55b041d44220 2023-10-05T19:40:26.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.607+0000 7f545de06700 1 -- 172.21.15.12:0/1284806226 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v28'21 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f5450014040 con 0x55b041d44220 2023-10-05T19:40:26.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.607+0000 7f545fad0e40 1 -- 172.21.15.12:0/1284806226 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d4c310 con 0x55b041d48020 2023-10-05T19:40:26.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.607+0000 7f545d605700 1 -- 172.21.15.12:0/1284806226 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v28'36 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f5458025020 con 0x55b041d48020 2023-10-05T19:40:26.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.611+0000 7f545fad0e40 1 -- 172.21.15.12:0/1284806226 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d4c310 con 0x55b041d40a40 2023-10-05T19:40:26.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.611+0000 7f545ce04700 1 -- 172.21.15.12:0/1284806226 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v28'17 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f5448008ae0 con 0x55b041d40a40 2023-10-05T19:40:26.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.611+0000 7f545fad0e40 1 -- 172.21.15.12:0/1284806226 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d4c930 con 0x55b041d48020 2023-10-05T19:40:26.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.615+0000 7f545d605700 1 -- 172.21.15.12:0/1284806226 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v28'15 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f5458025020 con 0x55b041d48020 2023-10-05T19:40:26.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.615+0000 7f545fad0e40 1 -- 172.21.15.12:0/1284806226 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d4c930 con 0x55b041d40a40 2023-10-05T19:40:26.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.619+0000 7f545ce04700 1 -- 172.21.15.12:0/1284806226 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v28'31 uv12 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f544805d040 con 0x55b041d40a40 2023-10-05T19:40:26.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.619+0000 7f545fad0e40 1 -- 172.21.15.12:0/1284806226 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d4ce20 con 0x55b041d3d310 2023-10-05T19:40:26.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.619+0000 7f545d605700 1 -- 172.21.15.12:0/1284806226 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v28'124 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f5458025020 con 0x55b041d3d310 2023-10-05T19:40:26.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.619+0000 7f545fad0e40 1 -- 172.21.15.12:0/1284806226 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d4ce20 con 0x55b041d44220 2023-10-05T19:40:26.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.623+0000 7f545de06700 1 -- 172.21.15.12:0/1284806226 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v28'29 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f5450014040 con 0x55b041d44220 2023-10-05T19:40:26.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.623+0000 7f545fad0e40 1 -- 172.21.15.12:0/1284806226 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d38cf0 con 0x55b041d48020 2023-10-05T19:40:26.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.627+0000 7f545d605700 1 -- 172.21.15.12:0/1284806226 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v28'37 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f5458025020 con 0x55b041d48020 2023-10-05T19:40:26.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.627+0000 7f545fad0e40 1 -- 172.21.15.12:0/1284806226 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d38cf0 con 0x55b041d44220 2023-10-05T19:40:26.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.631+0000 7f545de06700 1 -- 172.21.15.12:0/1284806226 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v28'15 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f5450014040 con 0x55b041d44220 2023-10-05T19:40:26.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.631+0000 7f545fad0e40 1 -- 172.21.15.12:0/1284806226 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d38cf0 con 0x55b041d3d310 2023-10-05T19:40:26.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.631+0000 7f545d605700 1 -- 172.21.15.12:0/1284806226 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v28'28 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f5458025020 con 0x55b041d3d310 2023-10-05T19:40:26.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.631+0000 7f545fad0e40 1 -- 172.21.15.12:0/1284806226 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d38cf0 con 0x55b041d3d310 2023-10-05T19:40:26.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.635+0000 7f545d605700 1 -- 172.21.15.12:0/1284806226 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v28'125 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f5458025020 con 0x55b041d3d310 2023-10-05T19:40:26.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.635+0000 7f545fad0e40 1 -- 172.21.15.12:0/1284806226 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d38cf0 con 0x55b041d44220 2023-10-05T19:40:26.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.639+0000 7f545de06700 1 -- 172.21.15.12:0/1284806226 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v28'30 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f5450014040 con 0x55b041d44220 2023-10-05T19:40:26.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.639+0000 7f545fad0e40 1 -- 172.21.15.12:0/1284806226 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d38cf0 con 0x55b041d48020 2023-10-05T19:40:26.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.639+0000 7f545d605700 1 -- 172.21.15.12:0/1284806226 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v28'25 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f5458025020 con 0x55b041d48020 2023-10-05T19:40:26.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.639+0000 7f545fad0e40 1 -- 172.21.15.12:0/1284806226 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d38cf0 con 0x55b041d44220 2023-10-05T19:40:26.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.643+0000 7f545de06700 1 -- 172.21.15.12:0/1284806226 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v28'17 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f5450014040 con 0x55b041d44220 2023-10-05T19:40:26.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.643+0000 7f545fad0e40 1 -- 172.21.15.12:0/1284806226 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d38cf0 con 0x55b041d44220 2023-10-05T19:40:26.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.643+0000 7f545de06700 1 -- 172.21.15.12:0/1284806226 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v28'23 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f5450014040 con 0x55b041d44220 2023-10-05T19:40:26.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.647+0000 7f545fad0e40 1 -- 172.21.15.12:0/1284806226 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d38cf0 con 0x55b041d44220 2023-10-05T19:40:26.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.647+0000 7f545de06700 1 -- 172.21.15.12:0/1284806226 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v28'12 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f5450014040 con 0x55b041d44220 2023-10-05T19:40:26.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.647+0000 7f545fad0e40 1 -- 172.21.15.12:0/1284806226 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d38cf0 con 0x55b041d44220 2023-10-05T19:40:26.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.651+0000 7f545de06700 1 -- 172.21.15.12:0/1284806226 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v28'27 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f5450014040 con 0x55b041d44220 2023-10-05T19:40:26.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.651+0000 7f545fad0e40 1 -- 172.21.15.12:0/1284806226 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d38cf0 con 0x55b041d40a40 2023-10-05T19:40:26.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.651+0000 7f545ce04700 1 -- 172.21.15.12:0/1284806226 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v28'20 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f544805d040 con 0x55b041d40a40 2023-10-05T19:40:26.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.651+0000 7f545fad0e40 1 -- 172.21.15.12:0/1284806226 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d38cf0 con 0x55b041d44220 2023-10-05T19:40:26.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.655+0000 7f545de06700 1 -- 172.21.15.12:0/1284806226 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v28'15 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f5450014040 con 0x55b041d44220 2023-10-05T19:40:26.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.655+0000 7f545fad0e40 1 -- 172.21.15.12:0/1284806226 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d38cf0 con 0x55b041d40a40 2023-10-05T19:40:26.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.655+0000 7f545ce04700 1 -- 172.21.15.12:0/1284806226 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v28'24 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f544805d040 con 0x55b041d40a40 2023-10-05T19:40:26.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.655+0000 7f545fad0e40 1 -- 172.21.15.12:0/1284806226 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d38cf0 con 0x55b041d44220 2023-10-05T19:40:26.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.659+0000 7f545de06700 1 -- 172.21.15.12:0/1284806226 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v28'31 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f5450014040 con 0x55b041d44220 2023-10-05T19:40:26.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.659+0000 7f545fad0e40 1 -- 172.21.15.12:0/1284806226 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d38cf0 con 0x55b041d40a40 2023-10-05T19:40:26.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.663+0000 7f545ce04700 1 -- 172.21.15.12:0/1284806226 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v28'32 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f544805d040 con 0x55b041d40a40 2023-10-05T19:40:26.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.663+0000 7f545fad0e40 1 -- 172.21.15.12:0/1284806226 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041d38cf0 con 0x55b041d48020 2023-10-05T19:40:26.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.663+0000 7f545d605700 1 -- 172.21.15.12:0/1284806226 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v28'21 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f5458025020 con 0x55b041d48020 2023-10-05T19:40:26.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.667+0000 7f539d7fa700 20 reqs_thread_entry: start 2023-10-05T19:40:26.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.667+0000 7f5396ffd700 20 reqs_thread_entry: start 2023-10-05T19:40:26.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.667+0000 7f545fad0e40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:26.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.667+0000 7f545fad0e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:26.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.667+0000 7f53957fa700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:57 2023 2023-10-05T19:40:26.676 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:26.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.667+0000 7f545fad0e40 20 get_system_obj_state: rctx=0x7ffd6d4a4138 obj=default.rgw.meta:users.uid:fud state=0x55b041bb20b0 s->prefetch_data=0 2023-10-05T19:40:26.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.667+0000 7f545fad0e40 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2023-10-05T19:40:26.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.667+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:102 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041dabb90 con 0x7f54580106e0 2023-10-05T19:40:26.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.667+0000 7f5394ff9700 20 reqs_thread_entry: start 2023-10-05T19:40:26.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.667+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(102 fud [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7f5450014040 con 0x7f54580106e0 2023-10-05T19:40:26.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.667+0000 7f545fad0e40 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x16 2023-10-05T19:40:26.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.667+0000 7f545fad0e40 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2023-10-05T19:40:26.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.667+0000 7f545fad0e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:26.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.667+0000 7f545fad0e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:26.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.667+0000 7f545fad0e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:26.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.667+0000 7f545fad0e40 10 cache get: name=default.rgw.meta+users.uid+fud : type miss (requested=0x13, cached=0x16) 2023-10-05T19:40:26.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.667+0000 7f545fad0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.667+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:103 8.e 8:75f48285:users.uid::fud:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041dadff0 con 0x7f54580106e0 2023-10-05T19:40:26.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.671+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(103 fud [call,call out=48b,read 0~346 out=346b,getxattrs out=75b] v0'0 uv2 ondisk = 0) v8 ==== 273+0+469 (crc 0 0 0) 0x7f5450014040 con 0x7f54580106e0 2023-10-05T19:40:26.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.671+0000 7f545fad0e40 20 rados_obj.operate() r=0 bl.length=346 2023-10-05T19:40:26.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.671+0000 7f545fad0e40 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x13 2023-10-05T19:40:26.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.671+0000 7f545fad0e40 10 moving default.rgw.meta+users.uid+fud to cache LRU end 2023-10-05T19:40:26.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.671+0000 7f545fad0e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:26.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.671+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:104 8.1 8:810902b1:users.uid::fud.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041dadff0 con 0x7f54580106e0 2023-10-05T19:40:26.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.671+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(104 fud.buckets [call out=15b] v0'0 uv2 ondisk = 0) v8 ==== 155+0+15 (crc 0 0 0) 0x7f5450014040 con 0x7f54580106e0 2023-10-05T19:40:26.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.671+0000 7f545fad0e40 10 removing key index: p5YnriCv1nAtykxBrupQ 2023-10-05T19:40:26.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.671+0000 7f545fad0e40 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2023-10-05T19:40:26.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.671+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:105 7.15 7:a93a5511:::notify.2:head [notify cookie 94215507461808 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041dacff0 con 0x55b041d134c0 2023-10-05T19:40:26.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.671+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== watch-notify(notify (1) cookie 94215506867280 notify 120259084291 ret 0) v3 ==== 212+0+0 (crc 0 0 0) 0x55b041d33190 con 0x55b041d134c0 2023-10-05T19:40:26.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.671+0000 7f5456ffd700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084291 cookie 94215506867280 notifier 4477 bl.length()=170 2023-10-05T19:40:26.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.671+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 <== osd.3 v2:172.21.15.12:6816/29951 27 ==== osd_op_reply(105 notify.2 [notify cookie 94215507461808 out=8b] v0'0 uv126 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f544805d040 con 0x55b041d134c0 2023-10-05T19:40:26.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.671+0000 7f5456ffd700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:106 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f543c003c50 con 0x55b041d134c0 2023-10-05T19:40:26.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.671+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 <== osd.3 v2:172.21.15.12:6816/29951 28 ==== osd_op_reply(106 notify.2 [notify-ack] v0'0 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f544805d040 con 0x55b041d134c0 2023-10-05T19:40:26.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.671+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 <== osd.3 v2:172.21.15.12:6816/29951 29 ==== watch-notify(notify_complete (2) cookie 94215507461808 notify 120259084291 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55b041d32db0 con 0x55b041d134c0 2023-10-05T19:40:26.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.671+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:107 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041dabb90 con 0x7f54580106e0 2023-10-05T19:40:26.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.675+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.0 v2:172.21.15.8:6800/27563 30 ==== osd_op_reply(107 p5YnriCv1nAtykxBrupQ [delete] v28'4 uv4 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5450014040 con 0x7f54580106e0 2023-10-05T19:40:26.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.675+0000 7f545fad0e40 10 removing email index: 2023-10-05T19:40:26.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.675+0000 7f545fad0e40 10 removing user buckets index 2023-10-05T19:40:26.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.675+0000 7f545fad0e40 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:users.uid:fud.buckets, ofs0, ns] 2023-10-05T19:40:26.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.675+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:108 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94215507461808 in=172b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041dacff0 con 0x7f54580106e0 2023-10-05T19:40:26.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.675+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.0 v2:172.21.15.8:6800/27563 31 ==== watch-notify(notify (1) cookie 94215506877712 notify 120259084306 ret 0) v3 ==== 202+0+0 (crc 0 0 0) 0x7f5450033820 con 0x7f54580106e0 2023-10-05T19:40:26.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.675+0000 7f54567fc700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084306 cookie 94215506877712 notifier 4477 bl.length()=160 2023-10-05T19:40:26.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.675+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.0 v2:172.21.15.8:6800/27563 32 ==== osd_op_reply(108 notify.6 [notify cookie 94215507461808 out=8b] v0'0 uv126 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f5450014040 con 0x7f54580106e0 2023-10-05T19:40:26.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.675+0000 7f54567fc700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:109 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f53ac0032d0 con 0x7f54580106e0 2023-10-05T19:40:26.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.679+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.0 v2:172.21.15.8:6800/27563 33 ==== osd_op_reply(109 notify.6 [notify-ack] v0'0 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5450014040 con 0x7f54580106e0 2023-10-05T19:40:26.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.679+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.0 v2:172.21.15.8:6800/27563 34 ==== watch-notify(notify_complete (2) cookie 94215507461808 notify 120259084306 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f5450033350 con 0x7f54580106e0 2023-10-05T19:40:26.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.679+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:110 8.1 8:810902b1:users.uid::fud.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041dabb90 con 0x7f54580106e0 2023-10-05T19:40:26.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.679+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.0 v2:172.21.15.8:6800/27563 35 ==== osd_op_reply(110 fud.buckets [delete] v28'4 uv4 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x7f5450014040 con 0x7f54580106e0 2023-10-05T19:40:26.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.679+0000 7f545fad0e40 10 removing user index: fud 2023-10-05T19:40:26.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.679+0000 7f545fad0e40 10 removing default.rgw.meta+users.uid+fud from cache 2023-10-05T19:40:26.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.679+0000 7f545fad0e40 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.uid:fud, ofs0, ns] 2023-10-05T19:40:26.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.679+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:111 7.3 7:c609908c:::notify.5:head [notify cookie 94215507457696 in=164b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041dacff0 con 0x7f54580106e0 2023-10-05T19:40:26.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.683+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.0 v2:172.21.15.8:6800/27563 36 ==== watch-notify(notify (1) cookie 94215506869488 notify 120259084307 ret 0) v3 ==== 194+0+0 (crc 0 0 0) 0x7f5450033350 con 0x7f54580106e0 2023-10-05T19:40:26.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.683+0000 7f54567fc700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084307 cookie 94215506869488 notifier 4477 bl.length()=152 2023-10-05T19:40:26.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.683+0000 7f54567fc700 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:112 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f53ac003290 con 0x7f54580106e0 2023-10-05T19:40:26.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.683+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.0 v2:172.21.15.8:6800/27563 37 ==== osd_op_reply(111 notify.5 [notify cookie 94215507457696 out=8b] v0'0 uv126 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f5450014040 con 0x7f54580106e0 2023-10-05T19:40:26.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.683+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.0 v2:172.21.15.8:6800/27563 38 ==== watch-notify(notify_complete (2) cookie 94215507457696 notify 120259084307 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f54500316e0 con 0x7f54580106e0 2023-10-05T19:40:26.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.683+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.0 v2:172.21.15.8:6800/27563 39 ==== osd_op_reply(112 notify.5 [notify-ack] v0'0 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5450014040 con 0x7f54580106e0 2023-10-05T19:40:26.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.683+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:113 8.e 8:75f48285:users.uid::fud:head [call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041dacba0 con 0x7f54580106e0 2023-10-05T19:40:26.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.687+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.0 v2:172.21.15.8:6800/27563 40 ==== osd_op_reply(113 fud [call,delete] v28'5 uv5 ondisk = 0) v8 ==== 189+0+0 (crc 0 0 0) 0x7f5450014040 con 0x7f54580106e0 2023-10-05T19:40:26.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.687+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:114 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94215506858240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041cf0810 con 0x55b041d0ed20 2023-10-05T19:40:26.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.687+0000 7f545d605700 1 -- 172.21.15.12:0/2747982595 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(114 notify.0 [watch unwatch cookie 94215506858240] v28'128 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5458025020 con 0x55b041d0ed20 2023-10-05T19:40:26.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.687+0000 7f545fad0e40 20 remove_watcher() i=0 2023-10-05T19:40:26.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.687+0000 7f545fad0e40 2 removed watcher, disabling cache 2023-10-05T19:40:26.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.687+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:115 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94215506862496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041cf0810 con 0x55b041c4ec20 2023-10-05T19:40:26.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.691+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(115 notify.1 [watch unwatch cookie 94215506862496] v28'128 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5450014040 con 0x55b041c4ec20 2023-10-05T19:40:26.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.691+0000 7f545fad0e40 20 remove_watcher() i=1 2023-10-05T19:40:26.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.691+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:116 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94215506867280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041cf0810 con 0x55b041d134c0 2023-10-05T19:40:26.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.695+0000 7f545ce04700 1 -- 172.21.15.12:0/2747982595 <== osd.3 v2:172.21.15.12:6816/29951 30 ==== osd_op_reply(116 notify.2 [watch unwatch cookie 94215506867280] v28'128 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f544805d040 con 0x55b041d134c0 2023-10-05T19:40:26.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.695+0000 7f545fad0e40 20 remove_watcher() i=2 2023-10-05T19:40:26.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.695+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:117 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94215506864704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041cf0810 con 0x55b041d0ed20 2023-10-05T19:40:26.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.699+0000 7f545d605700 1 -- 172.21.15.12:0/2747982595 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(117 notify.3 [watch unwatch cookie 94215506864704] v28'128 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5458025020 con 0x55b041d0ed20 2023-10-05T19:40:26.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.699+0000 7f545fad0e40 20 remove_watcher() i=3 2023-10-05T19:40:26.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.699+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:118 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94215506873984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041cf0810 con 0x55b041c4ec20 2023-10-05T19:40:26.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.699+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(118 notify.4 [watch unwatch cookie 94215506873984] v28'128 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5450014040 con 0x55b041c4ec20 2023-10-05T19:40:26.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.703+0000 7f545fad0e40 20 remove_watcher() i=4 2023-10-05T19:40:26.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.703+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:119 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94215506869488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041cf0810 con 0x7f54580106e0 2023-10-05T19:40:26.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.703+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.0 v2:172.21.15.8:6800/27563 41 ==== osd_op_reply(119 notify.5 [watch unwatch cookie 94215506869488] v28'128 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5450014040 con 0x7f54580106e0 2023-10-05T19:40:26.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.703+0000 7f545fad0e40 20 remove_watcher() i=5 2023-10-05T19:40:26.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.703+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:120 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94215506877712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041cf0810 con 0x7f54580106e0 2023-10-05T19:40:26.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.707+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.0 v2:172.21.15.8:6800/27563 42 ==== osd_op_reply(120 notify.6 [watch unwatch cookie 94215506877712] v28'128 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5450014040 con 0x7f54580106e0 2023-10-05T19:40:26.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.707+0000 7f545fad0e40 20 remove_watcher() i=6 2023-10-05T19:40:26.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.707+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:121 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94215506880000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041cf0810 con 0x7f54580106e0 2023-10-05T19:40:26.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.711+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.0 v2:172.21.15.8:6800/27563 43 ==== osd_op_reply(121 notify.7 [watch unwatch cookie 94215506880000] v28'128 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5450014040 con 0x7f54580106e0 2023-10-05T19:40:26.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.711+0000 7f545fad0e40 20 remove_watcher() i=7 2023-10-05T19:40:26.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.711+0000 7f542e7e4700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:26.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.711+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:122 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94215506882304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b041cf0810 con 0x55b041c4ec20 2023-10-05T19:40:26.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.715+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(122 data_loggenerations_metadata [watch unwatch cookie 94215506882304] v28'126 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f5450014040 con 0x55b041c4ec20 2023-10-05T19:40:26.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.715+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f54580106e0 msgr2=0x7f545801e590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:26.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.715+0000 7f545fad0e40 1 --2- 172.21.15.12:0/2747982595 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f54580106e0 0x7f545801e590 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:26.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.715+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55b041c4ec20 msgr2=0x55b041d0c7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:26.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.715+0000 7f545fad0e40 1 --2- 172.21.15.12:0/2747982595 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55b041c4ec20 0x55b041d0c7f0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:26.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.715+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55b041d0ed20 msgr2=0x55b041d11100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:26.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.715+0000 7f545fad0e40 1 --2- 172.21.15.12:0/2747982595 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55b041d0ed20 0x55b041d11100 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:26.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.715+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55b041d134c0 msgr2=0x55b041d158d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:26.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.715+0000 7f545fad0e40 1 --2- 172.21.15.12:0/2747982595 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55b041d134c0 0x55b041d158d0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:26.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.715+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5440030c50 msgr2=0x7f5440033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:26.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.715+0000 7f545fad0e40 1 --2- 172.21.15.12:0/2747982595 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5440030c50 0x7f5440033100 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5448005a90 tx=0x7f5448015000 comp rx=0 tx=0).stop 2023-10-05T19:40:26.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.715+0000 7f545de06700 1 -- 172.21.15.12:0/2747982595 reap_dead start 2023-10-05T19:40:26.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.715+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b041c4d910 msgr2=0x55b041d00d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:26.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.715+0000 7f545fad0e40 1 --2- 172.21.15.12:0/2747982595 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b041c4d910 0x55b041d00d00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f54580069f0 tx=0x7f5458014000 comp rx=0 tx=0).stop 2023-10-05T19:40:26.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.715+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 shutdown_connections 2023-10-05T19:40:26.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.715+0000 7f545fad0e40 1 --2- 172.21.15.12:0/2747982595 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b041c4d910 0x55b041d00d00 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:26.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.715+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 >> 172.21.15.12:0/2747982595 conn(0x55b041cd4850 msgr2=0x55b041cf45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:26.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.715+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 shutdown_connections 2023-10-05T19:40:26.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.715+0000 7f545fad0e40 1 -- 172.21.15.12:0/2747982595 wait complete. 2023-10-05T19:40:26.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.715+0000 7f545fad0e40 1 -- 172.21.15.12:0/1284806226 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55b041d40a40 msgr2=0x55b041d42e20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:26.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.715+0000 7f545fad0e40 1 --2- 172.21.15.12:0/1284806226 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55b041d40a40 0x55b041d42e20 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:26.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.715+0000 7f545fad0e40 1 -- 172.21.15.12:0/1284806226 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55b041d3d310 msgr2=0x55b041d3f720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:26.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.715+0000 7f545fad0e40 1 --2- 172.21.15.12:0/1284806226 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55b041d3d310 0x55b041d3f720 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:26.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.715+0000 7f545fad0e40 1 -- 172.21.15.12:0/1284806226 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55b041d48020 msgr2=0x55b041d4a400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:26.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.715+0000 7f545fad0e40 1 --2- 172.21.15.12:0/1284806226 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55b041d48020 0x55b041d4a400 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:26.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.715+0000 7f545fad0e40 1 -- 172.21.15.12:0/1284806226 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55b041d44220 msgr2=0x55b041d46600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:26.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.715+0000 7f545fad0e40 1 --2- 172.21.15.12:0/1284806226 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55b041d44220 0x55b041d46600 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:26.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.715+0000 7f545fad0e40 1 -- 172.21.15.12:0/1284806226 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f53a4030c50 msgr2=0x7f53a4033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:26.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.715+0000 7f545fad0e40 1 --2- 172.21.15.12:0/1284806226 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f53a4030c50 0x7f53a4033100 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x55b041d22540 tx=0x7f545000a9b0 comp rx=0 tx=0).stop 2023-10-05T19:40:26.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.715+0000 7f545fad0e40 1 -- 172.21.15.12:0/1284806226 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b041d2cce0 msgr2=0x55b041d327b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:26.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.715+0000 7f545fad0e40 1 --2- 172.21.15.12:0/1284806226 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b041d2cce0 0x55b041d327b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f544803b000 tx=0x7f5448034000 comp rx=0 tx=0).stop 2023-10-05T19:40:26.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.715+0000 7f545d605700 1 -- 172.21.15.12:0/1284806226 reap_dead start 2023-10-05T19:40:26.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.715+0000 7f545fad0e40 1 -- 172.21.15.12:0/1284806226 shutdown_connections 2023-10-05T19:40:26.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.715+0000 7f545fad0e40 1 -- 172.21.15.12:0/1284806226 >> 172.21.15.12:0/1284806226 conn(0x55b041d280a0 msgr2=0x55b041d26140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:26.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.719+0000 7f545fad0e40 1 -- 172.21.15.12:0/1284806226 shutdown_connections 2023-10-05T19:40:26.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.719+0000 7f545fad0e40 1 -- 172.21.15.12:0/1284806226 wait complete. 2023-10-05T19:40:26.732 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--tenant', 'testx', '--uid', 'tenanteduser', '--display-name', 'tenanted-user', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '1'] 2023-10-05T19:40:26.732 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--tenant', 'testx', '--uid', 'tenanteduser', '--display-name', 'tenanted-user', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '1'] 2023-10-05T19:40:26.733 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --tenant testx --uid tenanteduser --display-name tenanted-user --access-key p5YnriCv1nAtykxBrupQ --secret Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh --max-buckets 1 2023-10-05T19:40:26.823 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:26.823 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:26.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.815+0000 7f06761e4e40 1 Processor -- start 2023-10-05T19:40:26.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.815+0000 7f06761e4e40 1 -- start start 2023-10-05T19:40:26.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.815+0000 7f06761e4e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55ce7751a3b0 0x55ce7751c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:26.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.815+0000 7f06761e4e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55ce77462bd0 con 0x55ce77515dd0 2023-10-05T19:40:26.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.815+0000 7f06761e4e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x55ce7741fee0 con 0x55ce7751a3b0 2023-10-05T19:40:26.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.815+0000 7f0673518700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55ce7751a3b0 0x55ce7751c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:26.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.815+0000 7f0673518700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55ce7751a3b0 0x55ce7751c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58144/0 (socket says 172.21.15.12:58144) 2023-10-05T19:40:26.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.815+0000 7f0673d19700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x55ce77515dd0 0x55ce775161a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:35596/0 (socket says 172.21.15.12:35596) 2023-10-05T19:40:26.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.815+0000 7f0673518700 1 -- 172.21.15.12:0/3596387187 learned_addr learned my addr 172.21.15.12:0/3596387187 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:26.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.815+0000 7f0673518700 1 -- 172.21.15.12:0/3596387187 >> v1:172.21.15.8:6789/0 conn(0x55ce77515dd0 legacy=0x55ce775161a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:26.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.815+0000 7f0673518700 1 -- 172.21.15.12:0/3596387187 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce775169e0 con 0x55ce7751a3b0 2023-10-05T19:40:26.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.819+0000 7f0673518700 1 --2- 172.21.15.12:0/3596387187 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ce7751a3b0 0x55ce7751c790 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f065c00a5c0 tx=0x7f065c005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8af01969866aaf0e server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:26.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.819+0000 7f0672d17700 1 -- 172.21.15.12:0/3596387187 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f065c00e510 con 0x55ce7751a3b0 2023-10-05T19:40:26.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.819+0000 7f0672d17700 1 -- 172.21.15.12:0/3596387187 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f065c00e670 con 0x55ce7751a3b0 2023-10-05T19:40:26.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.819+0000 7f0672d17700 1 -- 172.21.15.12:0/3596387187 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f065c00e920 con 0x55ce7751a3b0 2023-10-05T19:40:26.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.819+0000 7f06761e4e40 1 -- 172.21.15.12:0/3596387187 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ce7751a3b0 msgr2=0x55ce7751c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:26.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.819+0000 7f06761e4e40 1 --2- 172.21.15.12:0/3596387187 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ce7751a3b0 0x55ce7751c790 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f065c00a5c0 tx=0x7f065c005e00 comp rx=0 tx=0).stop 2023-10-05T19:40:26.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.819+0000 7f06761e4e40 1 -- 172.21.15.12:0/3596387187 shutdown_connections 2023-10-05T19:40:26.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.819+0000 7f06761e4e40 1 --2- 172.21.15.12:0/3596387187 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ce7751a3b0 0x55ce7751c790 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:26.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.819+0000 7f06761e4e40 1 -- 172.21.15.12:0/3596387187 >> 172.21.15.12:0/3596387187 conn(0x55ce775117f0 msgr2=0x55ce77513c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:26.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.819+0000 7f06761e4e40 1 -- 172.21.15.12:0/3596387187 shutdown_connections 2023-10-05T19:40:26.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.819+0000 7f06761e4e40 1 -- 172.21.15.12:0/3596387187 wait complete. 2023-10-05T19:40:26.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.819+0000 7f06761e4e40 1 Processor -- start 2023-10-05T19:40:26.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.819+0000 7f06761e4e40 1 -- start start 2023-10-05T19:40:26.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.823+0000 7f06761e4e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ce77515dd0 0x55ce77534f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:26.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.823+0000 7f06761e4e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce773e3e30 con 0x55ce77515dd0 2023-10-05T19:40:26.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.823+0000 7f0673d19700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ce77515dd0 0x55ce77534f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:26.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.823+0000 7f0673d19700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ce77515dd0 0x55ce77534f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58154/0 (socket says 172.21.15.12:58154) 2023-10-05T19:40:26.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.823+0000 7f0673d19700 1 -- 172.21.15.12:0/1909769179 learned_addr learned my addr 172.21.15.12:0/1909769179 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:26.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.823+0000 7f0673d19700 1 -- 172.21.15.12:0/1909769179 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f065c00a180 con 0x55ce77515dd0 2023-10-05T19:40:26.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.823+0000 7f0673d19700 1 --2- 172.21.15.12:0/1909769179 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ce77515dd0 0x55ce77534f70 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f066c001cb0 tx=0x7f066c001860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8de57bfd72fba62 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:26.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.823+0000 7f0671d15700 1 -- 172.21.15.12:0/1909769179 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f066c014070 con 0x55ce77515dd0 2023-10-05T19:40:26.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.823+0000 7f0671d15700 1 -- 172.21.15.12:0/1909769179 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f066c00bd40 con 0x55ce77515dd0 2023-10-05T19:40:26.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.823+0000 7f0671d15700 1 -- 172.21.15.12:0/1909769179 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f066c00f450 con 0x55ce77515dd0 2023-10-05T19:40:26.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.823+0000 7f06761e4e40 1 -- 172.21.15.12:0/1909769179 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ce77515dd0 msgr2=0x55ce77534f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:26.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.823+0000 7f06761e4e40 1 --2- 172.21.15.12:0/1909769179 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ce77515dd0 0x55ce77534f70 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f066c001cb0 tx=0x7f066c001860 comp rx=0 tx=0).stop 2023-10-05T19:40:26.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.823+0000 7f06761e4e40 1 -- 172.21.15.12:0/1909769179 shutdown_connections 2023-10-05T19:40:26.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.823+0000 7f06761e4e40 1 --2- 172.21.15.12:0/1909769179 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ce77515dd0 0x55ce77534f70 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:26.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.823+0000 7f06761e4e40 1 -- 172.21.15.12:0/1909769179 >> 172.21.15.12:0/1909769179 conn(0x55ce775117f0 msgr2=0x55ce77530b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:26.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.823+0000 7f06761e4e40 1 -- 172.21.15.12:0/1909769179 shutdown_connections 2023-10-05T19:40:26.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.823+0000 7f06761e4e40 1 -- 172.21.15.12:0/1909769179 wait complete. 2023-10-05T19:40:26.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.827+0000 7f06761e4e40 1 Processor -- start 2023-10-05T19:40:26.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.827+0000 7f06761e4e40 1 -- start start 2023-10-05T19:40:26.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.827+0000 7f06761e4e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ce77515dd0 0x55ce7753bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:26.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.827+0000 7f06761e4e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce773b6ed0 con 0x55ce77515dd0 2023-10-05T19:40:26.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.827+0000 7f0673d19700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ce77515dd0 0x55ce7753bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:26.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.827+0000 7f0673d19700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ce77515dd0 0x55ce7753bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58166/0 (socket says 172.21.15.12:58166) 2023-10-05T19:40:26.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.827+0000 7f0673d19700 1 -- 172.21.15.12:0/2802956745 learned_addr learned my addr 172.21.15.12:0/2802956745 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:26.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.827+0000 7f0673d19700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce77462bd0 con 0x55ce77515dd0 2023-10-05T19:40:26.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.827+0000 7f0673d19700 1 --2- 172.21.15.12:0/2802956745 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ce77515dd0 0x55ce7753bd40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f066c000c00 tx=0x7f066c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:26.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.827+0000 7f066bfff700 1 -- 172.21.15.12:0/2802956745 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f066c00beb0 con 0x55ce77515dd0 2023-10-05T19:40:26.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.827+0000 7f066bfff700 1 -- 172.21.15.12:0/2802956745 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f066c00f450 con 0x55ce77515dd0 2023-10-05T19:40:26.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.827+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ce7753c280 con 0x55ce77515dd0 2023-10-05T19:40:26.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.827+0000 7f066bfff700 1 -- 172.21.15.12:0/2802956745 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f066c01cc80 con 0x55ce77515dd0 2023-10-05T19:40:26.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.827+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ce7753c6c0 con 0x55ce77515dd0 2023-10-05T19:40:26.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.831+0000 7f066bfff700 1 -- 172.21.15.12:0/2802956745 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f066c023020 con 0x55ce77515dd0 2023-10-05T19:40:26.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.831+0000 7f06761e4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.831+0000 7f066bfff700 1 --2- 172.21.15.12:0/2802956745 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f0654030c00 0x7f06540330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:26.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.831+0000 7f066bfff700 1 -- 172.21.15.12:0/2802956745 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f066c008030 con 0x55ce77515dd0 2023-10-05T19:40:26.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.831+0000 7f0673518700 1 --2- 172.21.15.12:0/2802956745 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f0654030c00 0x7f06540330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:26.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.831+0000 7f06761e4e40 1 --2- 172.21.15.12:0/2802956745 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55ce7751a3b0 0x55ce77547830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:26.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.831+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77547ee0 con 0x55ce7751a3b0 2023-10-05T19:40:26.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.831+0000 7f067451a700 1 --2- 172.21.15.12:0/2802956745 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55ce7751a3b0 0x55ce77547830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:26.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.831+0000 7f067451a700 1 --2- 172.21.15.12:0/2802956745 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55ce7751a3b0 0x55ce77547830 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:26.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.831+0000 7f0673518700 1 --2- 172.21.15.12:0/2802956745 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f0654030c00 0x7f06540330b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f065c005e40 tx=0x7f065c013000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:26.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.831+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f066400a500 con 0x55ce7751a3b0 2023-10-05T19:40:26.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.831+0000 7f06761e4e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:26.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.831+0000 7f06761e4e40 20 realm 2023-10-05T19:40:26.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.831+0000 7f06761e4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.831+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77544e90 con 0x55ce7751a3b0 2023-10-05T19:40:26.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.831+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f066400a500 con 0x55ce7751a3b0 2023-10-05T19:40:26.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.831+0000 7f06761e4e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:26.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.831+0000 7f06761e4e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:26.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.831+0000 7f06761e4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.831+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce775489c0 con 0x55ce7751a3b0 2023-10-05T19:40:26.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.835+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f066400a500 con 0x55ce7751a3b0 2023-10-05T19:40:26.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.835+0000 7f06761e4e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:26.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.835+0000 7f06761e4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.835+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce775485d0 con 0x55ce7751a3b0 2023-10-05T19:40:26.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.835+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f066400a500 con 0x55ce7751a3b0 2023-10-05T19:40:26.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.835+0000 7f06761e4e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:26.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.835+0000 7f06761e4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.835+0000 7f06761e4e40 1 --2- 172.21.15.12:0/2802956745 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55ce77549d60 0x55ce7754c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:26.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.835+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce7754c680 con 0x55ce77549d60 2023-10-05T19:40:26.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.835+0000 7f0673d19700 1 --2- 172.21.15.12:0/2802956745 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55ce77549d60 0x55ce7754c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:26.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.835+0000 7f0673d19700 1 --2- 172.21.15.12:0/2802956745 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55ce77549d60 0x55ce7754c140 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:26.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.835+0000 7f0673d19700 1 -- 172.21.15.12:0/2802956745 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f066c016070 con 0x55ce77549d60 2023-10-05T19:40:26.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.835+0000 7f06761e4e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:26.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.835+0000 7f06761e4e40 20 searching for the correct realm 2023-10-05T19:40:26.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.835+0000 7f06761e4e40 1 --2- 172.21.15.12:0/2802956745 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55ce7754e500 0x55ce77550910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:26.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.835+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77550fc0 con 0x55ce7754e500 2023-10-05T19:40:26.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.835+0000 7f0673518700 1 --2- 172.21.15.12:0/2802956745 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55ce7754e500 0x55ce77550910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:26.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.835+0000 7f0673518700 1 --2- 172.21.15.12:0/2802956745 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55ce7754e500 0x55ce77550910 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:26.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.839+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f065c017050 con 0x55ce7754e500 2023-10-05T19:40:26.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.839+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77550fc0 con 0x55ce77549d60 2023-10-05T19:40:26.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.839+0000 7f0673d19700 1 -- 172.21.15.12:0/2802956745 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f066c016070 con 0x55ce77549d60 2023-10-05T19:40:26.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.839+0000 7f0673d19700 1 --2- 172.21.15.12:0/2802956745 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f066c01a070 0x7f066c00e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:26.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.839+0000 7f0673d19700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77550fc0 con 0x7f066c01a070 2023-10-05T19:40:26.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.839+0000 7f067451a700 1 --2- 172.21.15.12:0/2802956745 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f066c01a070 0x7f066c00e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:26.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.839+0000 7f067451a700 1 --2- 172.21.15.12:0/2802956745 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f066c01a070 0x7f066c00e3b0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:26.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.843+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0664014040 con 0x7f066c01a070 2023-10-05T19:40:26.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.843+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77550fc0 con 0x55ce7751a3b0 2023-10-05T19:40:26.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.843+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0664014040 con 0x55ce7751a3b0 2023-10-05T19:40:26.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.843+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77550fc0 con 0x55ce7751a3b0 2023-10-05T19:40:26.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.843+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0664014040 con 0x55ce7751a3b0 2023-10-05T19:40:26.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.843+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77550fc0 con 0x7f066c01a070 2023-10-05T19:40:26.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.843+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0664014040 con 0x7f066c01a070 2023-10-05T19:40:26.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.843+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77550fc0 con 0x7f066c01a070 2023-10-05T19:40:26.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.843+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0664014040 con 0x7f066c01a070 2023-10-05T19:40:26.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.843+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77550fc0 con 0x55ce77549d60 2023-10-05T19:40:26.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.843+0000 7f0673d19700 1 -- 172.21.15.12:0/2802956745 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f066c016070 con 0x55ce77549d60 2023-10-05T19:40:26.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.843+0000 7f0673d19700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77550fc0 con 0x55ce7754e500 2023-10-05T19:40:26.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.847+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f065c017050 con 0x55ce7754e500 2023-10-05T19:40:26.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.847+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77550fc0 con 0x55ce7751a3b0 2023-10-05T19:40:26.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.847+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0664014040 con 0x55ce7751a3b0 2023-10-05T19:40:26.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.847+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77550fc0 con 0x7f066c01a070 2023-10-05T19:40:26.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.847+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0664014040 con 0x7f066c01a070 2023-10-05T19:40:26.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.847+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77550fc0 con 0x55ce7751a3b0 2023-10-05T19:40:26.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.847+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0664014040 con 0x55ce7751a3b0 2023-10-05T19:40:26.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.847+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77550fc0 con 0x55ce77549d60 2023-10-05T19:40:26.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.847+0000 7f0673d19700 1 -- 172.21.15.12:0/2802956745 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f066c016070 con 0x55ce77549d60 2023-10-05T19:40:26.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.847+0000 7f0673d19700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77550fc0 con 0x55ce7754e500 2023-10-05T19:40:26.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.851+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f065c017050 con 0x55ce7754e500 2023-10-05T19:40:26.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.851+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77550fc0 con 0x55ce7754e500 2023-10-05T19:40:26.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.851+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f065c017050 con 0x55ce7754e500 2023-10-05T19:40:26.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.851+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77550fc0 con 0x7f066c01a070 2023-10-05T19:40:26.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.851+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0664014040 con 0x7f066c01a070 2023-10-05T19:40:26.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.851+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77550fc0 con 0x55ce7754e500 2023-10-05T19:40:26.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.851+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f065c017050 con 0x55ce7754e500 2023-10-05T19:40:26.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.851+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77550fc0 con 0x55ce7754e500 2023-10-05T19:40:26.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.851+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f065c017050 con 0x55ce7754e500 2023-10-05T19:40:26.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.851+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77550fc0 con 0x55ce7751a3b0 2023-10-05T19:40:26.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.855+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0664014040 con 0x55ce7751a3b0 2023-10-05T19:40:26.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.855+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77550fc0 con 0x55ce7751a3b0 2023-10-05T19:40:26.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.855+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0664014040 con 0x55ce7751a3b0 2023-10-05T19:40:26.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.855+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77550fc0 con 0x7f066c01a070 2023-10-05T19:40:26.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.855+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0664014040 con 0x7f066c01a070 2023-10-05T19:40:26.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.855+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77550fc0 con 0x55ce7754e500 2023-10-05T19:40:26.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.855+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f065c017050 con 0x55ce7754e500 2023-10-05T19:40:26.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.855+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77550fc0 con 0x55ce77549d60 2023-10-05T19:40:26.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.855+0000 7f0673d19700 1 -- 172.21.15.12:0/2802956745 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f066c016070 con 0x55ce77549d60 2023-10-05T19:40:26.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.855+0000 7f0673d19700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77550fc0 con 0x55ce7751a3b0 2023-10-05T19:40:26.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.859+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0664014040 con 0x55ce7751a3b0 2023-10-05T19:40:26.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.859+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77550fc0 con 0x7f066c01a070 2023-10-05T19:40:26.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.859+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0664014040 con 0x7f066c01a070 2023-10-05T19:40:26.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.859+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77550fc0 con 0x55ce7754e500 2023-10-05T19:40:26.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.859+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f065c017050 con 0x55ce7754e500 2023-10-05T19:40:26.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.859+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77550fc0 con 0x7f066c01a070 2023-10-05T19:40:26.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.859+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0664014040 con 0x7f066c01a070 2023-10-05T19:40:26.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.859+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77550fc0 con 0x55ce77549d60 2023-10-05T19:40:26.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.863+0000 7f0673d19700 1 -- 172.21.15.12:0/2802956745 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f066c016070 con 0x55ce77549d60 2023-10-05T19:40:26.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.863+0000 7f0673d19700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77550fc0 con 0x55ce7754e500 2023-10-05T19:40:26.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.863+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f065c017050 con 0x55ce7754e500 2023-10-05T19:40:26.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.863+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77550fc0 con 0x7f066c01a070 2023-10-05T19:40:26.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.863+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0664014040 con 0x7f066c01a070 2023-10-05T19:40:26.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.863+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77550fc0 con 0x55ce7751a3b0 2023-10-05T19:40:26.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.863+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f0664014040 con 0x55ce7751a3b0 2023-10-05T19:40:26.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.863+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77550fc0 con 0x55ce77549d60 2023-10-05T19:40:26.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.863+0000 7f0673d19700 1 -- 172.21.15.12:0/2802956745 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f066c016070 con 0x55ce77549d60 2023-10-05T19:40:26.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.863+0000 7f06761e4e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:26.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.863+0000 7f06761e4e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:26.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.863+0000 7f06761e4e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:26.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.863+0000 7f06761e4e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:26.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.863+0000 7f06761e4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.863+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce7754d280 con 0x55ce7751a3b0 2023-10-05T19:40:26.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.867+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f0664014040 con 0x55ce7751a3b0 2023-10-05T19:40:26.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.867+0000 7f06761e4e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:26.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.867+0000 7f06761e4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.867+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce7754ceb0 con 0x55ce7751a3b0 2023-10-05T19:40:26.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.867+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f0664014040 con 0x55ce7751a3b0 2023-10-05T19:40:26.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.867+0000 7f06761e4e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:26.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.867+0000 7f06761e4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.867+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77551430 con 0x55ce7754e500 2023-10-05T19:40:26.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.867+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f065c017050 con 0x55ce7754e500 2023-10-05T19:40:26.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.867+0000 7f06761e4e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:26.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.867+0000 7f06761e4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.867+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce775517e0 con 0x55ce7754e500 2023-10-05T19:40:26.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.867+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f065c017050 con 0x55ce7754e500 2023-10-05T19:40:26.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.867+0000 7f06761e4e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:26.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.867+0000 7f06761e4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.867+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77551b90 con 0x55ce7751a3b0 2023-10-05T19:40:26.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.867+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f0664014040 con 0x55ce7751a3b0 2023-10-05T19:40:26.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.867+0000 7f06761e4e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:26.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.867+0000 7f06761e4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.867+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce7754d710 con 0x55ce7751a3b0 2023-10-05T19:40:26.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.871+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f0664014040 con 0x55ce7751a3b0 2023-10-05T19:40:26.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.871+0000 7f06761e4e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:26.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.871+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce7754d6e0 con 0x55ce7754e500 2023-10-05T19:40:26.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.871+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f065c017050 con 0x55ce7754e500 2023-10-05T19:40:26.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.871+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce7754d6e0 con 0x55ce77549d60 2023-10-05T19:40:26.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.871+0000 7f0673d19700 1 -- 172.21.15.12:0/2802956745 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f066c016070 con 0x55ce77549d60 2023-10-05T19:40:26.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.871+0000 7f0673d19700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce7754d6e0 con 0x7f066c01a070 2023-10-05T19:40:26.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.875+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0664014040 con 0x7f066c01a070 2023-10-05T19:40:26.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.875+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77551b90 con 0x55ce7751a3b0 2023-10-05T19:40:26.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.875+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0664014040 con 0x55ce7751a3b0 2023-10-05T19:40:26.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.875+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77551b90 con 0x55ce7751a3b0 2023-10-05T19:40:26.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.875+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0664014040 con 0x55ce7751a3b0 2023-10-05T19:40:26.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.875+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77551b90 con 0x7f066c01a070 2023-10-05T19:40:26.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.875+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0664014040 con 0x7f066c01a070 2023-10-05T19:40:26.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.875+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77551b90 con 0x7f066c01a070 2023-10-05T19:40:26.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.879+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0664014040 con 0x7f066c01a070 2023-10-05T19:40:26.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.879+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77551b90 con 0x55ce77549d60 2023-10-05T19:40:26.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.879+0000 7f0673d19700 1 -- 172.21.15.12:0/2802956745 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f066c016070 con 0x55ce77549d60 2023-10-05T19:40:26.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.879+0000 7f0673d19700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77551b90 con 0x55ce7754e500 2023-10-05T19:40:26.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.879+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f065c017050 con 0x55ce7754e500 2023-10-05T19:40:26.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.879+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77551b90 con 0x55ce7751a3b0 2023-10-05T19:40:26.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.879+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0664014040 con 0x55ce7751a3b0 2023-10-05T19:40:26.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.879+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77551b90 con 0x7f066c01a070 2023-10-05T19:40:26.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.879+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0664014040 con 0x7f066c01a070 2023-10-05T19:40:26.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.879+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77551b90 con 0x55ce7751a3b0 2023-10-05T19:40:26.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.883+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0664014040 con 0x55ce7751a3b0 2023-10-05T19:40:26.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.883+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77551b90 con 0x55ce77549d60 2023-10-05T19:40:26.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.883+0000 7f0673d19700 1 -- 172.21.15.12:0/2802956745 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f066c016070 con 0x55ce77549d60 2023-10-05T19:40:26.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.883+0000 7f0673d19700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77551b90 con 0x55ce7754e500 2023-10-05T19:40:26.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.883+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f065c017050 con 0x55ce7754e500 2023-10-05T19:40:26.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.883+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77551b90 con 0x55ce7754e500 2023-10-05T19:40:26.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.883+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f065c017050 con 0x55ce7754e500 2023-10-05T19:40:26.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.883+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77551b90 con 0x7f066c01a070 2023-10-05T19:40:26.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.883+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0664014040 con 0x7f066c01a070 2023-10-05T19:40:26.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.883+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77551b90 con 0x55ce7754e500 2023-10-05T19:40:26.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.887+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f065c017050 con 0x55ce7754e500 2023-10-05T19:40:26.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.887+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77551b90 con 0x55ce7754e500 2023-10-05T19:40:26.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.887+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f065c017050 con 0x55ce7754e500 2023-10-05T19:40:26.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.887+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77551b90 con 0x55ce7751a3b0 2023-10-05T19:40:26.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.887+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0664014040 con 0x55ce7751a3b0 2023-10-05T19:40:26.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.887+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77551b90 con 0x55ce7751a3b0 2023-10-05T19:40:26.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.887+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0664014040 con 0x55ce7751a3b0 2023-10-05T19:40:26.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.887+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77551b90 con 0x7f066c01a070 2023-10-05T19:40:26.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.891+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0664014040 con 0x7f066c01a070 2023-10-05T19:40:26.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.891+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77551b90 con 0x55ce7754e500 2023-10-05T19:40:26.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.891+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f065c017050 con 0x55ce7754e500 2023-10-05T19:40:26.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.891+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77551b90 con 0x55ce77549d60 2023-10-05T19:40:26.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.891+0000 7f0673d19700 1 -- 172.21.15.12:0/2802956745 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f066c016070 con 0x55ce77549d60 2023-10-05T19:40:26.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.891+0000 7f0673d19700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77551b90 con 0x55ce7751a3b0 2023-10-05T19:40:26.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.891+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0664014040 con 0x55ce7751a3b0 2023-10-05T19:40:26.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.891+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77551b90 con 0x7f066c01a070 2023-10-05T19:40:26.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.895+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0664014040 con 0x7f066c01a070 2023-10-05T19:40:26.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.895+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77551b90 con 0x55ce7754e500 2023-10-05T19:40:26.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.895+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f065c017050 con 0x55ce7754e500 2023-10-05T19:40:26.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.895+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77551b90 con 0x7f066c01a070 2023-10-05T19:40:26.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.895+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0664014040 con 0x7f066c01a070 2023-10-05T19:40:26.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.895+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77551b90 con 0x55ce77549d60 2023-10-05T19:40:26.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.895+0000 7f0673d19700 1 -- 172.21.15.12:0/2802956745 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f066c016070 con 0x55ce77549d60 2023-10-05T19:40:26.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.895+0000 7f0673d19700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77551b90 con 0x55ce7754e500 2023-10-05T19:40:26.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.895+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f065c017050 con 0x55ce7754e500 2023-10-05T19:40:26.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.895+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77551b90 con 0x7f066c01a070 2023-10-05T19:40:26.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.899+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0664014040 con 0x7f066c01a070 2023-10-05T19:40:26.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.899+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77551b90 con 0x55ce7751a3b0 2023-10-05T19:40:26.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.899+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f0664014040 con 0x55ce7751a3b0 2023-10-05T19:40:26.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.899+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77551b90 con 0x55ce77549d60 2023-10-05T19:40:26.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.899+0000 7f0673d19700 1 -- 172.21.15.12:0/2802956745 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f066c016070 con 0x55ce77549d60 2023-10-05T19:40:26.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.899+0000 7f06761e4e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:26.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.899+0000 7f06761e4e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:26.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.899+0000 7f06761e4e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:26.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.899+0000 7f06761e4e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:26.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.899+0000 7f06761e4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.899+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce7754d6e0 con 0x55ce7751a3b0 2023-10-05T19:40:26.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.899+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f0664014040 con 0x55ce7751a3b0 2023-10-05T19:40:26.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.899+0000 7f06761e4e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:26.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.899+0000 7f06761e4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.899+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77552480 con 0x55ce77549d60 2023-10-05T19:40:26.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.903+0000 7f0673d19700 1 -- 172.21.15.12:0/2802956745 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f066c016070 con 0x55ce77549d60 2023-10-05T19:40:26.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.903+0000 7f06761e4e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:26.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.903+0000 7f06761e4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.903+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77552d70 con 0x55ce7751a3b0 2023-10-05T19:40:26.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.903+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f0664014040 con 0x55ce7751a3b0 2023-10-05T19:40:26.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.903+0000 7f06761e4e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:26.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.903+0000 7f06761e4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.903+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77552d70 con 0x55ce7754e500 2023-10-05T19:40:26.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.903+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f065c017050 con 0x55ce7754e500 2023-10-05T19:40:26.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.903+0000 7f06761e4e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:26.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.903+0000 7f06761e4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.903+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce775534e0 con 0x7f066c01a070 2023-10-05T19:40:26.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.903+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f0664014040 con 0x7f066c01a070 2023-10-05T19:40:26.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.903+0000 7f06761e4e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:26.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.903+0000 7f06761e4e40 20 zone default found 2023-10-05T19:40:26.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.903+0000 7f06761e4e40 4 Realm: () 2023-10-05T19:40:26.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.903+0000 7f06761e4e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:26.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.903+0000 7f06761e4e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:26.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.903+0000 7f06761e4e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:26.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.903+0000 7f06761e4e40 20 zonegroup default 2023-10-05T19:40:26.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.903+0000 7f06761e4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.903+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77553490 con 0x7f066c01a070 2023-10-05T19:40:26.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.907+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f0664014040 con 0x7f066c01a070 2023-10-05T19:40:26.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.907+0000 7f06761e4e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:26.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.907+0000 7f06761e4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.907+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77554e10 con 0x55ce7754e500 2023-10-05T19:40:26.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.907+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f065c017050 con 0x55ce7754e500 2023-10-05T19:40:26.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.907+0000 7f06761e4e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:26.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.907+0000 7f06761e4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:26.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.907+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce775551c0 con 0x55ce7754e500 2023-10-05T19:40:26.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.907+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f065c017050 con 0x55ce7754e500 2023-10-05T19:40:26.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.907+0000 7f06761e4e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:26.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.907+0000 7f06761e4e40 20 started sync module instance, tier type = 2023-10-05T19:40:26.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.907+0000 7f06761e4e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:26.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.907+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce775544d0 con 0x55ce77549d60 2023-10-05T19:40:26.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.911+0000 7f0673d19700 1 -- 172.21.15.12:0/2802956745 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'129 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f066c016070 con 0x55ce77549d60 2023-10-05T19:40:26.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.911+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94345253693568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77555570 con 0x55ce77549d60 2023-10-05T19:40:26.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.911+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77555e10 con 0x55ce7751a3b0 2023-10-05T19:40:26.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.915+0000 7f0673d19700 1 -- 172.21.15.12:0/2802956745 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94345253693568] v28'130 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f066c016070 con 0x55ce77549d60 2023-10-05T19:40:26.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.915+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'129 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0664014040 con 0x55ce7751a3b0 2023-10-05T19:40:26.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.915+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94345253701840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77555e10 con 0x55ce7751a3b0 2023-10-05T19:40:26.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.915+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77556890 con 0x55ce7754e500 2023-10-05T19:40:26.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.915+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94345253701840] v28'130 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0664014040 con 0x55ce7751a3b0 2023-10-05T19:40:26.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.919+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'129 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f065c017050 con 0x55ce7754e500 2023-10-05T19:40:26.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.919+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94345253706016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77555d70 con 0x55ce7754e500 2023-10-05T19:40:26.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.919+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77557670 con 0x55ce77549d60 2023-10-05T19:40:26.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.919+0000 7f0673d19700 1 -- 172.21.15.12:0/2802956745 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'129 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f066c016070 con 0x55ce77549d60 2023-10-05T19:40:26.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.919+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94345253706016] v28'130 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f065c017050 con 0x55ce7754e500 2023-10-05T19:40:26.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.919+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94345253711280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77557200 con 0x55ce77549d60 2023-10-05T19:40:26.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.919+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77558570 con 0x55ce7751a3b0 2023-10-05T19:40:26.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.923+0000 7f0673d19700 1 -- 172.21.15.12:0/2802956745 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94345253711280] v28'130 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f066c016070 con 0x55ce77549d60 2023-10-05T19:40:26.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.923+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'129 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0664014040 con 0x55ce7751a3b0 2023-10-05T19:40:26.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.923+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94345253714464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77558570 con 0x55ce7751a3b0 2023-10-05T19:40:26.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.923+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77558f80 con 0x7f066c01a070 2023-10-05T19:40:26.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.927+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94345253714464] v28'130 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0664014040 con 0x55ce7751a3b0 2023-10-05T19:40:26.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.927+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'129 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0664014040 con 0x7f066c01a070 2023-10-05T19:40:26.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.927+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94345253716656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77558f80 con 0x7f066c01a070 2023-10-05T19:40:26.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.927+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77559990 con 0x7f066c01a070 2023-10-05T19:40:26.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.927+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(95 notify.5 [watch watch cookie 94345253716656] v28'130 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0664014040 con 0x7f066c01a070 2023-10-05T19:40:26.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.927+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(96 notify.6 [create] v28'129 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0664014040 con 0x7f066c01a070 2023-10-05T19:40:26.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.927+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94345253719232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce77559990 con 0x7f066c01a070 2023-10-05T19:40:26.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.927+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce7755a230 con 0x7f066c01a070 2023-10-05T19:40:26.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.931+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(98 notify.7 [create] v28'129 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0664014040 con 0x7f066c01a070 2023-10-05T19:40:26.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.931+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(97 notify.6 [watch watch cookie 94345253719232] v28'130 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0664014040 con 0x7f066c01a070 2023-10-05T19:40:26.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.931+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94345253723152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce7755a610 con 0x7f066c01a070 2023-10-05T19:40:26.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.931+0000 7f06761e4e40 20 add_watcher() i=0 2023-10-05T19:40:26.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.931+0000 7f06761e4e40 20 add_watcher() i=1 2023-10-05T19:40:26.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.931+0000 7f06761e4e40 20 add_watcher() i=2 2023-10-05T19:40:26.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.931+0000 7f06761e4e40 20 add_watcher() i=3 2023-10-05T19:40:26.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.931+0000 7f06761e4e40 20 add_watcher() i=4 2023-10-05T19:40:26.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.931+0000 7f06761e4e40 20 add_watcher() i=5 2023-10-05T19:40:26.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.931+0000 7f06761e4e40 20 add_watcher() i=6 2023-10-05T19:40:26.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.931+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94345253723152] v28'130 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0664014040 con 0x7f066c01a070 2023-10-05T19:40:26.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.931+0000 7f06761e4e40 20 add_watcher() i=7 2023-10-05T19:40:26.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.931+0000 7f06761e4e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:26.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.931+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce775595c0 con 0x55ce7751a3b0 2023-10-05T19:40:26.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.935+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f0664014040 con 0x55ce7751a3b0 2023-10-05T19:40:26.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.935+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94345253724384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce775595c0 con 0x55ce7751a3b0 2023-10-05T19:40:26.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.935+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94345253724384] v28'127 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f0664014040 con 0x55ce7751a3b0 2023-10-05T19:40:26.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.935+0000 7f06457e2700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:26.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.935+0000 7f06761e4e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:26.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.935+0000 7f06761e4e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:26.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.935+0000 7f06761e4e40 1 Processor -- start 2023-10-05T19:40:26.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.935+0000 7f06761e4e40 1 -- start start 2023-10-05T19:40:26.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.939+0000 7f06761e4e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ce77567c70 0x55ce77568040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:26.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.939+0000 7f06761e4e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce77568580 con 0x55ce77567c70 2023-10-05T19:40:26.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.939+0000 7f0673518700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ce77567c70 0x55ce77568040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:26.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.939+0000 7f0673518700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ce77567c70 0x55ce77568040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58170/0 (socket says 172.21.15.12:58170) 2023-10-05T19:40:26.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.939+0000 7f0673518700 1 -- 172.21.15.12:0/2298638067 learned_addr learned my addr 172.21.15.12:0/2298638067 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:26.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.939+0000 7f0673518700 1 -- 172.21.15.12:0/2298638067 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce775686c0 con 0x55ce77567c70 2023-10-05T19:40:26.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.939+0000 7f0673518700 1 --2- 172.21.15.12:0/2298638067 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ce77567c70 0x55ce77568040 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f065c003000 tx=0x7f065c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2434a57267e1d163 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:26.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.939+0000 7f0644fe1700 1 -- 172.21.15.12:0/2298638067 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f065c010be0 con 0x55ce77567c70 2023-10-05T19:40:26.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.939+0000 7f0644fe1700 1 -- 172.21.15.12:0/2298638067 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f065c010d40 con 0x55ce77567c70 2023-10-05T19:40:26.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.939+0000 7f0644fe1700 1 -- 172.21.15.12:0/2298638067 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f065c028430 con 0x55ce77567c70 2023-10-05T19:40:26.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.939+0000 7f06761e4e40 1 -- 172.21.15.12:0/2298638067 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ce77567c70 msgr2=0x55ce77568040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:26.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.939+0000 7f06761e4e40 1 --2- 172.21.15.12:0/2298638067 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ce77567c70 0x55ce77568040 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f065c003000 tx=0x7f065c010000 comp rx=0 tx=0).stop 2023-10-05T19:40:26.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.939+0000 7f06761e4e40 1 -- 172.21.15.12:0/2298638067 shutdown_connections 2023-10-05T19:40:26.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.939+0000 7f06761e4e40 1 --2- 172.21.15.12:0/2298638067 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ce77567c70 0x55ce77568040 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:26.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.939+0000 7f06761e4e40 1 -- 172.21.15.12:0/2298638067 >> 172.21.15.12:0/2298638067 conn(0x55ce77563030 msgr2=0x55ce77561180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:26.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.939+0000 7f06761e4e40 1 -- 172.21.15.12:0/2298638067 shutdown_connections 2023-10-05T19:40:26.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.939+0000 7f06761e4e40 1 -- 172.21.15.12:0/2298638067 wait complete. 2023-10-05T19:40:26.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.943+0000 7f06761e4e40 1 Processor -- start 2023-10-05T19:40:26.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.943+0000 7f06761e4e40 1 -- start start 2023-10-05T19:40:26.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.943+0000 7f06761e4e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ce77567c70 0x55ce7756d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:26.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.943+0000 7f06761e4e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce7756e720 con 0x55ce77567c70 2023-10-05T19:40:26.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.943+0000 7f0673518700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ce77567c70 0x55ce7756d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:26.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.943+0000 7f0673518700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ce77567c70 0x55ce7756d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58182/0 (socket says 172.21.15.12:58182) 2023-10-05T19:40:26.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.943+0000 7f0673518700 1 -- 172.21.15.12:0/1944759321 learned_addr learned my addr 172.21.15.12:0/1944759321 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:26.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.943+0000 7f0673518700 1 -- 172.21.15.12:0/1944759321 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce775169e0 con 0x55ce77567c70 2023-10-05T19:40:26.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.943+0000 7f0673518700 1 --2- 172.21.15.12:0/1944759321 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ce77567c70 0x55ce7756d720 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x55ce77561cf0 tx=0x7f065c033000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:26.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.943+0000 7f05beffd700 1 -- 172.21.15.12:0/1944759321 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f065c028ce0 con 0x55ce77567c70 2023-10-05T19:40:26.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.943+0000 7f05beffd700 1 -- 172.21.15.12:0/1944759321 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f065c028e40 con 0x55ce77567c70 2023-10-05T19:40:26.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.943+0000 7f06761e4e40 1 -- 172.21.15.12:0/1944759321 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ce7756dd20 con 0x55ce77567c70 2023-10-05T19:40:26.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.943+0000 7f05beffd700 1 -- 172.21.15.12:0/1944759321 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f065c04c470 con 0x55ce77567c70 2023-10-05T19:40:26.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.943+0000 7f06761e4e40 1 -- 172.21.15.12:0/1944759321 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ce7756e160 con 0x55ce77567c70 2023-10-05T19:40:26.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.947+0000 7f05beffd700 1 -- 172.21.15.12:0/1944759321 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f065c049070 con 0x55ce77567c70 2023-10-05T19:40:26.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.947+0000 7f05beffd700 1 --2- 172.21.15.12:0/1944759321 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f05b4030c00 0x7f05b40330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:26.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.947+0000 7f05beffd700 1 -- 172.21.15.12:0/1944759321 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f065c034040 con 0x55ce77567c70 2023-10-05T19:40:26.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.947+0000 7f06761e4e40 5 note: GC not initialized 2023-10-05T19:40:26.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.947+0000 7f067451a700 1 --2- 172.21.15.12:0/1944759321 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f05b4030c00 0x7f05b40330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:26.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.947+0000 7f067451a700 1 --2- 172.21.15.12:0/1944759321 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f05b4030c00 0x7f05b40330b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x55ce7755a570 tx=0x7f066400ab80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:26.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.947+0000 7f05b3fff700 20 reqs_thread_entry: start 2023-10-05T19:40:26.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.947+0000 7f05b1ffb700 20 reqs_thread_entry: start 2023-10-05T19:40:26.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.947+0000 7f06761e4e40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:26.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.947+0000 7f06761e4e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:26.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.947+0000 7f059ffff700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:57 2023 2023-10-05T19:40:26.958 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:26.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.947+0000 7f059f7fe700 20 reqs_thread_entry: start 2023-10-05T19:40:26.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.951+0000 7f06761e4e40 20 get_system_obj_state: rctx=0x7fffe915f7d8 obj=default.rgw.meta:users.uid:testx$tenanteduser state=0x55ce773ef0b0 s->prefetch_data=0 2023-10-05T19:40:26.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.951+0000 7f06761e4e40 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser : miss 2023-10-05T19:40:26.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.951+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:102 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce775d5f70 con 0x7f066c01a070 2023-10-05T19:40:26.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.951+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(102 testx$tenanteduser [call,getxattrs,stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 246+0+0 (crc 0 0 0) 0x7f0664014040 con 0x7f066c01a070 2023-10-05T19:40:26.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.951+0000 7f06761e4e40 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x0 2023-10-05T19:40:26.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.951+0000 7f06761e4e40 10 adding default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2023-10-05T19:40:26.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.951+0000 7f06761e4e40 20 get_system_obj_state: rctx=0x7fffe915f4c8 obj=default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ state=0x55ce773ef0b0 s->prefetch_data=0 2023-10-05T19:40:26.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.951+0000 7f06761e4e40 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : miss 2023-10-05T19:40:26.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.951+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:103 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce775d5f70 con 0x7f066c01a070 2023-10-05T19:40:26.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.951+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(103 p5YnriCv1nAtykxBrupQ [getxattrs,stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 206+0+0 (crc 0 0 0) 0x7f0664014040 con 0x7f066c01a070 2023-10-05T19:40:26.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.951+0000 7f06761e4e40 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x0 2023-10-05T19:40:26.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.951+0000 7f06761e4e40 10 adding default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2023-10-05T19:40:26.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.951+0000 7f06761e4e40 20 get_system_obj_state: rctx=0x7fffe915ed68 obj=default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ state=0x55ce773ef0b0 s->prefetch_data=0 2023-10-05T19:40:26.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.951+0000 7f06761e4e40 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2023-10-05T19:40:26.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.951+0000 7f06761e4e40 20 get_system_obj_state: rctx=0x7fffe915f408 obj=default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ state=0x55ce773ef0b0 s->prefetch_data=0 2023-10-05T19:40:26.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.951+0000 7f06761e4e40 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2023-10-05T19:40:26.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.951+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:104 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [delete,create,call version.set in=58b,writefull 0~385 in=385b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce775d8680 con 0x7f066c01a070 2023-10-05T19:40:26.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.955+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(104 testx$tenanteduser [delete,create,call,writefull 0~385] v28'5 uv5 ondisk = 0) v8 ==== 288+0+0 (crc 0 0 0) 0x7f0664014040 con 0x7f066c01a070 2023-10-05T19:40:26.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.955+0000 7f06761e4e40 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x17 2023-10-05T19:40:26.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.955+0000 7f06761e4e40 10 moving default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2023-10-05T19:40:26.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.955+0000 7f06761e4e40 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.uid:testx$tenanteduser, ofs0, ns] 2023-10-05T19:40:26.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.955+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:105 7.e 7:7759931f:::notify.3:head [notify cookie 94345254237440 in=588b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce775d7e40 con 0x55ce77549d60 2023-10-05T19:40:26.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.955+0000 7f0673d19700 1 -- 172.21.15.12:0/2802956745 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== watch-notify(notify (1) cookie 94345253711280 notify 120259084303 ret 0) v3 ==== 618+0+0 (crc 0 0 0) 0x55ce7753c6c0 con 0x55ce77549d60 2023-10-05T19:40:26.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.955+0000 7f0671d15700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084303 cookie 94345253711280 notifier 4483 bl.length()=576 2023-10-05T19:40:26.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.955+0000 7f0671d15700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x17 2023-10-05T19:40:26.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.955+0000 7f0673d19700 1 -- 172.21.15.12:0/2802956745 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(105 notify.3 [notify cookie 94345254237440 out=8b] v0'0 uv129 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f066c016070 con 0x55ce77549d60 2023-10-05T19:40:26.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.955+0000 7f0671d15700 10 rgw watcher librados: moving default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2023-10-05T19:40:26.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.955+0000 7f0671d15700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:106 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f0658003d90 con 0x55ce77549d60 2023-10-05T19:40:26.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.955+0000 7f0673d19700 1 -- 172.21.15.12:0/2802956745 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== watch-notify(notify_complete (2) cookie 94345254237440 notify 120259084303 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55ce7753c280 con 0x55ce77549d60 2023-10-05T19:40:26.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.955+0000 7f0673d19700 1 -- 172.21.15.12:0/2802956745 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(106 notify.3 [notify-ack] v0'0 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f066c016070 con 0x55ce77549d60 2023-10-05T19:40:26.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.955+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:107 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete,create,writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce775d8630 con 0x7f066c01a070 2023-10-05T19:40:26.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.959+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.0 v2:172.21.15.8:6800/27563 30 ==== osd_op_reply(107 p5YnriCv1nAtykxBrupQ [delete,create,writefull 0~22] v28'6 uv6 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x7f0664014040 con 0x7f066c01a070 2023-10-05T19:40:26.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.959+0000 7f06761e4e40 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2023-10-05T19:40:26.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.959+0000 7f06761e4e40 10 moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2023-10-05T19:40:26.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.959+0000 7f06761e4e40 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2023-10-05T19:40:26.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.959+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:108 7.15 7:a93a5511:::notify.2:head [notify cookie 94345254236752 in=204b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce775d8630 con 0x55ce7754e500 2023-10-05T19:40:26.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.959+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== watch-notify(notify (1) cookie 94345253706016 notify 120259084292 ret 0) v3 ==== 234+0+0 (crc 0 0 0) 0x55ce7756e160 con 0x55ce7754e500 2023-10-05T19:40:26.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.959+0000 7f0671d15700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084292 cookie 94345253706016 notifier 4483 bl.length()=192 2023-10-05T19:40:26.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.959+0000 7f0671d15700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2023-10-05T19:40:26.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.959+0000 7f0671d15700 10 rgw watcher librados: moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2023-10-05T19:40:26.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.959+0000 7f0671d15700 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:109 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f0658003d90 con 0x55ce7754e500 2023-10-05T19:40:26.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.963+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 <== osd.3 v2:172.21.15.12:6816/29951 27 ==== osd_op_reply(108 notify.2 [notify cookie 94345254236752 out=8b] v0'0 uv129 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f065c048030 con 0x55ce7754e500 2023-10-05T19:40:26.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.963+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 <== osd.3 v2:172.21.15.12:6816/29951 28 ==== watch-notify(notify_complete (2) cookie 94345254236752 notify 120259084292 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55ce7756dd20 con 0x55ce7754e500 2023-10-05T19:40:26.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.963+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 <== osd.3 v2:172.21.15.12:6816/29951 29 ==== osd_op_reply(109 notify.2 [notify-ack] v0'0 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f065c048030 con 0x55ce7754e500 2023-10-05T19:40:26.972 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:40:26.972 INFO:teuthology.orchestra.run.smithi012.stdout: "user_id": "testx$tenanteduser", 2023-10-05T19:40:26.972 INFO:teuthology.orchestra.run.smithi012.stdout: "display_name": "tenanted-user", 2023-10-05T19:40:26.972 INFO:teuthology.orchestra.run.smithi012.stdout: "email": "", 2023-10-05T19:40:26.973 INFO:teuthology.orchestra.run.smithi012.stdout: "suspended": 0, 2023-10-05T19:40:26.973 INFO:teuthology.orchestra.run.smithi012.stdout: "max_buckets": 1, 2023-10-05T19:40:26.973 INFO:teuthology.orchestra.run.smithi012.stdout: "subusers": [], 2023-10-05T19:40:26.973 INFO:teuthology.orchestra.run.smithi012.stdout: "keys": [ 2023-10-05T19:40:26.973 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:26.973 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "testx$tenanteduser", 2023-10-05T19:40:26.973 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key": "p5YnriCv1nAtykxBrupQ", 2023-10-05T19:40:26.973 INFO:teuthology.orchestra.run.smithi012.stdout: "secret_key": "Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh" 2023-10-05T19:40:26.973 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:26.974 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:26.974 INFO:teuthology.orchestra.run.smithi012.stdout: "swift_keys": [], 2023-10-05T19:40:26.974 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": [], 2023-10-05T19:40:26.974 INFO:teuthology.orchestra.run.smithi012.stdout: "op_mask": "read, write, delete", 2023-10-05T19:40:26.974 INFO:teuthology.orchestra.run.smithi012.stdout: "default_placement": "", 2023-10-05T19:40:26.974 INFO:teuthology.orchestra.run.smithi012.stdout: "default_storage_class": "", 2023-10-05T19:40:26.974 INFO:teuthology.orchestra.run.smithi012.stdout: "placement_tags": [], 2023-10-05T19:40:26.974 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket_quota": { 2023-10-05T19:40:26.974 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:26.974 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:26.975 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:26.975 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:26.975 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:26.975 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:26.975 INFO:teuthology.orchestra.run.smithi012.stdout: "user_quota": { 2023-10-05T19:40:26.975 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:26.975 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:26.975 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:26.975 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:26.975 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:26.976 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:26.976 INFO:teuthology.orchestra.run.smithi012.stdout: "temp_url_keys": [], 2023-10-05T19:40:26.976 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "rgw", 2023-10-05T19:40:26.976 INFO:teuthology.orchestra.run.smithi012.stdout: "mfa_ids": [] 2023-10-05T19:40:26.976 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:40:26.976 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:40:26.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.963+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:110 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94345253693568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce775d8630 con 0x55ce77549d60 2023-10-05T19:40:26.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.967+0000 7f0673d19700 1 -- 172.21.15.12:0/2802956745 <== osd.2 v2:172.21.15.12:6800/29934 23 ==== osd_op_reply(110 notify.0 [watch unwatch cookie 94345253693568] v28'131 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f066c016070 con 0x55ce77549d60 2023-10-05T19:40:26.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.967+0000 7f06761e4e40 20 remove_watcher() i=0 2023-10-05T19:40:26.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.967+0000 7f06761e4e40 2 removed watcher, disabling cache 2023-10-05T19:40:26.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.967+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:111 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94345253701840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce7752b810 con 0x55ce7751a3b0 2023-10-05T19:40:26.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.971+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(111 notify.1 [watch unwatch cookie 94345253701840] v28'131 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0664014040 con 0x55ce7751a3b0 2023-10-05T19:40:26.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.971+0000 7f06761e4e40 20 remove_watcher() i=1 2023-10-05T19:40:26.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.971+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:112 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94345253706016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce7752b810 con 0x55ce7754e500 2023-10-05T19:40:26.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.971+0000 7f0673518700 1 -- 172.21.15.12:0/2802956745 <== osd.3 v2:172.21.15.12:6816/29951 30 ==== osd_op_reply(112 notify.2 [watch unwatch cookie 94345253706016] v28'131 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f065c048030 con 0x55ce7754e500 2023-10-05T19:40:26.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.971+0000 7f06761e4e40 20 remove_watcher() i=2 2023-10-05T19:40:26.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.971+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:113 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94345253711280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce7752b810 con 0x55ce77549d60 2023-10-05T19:40:26.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.975+0000 7f0673d19700 1 -- 172.21.15.12:0/2802956745 <== osd.2 v2:172.21.15.12:6800/29934 24 ==== osd_op_reply(113 notify.3 [watch unwatch cookie 94345253711280] v28'131 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f066c016070 con 0x55ce77549d60 2023-10-05T19:40:26.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.975+0000 7f06761e4e40 20 remove_watcher() i=3 2023-10-05T19:40:26.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.975+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:114 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94345253714464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce7752b810 con 0x55ce7751a3b0 2023-10-05T19:40:26.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.979+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(114 notify.4 [watch unwatch cookie 94345253714464] v28'131 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0664014040 con 0x55ce7751a3b0 2023-10-05T19:40:26.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.979+0000 7f06761e4e40 20 remove_watcher() i=4 2023-10-05T19:40:26.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.979+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:115 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94345253716656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce7752b810 con 0x7f066c01a070 2023-10-05T19:40:26.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.979+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.0 v2:172.21.15.8:6800/27563 31 ==== osd_op_reply(115 notify.5 [watch unwatch cookie 94345253716656] v28'131 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0664014040 con 0x7f066c01a070 2023-10-05T19:40:26.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.979+0000 7f06761e4e40 20 remove_watcher() i=5 2023-10-05T19:40:26.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.979+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:116 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94345253719232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce7752b810 con 0x7f066c01a070 2023-10-05T19:40:26.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.983+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.0 v2:172.21.15.8:6800/27563 32 ==== osd_op_reply(116 notify.6 [watch unwatch cookie 94345253719232] v28'131 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0664014040 con 0x7f066c01a070 2023-10-05T19:40:26.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.983+0000 7f06761e4e40 20 remove_watcher() i=6 2023-10-05T19:40:26.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.983+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:117 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94345253723152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce7752b810 con 0x7f066c01a070 2023-10-05T19:40:26.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.987+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.0 v2:172.21.15.8:6800/27563 33 ==== osd_op_reply(117 notify.7 [watch unwatch cookie 94345253723152] v28'131 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0664014040 con 0x7f066c01a070 2023-10-05T19:40:26.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.987+0000 7f06761e4e40 20 remove_watcher() i=7 2023-10-05T19:40:26.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.991+0000 7f06457e2700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:26.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.991+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:118 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94345253724384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55ce7752b810 con 0x55ce7751a3b0 2023-10-05T19:40:27.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.991+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(118 data_loggenerations_metadata [watch unwatch cookie 94345253724384] v28'128 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f0664014040 con 0x55ce7751a3b0 2023-10-05T19:40:27.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.991+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f066c01a070 msgr2=0x7f066c00e3b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.991+0000 7f06761e4e40 1 --2- 172.21.15.12:0/2802956745 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f066c01a070 0x7f066c00e3b0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:27.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.991+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55ce7751a3b0 msgr2=0x55ce77547830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.991+0000 7f06761e4e40 1 --2- 172.21.15.12:0/2802956745 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55ce7751a3b0 0x55ce77547830 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:27.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.991+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55ce77549d60 msgr2=0x55ce7754c140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.991+0000 7f06761e4e40 1 --2- 172.21.15.12:0/2802956745 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55ce77549d60 0x55ce7754c140 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:27.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.991+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55ce7754e500 msgr2=0x55ce77550910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.991+0000 7f06761e4e40 1 --2- 172.21.15.12:0/2802956745 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55ce7754e500 0x55ce77550910 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:27.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.991+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f0654030c00 msgr2=0x7f06540330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.991+0000 7f06761e4e40 1 --2- 172.21.15.12:0/2802956745 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f0654030c00 0x7f06540330b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f065c005e40 tx=0x7f065c013000 comp rx=0 tx=0).stop 2023-10-05T19:40:27.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.991+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ce77515dd0 msgr2=0x55ce7753bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.991+0000 7f067451a700 1 -- 172.21.15.12:0/2802956745 reap_dead start 2023-10-05T19:40:27.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.991+0000 7f06761e4e40 1 --2- 172.21.15.12:0/2802956745 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ce77515dd0 0x55ce7753bd40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f066c000c00 tx=0x7f066c016000 comp rx=0 tx=0).stop 2023-10-05T19:40:27.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.991+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 shutdown_connections 2023-10-05T19:40:27.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.991+0000 7f06761e4e40 1 --2- 172.21.15.12:0/2802956745 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ce77515dd0 0x55ce7753bd40 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:27.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.991+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 >> 172.21.15.12:0/2802956745 conn(0x55ce775117f0 msgr2=0x55ce7752f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:27.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.995+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 shutdown_connections 2023-10-05T19:40:27.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.995+0000 7f06761e4e40 1 -- 172.21.15.12:0/2802956745 wait complete. 2023-10-05T19:40:27.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.995+0000 7f06761e4e40 1 -- 172.21.15.12:0/1944759321 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f05b4030c00 msgr2=0x7f05b40330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.995+0000 7f06761e4e40 1 --2- 172.21.15.12:0/1944759321 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f05b4030c00 0x7f05b40330b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x55ce7755a570 tx=0x7f066400ab80 comp rx=0 tx=0).stop 2023-10-05T19:40:27.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.995+0000 7f06761e4e40 1 -- 172.21.15.12:0/1944759321 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ce77567c70 msgr2=0x55ce7756d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.995+0000 7f06761e4e40 1 --2- 172.21.15.12:0/1944759321 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ce77567c70 0x55ce7756d720 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x55ce77561cf0 tx=0x7f065c033000 comp rx=0 tx=0).stop 2023-10-05T19:40:27.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.995+0000 7f06761e4e40 1 -- 172.21.15.12:0/1944759321 shutdown_connections 2023-10-05T19:40:27.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.995+0000 7f06761e4e40 1 --2- 172.21.15.12:0/1944759321 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f05b4030c00 0x7f05b40330b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:27.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.995+0000 7f06761e4e40 1 --2- 172.21.15.12:0/1944759321 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ce77567c70 0x55ce7756d720 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:27.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.995+0000 7f06761e4e40 1 -- 172.21.15.12:0/1944759321 >> 172.21.15.12:0/1944759321 conn(0x55ce77563030 msgr2=0x55ce775610d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:27.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.995+0000 7f06761e4e40 1 -- 172.21.15.12:0/1944759321 shutdown_connections 2023-10-05T19:40:27.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:26.995+0000 7f06761e4e40 1 -- 172.21.15.12:0/1944759321 wait complete. 2023-10-05T19:40:27.009 DEBUG:tasks.util.rgw: json result: {'user_id': 'testx$tenanteduser', 'display_name': 'tenanted-user', 'email': '', 'suspended': 0, 'max_buckets': 1, 'subusers': [], 'keys': [{'user': 'testx$tenanteduser', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:40:27.010 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'link', '--bucket', '/myfoo', '--tenant', 'testx', '--uid', 'tenanteduser'] 2023-10-05T19:40:27.010 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'link', '--bucket', '/myfoo', '--tenant', 'testx', '--uid', 'tenanteduser'] 2023-10-05T19:40:27.010 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket link --bucket /myfoo --tenant testx --uid tenanteduser 2023-10-05T19:40:27.054 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:27.054 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:27.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.047+0000 7fc6bcde7e40 1 Processor -- start 2023-10-05T19:40:27.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.047+0000 7fc6bcde7e40 1 -- start start 2023-10-05T19:40:27.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.047+0000 7fc6bcde7e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x558a60f8c3b0 0x558a60f8e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.047+0000 7fc6bcde7e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x558a60ed4bd0 con 0x558a60f87dd0 2023-10-05T19:40:27.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.047+0000 7fc6bcde7e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x558a60e91ee0 con 0x558a60f8c3b0 2023-10-05T19:40:27.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.047+0000 7fc6ba11b700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x558a60f8c3b0 0x558a60f8e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.047+0000 7fc6ba91c700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x558a60f87dd0 0x558a60f881a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:35608/0 (socket says 172.21.15.12:35608) 2023-10-05T19:40:27.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.047+0000 7fc6ba11b700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x558a60f8c3b0 0x558a60f8e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58192/0 (socket says 172.21.15.12:58192) 2023-10-05T19:40:27.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.047+0000 7fc6ba91c700 1 -- 172.21.15.12:0/4290972466 learned_addr learned my addr 172.21.15.12:0/4290972466 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:27.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.047+0000 7fc6ba11b700 1 -- 172.21.15.12:0/4290972466 >> v1:172.21.15.8:6789/0 conn(0x558a60f87dd0 legacy=0x558a60f881a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.047+0000 7fc6ba11b700 1 -- 172.21.15.12:0/4290972466 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a60f889e0 con 0x558a60f8c3b0 2023-10-05T19:40:27.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.047+0000 7fc6ba11b700 1 --2- 172.21.15.12:0/4290972466 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558a60f8c3b0 0x558a60f8e790 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc6a400a520 tx=0x7fc6a4005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bc7a2102fa3a86b server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.047+0000 7fc6b991a700 1 -- 172.21.15.12:0/4290972466 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6a400f590 con 0x558a60f8c3b0 2023-10-05T19:40:27.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.047+0000 7fc6b991a700 1 -- 172.21.15.12:0/4290972466 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6a400f6f0 con 0x558a60f8c3b0 2023-10-05T19:40:27.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.047+0000 7fc6b991a700 1 -- 172.21.15.12:0/4290972466 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6a400f880 con 0x558a60f8c3b0 2023-10-05T19:40:27.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.047+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/4290972466 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558a60f8c3b0 msgr2=0x558a60f8e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.047+0000 7fc6bcde7e40 1 --2- 172.21.15.12:0/4290972466 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558a60f8c3b0 0x558a60f8e790 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc6a400a520 tx=0x7fc6a4005d40 comp rx=0 tx=0).stop 2023-10-05T19:40:27.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.047+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/4290972466 shutdown_connections 2023-10-05T19:40:27.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.047+0000 7fc6bcde7e40 1 --2- 172.21.15.12:0/4290972466 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558a60f8c3b0 0x558a60f8e790 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:27.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.047+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/4290972466 >> 172.21.15.12:0/4290972466 conn(0x558a60f837f0 msgr2=0x558a60f85c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:27.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.047+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/4290972466 shutdown_connections 2023-10-05T19:40:27.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.047+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/4290972466 wait complete. 2023-10-05T19:40:27.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.051+0000 7fc6bcde7e40 1 Processor -- start 2023-10-05T19:40:27.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.051+0000 7fc6bcde7e40 1 -- start start 2023-10-05T19:40:27.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.051+0000 7fc6bcde7e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558a60f87dd0 0x558a60efd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.051+0000 7fc6bcde7e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a60e55e30 con 0x558a60f87dd0 2023-10-05T19:40:27.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.051+0000 7fc6ba91c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558a60f87dd0 0x558a60efd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.051+0000 7fc6ba91c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558a60f87dd0 0x558a60efd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58202/0 (socket says 172.21.15.12:58202) 2023-10-05T19:40:27.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.051+0000 7fc6ba91c700 1 -- 172.21.15.12:0/3033769492 learned_addr learned my addr 172.21.15.12:0/3033769492 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:27.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.051+0000 7fc6ba91c700 1 -- 172.21.15.12:0/3033769492 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a400a0e0 con 0x558a60f87dd0 2023-10-05T19:40:27.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.051+0000 7fc6ba91c700 1 --2- 172.21.15.12:0/3033769492 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558a60f87dd0 0x558a60efd880 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac001bf0 tx=0x7fc6ac0017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=502b55a5d95c0cd1 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.051+0000 7fc6b8918700 1 -- 172.21.15.12:0/3033769492 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6ac00bbf0 con 0x558a60f87dd0 2023-10-05T19:40:27.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.051+0000 7fc6b8918700 1 -- 172.21.15.12:0/3033769492 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6ac00bd50 con 0x558a60f87dd0 2023-10-05T19:40:27.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.051+0000 7fc6b8918700 1 -- 172.21.15.12:0/3033769492 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6ac00e470 con 0x558a60f87dd0 2023-10-05T19:40:27.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.051+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/3033769492 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558a60f87dd0 msgr2=0x558a60efd880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.051+0000 7fc6bcde7e40 1 --2- 172.21.15.12:0/3033769492 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558a60f87dd0 0x558a60efd880 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac001bf0 tx=0x7fc6ac0017a0 comp rx=0 tx=0).stop 2023-10-05T19:40:27.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.051+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/3033769492 shutdown_connections 2023-10-05T19:40:27.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.051+0000 7fc6bcde7e40 1 --2- 172.21.15.12:0/3033769492 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558a60f87dd0 0x558a60efd880 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:27.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.051+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/3033769492 >> 172.21.15.12:0/3033769492 conn(0x558a60f837f0 msgr2=0x558a60fa2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:27.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.051+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/3033769492 shutdown_connections 2023-10-05T19:40:27.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.051+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/3033769492 wait complete. 2023-10-05T19:40:27.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.051+0000 7fc6bcde7e40 1 Processor -- start 2023-10-05T19:40:27.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.051+0000 7fc6bcde7e40 1 -- start start 2023-10-05T19:40:27.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.055+0000 7fc6bcde7e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558a60f87dd0 0x558a60fafef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.055+0000 7fc6bcde7e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a60e28ed0 con 0x558a60f87dd0 2023-10-05T19:40:27.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.055+0000 7fc6ba91c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558a60f87dd0 0x558a60fafef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.055+0000 7fc6ba91c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558a60f87dd0 0x558a60fafef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58212/0 (socket says 172.21.15.12:58212) 2023-10-05T19:40:27.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.055+0000 7fc6ba91c700 1 -- 172.21.15.12:0/2531636316 learned_addr learned my addr 172.21.15.12:0/2531636316 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:27.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.055+0000 7fc6ba91c700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a60ed4bd0 con 0x558a60f87dd0 2023-10-05T19:40:27.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.055+0000 7fc6ba91c700 1 --2- 172.21.15.12:0/2531636316 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558a60f87dd0 0x558a60fafef0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x558a60fa3580 tx=0x7fc6ac016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.055+0000 7fc6b2ffd700 1 -- 172.21.15.12:0/2531636316 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6ac01f070 con 0x558a60f87dd0 2023-10-05T19:40:27.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.055+0000 7fc6b2ffd700 1 -- 172.21.15.12:0/2531636316 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6ac01a2e0 con 0x558a60f87dd0 2023-10-05T19:40:27.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.055+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558a60fb0430 con 0x558a60f87dd0 2023-10-05T19:40:27.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.055+0000 7fc6b2ffd700 1 -- 172.21.15.12:0/2531636316 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6ac00bec0 con 0x558a60f87dd0 2023-10-05T19:40:27.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.055+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558a60fb0870 con 0x558a60f87dd0 2023-10-05T19:40:27.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.055+0000 7fc6bcde7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.059+0000 7fc6b2ffd700 1 -- 172.21.15.12:0/2531636316 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fc6ac01a960 con 0x558a60f87dd0 2023-10-05T19:40:27.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.059+0000 7fc6b2ffd700 1 --2- 172.21.15.12:0/2531636316 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fc69c030c50 0x7fc69c033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.059+0000 7fc6b2ffd700 1 -- 172.21.15.12:0/2531636316 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7fc6ac044f60 con 0x558a60f87dd0 2023-10-05T19:40:27.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.059+0000 7fc6ba11b700 1 --2- 172.21.15.12:0/2531636316 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fc69c030c50 0x7fc69c033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.059+0000 7fc6bcde7e40 1 --2- 172.21.15.12:0/2531636316 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x558a60f8c3b0 0x558a60fbb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.059+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fbc020 con 0x558a60f8c3b0 2023-10-05T19:40:27.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.059+0000 7fc6bb11d700 1 --2- 172.21.15.12:0/2531636316 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x558a60f8c3b0 0x558a60fbb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.059+0000 7fc6ba11b700 1 --2- 172.21.15.12:0/2531636316 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fc69c030c50 0x7fc69c033100 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4005d80 tx=0x7fc6a4014000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.059+0000 7fc6bb11d700 1 --2- 172.21.15.12:0/2531636316 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x558a60f8c3b0 0x558a60fbb9b0 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.059+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fc6b400a500 con 0x558a60f8c3b0 2023-10-05T19:40:27.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.059+0000 7fc6bcde7e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:27.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.059+0000 7fc6bcde7e40 20 realm 2023-10-05T19:40:27.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.059+0000 7fc6bcde7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.059+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fb9010 con 0x558a60f8c3b0 2023-10-05T19:40:27.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.059+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fc6b400a500 con 0x558a60f8c3b0 2023-10-05T19:40:27.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.059+0000 7fc6bcde7e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:27.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.059+0000 7fc6bcde7e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:27.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.059+0000 7fc6bcde7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.059+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fbcb00 con 0x558a60f8c3b0 2023-10-05T19:40:27.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.059+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fc6b400a500 con 0x558a60f8c3b0 2023-10-05T19:40:27.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.059+0000 7fc6bcde7e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:27.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.059+0000 7fc6bcde7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.059+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fbc710 con 0x558a60f8c3b0 2023-10-05T19:40:27.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.063+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fc6b400a500 con 0x558a60f8c3b0 2023-10-05T19:40:27.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.063+0000 7fc6bcde7e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:27.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.063+0000 7fc6bcde7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.063+0000 7fc6bcde7e40 1 --2- 172.21.15.12:0/2531636316 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x558a60fbdea0 0x558a60fc0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.063+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc07c0 con 0x558a60fbdea0 2023-10-05T19:40:27.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.063+0000 7fc6ba91c700 1 --2- 172.21.15.12:0/2531636316 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x558a60fbdea0 0x558a60fc0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.063+0000 7fc6ba91c700 1 --2- 172.21.15.12:0/2531636316 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x558a60fbdea0 0x558a60fc0280 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.063+0000 7fc6ba91c700 1 -- 172.21.15.12:0/2531636316 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fc6ac014070 con 0x558a60fbdea0 2023-10-05T19:40:27.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.063+0000 7fc6bcde7e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:27.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.063+0000 7fc6bcde7e40 20 searching for the correct realm 2023-10-05T19:40:27.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.063+0000 7fc6bcde7e40 1 --2- 172.21.15.12:0/2531636316 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x558a60fc2640 0x558a60fc4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.063+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc5100 con 0x558a60fc2640 2023-10-05T19:40:27.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.063+0000 7fc6ba11b700 1 --2- 172.21.15.12:0/2531636316 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x558a60fc2640 0x558a60fc4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.063+0000 7fc6ba11b700 1 --2- 172.21.15.12:0/2531636316 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x558a60fc2640 0x558a60fc4a50 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.067+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fc6a400fcc0 con 0x558a60fc2640 2023-10-05T19:40:27.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.067+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc5100 con 0x558a60fbdea0 2023-10-05T19:40:27.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.067+0000 7fc6ba91c700 1 -- 172.21.15.12:0/2531636316 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6ac014070 con 0x558a60fbdea0 2023-10-05T19:40:27.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.067+0000 7fc6ba91c700 1 --2- 172.21.15.12:0/2531636316 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fc6ac02d020 0x7fc6ac03d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.067+0000 7fc6ba91c700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc5100 con 0x7fc6ac02d020 2023-10-05T19:40:27.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.067+0000 7fc6bb11d700 1 --2- 172.21.15.12:0/2531636316 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fc6ac02d020 0x7fc6ac03d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.067+0000 7fc6bb11d700 1 --2- 172.21.15.12:0/2531636316 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fc6ac02d020 0x7fc6ac03d8d0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.067+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6b4014040 con 0x7fc6ac02d020 2023-10-05T19:40:27.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.067+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc5100 con 0x558a60f8c3b0 2023-10-05T19:40:27.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.071+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6b4014040 con 0x558a60f8c3b0 2023-10-05T19:40:27.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.071+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc5100 con 0x558a60f8c3b0 2023-10-05T19:40:27.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.071+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6b4014040 con 0x558a60f8c3b0 2023-10-05T19:40:27.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.071+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc5100 con 0x7fc6ac02d020 2023-10-05T19:40:27.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.071+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6b4014040 con 0x7fc6ac02d020 2023-10-05T19:40:27.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.071+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc5100 con 0x7fc6ac02d020 2023-10-05T19:40:27.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.071+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6b4014040 con 0x7fc6ac02d020 2023-10-05T19:40:27.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.071+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc5100 con 0x558a60fbdea0 2023-10-05T19:40:27.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.071+0000 7fc6ba91c700 1 -- 172.21.15.12:0/2531636316 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6ac014070 con 0x558a60fbdea0 2023-10-05T19:40:27.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.071+0000 7fc6ba91c700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc5100 con 0x558a60fc2640 2023-10-05T19:40:27.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.075+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6a400fcc0 con 0x558a60fc2640 2023-10-05T19:40:27.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.075+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc5100 con 0x558a60f8c3b0 2023-10-05T19:40:27.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.075+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6b4014040 con 0x558a60f8c3b0 2023-10-05T19:40:27.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.075+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc5100 con 0x7fc6ac02d020 2023-10-05T19:40:27.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.075+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6b4014040 con 0x7fc6ac02d020 2023-10-05T19:40:27.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.075+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc5100 con 0x558a60f8c3b0 2023-10-05T19:40:27.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.075+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6b4014040 con 0x558a60f8c3b0 2023-10-05T19:40:27.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.075+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc5100 con 0x558a60fbdea0 2023-10-05T19:40:27.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.075+0000 7fc6ba91c700 1 -- 172.21.15.12:0/2531636316 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6ac014070 con 0x558a60fbdea0 2023-10-05T19:40:27.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.075+0000 7fc6ba91c700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc5100 con 0x558a60fc2640 2023-10-05T19:40:27.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.075+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6a400fcc0 con 0x558a60fc2640 2023-10-05T19:40:27.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.075+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc5100 con 0x558a60fc2640 2023-10-05T19:40:27.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.079+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6a400fcc0 con 0x558a60fc2640 2023-10-05T19:40:27.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.079+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc5100 con 0x7fc6ac02d020 2023-10-05T19:40:27.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.079+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6b4014040 con 0x7fc6ac02d020 2023-10-05T19:40:27.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.079+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc5100 con 0x558a60fc2640 2023-10-05T19:40:27.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.079+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6a400fcc0 con 0x558a60fc2640 2023-10-05T19:40:27.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.079+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc5100 con 0x558a60fc2640 2023-10-05T19:40:27.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.079+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6a400fcc0 con 0x558a60fc2640 2023-10-05T19:40:27.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.079+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc5100 con 0x558a60f8c3b0 2023-10-05T19:40:27.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.083+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6b4014040 con 0x558a60f8c3b0 2023-10-05T19:40:27.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.083+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc5100 con 0x558a60f8c3b0 2023-10-05T19:40:27.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.083+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6b4014040 con 0x558a60f8c3b0 2023-10-05T19:40:27.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.083+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc5100 con 0x7fc6ac02d020 2023-10-05T19:40:27.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.083+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6b4014040 con 0x7fc6ac02d020 2023-10-05T19:40:27.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.083+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc5100 con 0x558a60fc2640 2023-10-05T19:40:27.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.083+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6a400fcc0 con 0x558a60fc2640 2023-10-05T19:40:27.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.083+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc5100 con 0x558a60fbdea0 2023-10-05T19:40:27.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.083+0000 7fc6ba91c700 1 -- 172.21.15.12:0/2531636316 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6ac014070 con 0x558a60fbdea0 2023-10-05T19:40:27.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.087+0000 7fc6ba91c700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc5100 con 0x558a60f8c3b0 2023-10-05T19:40:27.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.087+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6b4014040 con 0x558a60f8c3b0 2023-10-05T19:40:27.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.087+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc5100 con 0x7fc6ac02d020 2023-10-05T19:40:27.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.087+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6b4014040 con 0x7fc6ac02d020 2023-10-05T19:40:27.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.087+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc5100 con 0x558a60fc2640 2023-10-05T19:40:27.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.087+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6a400fcc0 con 0x558a60fc2640 2023-10-05T19:40:27.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.087+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc5100 con 0x7fc6ac02d020 2023-10-05T19:40:27.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.087+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6b4014040 con 0x7fc6ac02d020 2023-10-05T19:40:27.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.087+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc5100 con 0x558a60fbdea0 2023-10-05T19:40:27.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.091+0000 7fc6ba91c700 1 -- 172.21.15.12:0/2531636316 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6ac014070 con 0x558a60fbdea0 2023-10-05T19:40:27.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.091+0000 7fc6ba91c700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc5100 con 0x558a60fc2640 2023-10-05T19:40:27.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.091+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6a400fcc0 con 0x558a60fc2640 2023-10-05T19:40:27.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.091+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc5100 con 0x7fc6ac02d020 2023-10-05T19:40:27.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.091+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6b4014040 con 0x7fc6ac02d020 2023-10-05T19:40:27.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.091+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc5100 con 0x558a60f8c3b0 2023-10-05T19:40:27.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.091+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fc6b4014040 con 0x558a60f8c3b0 2023-10-05T19:40:27.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.091+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc5100 con 0x558a60fbdea0 2023-10-05T19:40:27.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.095+0000 7fc6ba91c700 1 -- 172.21.15.12:0/2531636316 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fc6ac014070 con 0x558a60fbdea0 2023-10-05T19:40:27.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.095+0000 7fc6bcde7e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:27.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.095+0000 7fc6bcde7e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:27.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.095+0000 7fc6bcde7e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:27.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.095+0000 7fc6bcde7e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:27.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.095+0000 7fc6bcde7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.095+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc13c0 con 0x558a60f8c3b0 2023-10-05T19:40:27.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.095+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fc6b4014040 con 0x558a60f8c3b0 2023-10-05T19:40:27.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.095+0000 7fc6bcde7e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:27.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.095+0000 7fc6bcde7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.095+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc0ff0 con 0x558a60f8c3b0 2023-10-05T19:40:27.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.095+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fc6b4014040 con 0x558a60f8c3b0 2023-10-05T19:40:27.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.095+0000 7fc6bcde7e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:27.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.095+0000 7fc6bcde7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.095+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc5570 con 0x558a60fc2640 2023-10-05T19:40:27.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.095+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fc6a400fcc0 con 0x558a60fc2640 2023-10-05T19:40:27.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.095+0000 7fc6bcde7e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:27.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.095+0000 7fc6bcde7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.095+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc5920 con 0x558a60fc2640 2023-10-05T19:40:27.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.095+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7fc6a400fcc0 con 0x558a60fc2640 2023-10-05T19:40:27.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.095+0000 7fc6bcde7e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:27.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.095+0000 7fc6bcde7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.095+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc5cd0 con 0x558a60f8c3b0 2023-10-05T19:40:27.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.099+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fc6b4014040 con 0x558a60f8c3b0 2023-10-05T19:40:27.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.099+0000 7fc6bcde7e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:27.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.099+0000 7fc6bcde7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.099+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc60e0 con 0x558a60f8c3b0 2023-10-05T19:40:27.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.099+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fc6b4014040 con 0x558a60f8c3b0 2023-10-05T19:40:27.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.099+0000 7fc6bcde7e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:27.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.099+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc60b0 con 0x558a60fc2640 2023-10-05T19:40:27.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.099+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fc6a400fcc0 con 0x558a60fc2640 2023-10-05T19:40:27.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.099+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc60b0 con 0x558a60fbdea0 2023-10-05T19:40:27.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.103+0000 7fc6ba91c700 1 -- 172.21.15.12:0/2531636316 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6ac014070 con 0x558a60fbdea0 2023-10-05T19:40:27.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.103+0000 7fc6ba91c700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc60b0 con 0x7fc6ac02d020 2023-10-05T19:40:27.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.103+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6b4014040 con 0x7fc6ac02d020 2023-10-05T19:40:27.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.103+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc5cd0 con 0x558a60f8c3b0 2023-10-05T19:40:27.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.103+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6b4014040 con 0x558a60f8c3b0 2023-10-05T19:40:27.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.103+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc5cd0 con 0x558a60f8c3b0 2023-10-05T19:40:27.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.103+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6b4014040 con 0x558a60f8c3b0 2023-10-05T19:40:27.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.103+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc5cd0 con 0x7fc6ac02d020 2023-10-05T19:40:27.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.103+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6b4014040 con 0x7fc6ac02d020 2023-10-05T19:40:27.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.107+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc5cd0 con 0x7fc6ac02d020 2023-10-05T19:40:27.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.107+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6b4014040 con 0x7fc6ac02d020 2023-10-05T19:40:27.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.107+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc5cd0 con 0x558a60fbdea0 2023-10-05T19:40:27.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.107+0000 7fc6ba91c700 1 -- 172.21.15.12:0/2531636316 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6ac014070 con 0x558a60fbdea0 2023-10-05T19:40:27.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.107+0000 7fc6ba91c700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc6ac048040 con 0x558a60fc2640 2023-10-05T19:40:27.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.107+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6a400fcc0 con 0x558a60fc2640 2023-10-05T19:40:27.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.107+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc6a4039040 con 0x558a60f8c3b0 2023-10-05T19:40:27.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.107+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6b4014040 con 0x558a60f8c3b0 2023-10-05T19:40:27.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.107+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc6a4039040 con 0x7fc6ac02d020 2023-10-05T19:40:27.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.111+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6b4014040 con 0x7fc6ac02d020 2023-10-05T19:40:27.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.111+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc6a4039040 con 0x558a60f8c3b0 2023-10-05T19:40:27.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.111+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6b4014040 con 0x558a60f8c3b0 2023-10-05T19:40:27.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.111+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc6a4039040 con 0x558a60fbdea0 2023-10-05T19:40:27.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.111+0000 7fc6ba91c700 1 -- 172.21.15.12:0/2531636316 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6ac014070 con 0x558a60fbdea0 2023-10-05T19:40:27.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.111+0000 7fc6ba91c700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc6a4039040 con 0x558a60fc2640 2023-10-05T19:40:27.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.111+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6a400fcc0 con 0x558a60fc2640 2023-10-05T19:40:27.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.111+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc6a4039040 con 0x558a60fc2640 2023-10-05T19:40:27.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.111+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6a400fcc0 con 0x558a60fc2640 2023-10-05T19:40:27.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.115+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc6a4039040 con 0x7fc6ac02d020 2023-10-05T19:40:27.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.115+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6b4014040 con 0x7fc6ac02d020 2023-10-05T19:40:27.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.115+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc6a4039040 con 0x558a60fc2640 2023-10-05T19:40:27.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.115+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6a400fcc0 con 0x558a60fc2640 2023-10-05T19:40:27.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.115+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc6a4039040 con 0x558a60fc2640 2023-10-05T19:40:27.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.115+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6a400fcc0 con 0x558a60fc2640 2023-10-05T19:40:27.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.115+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc6a4039040 con 0x558a60f8c3b0 2023-10-05T19:40:27.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.115+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6b4014040 con 0x558a60f8c3b0 2023-10-05T19:40:27.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.115+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc6a4039040 con 0x558a60f8c3b0 2023-10-05T19:40:27.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.119+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6b4014040 con 0x558a60f8c3b0 2023-10-05T19:40:27.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.119+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc6a4039040 con 0x7fc6ac02d020 2023-10-05T19:40:27.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.119+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6b4014040 con 0x7fc6ac02d020 2023-10-05T19:40:27.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.119+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc6a4039040 con 0x558a60fc2640 2023-10-05T19:40:27.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.119+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6a400fcc0 con 0x558a60fc2640 2023-10-05T19:40:27.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.119+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc6a4039040 con 0x558a60fbdea0 2023-10-05T19:40:27.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.119+0000 7fc6ba91c700 1 -- 172.21.15.12:0/2531636316 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6ac014070 con 0x558a60fbdea0 2023-10-05T19:40:27.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.119+0000 7fc6ba91c700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc6a4039040 con 0x558a60f8c3b0 2023-10-05T19:40:27.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.119+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6b4014040 con 0x558a60f8c3b0 2023-10-05T19:40:27.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.119+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc6a4039040 con 0x7fc6ac02d020 2023-10-05T19:40:27.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.123+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6b4014040 con 0x7fc6ac02d020 2023-10-05T19:40:27.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.123+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc6a4039040 con 0x558a60fc2640 2023-10-05T19:40:27.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.123+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6a400fcc0 con 0x558a60fc2640 2023-10-05T19:40:27.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.123+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc6a4039040 con 0x7fc6ac02d020 2023-10-05T19:40:27.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.123+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6b4014040 con 0x7fc6ac02d020 2023-10-05T19:40:27.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.123+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc6a4039040 con 0x558a60fbdea0 2023-10-05T19:40:27.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.123+0000 7fc6ba91c700 1 -- 172.21.15.12:0/2531636316 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6ac014070 con 0x558a60fbdea0 2023-10-05T19:40:27.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.123+0000 7fc6ba91c700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc6a4039040 con 0x558a60fc2640 2023-10-05T19:40:27.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.127+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6a400fcc0 con 0x558a60fc2640 2023-10-05T19:40:27.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.127+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc6a4039040 con 0x7fc6ac02d020 2023-10-05T19:40:27.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.127+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6b4014040 con 0x7fc6ac02d020 2023-10-05T19:40:27.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.127+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc6a4039040 con 0x558a60f8c3b0 2023-10-05T19:40:27.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.127+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fc6b4014040 con 0x558a60f8c3b0 2023-10-05T19:40:27.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.127+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc6a4039040 con 0x558a60fbdea0 2023-10-05T19:40:27.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.127+0000 7fc6ba91c700 1 -- 172.21.15.12:0/2531636316 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fc6ac014070 con 0x558a60fbdea0 2023-10-05T19:40:27.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.127+0000 7fc6bcde7e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:27.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.127+0000 7fc6bcde7e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:27.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.127+0000 7fc6bcde7e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:27.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.127+0000 7fc6bcde7e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:27.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.127+0000 7fc6bcde7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.127+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc60b0 con 0x558a60f8c3b0 2023-10-05T19:40:27.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.131+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fc6b4014040 con 0x558a60f8c3b0 2023-10-05T19:40:27.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.131+0000 7fc6bcde7e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:27.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.131+0000 7fc6bcde7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.131+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc6510 con 0x558a60fbdea0 2023-10-05T19:40:27.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.131+0000 7fc6ba91c700 1 -- 172.21.15.12:0/2531636316 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fc6ac014070 con 0x558a60fbdea0 2023-10-05T19:40:27.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.131+0000 7fc6bcde7e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:27.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.131+0000 7fc6bcde7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.131+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc6e00 con 0x558a60f8c3b0 2023-10-05T19:40:27.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.131+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fc6b4014040 con 0x558a60f8c3b0 2023-10-05T19:40:27.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.131+0000 7fc6bcde7e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:27.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.131+0000 7fc6bcde7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.131+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc6e00 con 0x558a60fc2640 2023-10-05T19:40:27.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.131+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fc6a400fcc0 con 0x558a60fc2640 2023-10-05T19:40:27.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.131+0000 7fc6bcde7e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:27.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.131+0000 7fc6bcde7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.131+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc7570 con 0x7fc6ac02d020 2023-10-05T19:40:27.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.131+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7fc6b4014040 con 0x7fc6ac02d020 2023-10-05T19:40:27.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.131+0000 7fc6bcde7e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:27.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.131+0000 7fc6bcde7e40 20 zone default found 2023-10-05T19:40:27.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.131+0000 7fc6bcde7e40 4 Realm: () 2023-10-05T19:40:27.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.131+0000 7fc6bcde7e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:27.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.131+0000 7fc6bcde7e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:27.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.131+0000 7fc6bcde7e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:27.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.131+0000 7fc6bcde7e40 20 zonegroup default 2023-10-05T19:40:27.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.131+0000 7fc6bcde7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.131+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc72b0 con 0x7fc6ac02d020 2023-10-05T19:40:27.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.135+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fc6b4014040 con 0x7fc6ac02d020 2023-10-05T19:40:27.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.135+0000 7fc6bcde7e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:27.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.135+0000 7fc6bcde7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.135+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc8c30 con 0x558a60fc2640 2023-10-05T19:40:27.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.135+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fc6a400fcc0 con 0x558a60fc2640 2023-10-05T19:40:27.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.135+0000 7fc6bcde7e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:27.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.135+0000 7fc6bcde7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.135+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc8fe0 con 0x558a60fc2640 2023-10-05T19:40:27.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.135+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fc6a400fcc0 con 0x558a60fc2640 2023-10-05T19:40:27.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.135+0000 7fc6bcde7e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:27.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.135+0000 7fc6bcde7e40 20 started sync module instance, tier type = 2023-10-05T19:40:27.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.135+0000 7fc6bcde7e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:27.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.135+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc82f0 con 0x558a60fbdea0 2023-10-05T19:40:27.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.139+0000 7fc6ba91c700 1 -- 172.21.15.12:0/2531636316 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'132 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc6ac014070 con 0x558a60fbdea0 2023-10-05T19:40:27.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.139+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94052821010160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc6a30 con 0x558a60fbdea0 2023-10-05T19:40:27.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.139+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fc9db0 con 0x558a60f8c3b0 2023-10-05T19:40:27.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.143+0000 7fc6ba91c700 1 -- 172.21.15.12:0/2531636316 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94052821010160] v28'133 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc6ac014070 con 0x558a60fbdea0 2023-10-05T19:40:27.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.143+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'132 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc6b4014040 con 0x558a60f8c3b0 2023-10-05T19:40:27.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.143+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94052821013392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fca1b0 con 0x558a60f8c3b0 2023-10-05T19:40:27.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.143+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fcaa90 con 0x558a60fc2640 2023-10-05T19:40:27.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.147+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'132 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc6a400fcc0 con 0x558a60fc2640 2023-10-05T19:40:27.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.147+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94052821017008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fca600 con 0x558a60fc2640 2023-10-05T19:40:27.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.147+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fcb9e0 con 0x558a60fbdea0 2023-10-05T19:40:27.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.147+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94052821013392] v28'133 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc6b4014040 con 0x558a60f8c3b0 2023-10-05T19:40:27.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.147+0000 7fc6ba91c700 1 -- 172.21.15.12:0/2531636316 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'132 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc6ac014070 con 0x558a60fbdea0 2023-10-05T19:40:27.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.147+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94052821014496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fcb9e0 con 0x558a60fbdea0 2023-10-05T19:40:27.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.147+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fcc280 con 0x558a60f8c3b0 2023-10-05T19:40:27.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.147+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94052821017008] v28'133 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc6a400fcc0 con 0x558a60fc2640 2023-10-05T19:40:27.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.151+0000 7fc6ba91c700 1 -- 172.21.15.12:0/2531636316 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94052821014496] v28'133 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc6ac014070 con 0x558a60fbdea0 2023-10-05T19:40:27.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.151+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'132 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc6b4014040 con 0x558a60f8c3b0 2023-10-05T19:40:27.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.151+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94052821021936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fcbe30 con 0x558a60f8c3b0 2023-10-05T19:40:27.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.151+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fccb70 con 0x7fc6ac02d020 2023-10-05T19:40:27.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.151+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'132 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc6b4014040 con 0x7fc6ac02d020 2023-10-05T19:40:27.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.151+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94052821025856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fccd80 con 0x7fc6ac02d020 2023-10-05T19:40:27.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.151+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fcd620 con 0x7fc6ac02d020 2023-10-05T19:40:27.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.155+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94052821021936] v28'133 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc6b4014040 con 0x558a60f8c3b0 2023-10-05T19:40:27.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.155+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(95 notify.5 [watch watch cookie 94052821025856] v28'133 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc6b4014040 con 0x7fc6ac02d020 2023-10-05T19:40:27.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.155+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(96 notify.6 [create] v28'132 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc6b4014040 con 0x7fc6ac02d020 2023-10-05T19:40:27.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.155+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94052821026960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fcd1d0 con 0x7fc6ac02d020 2023-10-05T19:40:27.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.155+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fcda70 con 0x7fc6ac02d020 2023-10-05T19:40:27.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.159+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(98 notify.7 [create] v28'132 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc6b4014040 con 0x7fc6ac02d020 2023-10-05T19:40:27.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.159+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(97 notify.6 [watch watch cookie 94052821026960] v28'133 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc6b4014040 con 0x7fc6ac02d020 2023-10-05T19:40:27.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.159+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94052821029888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fcd850 con 0x7fc6ac02d020 2023-10-05T19:40:27.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.159+0000 7fc6bcde7e40 20 add_watcher() i=0 2023-10-05T19:40:27.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.159+0000 7fc6bcde7e40 20 add_watcher() i=1 2023-10-05T19:40:27.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.159+0000 7fc6bcde7e40 20 add_watcher() i=2 2023-10-05T19:40:27.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.159+0000 7fc6bcde7e40 20 add_watcher() i=3 2023-10-05T19:40:27.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.159+0000 7fc6bcde7e40 20 add_watcher() i=4 2023-10-05T19:40:27.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.159+0000 7fc6bcde7e40 20 add_watcher() i=5 2023-10-05T19:40:27.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.159+0000 7fc6bcde7e40 20 add_watcher() i=6 2023-10-05T19:40:27.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.159+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94052821029888] v28'133 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc6b4014040 con 0x7fc6ac02d020 2023-10-05T19:40:27.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.159+0000 7fc6bcde7e40 20 add_watcher() i=7 2023-10-05T19:40:27.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.159+0000 7fc6bcde7e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:27.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.159+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fd0cc0 con 0x558a60f8c3b0 2023-10-05T19:40:27.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.159+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fc6b4014040 con 0x558a60f8c3b0 2023-10-05T19:40:27.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.159+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94052821028064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a60fce180 con 0x558a60f8c3b0 2023-10-05T19:40:27.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.163+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94052821028064] v28'129 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fc6b4014040 con 0x558a60f8c3b0 2023-10-05T19:40:27.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.163+0000 7fc607fff700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:27.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.163+0000 7fc6bcde7e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:27.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.163+0000 7fc6bcde7e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:27.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.163+0000 7fc6bcde7e40 1 Processor -- start 2023-10-05T19:40:27.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.163+0000 7fc6bcde7e40 1 -- start start 2023-10-05T19:40:27.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.163+0000 7fc6bcde7e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558a60fdb330 0x558a60fdb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.163+0000 7fc6bcde7e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a60fdbc40 con 0x558a60fdb330 2023-10-05T19:40:27.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.163+0000 7fc6ba11b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558a60fdb330 0x558a60fdb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.163+0000 7fc6ba11b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558a60fdb330 0x558a60fdb700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58216/0 (socket says 172.21.15.12:58216) 2023-10-05T19:40:27.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.163+0000 7fc6ba11b700 1 -- 172.21.15.12:0/4171679620 learned_addr learned my addr 172.21.15.12:0/4171679620 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:27.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.167+0000 7fc6ba11b700 1 -- 172.21.15.12:0/4171679620 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a60fdbd80 con 0x558a60fdb330 2023-10-05T19:40:27.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.167+0000 7fc6ba11b700 1 --2- 172.21.15.12:0/4171679620 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558a60fdb330 0x558a60fdb700 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4003000 tx=0x7fc6a4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2743f8bf2227b3e server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.167+0000 7fc6077fe700 1 -- 172.21.15.12:0/4171679620 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6a401ccd0 con 0x558a60fdb330 2023-10-05T19:40:27.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.167+0000 7fc6077fe700 1 -- 172.21.15.12:0/4171679620 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6a401ce50 con 0x558a60fdb330 2023-10-05T19:40:27.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.167+0000 7fc6077fe700 1 -- 172.21.15.12:0/4171679620 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6a4029420 con 0x558a60fdb330 2023-10-05T19:40:27.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.167+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/4171679620 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558a60fdb330 msgr2=0x558a60fdb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.167+0000 7fc6bcde7e40 1 --2- 172.21.15.12:0/4171679620 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558a60fdb330 0x558a60fdb700 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4003000 tx=0x7fc6a4011000 comp rx=0 tx=0).stop 2023-10-05T19:40:27.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.167+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/4171679620 shutdown_connections 2023-10-05T19:40:27.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.167+0000 7fc6bcde7e40 1 --2- 172.21.15.12:0/4171679620 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558a60fdb330 0x558a60fdb700 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:27.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.167+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/4171679620 >> 172.21.15.12:0/4171679620 conn(0x558a60fd66f0 msgr2=0x558a60fd4840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:27.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.167+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/4171679620 shutdown_connections 2023-10-05T19:40:27.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.167+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/4171679620 wait complete. 2023-10-05T19:40:27.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.167+0000 7fc6bcde7e40 1 Processor -- start 2023-10-05T19:40:27.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.167+0000 7fc6bcde7e40 1 -- start start 2023-10-05T19:40:27.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.167+0000 7fc6bcde7e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558a60fdb330 0x558a60fe0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.167+0000 7fc6bcde7e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a60fe1e60 con 0x558a60fdb330 2023-10-05T19:40:27.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.171+0000 7fc6ba11b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558a60fdb330 0x558a60fe0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.171+0000 7fc6ba11b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558a60fdb330 0x558a60fe0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58220/0 (socket says 172.21.15.12:58220) 2023-10-05T19:40:27.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.171+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2785908888 learned_addr learned my addr 172.21.15.12:0/2785908888 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:27.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.171+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2785908888 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a402e040 con 0x558a60fdb330 2023-10-05T19:40:27.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.171+0000 7fc6ba11b700 1 --2- 172.21.15.12:0/2785908888 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558a60fdb330 0x558a60fe0e60 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4047040 tx=0x7fc6a4042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.171+0000 7fc605ffb700 1 -- 172.21.15.12:0/2785908888 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6a4010d60 con 0x558a60fdb330 2023-10-05T19:40:27.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.171+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2785908888 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558a60fe1460 con 0x558a60fdb330 2023-10-05T19:40:27.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.171+0000 7fc605ffb700 1 -- 172.21.15.12:0/2785908888 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6a4050450 con 0x558a60fdb330 2023-10-05T19:40:27.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.171+0000 7fc605ffb700 1 -- 172.21.15.12:0/2785908888 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6a4050640 con 0x558a60fdb330 2023-10-05T19:40:27.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.171+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2785908888 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558a60fe1900 con 0x558a60fdb330 2023-10-05T19:40:27.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.171+0000 7fc605ffb700 1 -- 172.21.15.12:0/2785908888 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fc6a404e440 con 0x558a60fdb330 2023-10-05T19:40:27.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.171+0000 7fc605ffb700 1 --2- 172.21.15.12:0/2785908888 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fc5f8030c00 0x7fc5f80330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.171+0000 7fc605ffb700 1 -- 172.21.15.12:0/2785908888 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7fc6a4051e60 con 0x558a60fdb330 2023-10-05T19:40:27.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.175+0000 7fc6bb11d700 1 --2- 172.21.15.12:0/2785908888 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fc5f8030c00 0x7fc5f80330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.175+0000 7fc6bcde7e40 5 note: GC not initialized 2023-10-05T19:40:27.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.175+0000 7fc6bb11d700 1 --2- 172.21.15.12:0/2785908888 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fc5f8030c00 0x7fc5f80330b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x558a60fcdb40 tx=0x7fc6b400ab80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.175+0000 7fc5feffd700 20 reqs_thread_entry: start 2023-10-05T19:40:27.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.175+0000 7fc6bcde7e40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:27.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.175+0000 7fc5fcff9700 20 reqs_thread_entry: start 2023-10-05T19:40:27.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.175+0000 7fc6bcde7e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:27.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.175+0000 7fc5eaffd700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:57 2023 2023-10-05T19:40:27.186 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:27.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.175+0000 7fc5ea7fc700 20 reqs_thread_entry: start 2023-10-05T19:40:27.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.175+0000 7fc6bcde7e40 20 get_system_obj_state: rctx=0x7ffe5be090c8 obj=default.rgw.meta:users.uid:testx$tenanteduser state=0x558a60e610b0 s->prefetch_data=0 2023-10-05T19:40:27.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.175+0000 7fc6bcde7e40 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser : miss 2023-10-05T19:40:27.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.175+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:102 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a61049770 con 0x7fc6ac02d020 2023-10-05T19:40:27.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.179+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(102 testx$tenanteduser [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 246+0+139 (crc 0 0 0) 0x7fc6b4014040 con 0x7fc6ac02d020 2023-10-05T19:40:27.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.179+0000 7fc6bcde7e40 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x16 2023-10-05T19:40:27.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.179+0000 7fc6bcde7e40 10 adding default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2023-10-05T19:40:27.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.179+0000 7fc6bcde7e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:27.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.179+0000 7fc6bcde7e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:27.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.179+0000 7fc6bcde7e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:27.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.179+0000 7fc6bcde7e40 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser : type miss (requested=0x13, cached=0x16) 2023-10-05T19:40:27.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.179+0000 7fc6bcde7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.179+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:103 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a6104c200 con 0x7fc6ac02d020 2023-10-05T19:40:27.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.179+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(103 testx$tenanteduser [call,call out=48b,read 0~385 out=385b,getxattrs out=75b] v0'0 uv5 ondisk = 0) v8 ==== 288+0+508 (crc 0 0 0) 0x7fc6b4014040 con 0x7fc6ac02d020 2023-10-05T19:40:27.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.179+0000 7fc6bcde7e40 20 rados_obj.operate() r=0 bl.length=385 2023-10-05T19:40:27.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.179+0000 7fc6bcde7e40 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x13 2023-10-05T19:40:27.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.179+0000 7fc6bcde7e40 10 moving default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2023-10-05T19:40:27.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.179+0000 7fc6bcde7e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:27.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.179+0000 7fc6bcde7e40 20 get_system_obj_state: rctx=0x7ffe5be08470 obj=default.rgw.meta:root:myfoo state=0x558a60e610b0 s->prefetch_data=0 2023-10-05T19:40:27.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.179+0000 7fc6bcde7e40 10 cache get: name=default.rgw.meta+root+myfoo : miss 2023-10-05T19:40:27.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.179+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:104 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a6104ba10 con 0x558a60fbdea0 2023-10-05T19:40:27.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.179+0000 7fc6ba91c700 1 -- 172.21.15.12:0/2531636316 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(104 myfoo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv23 ondisk = 0) v8 ==== 233+0+139 (crc 0 0 0) 0x7fc6ac014070 con 0x558a60fbdea0 2023-10-05T19:40:27.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.179+0000 7fc6bcde7e40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x16 2023-10-05T19:40:27.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.179+0000 7fc6bcde7e40 10 adding default.rgw.meta+root+myfoo to cache LRU end 2023-10-05T19:40:27.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.179+0000 7fc6bcde7e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:27.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.179+0000 7fc6bcde7e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:27.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.179+0000 7fc6bcde7e40 10 cache get: name=default.rgw.meta+root+myfoo : type miss (requested=0x11, cached=0x16) 2023-10-05T19:40:27.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.179+0000 7fc6bcde7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.179+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:105 8.9 8:91b71841:root::myfoo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a6104dfb0 con 0x558a60fbdea0 2023-10-05T19:40:27.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.183+0000 7fc6ba91c700 1 -- 172.21.15.12:0/2531636316 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(105 myfoo [call,call out=48b,read 0~165 out=165b] v0'0 uv23 ondisk = 0) v8 ==== 233+0+213 (crc 0 0 0) 0x7fc6ac014070 con 0x558a60fbdea0 2023-10-05T19:40:27.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.183+0000 7fc6bcde7e40 20 rados_obj.operate() r=0 bl.length=165 2023-10-05T19:40:27.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.183+0000 7fc6bcde7e40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2023-10-05T19:40:27.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.183+0000 7fc6bcde7e40 10 moving default.rgw.meta+root+myfoo to cache LRU end 2023-10-05T19:40:27.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.183+0000 7fc6bcde7e40 20 get_system_obj_state: rctx=0x7ffe5be08470 obj=default.rgw.meta:root:.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 state=0x558a60e372b0 s->prefetch_data=0 2023-10-05T19:40:27.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.183+0000 7fc6bcde7e40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 : miss 2023-10-05T19:40:27.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.183+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:106 8.e 8:70176bbd:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a6104d4e0 con 0x7fc6ac02d020 2023-10-05T19:40:27.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.183+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(106 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 [call out=48b,getxattrs out=218b,stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 290+0+282 (crc 0 0 0) 0x7fc6b4042040 con 0x7fc6ac02d020 2023-10-05T19:40:27.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.183+0000 7fc6bcde7e40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 info.flags=0x16 2023-10-05T19:40:27.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.183+0000 7fc6bcde7e40 10 adding default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 to cache LRU end 2023-10-05T19:40:27.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.183+0000 7fc6bcde7e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:27.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.183+0000 7fc6bcde7e40 10 updating xattr: name=user.rgw.acl bl.length()=123 2023-10-05T19:40:27.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.183+0000 7fc6bcde7e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:27.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.183+0000 7fc6bcde7e40 20 Read xattr: user.rgw.acl 2023-10-05T19:40:27.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.183+0000 7fc6bcde7e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:27.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.183+0000 7fc6bcde7e40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 : type miss (requested=0x13, cached=0x16) 2023-10-05T19:40:27.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.183+0000 7fc6bcde7e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.183+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:107 8.e 8:70176bbd:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a61050550 con 0x7fc6ac02d020 2023-10-05T19:40:27.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.183+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.0 v2:172.21.15.8:6800/27563 30 ==== osd_op_reply(107 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 [call,call out=48b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv4 ondisk = 0) v8 ==== 332+0+618 (crc 0 0 0) 0x7fc6b4042040 con 0x7fc6ac02d020 2023-10-05T19:40:27.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.183+0000 7fc6bcde7e40 20 rados_obj.operate() r=0 bl.length=352 2023-10-05T19:40:27.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.183+0000 7fc6bcde7e40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 info.flags=0x13 2023-10-05T19:40:27.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.183+0000 7fc6bcde7e40 10 moving default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 to cache LRU end 2023-10-05T19:40:27.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.183+0000 7fc6bcde7e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:27.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.183+0000 7fc6bcde7e40 10 updating xattr: name=user.rgw.acl bl.length()=123 2023-10-05T19:40:27.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.183+0000 7fc6bcde7e40 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 2023-10-05T19:40:27.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.183+0000 7fc6bcde7e40 20 get_system_obj_state: rctx=0x7ffe5be08408 obj=default.rgw.meta:users.uid:testx$tenanteduser state=0x558a60e372b0 s->prefetch_data=0 2023-10-05T19:40:27.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.183+0000 7fc6bcde7e40 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser : type miss (requested=0x16, cached=0x13) 2023-10-05T19:40:27.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.183+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:108 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a61050320 con 0x7fc6ac02d020 2023-10-05T19:40:27.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.187+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.0 v2:172.21.15.8:6800/27563 31 ==== osd_op_reply(108 testx$tenanteduser [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 246+0+139 (crc 0 0 0) 0x7fc6b4042040 con 0x7fc6ac02d020 2023-10-05T19:40:27.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.187+0000 7fc6bcde7e40 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x16 2023-10-05T19:40:27.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.187+0000 7fc6bcde7e40 10 moving default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2023-10-05T19:40:27.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.187+0000 7fc6bcde7e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:27.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.187+0000 7fc6bcde7e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:27.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.187+0000 7fc6bcde7e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:27.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.187+0000 7fc6bcde7e40 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser : hit (requested=0x13, cached=0x17) 2023-10-05T19:40:27.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.187+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:109 8.9 8:96d98480:users.uid::foo.buckets:head [call user.remove_bucket in=58b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a61052380 con 0x558a60fbdea0 2023-10-05T19:40:27.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.187+0000 7fc6ba91c700 1 -- 172.21.15.12:0/2531636316 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(109 foo.buckets [call] v28'24 uv24 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x7fc6ac014070 con 0x558a60fbdea0 2023-10-05T19:40:27.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.187+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:110 8.1f 8:f8fdc702:root::.bucket.meta.testx%3amyfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1:head [create,call version.inc in=38b,writefull 0~371 in=371b,setxattr user.rgw.acl (203) in=215b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a61055f10 con 0x558a60fbdea0 2023-10-05T19:40:27.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.191+0000 7fc6ba91c700 1 -- 172.21.15.12:0/2531636316 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(110 .bucket.meta.testx:myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 [create,call,writefull 0~371,setxattr (203)] v28'1 uv1 ondisk = 0) v8 ==== 338+0+0 (crc 0 0 0) 0x7fc6ac014070 con 0x558a60fbdea0 2023-10-05T19:40:27.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.191+0000 7fc6bcde7e40 10 cache put: name=default.rgw.meta+root+.bucket.meta.testx:myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 info.flags=0x7 2023-10-05T19:40:27.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.191+0000 7fc6bcde7e40 10 adding default.rgw.meta+root+.bucket.meta.testx:myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 to cache LRU end 2023-10-05T19:40:27.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.191+0000 7fc6bcde7e40 10 updating xattr: name=user.rgw.acl bl.length()=203 2023-10-05T19:40:27.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.191+0000 7fc6bcde7e40 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.testx:myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1, ofs0, ns] 2023-10-05T19:40:27.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.191+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:111 7.15 7:a93a5511:::notify.2:head [notify cookie 94052821588848 in=818b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a61053d20 con 0x558a60fc2640 2023-10-05T19:40:27.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.191+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== watch-notify(notify (1) cookie 94052821017008 notify 120259084293 ret 0) v3 ==== 848+0+0 (crc 0 0 0) 0x558a60fe1900 con 0x558a60fc2640 2023-10-05T19:40:27.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.191+0000 7fc6b8918700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084293 cookie 94052821017008 notifier 4489 bl.length()=806 2023-10-05T19:40:27.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.191+0000 7fc6b8918700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.testx:myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 info.flags=0x7 2023-10-05T19:40:27.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.191+0000 7fc6b8918700 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.testx:myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 to cache LRU end 2023-10-05T19:40:27.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.191+0000 7fc6b8918700 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2023-10-05T19:40:27.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.191+0000 7fc6b8918700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:112 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc6a0003d90 con 0x558a60fc2640 2023-10-05T19:40:27.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.191+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 <== osd.3 v2:172.21.15.12:6816/29951 27 ==== osd_op_reply(111 notify.2 [notify cookie 94052821588848 out=8b] v0'0 uv132 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fc6a404b040 con 0x558a60fc2640 2023-10-05T19:40:27.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.195+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 <== osd.3 v2:172.21.15.12:6816/29951 28 ==== osd_op_reply(112 notify.2 [notify-ack] v0'0 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc6a404b040 con 0x558a60fc2640 2023-10-05T19:40:27.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.195+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 <== osd.3 v2:172.21.15.12:6816/29951 29 ==== watch-notify(notify_complete (2) cookie 94052821588848 notify 120259084293 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x558a60fe1460 con 0x558a60fc2640 2023-10-05T19:40:27.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.195+0000 7fc6bcde7e40 20 handle_bi_update(): bucket=testx:myfoo[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1]): orig_sources= new_sources= 2023-10-05T19:40:27.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.195+0000 7fc6bcde7e40 20 handle_bi_update(): bucket=testx:myfoo[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1]): potential sources added=[] removed=[] 2023-10-05T19:40:27.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.195+0000 7fc6bcde7e40 20 handle_bi_update(): bucket=testx:myfoo[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1]): orig_dests= new_dests= 2023-10-05T19:40:27.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.195+0000 7fc6bcde7e40 20 handle_bi_update(): bucket=testx:myfoo[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1]): potential dests added=[] removed=[] 2023-10-05T19:40:27.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.195+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:113 8.c 8:37c7a57e:users.uid::testx$tenanteduser.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a61054020 con 0x558a60f8c3b0 2023-10-05T19:40:27.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.199+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(113 testx$tenanteduser.buckets [call] v28'1 uv1 ondisk = 0) v8 ==== 170+0+0 (crc 0 0 0) 0x7fc6b4042040 con 0x558a60f8c3b0 2023-10-05T19:40:27.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.199+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:114 8.5 8:a7a6c9d8:root::testx%2fmyfoo:head [delete,create,call version.inc in=38b,writefull 0~193 in=193b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a61054020 con 0x558a60f8c3b0 2023-10-05T19:40:27.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.199+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(114 testx/myfoo [delete,create,call,writefull 0~193] v28'1 uv1 ondisk = 0) v8 ==== 281+0+0 (crc 0 0 0) 0x7fc6b4042040 con 0x558a60f8c3b0 2023-10-05T19:40:27.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.199+0000 7fc6bcde7e40 10 cache put: name=default.rgw.meta+root+testx/myfoo info.flags=0x7 2023-10-05T19:40:27.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.199+0000 7fc6bcde7e40 10 adding default.rgw.meta+root+testx/myfoo to cache LRU end 2023-10-05T19:40:27.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.199+0000 7fc6bcde7e40 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:testx/myfoo, ofs0, ns] 2023-10-05T19:40:27.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.199+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:115 7.d 7:b4812045:::notify.4:head [notify cookie 94052821575360 in=360b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a61054020 con 0x558a60f8c3b0 2023-10-05T19:40:27.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.203+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== watch-notify(notify (1) cookie 94052821021936 notify 120259084299 ret 0) v3 ==== 390+0+0 (crc 0 0 0) 0x7fc6b4033630 con 0x558a60f8c3b0 2023-10-05T19:40:27.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.203+0000 7fc6b3fff700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084299 cookie 94052821021936 notifier 4489 bl.length()=348 2023-10-05T19:40:27.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.203+0000 7fc6b3fff700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+testx/myfoo info.flags=0x7 2023-10-05T19:40:27.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.203+0000 7fc6b3fff700 10 rgw watcher librados: moving default.rgw.meta+root+testx/myfoo to cache LRU end 2023-10-05T19:40:27.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.203+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(115 notify.4 [notify cookie 94052821575360 out=8b] v0'0 uv132 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fc6b4042040 con 0x558a60f8c3b0 2023-10-05T19:40:27.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.203+0000 7fc6b3fff700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:116 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc60c003400 con 0x558a60f8c3b0 2023-10-05T19:40:27.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.203+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== watch-notify(notify_complete (2) cookie 94052821575360 notify 120259084299 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7fc6b40330d0 con 0x558a60f8c3b0 2023-10-05T19:40:27.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.203+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== osd_op_reply(116 notify.4 [notify-ack] v0'0 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc6b4042040 con 0x558a60f8c3b0 2023-10-05T19:40:27.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.203+0000 7fc6bcde7e40 10 removing default.rgw.meta+root+myfoo from cache 2023-10-05T19:40:27.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.203+0000 7fc6bcde7e40 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2023-10-05T19:40:27.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.203+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:117 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94052821542160 in=161b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a61054020 con 0x7fc6ac02d020 2023-10-05T19:40:27.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.203+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.0 v2:172.21.15.8:6800/27563 32 ==== watch-notify(notify (1) cookie 94052821026960 notify 120259084308 ret 0) v3 ==== 191+0+0 (crc 0 0 0) 0x7fc6b40330d0 con 0x7fc6ac02d020 2023-10-05T19:40:27.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.203+0000 7fc6b3fff700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084308 cookie 94052821026960 notifier 4489 bl.length()=149 2023-10-05T19:40:27.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.203+0000 7fc6b3fff700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:118 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc60c003440 con 0x7fc6ac02d020 2023-10-05T19:40:27.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.203+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.0 v2:172.21.15.8:6800/27563 33 ==== osd_op_reply(117 notify.6 [notify cookie 94052821542160 out=8b] v0'0 uv132 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fc6b4042040 con 0x7fc6ac02d020 2023-10-05T19:40:27.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.203+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.0 v2:172.21.15.8:6800/27563 34 ==== osd_op_reply(118 notify.6 [notify-ack] v0'0 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc6b4042040 con 0x7fc6ac02d020 2023-10-05T19:40:27.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.203+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.0 v2:172.21.15.8:6800/27563 35 ==== watch-notify(notify_complete (2) cookie 94052821542160 notify 120259084308 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7fc6b4033430 con 0x7fc6ac02d020 2023-10-05T19:40:27.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.203+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:119 8.9 8:91b71841:root::myfoo:head [call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a61054020 con 0x558a60fbdea0 2023-10-05T19:40:27.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.207+0000 7fc6ba91c700 1 -- 172.21.15.12:0/2531636316 <== osd.2 v2:172.21.15.12:6800/29934 23 ==== osd_op_reply(119 myfoo [call,delete] v28'25 uv25 ondisk = 0) v8 ==== 191+0+0 (crc 0 0 0) 0x7fc6ac014070 con 0x558a60fbdea0 2023-10-05T19:40:27.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.207+0000 7fc6bcde7e40 10 removing default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 from cache 2023-10-05T19:40:27.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.207+0000 7fc6bcde7e40 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1, ofs0, ns] 2023-10-05T19:40:27.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.207+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:120 7.e 7:7759931f:::notify.3:head [notify cookie 94052821558576 in=218b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a61054020 con 0x558a60fbdea0 2023-10-05T19:40:27.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.207+0000 7fc6ba91c700 1 -- 172.21.15.12:0/2531636316 <== osd.2 v2:172.21.15.12:6800/29934 24 ==== watch-notify(notify (1) cookie 94052821014496 notify 120259084304 ret 0) v3 ==== 248+0+0 (crc 0 0 0) 0x7fc6ac05a040 con 0x558a60fbdea0 2023-10-05T19:40:27.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.207+0000 7fc6b8918700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084304 cookie 94052821014496 notifier 4489 bl.length()=206 2023-10-05T19:40:27.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.207+0000 7fc6b8918700 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:121 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc6a0003d90 con 0x558a60fbdea0 2023-10-05T19:40:27.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.207+0000 7fc6ba91c700 1 -- 172.21.15.12:0/2531636316 <== osd.2 v2:172.21.15.12:6800/29934 25 ==== osd_op_reply(120 notify.3 [notify cookie 94052821558576 out=8b] v0'0 uv132 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fc6ac014070 con 0x558a60fbdea0 2023-10-05T19:40:27.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.211+0000 7fc6ba91c700 1 -- 172.21.15.12:0/2531636316 <== osd.2 v2:172.21.15.12:6800/29934 26 ==== watch-notify(notify_complete (2) cookie 94052821558576 notify 120259084304 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x558a60fb0870 con 0x558a60fbdea0 2023-10-05T19:40:27.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.211+0000 7fc6ba91c700 1 -- 172.21.15.12:0/2531636316 <== osd.2 v2:172.21.15.12:6800/29934 27 ==== osd_op_reply(121 notify.3 [notify-ack] v0'0 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc6ac014070 con 0x558a60fbdea0 2023-10-05T19:40:27.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.211+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:122 8.e 8:70176bbd:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1:head [call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a61054020 con 0x7fc6ac02d020 2023-10-05T19:40:27.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.211+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.0 v2:172.21.15.8:6800/27563 36 ==== osd_op_reply(122 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 [call,delete] v28'6 uv6 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x7fc6b4042040 con 0x7fc6ac02d020 2023-10-05T19:40:27.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.211+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:123 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94052821010160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a61054020 con 0x558a60fbdea0 2023-10-05T19:40:27.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.215+0000 7fc6ba91c700 1 -- 172.21.15.12:0/2531636316 <== osd.2 v2:172.21.15.12:6800/29934 28 ==== osd_op_reply(123 notify.0 [watch unwatch cookie 94052821010160] v28'134 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc6ac014070 con 0x558a60fbdea0 2023-10-05T19:40:27.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.215+0000 7fc6bcde7e40 20 remove_watcher() i=0 2023-10-05T19:40:27.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.215+0000 7fc6bcde7e40 2 removed watcher, disabling cache 2023-10-05T19:40:27.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.215+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:124 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94052821013392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a61054020 con 0x558a60f8c3b0 2023-10-05T19:40:27.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.219+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.1 v2:172.21.15.12:6808/29933 39 ==== osd_op_reply(124 notify.1 [watch unwatch cookie 94052821013392] v28'134 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc6b4042040 con 0x558a60f8c3b0 2023-10-05T19:40:27.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.219+0000 7fc6bcde7e40 20 remove_watcher() i=1 2023-10-05T19:40:27.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.219+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:125 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94052821017008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a61054020 con 0x558a60fc2640 2023-10-05T19:40:27.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.223+0000 7fc6ba11b700 1 -- 172.21.15.12:0/2531636316 <== osd.3 v2:172.21.15.12:6816/29951 30 ==== osd_op_reply(125 notify.2 [watch unwatch cookie 94052821017008] v28'134 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc6a404b040 con 0x558a60fc2640 2023-10-05T19:40:27.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.223+0000 7fc6bcde7e40 20 remove_watcher() i=2 2023-10-05T19:40:27.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.223+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:126 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94052821014496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a61054020 con 0x558a60fbdea0 2023-10-05T19:40:27.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.223+0000 7fc6ba91c700 1 -- 172.21.15.12:0/2531636316 <== osd.2 v2:172.21.15.12:6800/29934 29 ==== osd_op_reply(126 notify.3 [watch unwatch cookie 94052821014496] v28'134 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc6ac014070 con 0x558a60fbdea0 2023-10-05T19:40:27.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.223+0000 7fc6bcde7e40 20 remove_watcher() i=3 2023-10-05T19:40:27.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.223+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:127 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94052821021936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a61054020 con 0x558a60f8c3b0 2023-10-05T19:40:27.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.227+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.1 v2:172.21.15.12:6808/29933 40 ==== osd_op_reply(127 notify.4 [watch unwatch cookie 94052821021936] v28'134 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc6b4042040 con 0x558a60f8c3b0 2023-10-05T19:40:27.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.227+0000 7fc6bcde7e40 20 remove_watcher() i=4 2023-10-05T19:40:27.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.227+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:128 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94052821025856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a61054020 con 0x7fc6ac02d020 2023-10-05T19:40:27.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.231+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.0 v2:172.21.15.8:6800/27563 37 ==== osd_op_reply(128 notify.5 [watch unwatch cookie 94052821025856] v28'134 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc6b4042040 con 0x7fc6ac02d020 2023-10-05T19:40:27.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.231+0000 7fc6bcde7e40 20 remove_watcher() i=5 2023-10-05T19:40:27.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.231+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:129 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94052821026960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a61054020 con 0x7fc6ac02d020 2023-10-05T19:40:27.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.235+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.0 v2:172.21.15.8:6800/27563 38 ==== osd_op_reply(129 notify.6 [watch unwatch cookie 94052821026960] v28'134 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc6b4042040 con 0x7fc6ac02d020 2023-10-05T19:40:27.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.235+0000 7fc6bcde7e40 20 remove_watcher() i=6 2023-10-05T19:40:27.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.235+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:130 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94052821029888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a61054020 con 0x7fc6ac02d020 2023-10-05T19:40:27.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.235+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.0 v2:172.21.15.8:6800/27563 39 ==== osd_op_reply(130 notify.7 [watch unwatch cookie 94052821029888] v28'134 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc6b4042040 con 0x7fc6ac02d020 2023-10-05T19:40:27.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.239+0000 7fc6bcde7e40 20 remove_watcher() i=7 2023-10-05T19:40:27.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.239+0000 7fc607fff700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:27.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.239+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:131 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94052821028064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x558a61054020 con 0x558a60f8c3b0 2023-10-05T19:40:27.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.243+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 <== osd.1 v2:172.21.15.12:6808/29933 41 ==== osd_op_reply(131 data_loggenerations_metadata [watch unwatch cookie 94052821028064] v28'130 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fc6b4042040 con 0x558a60f8c3b0 2023-10-05T19:40:27.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.243+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fc6ac02d020 msgr2=0x7fc6ac03d8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.243+0000 7fc6bcde7e40 1 --2- 172.21.15.12:0/2531636316 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fc6ac02d020 0x7fc6ac03d8d0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:27.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.243+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x558a60f8c3b0 msgr2=0x558a60fbb9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.243+0000 7fc6bcde7e40 1 --2- 172.21.15.12:0/2531636316 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x558a60f8c3b0 0x558a60fbb9b0 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:27.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.243+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x558a60fbdea0 msgr2=0x558a60fc0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.243+0000 7fc6bcde7e40 1 --2- 172.21.15.12:0/2531636316 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x558a60fbdea0 0x558a60fc0280 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:27.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.243+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x558a60fc2640 msgr2=0x558a60fc4a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.243+0000 7fc6bcde7e40 1 --2- 172.21.15.12:0/2531636316 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x558a60fc2640 0x558a60fc4a50 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:27.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.243+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fc69c030c50 msgr2=0x7fc69c033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.243+0000 7fc6bcde7e40 1 --2- 172.21.15.12:0/2531636316 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fc69c030c50 0x7fc69c033100 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4005d80 tx=0x7fc6a4014000 comp rx=0 tx=0).stop 2023-10-05T19:40:27.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.243+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558a60f87dd0 msgr2=0x558a60fafef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.243+0000 7fc6bcde7e40 1 --2- 172.21.15.12:0/2531636316 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558a60f87dd0 0x558a60fafef0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x558a60fa3580 tx=0x7fc6ac016000 comp rx=0 tx=0).stop 2023-10-05T19:40:27.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.243+0000 7fc6bb11d700 1 -- 172.21.15.12:0/2531636316 reap_dead start 2023-10-05T19:40:27.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.243+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 shutdown_connections 2023-10-05T19:40:27.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.243+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 >> 172.21.15.12:0/2531636316 conn(0x558a60f837f0 msgr2=0x558a60fa14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:27.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.243+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 shutdown_connections 2023-10-05T19:40:27.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.243+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2531636316 wait complete. 2023-10-05T19:40:27.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.243+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2785908888 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fc5f8030c00 msgr2=0x7fc5f80330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.243+0000 7fc6bcde7e40 1 --2- 172.21.15.12:0/2785908888 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fc5f8030c00 0x7fc5f80330b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x558a60fcdb40 tx=0x7fc6b400ab80 comp rx=0 tx=0).stop 2023-10-05T19:40:27.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.243+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2785908888 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558a60fdb330 msgr2=0x558a60fe0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.243+0000 7fc6bcde7e40 1 --2- 172.21.15.12:0/2785908888 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558a60fdb330 0x558a60fe0e60 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4047040 tx=0x7fc6a4042000 comp rx=0 tx=0).stop 2023-10-05T19:40:27.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.243+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2785908888 shutdown_connections 2023-10-05T19:40:27.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.243+0000 7fc6bcde7e40 1 --2- 172.21.15.12:0/2785908888 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fc5f8030c00 0x7fc5f80330b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:27.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.243+0000 7fc6bcde7e40 1 --2- 172.21.15.12:0/2785908888 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558a60fdb330 0x558a60fe0e60 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:27.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.243+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2785908888 >> 172.21.15.12:0/2785908888 conn(0x558a60fd66f0 msgr2=0x558a60fd4790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:27.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.243+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2785908888 shutdown_connections 2023-10-05T19:40:27.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.247+0000 7fc6bcde7e40 1 -- 172.21.15.12:0/2785908888 wait complete. 2023-10-05T19:40:27.259 INFO:tasks.util.rgw:rgwadmin: client.0 : ['metadata', 'get', 'bucket:testx/myfoo'] 2023-10-05T19:40:27.260 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'metadata', 'get', 'bucket:testx/myfoo'] 2023-10-05T19:40:27.260 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph metadata get bucket:testx/myfoo 2023-10-05T19:40:27.299 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:27.299 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:27.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.291+0000 7fd35ad9de40 1 Processor -- start 2023-10-05T19:40:27.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.291+0000 7fd35ad9de40 1 -- start start 2023-10-05T19:40:27.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.291+0000 7fd35ad9de40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x561b6cf9c410 0x561b6cf9e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.291+0000 7fd35ad9de40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x561b6cee4bd0 con 0x561b6cf97e30 2023-10-05T19:40:27.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.291+0000 7fd35ad9de40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x561b6cea1ee0 con 0x561b6cf9c410 2023-10-05T19:40:27.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.291+0000 7fd3580d1700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x561b6cf9c410 0x561b6cf9e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.291+0000 7fd3580d1700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x561b6cf9c410 0x561b6cf9e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58226/0 (socket says 172.21.15.12:58226) 2023-10-05T19:40:27.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.291+0000 7fd3580d1700 1 -- 172.21.15.12:0/2486863201 learned_addr learned my addr 172.21.15.12:0/2486863201 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:27.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.291+0000 7fd3580d1700 1 -- 172.21.15.12:0/2486863201 >> v1:172.21.15.8:6789/0 conn(0x561b6cf97e30 legacy=0x561b6cf98200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.291+0000 7fd3580d1700 1 -- 172.21.15.12:0/2486863201 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b6cf98a40 con 0x561b6cf9c410 2023-10-05T19:40:27.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.295+0000 7fd3580d1700 1 --2- 172.21.15.12:0/2486863201 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561b6cf9c410 0x561b6cf9e7f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd34000a5c0 tx=0x7fd340005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdf637c767ba0636 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.295+0000 7fd3578d0700 1 -- 172.21.15.12:0/2486863201 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd34000e510 con 0x561b6cf9c410 2023-10-05T19:40:27.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.295+0000 7fd3578d0700 1 -- 172.21.15.12:0/2486863201 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd34000e670 con 0x561b6cf9c410 2023-10-05T19:40:27.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.295+0000 7fd3578d0700 1 -- 172.21.15.12:0/2486863201 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd34000e920 con 0x561b6cf9c410 2023-10-05T19:40:27.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.295+0000 7fd35ad9de40 1 -- 172.21.15.12:0/2486863201 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561b6cf9c410 msgr2=0x561b6cf9e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.295+0000 7fd35ad9de40 1 --2- 172.21.15.12:0/2486863201 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561b6cf9c410 0x561b6cf9e7f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd34000a5c0 tx=0x7fd340005e00 comp rx=0 tx=0).stop 2023-10-05T19:40:27.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.295+0000 7fd35ad9de40 1 -- 172.21.15.12:0/2486863201 shutdown_connections 2023-10-05T19:40:27.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.295+0000 7fd35ad9de40 1 --2- 172.21.15.12:0/2486863201 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561b6cf9c410 0x561b6cf9e7f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:27.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.295+0000 7fd35ad9de40 1 -- 172.21.15.12:0/2486863201 >> 172.21.15.12:0/2486863201 conn(0x561b6cf93850 msgr2=0x561b6cf95c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:27.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.295+0000 7fd35ad9de40 1 -- 172.21.15.12:0/2486863201 shutdown_connections 2023-10-05T19:40:27.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.295+0000 7fd35ad9de40 1 -- 172.21.15.12:0/2486863201 wait complete. 2023-10-05T19:40:27.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.295+0000 7fd35ad9de40 1 Processor -- start 2023-10-05T19:40:27.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.295+0000 7fd35ad9de40 1 -- start start 2023-10-05T19:40:27.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.295+0000 7fd35ad9de40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561b6cf97e30 0x561b6cfb6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.295+0000 7fd35ad9de40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b6ce65e30 con 0x561b6cf97e30 2023-10-05T19:40:27.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.295+0000 7fd3588d2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561b6cf97e30 0x561b6cfb6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.295+0000 7fd3588d2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561b6cf97e30 0x561b6cfb6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58230/0 (socket says 172.21.15.12:58230) 2023-10-05T19:40:27.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.295+0000 7fd3588d2700 1 -- 172.21.15.12:0/477064957 learned_addr learned my addr 172.21.15.12:0/477064957 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:27.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.295+0000 7fd3588d2700 1 -- 172.21.15.12:0/477064957 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd34000a180 con 0x561b6cf97e30 2023-10-05T19:40:27.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.299+0000 7fd3588d2700 1 --2- 172.21.15.12:0/477064957 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561b6cf97e30 0x561b6cfb6fc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd348001d30 tx=0x7fd348001900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d421cd981dfda005 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.299+0000 7fd3568ce700 1 -- 172.21.15.12:0/477064957 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd348014070 con 0x561b6cf97e30 2023-10-05T19:40:27.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.299+0000 7fd3568ce700 1 -- 172.21.15.12:0/477064957 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd348008070 con 0x561b6cf97e30 2023-10-05T19:40:27.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.299+0000 7fd3568ce700 1 -- 172.21.15.12:0/477064957 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd34800bd30 con 0x561b6cf97e30 2023-10-05T19:40:27.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.299+0000 7fd35ad9de40 1 -- 172.21.15.12:0/477064957 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561b6cf97e30 msgr2=0x561b6cfb6fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.299+0000 7fd35ad9de40 1 --2- 172.21.15.12:0/477064957 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561b6cf97e30 0x561b6cfb6fc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd348001d30 tx=0x7fd348001900 comp rx=0 tx=0).stop 2023-10-05T19:40:27.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.299+0000 7fd35ad9de40 1 -- 172.21.15.12:0/477064957 shutdown_connections 2023-10-05T19:40:27.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.299+0000 7fd35ad9de40 1 --2- 172.21.15.12:0/477064957 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561b6cf97e30 0x561b6cfb6fc0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:27.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.299+0000 7fd35ad9de40 1 -- 172.21.15.12:0/477064957 >> 172.21.15.12:0/477064957 conn(0x561b6cf93850 msgr2=0x561b6cfb2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:27.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.299+0000 7fd35ad9de40 1 -- 172.21.15.12:0/477064957 shutdown_connections 2023-10-05T19:40:27.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.299+0000 7fd35ad9de40 1 -- 172.21.15.12:0/477064957 wait complete. 2023-10-05T19:40:27.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.299+0000 7fd35ad9de40 1 Processor -- start 2023-10-05T19:40:27.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.299+0000 7fd35ad9de40 1 -- start start 2023-10-05T19:40:27.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.299+0000 7fd35ad9de40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561b6cf97e30 0x561b6cf0daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.299+0000 7fd35ad9de40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b6ce38ed0 con 0x561b6cf97e30 2023-10-05T19:40:27.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.299+0000 7fd3588d2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561b6cf97e30 0x561b6cf0daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.299+0000 7fd3588d2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561b6cf97e30 0x561b6cf0daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58246/0 (socket says 172.21.15.12:58246) 2023-10-05T19:40:27.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.299+0000 7fd3588d2700 1 -- 172.21.15.12:0/1640501558 learned_addr learned my addr 172.21.15.12:0/1640501558 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:27.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.299+0000 7fd3588d2700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd34800a040 con 0x561b6cf97e30 2023-10-05T19:40:27.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.299+0000 7fd3588d2700 1 --2- 172.21.15.12:0/1640501558 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561b6cf97e30 0x561b6cf0daf0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x561b6cfb36a0 tx=0x7fd348016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.303+0000 7fd3550cb700 1 -- 172.21.15.12:0/1640501558 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd34800bee0 con 0x561b6cf97e30 2023-10-05T19:40:27.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.303+0000 7fd3550cb700 1 -- 172.21.15.12:0/1640501558 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd348021070 con 0x561b6cf97e30 2023-10-05T19:40:27.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.303+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561b6cf0e030 con 0x561b6cf97e30 2023-10-05T19:40:27.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.303+0000 7fd3550cb700 1 -- 172.21.15.12:0/1640501558 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd34801a600 con 0x561b6cf97e30 2023-10-05T19:40:27.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.303+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561b6cf0c3f0 con 0x561b6cf97e30 2023-10-05T19:40:27.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.303+0000 7fd35ad9de40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.303+0000 7fd3550cb700 1 -- 172.21.15.12:0/1640501558 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fd34801a7b0 con 0x561b6cf97e30 2023-10-05T19:40:27.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.303+0000 7fd3550cb700 1 --2- 172.21.15.12:0/1640501558 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd338030c50 0x7fd338033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.303+0000 7fd3550cb700 1 -- 172.21.15.12:0/1640501558 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7fd348044c50 con 0x561b6cf97e30 2023-10-05T19:40:27.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.303+0000 7fd3580d1700 1 --2- 172.21.15.12:0/1640501558 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd338030c50 0x7fd338033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.303+0000 7fd35ad9de40 1 --2- 172.21.15.12:0/1640501558 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x561b6cf9c410 0x561b6cfcbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.303+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfcc150 con 0x561b6cf9c410 2023-10-05T19:40:27.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.303+0000 7fd3590d3700 1 --2- 172.21.15.12:0/1640501558 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x561b6cf9c410 0x561b6cfcbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.303+0000 7fd3580d1700 1 --2- 172.21.15.12:0/1640501558 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd338030c50 0x7fd338033100 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd340000f80 tx=0x7fd340013000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.307+0000 7fd3590d3700 1 --2- 172.21.15.12:0/1640501558 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x561b6cf9c410 0x561b6cfcbaa0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.307+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd35000a500 con 0x561b6cf9c410 2023-10-05T19:40:27.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.307+0000 7fd35ad9de40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:27.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.307+0000 7fd35ad9de40 20 realm 2023-10-05T19:40:27.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.307+0000 7fd35ad9de40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.307+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfc9100 con 0x561b6cf9c410 2023-10-05T19:40:27.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.307+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd35000a500 con 0x561b6cf9c410 2023-10-05T19:40:27.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.307+0000 7fd35ad9de40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:27.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.307+0000 7fd35ad9de40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:27.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.307+0000 7fd35ad9de40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.307+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfccc00 con 0x561b6cf9c410 2023-10-05T19:40:27.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.307+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd35000a500 con 0x561b6cf9c410 2023-10-05T19:40:27.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.307+0000 7fd35ad9de40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:27.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.307+0000 7fd35ad9de40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.307+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfcc7e0 con 0x561b6cf9c410 2023-10-05T19:40:27.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.307+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fd35000a500 con 0x561b6cf9c410 2023-10-05T19:40:27.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.307+0000 7fd35ad9de40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:27.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.307+0000 7fd35ad9de40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.307+0000 7fd35ad9de40 1 --2- 172.21.15.12:0/1640501558 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x561b6cfcdb90 0x561b6cfcff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.307+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd06e0 con 0x561b6cfcdb90 2023-10-05T19:40:27.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.307+0000 7fd3588d2700 1 --2- 172.21.15.12:0/1640501558 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x561b6cfcdb90 0x561b6cfcff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.307+0000 7fd3588d2700 1 --2- 172.21.15.12:0/1640501558 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x561b6cfcdb90 0x561b6cfcff70 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.311+0000 7fd3588d2700 1 -- 172.21.15.12:0/1640501558 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fd348016070 con 0x561b6cfcdb90 2023-10-05T19:40:27.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.311+0000 7fd35ad9de40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:27.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.311+0000 7fd35ad9de40 20 searching for the correct realm 2023-10-05T19:40:27.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.311+0000 7fd35ad9de40 1 --2- 172.21.15.12:0/1640501558 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x561b6cfd2510 0x561b6cfd48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.311+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd4e30 con 0x561b6cfd2510 2023-10-05T19:40:27.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.311+0000 7fd3580d1700 1 --2- 172.21.15.12:0/1640501558 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x561b6cfd2510 0x561b6cfd48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.311+0000 7fd3580d1700 1 --2- 172.21.15.12:0/1640501558 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x561b6cfd2510 0x561b6cfd48f0 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.311+0000 7fd3580d1700 1 -- 172.21.15.12:0/1640501558 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fd340017050 con 0x561b6cfd2510 2023-10-05T19:40:27.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.311+0000 7fd3580d1700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd4e30 con 0x561b6cfcdb90 2023-10-05T19:40:27.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.315+0000 7fd3588d2700 1 -- 172.21.15.12:0/1640501558 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd348016070 con 0x561b6cfcdb90 2023-10-05T19:40:27.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.315+0000 7fd3588d2700 1 --2- 172.21.15.12:0/1640501558 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fd3480176e0 0x7fd3480129f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.315+0000 7fd3588d2700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd4e30 con 0x7fd3480176e0 2023-10-05T19:40:27.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.315+0000 7fd3590d3700 1 --2- 172.21.15.12:0/1640501558 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fd3480176e0 0x7fd3480129f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.315+0000 7fd3590d3700 1 --2- 172.21.15.12:0/1640501558 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fd3480176e0 0x7fd3480129f0 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.315+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd350014040 con 0x7fd3480176e0 2023-10-05T19:40:27.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.315+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd4e30 con 0x561b6cf9c410 2023-10-05T19:40:27.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.315+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd350014040 con 0x561b6cf9c410 2023-10-05T19:40:27.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.315+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd4e30 con 0x561b6cf9c410 2023-10-05T19:40:27.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.319+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd350014040 con 0x561b6cf9c410 2023-10-05T19:40:27.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.319+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd4e30 con 0x7fd3480176e0 2023-10-05T19:40:27.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.319+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd350014040 con 0x7fd3480176e0 2023-10-05T19:40:27.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.319+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd4e30 con 0x7fd3480176e0 2023-10-05T19:40:27.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.319+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd350014040 con 0x7fd3480176e0 2023-10-05T19:40:27.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.319+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd4e30 con 0x561b6cfcdb90 2023-10-05T19:40:27.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.319+0000 7fd3588d2700 1 -- 172.21.15.12:0/1640501558 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd348016070 con 0x561b6cfcdb90 2023-10-05T19:40:27.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.319+0000 7fd3588d2700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd4e30 con 0x561b6cfd2510 2023-10-05T19:40:27.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.323+0000 7fd3580d1700 1 -- 172.21.15.12:0/1640501558 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd340017050 con 0x561b6cfd2510 2023-10-05T19:40:27.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.323+0000 7fd3580d1700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd4e30 con 0x561b6cf9c410 2023-10-05T19:40:27.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.323+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd350014040 con 0x561b6cf9c410 2023-10-05T19:40:27.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.323+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd4e30 con 0x7fd3480176e0 2023-10-05T19:40:27.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.323+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd350014040 con 0x7fd3480176e0 2023-10-05T19:40:27.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.323+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd4e30 con 0x561b6cf9c410 2023-10-05T19:40:27.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.323+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd350014040 con 0x561b6cf9c410 2023-10-05T19:40:27.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.323+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd4e30 con 0x561b6cfcdb90 2023-10-05T19:40:27.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.323+0000 7fd3588d2700 1 -- 172.21.15.12:0/1640501558 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd348016070 con 0x561b6cfcdb90 2023-10-05T19:40:27.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.323+0000 7fd3588d2700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd4e30 con 0x561b6cfd2510 2023-10-05T19:40:27.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.327+0000 7fd3580d1700 1 -- 172.21.15.12:0/1640501558 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd340017050 con 0x561b6cfd2510 2023-10-05T19:40:27.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.327+0000 7fd3580d1700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd4e30 con 0x561b6cfd2510 2023-10-05T19:40:27.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.327+0000 7fd3580d1700 1 -- 172.21.15.12:0/1640501558 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd340017050 con 0x561b6cfd2510 2023-10-05T19:40:27.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.327+0000 7fd3580d1700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd4e30 con 0x7fd3480176e0 2023-10-05T19:40:27.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.327+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd350014040 con 0x7fd3480176e0 2023-10-05T19:40:27.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.327+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd4e30 con 0x561b6cfd2510 2023-10-05T19:40:27.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.327+0000 7fd3580d1700 1 -- 172.21.15.12:0/1640501558 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd340017050 con 0x561b6cfd2510 2023-10-05T19:40:27.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.327+0000 7fd3580d1700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd4e30 con 0x561b6cfd2510 2023-10-05T19:40:27.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.331+0000 7fd3580d1700 1 -- 172.21.15.12:0/1640501558 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd340017050 con 0x561b6cfd2510 2023-10-05T19:40:27.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.331+0000 7fd3580d1700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd4e30 con 0x561b6cf9c410 2023-10-05T19:40:27.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.331+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd350014040 con 0x561b6cf9c410 2023-10-05T19:40:27.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.331+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd4e30 con 0x561b6cf9c410 2023-10-05T19:40:27.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.331+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd350014040 con 0x561b6cf9c410 2023-10-05T19:40:27.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.331+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd4e30 con 0x7fd3480176e0 2023-10-05T19:40:27.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.331+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd350014040 con 0x7fd3480176e0 2023-10-05T19:40:27.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.331+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd4e30 con 0x561b6cfd2510 2023-10-05T19:40:27.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.335+0000 7fd3580d1700 1 -- 172.21.15.12:0/1640501558 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd340017050 con 0x561b6cfd2510 2023-10-05T19:40:27.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.335+0000 7fd3580d1700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd4e30 con 0x561b6cfcdb90 2023-10-05T19:40:27.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.335+0000 7fd3588d2700 1 -- 172.21.15.12:0/1640501558 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd348016070 con 0x561b6cfcdb90 2023-10-05T19:40:27.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.335+0000 7fd3588d2700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd4e30 con 0x561b6cf9c410 2023-10-05T19:40:27.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.335+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd350014040 con 0x561b6cf9c410 2023-10-05T19:40:27.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.335+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd4e30 con 0x7fd3480176e0 2023-10-05T19:40:27.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.335+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd350014040 con 0x7fd3480176e0 2023-10-05T19:40:27.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.335+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd4e30 con 0x561b6cfd2510 2023-10-05T19:40:27.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.339+0000 7fd3580d1700 1 -- 172.21.15.12:0/1640501558 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd340017050 con 0x561b6cfd2510 2023-10-05T19:40:27.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.339+0000 7fd3580d1700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd4e30 con 0x7fd3480176e0 2023-10-05T19:40:27.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.339+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd350014040 con 0x7fd3480176e0 2023-10-05T19:40:27.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.339+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd4e30 con 0x561b6cfcdb90 2023-10-05T19:40:27.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.339+0000 7fd3588d2700 1 -- 172.21.15.12:0/1640501558 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd348016070 con 0x561b6cfcdb90 2023-10-05T19:40:27.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.339+0000 7fd3588d2700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd4e30 con 0x561b6cfd2510 2023-10-05T19:40:27.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.339+0000 7fd3580d1700 1 -- 172.21.15.12:0/1640501558 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd340017050 con 0x561b6cfd2510 2023-10-05T19:40:27.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.339+0000 7fd3580d1700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd4e30 con 0x7fd3480176e0 2023-10-05T19:40:27.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.339+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd350014040 con 0x7fd3480176e0 2023-10-05T19:40:27.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.339+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd4e30 con 0x561b6cf9c410 2023-10-05T19:40:27.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.343+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fd350014040 con 0x561b6cf9c410 2023-10-05T19:40:27.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.343+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd4e30 con 0x561b6cfcdb90 2023-10-05T19:40:27.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.343+0000 7fd3588d2700 1 -- 172.21.15.12:0/1640501558 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fd348016070 con 0x561b6cfcdb90 2023-10-05T19:40:27.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.343+0000 7fd35ad9de40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:27.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.343+0000 7fd35ad9de40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:27.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.343+0000 7fd35ad9de40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:27.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.343+0000 7fd35ad9de40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:27.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.343+0000 7fd35ad9de40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.343+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd1290 con 0x561b6cf9c410 2023-10-05T19:40:27.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.343+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd350014040 con 0x561b6cf9c410 2023-10-05T19:40:27.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.343+0000 7fd35ad9de40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:27.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.343+0000 7fd35ad9de40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.343+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd1be0 con 0x561b6cf9c410 2023-10-05T19:40:27.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.343+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fd350014040 con 0x561b6cf9c410 2023-10-05T19:40:27.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.343+0000 7fd35ad9de40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:27.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.343+0000 7fd35ad9de40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.343+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd0e50 con 0x561b6cfd2510 2023-10-05T19:40:27.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.347+0000 7fd3580d1700 1 -- 172.21.15.12:0/1640501558 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fd340017050 con 0x561b6cfd2510 2023-10-05T19:40:27.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.347+0000 7fd35ad9de40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:27.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.347+0000 7fd35ad9de40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.347+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd5760 con 0x561b6cfd2510 2023-10-05T19:40:27.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.347+0000 7fd3580d1700 1 -- 172.21.15.12:0/1640501558 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7fd340017050 con 0x561b6cfd2510 2023-10-05T19:40:27.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.347+0000 7fd35ad9de40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:27.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.347+0000 7fd35ad9de40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.347+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd5b10 con 0x561b6cf9c410 2023-10-05T19:40:27.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.347+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd350014040 con 0x561b6cf9c410 2023-10-05T19:40:27.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.347+0000 7fd35ad9de40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:27.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.347+0000 7fd35ad9de40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.347+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd5ec0 con 0x561b6cf9c410 2023-10-05T19:40:27.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.347+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fd350014040 con 0x561b6cf9c410 2023-10-05T19:40:27.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.347+0000 7fd35ad9de40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:27.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.347+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd6ed0 con 0x561b6cfd2510 2023-10-05T19:40:27.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.351+0000 7fd3580d1700 1 -- 172.21.15.12:0/1640501558 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fd340017050 con 0x561b6cfd2510 2023-10-05T19:40:27.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.351+0000 7fd3580d1700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd6ed0 con 0x561b6cfcdb90 2023-10-05T19:40:27.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.351+0000 7fd3588d2700 1 -- 172.21.15.12:0/1640501558 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd348016070 con 0x561b6cfcdb90 2023-10-05T19:40:27.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.351+0000 7fd3588d2700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd6ed0 con 0x7fd3480176e0 2023-10-05T19:40:27.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.355+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd350014040 con 0x7fd3480176e0 2023-10-05T19:40:27.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.355+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd5b10 con 0x561b6cf9c410 2023-10-05T19:40:27.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.355+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd350014040 con 0x561b6cf9c410 2023-10-05T19:40:27.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.355+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd5b10 con 0x561b6cf9c410 2023-10-05T19:40:27.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.355+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd350014040 con 0x561b6cf9c410 2023-10-05T19:40:27.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.355+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd5b10 con 0x7fd3480176e0 2023-10-05T19:40:27.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.355+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd350014040 con 0x7fd3480176e0 2023-10-05T19:40:27.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.355+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd5b10 con 0x7fd3480176e0 2023-10-05T19:40:27.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.355+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd350014040 con 0x7fd3480176e0 2023-10-05T19:40:27.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.355+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd5b10 con 0x561b6cfcdb90 2023-10-05T19:40:27.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.359+0000 7fd3588d2700 1 -- 172.21.15.12:0/1640501558 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd348016070 con 0x561b6cfcdb90 2023-10-05T19:40:27.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.359+0000 7fd3588d2700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd5b10 con 0x561b6cfd2510 2023-10-05T19:40:27.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.359+0000 7fd3580d1700 1 -- 172.21.15.12:0/1640501558 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd340017050 con 0x561b6cfd2510 2023-10-05T19:40:27.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.359+0000 7fd3580d1700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd5b10 con 0x561b6cf9c410 2023-10-05T19:40:27.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.359+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd350014040 con 0x561b6cf9c410 2023-10-05T19:40:27.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.359+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd5b10 con 0x7fd3480176e0 2023-10-05T19:40:27.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.359+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd350014040 con 0x7fd3480176e0 2023-10-05T19:40:27.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.359+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd5b10 con 0x561b6cf9c410 2023-10-05T19:40:27.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.359+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd350014040 con 0x561b6cf9c410 2023-10-05T19:40:27.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.363+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd5b10 con 0x561b6cfcdb90 2023-10-05T19:40:27.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.363+0000 7fd3588d2700 1 -- 172.21.15.12:0/1640501558 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd348016070 con 0x561b6cfcdb90 2023-10-05T19:40:27.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.363+0000 7fd3588d2700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd5b10 con 0x561b6cfd2510 2023-10-05T19:40:27.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.363+0000 7fd3580d1700 1 -- 172.21.15.12:0/1640501558 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd340017050 con 0x561b6cfd2510 2023-10-05T19:40:27.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.363+0000 7fd3580d1700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd5b10 con 0x561b6cfd2510 2023-10-05T19:40:27.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.363+0000 7fd3580d1700 1 -- 172.21.15.12:0/1640501558 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd340017050 con 0x561b6cfd2510 2023-10-05T19:40:27.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.363+0000 7fd3580d1700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd5b10 con 0x7fd3480176e0 2023-10-05T19:40:27.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.363+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd350014040 con 0x7fd3480176e0 2023-10-05T19:40:27.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.363+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd5b10 con 0x561b6cfd2510 2023-10-05T19:40:27.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.367+0000 7fd3580d1700 1 -- 172.21.15.12:0/1640501558 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd340017050 con 0x561b6cfd2510 2023-10-05T19:40:27.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.367+0000 7fd3580d1700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd5b10 con 0x561b6cfd2510 2023-10-05T19:40:27.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.367+0000 7fd3580d1700 1 -- 172.21.15.12:0/1640501558 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd340017050 con 0x561b6cfd2510 2023-10-05T19:40:27.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.367+0000 7fd3580d1700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd5b10 con 0x561b6cf9c410 2023-10-05T19:40:27.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.367+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd350014040 con 0x561b6cf9c410 2023-10-05T19:40:27.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.367+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd5b10 con 0x561b6cf9c410 2023-10-05T19:40:27.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.367+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd350014040 con 0x561b6cf9c410 2023-10-05T19:40:27.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.367+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd5b10 con 0x7fd3480176e0 2023-10-05T19:40:27.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.367+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd350014040 con 0x7fd3480176e0 2023-10-05T19:40:27.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.367+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd5b10 con 0x561b6cfd2510 2023-10-05T19:40:27.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.371+0000 7fd3580d1700 1 -- 172.21.15.12:0/1640501558 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd340017050 con 0x561b6cfd2510 2023-10-05T19:40:27.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.371+0000 7fd3580d1700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd5b10 con 0x561b6cfcdb90 2023-10-05T19:40:27.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.371+0000 7fd3588d2700 1 -- 172.21.15.12:0/1640501558 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd348016070 con 0x561b6cfcdb90 2023-10-05T19:40:27.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.371+0000 7fd3588d2700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd5b10 con 0x561b6cf9c410 2023-10-05T19:40:27.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.371+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd350014040 con 0x561b6cf9c410 2023-10-05T19:40:27.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.371+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd5b10 con 0x7fd3480176e0 2023-10-05T19:40:27.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.371+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd350014040 con 0x7fd3480176e0 2023-10-05T19:40:27.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.371+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd5b10 con 0x561b6cfd2510 2023-10-05T19:40:27.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.371+0000 7fd3580d1700 1 -- 172.21.15.12:0/1640501558 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd340017050 con 0x561b6cfd2510 2023-10-05T19:40:27.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.371+0000 7fd3580d1700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd5b10 con 0x7fd3480176e0 2023-10-05T19:40:27.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.375+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd350014040 con 0x7fd3480176e0 2023-10-05T19:40:27.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.375+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd5b10 con 0x561b6cfcdb90 2023-10-05T19:40:27.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.375+0000 7fd3588d2700 1 -- 172.21.15.12:0/1640501558 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd348016070 con 0x561b6cfcdb90 2023-10-05T19:40:27.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.375+0000 7fd3588d2700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd5b10 con 0x561b6cfd2510 2023-10-05T19:40:27.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.375+0000 7fd3580d1700 1 -- 172.21.15.12:0/1640501558 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd340017050 con 0x561b6cfd2510 2023-10-05T19:40:27.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.375+0000 7fd3580d1700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd5b10 con 0x7fd3480176e0 2023-10-05T19:40:27.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.375+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd350014040 con 0x7fd3480176e0 2023-10-05T19:40:27.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.375+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd5b10 con 0x561b6cf9c410 2023-10-05T19:40:27.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.379+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fd350014040 con 0x561b6cf9c410 2023-10-05T19:40:27.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.379+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd5b10 con 0x561b6cfcdb90 2023-10-05T19:40:27.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.379+0000 7fd3588d2700 1 -- 172.21.15.12:0/1640501558 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fd348016070 con 0x561b6cfcdb90 2023-10-05T19:40:27.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.379+0000 7fd35ad9de40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:27.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.379+0000 7fd35ad9de40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:27.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.379+0000 7fd35ad9de40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:27.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.379+0000 7fd35ad9de40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:27.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.379+0000 7fd35ad9de40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.379+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd5ec0 con 0x561b6cf9c410 2023-10-05T19:40:27.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.379+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fd350014040 con 0x561b6cf9c410 2023-10-05T19:40:27.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.379+0000 7fd35ad9de40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:27.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.379+0000 7fd35ad9de40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.379+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd6270 con 0x561b6cfcdb90 2023-10-05T19:40:27.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.379+0000 7fd3588d2700 1 -- 172.21.15.12:0/1640501558 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fd348016070 con 0x561b6cfcdb90 2023-10-05T19:40:27.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.379+0000 7fd35ad9de40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:27.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.379+0000 7fd35ad9de40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.379+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd6b10 con 0x561b6cf9c410 2023-10-05T19:40:27.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.379+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fd350014040 con 0x561b6cf9c410 2023-10-05T19:40:27.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.379+0000 7fd35ad9de40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:27.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.379+0000 7fd35ad9de40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.379+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd6b10 con 0x561b6cfd2510 2023-10-05T19:40:27.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.383+0000 7fd3580d1700 1 -- 172.21.15.12:0/1640501558 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fd340017050 con 0x561b6cfd2510 2023-10-05T19:40:27.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.383+0000 7fd35ad9de40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:27.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.383+0000 7fd35ad9de40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.383+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd66a0 con 0x7fd3480176e0 2023-10-05T19:40:27.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.383+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7fd350014040 con 0x7fd3480176e0 2023-10-05T19:40:27.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.383+0000 7fd35ad9de40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:27.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.383+0000 7fd35ad9de40 20 zone default found 2023-10-05T19:40:27.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.383+0000 7fd35ad9de40 4 Realm: () 2023-10-05T19:40:27.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.383+0000 7fd35ad9de40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:27.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.383+0000 7fd35ad9de40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:27.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.383+0000 7fd35ad9de40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:27.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.383+0000 7fd35ad9de40 20 zonegroup default 2023-10-05T19:40:27.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.383+0000 7fd35ad9de40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.383+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd66a0 con 0x7fd3480176e0 2023-10-05T19:40:27.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.383+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fd350014040 con 0x7fd3480176e0 2023-10-05T19:40:27.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.383+0000 7fd35ad9de40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:27.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.383+0000 7fd35ad9de40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.383+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd66a0 con 0x561b6cfd2510 2023-10-05T19:40:27.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.383+0000 7fd3580d1700 1 -- 172.21.15.12:0/1640501558 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fd340017050 con 0x561b6cfd2510 2023-10-05T19:40:27.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.383+0000 7fd35ad9de40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:27.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.383+0000 7fd35ad9de40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.383+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd8d30 con 0x561b6cfd2510 2023-10-05T19:40:27.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.387+0000 7fd3580d1700 1 -- 172.21.15.12:0/1640501558 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fd340017050 con 0x561b6cfd2510 2023-10-05T19:40:27.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.387+0000 7fd35ad9de40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:27.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.387+0000 7fd35ad9de40 20 started sync module instance, tier type = 2023-10-05T19:40:27.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.387+0000 7fd35ad9de40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:27.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.387+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd83f0 con 0x561b6cfcdb90 2023-10-05T19:40:27.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.387+0000 7fd3588d2700 1 -- 172.21.15.12:0/1640501558 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'135 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd348016070 con 0x561b6cfcdb90 2023-10-05T19:40:27.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.387+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94675792660464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd8020 con 0x561b6cfcdb90 2023-10-05T19:40:27.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.387+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd9ac0 con 0x561b6cf9c410 2023-10-05T19:40:27.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.391+0000 7fd3588d2700 1 -- 172.21.15.12:0/1640501558 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94675792660464] v28'136 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd348016070 con 0x561b6cfcdb90 2023-10-05T19:40:27.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.391+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'135 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd350014040 con 0x561b6cf9c410 2023-10-05T19:40:27.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.391+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94675792662752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd9a20 con 0x561b6cf9c410 2023-10-05T19:40:27.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.391+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfda2c0 con 0x561b6cfd2510 2023-10-05T19:40:27.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.395+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94675792662752] v28'136 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd350014040 con 0x561b6cf9c410 2023-10-05T19:40:27.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.395+0000 7fd3580d1700 1 -- 172.21.15.12:0/1640501558 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'135 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd340017050 con 0x561b6cfd2510 2023-10-05T19:40:27.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.395+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94675792664144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfda670 con 0x561b6cfd2510 2023-10-05T19:40:27.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.395+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfdaf10 con 0x561b6cfcdb90 2023-10-05T19:40:27.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.399+0000 7fd3588d2700 1 -- 172.21.15.12:0/1640501558 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'135 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd348016070 con 0x561b6cfcdb90 2023-10-05T19:40:27.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.399+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94675792669504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfdb800 con 0x561b6cfcdb90 2023-10-05T19:40:27.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.399+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfdc0e0 con 0x561b6cf9c410 2023-10-05T19:40:27.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.399+0000 7fd3580d1700 1 -- 172.21.15.12:0/1640501558 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94675792664144] v28'136 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd340017050 con 0x561b6cfd2510 2023-10-05T19:40:27.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.399+0000 7fd3588d2700 1 -- 172.21.15.12:0/1640501558 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94675792669504] v28'136 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd348016070 con 0x561b6cfcdb90 2023-10-05T19:40:27.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.403+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'135 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd350014040 con 0x561b6cf9c410 2023-10-05T19:40:27.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.403+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94675792665248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfdb800 con 0x561b6cf9c410 2023-10-05T19:40:27.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.403+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfdc9d0 con 0x7fd3480176e0 2023-10-05T19:40:27.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.403+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'135 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd350014040 con 0x7fd3480176e0 2023-10-05T19:40:27.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.403+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94675792675680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfdcca0 con 0x7fd3480176e0 2023-10-05T19:40:27.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.403+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfdd540 con 0x7fd3480176e0 2023-10-05T19:40:27.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.403+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94675792665248] v28'136 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd350014040 con 0x561b6cf9c410 2023-10-05T19:40:27.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.407+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(95 notify.5 [watch watch cookie 94675792675680] v28'136 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd350014040 con 0x7fd3480176e0 2023-10-05T19:40:27.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.407+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(96 notify.6 [create] v28'135 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd350014040 con 0x7fd3480176e0 2023-10-05T19:40:27.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.407+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94675792676784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfdd0f0 con 0x7fd3480176e0 2023-10-05T19:40:27.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.407+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfdd990 con 0x7fd3480176e0 2023-10-05T19:40:27.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.407+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(97 notify.6 [watch watch cookie 94675792676784] v28'136 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd350014040 con 0x7fd3480176e0 2023-10-05T19:40:27.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.411+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(98 notify.7 [create] v28'135 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd350014040 con 0x7fd3480176e0 2023-10-05T19:40:27.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.411+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94675792678448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfdd770 con 0x7fd3480176e0 2023-10-05T19:40:27.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.411+0000 7fd35ad9de40 20 add_watcher() i=0 2023-10-05T19:40:27.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.411+0000 7fd35ad9de40 20 add_watcher() i=1 2023-10-05T19:40:27.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.411+0000 7fd35ad9de40 20 add_watcher() i=2 2023-10-05T19:40:27.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.411+0000 7fd35ad9de40 20 add_watcher() i=3 2023-10-05T19:40:27.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.411+0000 7fd35ad9de40 20 add_watcher() i=4 2023-10-05T19:40:27.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.411+0000 7fd35ad9de40 20 add_watcher() i=5 2023-10-05T19:40:27.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.411+0000 7fd35ad9de40 20 add_watcher() i=6 2023-10-05T19:40:27.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.411+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94675792678448] v28'136 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd350014040 con 0x7fd3480176e0 2023-10-05T19:40:27.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.411+0000 7fd35ad9de40 20 add_watcher() i=7 2023-10-05T19:40:27.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.411+0000 7fd35ad9de40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:27.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.411+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfe0c40 con 0x561b6cf9c410 2023-10-05T19:40:27.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.411+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fd350014040 con 0x561b6cf9c410 2023-10-05T19:40:27.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.411+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94675792680016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfde100 con 0x561b6cf9c410 2023-10-05T19:40:27.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.415+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94675792680016] v28'131 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fd350014040 con 0x561b6cf9c410 2023-10-05T19:40:27.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.415+0000 7fd32a7e4700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:27.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.415+0000 7fd35ad9de40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:27.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.415+0000 7fd35ad9de40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:27.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.415+0000 7fd35ad9de40 1 Processor -- start 2023-10-05T19:40:27.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.415+0000 7fd35ad9de40 1 -- start start 2023-10-05T19:40:27.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.415+0000 7fd35ad9de40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561b6cfeb040 0x561b6cfeb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.415+0000 7fd35ad9de40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b6cfeb950 con 0x561b6cfeb040 2023-10-05T19:40:27.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.415+0000 7fd3580d1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561b6cfeb040 0x561b6cfeb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.419+0000 7fd3580d1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561b6cfeb040 0x561b6cfeb410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58254/0 (socket says 172.21.15.12:58254) 2023-10-05T19:40:27.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.419+0000 7fd3580d1700 1 -- 172.21.15.12:0/3684157411 learned_addr learned my addr 172.21.15.12:0/3684157411 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:27.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.419+0000 7fd3580d1700 1 -- 172.21.15.12:0/3684157411 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b6cfeba90 con 0x561b6cfeb040 2023-10-05T19:40:27.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.419+0000 7fd3580d1700 1 --2- 172.21.15.12:0/3684157411 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561b6cfeb040 0x561b6cfeb410 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x561b6cf98eb0 tx=0x7fd340010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e3bea674646d436 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.419+0000 7fd329fe3700 1 -- 172.21.15.12:0/3684157411 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd340010bc0 con 0x561b6cfeb040 2023-10-05T19:40:27.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.419+0000 7fd329fe3700 1 -- 172.21.15.12:0/3684157411 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd340036040 con 0x561b6cfeb040 2023-10-05T19:40:27.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.419+0000 7fd329fe3700 1 -- 172.21.15.12:0/3684157411 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd340010d60 con 0x561b6cfeb040 2023-10-05T19:40:27.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.419+0000 7fd35ad9de40 1 -- 172.21.15.12:0/3684157411 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561b6cfeb040 msgr2=0x561b6cfeb410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.419+0000 7fd35ad9de40 1 --2- 172.21.15.12:0/3684157411 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561b6cfeb040 0x561b6cfeb410 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x561b6cf98eb0 tx=0x7fd340010000 comp rx=0 tx=0).stop 2023-10-05T19:40:27.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.419+0000 7fd35ad9de40 1 -- 172.21.15.12:0/3684157411 shutdown_connections 2023-10-05T19:40:27.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.419+0000 7fd35ad9de40 1 --2- 172.21.15.12:0/3684157411 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561b6cfeb040 0x561b6cfeb410 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:27.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.419+0000 7fd35ad9de40 1 -- 172.21.15.12:0/3684157411 >> 172.21.15.12:0/3684157411 conn(0x561b6cfe6400 msgr2=0x561b6cfe45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:27.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.419+0000 7fd35ad9de40 1 -- 172.21.15.12:0/3684157411 shutdown_connections 2023-10-05T19:40:27.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.419+0000 7fd35ad9de40 1 -- 172.21.15.12:0/3684157411 wait complete. 2023-10-05T19:40:27.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.419+0000 7fd35ad9de40 1 Processor -- start 2023-10-05T19:40:27.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.419+0000 7fd35ad9de40 1 -- start start 2023-10-05T19:40:27.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.419+0000 7fd35ad9de40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561b6cfeb040 0x561b6cff0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.419+0000 7fd35ad9de40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b6cff1ab0 con 0x561b6cfeb040 2023-10-05T19:40:27.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.423+0000 7fd3580d1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561b6cfeb040 0x561b6cff0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.423+0000 7fd3580d1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561b6cfeb040 0x561b6cff0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58268/0 (socket says 172.21.15.12:58268) 2023-10-05T19:40:27.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.423+0000 7fd3580d1700 1 -- 172.21.15.12:0/2671933324 learned_addr learned my addr 172.21.15.12:0/2671933324 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:27.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.423+0000 7fd3580d1700 1 -- 172.21.15.12:0/2671933324 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b6cf98a40 con 0x561b6cfeb040 2023-10-05T19:40:27.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.423+0000 7fd3580d1700 1 --2- 172.21.15.12:0/2671933324 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561b6cfeb040 0x561b6cff0ab0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd34003c040 tx=0x7fd340037000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.423+0000 7fd29bfff700 1 -- 172.21.15.12:0/2671933324 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd340030c90 con 0x561b6cfeb040 2023-10-05T19:40:27.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.423+0000 7fd29bfff700 1 -- 172.21.15.12:0/2671933324 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd340030e10 con 0x561b6cfeb040 2023-10-05T19:40:27.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.423+0000 7fd35ad9de40 1 -- 172.21.15.12:0/2671933324 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561b6cff10b0 con 0x561b6cfeb040 2023-10-05T19:40:27.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.423+0000 7fd29bfff700 1 -- 172.21.15.12:0/2671933324 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd34004c440 con 0x561b6cfeb040 2023-10-05T19:40:27.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.423+0000 7fd35ad9de40 1 -- 172.21.15.12:0/2671933324 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561b6cff1550 con 0x561b6cfeb040 2023-10-05T19:40:27.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.423+0000 7fd29bfff700 1 -- 172.21.15.12:0/2671933324 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fd340027400 con 0x561b6cfeb040 2023-10-05T19:40:27.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.423+0000 7fd29bfff700 1 --2- 172.21.15.12:0/2671933324 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd2a0030c50 0x7fd2a0033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.427+0000 7fd29bfff700 1 -- 172.21.15.12:0/2671933324 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7fd34004dc60 con 0x561b6cfeb040 2023-10-05T19:40:27.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.427+0000 7fd3590d3700 1 --2- 172.21.15.12:0/2671933324 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd2a0030c50 0x7fd2a0033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.427+0000 7fd35ad9de40 5 note: GC not initialized 2023-10-05T19:40:27.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.427+0000 7fd3590d3700 1 --2- 172.21.15.12:0/2671933324 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd2a0030c50 0x7fd2a0033100 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x561b6cfdd9e0 tx=0x7fd35000a9b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.427+0000 7fd2997fa700 20 reqs_thread_entry: start 2023-10-05T19:40:27.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.427+0000 7fd35ad9de40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:27.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.427+0000 7fd292ffd700 20 reqs_thread_entry: start 2023-10-05T19:40:27.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.427+0000 7fd35ad9de40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:27.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.427+0000 7fd2917fa700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:57 2023 2023-10-05T19:40:27.437 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:27.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.427+0000 7fd290ff9700 20 reqs_thread_entry: start 2023-10-05T19:40:27.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.427+0000 7fd35ad9de40 20 get_system_obj_state: rctx=0x7ffe40601008 obj=default.rgw.meta:root:testx/myfoo state=0x561b6ce710b0 s->prefetch_data=0 2023-10-05T19:40:27.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.427+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 8.5 8:a7a6c9d8:root::testx%2fmyfoo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6d058fd0 con 0x561b6cf9c410 2023-10-05T19:40:27.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.431+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(102 testx/myfoo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 239+0+139 (crc 0 0 0) 0x7fd350014040 con 0x561b6cf9c410 2023-10-05T19:40:27.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.431+0000 7fd35ad9de40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:27.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.431+0000 7fd35ad9de40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:27.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.431+0000 7fd35ad9de40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.431+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 8.5 8:a7a6c9d8:root::testx%2fmyfoo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6d058fb0 con 0x561b6cf9c410 2023-10-05T19:40:27.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.431+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(103 testx/myfoo [call,call out=48b,read 0~193 out=193b,getxattrs out=75b] v0'0 uv1 ondisk = 0) v8 ==== 281+0+316 (crc 0 0 0) 0x7fd350014040 con 0x561b6cf9c410 2023-10-05T19:40:27.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.431+0000 7fd35ad9de40 20 rados_obj.operate() r=0 bl.length=193 2023-10-05T19:40:27.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.431+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:104 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94675792660464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfad860 con 0x561b6cfcdb90 2023-10-05T19:40:27.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.435+0000 7fd3588d2700 1 -- 172.21.15.12:0/1640501558 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(104 notify.0 [watch unwatch cookie 94675792660464] v28'137 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd348016070 con 0x561b6cfcdb90 2023-10-05T19:40:27.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.435+0000 7fd35ad9de40 20 remove_watcher() i=0 2023-10-05T19:40:27.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.435+0000 7fd35ad9de40 2 removed watcher, disabling cache 2023-10-05T19:40:27.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.435+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:105 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94675792662752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfad860 con 0x561b6cf9c410 2023-10-05T19:40:27.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.439+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(105 notify.1 [watch unwatch cookie 94675792662752] v28'137 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd350014040 con 0x561b6cf9c410 2023-10-05T19:40:27.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.439+0000 7fd35ad9de40 20 remove_watcher() i=1 2023-10-05T19:40:27.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.439+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:106 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94675792664144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfad860 con 0x561b6cfd2510 2023-10-05T19:40:27.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.439+0000 7fd3580d1700 1 -- 172.21.15.12:0/1640501558 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(106 notify.2 [watch unwatch cookie 94675792664144] v28'137 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd340040020 con 0x561b6cfd2510 2023-10-05T19:40:27.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.439+0000 7fd35ad9de40 20 remove_watcher() i=2 2023-10-05T19:40:27.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.439+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:107 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94675792669504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfad860 con 0x561b6cfcdb90 2023-10-05T19:40:27.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.443+0000 7fd3588d2700 1 -- 172.21.15.12:0/1640501558 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(107 notify.3 [watch unwatch cookie 94675792669504] v28'137 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd348016070 con 0x561b6cfcdb90 2023-10-05T19:40:27.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.443+0000 7fd35ad9de40 20 remove_watcher() i=3 2023-10-05T19:40:27.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.443+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:108 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94675792665248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfad860 con 0x561b6cf9c410 2023-10-05T19:40:27.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.447+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(108 notify.4 [watch unwatch cookie 94675792665248] v28'137 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd350014040 con 0x561b6cf9c410 2023-10-05T19:40:27.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.447+0000 7fd35ad9de40 20 remove_watcher() i=4 2023-10-05T19:40:27.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.447+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:109 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94675792675680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfd9b40 con 0x7fd3480176e0 2023-10-05T19:40:27.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.447+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(109 notify.5 [watch unwatch cookie 94675792675680] v28'137 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd350014040 con 0x7fd3480176e0 2023-10-05T19:40:27.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.447+0000 7fd35ad9de40 20 remove_watcher() i=5 2023-10-05T19:40:27.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.447+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:110 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94675792676784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfad860 con 0x7fd3480176e0 2023-10-05T19:40:27.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.451+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(110 notify.6 [watch unwatch cookie 94675792676784] v28'137 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd350014040 con 0x7fd3480176e0 2023-10-05T19:40:27.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.451+0000 7fd35ad9de40 20 remove_watcher() i=6 2023-10-05T19:40:27.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.451+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:111 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94675792678448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6cfdc850 con 0x7fd3480176e0 2023-10-05T19:40:27.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.455+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(111 notify.7 [watch unwatch cookie 94675792678448] v28'137 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd350014040 con 0x7fd3480176e0 2023-10-05T19:40:27.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.455+0000 7fd35ad9de40 20 remove_watcher() i=7 2023-10-05T19:40:27.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.455+0000 7fd32a7e4700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:27.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.455+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:112 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94675792680016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561b6d018670 con 0x561b6cf9c410 2023-10-05T19:40:27.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.459+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(112 data_loggenerations_metadata [watch unwatch cookie 94675792680016] v28'132 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fd350014040 con 0x561b6cf9c410 2023-10-05T19:40:27.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.459+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fd3480176e0 msgr2=0x7fd3480129f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.459+0000 7fd35ad9de40 1 --2- 172.21.15.12:0/1640501558 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fd3480176e0 0x7fd3480129f0 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:27.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.459+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x561b6cf9c410 msgr2=0x561b6cfcbaa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.459+0000 7fd35ad9de40 1 --2- 172.21.15.12:0/1640501558 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x561b6cf9c410 0x561b6cfcbaa0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:27.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.459+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x561b6cfcdb90 msgr2=0x561b6cfcff70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.459+0000 7fd35ad9de40 1 --2- 172.21.15.12:0/1640501558 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x561b6cfcdb90 0x561b6cfcff70 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:27.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.459+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x561b6cfd2510 msgr2=0x561b6cfd48f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.459+0000 7fd35ad9de40 1 --2- 172.21.15.12:0/1640501558 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x561b6cfd2510 0x561b6cfd48f0 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:27.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.459+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd338030c50 msgr2=0x7fd338033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.459+0000 7fd35ad9de40 1 --2- 172.21.15.12:0/1640501558 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd338030c50 0x7fd338033100 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd340000f80 tx=0x7fd340013000 comp rx=0 tx=0).stop 2023-10-05T19:40:27.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.459+0000 7fd3590d3700 1 -- 172.21.15.12:0/1640501558 reap_dead start 2023-10-05T19:40:27.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.459+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561b6cf97e30 msgr2=0x561b6cf0daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.459+0000 7fd35ad9de40 1 --2- 172.21.15.12:0/1640501558 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561b6cf97e30 0x561b6cf0daf0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x561b6cfb36a0 tx=0x7fd348016000 comp rx=0 tx=0).stop 2023-10-05T19:40:27.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.459+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 shutdown_connections 2023-10-05T19:40:27.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.459+0000 7fd35ad9de40 1 --2- 172.21.15.12:0/1640501558 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561b6cf97e30 0x561b6cf0daf0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:27.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.459+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 >> 172.21.15.12:0/1640501558 conn(0x561b6cf93850 msgr2=0x561b6cfb1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:27.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.459+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 shutdown_connections 2023-10-05T19:40:27.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.459+0000 7fd35ad9de40 1 -- 172.21.15.12:0/1640501558 wait complete. 2023-10-05T19:40:27.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.463+0000 7fd35ad9de40 1 -- 172.21.15.12:0/2671933324 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd2a0030c50 msgr2=0x7fd2a0033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.463+0000 7fd35ad9de40 1 --2- 172.21.15.12:0/2671933324 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd2a0030c50 0x7fd2a0033100 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x561b6cfdd9e0 tx=0x7fd35000a9b0 comp rx=0 tx=0).stop 2023-10-05T19:40:27.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.463+0000 7fd35ad9de40 1 -- 172.21.15.12:0/2671933324 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561b6cfeb040 msgr2=0x561b6cff0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.463+0000 7fd35ad9de40 1 --2- 172.21.15.12:0/2671933324 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561b6cfeb040 0x561b6cff0ab0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd34003c040 tx=0x7fd340037000 comp rx=0 tx=0).stop 2023-10-05T19:40:27.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.463+0000 7fd35ad9de40 1 -- 172.21.15.12:0/2671933324 shutdown_connections 2023-10-05T19:40:27.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.463+0000 7fd35ad9de40 1 --2- 172.21.15.12:0/2671933324 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd2a0030c50 0x7fd2a0033100 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:27.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.463+0000 7fd35ad9de40 1 --2- 172.21.15.12:0/2671933324 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561b6cfeb040 0x561b6cff0ab0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:27.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.463+0000 7fd35ad9de40 1 -- 172.21.15.12:0/2671933324 >> 172.21.15.12:0/2671933324 conn(0x561b6cfe6400 msgr2=0x561b6cfe4500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:27.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.463+0000 7fd35ad9de40 1 -- 172.21.15.12:0/2671933324 shutdown_connections 2023-10-05T19:40:27.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.463+0000 7fd35ad9de40 1 -- 172.21.15.12:0/2671933324 wait complete. 2023-10-05T19:40:27.473 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:40:27.474 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "bucket:testx/myfoo", 2023-10-05T19:40:27.474 INFO:teuthology.orchestra.run.smithi012.stdout: "ver": { 2023-10-05T19:40:27.474 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "R4kPhLoxLBV0FfF85d5uWVpA", 2023-10-05T19:40:27.474 INFO:teuthology.orchestra.run.smithi012.stdout: "ver": 2 2023-10-05T19:40:27.474 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:27.474 INFO:teuthology.orchestra.run.smithi012.stdout: "mtime": "2023-10-05T19:40:27.202911Z", 2023-10-05T19:40:27.474 INFO:teuthology.orchestra.run.smithi012.stdout: "data": { 2023-10-05T19:40:27.474 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": { 2023-10-05T19:40:27.474 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "myfoo", 2023-10-05T19:40:27.474 INFO:teuthology.orchestra.run.smithi012.stdout: "marker": "93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1", 2023-10-05T19:40:27.474 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket_id": "93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1", 2023-10-05T19:40:27.475 INFO:teuthology.orchestra.run.smithi012.stdout: "tenant": "testx", 2023-10-05T19:40:27.475 INFO:teuthology.orchestra.run.smithi012.stdout: "explicit_placement": { 2023-10-05T19:40:27.475 INFO:teuthology.orchestra.run.smithi012.stdout: "data_pool": "", 2023-10-05T19:40:27.475 INFO:teuthology.orchestra.run.smithi012.stdout: "data_extra_pool": "", 2023-10-05T19:40:27.475 INFO:teuthology.orchestra.run.smithi012.stdout: "index_pool": "" 2023-10-05T19:40:27.475 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:27.475 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:27.475 INFO:teuthology.orchestra.run.smithi012.stdout: "owner": "testx$tenanteduser", 2023-10-05T19:40:27.475 INFO:teuthology.orchestra.run.smithi012.stdout: "creation_time": "2023-10-05T19:40:22.857926Z", 2023-10-05T19:40:27.475 INFO:teuthology.orchestra.run.smithi012.stdout: "linked": "true", 2023-10-05T19:40:27.475 INFO:teuthology.orchestra.run.smithi012.stdout: "has_bucket_info": "false" 2023-10-05T19:40:27.476 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:27.476 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:40:27.477 DEBUG:tasks.util.rgw: json result: {'key': 'bucket:testx/myfoo', 'ver': {'tag': 'R4kPhLoxLBV0FfF85d5uWVpA', 'ver': 2}, 'mtime': '2023-10-05T19:40:27.202911Z', 'data': {'bucket': {'name': 'myfoo', 'marker': '93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1', 'bucket_id': '93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1', 'tenant': 'testx', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}}, 'owner': 'testx$tenanteduser', 'creation_time': '2023-10-05T19:40:22.857926Z', 'linked': 'true', 'has_bucket_info': 'false'}} 2023-10-05T19:40:27.477 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'link', '--bucket', 'testx/myfoo', '--uid', 'foo'] 2023-10-05T19:40:27.477 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'link', '--bucket', 'testx/myfoo', '--uid', 'foo'] 2023-10-05T19:40:27.478 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket link --bucket testx/myfoo --uid foo 2023-10-05T19:40:27.521 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:27.521 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:27.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.511+0000 7fce8f072e40 1 Processor -- start 2023-10-05T19:40:27.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.511+0000 7fce8f072e40 1 -- start start 2023-10-05T19:40:27.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.511+0000 7fce8f072e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x562a0b0ca7f0 0x562a0b0c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.511+0000 7fce8f072e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x562a0b016bd0 con 0x562a0b0ca420 2023-10-05T19:40:27.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.511+0000 7fce8f072e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x562a0afd3ee0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.515+0000 7fce8c3a6700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x562a0b0ca7f0 0x562a0b0c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.515+0000 7fce8cba7700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x562a0b0ca420 0x562a0b0c8fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:35618/0 (socket says 172.21.15.12:35618) 2023-10-05T19:40:27.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.515+0000 7fce8c3a6700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x562a0b0ca7f0 0x562a0b0c96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58276/0 (socket says 172.21.15.12:58276) 2023-10-05T19:40:27.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.515+0000 7fce8cba7700 1 -- 172.21.15.12:0/2901229895 learned_addr learned my addr 172.21.15.12:0/2901229895 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:27.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.515+0000 7fce8c3a6700 1 -- 172.21.15.12:0/2901229895 >> v1:172.21.15.8:6789/0 conn(0x562a0b0ca420 legacy=0x562a0b0c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.515+0000 7fce8c3a6700 1 -- 172.21.15.12:0/2901229895 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a0b0c9ce0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.515+0000 7fce8c3a6700 1 --2- 172.21.15.12:0/2901229895 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562a0b0ca7f0 0x562a0b0c96e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fce7400a5d0 tx=0x7fce74005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fec09f50b53857dd server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.515+0000 7fce8bba5700 1 -- 172.21.15.12:0/2901229895 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fce740086a0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.515+0000 7fce8bba5700 1 -- 172.21.15.12:0/2901229895 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce74008800 con 0x562a0b0ca7f0 2023-10-05T19:40:27.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.515+0000 7fce8bba5700 1 -- 172.21.15.12:0/2901229895 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fce74008ad0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.515+0000 7fce8f072e40 1 -- 172.21.15.12:0/2901229895 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562a0b0ca7f0 msgr2=0x562a0b0c96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.515+0000 7fce8f072e40 1 --2- 172.21.15.12:0/2901229895 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562a0b0ca7f0 0x562a0b0c96e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fce7400a5d0 tx=0x7fce74005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:27.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.515+0000 7fce8f072e40 1 -- 172.21.15.12:0/2901229895 shutdown_connections 2023-10-05T19:40:27.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.515+0000 7fce8f072e40 1 --2- 172.21.15.12:0/2901229895 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562a0b0ca7f0 0x562a0b0c96e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:27.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.515+0000 7fce8f072e40 1 -- 172.21.15.12:0/2901229895 >> 172.21.15.12:0/2901229895 conn(0x562a0b0c4730 msgr2=0x562a0b0c6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:27.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.515+0000 7fce8f072e40 1 -- 172.21.15.12:0/2901229895 shutdown_connections 2023-10-05T19:40:27.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.515+0000 7fce8f072e40 1 -- 172.21.15.12:0/2901229895 wait complete. 2023-10-05T19:40:27.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.515+0000 7fce8f072e40 1 Processor -- start 2023-10-05T19:40:27.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.515+0000 7fce8f072e40 1 -- start start 2023-10-05T19:40:27.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.519+0000 7fce8f072e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562a0b0ca420 0x562a0b0eb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.519+0000 7fce8f072e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a0af97e30 con 0x562a0b0ca420 2023-10-05T19:40:27.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.519+0000 7fce8cba7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562a0b0ca420 0x562a0b0eb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.519+0000 7fce8cba7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562a0b0ca420 0x562a0b0eb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58288/0 (socket says 172.21.15.12:58288) 2023-10-05T19:40:27.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.519+0000 7fce8cba7700 1 -- 172.21.15.12:0/658249818 learned_addr learned my addr 172.21.15.12:0/658249818 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:27.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.519+0000 7fce8cba7700 1 -- 172.21.15.12:0/658249818 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce74005cf0 con 0x562a0b0ca420 2023-10-05T19:40:27.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.519+0000 7fce8cba7700 1 --2- 172.21.15.12:0/658249818 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562a0b0ca420 0x562a0b0eb130 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fce84001bf0 tx=0x7fce840017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53562cf8afed8569 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.519+0000 7fce8aba3700 1 -- 172.21.15.12:0/658249818 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fce8400bbf0 con 0x562a0b0ca420 2023-10-05T19:40:27.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.519+0000 7fce8aba3700 1 -- 172.21.15.12:0/658249818 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce8400bd50 con 0x562a0b0ca420 2023-10-05T19:40:27.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.519+0000 7fce8aba3700 1 -- 172.21.15.12:0/658249818 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fce8400e470 con 0x562a0b0ca420 2023-10-05T19:40:27.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.519+0000 7fce8f072e40 1 -- 172.21.15.12:0/658249818 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562a0b0ca420 msgr2=0x562a0b0eb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.519+0000 7fce8f072e40 1 --2- 172.21.15.12:0/658249818 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562a0b0ca420 0x562a0b0eb130 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fce84001bf0 tx=0x7fce840017a0 comp rx=0 tx=0).stop 2023-10-05T19:40:27.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.519+0000 7fce8f072e40 1 -- 172.21.15.12:0/658249818 shutdown_connections 2023-10-05T19:40:27.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.519+0000 7fce8f072e40 1 --2- 172.21.15.12:0/658249818 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562a0b0ca420 0x562a0b0eb130 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:27.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.519+0000 7fce8f072e40 1 -- 172.21.15.12:0/658249818 >> 172.21.15.12:0/658249818 conn(0x562a0b0c4730 msgr2=0x562a0b0e6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:27.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.519+0000 7fce8f072e40 1 -- 172.21.15.12:0/658249818 shutdown_connections 2023-10-05T19:40:27.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.519+0000 7fce8f072e40 1 -- 172.21.15.12:0/658249818 wait complete. 2023-10-05T19:40:27.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.519+0000 7fce8f072e40 1 Processor -- start 2023-10-05T19:40:27.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.519+0000 7fce8f072e40 1 -- start start 2023-10-05T19:40:27.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.519+0000 7fce8f072e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562a0b0ca420 0x562a0b0f1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.519+0000 7fce8f072e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a0af6aed0 con 0x562a0b0ca420 2023-10-05T19:40:27.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.519+0000 7fce8cba7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562a0b0ca420 0x562a0b0f1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.519+0000 7fce8cba7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562a0b0ca420 0x562a0b0f1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58298/0 (socket says 172.21.15.12:58298) 2023-10-05T19:40:27.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.519+0000 7fce8cba7700 1 -- 172.21.15.12:0/3227782683 learned_addr learned my addr 172.21.15.12:0/3227782683 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:27.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.523+0000 7fce8cba7700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a0b016bd0 con 0x562a0b0ca420 2023-10-05T19:40:27.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.523+0000 7fce8cba7700 1 --2- 172.21.15.12:0/3227782683 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562a0b0ca420 0x562a0b0f1f00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x562a0b0e7810 tx=0x7fce84016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.523+0000 7fce893a0700 1 -- 172.21.15.12:0/3227782683 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fce8401f070 con 0x562a0b0ca420 2023-10-05T19:40:27.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.523+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562a0b0f2440 con 0x562a0b0ca420 2023-10-05T19:40:27.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.523+0000 7fce893a0700 1 -- 172.21.15.12:0/3227782683 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce84018640 con 0x562a0b0ca420 2023-10-05T19:40:27.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.523+0000 7fce893a0700 1 -- 172.21.15.12:0/3227782683 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fce84018910 con 0x562a0b0ca420 2023-10-05T19:40:27.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.523+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562a0b0f2880 con 0x562a0b0ca420 2023-10-05T19:40:27.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.523+0000 7fce8f072e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.523+0000 7fce893a0700 1 -- 172.21.15.12:0/3227782683 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fce84018b00 con 0x562a0b0ca420 2023-10-05T19:40:27.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.523+0000 7fce893a0700 1 --2- 172.21.15.12:0/3227782683 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fce70030bf0 0x7fce700330a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.523+0000 7fce893a0700 1 -- 172.21.15.12:0/3227782683 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7fce84044fb0 con 0x562a0b0ca420 2023-10-05T19:40:27.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.523+0000 7fce8c3a6700 1 --2- 172.21.15.12:0/3227782683 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fce70030bf0 0x7fce700330a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.527+0000 7fce8f072e40 1 --2- 172.21.15.12:0/3227782683 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x562a0b0ca7f0 0x562a0b0fda80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.527+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b0fe090 con 0x562a0b0ca7f0 2023-10-05T19:40:27.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.527+0000 7fce8d3a8700 1 --2- 172.21.15.12:0/3227782683 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x562a0b0ca7f0 0x562a0b0fda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.527+0000 7fce8c3a6700 1 --2- 172.21.15.12:0/3227782683 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fce70030bf0 0x7fce700330a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fce74005a90 tx=0x7fce74015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.527+0000 7fce8d3a8700 1 --2- 172.21.15.12:0/3227782683 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x562a0b0ca7f0 0x562a0b0fda80 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.527+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fce7c00a5a0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.527+0000 7fce8f072e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:27.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.527+0000 7fce8f072e40 20 realm 2023-10-05T19:40:27.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.527+0000 7fce8f072e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.527+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b0fb0e0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.527+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fce7c00a5a0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.527+0000 7fce8f072e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:27.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.527+0000 7fce8f072e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:27.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.527+0000 7fce8f072e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.527+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b0feb70 con 0x562a0b0ca7f0 2023-10-05T19:40:27.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.527+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fce7c00a5a0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.527+0000 7fce8f072e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:27.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.527+0000 7fce8f072e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.527+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b0fe780 con 0x562a0b0ca7f0 2023-10-05T19:40:27.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.527+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fce7c00a5a0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.527+0000 7fce8f072e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:27.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.527+0000 7fce8f072e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.527+0000 7fce8f072e40 1 --2- 172.21.15.12:0/3227782683 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x562a0b0fff10 0x562a0b1022f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.527+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b102830 con 0x562a0b0fff10 2023-10-05T19:40:27.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.527+0000 7fce8cba7700 1 --2- 172.21.15.12:0/3227782683 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x562a0b0fff10 0x562a0b1022f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.531+0000 7fce8cba7700 1 --2- 172.21.15.12:0/3227782683 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x562a0b0fff10 0x562a0b1022f0 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.531+0000 7fce8cba7700 1 -- 172.21.15.12:0/3227782683 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fce84014070 con 0x562a0b0fff10 2023-10-05T19:40:27.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.531+0000 7fce8f072e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:27.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.531+0000 7fce8f072e40 20 searching for the correct realm 2023-10-05T19:40:27.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.531+0000 7fce8f072e40 1 --2- 172.21.15.12:0/3227782683 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x562a0b1046b0 0x562a0b106ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.531+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107170 con 0x562a0b1046b0 2023-10-05T19:40:27.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.531+0000 7fce8c3a6700 1 --2- 172.21.15.12:0/3227782683 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x562a0b1046b0 0x562a0b106ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.531+0000 7fce8c3a6700 1 --2- 172.21.15.12:0/3227782683 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x562a0b1046b0 0x562a0b106ac0 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.535+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fce74012450 con 0x562a0b1046b0 2023-10-05T19:40:27.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.535+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107170 con 0x562a0b0fff10 2023-10-05T19:40:27.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.535+0000 7fce8cba7700 1 -- 172.21.15.12:0/3227782683 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce84014070 con 0x562a0b0fff10 2023-10-05T19:40:27.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.535+0000 7fce8cba7700 1 --2- 172.21.15.12:0/3227782683 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fce8402d020 0x7fce8400d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.535+0000 7fce8cba7700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107170 con 0x7fce8402d020 2023-10-05T19:40:27.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.535+0000 7fce8d3a8700 1 --2- 172.21.15.12:0/3227782683 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fce8402d020 0x7fce8400d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.535+0000 7fce8d3a8700 1 --2- 172.21.15.12:0/3227782683 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fce8402d020 0x7fce8400d3d0 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.535+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce7c00a5a0 con 0x7fce8402d020 2023-10-05T19:40:27.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.535+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107170 con 0x562a0b0ca7f0 2023-10-05T19:40:27.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.535+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce7c00a5a0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.539+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107170 con 0x562a0b0ca7f0 2023-10-05T19:40:27.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.539+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce7c00a5a0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.539+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107170 con 0x7fce8402d020 2023-10-05T19:40:27.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.539+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce7c00a5a0 con 0x7fce8402d020 2023-10-05T19:40:27.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.539+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107170 con 0x7fce8402d020 2023-10-05T19:40:27.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.539+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce7c00a5a0 con 0x7fce8402d020 2023-10-05T19:40:27.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.539+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107170 con 0x562a0b0fff10 2023-10-05T19:40:27.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.539+0000 7fce8cba7700 1 -- 172.21.15.12:0/3227782683 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce84014070 con 0x562a0b0fff10 2023-10-05T19:40:27.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.539+0000 7fce8cba7700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107170 con 0x562a0b1046b0 2023-10-05T19:40:27.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.543+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce74012450 con 0x562a0b1046b0 2023-10-05T19:40:27.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.543+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107170 con 0x562a0b0ca7f0 2023-10-05T19:40:27.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.543+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce7c00a5a0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.543+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107170 con 0x7fce8402d020 2023-10-05T19:40:27.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.543+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce7c00a5a0 con 0x7fce8402d020 2023-10-05T19:40:27.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.543+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107170 con 0x562a0b0ca7f0 2023-10-05T19:40:27.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.543+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce7c00a5a0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.543+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107170 con 0x562a0b0fff10 2023-10-05T19:40:27.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.543+0000 7fce8cba7700 1 -- 172.21.15.12:0/3227782683 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce84014070 con 0x562a0b0fff10 2023-10-05T19:40:27.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.543+0000 7fce8cba7700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107170 con 0x562a0b1046b0 2023-10-05T19:40:27.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.547+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce74012450 con 0x562a0b1046b0 2023-10-05T19:40:27.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.547+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107170 con 0x562a0b1046b0 2023-10-05T19:40:27.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.547+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce74012450 con 0x562a0b1046b0 2023-10-05T19:40:27.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.547+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107170 con 0x7fce8402d020 2023-10-05T19:40:27.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.547+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce7c00a5a0 con 0x7fce8402d020 2023-10-05T19:40:27.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.547+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107170 con 0x562a0b1046b0 2023-10-05T19:40:27.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.547+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce74012450 con 0x562a0b1046b0 2023-10-05T19:40:27.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.547+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107170 con 0x562a0b1046b0 2023-10-05T19:40:27.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.547+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce74012450 con 0x562a0b1046b0 2023-10-05T19:40:27.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.547+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107170 con 0x562a0b0ca7f0 2023-10-05T19:40:27.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.547+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce7c00a5a0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.547+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107170 con 0x562a0b0ca7f0 2023-10-05T19:40:27.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.551+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce7c00a5a0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.551+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107170 con 0x7fce8402d020 2023-10-05T19:40:27.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.551+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce7c00a5a0 con 0x7fce8402d020 2023-10-05T19:40:27.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.551+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107170 con 0x562a0b1046b0 2023-10-05T19:40:27.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.551+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce74012450 con 0x562a0b1046b0 2023-10-05T19:40:27.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.551+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107170 con 0x562a0b0fff10 2023-10-05T19:40:27.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.551+0000 7fce8cba7700 1 -- 172.21.15.12:0/3227782683 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce84014070 con 0x562a0b0fff10 2023-10-05T19:40:27.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.551+0000 7fce8cba7700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107170 con 0x562a0b0ca7f0 2023-10-05T19:40:27.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.555+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce7c00a5a0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.555+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107170 con 0x7fce8402d020 2023-10-05T19:40:27.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.555+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce7c00a5a0 con 0x7fce8402d020 2023-10-05T19:40:27.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.555+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107170 con 0x562a0b1046b0 2023-10-05T19:40:27.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.555+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce74012450 con 0x562a0b1046b0 2023-10-05T19:40:27.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.555+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107170 con 0x7fce8402d020 2023-10-05T19:40:27.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.555+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce7c00a5a0 con 0x7fce8402d020 2023-10-05T19:40:27.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.555+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107170 con 0x562a0b0fff10 2023-10-05T19:40:27.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.555+0000 7fce8cba7700 1 -- 172.21.15.12:0/3227782683 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce84014070 con 0x562a0b0fff10 2023-10-05T19:40:27.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.559+0000 7fce8cba7700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107170 con 0x562a0b1046b0 2023-10-05T19:40:27.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.559+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce74012450 con 0x562a0b1046b0 2023-10-05T19:40:27.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.559+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107170 con 0x7fce8402d020 2023-10-05T19:40:27.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.559+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce7c00a5a0 con 0x7fce8402d020 2023-10-05T19:40:27.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.559+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107170 con 0x562a0b0ca7f0 2023-10-05T19:40:27.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.559+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fce7c00a5a0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.559+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107170 con 0x562a0b0fff10 2023-10-05T19:40:27.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.559+0000 7fce8cba7700 1 -- 172.21.15.12:0/3227782683 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fce84014070 con 0x562a0b0fff10 2023-10-05T19:40:27.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.559+0000 7fce8f072e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:27.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.559+0000 7fce8f072e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:27.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.559+0000 7fce8f072e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:27.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.559+0000 7fce8f072e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:27.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.559+0000 7fce8f072e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.559+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b103430 con 0x562a0b0ca7f0 2023-10-05T19:40:27.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.563+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fce7c00a5a0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.563+0000 7fce8f072e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:27.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.563+0000 7fce8f072e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.563+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b103060 con 0x562a0b0ca7f0 2023-10-05T19:40:27.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.563+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fce7c00a5a0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.563+0000 7fce8f072e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:27.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.563+0000 7fce8f072e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.563+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b1075e0 con 0x562a0b1046b0 2023-10-05T19:40:27.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.563+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fce74012450 con 0x562a0b1046b0 2023-10-05T19:40:27.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.563+0000 7fce8f072e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:27.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.563+0000 7fce8f072e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.563+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107990 con 0x562a0b1046b0 2023-10-05T19:40:27.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.563+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7fce74012450 con 0x562a0b1046b0 2023-10-05T19:40:27.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.563+0000 7fce8f072e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:27.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.563+0000 7fce8f072e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.563+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107d40 con 0x562a0b0ca7f0 2023-10-05T19:40:27.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.563+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fce7c00a5a0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.563+0000 7fce8f072e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:27.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.563+0000 7fce8f072e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.563+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b108150 con 0x562a0b0ca7f0 2023-10-05T19:40:27.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.563+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fce7c00a5a0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.567+0000 7fce8f072e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:27.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.567+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b108120 con 0x562a0b1046b0 2023-10-05T19:40:27.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.567+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fce74012450 con 0x562a0b1046b0 2023-10-05T19:40:27.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.567+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b108120 con 0x562a0b0fff10 2023-10-05T19:40:27.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.567+0000 7fce8cba7700 1 -- 172.21.15.12:0/3227782683 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce84014070 con 0x562a0b0fff10 2023-10-05T19:40:27.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.567+0000 7fce8cba7700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b108120 con 0x7fce8402d020 2023-10-05T19:40:27.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.571+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce7c00a5a0 con 0x7fce8402d020 2023-10-05T19:40:27.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.571+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107d40 con 0x562a0b0ca7f0 2023-10-05T19:40:27.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.571+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce7c00a5a0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.571+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107d40 con 0x562a0b0ca7f0 2023-10-05T19:40:27.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.571+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce7c00a5a0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.571+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107d40 con 0x7fce8402d020 2023-10-05T19:40:27.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.571+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce7c00a5a0 con 0x7fce8402d020 2023-10-05T19:40:27.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.571+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107d40 con 0x7fce8402d020 2023-10-05T19:40:27.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.571+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce7c00a5a0 con 0x7fce8402d020 2023-10-05T19:40:27.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.575+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107d40 con 0x562a0b0fff10 2023-10-05T19:40:27.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.575+0000 7fce8cba7700 1 -- 172.21.15.12:0/3227782683 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce84014070 con 0x562a0b0fff10 2023-10-05T19:40:27.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.575+0000 7fce8cba7700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107d40 con 0x562a0b1046b0 2023-10-05T19:40:27.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.575+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce74012450 con 0x562a0b1046b0 2023-10-05T19:40:27.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.575+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107d40 con 0x562a0b0ca7f0 2023-10-05T19:40:27.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.575+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce7c00a5a0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.575+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107d40 con 0x7fce8402d020 2023-10-05T19:40:27.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.575+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce7c00a5a0 con 0x7fce8402d020 2023-10-05T19:40:27.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.575+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107d40 con 0x562a0b0ca7f0 2023-10-05T19:40:27.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.579+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce7c00a5a0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.579+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107d40 con 0x562a0b0fff10 2023-10-05T19:40:27.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.579+0000 7fce8cba7700 1 -- 172.21.15.12:0/3227782683 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce84014070 con 0x562a0b0fff10 2023-10-05T19:40:27.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.579+0000 7fce8cba7700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107d40 con 0x562a0b1046b0 2023-10-05T19:40:27.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.579+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce74012450 con 0x562a0b1046b0 2023-10-05T19:40:27.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.579+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107d40 con 0x562a0b1046b0 2023-10-05T19:40:27.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.579+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce74012450 con 0x562a0b1046b0 2023-10-05T19:40:27.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.579+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107d40 con 0x7fce8402d020 2023-10-05T19:40:27.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.579+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce7c00a5a0 con 0x7fce8402d020 2023-10-05T19:40:27.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.579+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107d40 con 0x562a0b1046b0 2023-10-05T19:40:27.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.583+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce74012450 con 0x562a0b1046b0 2023-10-05T19:40:27.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.583+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107d40 con 0x562a0b1046b0 2023-10-05T19:40:27.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.583+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce74012450 con 0x562a0b1046b0 2023-10-05T19:40:27.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.583+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107d40 con 0x562a0b0ca7f0 2023-10-05T19:40:27.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.583+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce7c00a5a0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.583+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107d40 con 0x562a0b0ca7f0 2023-10-05T19:40:27.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.583+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce7c00a5a0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.583+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107d40 con 0x7fce8402d020 2023-10-05T19:40:27.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.587+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce7c00a5a0 con 0x7fce8402d020 2023-10-05T19:40:27.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.587+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107d40 con 0x562a0b1046b0 2023-10-05T19:40:27.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.587+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce74012450 con 0x562a0b1046b0 2023-10-05T19:40:27.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.587+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107d40 con 0x562a0b0fff10 2023-10-05T19:40:27.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.587+0000 7fce8cba7700 1 -- 172.21.15.12:0/3227782683 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce84014070 con 0x562a0b0fff10 2023-10-05T19:40:27.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.587+0000 7fce8cba7700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107d40 con 0x562a0b0ca7f0 2023-10-05T19:40:27.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.587+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce7c00a5a0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.587+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107d40 con 0x7fce8402d020 2023-10-05T19:40:27.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.587+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce7c00a5a0 con 0x7fce8402d020 2023-10-05T19:40:27.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.587+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107d40 con 0x562a0b1046b0 2023-10-05T19:40:27.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.591+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce74012450 con 0x562a0b1046b0 2023-10-05T19:40:27.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.591+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107d40 con 0x7fce8402d020 2023-10-05T19:40:27.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.591+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce7c00a5a0 con 0x7fce8402d020 2023-10-05T19:40:27.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.591+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107d40 con 0x562a0b0fff10 2023-10-05T19:40:27.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.591+0000 7fce8cba7700 1 -- 172.21.15.12:0/3227782683 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce84014070 con 0x562a0b0fff10 2023-10-05T19:40:27.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.591+0000 7fce8cba7700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107d40 con 0x562a0b1046b0 2023-10-05T19:40:27.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.591+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce74012450 con 0x562a0b1046b0 2023-10-05T19:40:27.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.591+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107d40 con 0x7fce8402d020 2023-10-05T19:40:27.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.595+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fce7c00a5a0 con 0x7fce8402d020 2023-10-05T19:40:27.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.595+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107d40 con 0x562a0b0ca7f0 2023-10-05T19:40:27.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.595+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fce7c00a5a0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.595+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b107d40 con 0x562a0b0fff10 2023-10-05T19:40:27.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.595+0000 7fce8cba7700 1 -- 172.21.15.12:0/3227782683 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fce84014070 con 0x562a0b0fff10 2023-10-05T19:40:27.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.595+0000 7fce8f072e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:27.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.595+0000 7fce8f072e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:27.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.595+0000 7fce8f072e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:27.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.595+0000 7fce8f072e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:27.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.595+0000 7fce8f072e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.595+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b108120 con 0x562a0b0ca7f0 2023-10-05T19:40:27.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.595+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fce7c00a5a0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.595+0000 7fce8f072e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:27.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.595+0000 7fce8f072e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.595+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b108580 con 0x562a0b0fff10 2023-10-05T19:40:27.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.595+0000 7fce8cba7700 1 -- 172.21.15.12:0/3227782683 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fce84014070 con 0x562a0b0fff10 2023-10-05T19:40:27.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.595+0000 7fce8f072e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:27.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.595+0000 7fce8f072e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.595+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b108e70 con 0x562a0b0ca7f0 2023-10-05T19:40:27.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.599+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fce7c00a5a0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.599+0000 7fce8f072e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:27.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.599+0000 7fce8f072e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.599+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b108e70 con 0x562a0b1046b0 2023-10-05T19:40:27.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.599+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fce74012450 con 0x562a0b1046b0 2023-10-05T19:40:27.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.599+0000 7fce8f072e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:27.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.599+0000 7fce8f072e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.599+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b1095e0 con 0x7fce8402d020 2023-10-05T19:40:27.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.599+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7fce7c00a5a0 con 0x7fce8402d020 2023-10-05T19:40:27.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.599+0000 7fce8f072e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:27.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.599+0000 7fce8f072e40 20 zone default found 2023-10-05T19:40:27.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.599+0000 7fce8f072e40 4 Realm: () 2023-10-05T19:40:27.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.599+0000 7fce8f072e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:27.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.599+0000 7fce8f072e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:27.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.599+0000 7fce8f072e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:27.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.599+0000 7fce8f072e40 20 zonegroup default 2023-10-05T19:40:27.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.599+0000 7fce8f072e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.599+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b109350 con 0x7fce8402d020 2023-10-05T19:40:27.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.599+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fce7c00a5a0 con 0x7fce8402d020 2023-10-05T19:40:27.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.599+0000 7fce8f072e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:27.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.599+0000 7fce8f072e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.599+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b10acd0 con 0x562a0b1046b0 2023-10-05T19:40:27.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.603+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fce74012450 con 0x562a0b1046b0 2023-10-05T19:40:27.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.603+0000 7fce8f072e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:27.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.603+0000 7fce8f072e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.603+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b10b080 con 0x562a0b1046b0 2023-10-05T19:40:27.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.603+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fce74012450 con 0x562a0b1046b0 2023-10-05T19:40:27.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.603+0000 7fce8f072e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:27.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.603+0000 7fce8f072e40 20 started sync module instance, tier type = 2023-10-05T19:40:27.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.603+0000 7fce8f072e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:27.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.603+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b10a390 con 0x562a0b0fff10 2023-10-05T19:40:27.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.607+0000 7fce8cba7700 1 -- 172.21.15.12:0/3227782683 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'138 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fce84014070 con 0x562a0b0fff10 2023-10-05T19:40:27.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.607+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94738574256016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b108aa0 con 0x562a0b0fff10 2023-10-05T19:40:27.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.607+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b10be10 con 0x562a0b0ca7f0 2023-10-05T19:40:27.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.607+0000 7fce8cba7700 1 -- 172.21.15.12:0/3227782683 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94738574256016] v28'139 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fce84014070 con 0x562a0b0fff10 2023-10-05T19:40:27.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.607+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'138 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fce7c00a5a0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.607+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94738574259536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b10c210 con 0x562a0b0ca7f0 2023-10-05T19:40:27.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.607+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b10cab0 con 0x562a0b1046b0 2023-10-05T19:40:27.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.611+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'138 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fce74012450 con 0x562a0b1046b0 2023-10-05T19:40:27.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.611+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94738574263088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b10d3a0 con 0x562a0b1046b0 2023-10-05T19:40:27.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.611+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b10ddb0 con 0x562a0b0fff10 2023-10-05T19:40:27.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.611+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94738574259536] v28'139 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fce7c00a5a0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.615+0000 7fce8cba7700 1 -- 172.21.15.12:0/3227782683 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'138 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fce84014070 con 0x562a0b0fff10 2023-10-05T19:40:27.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.615+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94738574260640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b10d9e0 con 0x562a0b0fff10 2023-10-05T19:40:27.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.615+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b10ed10 con 0x562a0b0ca7f0 2023-10-05T19:40:27.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.615+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94738574263088] v28'139 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fce74012450 con 0x562a0b1046b0 2023-10-05T19:40:27.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.615+0000 7fce8cba7700 1 -- 172.21.15.12:0/3227782683 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94738574260640] v28'139 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fce84014070 con 0x562a0b0fff10 2023-10-05T19:40:27.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.615+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'138 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fce7c00a5a0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.615+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94738574270816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b10e820 con 0x562a0b0ca7f0 2023-10-05T19:40:27.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.615+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b10f230 con 0x7fce8402d020 2023-10-05T19:40:27.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.619+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94738574270816] v28'139 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fce7c00a5a0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.619+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'138 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fce7c00a5a0 con 0x7fce8402d020 2023-10-05T19:40:27.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.619+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94738574266032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b10f230 con 0x7fce8402d020 2023-10-05T19:40:27.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.619+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b10fad0 con 0x7fce8402d020 2023-10-05T19:40:27.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.623+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(95 notify.5 [watch watch cookie 94738574266032] v28'139 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fce7c00a5a0 con 0x7fce8402d020 2023-10-05T19:40:27.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.623+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(96 notify.6 [create] v28'138 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fce7c00a5a0 con 0x7fce8402d020 2023-10-05T19:40:27.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.623+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94738574274416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b10f8b0 con 0x7fce8402d020 2023-10-05T19:40:27.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.623+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b110530 con 0x7fce8402d020 2023-10-05T19:40:27.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.623+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(97 notify.6 [watch watch cookie 94738574274416] v28'139 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fce7c00a5a0 con 0x7fce8402d020 2023-10-05T19:40:27.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.623+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(98 notify.7 [create] v28'138 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fce7c00a5a0 con 0x7fce8402d020 2023-10-05T19:40:27.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.623+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94738574275792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b110530 con 0x7fce8402d020 2023-10-05T19:40:27.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.623+0000 7fce8f072e40 20 add_watcher() i=0 2023-10-05T19:40:27.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.623+0000 7fce8f072e40 20 add_watcher() i=1 2023-10-05T19:40:27.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.623+0000 7fce8f072e40 20 add_watcher() i=2 2023-10-05T19:40:27.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.623+0000 7fce8f072e40 20 add_watcher() i=3 2023-10-05T19:40:27.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.623+0000 7fce8f072e40 20 add_watcher() i=4 2023-10-05T19:40:27.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.623+0000 7fce8f072e40 20 add_watcher() i=5 2023-10-05T19:40:27.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.623+0000 7fce8f072e40 20 add_watcher() i=6 2023-10-05T19:40:27.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.627+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94738574275792] v28'139 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fce7c00a5a0 con 0x7fce8402d020 2023-10-05T19:40:27.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.627+0000 7fce8f072e40 20 add_watcher() i=7 2023-10-05T19:40:27.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.627+0000 7fce8f072e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:27.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.627+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b1136d0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.627+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fce7c00a5a0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.627+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94738574279088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b110c00 con 0x562a0b0ca7f0 2023-10-05T19:40:27.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.631+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94738574279088] v28'133 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fce7c00a5a0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.631+0000 7fce5d7f2700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:27.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.631+0000 7fce8f072e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:27.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.631+0000 7fce8f072e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:27.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.631+0000 7fce8f072e40 1 Processor -- start 2023-10-05T19:40:27.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.631+0000 7fce8f072e40 1 -- start start 2023-10-05T19:40:27.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.631+0000 7fce8f072e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562a0b11dee0 0x562a0b11e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.631+0000 7fce8f072e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a0b11e7f0 con 0x562a0b11dee0 2023-10-05T19:40:27.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.631+0000 7fce8c3a6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562a0b11dee0 0x562a0b11e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.631+0000 7fce8c3a6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562a0b11dee0 0x562a0b11e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58302/0 (socket says 172.21.15.12:58302) 2023-10-05T19:40:27.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.631+0000 7fce8c3a6700 1 -- 172.21.15.12:0/2451014387 learned_addr learned my addr 172.21.15.12:0/2451014387 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:27.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.631+0000 7fce8c3a6700 1 -- 172.21.15.12:0/2451014387 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a0b11e930 con 0x562a0b11dee0 2023-10-05T19:40:27.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.635+0000 7fce8c3a6700 1 --2- 172.21.15.12:0/2451014387 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562a0b11dee0 0x562a0b11e2b0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x562a0b0c9cb0 tx=0x7fce74012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1974ca831f643047 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.635+0000 7fce5cff1700 1 -- 172.21.15.12:0/2451014387 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fce74030ce0 con 0x562a0b11dee0 2023-10-05T19:40:27.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.635+0000 7fce5cff1700 1 -- 172.21.15.12:0/2451014387 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce74030e40 con 0x562a0b11dee0 2023-10-05T19:40:27.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.635+0000 7fce5cff1700 1 -- 172.21.15.12:0/2451014387 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fce74012b70 con 0x562a0b11dee0 2023-10-05T19:40:27.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.635+0000 7fce8f072e40 1 -- 172.21.15.12:0/2451014387 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562a0b11dee0 msgr2=0x562a0b11e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.635+0000 7fce8f072e40 1 --2- 172.21.15.12:0/2451014387 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562a0b11dee0 0x562a0b11e2b0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x562a0b0c9cb0 tx=0x7fce74012000 comp rx=0 tx=0).stop 2023-10-05T19:40:27.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.635+0000 7fce8f072e40 1 -- 172.21.15.12:0/2451014387 shutdown_connections 2023-10-05T19:40:27.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.635+0000 7fce8f072e40 1 --2- 172.21.15.12:0/2451014387 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562a0b11dee0 0x562a0b11e2b0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:27.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.635+0000 7fce8f072e40 1 -- 172.21.15.12:0/2451014387 >> 172.21.15.12:0/2451014387 conn(0x562a0b1192a0 msgr2=0x562a0b1173f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:27.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.635+0000 7fce8f072e40 1 -- 172.21.15.12:0/2451014387 shutdown_connections 2023-10-05T19:40:27.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.635+0000 7fce8f072e40 1 -- 172.21.15.12:0/2451014387 wait complete. 2023-10-05T19:40:27.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.635+0000 7fce8f072e40 1 Processor -- start 2023-10-05T19:40:27.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.635+0000 7fce8f072e40 1 -- start start 2023-10-05T19:40:27.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.635+0000 7fce8f072e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562a0b11dee0 0x562a0b123910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.635+0000 7fce8f072e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a0b124910 con 0x562a0b11dee0 2023-10-05T19:40:27.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.635+0000 7fce8c3a6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562a0b11dee0 0x562a0b123910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.635+0000 7fce8c3a6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562a0b11dee0 0x562a0b123910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58318/0 (socket says 172.21.15.12:58318) 2023-10-05T19:40:27.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.635+0000 7fce8c3a6700 1 -- 172.21.15.12:0/81053846 learned_addr learned my addr 172.21.15.12:0/81053846 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:27.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.639+0000 7fce8c3a6700 1 -- 172.21.15.12:0/81053846 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a0b0c9ce0 con 0x562a0b11dee0 2023-10-05T19:40:27.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.639+0000 7fce8c3a6700 1 --2- 172.21.15.12:0/81053846 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562a0b11dee0 0x562a0b123910 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x562a0b117f60 tx=0x7fce74034000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.639+0000 7fcdd6ffd700 1 -- 172.21.15.12:0/81053846 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fce74041c90 con 0x562a0b11dee0 2023-10-05T19:40:27.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.639+0000 7fcdd6ffd700 1 -- 172.21.15.12:0/81053846 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce74041df0 con 0x562a0b11dee0 2023-10-05T19:40:27.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.639+0000 7fcdd6ffd700 1 -- 172.21.15.12:0/81053846 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fce74027400 con 0x562a0b11dee0 2023-10-05T19:40:27.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.639+0000 7fce8f072e40 1 -- 172.21.15.12:0/81053846 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562a0b123f10 con 0x562a0b11dee0 2023-10-05T19:40:27.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.639+0000 7fce8f072e40 1 -- 172.21.15.12:0/81053846 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562a0b124350 con 0x562a0b11dee0 2023-10-05T19:40:27.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.639+0000 7fcdd6ffd700 1 -- 172.21.15.12:0/81053846 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fce74049070 con 0x562a0b11dee0 2023-10-05T19:40:27.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.639+0000 7fcdd6ffd700 1 --2- 172.21.15.12:0/81053846 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fcdcc030c00 0x7fcdcc0330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.639+0000 7fcdd6ffd700 1 -- 172.21.15.12:0/81053846 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7fce74055b00 con 0x562a0b11dee0 2023-10-05T19:40:27.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.639+0000 7fce8d3a8700 1 --2- 172.21.15.12:0/81053846 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fcdcc030c00 0x7fcdcc0330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.639+0000 7fce8f072e40 5 note: GC not initialized 2023-10-05T19:40:27.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.643+0000 7fce8d3a8700 1 --2- 172.21.15.12:0/81053846 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fcdcc030c00 0x7fcdcc0330b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x562a0b110b60 tx=0x7fce7c00aeb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.643+0000 7fcdcbfff700 20 reqs_thread_entry: start 2023-10-05T19:40:27.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.643+0000 7fcdc9ffb700 20 reqs_thread_entry: start 2023-10-05T19:40:27.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.643+0000 7fce8f072e40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:27.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.643+0000 7fce8f072e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:27.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.643+0000 7fcdb7fff700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:57 2023 2023-10-05T19:40:27.653 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:27.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.643+0000 7fce8f072e40 20 get_system_obj_state: rctx=0x7ffd250552d8 obj=default.rgw.meta:users.uid:foo state=0x562a0afa30b0 s->prefetch_data=0 2023-10-05T19:40:27.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.643+0000 7fce8f072e40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2023-10-05T19:40:27.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.643+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b18c160 con 0x562a0b0ca7f0 2023-10-05T19:40:27.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.643+0000 7fcdb77fe700 20 reqs_thread_entry: start 2023-10-05T19:40:27.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.643+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(102 foo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7fce7c00a5a0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.647+0000 7fce8f072e40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x16 2023-10-05T19:40:27.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.647+0000 7fce8f072e40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:27.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.647+0000 7fce8f072e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:27.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.647+0000 7fce8f072e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:27.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.647+0000 7fce8f072e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:27.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.647+0000 7fce8f072e40 10 cache get: name=default.rgw.meta+users.uid+foo : type miss (requested=0x13, cached=0x16) 2023-10-05T19:40:27.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.647+0000 7fce8f072e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.647+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b18ebf0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.647+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(103 foo [call,call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv11 ondisk = 0) v8 ==== 273+0+480 (crc 0 0 0) 0x7fce7c00a5a0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.647+0000 7fce8f072e40 20 rados_obj.operate() r=0 bl.length=357 2023-10-05T19:40:27.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.647+0000 7fce8f072e40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2023-10-05T19:40:27.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.647+0000 7fce8f072e40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:27.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.647+0000 7fce8f072e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:27.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.647+0000 7fce8f072e40 20 get_system_obj_state: rctx=0x7ffd25054680 obj=default.rgw.meta:root:testx/myfoo state=0x562a0afa30b0 s->prefetch_data=0 2023-10-05T19:40:27.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.647+0000 7fce8f072e40 10 cache get: name=default.rgw.meta+root+testx/myfoo : miss 2023-10-05T19:40:27.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.647+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:104 8.5 8:a7a6c9d8:root::testx%2fmyfoo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b18e400 con 0x562a0b0ca7f0 2023-10-05T19:40:27.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.647+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(104 testx/myfoo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 239+0+139 (crc 0 0 0) 0x7fce7c00a5a0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.647+0000 7fce8f072e40 10 cache put: name=default.rgw.meta+root+testx/myfoo info.flags=0x16 2023-10-05T19:40:27.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.647+0000 7fce8f072e40 10 adding default.rgw.meta+root+testx/myfoo to cache LRU end 2023-10-05T19:40:27.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.647+0000 7fce8f072e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:27.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.647+0000 7fce8f072e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:27.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.647+0000 7fce8f072e40 10 cache get: name=default.rgw.meta+root+testx/myfoo : type miss (requested=0x11, cached=0x16) 2023-10-05T19:40:27.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.647+0000 7fce8f072e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.647+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:105 8.5 8:a7a6c9d8:root::testx%2fmyfoo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b1909a0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.647+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(105 testx/myfoo [call,call out=48b,read 0~193 out=193b] v0'0 uv1 ondisk = 0) v8 ==== 239+0+241 (crc 0 0 0) 0x7fce7c00a5a0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.647+0000 7fce8f072e40 20 rados_obj.operate() r=0 bl.length=193 2023-10-05T19:40:27.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.647+0000 7fce8f072e40 10 cache put: name=default.rgw.meta+root+testx/myfoo info.flags=0x11 2023-10-05T19:40:27.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.647+0000 7fce8f072e40 10 moving default.rgw.meta+root+testx/myfoo to cache LRU end 2023-10-05T19:40:27.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.647+0000 7fce8f072e40 20 get_system_obj_state: rctx=0x7ffd25054680 obj=default.rgw.meta:root:.bucket.meta.testx:myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 state=0x562a0af792b0 s->prefetch_data=0 2023-10-05T19:40:27.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.647+0000 7fce8f072e40 10 cache get: name=default.rgw.meta+root+.bucket.meta.testx:myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 : miss 2023-10-05T19:40:27.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.647+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:106 8.1f 8:f8fdc702:root::.bucket.meta.testx%3amyfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b18fd60 con 0x562a0b0fff10 2023-10-05T19:40:27.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.647+0000 7fce8cba7700 1 -- 172.21.15.12:0/3227782683 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(106 .bucket.meta.testx:myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 [call out=48b,getxattrs out=298b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 296+0+362 (crc 0 0 0) 0x7fce84014070 con 0x562a0b0fff10 2023-10-05T19:40:27.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.647+0000 7fce8f072e40 10 cache put: name=default.rgw.meta+root+.bucket.meta.testx:myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 info.flags=0x16 2023-10-05T19:40:27.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.647+0000 7fce8f072e40 10 adding default.rgw.meta+root+.bucket.meta.testx:myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 to cache LRU end 2023-10-05T19:40:27.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.647+0000 7fce8f072e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:27.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.647+0000 7fce8f072e40 10 updating xattr: name=user.rgw.acl bl.length()=203 2023-10-05T19:40:27.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.647+0000 7fce8f072e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:27.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.647+0000 7fce8f072e40 20 Read xattr: user.rgw.acl 2023-10-05T19:40:27.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.647+0000 7fce8f072e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:27.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.647+0000 7fce8f072e40 10 cache get: name=default.rgw.meta+root+.bucket.meta.testx:myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 : type miss (requested=0x13, cached=0x16) 2023-10-05T19:40:27.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.647+0000 7fce8f072e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.647+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:107 8.1f 8:f8fdc702:root::.bucket.meta.testx%3amyfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b192a20 con 0x562a0b0fff10 2023-10-05T19:40:27.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.651+0000 7fce8cba7700 1 -- 172.21.15.12:0/3227782683 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(107 .bucket.meta.testx:myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 [call,call out=48b,read 0~371 out=371b,getxattrs out=298b] v0'0 uv1 ondisk = 0) v8 ==== 338+0+717 (crc 0 0 0) 0x7fce84014070 con 0x562a0b0fff10 2023-10-05T19:40:27.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.651+0000 7fce8f072e40 20 rados_obj.operate() r=0 bl.length=371 2023-10-05T19:40:27.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.651+0000 7fce8f072e40 10 cache put: name=default.rgw.meta+root+.bucket.meta.testx:myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 info.flags=0x13 2023-10-05T19:40:27.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.651+0000 7fce8f072e40 10 moving default.rgw.meta+root+.bucket.meta.testx:myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 to cache LRU end 2023-10-05T19:40:27.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.651+0000 7fce8f072e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:27.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.651+0000 7fce8f072e40 10 updating xattr: name=user.rgw.acl bl.length()=203 2023-10-05T19:40:27.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.651+0000 7fce8f072e40 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.testx:myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 2023-10-05T19:40:27.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.651+0000 7fce8f072e40 20 get_system_obj_state: rctx=0x7ffd25054618 obj=default.rgw.meta:users.uid:foo state=0x562a0af792b0 s->prefetch_data=0 2023-10-05T19:40:27.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.651+0000 7fce8f072e40 10 cache get: name=default.rgw.meta+users.uid+foo : type miss (requested=0x16, cached=0x13) 2023-10-05T19:40:27.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.651+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:108 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b193150 con 0x562a0b0ca7f0 2023-10-05T19:40:27.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.651+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(108 foo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7fce7c00a5a0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.651+0000 7fce8f072e40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x16 2023-10-05T19:40:27.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.651+0000 7fce8f072e40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:27.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.651+0000 7fce8f072e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:27.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.651+0000 7fce8f072e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:27.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.651+0000 7fce8f072e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:27.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.651+0000 7fce8f072e40 10 cache get: name=default.rgw.meta+users.uid+foo : hit (requested=0x13, cached=0x17) 2023-10-05T19:40:27.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.651+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:109 8.c 8:37c7a57e:users.uid::testx$tenanteduser.buckets:head [call user.remove_bucket in=58b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b195170 con 0x562a0b0ca7f0 2023-10-05T19:40:27.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.655+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== osd_op_reply(109 testx$tenanteduser.buckets [call] v28'2 uv2 ondisk = 0) v8 ==== 170+0+0 (crc 0 0 0) 0x7fce7c00a5a0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.655+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:110 8.e 8:70176bbd:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1:head [create,call version.inc in=38b,writefull 0~352 in=352b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b198a40 con 0x7fce8402d020 2023-10-05T19:40:27.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.655+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(110 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 [create,call,writefull 0~352,setxattr (123)] v28'7 uv7 ondisk = 0) v8 ==== 332+0+0 (crc 0 0 0) 0x7fce7c00a5a0 con 0x7fce8402d020 2023-10-05T19:40:27.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.655+0000 7fce8f072e40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 info.flags=0x7 2023-10-05T19:40:27.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.655+0000 7fce8f072e40 10 adding default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 to cache LRU end 2023-10-05T19:40:27.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.655+0000 7fce8f072e40 10 updating xattr: name=user.rgw.acl bl.length()=123 2023-10-05T19:40:27.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.655+0000 7fce8f072e40 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1, ofs0, ns] 2023-10-05T19:40:27.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.655+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:111 7.e 7:7759931f:::notify.3:head [notify cookie 94738574836752 in=713b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b196670 con 0x562a0b0fff10 2023-10-05T19:40:27.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.659+0000 7fce8cba7700 1 -- 172.21.15.12:0/3227782683 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== watch-notify(notify (1) cookie 94738574260640 notify 120259084305 ret 0) v3 ==== 743+0+0 (crc 0 0 0) 0x562a0b0f2880 con 0x562a0b0fff10 2023-10-05T19:40:27.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.659+0000 7fce8a3a2700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084305 cookie 94738574260640 notifier 4501 bl.length()=701 2023-10-05T19:40:27.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.659+0000 7fce8a3a2700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 info.flags=0x7 2023-10-05T19:40:27.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.659+0000 7fce8a3a2700 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 to cache LRU end 2023-10-05T19:40:27.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.659+0000 7fce8a3a2700 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2023-10-05T19:40:27.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.659+0000 7fce8a3a2700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:112 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fce64003d90 con 0x562a0b0fff10 2023-10-05T19:40:27.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.659+0000 7fce8cba7700 1 -- 172.21.15.12:0/3227782683 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(111 notify.3 [notify cookie 94738574836752 out=8b] v0'0 uv138 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fce84014070 con 0x562a0b0fff10 2023-10-05T19:40:27.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.659+0000 7fce8cba7700 1 -- 172.21.15.12:0/3227782683 <== osd.2 v2:172.21.15.12:6800/29934 23 ==== osd_op_reply(112 notify.3 [notify-ack] v0'0 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fce84014070 con 0x562a0b0fff10 2023-10-05T19:40:27.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.659+0000 7fce8cba7700 1 -- 172.21.15.12:0/3227782683 <== osd.2 v2:172.21.15.12:6800/29934 24 ==== watch-notify(notify_complete (2) cookie 94738574836752 notify 120259084305 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x562a0b0f2440 con 0x562a0b0fff10 2023-10-05T19:40:27.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.659+0000 7fce8f072e40 20 handle_bi_update(): bucket=:myfoo[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1]): orig_sources= new_sources= 2023-10-05T19:40:27.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.659+0000 7fce8f072e40 20 handle_bi_update(): bucket=:myfoo[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1]): potential sources added=[] removed=[] 2023-10-05T19:40:27.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.659+0000 7fce8f072e40 20 handle_bi_update(): bucket=:myfoo[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1]): orig_dests= new_dests= 2023-10-05T19:40:27.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.659+0000 7fce8f072e40 20 handle_bi_update(): bucket=:myfoo[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1]): potential dests added=[] removed=[] 2023-10-05T19:40:27.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.659+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:113 8.9 8:96d98480:users.uid::foo.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b198410 con 0x562a0b0fff10 2023-10-05T19:40:27.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.663+0000 7fce8cba7700 1 -- 172.21.15.12:0/3227782683 <== osd.2 v2:172.21.15.12:6800/29934 25 ==== osd_op_reply(113 foo.buckets [call] v28'26 uv26 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x7fce84014070 con 0x562a0b0fff10 2023-10-05T19:40:27.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.663+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:114 8.9 8:91b71841:root::myfoo:head [delete,create,call version.inc in=38b,writefull 0~165 in=165b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b196180 con 0x562a0b0fff10 2023-10-05T19:40:27.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.663+0000 7fce8cba7700 1 -- 172.21.15.12:0/3227782683 <== osd.2 v2:172.21.15.12:6800/29934 26 ==== osd_op_reply(114 myfoo [delete,create,call,writefull 0~165] v28'27 uv27 ondisk = 0) v8 ==== 275+0+0 (crc 0 0 0) 0x7fce84014070 con 0x562a0b0fff10 2023-10-05T19:40:27.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.663+0000 7fce8f072e40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2023-10-05T19:40:27.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.663+0000 7fce8f072e40 10 adding default.rgw.meta+root+myfoo to cache LRU end 2023-10-05T19:40:27.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.663+0000 7fce8f072e40 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2023-10-05T19:40:27.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.663+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:115 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94738574830736 in=326b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b196180 con 0x7fce8402d020 2023-10-05T19:40:27.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.667+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== watch-notify(notify (1) cookie 94738574274416 notify 120259084309 ret 0) v3 ==== 356+0+0 (crc 0 0 0) 0x7fce7c02d040 con 0x7fce8402d020 2023-10-05T19:40:27.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.667+0000 7fce8aba3700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084309 cookie 94738574274416 notifier 4501 bl.length()=314 2023-10-05T19:40:27.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.667+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(115 notify.6 [notify cookie 94738574830736 out=8b] v0'0 uv138 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fce7c00a5a0 con 0x7fce8402d020 2023-10-05T19:40:27.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.667+0000 7fce8aba3700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2023-10-05T19:40:27.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.667+0000 7fce8aba3700 10 rgw watcher librados: moving default.rgw.meta+root+myfoo to cache LRU end 2023-10-05T19:40:27.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.667+0000 7fce8aba3700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:116 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fcddc003400 con 0x7fce8402d020 2023-10-05T19:40:27.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.667+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.0 v2:172.21.15.8:6800/27563 30 ==== watch-notify(notify_complete (2) cookie 94738574830736 notify 120259084309 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7fce7c037ce0 con 0x7fce8402d020 2023-10-05T19:40:27.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.667+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.0 v2:172.21.15.8:6800/27563 31 ==== osd_op_reply(116 notify.6 [notify-ack] v0'0 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fce7c00a5a0 con 0x7fce8402d020 2023-10-05T19:40:27.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.667+0000 7fce8f072e40 10 removing default.rgw.meta+root+testx/myfoo from cache 2023-10-05T19:40:27.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.667+0000 7fce8f072e40 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:root:testx/myfoo, ofs0, ns] 2023-10-05T19:40:27.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.667+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:117 7.d 7:b4812045:::notify.4:head [notify cookie 94738574830672 in=167b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b194650 con 0x562a0b0ca7f0 2023-10-05T19:40:27.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.667+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.1 v2:172.21.15.12:6808/29933 39 ==== watch-notify(notify (1) cookie 94738574270816 notify 120259084300 ret 0) v3 ==== 197+0+0 (crc 0 0 0) 0x7fce7c037ce0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.667+0000 7fce8aba3700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084300 cookie 94738574270816 notifier 4501 bl.length()=155 2023-10-05T19:40:27.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.667+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.1 v2:172.21.15.12:6808/29933 40 ==== osd_op_reply(117 notify.4 [notify cookie 94738574830672 out=8b] v0'0 uv138 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fce7c00a5a0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.667+0000 7fce8aba3700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:118 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fcddc003440 con 0x562a0b0ca7f0 2023-10-05T19:40:27.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.671+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.1 v2:172.21.15.12:6808/29933 41 ==== watch-notify(notify_complete (2) cookie 94738574830672 notify 120259084300 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7fce7c037040 con 0x562a0b0ca7f0 2023-10-05T19:40:27.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.671+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.1 v2:172.21.15.12:6808/29933 42 ==== osd_op_reply(118 notify.4 [notify-ack] v0'0 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fce7c00a5a0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.671+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:119 8.5 8:a7a6c9d8:root::testx%2fmyfoo:head [call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b194160 con 0x562a0b0ca7f0 2023-10-05T19:40:27.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.671+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.1 v2:172.21.15.12:6808/29933 43 ==== osd_op_reply(119 testx/myfoo [call,delete] v28'2 uv2 ondisk = 0) v8 ==== 197+0+0 (crc 0 0 0) 0x7fce7c00a5a0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.671+0000 7fce8f072e40 10 removing default.rgw.meta+root+.bucket.meta.testx:myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 from cache 2023-10-05T19:40:27.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.671+0000 7fce8f072e40 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.testx:myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1, ofs0, ns] 2023-10-05T19:40:27.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.671+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:120 7.15 7:a93a5511:::notify.2:head [notify cookie 94738574803824 in=224b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b197400 con 0x562a0b1046b0 2023-10-05T19:40:27.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.675+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== watch-notify(notify (1) cookie 94738574263088 notify 120259084294 ret 0) v3 ==== 254+0+0 (crc 0 0 0) 0x562a0b124350 con 0x562a0b1046b0 2023-10-05T19:40:27.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.675+0000 7fce8a3a2700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084294 cookie 94738574263088 notifier 4501 bl.length()=212 2023-10-05T19:40:27.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.675+0000 7fce8a3a2700 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:121 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fce64003d90 con 0x562a0b1046b0 2023-10-05T19:40:27.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.675+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 <== osd.3 v2:172.21.15.12:6816/29951 27 ==== osd_op_reply(120 notify.2 [notify cookie 94738574803824 out=8b] v0'0 uv138 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fce74012450 con 0x562a0b1046b0 2023-10-05T19:40:27.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.675+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 <== osd.3 v2:172.21.15.12:6816/29951 28 ==== watch-notify(notify_complete (2) cookie 94738574803824 notify 120259084294 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x562a0b123f10 con 0x562a0b1046b0 2023-10-05T19:40:27.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.675+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 <== osd.3 v2:172.21.15.12:6816/29951 29 ==== osd_op_reply(121 notify.2 [notify-ack] v0'0 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fce7405a040 con 0x562a0b1046b0 2023-10-05T19:40:27.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.675+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:122 8.1f 8:f8fdc702:root::.bucket.meta.testx%3amyfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1:head [call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b197400 con 0x562a0b0fff10 2023-10-05T19:40:27.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.675+0000 7fce8cba7700 1 -- 172.21.15.12:0/3227782683 <== osd.2 v2:172.21.15.12:6800/29934 27 ==== osd_op_reply(122 .bucket.meta.testx:myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 [call,delete] v28'2 uv2 ondisk = 0) v8 ==== 254+0+0 (crc 0 0 0) 0x7fce84014070 con 0x562a0b0fff10 2023-10-05T19:40:27.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.679+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:123 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94738574256016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b197400 con 0x562a0b0fff10 2023-10-05T19:40:27.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.679+0000 7fce8cba7700 1 -- 172.21.15.12:0/3227782683 <== osd.2 v2:172.21.15.12:6800/29934 28 ==== osd_op_reply(123 notify.0 [watch unwatch cookie 94738574256016] v28'140 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fce84014070 con 0x562a0b0fff10 2023-10-05T19:40:27.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.679+0000 7fce8f072e40 20 remove_watcher() i=0 2023-10-05T19:40:27.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.679+0000 7fce8f072e40 2 removed watcher, disabling cache 2023-10-05T19:40:27.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.679+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:124 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94738574259536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b197400 con 0x562a0b0ca7f0 2023-10-05T19:40:27.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.683+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.1 v2:172.21.15.12:6808/29933 44 ==== osd_op_reply(124 notify.1 [watch unwatch cookie 94738574259536] v28'140 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fce7c00a5a0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.683+0000 7fce8f072e40 20 remove_watcher() i=1 2023-10-05T19:40:27.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.683+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:125 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94738574263088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b197400 con 0x562a0b1046b0 2023-10-05T19:40:27.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.687+0000 7fce8c3a6700 1 -- 172.21.15.12:0/3227782683 <== osd.3 v2:172.21.15.12:6816/29951 30 ==== osd_op_reply(125 notify.2 [watch unwatch cookie 94738574263088] v28'140 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fce7405a040 con 0x562a0b1046b0 2023-10-05T19:40:27.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.687+0000 7fce8f072e40 20 remove_watcher() i=2 2023-10-05T19:40:27.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.687+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:126 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94738574260640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b0e1a10 con 0x562a0b0fff10 2023-10-05T19:40:27.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.687+0000 7fce8cba7700 1 -- 172.21.15.12:0/3227782683 <== osd.2 v2:172.21.15.12:6800/29934 29 ==== osd_op_reply(126 notify.3 [watch unwatch cookie 94738574260640] v28'140 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fce84014070 con 0x562a0b0fff10 2023-10-05T19:40:27.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.687+0000 7fce8f072e40 20 remove_watcher() i=3 2023-10-05T19:40:27.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.687+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:127 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94738574270816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b0e1a10 con 0x562a0b0ca7f0 2023-10-05T19:40:27.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.691+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.1 v2:172.21.15.12:6808/29933 45 ==== osd_op_reply(127 notify.4 [watch unwatch cookie 94738574270816] v28'140 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fce7c00a5a0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.691+0000 7fce8f072e40 20 remove_watcher() i=4 2023-10-05T19:40:27.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.691+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:128 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94738574266032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b0e1a10 con 0x7fce8402d020 2023-10-05T19:40:27.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.695+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.0 v2:172.21.15.8:6800/27563 32 ==== osd_op_reply(128 notify.5 [watch unwatch cookie 94738574266032] v28'140 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fce7c00a5a0 con 0x7fce8402d020 2023-10-05T19:40:27.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.695+0000 7fce8f072e40 20 remove_watcher() i=5 2023-10-05T19:40:27.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.695+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:129 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94738574274416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b0e1a10 con 0x7fce8402d020 2023-10-05T19:40:27.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.695+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.0 v2:172.21.15.8:6800/27563 33 ==== osd_op_reply(129 notify.6 [watch unwatch cookie 94738574274416] v28'140 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fce7c00a5a0 con 0x7fce8402d020 2023-10-05T19:40:27.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.695+0000 7fce8f072e40 20 remove_watcher() i=6 2023-10-05T19:40:27.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.695+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:130 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94738574275792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b0e1a10 con 0x7fce8402d020 2023-10-05T19:40:27.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.699+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.0 v2:172.21.15.8:6800/27563 34 ==== osd_op_reply(130 notify.7 [watch unwatch cookie 94738574275792] v28'140 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fce7c00a5a0 con 0x7fce8402d020 2023-10-05T19:40:27.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.699+0000 7fce8f072e40 20 remove_watcher() i=7 2023-10-05T19:40:27.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.703+0000 7fce5d7f2700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:27.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.703+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:131 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94738574279088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x562a0b0e1a10 con 0x562a0b0ca7f0 2023-10-05T19:40:27.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.707+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 <== osd.1 v2:172.21.15.12:6808/29933 46 ==== osd_op_reply(131 data_loggenerations_metadata [watch unwatch cookie 94738574279088] v28'134 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fce7c00a5a0 con 0x562a0b0ca7f0 2023-10-05T19:40:27.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.707+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fce8402d020 msgr2=0x7fce8400d3d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.707+0000 7fce8f072e40 1 --2- 172.21.15.12:0/3227782683 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fce8402d020 0x7fce8400d3d0 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:27.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.707+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x562a0b0ca7f0 msgr2=0x562a0b0fda80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.707+0000 7fce8f072e40 1 --2- 172.21.15.12:0/3227782683 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x562a0b0ca7f0 0x562a0b0fda80 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:27.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.707+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x562a0b0fff10 msgr2=0x562a0b1022f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.707+0000 7fce8f072e40 1 --2- 172.21.15.12:0/3227782683 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x562a0b0fff10 0x562a0b1022f0 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:27.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.707+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x562a0b1046b0 msgr2=0x562a0b106ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.707+0000 7fce8f072e40 1 --2- 172.21.15.12:0/3227782683 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x562a0b1046b0 0x562a0b106ac0 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:27.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.707+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fce70030bf0 msgr2=0x7fce700330a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.707+0000 7fce8f072e40 1 --2- 172.21.15.12:0/3227782683 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fce70030bf0 0x7fce700330a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fce74005a90 tx=0x7fce74015000 comp rx=0 tx=0).stop 2023-10-05T19:40:27.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.707+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562a0b0ca420 msgr2=0x562a0b0f1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.707+0000 7fce8f072e40 1 --2- 172.21.15.12:0/3227782683 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562a0b0ca420 0x562a0b0f1f00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x562a0b0e7810 tx=0x7fce84016000 comp rx=0 tx=0).stop 2023-10-05T19:40:27.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.707+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 reap_dead start 2023-10-05T19:40:27.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.707+0000 7fce8d3a8700 1 -- 172.21.15.12:0/3227782683 reap_dead start 2023-10-05T19:40:27.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.707+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 shutdown_connections 2023-10-05T19:40:27.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.707+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 >> 172.21.15.12:0/3227782683 conn(0x562a0b0c4730 msgr2=0x562a0b0e5770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:27.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.707+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 shutdown_connections 2023-10-05T19:40:27.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.707+0000 7fce8f072e40 1 -- 172.21.15.12:0/3227782683 wait complete. 2023-10-05T19:40:27.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.707+0000 7fce8f072e40 1 -- 172.21.15.12:0/81053846 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fcdcc030c00 msgr2=0x7fcdcc0330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.707+0000 7fce8f072e40 1 --2- 172.21.15.12:0/81053846 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fcdcc030c00 0x7fcdcc0330b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x562a0b110b60 tx=0x7fce7c00aeb0 comp rx=0 tx=0).stop 2023-10-05T19:40:27.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.707+0000 7fce8f072e40 1 -- 172.21.15.12:0/81053846 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562a0b11dee0 msgr2=0x562a0b123910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.707+0000 7fce8f072e40 1 --2- 172.21.15.12:0/81053846 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562a0b11dee0 0x562a0b123910 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x562a0b117f60 tx=0x7fce74034000 comp rx=0 tx=0).stop 2023-10-05T19:40:27.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.707+0000 7fce8f072e40 1 -- 172.21.15.12:0/81053846 shutdown_connections 2023-10-05T19:40:27.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.707+0000 7fce8f072e40 1 --2- 172.21.15.12:0/81053846 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fcdcc030c00 0x7fcdcc0330b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:27.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.707+0000 7fce8f072e40 1 --2- 172.21.15.12:0/81053846 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562a0b11dee0 0x562a0b123910 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:27.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.707+0000 7fce8f072e40 1 -- 172.21.15.12:0/81053846 >> 172.21.15.12:0/81053846 conn(0x562a0b1192a0 msgr2=0x562a0b117340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:27.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.707+0000 7fce8f072e40 1 -- 172.21.15.12:0/81053846 shutdown_connections 2023-10-05T19:40:27.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.707+0000 7fce8f072e40 1 -- 172.21.15.12:0/81053846 wait complete. 2023-10-05T19:40:27.723 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--tenant', 'testx', '--uid', 'tenanteduser'] 2023-10-05T19:40:27.723 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--tenant', 'testx', '--uid', 'tenanteduser'] 2023-10-05T19:40:27.723 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --tenant testx --uid tenanteduser 2023-10-05T19:40:27.806 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:27.806 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.799+0000 7f30d5234e40 1 Processor -- start 2023-10-05T19:40:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.799+0000 7f30d5234e40 1 -- start start 2023-10-05T19:40:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.799+0000 7f30d5234e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x559f03d20e40 0x559f03d21210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.799+0000 7f30d5234e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x559f03c6dbd0 con 0x559f03d21750 2023-10-05T19:40:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.799+0000 7f30d5234e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x559f03c2aee0 con 0x559f03d20e40 2023-10-05T19:40:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.799+0000 7f30d2d69700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x559f03d20e40 0x559f03d21210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.799+0000 7f30d2568700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x559f03d21750 0x559f03d221f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:35634/0 (socket says 172.21.15.12:35634) 2023-10-05T19:40:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.799+0000 7f30d2568700 1 -- 172.21.15.12:0/635392765 learned_addr learned my addr 172.21.15.12:0/635392765 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:27.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.799+0000 7f30d2d69700 1 -- 172.21.15.12:0/635392765 >> v1:172.21.15.8:6789/0 conn(0x559f03d21750 legacy=0x559f03d221f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.799+0000 7f30d1d67700 1 -- 172.21.15.12:0/635392765 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3896939213 0 0) 0x559f03c6dbd0 con 0x559f03d21750 2023-10-05T19:40:27.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.799+0000 7f30d2d69700 1 -- 172.21.15.12:0/635392765 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f03d22900 con 0x559f03d20e40 2023-10-05T19:40:27.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.799+0000 7f30d2d69700 1 --2- 172.21.15.12:0/635392765 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559f03d20e40 0x559f03d21210 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f30cc00a5d0 tx=0x7f30cc005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae5bb744d54e08fc server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.799+0000 7f30d1d67700 1 -- 172.21.15.12:0/635392765 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30cc0086a0 con 0x559f03d20e40 2023-10-05T19:40:27.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.799+0000 7f30d1d67700 1 -- 172.21.15.12:0/635392765 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30cc008800 con 0x559f03d20e40 2023-10-05T19:40:27.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.799+0000 7f30d1d67700 1 -- 172.21.15.12:0/635392765 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30cc008ad0 con 0x559f03d20e40 2023-10-05T19:40:27.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.803+0000 7f30d5234e40 1 -- 172.21.15.12:0/635392765 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559f03d20e40 msgr2=0x559f03d21210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.803+0000 7f30d5234e40 1 --2- 172.21.15.12:0/635392765 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559f03d20e40 0x559f03d21210 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f30cc00a5d0 tx=0x7f30cc005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:27.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.803+0000 7f30d5234e40 1 -- 172.21.15.12:0/635392765 shutdown_connections 2023-10-05T19:40:27.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.803+0000 7f30d5234e40 1 --2- 172.21.15.12:0/635392765 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559f03d20e40 0x559f03d21210 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:27.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.803+0000 7f30d5234e40 1 -- 172.21.15.12:0/635392765 >> 172.21.15.12:0/635392765 conn(0x559f03d1c860 msgr2=0x559f03d1ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:27.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.803+0000 7f30d5234e40 1 -- 172.21.15.12:0/635392765 shutdown_connections 2023-10-05T19:40:27.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.803+0000 7f30d5234e40 1 -- 172.21.15.12:0/635392765 wait complete. 2023-10-05T19:40:27.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.803+0000 7f30d5234e40 1 Processor -- start 2023-10-05T19:40:27.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.803+0000 7f30d5234e40 1 -- start start 2023-10-05T19:40:27.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.803+0000 7f30d5234e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559f03d20e40 0x559f03c968f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.803+0000 7f30d5234e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f03beee30 con 0x559f03d20e40 2023-10-05T19:40:27.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.803+0000 7f30d2d69700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559f03d20e40 0x559f03c968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.803+0000 7f30d2d69700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559f03d20e40 0x559f03c968f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58334/0 (socket says 172.21.15.12:58334) 2023-10-05T19:40:27.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.803+0000 7f30d2d69700 1 -- 172.21.15.12:0/777743008 learned_addr learned my addr 172.21.15.12:0/777743008 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:27.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.803+0000 7f30d2d69700 1 -- 172.21.15.12:0/777743008 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30cc005cf0 con 0x559f03d20e40 2023-10-05T19:40:27.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.803+0000 7f30d2d69700 1 --2- 172.21.15.12:0/777743008 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559f03d20e40 0x559f03c968f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f30cc005cc0 tx=0x7f30cc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff9ec169a0e0fefb server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.803+0000 7f30d0d65700 1 -- 172.21.15.12:0/777743008 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30cc008970 con 0x559f03d20e40 2023-10-05T19:40:27.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.803+0000 7f30d0d65700 1 -- 172.21.15.12:0/777743008 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30cc008ad0 con 0x559f03d20e40 2023-10-05T19:40:27.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.803+0000 7f30d0d65700 1 -- 172.21.15.12:0/777743008 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30cc008dc0 con 0x559f03d20e40 2023-10-05T19:40:27.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.807+0000 7f30d5234e40 1 -- 172.21.15.12:0/777743008 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559f03d20e40 msgr2=0x559f03c968f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.807+0000 7f30d5234e40 1 --2- 172.21.15.12:0/777743008 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559f03d20e40 0x559f03c968f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f30cc005cc0 tx=0x7f30cc015000 comp rx=0 tx=0).stop 2023-10-05T19:40:27.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.807+0000 7f30d5234e40 1 -- 172.21.15.12:0/777743008 shutdown_connections 2023-10-05T19:40:27.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.807+0000 7f30d5234e40 1 --2- 172.21.15.12:0/777743008 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559f03d20e40 0x559f03c968f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:27.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.807+0000 7f30d5234e40 1 -- 172.21.15.12:0/777743008 >> 172.21.15.12:0/777743008 conn(0x559f03d1c860 msgr2=0x559f03d3b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:27.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.807+0000 7f30d5234e40 1 -- 172.21.15.12:0/777743008 shutdown_connections 2023-10-05T19:40:27.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.807+0000 7f30d5234e40 1 -- 172.21.15.12:0/777743008 wait complete. 2023-10-05T19:40:27.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.807+0000 7f30d5234e40 1 Processor -- start 2023-10-05T19:40:27.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.807+0000 7f30d5234e40 1 -- start start 2023-10-05T19:40:27.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.807+0000 7f30d5234e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559f03d20e40 0x559f03d48b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.807+0000 7f30d5234e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f03bc1ed0 con 0x559f03d20e40 2023-10-05T19:40:27.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.807+0000 7f30d2d69700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559f03d20e40 0x559f03d48b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.807+0000 7f30d2d69700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559f03d20e40 0x559f03d48b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58338/0 (socket says 172.21.15.12:58338) 2023-10-05T19:40:27.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.807+0000 7f30d2d69700 1 -- 172.21.15.12:0/3508051254 learned_addr learned my addr 172.21.15.12:0/3508051254 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:27.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.807+0000 7f30d2d69700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f03d22900 con 0x559f03d20e40 2023-10-05T19:40:27.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.807+0000 7f30d2d69700 1 --2- 172.21.15.12:0/3508051254 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559f03d20e40 0x559f03d48b10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f30cc01e040 tx=0x7f30cc01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.807+0000 7f30caffd700 1 -- 172.21.15.12:0/3508051254 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30cc012b70 con 0x559f03d20e40 2023-10-05T19:40:27.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.807+0000 7f30caffd700 1 -- 172.21.15.12:0/3508051254 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30cc02a070 con 0x559f03d20e40 2023-10-05T19:40:27.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.807+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559f03d49050 con 0x559f03d20e40 2023-10-05T19:40:27.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.807+0000 7f30caffd700 1 -- 172.21.15.12:0/3508051254 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30cc012e30 con 0x559f03d20e40 2023-10-05T19:40:27.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.807+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f03d494f0 con 0x559f03d20e40 2023-10-05T19:40:27.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.811+0000 7f30d5234e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.811+0000 7f30caffd700 1 -- 172.21.15.12:0/3508051254 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f30cc019070 con 0x559f03d20e40 2023-10-05T19:40:27.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.811+0000 7f30caffd700 1 --2- 172.21.15.12:0/3508051254 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f30b4030c90 0x7f30b4033140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.811+0000 7f30caffd700 1 -- 172.21.15.12:0/3508051254 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f30cc02c070 con 0x559f03d20e40 2023-10-05T19:40:27.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.811+0000 7f30d2568700 1 --2- 172.21.15.12:0/3508051254 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f30b4030c90 0x7f30b4033140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.811+0000 7f30d5234e40 1 --2- 172.21.15.12:0/3508051254 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x559f03d21750 0x559f03d546d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.811+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d54cf0 con 0x559f03d21750 2023-10-05T19:40:27.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.811+0000 7f30d356a700 1 --2- 172.21.15.12:0/3508051254 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x559f03d21750 0x559f03d546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.811+0000 7f30d356a700 1 --2- 172.21.15.12:0/3508051254 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x559f03d21750 0x559f03d546d0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.811+0000 7f30d2568700 1 --2- 172.21.15.12:0/3508051254 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f30b4030c90 0x7f30b4033140 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f30bc001560 tx=0x7f30bc006650 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.815+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f30c400a500 con 0x559f03d21750 2023-10-05T19:40:27.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.815+0000 7f30d5234e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:27.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.815+0000 7f30d5234e40 20 realm 2023-10-05T19:40:27.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.815+0000 7f30d5234e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.815+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d51d30 con 0x559f03d21750 2023-10-05T19:40:27.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.815+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f30c400a500 con 0x559f03d21750 2023-10-05T19:40:27.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.815+0000 7f30d5234e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:27.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.815+0000 7f30d5234e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:27.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.815+0000 7f30d5234e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.815+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d557d0 con 0x559f03d21750 2023-10-05T19:40:27.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.815+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f30c400a500 con 0x559f03d21750 2023-10-05T19:40:27.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.815+0000 7f30d5234e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:27.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.815+0000 7f30d5234e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.815+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d553e0 con 0x559f03d21750 2023-10-05T19:40:27.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.815+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f30c400a500 con 0x559f03d21750 2023-10-05T19:40:27.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.815+0000 7f30d5234e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:27.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.815+0000 7f30d5234e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.815+0000 7f30d5234e40 1 --2- 172.21.15.12:0/3508051254 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x559f03d56b70 0x559f03d58f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.815+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d59490 con 0x559f03d56b70 2023-10-05T19:40:27.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.815+0000 7f30d2d69700 1 --2- 172.21.15.12:0/3508051254 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x559f03d56b70 0x559f03d58f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.815+0000 7f30d2d69700 1 --2- 172.21.15.12:0/3508051254 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x559f03d56b70 0x559f03d58f50 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.815+0000 7f30d2d69700 1 -- 172.21.15.12:0/3508051254 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f30cc042040 con 0x559f03d56b70 2023-10-05T19:40:27.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.815+0000 7f30d5234e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:27.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.815+0000 7f30d5234e40 20 searching for the correct realm 2023-10-05T19:40:27.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.815+0000 7f30d5234e40 1 --2- 172.21.15.12:0/3508051254 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x559f03d5b310 0x559f03d5d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.815+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5ddd0 con 0x559f03d5b310 2023-10-05T19:40:27.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.815+0000 7f30d2568700 1 --2- 172.21.15.12:0/3508051254 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x559f03d5b310 0x559f03d5d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.819+0000 7f30d2568700 1 --2- 172.21.15.12:0/3508051254 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x559f03d5b310 0x559f03d5d720 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.819+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f30bc004520 con 0x559f03d5b310 2023-10-05T19:40:27.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.819+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5ddd0 con 0x559f03d56b70 2023-10-05T19:40:27.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.819+0000 7f30d2d69700 1 -- 172.21.15.12:0/3508051254 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30cc042040 con 0x559f03d56b70 2023-10-05T19:40:27.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.819+0000 7f30d2d69700 1 --2- 172.21.15.12:0/3508051254 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f30cc007030 0x7f30cc031480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.819+0000 7f30d2d69700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5ddd0 con 0x7f30cc007030 2023-10-05T19:40:27.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.819+0000 7f30d356a700 1 --2- 172.21.15.12:0/3508051254 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f30cc007030 0x7f30cc031480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.823+0000 7f30d356a700 1 --2- 172.21.15.12:0/3508051254 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f30cc007030 0x7f30cc031480 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.823+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30c4014040 con 0x7f30cc007030 2023-10-05T19:40:27.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.823+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5ddd0 con 0x559f03d21750 2023-10-05T19:40:27.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.823+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d21750 2023-10-05T19:40:27.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.823+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5ddd0 con 0x559f03d21750 2023-10-05T19:40:27.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.823+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d21750 2023-10-05T19:40:27.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.823+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5ddd0 con 0x7f30cc007030 2023-10-05T19:40:27.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.823+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30c4014040 con 0x7f30cc007030 2023-10-05T19:40:27.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.823+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5ddd0 con 0x7f30cc007030 2023-10-05T19:40:27.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.827+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30c4014040 con 0x7f30cc007030 2023-10-05T19:40:27.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.827+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5ddd0 con 0x559f03d56b70 2023-10-05T19:40:27.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.827+0000 7f30d2d69700 1 -- 172.21.15.12:0/3508051254 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30cc042040 con 0x559f03d56b70 2023-10-05T19:40:27.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.827+0000 7f30d2d69700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5ddd0 con 0x559f03d5b310 2023-10-05T19:40:27.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.827+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30bc004520 con 0x559f03d5b310 2023-10-05T19:40:27.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.827+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5ddd0 con 0x559f03d21750 2023-10-05T19:40:27.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.827+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d21750 2023-10-05T19:40:27.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.827+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5ddd0 con 0x7f30cc007030 2023-10-05T19:40:27.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.827+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30c4014040 con 0x7f30cc007030 2023-10-05T19:40:27.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.827+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5ddd0 con 0x559f03d21750 2023-10-05T19:40:27.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.831+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d21750 2023-10-05T19:40:27.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.831+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5ddd0 con 0x559f03d56b70 2023-10-05T19:40:27.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.831+0000 7f30d2d69700 1 -- 172.21.15.12:0/3508051254 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30cc042040 con 0x559f03d56b70 2023-10-05T19:40:27.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.831+0000 7f30d2d69700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5ddd0 con 0x559f03d5b310 2023-10-05T19:40:27.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.831+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30bc004520 con 0x559f03d5b310 2023-10-05T19:40:27.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.831+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5ddd0 con 0x559f03d5b310 2023-10-05T19:40:27.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.831+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30bc004520 con 0x559f03d5b310 2023-10-05T19:40:27.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.831+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5ddd0 con 0x7f30cc007030 2023-10-05T19:40:27.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.831+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30c4014040 con 0x7f30cc007030 2023-10-05T19:40:27.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.831+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5ddd0 con 0x559f03d5b310 2023-10-05T19:40:27.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.835+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30bc004520 con 0x559f03d5b310 2023-10-05T19:40:27.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.835+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5ddd0 con 0x559f03d5b310 2023-10-05T19:40:27.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.835+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30bc004520 con 0x559f03d5b310 2023-10-05T19:40:27.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.835+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5ddd0 con 0x559f03d21750 2023-10-05T19:40:27.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.835+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d21750 2023-10-05T19:40:27.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.835+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5ddd0 con 0x559f03d21750 2023-10-05T19:40:27.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.835+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d21750 2023-10-05T19:40:27.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.835+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5ddd0 con 0x7f30cc007030 2023-10-05T19:40:27.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.835+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30c4014040 con 0x7f30cc007030 2023-10-05T19:40:27.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.835+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5ddd0 con 0x559f03d5b310 2023-10-05T19:40:27.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.839+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30bc004520 con 0x559f03d5b310 2023-10-05T19:40:27.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.839+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5ddd0 con 0x559f03d56b70 2023-10-05T19:40:27.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.839+0000 7f30d2d69700 1 -- 172.21.15.12:0/3508051254 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30cc042040 con 0x559f03d56b70 2023-10-05T19:40:27.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.839+0000 7f30d2d69700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5ddd0 con 0x559f03d21750 2023-10-05T19:40:27.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.839+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d21750 2023-10-05T19:40:27.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.839+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5ddd0 con 0x7f30cc007030 2023-10-05T19:40:27.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.839+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30c4014040 con 0x7f30cc007030 2023-10-05T19:40:27.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.839+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5ddd0 con 0x559f03d5b310 2023-10-05T19:40:27.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.839+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30bc004520 con 0x559f03d5b310 2023-10-05T19:40:27.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.839+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5ddd0 con 0x7f30cc007030 2023-10-05T19:40:27.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.843+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30c4014040 con 0x7f30cc007030 2023-10-05T19:40:27.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.843+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5ddd0 con 0x559f03d56b70 2023-10-05T19:40:27.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.843+0000 7f30d2d69700 1 -- 172.21.15.12:0/3508051254 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30cc042040 con 0x559f03d56b70 2023-10-05T19:40:27.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.843+0000 7f30d2d69700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5ddd0 con 0x559f03d5b310 2023-10-05T19:40:27.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.843+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30bc004520 con 0x559f03d5b310 2023-10-05T19:40:27.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.843+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5ddd0 con 0x7f30cc007030 2023-10-05T19:40:27.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.843+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30c4014040 con 0x7f30cc007030 2023-10-05T19:40:27.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.843+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5ddd0 con 0x559f03d21750 2023-10-05T19:40:27.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.847+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d21750 2023-10-05T19:40:27.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.847+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5ddd0 con 0x559f03d56b70 2023-10-05T19:40:27.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.847+0000 7f30d2d69700 1 -- 172.21.15.12:0/3508051254 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f30cc042040 con 0x559f03d56b70 2023-10-05T19:40:27.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.847+0000 7f30d5234e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:27.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.847+0000 7f30d5234e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:27.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.847+0000 7f30d5234e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:27.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.847+0000 7f30d5234e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:27.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.847+0000 7f30d5234e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.847+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5a090 con 0x559f03d21750 2023-10-05T19:40:27.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.847+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d21750 2023-10-05T19:40:27.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.847+0000 7f30d5234e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:27.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.847+0000 7f30d5234e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.847+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d59cc0 con 0x559f03d21750 2023-10-05T19:40:27.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.847+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d21750 2023-10-05T19:40:27.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.847+0000 7f30d5234e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:27.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.847+0000 7f30d5234e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.847+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5e240 con 0x559f03d5b310 2023-10-05T19:40:27.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.847+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f30bc004520 con 0x559f03d5b310 2023-10-05T19:40:27.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.847+0000 7f30d5234e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:27.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.851+0000 7f30d5234e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.851+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5e5f0 con 0x559f03d5b310 2023-10-05T19:40:27.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.851+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f30bc004520 con 0x559f03d5b310 2023-10-05T19:40:27.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.851+0000 7f30d5234e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:27.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.851+0000 7f30d5234e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.851+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5e9a0 con 0x559f03d21750 2023-10-05T19:40:27.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.851+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d21750 2023-10-05T19:40:27.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.851+0000 7f30d5234e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:27.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.851+0000 7f30d5234e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.851+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5a5b0 con 0x559f03d21750 2023-10-05T19:40:27.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.851+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d21750 2023-10-05T19:40:27.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.851+0000 7f30d5234e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:27.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.851+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5a5b0 con 0x559f03d5b310 2023-10-05T19:40:27.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.855+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f30bc004520 con 0x559f03d5b310 2023-10-05T19:40:27.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.855+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5a5b0 con 0x559f03d56b70 2023-10-05T19:40:27.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.855+0000 7f30d2d69700 1 -- 172.21.15.12:0/3508051254 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30cc042040 con 0x559f03d56b70 2023-10-05T19:40:27.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.855+0000 7f30d2d69700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5a5b0 con 0x7f30cc007030 2023-10-05T19:40:27.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.855+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30c4014040 con 0x7f30cc007030 2023-10-05T19:40:27.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.855+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5e9a0 con 0x559f03d21750 2023-10-05T19:40:27.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.855+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d21750 2023-10-05T19:40:27.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.855+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5e9a0 con 0x559f03d21750 2023-10-05T19:40:27.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.859+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d21750 2023-10-05T19:40:27.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.859+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5e9a0 con 0x7f30cc007030 2023-10-05T19:40:27.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.859+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30c4014040 con 0x7f30cc007030 2023-10-05T19:40:27.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.859+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5e9a0 con 0x7f30cc007030 2023-10-05T19:40:27.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.859+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30c4014040 con 0x7f30cc007030 2023-10-05T19:40:27.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.859+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5e9a0 con 0x559f03d56b70 2023-10-05T19:40:27.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.859+0000 7f30d2d69700 1 -- 172.21.15.12:0/3508051254 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30cc042040 con 0x559f03d56b70 2023-10-05T19:40:27.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.859+0000 7f30d2d69700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5e9a0 con 0x559f03d5b310 2023-10-05T19:40:27.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.863+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30bc02d040 con 0x559f03d5b310 2023-10-05T19:40:27.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.863+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5e9a0 con 0x559f03d21750 2023-10-05T19:40:27.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.863+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d21750 2023-10-05T19:40:27.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.863+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5e9a0 con 0x7f30cc007030 2023-10-05T19:40:27.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.863+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30c4014040 con 0x7f30cc007030 2023-10-05T19:40:27.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.863+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5e9a0 con 0x559f03d21750 2023-10-05T19:40:27.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.863+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d21750 2023-10-05T19:40:27.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.863+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5e9a0 con 0x559f03d56b70 2023-10-05T19:40:27.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.867+0000 7f30d2d69700 1 -- 172.21.15.12:0/3508051254 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30cc042040 con 0x559f03d56b70 2023-10-05T19:40:27.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.867+0000 7f30d2d69700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5e9a0 con 0x559f03d5b310 2023-10-05T19:40:27.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.867+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30bc02d040 con 0x559f03d5b310 2023-10-05T19:40:27.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.867+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5e9a0 con 0x559f03d5b310 2023-10-05T19:40:27.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.867+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30bc02d040 con 0x559f03d5b310 2023-10-05T19:40:27.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.867+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5e9a0 con 0x7f30cc007030 2023-10-05T19:40:27.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.867+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30c4014040 con 0x7f30cc007030 2023-10-05T19:40:27.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.867+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5e9a0 con 0x559f03d5b310 2023-10-05T19:40:27.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.867+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30bc02d040 con 0x559f03d5b310 2023-10-05T19:40:27.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.871+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5e9a0 con 0x559f03d5b310 2023-10-05T19:40:27.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.871+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30bc02d040 con 0x559f03d5b310 2023-10-05T19:40:27.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.871+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5e9a0 con 0x559f03d21750 2023-10-05T19:40:27.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.871+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d21750 2023-10-05T19:40:27.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.871+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5e9a0 con 0x559f03d21750 2023-10-05T19:40:27.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.871+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d21750 2023-10-05T19:40:27.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.871+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5e9a0 con 0x7f30cc007030 2023-10-05T19:40:27.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.871+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30c4014040 con 0x7f30cc007030 2023-10-05T19:40:27.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.871+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5e9a0 con 0x559f03d5b310 2023-10-05T19:40:27.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.875+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30bc02d040 con 0x559f03d5b310 2023-10-05T19:40:27.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.875+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5e9a0 con 0x559f03d56b70 2023-10-05T19:40:27.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.875+0000 7f30d2d69700 1 -- 172.21.15.12:0/3508051254 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30cc042040 con 0x559f03d56b70 2023-10-05T19:40:27.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.875+0000 7f30d2d69700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5e9a0 con 0x559f03d21750 2023-10-05T19:40:27.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.875+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d21750 2023-10-05T19:40:27.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.875+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5e9a0 con 0x7f30cc007030 2023-10-05T19:40:27.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.875+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30c4014040 con 0x7f30cc007030 2023-10-05T19:40:27.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.875+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5e9a0 con 0x559f03d5b310 2023-10-05T19:40:27.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.875+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30bc02d040 con 0x559f03d5b310 2023-10-05T19:40:27.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.875+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5e9a0 con 0x7f30cc007030 2023-10-05T19:40:27.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.879+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30c4014040 con 0x7f30cc007030 2023-10-05T19:40:27.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.879+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5e9a0 con 0x559f03d56b70 2023-10-05T19:40:27.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.879+0000 7f30d2d69700 1 -- 172.21.15.12:0/3508051254 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30cc042040 con 0x559f03d56b70 2023-10-05T19:40:27.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.879+0000 7f30d2d69700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5e9a0 con 0x559f03d5b310 2023-10-05T19:40:27.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.879+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30bc02d040 con 0x559f03d5b310 2023-10-05T19:40:27.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.879+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5e9a0 con 0x7f30cc007030 2023-10-05T19:40:27.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.879+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f30c4014040 con 0x7f30cc007030 2023-10-05T19:40:27.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.879+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5e9a0 con 0x559f03d21750 2023-10-05T19:40:27.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.883+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d21750 2023-10-05T19:40:27.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.883+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5e9a0 con 0x559f03d56b70 2023-10-05T19:40:27.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.883+0000 7f30d2d69700 1 -- 172.21.15.12:0/3508051254 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f30cc042040 con 0x559f03d56b70 2023-10-05T19:40:27.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.883+0000 7f30d5234e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:27.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.883+0000 7f30d5234e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:27.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.883+0000 7f30d5234e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:27.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.883+0000 7f30d5234e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:27.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.883+0000 7f30d5234e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.883+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5a5b0 con 0x559f03d21750 2023-10-05T19:40:27.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.883+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d21750 2023-10-05T19:40:27.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.883+0000 7f30d5234e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:27.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.883+0000 7f30d5234e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.883+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5f340 con 0x559f03d56b70 2023-10-05T19:40:27.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.883+0000 7f30d2d69700 1 -- 172.21.15.12:0/3508051254 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f30cc042040 con 0x559f03d56b70 2023-10-05T19:40:27.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.883+0000 7f30d5234e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:27.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.883+0000 7f30d5234e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.883+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5fc30 con 0x559f03d21750 2023-10-05T19:40:27.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.887+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d21750 2023-10-05T19:40:27.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.887+0000 7f30d5234e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:27.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.887+0000 7f30d5234e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.887+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5fc30 con 0x559f03d5b310 2023-10-05T19:40:27.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.887+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f30bc02d040 con 0x559f03d5b310 2023-10-05T19:40:27.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.887+0000 7f30d5234e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:27.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.887+0000 7f30d5234e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.887+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d603a0 con 0x7f30cc007030 2023-10-05T19:40:27.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.887+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f30c4014040 con 0x7f30cc007030 2023-10-05T19:40:27.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.887+0000 7f30d5234e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:27.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.887+0000 7f30d5234e40 20 zone default found 2023-10-05T19:40:27.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.887+0000 7f30d5234e40 4 Realm: () 2023-10-05T19:40:27.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.887+0000 7f30d5234e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:27.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.887+0000 7f30d5234e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:27.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.887+0000 7f30d5234e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:27.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.887+0000 7f30d5234e40 20 zonegroup default 2023-10-05T19:40:27.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.887+0000 7f30d5234e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.887+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5ffe0 con 0x7f30cc007030 2023-10-05T19:40:27.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.887+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f30c4014040 con 0x7f30cc007030 2023-10-05T19:40:27.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.887+0000 7f30d5234e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:27.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.887+0000 7f30d5234e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.887+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d61960 con 0x559f03d5b310 2023-10-05T19:40:27.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.891+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f30bc02d040 con 0x559f03d5b310 2023-10-05T19:40:27.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.891+0000 7f30d5234e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:27.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.891+0000 7f30d5234e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:27.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.891+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d61d10 con 0x559f03d5b310 2023-10-05T19:40:27.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.891+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f30bc03c040 con 0x559f03d5b310 2023-10-05T19:40:27.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.891+0000 7f30d5234e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:27.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.891+0000 7f30d5234e40 20 started sync module instance, tier type = 2023-10-05T19:40:27.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.891+0000 7f30d5234e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:27.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.891+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d61020 con 0x559f03d56b70 2023-10-05T19:40:27.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.895+0000 7f30d2d69700 1 -- 172.21.15.12:0/3508051254 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'141 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f30cc042040 con 0x559f03d56b70 2023-10-05T19:40:27.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.895+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94141452522528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d5f860 con 0x559f03d56b70 2023-10-05T19:40:27.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.895+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d62b30 con 0x559f03d21750 2023-10-05T19:40:27.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.895+0000 7f30d2d69700 1 -- 172.21.15.12:0/3508051254 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94141452522528] v28'142 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f30cc042040 con 0x559f03d56b70 2023-10-05T19:40:27.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.895+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'141 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d21750 2023-10-05T19:40:27.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.895+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94141452525760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d62b30 con 0x559f03d21750 2023-10-05T19:40:27.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.899+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d63420 con 0x559f03d5b310 2023-10-05T19:40:27.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.899+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'141 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f30bc03c040 con 0x559f03d5b310 2023-10-05T19:40:27.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.899+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94141452529456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d63d10 con 0x559f03d5b310 2023-10-05T19:40:27.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.899+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d64720 con 0x559f03d56b70 2023-10-05T19:40:27.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.899+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94141452525760] v28'142 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d21750 2023-10-05T19:40:27.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.903+0000 7f30d2d69700 1 -- 172.21.15.12:0/3508051254 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'141 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f30cc042040 con 0x559f03d56b70 2023-10-05T19:40:27.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.903+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94141452534320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d62760 con 0x559f03d56b70 2023-10-05T19:40:27.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.903+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d65330 con 0x559f03d21750 2023-10-05T19:40:27.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.903+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94141452529456] v28'142 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f30bc03c040 con 0x559f03d5b310 2023-10-05T19:40:27.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.903+0000 7f30d2d69700 1 -- 172.21.15.12:0/3508051254 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94141452534320] v28'142 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f30cc042040 con 0x559f03d56b70 2023-10-05T19:40:27.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.903+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'141 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d21750 2023-10-05T19:40:27.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.903+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94141452531744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d62760 con 0x559f03d21750 2023-10-05T19:40:27.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.903+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d647f0 con 0x7f30cc007030 2023-10-05T19:40:27.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.907+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'141 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f30c4014040 con 0x7f30cc007030 2023-10-05T19:40:27.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.907+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94141452539552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d62760 con 0x7f30cc007030 2023-10-05T19:40:27.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.907+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d666d0 con 0x7f30cc007030 2023-10-05T19:40:27.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.907+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94141452531744] v28'142 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d21750 2023-10-05T19:40:27.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.911+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(95 notify.5 [watch watch cookie 94141452539552] v28'142 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f30c4014040 con 0x7f30cc007030 2023-10-05T19:40:27.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.911+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(96 notify.6 [create] v28'141 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f30c4014040 con 0x7f30cc007030 2023-10-05T19:40:27.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.911+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94141452537344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d661e0 con 0x7f30cc007030 2023-10-05T19:40:27.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.911+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d66bf0 con 0x7f30cc007030 2023-10-05T19:40:27.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.911+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(97 notify.6 [watch watch cookie 94141452537344] v28'142 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f30c4014040 con 0x7f30cc007030 2023-10-05T19:40:27.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.915+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(98 notify.7 [create] v28'141 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f30c4014040 con 0x7f30cc007030 2023-10-05T19:40:27.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.915+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94141452540656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d66bf0 con 0x7f30cc007030 2023-10-05T19:40:27.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.915+0000 7f30d5234e40 20 add_watcher() i=0 2023-10-05T19:40:27.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.915+0000 7f30d5234e40 20 add_watcher() i=1 2023-10-05T19:40:27.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.915+0000 7f30d5234e40 20 add_watcher() i=2 2023-10-05T19:40:27.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.915+0000 7f30d5234e40 20 add_watcher() i=3 2023-10-05T19:40:27.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.915+0000 7f30d5234e40 20 add_watcher() i=4 2023-10-05T19:40:27.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.915+0000 7f30d5234e40 20 add_watcher() i=5 2023-10-05T19:40:27.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.915+0000 7f30d5234e40 20 add_watcher() i=6 2023-10-05T19:40:27.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.915+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94141452540656] v28'142 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f30c4014040 con 0x7f30cc007030 2023-10-05T19:40:27.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.915+0000 7f30d5234e40 20 add_watcher() i=7 2023-10-05T19:40:27.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.915+0000 7f30d5234e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:27.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.915+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d69d90 con 0x559f03d21750 2023-10-05T19:40:27.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.919+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d21750 2023-10-05T19:40:27.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.919+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94141452544112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d67320 con 0x559f03d21750 2023-10-05T19:40:27.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.919+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94141452544112] v28'135 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d21750 2023-10-05T19:40:27.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.919+0000 7f301ffff700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:27.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.919+0000 7f30d5234e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:27.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.919+0000 7f30d5234e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:27.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.919+0000 7f30d5234e40 1 Processor -- start 2023-10-05T19:40:27.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.919+0000 7f30d5234e40 1 -- start start 2023-10-05T19:40:27.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.919+0000 7f30d5234e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559f03d74580 0x559f03d74950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.919+0000 7f30d5234e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f03d74e90 con 0x559f03d74580 2023-10-05T19:40:27.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.923+0000 7f30d2568700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559f03d74580 0x559f03d74950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.923+0000 7f30d2568700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559f03d74580 0x559f03d74950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58344/0 (socket says 172.21.15.12:58344) 2023-10-05T19:40:27.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.923+0000 7f30d2568700 1 -- 172.21.15.12:0/1645236821 learned_addr learned my addr 172.21.15.12:0/1645236821 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:27.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.923+0000 7f30d2568700 1 -- 172.21.15.12:0/1645236821 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f03d74fd0 con 0x559f03d74580 2023-10-05T19:40:27.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.923+0000 7f30d2568700 1 --2- 172.21.15.12:0/1645236821 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559f03d74580 0x559f03d74950 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x559f03d5e180 tx=0x7f30bc027040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f8f469530b0a6d server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.923+0000 7f301f7fe700 1 -- 172.21.15.12:0/1645236821 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30bc039040 con 0x559f03d74580 2023-10-05T19:40:27.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.923+0000 7f301f7fe700 1 -- 172.21.15.12:0/1645236821 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30bc03a040 con 0x559f03d74580 2023-10-05T19:40:27.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.923+0000 7f301f7fe700 1 -- 172.21.15.12:0/1645236821 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30bc02a080 con 0x559f03d74580 2023-10-05T19:40:27.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.923+0000 7f30d5234e40 1 -- 172.21.15.12:0/1645236821 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559f03d74580 msgr2=0x559f03d74950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:27.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.923+0000 7f30d5234e40 1 --2- 172.21.15.12:0/1645236821 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559f03d74580 0x559f03d74950 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x559f03d5e180 tx=0x7f30bc027040 comp rx=0 tx=0).stop 2023-10-05T19:40:27.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.923+0000 7f30d5234e40 1 -- 172.21.15.12:0/1645236821 shutdown_connections 2023-10-05T19:40:27.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.923+0000 7f30d5234e40 1 --2- 172.21.15.12:0/1645236821 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559f03d74580 0x559f03d74950 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:27.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.923+0000 7f30d5234e40 1 -- 172.21.15.12:0/1645236821 >> 172.21.15.12:0/1645236821 conn(0x559f03d6f940 msgr2=0x559f03d6da90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:27.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.923+0000 7f30d5234e40 1 -- 172.21.15.12:0/1645236821 shutdown_connections 2023-10-05T19:40:27.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.923+0000 7f30d5234e40 1 -- 172.21.15.12:0/1645236821 wait complete. 2023-10-05T19:40:27.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.923+0000 7f30d5234e40 1 Processor -- start 2023-10-05T19:40:27.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.923+0000 7f30d5234e40 1 -- start start 2023-10-05T19:40:27.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.927+0000 7f30d5234e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559f03d74580 0x559f03d7a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.927+0000 7f30d5234e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f03d7b0d0 con 0x559f03d74580 2023-10-05T19:40:27.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.927+0000 7f30d2568700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559f03d74580 0x559f03d7a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.927+0000 7f30d2568700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559f03d74580 0x559f03d7a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58354/0 (socket says 172.21.15.12:58354) 2023-10-05T19:40:27.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.927+0000 7f30d2568700 1 -- 172.21.15.12:0/1375168435 learned_addr learned my addr 172.21.15.12:0/1375168435 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:27.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.927+0000 7f30d2568700 1 -- 172.21.15.12:0/1375168435 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30bc004dd0 con 0x559f03d74580 2023-10-05T19:40:27.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.927+0000 7f30d2568700 1 --2- 172.21.15.12:0/1375168435 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559f03d74580 0x559f03d7a0d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f30bc006760 tx=0x7f30bc041070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.927+0000 7f301dffb700 1 -- 172.21.15.12:0/1375168435 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30bc039040 con 0x559f03d74580 2023-10-05T19:40:27.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.927+0000 7f30d5234e40 1 -- 172.21.15.12:0/1375168435 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559f03d7a6d0 con 0x559f03d74580 2023-10-05T19:40:27.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.927+0000 7f301dffb700 1 -- 172.21.15.12:0/1375168435 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30bc02b040 con 0x559f03d74580 2023-10-05T19:40:27.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.927+0000 7f301dffb700 1 -- 172.21.15.12:0/1375168435 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30bc01a040 con 0x559f03d74580 2023-10-05T19:40:27.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.927+0000 7f30d5234e40 1 -- 172.21.15.12:0/1375168435 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f03d7ab10 con 0x559f03d74580 2023-10-05T19:40:27.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.927+0000 7f301dffb700 1 -- 172.21.15.12:0/1375168435 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f30bc025c60 con 0x559f03d74580 2023-10-05T19:40:27.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.931+0000 7f301dffb700 1 --2- 172.21.15.12:0/1375168435 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3010030c00 0x7f30100330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.931+0000 7f301dffb700 1 -- 172.21.15.12:0/1375168435 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f30bc05db00 con 0x559f03d74580 2023-10-05T19:40:27.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.931+0000 7f30d356a700 1 --2- 172.21.15.12:0/1375168435 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3010030c00 0x7f30100330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.931+0000 7f30d5234e40 1 --2- 172.21.15.12:0/1375168435 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x559f03d84cf0 0x559f03d87100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.931+0000 7f30d5234e40 1 -- 172.21.15.12:0/1375168435 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d877a0 con 0x559f03d84cf0 2023-10-05T19:40:27.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.931+0000 7f30d2d69700 1 --2- 172.21.15.12:0/1375168435 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x559f03d84cf0 0x559f03d87100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.931+0000 7f30d356a700 1 --2- 172.21.15.12:0/1375168435 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3010030c00 0x7f30100330b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x559f03d67280 tx=0x7f30c400ab60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.931+0000 7f30d2d69700 1 --2- 172.21.15.12:0/1375168435 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x559f03d84cf0 0x559f03d87100 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.935+0000 7f30d2d69700 1 -- 172.21.15.12:0/1375168435 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v28'25 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f30cc042040 con 0x559f03d84cf0 2023-10-05T19:40:27.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.935+0000 7f30d5234e40 1 --2- 172.21.15.12:0/1375168435 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x559f03d88470 0x559f03d8a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.935+0000 7f30d5234e40 1 -- 172.21.15.12:0/1375168435 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d8af80 con 0x559f03d88470 2023-10-05T19:40:27.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.935+0000 7f30d2568700 1 --2- 172.21.15.12:0/1375168435 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x559f03d88470 0x559f03d8a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.935+0000 7f30d2568700 1 --2- 172.21.15.12:0/1375168435 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x559f03d88470 0x559f03d8a850 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.939+0000 7f30d2568700 1 -- 172.21.15.12:0/1375168435 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v28'16 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f30bc03c040 con 0x559f03d88470 2023-10-05T19:40:27.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.939+0000 7f30d5234e40 1 --2- 172.21.15.12:0/1375168435 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x559f03d8bc50 0x559f03d8e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.939+0000 7f30d5234e40 1 -- 172.21.15.12:0/1375168435 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d8e760 con 0x559f03d8bc50 2023-10-05T19:40:27.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.939+0000 7f30d356a700 1 --2- 172.21.15.12:0/1375168435 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x559f03d8bc50 0x559f03d8e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.939+0000 7f30d356a700 1 --2- 172.21.15.12:0/1375168435 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x559f03d8bc50 0x559f03d8e030 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.939+0000 7f30d356a700 1 -- 172.21.15.12:0/1375168435 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v28'16 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d8bc50 2023-10-05T19:40:27.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.943+0000 7f30d5234e40 1 -- 172.21.15.12:0/1375168435 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d8edb0 con 0x559f03d84cf0 2023-10-05T19:40:27.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.943+0000 7f30d2d69700 1 -- 172.21.15.12:0/1375168435 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v28'136 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f30cc042040 con 0x559f03d84cf0 2023-10-05T19:40:27.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.943+0000 7f30d5234e40 1 --2- 172.21.15.12:0/1375168435 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x559f03d8fa50 0x559f03d91e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:27.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.943+0000 7f30d5234e40 1 -- 172.21.15.12:0/1375168435 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d92490 con 0x559f03d8fa50 2023-10-05T19:40:27.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.943+0000 7f30d2d69700 1 --2- 172.21.15.12:0/1375168435 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x559f03d8fa50 0x559f03d91e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:27.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.943+0000 7f30d2d69700 1 --2- 172.21.15.12:0/1375168435 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x559f03d8fa50 0x559f03d91e30 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:27.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.947+0000 7f30d2d69700 1 -- 172.21.15.12:0/1375168435 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v28'38 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f30cc042040 con 0x559f03d8fa50 2023-10-05T19:40:27.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.947+0000 7f30d5234e40 1 -- 172.21.15.12:0/1375168435 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d92ae0 con 0x559f03d8bc50 2023-10-05T19:40:27.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.951+0000 7f30d356a700 1 -- 172.21.15.12:0/1375168435 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v28'28 uv8 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d8bc50 2023-10-05T19:40:27.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.951+0000 7f30d5234e40 1 -- 172.21.15.12:0/1375168435 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d93100 con 0x559f03d84cf0 2023-10-05T19:40:27.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.951+0000 7f30d2d69700 1 -- 172.21.15.12:0/1375168435 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v28'18 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f30cc042040 con 0x559f03d84cf0 2023-10-05T19:40:27.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.951+0000 7f30d5234e40 1 -- 172.21.15.12:0/1375168435 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d935f0 con 0x559f03d88470 2023-10-05T19:40:27.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.955+0000 7f30d2568700 1 -- 172.21.15.12:0/1375168435 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v28'25 uv8 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f30bc03c040 con 0x559f03d88470 2023-10-05T19:40:27.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.955+0000 7f30d5234e40 1 -- 172.21.15.12:0/1375168435 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d806d0 con 0x559f03d8fa50 2023-10-05T19:40:27.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.959+0000 7f30d2d69700 1 -- 172.21.15.12:0/1375168435 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v28'16 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f30cc042040 con 0x559f03d8fa50 2023-10-05T19:40:27.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.959+0000 7f30d5234e40 1 -- 172.21.15.12:0/1375168435 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d806d0 con 0x559f03d8bc50 2023-10-05T19:40:27.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.963+0000 7f30d356a700 1 -- 172.21.15.12:0/1375168435 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v28'22 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d8bc50 2023-10-05T19:40:27.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.963+0000 7f30d5234e40 1 -- 172.21.15.12:0/1375168435 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d94110 con 0x559f03d8fa50 2023-10-05T19:40:27.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.963+0000 7f30d2d69700 1 -- 172.21.15.12:0/1375168435 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v28'39 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f30cc042040 con 0x559f03d8fa50 2023-10-05T19:40:27.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.963+0000 7f30d5234e40 1 -- 172.21.15.12:0/1375168435 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d94110 con 0x559f03d88470 2023-10-05T19:40:27.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.967+0000 7f30d2568700 1 -- 172.21.15.12:0/1375168435 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v28'18 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f30bc03c040 con 0x559f03d88470 2023-10-05T19:40:27.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.967+0000 7f30d5234e40 1 -- 172.21.15.12:0/1375168435 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d94730 con 0x559f03d8fa50 2023-10-05T19:40:27.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.971+0000 7f30d2d69700 1 -- 172.21.15.12:0/1375168435 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v28'16 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f30cc042040 con 0x559f03d8fa50 2023-10-05T19:40:27.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.971+0000 7f30d5234e40 1 -- 172.21.15.12:0/1375168435 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d94730 con 0x559f03d88470 2023-10-05T19:40:27.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.971+0000 7f30d2568700 1 -- 172.21.15.12:0/1375168435 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v28'33 uv12 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f30bc03c040 con 0x559f03d88470 2023-10-05T19:40:27.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.971+0000 7f30d5234e40 1 -- 172.21.15.12:0/1375168435 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d94870 con 0x559f03d84cf0 2023-10-05T19:40:27.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.975+0000 7f30d2d69700 1 -- 172.21.15.12:0/1375168435 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v28'137 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f30cc042040 con 0x559f03d84cf0 2023-10-05T19:40:27.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.975+0000 7f30d5234e40 1 -- 172.21.15.12:0/1375168435 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d94870 con 0x559f03d8bc50 2023-10-05T19:40:27.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.979+0000 7f30d356a700 1 -- 172.21.15.12:0/1375168435 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v28'32 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d8bc50 2023-10-05T19:40:27.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.979+0000 7f30d5234e40 1 -- 172.21.15.12:0/1375168435 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d94ae0 con 0x559f03d8fa50 2023-10-05T19:40:27.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.983+0000 7f30d2d69700 1 -- 172.21.15.12:0/1375168435 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v28'40 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f30cc042040 con 0x559f03d8fa50 2023-10-05T19:40:27.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.983+0000 7f30d5234e40 1 -- 172.21.15.12:0/1375168435 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d94ae0 con 0x559f03d8bc50 2023-10-05T19:40:27.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.983+0000 7f30d356a700 1 -- 172.21.15.12:0/1375168435 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v28'16 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d8bc50 2023-10-05T19:40:27.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.983+0000 7f30d5234e40 1 -- 172.21.15.12:0/1375168435 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d94ae0 con 0x559f03d84cf0 2023-10-05T19:40:27.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.987+0000 7f30d2d69700 1 -- 172.21.15.12:0/1375168435 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v28'29 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f30cc042040 con 0x559f03d84cf0 2023-10-05T19:40:27.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.987+0000 7f30d5234e40 1 -- 172.21.15.12:0/1375168435 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d94ae0 con 0x559f03d84cf0 2023-10-05T19:40:27.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.991+0000 7f30d2d69700 1 -- 172.21.15.12:0/1375168435 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v28'138 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f30cc042040 con 0x559f03d84cf0 2023-10-05T19:40:27.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.991+0000 7f30d5234e40 1 -- 172.21.15.12:0/1375168435 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d94ae0 con 0x559f03d8bc50 2023-10-05T19:40:28.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.991+0000 7f30d356a700 1 -- 172.21.15.12:0/1375168435 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v28'33 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d8bc50 2023-10-05T19:40:28.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.991+0000 7f30d5234e40 1 -- 172.21.15.12:0/1375168435 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d94ae0 con 0x559f03d8fa50 2023-10-05T19:40:28.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.995+0000 7f30d2d69700 1 -- 172.21.15.12:0/1375168435 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v28'26 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f30cc042040 con 0x559f03d8fa50 2023-10-05T19:40:28.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.995+0000 7f30d5234e40 1 -- 172.21.15.12:0/1375168435 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d94b70 con 0x559f03d8bc50 2023-10-05T19:40:28.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.999+0000 7f30d356a700 1 -- 172.21.15.12:0/1375168435 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v28'18 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d8bc50 2023-10-05T19:40:28.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.999+0000 7f30d5234e40 1 -- 172.21.15.12:0/1375168435 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d94c00 con 0x559f03d8bc50 2023-10-05T19:40:28.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.999+0000 7f30d356a700 1 -- 172.21.15.12:0/1375168435 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v28'24 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d8bc50 2023-10-05T19:40:28.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:27.999+0000 7f30d5234e40 1 -- 172.21.15.12:0/1375168435 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d94c00 con 0x559f03d8bc50 2023-10-05T19:40:28.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.003+0000 7f30d356a700 1 -- 172.21.15.12:0/1375168435 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v28'13 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d8bc50 2023-10-05T19:40:28.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.003+0000 7f30d5234e40 1 -- 172.21.15.12:0/1375168435 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d94c00 con 0x559f03d8bc50 2023-10-05T19:40:28.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.007+0000 7f30d356a700 1 -- 172.21.15.12:0/1375168435 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v28'29 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d8bc50 2023-10-05T19:40:28.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.007+0000 7f30d5234e40 1 -- 172.21.15.12:0/1375168435 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d94c00 con 0x559f03d88470 2023-10-05T19:40:28.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.007+0000 7f30d2568700 1 -- 172.21.15.12:0/1375168435 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v28'21 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f30bc03c040 con 0x559f03d88470 2023-10-05T19:40:28.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.007+0000 7f30d5234e40 1 -- 172.21.15.12:0/1375168435 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d94c00 con 0x559f03d8bc50 2023-10-05T19:40:28.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.011+0000 7f30d356a700 1 -- 172.21.15.12:0/1375168435 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v28'16 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d8bc50 2023-10-05T19:40:28.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.011+0000 7f30d5234e40 1 -- 172.21.15.12:0/1375168435 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d94c00 con 0x559f03d88470 2023-10-05T19:40:28.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.015+0000 7f30d2568700 1 -- 172.21.15.12:0/1375168435 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v28'26 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f30bc03c040 con 0x559f03d88470 2023-10-05T19:40:28.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.015+0000 7f30d5234e40 1 -- 172.21.15.12:0/1375168435 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d94c00 con 0x559f03d8bc50 2023-10-05T19:40:28.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.015+0000 7f30d356a700 1 -- 172.21.15.12:0/1375168435 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v28'34 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d8bc50 2023-10-05T19:40:28.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.015+0000 7f30d5234e40 1 -- 172.21.15.12:0/1375168435 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d94c00 con 0x559f03d88470 2023-10-05T19:40:28.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.019+0000 7f30d2568700 1 -- 172.21.15.12:0/1375168435 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v28'34 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f30bc03c040 con 0x559f03d88470 2023-10-05T19:40:28.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.019+0000 7f30d5234e40 1 -- 172.21.15.12:0/1375168435 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d94c00 con 0x559f03d8fa50 2023-10-05T19:40:28.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.023+0000 7f30d2d69700 1 -- 172.21.15.12:0/1375168435 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v28'22 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f30cc042040 con 0x559f03d8fa50 2023-10-05T19:40:28.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.023+0000 7f3016ffd700 20 reqs_thread_entry: start 2023-10-05T19:40:28.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.023+0000 7f3014ff9700 20 reqs_thread_entry: start 2023-10-05T19:40:28.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.023+0000 7f30d5234e40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:28.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.023+0000 7f30d5234e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:28.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.023+0000 7f3002ffd700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:58 2023 2023-10-05T19:40:28.034 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:28.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.027+0000 7f30027fc700 20 reqs_thread_entry: start 2023-10-05T19:40:28.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.027+0000 7f30d5234e40 20 get_system_obj_state: rctx=0x7ffe32de7db8 obj=default.rgw.meta:users.uid:testx$tenanteduser state=0x559f03bfa0b0 s->prefetch_data=0 2023-10-05T19:40:28.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.027+0000 7f30d5234e40 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser : miss 2023-10-05T19:40:28.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.027+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:102 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03df3760 con 0x7f30cc007030 2023-10-05T19:40:28.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.027+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(102 testx$tenanteduser [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 246+0+139 (crc 0 0 0) 0x7f30c4014040 con 0x7f30cc007030 2023-10-05T19:40:28.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.027+0000 7f30d5234e40 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x16 2023-10-05T19:40:28.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.027+0000 7f30d5234e40 10 adding default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2023-10-05T19:40:28.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.027+0000 7f30d5234e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:28.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.027+0000 7f30d5234e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:28.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.027+0000 7f30d5234e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:28.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.027+0000 7f30d5234e40 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser : type miss (requested=0x13, cached=0x16) 2023-10-05T19:40:28.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.027+0000 7f30d5234e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.027+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:103 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03df61f0 con 0x7f30cc007030 2023-10-05T19:40:28.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.027+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(103 testx$tenanteduser [call,call out=48b,read 0~385 out=385b,getxattrs out=75b] v0'0 uv5 ondisk = 0) v8 ==== 288+0+508 (crc 0 0 0) 0x7f30c4014040 con 0x7f30cc007030 2023-10-05T19:40:28.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.027+0000 7f30d5234e40 20 rados_obj.operate() r=0 bl.length=385 2023-10-05T19:40:28.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.027+0000 7f30d5234e40 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x13 2023-10-05T19:40:28.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.027+0000 7f30d5234e40 10 moving default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2023-10-05T19:40:28.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.027+0000 7f30d5234e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:28.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.027+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:104 8.c 8:37c7a57e:users.uid::testx$tenanteduser.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03df61f0 con 0x559f03d21750 2023-10-05T19:40:28.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.027+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(104 testx$tenanteduser.buckets [call out=15b] v0'0 uv2 ondisk = 0) v8 ==== 170+0+15 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d21750 2023-10-05T19:40:28.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.027+0000 7f30d5234e40 10 removing key index: p5YnriCv1nAtykxBrupQ 2023-10-05T19:40:28.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.027+0000 7f30d5234e40 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2023-10-05T19:40:28.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.027+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:105 7.15 7:a93a5511:::notify.2:head [notify cookie 94141453126896 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03df6640 con 0x559f03d5b310 2023-10-05T19:40:28.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.027+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== watch-notify(notify (1) cookie 94141452529456 notify 120259084295 ret 0) v3 ==== 212+0+0 (crc 0 0 0) 0x559f03d7ab10 con 0x559f03d5b310 2023-10-05T19:40:28.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.027+0000 7f30d0d65700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084295 cookie 94141452529456 notifier 4507 bl.length()=170 2023-10-05T19:40:28.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.031+0000 7f30d0d65700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:106 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f30b8003c50 con 0x559f03d5b310 2023-10-05T19:40:28.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.031+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 <== osd.3 v2:172.21.15.12:6816/29951 27 ==== osd_op_reply(105 notify.2 [notify cookie 94141453126896 out=8b] v0'0 uv141 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f30bc03c040 con 0x559f03d5b310 2023-10-05T19:40:28.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.031+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 <== osd.3 v2:172.21.15.12:6816/29951 28 ==== watch-notify(notify_complete (2) cookie 94141453126896 notify 120259084295 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x559f03d7a6d0 con 0x559f03d5b310 2023-10-05T19:40:28.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.031+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 <== osd.3 v2:172.21.15.12:6816/29951 29 ==== osd_op_reply(106 notify.2 [notify-ack] v0'0 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f30bc03c040 con 0x559f03d5b310 2023-10-05T19:40:28.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.031+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:107 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03df41d0 con 0x7f30cc007030 2023-10-05T19:40:28.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.035+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(107 p5YnriCv1nAtykxBrupQ [delete] v28'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f30c4014040 con 0x7f30cc007030 2023-10-05T19:40:28.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.035+0000 7f30d5234e40 10 removing email index: 2023-10-05T19:40:28.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.035+0000 7f30d5234e40 10 removing user buckets index 2023-10-05T19:40:28.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.035+0000 7f30d5234e40 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:users.uid:testx$tenanteduser.buckets, ofs0, ns] 2023-10-05T19:40:28.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.035+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:108 7.d 7:b4812045:::notify.4:head [notify cookie 94141453126896 in=187b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03df6640 con 0x559f03d21750 2023-10-05T19:40:28.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.035+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== watch-notify(notify (1) cookie 94141452531744 notify 120259084301 ret 0) v3 ==== 217+0+0 (crc 0 0 0) 0x7f30c4026440 con 0x559f03d21750 2023-10-05T19:40:28.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.035+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(108 notify.4 [notify cookie 94141453126896 out=8b] v0'0 uv141 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d21750 2023-10-05T19:40:28.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.035+0000 7f30cbfff700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084301 cookie 94141452531744 notifier 4507 bl.length()=175 2023-10-05T19:40:28.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.035+0000 7f30cbfff700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:109 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f30240032c0 con 0x559f03d21750 2023-10-05T19:40:28.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.035+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== watch-notify(notify_complete (2) cookie 94141453126896 notify 120259084301 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f30c40266a0 con 0x559f03d21750 2023-10-05T19:40:28.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.035+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(109 notify.4 [notify-ack] v0'0 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d21750 2023-10-05T19:40:28.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.035+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:110 8.c 8:37c7a57e:users.uid::testx$tenanteduser.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03df41d0 con 0x559f03d21750 2023-10-05T19:40:28.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.039+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== osd_op_reply(110 testx$tenanteduser.buckets [delete] v28'3 uv3 ondisk = 0) v8 ==== 170+0+0 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d21750 2023-10-05T19:40:28.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.039+0000 7f30d5234e40 10 removing user index: testx$tenanteduser 2023-10-05T19:40:28.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.039+0000 7f30d5234e40 10 removing default.rgw.meta+users.uid+testx$tenanteduser from cache 2023-10-05T19:40:28.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.039+0000 7f30d5234e40 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:users.uid:testx$tenanteduser, ofs0, ns] 2023-10-05T19:40:28.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.039+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:111 7.e 7:7759931f:::notify.3:head [notify cookie 94141453122784 in=179b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03df5630 con 0x559f03d56b70 2023-10-05T19:40:28.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.039+0000 7f30d2d69700 1 -- 172.21.15.12:0/3508051254 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== watch-notify(notify (1) cookie 94141452534320 notify 120259084306 ret 0) v3 ==== 209+0+0 (crc 0 0 0) 0x559f03d494f0 con 0x559f03d56b70 2023-10-05T19:40:28.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.039+0000 7f30cbfff700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084306 cookie 94141452534320 notifier 4507 bl.length()=167 2023-10-05T19:40:28.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.039+0000 7f30cbfff700 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:112 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f3024003330 con 0x559f03d56b70 2023-10-05T19:40:28.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.039+0000 7f30d2d69700 1 -- 172.21.15.12:0/3508051254 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(111 notify.3 [notify cookie 94141453122784 out=8b] v0'0 uv141 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f30cc042040 con 0x559f03d56b70 2023-10-05T19:40:28.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.039+0000 7f30d2d69700 1 -- 172.21.15.12:0/3508051254 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== watch-notify(notify_complete (2) cookie 94141453122784 notify 120259084306 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x559f03d49050 con 0x559f03d56b70 2023-10-05T19:40:28.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.039+0000 7f30d2d69700 1 -- 172.21.15.12:0/3508051254 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(112 notify.3 [notify-ack] v0'0 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f30cc042040 con 0x559f03d56b70 2023-10-05T19:40:28.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.039+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:113 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03df51e0 con 0x7f30cc007030 2023-10-05T19:40:28.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.043+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.0 v2:172.21.15.8:6800/27563 30 ==== osd_op_reply(113 testx$tenanteduser [call,delete] v28'8 uv8 ondisk = 0) v8 ==== 204+0+0 (crc 0 0 0) 0x7f30c4014040 con 0x7f30cc007030 2023-10-05T19:40:28.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.043+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:114 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94141452522528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d36590 con 0x559f03d56b70 2023-10-05T19:40:28.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.047+0000 7f30d2d69700 1 -- 172.21.15.12:0/3508051254 <== osd.2 v2:172.21.15.12:6800/29934 23 ==== osd_op_reply(114 notify.0 [watch unwatch cookie 94141452522528] v28'143 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f30cc042040 con 0x559f03d56b70 2023-10-05T19:40:28.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.047+0000 7f30d5234e40 20 remove_watcher() i=0 2023-10-05T19:40:28.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.047+0000 7f30d5234e40 2 removed watcher, disabling cache 2023-10-05T19:40:28.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.047+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:115 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94141452525760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d61420 con 0x559f03d21750 2023-10-05T19:40:28.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.047+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.1 v2:172.21.15.12:6808/29933 39 ==== osd_op_reply(115 notify.1 [watch unwatch cookie 94141452525760] v28'143 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d21750 2023-10-05T19:40:28.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.047+0000 7f30d5234e40 20 remove_watcher() i=1 2023-10-05T19:40:28.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.047+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:116 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94141452529456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d61420 con 0x559f03d5b310 2023-10-05T19:40:28.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.051+0000 7f30d2568700 1 -- 172.21.15.12:0/3508051254 <== osd.3 v2:172.21.15.12:6816/29951 30 ==== osd_op_reply(116 notify.2 [watch unwatch cookie 94141452529456] v28'143 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f30bc03c040 con 0x559f03d5b310 2023-10-05T19:40:28.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.051+0000 7f30d5234e40 20 remove_watcher() i=2 2023-10-05T19:40:28.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.051+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:117 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94141452534320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d62f30 con 0x559f03d56b70 2023-10-05T19:40:28.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.055+0000 7f30d2d69700 1 -- 172.21.15.12:0/3508051254 <== osd.2 v2:172.21.15.12:6800/29934 24 ==== osd_op_reply(117 notify.3 [watch unwatch cookie 94141452534320] v28'143 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f30cc042040 con 0x559f03d56b70 2023-10-05T19:40:28.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.055+0000 7f30d5234e40 20 remove_watcher() i=3 2023-10-05T19:40:28.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.055+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:118 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94141452531744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d64230 con 0x559f03d21750 2023-10-05T19:40:28.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.055+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.1 v2:172.21.15.12:6808/29933 40 ==== osd_op_reply(118 notify.4 [watch unwatch cookie 94141452531744] v28'143 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d21750 2023-10-05T19:40:28.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.055+0000 7f30d5234e40 20 remove_watcher() i=4 2023-10-05T19:40:28.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.055+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:119 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94141452539552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d64230 con 0x7f30cc007030 2023-10-05T19:40:28.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.059+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.0 v2:172.21.15.8:6800/27563 31 ==== osd_op_reply(119 notify.5 [watch unwatch cookie 94141452539552] v28'143 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f30c4014040 con 0x7f30cc007030 2023-10-05T19:40:28.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.059+0000 7f30d5234e40 20 remove_watcher() i=5 2023-10-05T19:40:28.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.059+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:120 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94141452537344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d656a0 con 0x7f30cc007030 2023-10-05T19:40:28.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.063+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.0 v2:172.21.15.8:6800/27563 32 ==== osd_op_reply(120 notify.6 [watch unwatch cookie 94141452537344] v28'143 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f30c4014040 con 0x7f30cc007030 2023-10-05T19:40:28.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.063+0000 7f30d5234e40 20 remove_watcher() i=6 2023-10-05T19:40:28.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.063+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:121 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94141452540656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d64e00 con 0x7f30cc007030 2023-10-05T19:40:28.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.067+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.0 v2:172.21.15.8:6800/27563 33 ==== osd_op_reply(121 notify.7 [watch unwatch cookie 94141452540656] v28'143 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f30c4014040 con 0x7f30cc007030 2023-10-05T19:40:28.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.067+0000 7f30d5234e40 20 remove_watcher() i=7 2023-10-05T19:40:28.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.067+0000 7f301ffff700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:28.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.067+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:122 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94141452544112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559f03d64e00 con 0x559f03d21750 2023-10-05T19:40:28.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.071+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 <== osd.1 v2:172.21.15.12:6808/29933 41 ==== osd_op_reply(122 data_loggenerations_metadata [watch unwatch cookie 94141452544112] v28'139 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f30c4014040 con 0x559f03d21750 2023-10-05T19:40:28.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.071+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f30cc007030 msgr2=0x7f30cc031480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.071+0000 7f30d5234e40 1 --2- 172.21.15.12:0/3508051254 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f30cc007030 0x7f30cc031480 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.071+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x559f03d21750 msgr2=0x559f03d546d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.071+0000 7f30d5234e40 1 --2- 172.21.15.12:0/3508051254 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x559f03d21750 0x559f03d546d0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.071+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x559f03d56b70 msgr2=0x559f03d58f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.071+0000 7f30d5234e40 1 --2- 172.21.15.12:0/3508051254 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x559f03d56b70 0x559f03d58f50 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.071+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x559f03d5b310 msgr2=0x559f03d5d720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.071+0000 7f30d5234e40 1 --2- 172.21.15.12:0/3508051254 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x559f03d5b310 0x559f03d5d720 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.071+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f30b4030c90 msgr2=0x7f30b4033140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.071+0000 7f30d5234e40 1 --2- 172.21.15.12:0/3508051254 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f30b4030c90 0x7f30b4033140 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f30bc001560 tx=0x7f30bc006650 comp rx=0 tx=0).stop 2023-10-05T19:40:28.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.071+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559f03d20e40 msgr2=0x559f03d48b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.071+0000 7f30d356a700 1 -- 172.21.15.12:0/3508051254 reap_dead start 2023-10-05T19:40:28.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.071+0000 7f30d5234e40 1 --2- 172.21.15.12:0/3508051254 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559f03d20e40 0x559f03d48b10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f30cc01e040 tx=0x7f30cc01c000 comp rx=0 tx=0).stop 2023-10-05T19:40:28.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.071+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 shutdown_connections 2023-10-05T19:40:28.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.071+0000 7f30d5234e40 1 --2- 172.21.15.12:0/3508051254 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559f03d20e40 0x559f03d48b10 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.071+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 >> 172.21.15.12:0/3508051254 conn(0x559f03d1c860 msgr2=0x559f03d3a330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:28.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.071+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 shutdown_connections 2023-10-05T19:40:28.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.071+0000 7f30d5234e40 1 -- 172.21.15.12:0/3508051254 wait complete. 2023-10-05T19:40:28.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.071+0000 7f30d5234e40 1 -- 172.21.15.12:0/1375168435 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x559f03d88470 msgr2=0x559f03d8a850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.071+0000 7f30d5234e40 1 --2- 172.21.15.12:0/1375168435 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x559f03d88470 0x559f03d8a850 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.071+0000 7f30d5234e40 1 -- 172.21.15.12:0/1375168435 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x559f03d84cf0 msgr2=0x559f03d87100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.071+0000 7f30d5234e40 1 --2- 172.21.15.12:0/1375168435 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x559f03d84cf0 0x559f03d87100 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.071+0000 7f30d5234e40 1 -- 172.21.15.12:0/1375168435 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x559f03d8fa50 msgr2=0x559f03d91e30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.071+0000 7f30d5234e40 1 --2- 172.21.15.12:0/1375168435 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x559f03d8fa50 0x559f03d91e30 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.071+0000 7f30d5234e40 1 -- 172.21.15.12:0/1375168435 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x559f03d8bc50 msgr2=0x559f03d8e030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.071+0000 7f30d5234e40 1 --2- 172.21.15.12:0/1375168435 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x559f03d8bc50 0x559f03d8e030 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.071+0000 7f30d5234e40 1 -- 172.21.15.12:0/1375168435 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3010030c00 msgr2=0x7f30100330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.071+0000 7f30d5234e40 1 --2- 172.21.15.12:0/1375168435 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3010030c00 0x7f30100330b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x559f03d67280 tx=0x7f30c400ab60 comp rx=0 tx=0).stop 2023-10-05T19:40:28.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.071+0000 7f30d5234e40 1 -- 172.21.15.12:0/1375168435 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559f03d74580 msgr2=0x559f03d7a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.071+0000 7f30d5234e40 1 --2- 172.21.15.12:0/1375168435 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559f03d74580 0x559f03d7a0d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f30bc006760 tx=0x7f30bc041070 comp rx=0 tx=0).stop 2023-10-05T19:40:28.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.071+0000 7f30d2d69700 1 -- 172.21.15.12:0/1375168435 reap_dead start 2023-10-05T19:40:28.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.071+0000 7f30d5234e40 1 -- 172.21.15.12:0/1375168435 shutdown_connections 2023-10-05T19:40:28.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.071+0000 7f30d5234e40 1 --2- 172.21.15.12:0/1375168435 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559f03d74580 0x559f03d7a0d0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.071+0000 7f30d5234e40 1 -- 172.21.15.12:0/1375168435 >> 172.21.15.12:0/1375168435 conn(0x559f03d6f940 msgr2=0x559f03d6d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:28.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.075+0000 7f30d5234e40 1 -- 172.21.15.12:0/1375168435 shutdown_connections 2023-10-05T19:40:28.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.075+0000 7f30d5234e40 1 -- 172.21.15.12:0/1375168435 wait complete. 2023-10-05T19:40:28.099 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=4 bytes_in=0 failed=False 2023-10-05T19:40:28.104 INFO:tasks.radosgw_admin:RL: get_obj myfoo foo bytes_out=0 bytes_in=4 failed=False 2023-10-05T19:40:28.151 INFO:tasks.radosgw_admin:RL: list_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2023-10-05T19:40:28.151 INFO:tasks.util.rgw:rgwadmin: client.0 : ['object', 'rm', '--bucket', 'myfoo', '--object', 'four'] 2023-10-05T19:40:28.151 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'object', 'rm', '--bucket', 'myfoo', '--object', 'four'] 2023-10-05T19:40:28.151 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph object rm --bucket myfoo --object four 2023-10-05T19:40:28.194 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:28.194 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:28.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.187+0000 7f3f1010ce40 1 Processor -- start 2023-10-05T19:40:28.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.187+0000 7f3f1010ce40 1 -- start start 2023-10-05T19:40:28.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.187+0000 7f3f1010ce40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x5603676ee920 0x560367782f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:28.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.187+0000 7f3f1010ce40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5603676c6bd0 con 0x5603676efc30 2023-10-05T19:40:28.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.187+0000 7f3f1010ce40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x560367683ee0 con 0x5603676ee920 2023-10-05T19:40:28.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.187+0000 7f3f0d440700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x5603676ee920 0x560367782f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:28.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.187+0000 7f3f0dc41700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x5603676efc30 0x5603676ee210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:35648/0 (socket says 172.21.15.12:35648) 2023-10-05T19:40:28.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.187+0000 7f3f0dc41700 1 -- 172.21.15.12:0/3554792075 learned_addr learned my addr 172.21.15.12:0/3554792075 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:28.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.187+0000 7f3f0cc3f700 1 -- 172.21.15.12:0/3554792075 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 122828365 0 0) 0x5603676c6bd0 con 0x5603676efc30 2023-10-05T19:40:28.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.187+0000 7f3f0d440700 1 -- 172.21.15.12:0/3554792075 >> v1:172.21.15.8:6789/0 conn(0x5603676efc30 legacy=0x5603676ee210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.187+0000 7f3f0d440700 1 -- 172.21.15.12:0/3554792075 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603676f00c0 con 0x5603676ee920 2023-10-05T19:40:28.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.187+0000 7f3f0d440700 1 --2- 172.21.15.12:0/3554792075 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5603676ee920 0x560367782f70 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3ef800a5d0 tx=0x7f3ef8005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acfc1e8f0486c0f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:28.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.187+0000 7f3f0cc3f700 1 -- 172.21.15.12:0/3554792075 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3ef80086a0 con 0x5603676ee920 2023-10-05T19:40:28.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.187+0000 7f3f0cc3f700 1 -- 172.21.15.12:0/3554792075 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef8008800 con 0x5603676ee920 2023-10-05T19:40:28.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.187+0000 7f3f0cc3f700 1 -- 172.21.15.12:0/3554792075 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3ef8008ad0 con 0x5603676ee920 2023-10-05T19:40:28.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.191+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3554792075 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5603676ee920 msgr2=0x560367782f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.191+0000 7f3f1010ce40 1 --2- 172.21.15.12:0/3554792075 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5603676ee920 0x560367782f70 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3ef800a5d0 tx=0x7f3ef8005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:28.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.191+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3554792075 shutdown_connections 2023-10-05T19:40:28.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.191+0000 7f3f1010ce40 1 --2- 172.21.15.12:0/3554792075 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5603676ee920 0x560367782f70 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.191+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3554792075 >> 172.21.15.12:0/3554792075 conn(0x560367775860 msgr2=0x560367777c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:28.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.191+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3554792075 shutdown_connections 2023-10-05T19:40:28.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.191+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3554792075 wait complete. 2023-10-05T19:40:28.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.191+0000 7f3f1010ce40 1 Processor -- start 2023-10-05T19:40:28.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.191+0000 7f3f1010ce40 1 -- start start 2023-10-05T19:40:28.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.191+0000 7f3f1010ce40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5603676ee920 0x56036779af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:28.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.191+0000 7f3f1010ce40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560367647e30 con 0x5603676ee920 2023-10-05T19:40:28.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.191+0000 7f3f0dc41700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5603676ee920 0x56036779af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:28.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.191+0000 7f3f0dc41700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5603676ee920 0x56036779af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58370/0 (socket says 172.21.15.12:58370) 2023-10-05T19:40:28.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.191+0000 7f3f0dc41700 1 -- 172.21.15.12:0/2409225717 learned_addr learned my addr 172.21.15.12:0/2409225717 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:28.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.191+0000 7f3f0dc41700 1 -- 172.21.15.12:0/2409225717 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef8005cf0 con 0x5603676ee920 2023-10-05T19:40:28.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.191+0000 7f3f0dc41700 1 --2- 172.21.15.12:0/2409225717 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5603676ee920 0x56036779af80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3f08001630 tx=0x7f3f080069b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73b4e82b851b1e01 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:28.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.191+0000 7f3f077fe700 1 -- 172.21.15.12:0/2409225717 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f08006320 con 0x5603676ee920 2023-10-05T19:40:28.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.191+0000 7f3f077fe700 1 -- 172.21.15.12:0/2409225717 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f080044c0 con 0x5603676ee920 2023-10-05T19:40:28.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.191+0000 7f3f077fe700 1 -- 172.21.15.12:0/2409225717 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f08004650 con 0x5603676ee920 2023-10-05T19:40:28.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.195+0000 7f3f1010ce40 1 -- 172.21.15.12:0/2409225717 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5603676ee920 msgr2=0x56036779af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.195+0000 7f3f1010ce40 1 --2- 172.21.15.12:0/2409225717 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5603676ee920 0x56036779af80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3f08001630 tx=0x7f3f080069b0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.195+0000 7f3f1010ce40 1 -- 172.21.15.12:0/2409225717 shutdown_connections 2023-10-05T19:40:28.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.195+0000 7f3f1010ce40 1 --2- 172.21.15.12:0/2409225717 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5603676ee920 0x56036779af80 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.195+0000 7f3f1010ce40 1 -- 172.21.15.12:0/2409225717 >> 172.21.15.12:0/2409225717 conn(0x560367775860 msgr2=0x560367796b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:28.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.195+0000 7f3f1010ce40 1 -- 172.21.15.12:0/2409225717 shutdown_connections 2023-10-05T19:40:28.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.195+0000 7f3f1010ce40 1 -- 172.21.15.12:0/2409225717 wait complete. 2023-10-05T19:40:28.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.195+0000 7f3f1010ce40 1 Processor -- start 2023-10-05T19:40:28.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.195+0000 7f3f1010ce40 1 -- start start 2023-10-05T19:40:28.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.195+0000 7f3f1010ce40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5603676efc30 0x5603677a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:28.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.195+0000 7f3f1010ce40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56036761aed0 con 0x5603676efc30 2023-10-05T19:40:28.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.195+0000 7f3f0dc41700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5603676efc30 0x5603677a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:28.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.195+0000 7f3f0dc41700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5603676efc30 0x5603677a1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58374/0 (socket says 172.21.15.12:58374) 2023-10-05T19:40:28.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.195+0000 7f3f0dc41700 1 -- 172.21.15.12:0/3341206312 learned_addr learned my addr 172.21.15.12:0/3341206312 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:28.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.195+0000 7f3f0dc41700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f08006c90 con 0x5603676efc30 2023-10-05T19:40:28.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.195+0000 7f3f0dc41700 1 --2- 172.21.15.12:0/3341206312 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5603676efc30 0x5603677a1ec0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3f08001560 tx=0x7f3f08014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:28.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.195+0000 7f3f05ffb700 1 -- 172.21.15.12:0/3341206312 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f08013690 con 0x5603676efc30 2023-10-05T19:40:28.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.195+0000 7f3f05ffb700 1 -- 172.21.15.12:0/3341206312 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f080137f0 con 0x5603676efc30 2023-10-05T19:40:28.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.195+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5603677a2400 con 0x5603676efc30 2023-10-05T19:40:28.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.195+0000 7f3f05ffb700 1 -- 172.21.15.12:0/3341206312 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f08013ac0 con 0x5603676efc30 2023-10-05T19:40:28.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.195+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5603677a2840 con 0x5603676efc30 2023-10-05T19:40:28.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.199+0000 7f3f1010ce40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.199+0000 7f3f05ffb700 1 -- 172.21.15.12:0/3341206312 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f3f0801a070 con 0x5603676efc30 2023-10-05T19:40:28.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.199+0000 7f3f05ffb700 1 --2- 172.21.15.12:0/3341206312 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3ef0030c50 0x7f3ef0033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:28.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.199+0000 7f3f05ffb700 1 -- 172.21.15.12:0/3341206312 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f3f08044c60 con 0x5603676efc30 2023-10-05T19:40:28.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.199+0000 7f3f0d440700 1 --2- 172.21.15.12:0/3341206312 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3ef0030c50 0x7f3ef0033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:28.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.199+0000 7f3f1010ce40 1 --2- 172.21.15.12:0/3341206312 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5603677aba30 0x5603677ade10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:28.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.199+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677ae480 con 0x5603677aba30 2023-10-05T19:40:28.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.199+0000 7f3f0e442700 1 --2- 172.21.15.12:0/3341206312 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5603677aba30 0x5603677ade10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:28.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.199+0000 7f3f0d440700 1 --2- 172.21.15.12:0/3341206312 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3ef0030c50 0x7f3ef0033100 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8005a90 tx=0x7f3ef8015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:28.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.199+0000 7f3f0e442700 1 --2- 172.21.15.12:0/3341206312 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5603677aba30 0x5603677ade10 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:28.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.199+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3f0000a500 con 0x5603677aba30 2023-10-05T19:40:28.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.203+0000 7f3f1010ce40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:28.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.203+0000 7f3f1010ce40 20 realm 2023-10-05T19:40:28.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.203+0000 7f3f1010ce40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.203+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677ab0a0 con 0x5603677aba30 2023-10-05T19:40:28.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.203+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3f0000a500 con 0x5603677aba30 2023-10-05T19:40:28.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.203+0000 7f3f1010ce40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:28.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.203+0000 7f3f1010ce40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:28.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.203+0000 7f3f1010ce40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.203+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677aef60 con 0x5603677aba30 2023-10-05T19:40:28.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.203+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3f0000a500 con 0x5603677aba30 2023-10-05T19:40:28.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.203+0000 7f3f1010ce40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:28.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.203+0000 7f3f1010ce40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.203+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677aeb70 con 0x5603677aba30 2023-10-05T19:40:28.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.203+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f3f0000a500 con 0x5603677aba30 2023-10-05T19:40:28.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.203+0000 7f3f1010ce40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:28.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.203+0000 7f3f1010ce40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.203+0000 7f3f1010ce40 1 --2- 172.21.15.12:0/3341206312 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5603677b0300 0x5603677b26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:28.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.203+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b2c20 con 0x5603677b0300 2023-10-05T19:40:28.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.203+0000 7f3f0dc41700 1 --2- 172.21.15.12:0/3341206312 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5603677b0300 0x5603677b26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:28.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.203+0000 7f3f0dc41700 1 --2- 172.21.15.12:0/3341206312 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5603677b0300 0x5603677b26e0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:28.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.203+0000 7f3f0dc41700 1 -- 172.21.15.12:0/3341206312 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f3f08025020 con 0x5603677b0300 2023-10-05T19:40:28.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.203+0000 7f3f1010ce40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:28.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.203+0000 7f3f1010ce40 20 searching for the correct realm 2023-10-05T19:40:28.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.203+0000 7f3f1010ce40 1 --2- 172.21.15.12:0/3341206312 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5603677b4aa0 0x5603677b6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:28.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.203+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b7560 con 0x5603677b4aa0 2023-10-05T19:40:28.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.203+0000 7f3f0d440700 1 --2- 172.21.15.12:0/3341206312 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5603677b4aa0 0x5603677b6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:28.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.207+0000 7f3f0d440700 1 --2- 172.21.15.12:0/3341206312 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5603677b4aa0 0x5603677b6eb0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:28.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.207+0000 7f3f0d440700 1 -- 172.21.15.12:0/3341206312 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f3ef8008ae0 con 0x5603677b4aa0 2023-10-05T19:40:28.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.207+0000 7f3f0d440700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b7560 con 0x5603677b0300 2023-10-05T19:40:28.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.207+0000 7f3f0dc41700 1 -- 172.21.15.12:0/3341206312 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3f08025020 con 0x5603677b0300 2023-10-05T19:40:28.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.207+0000 7f3f0dc41700 1 --2- 172.21.15.12:0/3341206312 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x5603676ee920 0x7f3f080113e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:28.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.207+0000 7f3f0dc41700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b7560 con 0x5603676ee920 2023-10-05T19:40:28.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.207+0000 7f3f0e442700 1 --2- 172.21.15.12:0/3341206312 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x5603676ee920 0x7f3f080113e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:28.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.207+0000 7f3f0e442700 1 --2- 172.21.15.12:0/3341206312 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x5603676ee920 0x7f3f080113e0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:28.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.211+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3f00014040 con 0x5603676ee920 2023-10-05T19:40:28.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.211+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b7560 con 0x5603677aba30 2023-10-05T19:40:28.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.211+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3f00014040 con 0x5603677aba30 2023-10-05T19:40:28.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.211+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b7560 con 0x5603677aba30 2023-10-05T19:40:28.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.211+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3f00014040 con 0x5603677aba30 2023-10-05T19:40:28.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.211+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b7560 con 0x5603676ee920 2023-10-05T19:40:28.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.211+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3f00014040 con 0x5603676ee920 2023-10-05T19:40:28.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.211+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b7560 con 0x5603676ee920 2023-10-05T19:40:28.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.211+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3f00014040 con 0x5603676ee920 2023-10-05T19:40:28.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.211+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b7560 con 0x5603677b0300 2023-10-05T19:40:28.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.211+0000 7f3f0dc41700 1 -- 172.21.15.12:0/3341206312 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3f08025020 con 0x5603677b0300 2023-10-05T19:40:28.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.211+0000 7f3f0dc41700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b7560 con 0x5603677b4aa0 2023-10-05T19:40:28.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.215+0000 7f3f0d440700 1 -- 172.21.15.12:0/3341206312 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ef8008ae0 con 0x5603677b4aa0 2023-10-05T19:40:28.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.215+0000 7f3f0d440700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b7560 con 0x5603677aba30 2023-10-05T19:40:28.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.215+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3f00014040 con 0x5603677aba30 2023-10-05T19:40:28.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.215+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b7560 con 0x5603676ee920 2023-10-05T19:40:28.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.215+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3f00014040 con 0x5603676ee920 2023-10-05T19:40:28.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.215+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b7560 con 0x5603677aba30 2023-10-05T19:40:28.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.215+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3f00014040 con 0x5603677aba30 2023-10-05T19:40:28.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.215+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b7560 con 0x5603677b0300 2023-10-05T19:40:28.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.215+0000 7f3f0dc41700 1 -- 172.21.15.12:0/3341206312 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3f08025020 con 0x5603677b0300 2023-10-05T19:40:28.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.215+0000 7f3f0dc41700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b7560 con 0x5603677b4aa0 2023-10-05T19:40:28.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.219+0000 7f3f0d440700 1 -- 172.21.15.12:0/3341206312 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ef8008ae0 con 0x5603677b4aa0 2023-10-05T19:40:28.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.219+0000 7f3f0d440700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b7560 con 0x5603677b4aa0 2023-10-05T19:40:28.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.219+0000 7f3f0d440700 1 -- 172.21.15.12:0/3341206312 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ef8008ae0 con 0x5603677b4aa0 2023-10-05T19:40:28.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.219+0000 7f3f0d440700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b7560 con 0x5603676ee920 2023-10-05T19:40:28.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.219+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3f00014040 con 0x5603676ee920 2023-10-05T19:40:28.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.219+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b7560 con 0x5603677b4aa0 2023-10-05T19:40:28.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.219+0000 7f3f0d440700 1 -- 172.21.15.12:0/3341206312 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ef8008ae0 con 0x5603677b4aa0 2023-10-05T19:40:28.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.219+0000 7f3f0d440700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b7560 con 0x5603677b4aa0 2023-10-05T19:40:28.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.219+0000 7f3f0d440700 1 -- 172.21.15.12:0/3341206312 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ef8008ae0 con 0x5603677b4aa0 2023-10-05T19:40:28.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.219+0000 7f3f0d440700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b7560 con 0x5603677aba30 2023-10-05T19:40:28.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.219+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3f00014040 con 0x5603677aba30 2023-10-05T19:40:28.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.219+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b7560 con 0x5603677aba30 2023-10-05T19:40:28.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.223+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3f00014040 con 0x5603677aba30 2023-10-05T19:40:28.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.223+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b7560 con 0x5603676ee920 2023-10-05T19:40:28.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.223+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3f00014040 con 0x5603676ee920 2023-10-05T19:40:28.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.223+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b7560 con 0x5603677b4aa0 2023-10-05T19:40:28.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.223+0000 7f3f0d440700 1 -- 172.21.15.12:0/3341206312 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ef8008ae0 con 0x5603677b4aa0 2023-10-05T19:40:28.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.223+0000 7f3f0d440700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b7560 con 0x5603677b0300 2023-10-05T19:40:28.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.223+0000 7f3f0dc41700 1 -- 172.21.15.12:0/3341206312 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3f08025020 con 0x5603677b0300 2023-10-05T19:40:28.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.223+0000 7f3f0dc41700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b7560 con 0x5603677aba30 2023-10-05T19:40:28.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.223+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3f00014040 con 0x5603677aba30 2023-10-05T19:40:28.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.223+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b7560 con 0x5603676ee920 2023-10-05T19:40:28.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.227+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3f00014040 con 0x5603676ee920 2023-10-05T19:40:28.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.227+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b7560 con 0x5603677b4aa0 2023-10-05T19:40:28.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.227+0000 7f3f0d440700 1 -- 172.21.15.12:0/3341206312 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ef8008ae0 con 0x5603677b4aa0 2023-10-05T19:40:28.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.227+0000 7f3f0d440700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b7560 con 0x5603676ee920 2023-10-05T19:40:28.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.227+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3f00014040 con 0x5603676ee920 2023-10-05T19:40:28.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.227+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b7560 con 0x5603677b0300 2023-10-05T19:40:28.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.227+0000 7f3f0dc41700 1 -- 172.21.15.12:0/3341206312 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3f08025020 con 0x5603677b0300 2023-10-05T19:40:28.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.227+0000 7f3f0dc41700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b7560 con 0x5603677b4aa0 2023-10-05T19:40:28.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.227+0000 7f3f0d440700 1 -- 172.21.15.12:0/3341206312 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ef8008ae0 con 0x5603677b4aa0 2023-10-05T19:40:28.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.227+0000 7f3f0d440700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b7560 con 0x5603676ee920 2023-10-05T19:40:28.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.231+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3f00014040 con 0x5603676ee920 2023-10-05T19:40:28.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.231+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b7560 con 0x5603677aba30 2023-10-05T19:40:28.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.231+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f3f00014040 con 0x5603677aba30 2023-10-05T19:40:28.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.231+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b7560 con 0x5603677b0300 2023-10-05T19:40:28.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.231+0000 7f3f0dc41700 1 -- 172.21.15.12:0/3341206312 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f3f08025020 con 0x5603677b0300 2023-10-05T19:40:28.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.231+0000 7f3f1010ce40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:28.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.231+0000 7f3f1010ce40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:28.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.231+0000 7f3f1010ce40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:28.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.231+0000 7f3f1010ce40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:28.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.231+0000 7f3f1010ce40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.231+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b3820 con 0x5603677aba30 2023-10-05T19:40:28.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.231+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3f00014040 con 0x5603677aba30 2023-10-05T19:40:28.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.231+0000 7f3f1010ce40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:28.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.231+0000 7f3f1010ce40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.231+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b3450 con 0x5603677aba30 2023-10-05T19:40:28.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.235+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f3f00014040 con 0x5603677aba30 2023-10-05T19:40:28.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.235+0000 7f3f1010ce40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:28.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.235+0000 7f3f1010ce40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.235+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b79d0 con 0x5603677b4aa0 2023-10-05T19:40:28.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.235+0000 7f3f0d440700 1 -- 172.21.15.12:0/3341206312 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f3ef8008ae0 con 0x5603677b4aa0 2023-10-05T19:40:28.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.235+0000 7f3f1010ce40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:28.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.235+0000 7f3f1010ce40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.235+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b7d80 con 0x5603677b4aa0 2023-10-05T19:40:28.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.235+0000 7f3f0d440700 1 -- 172.21.15.12:0/3341206312 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f3ef8008ae0 con 0x5603677b4aa0 2023-10-05T19:40:28.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.235+0000 7f3f1010ce40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:28.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.235+0000 7f3f1010ce40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.235+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b8130 con 0x5603677aba30 2023-10-05T19:40:28.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.235+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3f00014040 con 0x5603677aba30 2023-10-05T19:40:28.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.235+0000 7f3f1010ce40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:28.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.235+0000 7f3f1010ce40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.235+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b8540 con 0x5603677aba30 2023-10-05T19:40:28.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.235+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f3f00014040 con 0x5603677aba30 2023-10-05T19:40:28.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.235+0000 7f3f1010ce40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:28.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.235+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b8510 con 0x5603677b4aa0 2023-10-05T19:40:28.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.238+0000 7f3f0d440700 1 -- 172.21.15.12:0/3341206312 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f3ef8008ae0 con 0x5603677b4aa0 2023-10-05T19:40:28.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.238+0000 7f3f0d440700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b8510 con 0x5603677b0300 2023-10-05T19:40:28.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.238+0000 7f3f0dc41700 1 -- 172.21.15.12:0/3341206312 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3f08025020 con 0x5603677b0300 2023-10-05T19:40:28.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.238+0000 7f3f0dc41700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b8510 con 0x5603676ee920 2023-10-05T19:40:28.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.238+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3f00014040 con 0x5603676ee920 2023-10-05T19:40:28.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.238+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b8130 con 0x5603677aba30 2023-10-05T19:40:28.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.242+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3f00014040 con 0x5603677aba30 2023-10-05T19:40:28.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.242+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b8130 con 0x5603677aba30 2023-10-05T19:40:28.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.242+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3f00014040 con 0x5603677aba30 2023-10-05T19:40:28.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.242+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b8130 con 0x5603676ee920 2023-10-05T19:40:28.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.242+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3f00014040 con 0x5603676ee920 2023-10-05T19:40:28.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.242+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b8130 con 0x5603676ee920 2023-10-05T19:40:28.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.242+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3f00014040 con 0x5603676ee920 2023-10-05T19:40:28.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.242+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b8130 con 0x5603677b0300 2023-10-05T19:40:28.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.242+0000 7f3f0dc41700 1 -- 172.21.15.12:0/3341206312 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3f08025020 con 0x5603677b0300 2023-10-05T19:40:28.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.246+0000 7f3f0dc41700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b8130 con 0x5603677b4aa0 2023-10-05T19:40:28.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.246+0000 7f3f0d440700 1 -- 172.21.15.12:0/3341206312 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ef8008ae0 con 0x5603677b4aa0 2023-10-05T19:40:28.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.246+0000 7f3f0d440700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b8130 con 0x5603677aba30 2023-10-05T19:40:28.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.246+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3f00014040 con 0x5603677aba30 2023-10-05T19:40:28.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.246+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b8130 con 0x5603676ee920 2023-10-05T19:40:28.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.246+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3f00014040 con 0x5603676ee920 2023-10-05T19:40:28.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.246+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b8130 con 0x5603677aba30 2023-10-05T19:40:28.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.246+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3f00014040 con 0x5603677aba30 2023-10-05T19:40:28.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.246+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b8130 con 0x5603677b0300 2023-10-05T19:40:28.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.250+0000 7f3f0dc41700 1 -- 172.21.15.12:0/3341206312 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3f08025020 con 0x5603677b0300 2023-10-05T19:40:28.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.250+0000 7f3f0dc41700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b8130 con 0x5603677b4aa0 2023-10-05T19:40:28.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.250+0000 7f3f0d440700 1 -- 172.21.15.12:0/3341206312 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ef8008ae0 con 0x5603677b4aa0 2023-10-05T19:40:28.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.250+0000 7f3f0d440700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b8130 con 0x5603677b4aa0 2023-10-05T19:40:28.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.250+0000 7f3f0d440700 1 -- 172.21.15.12:0/3341206312 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ef8008ae0 con 0x5603677b4aa0 2023-10-05T19:40:28.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.250+0000 7f3f0d440700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b8130 con 0x5603676ee920 2023-10-05T19:40:28.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.250+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3f00014040 con 0x5603676ee920 2023-10-05T19:40:28.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.250+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b8130 con 0x5603677b4aa0 2023-10-05T19:40:28.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.250+0000 7f3f0d440700 1 -- 172.21.15.12:0/3341206312 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ef8008ae0 con 0x5603677b4aa0 2023-10-05T19:40:28.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.250+0000 7f3f0d440700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b8130 con 0x5603677b4aa0 2023-10-05T19:40:28.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.250+0000 7f3f0d440700 1 -- 172.21.15.12:0/3341206312 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ef8008ae0 con 0x5603677b4aa0 2023-10-05T19:40:28.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.250+0000 7f3f0d440700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b8130 con 0x5603677aba30 2023-10-05T19:40:28.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.254+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3f00014040 con 0x5603677aba30 2023-10-05T19:40:28.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.254+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b8130 con 0x5603677aba30 2023-10-05T19:40:28.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.254+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3f00014040 con 0x5603677aba30 2023-10-05T19:40:28.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.254+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b8130 con 0x5603676ee920 2023-10-05T19:40:28.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.254+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3f00014040 con 0x5603676ee920 2023-10-05T19:40:28.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.254+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b8130 con 0x5603677b4aa0 2023-10-05T19:40:28.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.254+0000 7f3f0d440700 1 -- 172.21.15.12:0/3341206312 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ef8008ae0 con 0x5603677b4aa0 2023-10-05T19:40:28.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.254+0000 7f3f0d440700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b8130 con 0x5603677b0300 2023-10-05T19:40:28.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.258+0000 7f3f0dc41700 1 -- 172.21.15.12:0/3341206312 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3f08025020 con 0x5603677b0300 2023-10-05T19:40:28.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.258+0000 7f3f0dc41700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b8130 con 0x5603677aba30 2023-10-05T19:40:28.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.258+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3f00014040 con 0x5603677aba30 2023-10-05T19:40:28.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.258+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b8130 con 0x5603676ee920 2023-10-05T19:40:28.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.258+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3f00014040 con 0x5603676ee920 2023-10-05T19:40:28.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.258+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b8130 con 0x5603677b4aa0 2023-10-05T19:40:28.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.258+0000 7f3f0d440700 1 -- 172.21.15.12:0/3341206312 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ef8008ae0 con 0x5603677b4aa0 2023-10-05T19:40:28.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.258+0000 7f3f0d440700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b8130 con 0x5603676ee920 2023-10-05T19:40:28.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.258+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3f00014040 con 0x5603676ee920 2023-10-05T19:40:28.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.258+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b8130 con 0x5603677b0300 2023-10-05T19:40:28.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.262+0000 7f3f0dc41700 1 -- 172.21.15.12:0/3341206312 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3f08025020 con 0x5603677b0300 2023-10-05T19:40:28.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.262+0000 7f3f0dc41700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b8130 con 0x5603677b4aa0 2023-10-05T19:40:28.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.262+0000 7f3f0d440700 1 -- 172.21.15.12:0/3341206312 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ef8008ae0 con 0x5603677b4aa0 2023-10-05T19:40:28.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.262+0000 7f3f0d440700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b8130 con 0x5603676ee920 2023-10-05T19:40:28.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.262+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3f00014040 con 0x5603676ee920 2023-10-05T19:40:28.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.262+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b8130 con 0x5603677aba30 2023-10-05T19:40:28.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.262+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f3f00014040 con 0x5603677aba30 2023-10-05T19:40:28.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.262+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b8130 con 0x5603677b0300 2023-10-05T19:40:28.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.266+0000 7f3f0dc41700 1 -- 172.21.15.12:0/3341206312 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f3f08025020 con 0x5603677b0300 2023-10-05T19:40:28.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.266+0000 7f3f1010ce40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:28.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.266+0000 7f3f1010ce40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:28.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.266+0000 7f3f1010ce40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:28.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.266+0000 7f3f1010ce40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:28.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.266+0000 7f3f1010ce40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.266+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b8510 con 0x5603677aba30 2023-10-05T19:40:28.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.266+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f3f00014040 con 0x5603677aba30 2023-10-05T19:40:28.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.266+0000 7f3f1010ce40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:28.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.266+0000 7f3f1010ce40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.266+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b8970 con 0x5603677b0300 2023-10-05T19:40:28.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.266+0000 7f3f0dc41700 1 -- 172.21.15.12:0/3341206312 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f3f08025020 con 0x5603677b0300 2023-10-05T19:40:28.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.266+0000 7f3f1010ce40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:28.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.266+0000 7f3f1010ce40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.266+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b9260 con 0x5603677aba30 2023-10-05T19:40:28.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.266+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f3f00014040 con 0x5603677aba30 2023-10-05T19:40:28.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.266+0000 7f3f1010ce40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:28.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.266+0000 7f3f1010ce40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.266+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b9260 con 0x5603677b4aa0 2023-10-05T19:40:28.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.266+0000 7f3f0d440700 1 -- 172.21.15.12:0/3341206312 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f3ef8008ae0 con 0x5603677b4aa0 2023-10-05T19:40:28.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.266+0000 7f3f1010ce40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:28.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.266+0000 7f3f1010ce40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.266+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b99d0 con 0x5603676ee920 2023-10-05T19:40:28.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.270+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f3f00014040 con 0x5603676ee920 2023-10-05T19:40:28.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.270+0000 7f3f1010ce40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:28.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.270+0000 7f3f1010ce40 20 zone default found 2023-10-05T19:40:28.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.270+0000 7f3f1010ce40 4 Realm: () 2023-10-05T19:40:28.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.270+0000 7f3f1010ce40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:28.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.270+0000 7f3f1010ce40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:28.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.270+0000 7f3f1010ce40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:28.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.270+0000 7f3f1010ce40 20 zonegroup default 2023-10-05T19:40:28.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.270+0000 7f3f1010ce40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.270+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677b96a0 con 0x5603676ee920 2023-10-05T19:40:28.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.270+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f3f00014040 con 0x5603676ee920 2023-10-05T19:40:28.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.270+0000 7f3f1010ce40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:28.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.270+0000 7f3f1010ce40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.270+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677bb020 con 0x5603677b4aa0 2023-10-05T19:40:28.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.270+0000 7f3f0d440700 1 -- 172.21.15.12:0/3341206312 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f3ef8008ae0 con 0x5603677b4aa0 2023-10-05T19:40:28.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.270+0000 7f3f1010ce40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:28.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.270+0000 7f3f1010ce40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.270+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677bb3d0 con 0x5603677b4aa0 2023-10-05T19:40:28.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.270+0000 7f3f0d440700 1 -- 172.21.15.12:0/3341206312 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f3ef8008ae0 con 0x5603677b4aa0 2023-10-05T19:40:28.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.270+0000 7f3f1010ce40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:28.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.270+0000 7f3f1010ce40 20 started sync module instance, tier type = 2023-10-05T19:40:28.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.270+0000 7f3f1010ce40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:28.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.270+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677babd0 con 0x5603677b0300 2023-10-05T19:40:28.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.274+0000 7f3f0dc41700 1 -- 172.21.15.12:0/3341206312 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'144 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3f08025020 con 0x5603677b0300 2023-10-05T19:40:28.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.274+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94572621047520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677bbc70 con 0x5603677b0300 2023-10-05T19:40:28.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.274+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677bc510 con 0x5603677aba30 2023-10-05T19:40:28.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.278+0000 7f3f0dc41700 1 -- 172.21.15.12:0/3341206312 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94572621047520] v28'145 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3f08025020 con 0x5603677b0300 2023-10-05T19:40:28.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.278+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'144 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3f00014040 con 0x5603677aba30 2023-10-05T19:40:28.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.278+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94572621051776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677bc510 con 0x5603677aba30 2023-10-05T19:40:28.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.278+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677bcf20 con 0x5603677b4aa0 2023-10-05T19:40:28.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.278+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94572621051776] v28'145 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3f00014040 con 0x5603677aba30 2023-10-05T19:40:28.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.282+0000 7f3f0d440700 1 -- 172.21.15.12:0/3341206312 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'144 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3ef8008ae0 con 0x5603677b4aa0 2023-10-05T19:40:28.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.282+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94572621053984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677bcb50 con 0x5603677b4aa0 2023-10-05T19:40:28.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.282+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677bdcb0 con 0x5603677b0300 2023-10-05T19:40:28.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.282+0000 7f3f0dc41700 1 -- 172.21.15.12:0/3341206312 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'144 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3f08025020 con 0x5603677b0300 2023-10-05T19:40:28.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.282+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94572621060032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677be550 con 0x5603677b0300 2023-10-05T19:40:28.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.282+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677bf0c0 con 0x5603677aba30 2023-10-05T19:40:28.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.282+0000 7f3f0d440700 1 -- 172.21.15.12:0/3341206312 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94572621053984] v28'145 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3ef8008ae0 con 0x5603677b4aa0 2023-10-05T19:40:28.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.286+0000 7f3f0dc41700 1 -- 172.21.15.12:0/3341206312 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94572621060032] v28'145 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3f08025020 con 0x5603677b0300 2023-10-05T19:40:28.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.286+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'144 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3f00014040 con 0x5603677aba30 2023-10-05T19:40:28.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.286+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94572621058768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677be060 con 0x5603677aba30 2023-10-05T19:40:28.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.286+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677bf9b0 con 0x5603676ee920 2023-10-05T19:40:28.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.290+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94572621058768] v28'145 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3f00014040 con 0x5603677aba30 2023-10-05T19:40:28.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.290+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'144 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3f00014040 con 0x5603676ee920 2023-10-05T19:40:28.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.290+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94572621065104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677be060 con 0x5603676ee920 2023-10-05T19:40:28.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.290+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677bfe80 con 0x5603676ee920 2023-10-05T19:40:28.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.290+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(95 notify.5 [watch watch cookie 94572621065104] v28'145 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3f00014040 con 0x5603676ee920 2023-10-05T19:40:28.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.294+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(96 notify.6 [create] v28'144 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3f00014040 con 0x5603676ee920 2023-10-05T19:40:28.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.294+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94572621066992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677bfe40 con 0x5603676ee920 2023-10-05T19:40:28.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.294+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677c0850 con 0x5603676ee920 2023-10-05T19:40:28.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.294+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(97 notify.6 [watch watch cookie 94572621066992] v28'145 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3f00014040 con 0x5603676ee920 2023-10-05T19:40:28.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.294+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(98 notify.7 [create] v28'144 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3f00014040 con 0x5603676ee920 2023-10-05T19:40:28.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.294+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94572621068624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677bfda0 con 0x5603676ee920 2023-10-05T19:40:28.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.294+0000 7f3f1010ce40 20 add_watcher() i=0 2023-10-05T19:40:28.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.294+0000 7f3f1010ce40 20 add_watcher() i=1 2023-10-05T19:40:28.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.294+0000 7f3f1010ce40 20 add_watcher() i=2 2023-10-05T19:40:28.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.294+0000 7f3f1010ce40 20 add_watcher() i=3 2023-10-05T19:40:28.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.294+0000 7f3f1010ce40 20 add_watcher() i=4 2023-10-05T19:40:28.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.294+0000 7f3f1010ce40 20 add_watcher() i=5 2023-10-05T19:40:28.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.294+0000 7f3f1010ce40 20 add_watcher() i=6 2023-10-05T19:40:28.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.298+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94572621068624] v28'145 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3f00014040 con 0x5603676ee920 2023-10-05T19:40:28.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.298+0000 7f3f1010ce40 20 add_watcher() i=7 2023-10-05T19:40:28.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.298+0000 7f3f1010ce40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:28.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.298+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677c3bd0 con 0x5603677aba30 2023-10-05T19:40:28.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.298+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f3f00014040 con 0x5603677aba30 2023-10-05T19:40:28.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.298+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94572621071488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677c1090 con 0x5603677aba30 2023-10-05T19:40:28.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.302+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94572621071488] v28'140 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f3f00014040 con 0x5603677aba30 2023-10-05T19:40:28.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.302+0000 7f3edefe5700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:28.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.302+0000 7f3f1010ce40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:28.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.302+0000 7f3f1010ce40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:28.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.302+0000 7f3f1010ce40 1 Processor -- start 2023-10-05T19:40:28.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.302+0000 7f3f1010ce40 1 -- start start 2023-10-05T19:40:28.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.302+0000 7f3f1010ce40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5603677ce210 0x5603677ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:28.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.302+0000 7f3f1010ce40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603677ceb20 con 0x5603677ce210 2023-10-05T19:40:28.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.302+0000 7f3f0d440700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5603677ce210 0x5603677ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:28.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.302+0000 7f3f0d440700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5603677ce210 0x5603677ce5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58384/0 (socket says 172.21.15.12:58384) 2023-10-05T19:40:28.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.302+0000 7f3f0d440700 1 -- 172.21.15.12:0/3456592822 learned_addr learned my addr 172.21.15.12:0/3456592822 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:28.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.302+0000 7f3f0d440700 1 -- 172.21.15.12:0/3456592822 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603677cec60 con 0x5603677ce210 2023-10-05T19:40:28.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.302+0000 7f3f0d440700 1 --2- 172.21.15.12:0/3456592822 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5603677ce210 0x5603677ce5e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x5603676f0090 tx=0x7f3ef8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48537b03201568a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:28.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.302+0000 7f3ede7e4700 1 -- 172.21.15.12:0/3456592822 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3ef8030930 con 0x5603677ce210 2023-10-05T19:40:28.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.302+0000 7f3ede7e4700 1 -- 172.21.15.12:0/3456592822 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef8030ad0 con 0x5603677ce210 2023-10-05T19:40:28.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.302+0000 7f3ede7e4700 1 -- 172.21.15.12:0/3456592822 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3ef8030cc0 con 0x5603677ce210 2023-10-05T19:40:28.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.306+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3456592822 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5603677ce210 msgr2=0x5603677ce5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.306+0000 7f3f1010ce40 1 --2- 172.21.15.12:0/3456592822 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5603677ce210 0x5603677ce5e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x5603676f0090 tx=0x7f3ef8012000 comp rx=0 tx=0).stop 2023-10-05T19:40:28.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.306+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3456592822 shutdown_connections 2023-10-05T19:40:28.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.306+0000 7f3f1010ce40 1 --2- 172.21.15.12:0/3456592822 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5603677ce210 0x5603677ce5e0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.306+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3456592822 >> 172.21.15.12:0/3456592822 conn(0x5603677c95d0 msgr2=0x5603677c7720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:28.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.306+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3456592822 shutdown_connections 2023-10-05T19:40:28.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.306+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3456592822 wait complete. 2023-10-05T19:40:28.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.306+0000 7f3f1010ce40 1 Processor -- start 2023-10-05T19:40:28.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.306+0000 7f3f1010ce40 1 -- start start 2023-10-05T19:40:28.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.306+0000 7f3f1010ce40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5603677ce210 0x5603677d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:28.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.306+0000 7f3f1010ce40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603677d4d40 con 0x5603677ce210 2023-10-05T19:40:28.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.306+0000 7f3f0d440700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5603677ce210 0x5603677d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:28.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.306+0000 7f3f0d440700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5603677ce210 0x5603677d3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58398/0 (socket says 172.21.15.12:58398) 2023-10-05T19:40:28.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.306+0000 7f3f0d440700 1 -- 172.21.15.12:0/1926475964 learned_addr learned my addr 172.21.15.12:0/1926475964 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:28.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.306+0000 7f3f0d440700 1 -- 172.21.15.12:0/1926475964 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603676f00c0 con 0x5603677ce210 2023-10-05T19:40:28.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.306+0000 7f3f0d440700 1 --2- 172.21.15.12:0/1926475964 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5603677ce210 0x5603677d3d40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3ef803b000 tx=0x7f3ef8034000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:28.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.306+0000 7f3edcfe1700 1 -- 172.21.15.12:0/1926475964 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3ef8027400 con 0x5603677ce210 2023-10-05T19:40:28.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.306+0000 7f3edcfe1700 1 -- 172.21.15.12:0/1926475964 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef8027580 con 0x5603677ce210 2023-10-05T19:40:28.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.306+0000 7f3edcfe1700 1 -- 172.21.15.12:0/1926475964 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3ef8027770 con 0x5603677ce210 2023-10-05T19:40:28.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.306+0000 7f3f1010ce40 1 -- 172.21.15.12:0/1926475964 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5603677d4340 con 0x5603677ce210 2023-10-05T19:40:28.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.306+0000 7f3f1010ce40 1 -- 172.21.15.12:0/1926475964 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5603677d4780 con 0x5603677ce210 2023-10-05T19:40:28.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.310+0000 7f3edcfe1700 1 -- 172.21.15.12:0/1926475964 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f3ef80279e0 con 0x5603677ce210 2023-10-05T19:40:28.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.310+0000 7f3edcfe1700 1 --2- 172.21.15.12:0/1926475964 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3e54030c50 0x7f3e54033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:28.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.310+0000 7f3edcfe1700 1 -- 172.21.15.12:0/1926475964 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f3ef8054b10 con 0x5603677ce210 2023-10-05T19:40:28.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.310+0000 7f3f0e442700 1 --2- 172.21.15.12:0/1926475964 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3e54030c50 0x7f3e54033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:28.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.310+0000 7f3f1010ce40 1 --2- 172.21.15.12:0/1926475964 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5603677de930 0x5603677e0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:28.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.310+0000 7f3f1010ce40 1 -- 172.21.15.12:0/1926475964 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677e13b0 con 0x5603677de930 2023-10-05T19:40:28.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.310+0000 7f3f0dc41700 1 --2- 172.21.15.12:0/1926475964 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5603677de930 0x5603677e0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:28.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.310+0000 7f3f0e442700 1 --2- 172.21.15.12:0/1926475964 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3e54030c50 0x7f3e54033100 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x5603677c0940 tx=0x7f3f0000a9d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:28.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.310+0000 7f3f0dc41700 1 --2- 172.21.15.12:0/1926475964 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5603677de930 0x5603677e0d10 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:28.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.314+0000 7f3f0dc41700 1 -- 172.21.15.12:0/1926475964 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v28'26 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f3f08025020 con 0x5603677de930 2023-10-05T19:40:28.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.314+0000 7f3f1010ce40 1 --2- 172.21.15.12:0/1926475964 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x5603677e2080 0x5603677e4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:28.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.314+0000 7f3f1010ce40 1 -- 172.21.15.12:0/1926475964 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677e4bc0 con 0x5603677e2080 2023-10-05T19:40:28.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.314+0000 7f3f0d440700 1 --2- 172.21.15.12:0/1926475964 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x5603677e2080 0x5603677e4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:28.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.314+0000 7f3f0d440700 1 --2- 172.21.15.12:0/1926475964 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x5603677e2080 0x5603677e4490 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:28.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.318+0000 7f3f0d440700 1 -- 172.21.15.12:0/1926475964 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v28'17 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f3ef8008ae0 con 0x5603677e2080 2023-10-05T19:40:28.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.318+0000 7f3f1010ce40 1 --2- 172.21.15.12:0/1926475964 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5603677e5890 0x5603677e7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:28.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.318+0000 7f3f1010ce40 1 -- 172.21.15.12:0/1926475964 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677e83a0 con 0x5603677e5890 2023-10-05T19:40:28.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.318+0000 7f3f0e442700 1 --2- 172.21.15.12:0/1926475964 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5603677e5890 0x5603677e7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:28.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.318+0000 7f3f0e442700 1 --2- 172.21.15.12:0/1926475964 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5603677e5890 0x5603677e7c70 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:28.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.322+0000 7f3f0e442700 1 -- 172.21.15.12:0/1926475964 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v28'17 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f3f00014040 con 0x5603677e5890 2023-10-05T19:40:28.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.322+0000 7f3f1010ce40 1 -- 172.21.15.12:0/1926475964 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677e89f0 con 0x5603677de930 2023-10-05T19:40:28.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.322+0000 7f3f0dc41700 1 -- 172.21.15.12:0/1926475964 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v28'141 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f3f08025020 con 0x5603677de930 2023-10-05T19:40:28.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.322+0000 7f3f1010ce40 1 --2- 172.21.15.12:0/1926475964 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5603677e9690 0x5603677eba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:28.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.322+0000 7f3f1010ce40 1 -- 172.21.15.12:0/1926475964 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677ec120 con 0x5603677e9690 2023-10-05T19:40:28.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.322+0000 7f3f0dc41700 1 --2- 172.21.15.12:0/1926475964 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5603677e9690 0x5603677eba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:28.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.326+0000 7f3f0dc41700 1 --2- 172.21.15.12:0/1926475964 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5603677e9690 0x5603677eba70 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:28.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.326+0000 7f3f0dc41700 1 -- 172.21.15.12:0/1926475964 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v28'41 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f3f08025020 con 0x5603677e9690 2023-10-05T19:40:28.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.326+0000 7f3f1010ce40 1 -- 172.21.15.12:0/1926475964 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677ec740 con 0x5603677e5890 2023-10-05T19:40:28.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.330+0000 7f3f0e442700 1 -- 172.21.15.12:0/1926475964 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v28'30 uv8 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f3f00014040 con 0x5603677e5890 2023-10-05T19:40:28.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.330+0000 7f3f1010ce40 1 -- 172.21.15.12:0/1926475964 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677ecd60 con 0x5603677de930 2023-10-05T19:40:28.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.334+0000 7f3f0dc41700 1 -- 172.21.15.12:0/1926475964 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v28'19 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f3f08025020 con 0x5603677de930 2023-10-05T19:40:28.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.334+0000 7f3f1010ce40 1 -- 172.21.15.12:0/1926475964 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677ecd60 con 0x5603677e2080 2023-10-05T19:40:28.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.334+0000 7f3f0d440700 1 -- 172.21.15.12:0/1926475964 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v28'27 uv8 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f3ef8008ae0 con 0x5603677e2080 2023-10-05T19:40:28.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.334+0000 7f3f1010ce40 1 -- 172.21.15.12:0/1926475964 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677ed3d0 con 0x5603677e9690 2023-10-05T19:40:28.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.338+0000 7f3f0dc41700 1 -- 172.21.15.12:0/1926475964 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v28'17 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f3f08025020 con 0x5603677e9690 2023-10-05T19:40:28.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.338+0000 7f3f1010ce40 1 -- 172.21.15.12:0/1926475964 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677ed3d0 con 0x5603677e5890 2023-10-05T19:40:28.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.342+0000 7f3f0e442700 1 -- 172.21.15.12:0/1926475964 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v28'23 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f3f00014040 con 0x5603677e5890 2023-10-05T19:40:28.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.342+0000 7f3f1010ce40 1 -- 172.21.15.12:0/1926475964 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677ed9f0 con 0x5603677e9690 2023-10-05T19:40:28.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.342+0000 7f3f0dc41700 1 -- 172.21.15.12:0/1926475964 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v28'42 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3f08025020 con 0x5603677e9690 2023-10-05T19:40:28.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.342+0000 7f3f1010ce40 1 -- 172.21.15.12:0/1926475964 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677ed9f0 con 0x5603677e2080 2023-10-05T19:40:28.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.346+0000 7f3f0d440700 1 -- 172.21.15.12:0/1926475964 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v28'19 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3ef8008ae0 con 0x5603677e2080 2023-10-05T19:40:28.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.346+0000 7f3f1010ce40 1 -- 172.21.15.12:0/1926475964 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677ee010 con 0x5603677e9690 2023-10-05T19:40:28.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.350+0000 7f3f0dc41700 1 -- 172.21.15.12:0/1926475964 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v28'17 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3f08025020 con 0x5603677e9690 2023-10-05T19:40:28.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.350+0000 7f3f1010ce40 1 -- 172.21.15.12:0/1926475964 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677ee010 con 0x5603677e2080 2023-10-05T19:40:28.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.354+0000 7f3f0d440700 1 -- 172.21.15.12:0/1926475964 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v28'35 uv12 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3ef8008ae0 con 0x5603677e2080 2023-10-05T19:40:28.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.354+0000 7f3f1010ce40 1 -- 172.21.15.12:0/1926475964 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677ee500 con 0x5603677de930 2023-10-05T19:40:28.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.354+0000 7f3f0dc41700 1 -- 172.21.15.12:0/1926475964 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v28'142 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3f08025020 con 0x5603677de930 2023-10-05T19:40:28.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.354+0000 7f3f1010ce40 1 -- 172.21.15.12:0/1926475964 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677ee500 con 0x5603677e5890 2023-10-05T19:40:28.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.358+0000 7f3f0e442700 1 -- 172.21.15.12:0/1926475964 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v28'35 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3f00014040 con 0x5603677e5890 2023-10-05T19:40:28.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.358+0000 7f3f1010ce40 1 -- 172.21.15.12:0/1926475964 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677da280 con 0x5603677e9690 2023-10-05T19:40:28.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.362+0000 7f3f0dc41700 1 -- 172.21.15.12:0/1926475964 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v28'43 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3f08025020 con 0x5603677e9690 2023-10-05T19:40:28.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.362+0000 7f3f1010ce40 1 -- 172.21.15.12:0/1926475964 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677da280 con 0x5603677e5890 2023-10-05T19:40:28.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.362+0000 7f3f0e442700 1 -- 172.21.15.12:0/1926475964 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v28'17 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3f00014040 con 0x5603677e5890 2023-10-05T19:40:28.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.362+0000 7f3f1010ce40 1 -- 172.21.15.12:0/1926475964 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677da280 con 0x5603677de930 2023-10-05T19:40:28.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.366+0000 7f3f0dc41700 1 -- 172.21.15.12:0/1926475964 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v28'33 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3f08025020 con 0x5603677de930 2023-10-05T19:40:28.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.366+0000 7f3f1010ce40 1 -- 172.21.15.12:0/1926475964 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677da280 con 0x5603677de930 2023-10-05T19:40:28.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.370+0000 7f3f0dc41700 1 -- 172.21.15.12:0/1926475964 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v28'143 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3f08025020 con 0x5603677de930 2023-10-05T19:40:28.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.370+0000 7f3f1010ce40 1 -- 172.21.15.12:0/1926475964 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677da280 con 0x5603677e5890 2023-10-05T19:40:28.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.370+0000 7f3f0e442700 1 -- 172.21.15.12:0/1926475964 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v28'36 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3f00014040 con 0x5603677e5890 2023-10-05T19:40:28.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.370+0000 7f3f1010ce40 1 -- 172.21.15.12:0/1926475964 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677da280 con 0x5603677e9690 2023-10-05T19:40:28.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.374+0000 7f3f0dc41700 1 -- 172.21.15.12:0/1926475964 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v28'27 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3f08025020 con 0x5603677e9690 2023-10-05T19:40:28.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.374+0000 7f3f1010ce40 1 -- 172.21.15.12:0/1926475964 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677da280 con 0x5603677e5890 2023-10-05T19:40:28.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.378+0000 7f3f0e442700 1 -- 172.21.15.12:0/1926475964 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v28'19 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3f00014040 con 0x5603677e5890 2023-10-05T19:40:28.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.378+0000 7f3f1010ce40 1 -- 172.21.15.12:0/1926475964 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677da280 con 0x5603677e5890 2023-10-05T19:40:28.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.378+0000 7f3f0e442700 1 -- 172.21.15.12:0/1926475964 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v28'25 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3f00014040 con 0x5603677e5890 2023-10-05T19:40:28.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.378+0000 7f3f1010ce40 1 -- 172.21.15.12:0/1926475964 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677da280 con 0x5603677e5890 2023-10-05T19:40:28.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.382+0000 7f3f0e442700 1 -- 172.21.15.12:0/1926475964 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v28'14 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3f00014040 con 0x5603677e5890 2023-10-05T19:40:28.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.382+0000 7f3f1010ce40 1 -- 172.21.15.12:0/1926475964 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677da280 con 0x5603677e5890 2023-10-05T19:40:28.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.386+0000 7f3f0e442700 1 -- 172.21.15.12:0/1926475964 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v28'31 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3f00014040 con 0x5603677e5890 2023-10-05T19:40:28.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.386+0000 7f3f1010ce40 1 -- 172.21.15.12:0/1926475964 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677da280 con 0x5603677e2080 2023-10-05T19:40:28.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.390+0000 7f3f0d440700 1 -- 172.21.15.12:0/1926475964 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v28'22 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3ef8008ae0 con 0x5603677e2080 2023-10-05T19:40:28.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.390+0000 7f3f1010ce40 1 -- 172.21.15.12:0/1926475964 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677da280 con 0x5603677e5890 2023-10-05T19:40:28.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.390+0000 7f3f0e442700 1 -- 172.21.15.12:0/1926475964 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v28'17 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3f00014040 con 0x5603677e5890 2023-10-05T19:40:28.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.390+0000 7f3f1010ce40 1 -- 172.21.15.12:0/1926475964 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677da280 con 0x5603677e2080 2023-10-05T19:40:28.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.394+0000 7f3f0d440700 1 -- 172.21.15.12:0/1926475964 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v28'28 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3ef8008ae0 con 0x5603677e2080 2023-10-05T19:40:28.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.394+0000 7f3f1010ce40 1 -- 172.21.15.12:0/1926475964 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677da280 con 0x5603677e5890 2023-10-05T19:40:28.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.398+0000 7f3f0e442700 1 -- 172.21.15.12:0/1926475964 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v28'37 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3f00014040 con 0x5603677e5890 2023-10-05T19:40:28.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.398+0000 7f3f1010ce40 1 -- 172.21.15.12:0/1926475964 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677da280 con 0x5603677e2080 2023-10-05T19:40:28.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.398+0000 7f3f0d440700 1 -- 172.21.15.12:0/1926475964 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v28'36 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3ef8008ae0 con 0x5603677e2080 2023-10-05T19:40:28.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.398+0000 7f3f1010ce40 1 -- 172.21.15.12:0/1926475964 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5603677da280 con 0x5603677e9690 2023-10-05T19:40:28.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.402+0000 7f3f0dc41700 1 -- 172.21.15.12:0/1926475964 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v28'23 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3f08025020 con 0x5603677e9690 2023-10-05T19:40:28.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.406+0000 7f3e4dffb700 20 reqs_thread_entry: start 2023-10-05T19:40:28.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.406+0000 7f3e3f7fe700 20 reqs_thread_entry: start 2023-10-05T19:40:28.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.406+0000 7f3f1010ce40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:28.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.406+0000 7f3f1010ce40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:28.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.406+0000 7f3e3dffb700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:58 2023 2023-10-05T19:40:28.438 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:28.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.406+0000 7f3e3d7fa700 20 reqs_thread_entry: start 2023-10-05T19:40:28.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.406+0000 7f3f1010ce40 20 get_system_obj_state: rctx=0x7fff95d1f4f0 obj=default.rgw.meta:root:myfoo state=0x5603676530b0 s->prefetch_data=0 2023-10-05T19:40:28.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.406+0000 7f3f1010ce40 10 cache get: name=default.rgw.meta+root+myfoo : miss 2023-10-05T19:40:28.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.406+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:102 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x56036784d7b0 con 0x5603677b0300 2023-10-05T19:40:28.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.406+0000 7f3f0dc41700 1 -- 172.21.15.12:0/3341206312 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(102 myfoo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv27 ondisk = 0) v8 ==== 233+0+139 (crc 0 0 0) 0x7f3f08025020 con 0x5603677b0300 2023-10-05T19:40:28.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.406+0000 7f3f1010ce40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x16 2023-10-05T19:40:28.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.406+0000 7f3f1010ce40 10 adding default.rgw.meta+root+myfoo to cache LRU end 2023-10-05T19:40:28.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.406+0000 7f3f1010ce40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:28.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.406+0000 7f3f1010ce40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:28.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.406+0000 7f3f1010ce40 10 cache get: name=default.rgw.meta+root+myfoo : type miss (requested=0x11, cached=0x16) 2023-10-05T19:40:28.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.406+0000 7f3f1010ce40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.406+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:103 8.9 8:91b71841:root::myfoo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x56036784d6a0 con 0x5603677b0300 2023-10-05T19:40:28.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.410+0000 7f3f0dc41700 1 -- 172.21.15.12:0/3341206312 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(103 myfoo [call,call out=48b,read 0~165 out=165b] v0'0 uv27 ondisk = 0) v8 ==== 233+0+213 (crc 0 0 0) 0x7f3f08025020 con 0x5603677b0300 2023-10-05T19:40:28.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.410+0000 7f3f1010ce40 20 rados_obj.operate() r=0 bl.length=165 2023-10-05T19:40:28.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.410+0000 7f3f1010ce40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2023-10-05T19:40:28.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.410+0000 7f3f1010ce40 10 moving default.rgw.meta+root+myfoo to cache LRU end 2023-10-05T19:40:28.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.410+0000 7f3f1010ce40 20 get_system_obj_state: rctx=0x7fff95d1f4f0 obj=default.rgw.meta:root:.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 state=0x5603676292b0 s->prefetch_data=0 2023-10-05T19:40:28.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.410+0000 7f3f1010ce40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 : miss 2023-10-05T19:40:28.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.410+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:104 8.e 8:70176bbd:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x56036784f230 con 0x5603676ee920 2023-10-05T19:40:28.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.410+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(104 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 [call out=48b,getxattrs out=218b,stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 290+0+282 (crc 0 0 0) 0x7f3f00014040 con 0x5603676ee920 2023-10-05T19:40:28.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.410+0000 7f3f1010ce40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 info.flags=0x16 2023-10-05T19:40:28.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.410+0000 7f3f1010ce40 10 adding default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 to cache LRU end 2023-10-05T19:40:28.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.410+0000 7f3f1010ce40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:28.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.410+0000 7f3f1010ce40 10 updating xattr: name=user.rgw.acl bl.length()=123 2023-10-05T19:40:28.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.410+0000 7f3f1010ce40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:28.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.410+0000 7f3f1010ce40 20 Read xattr: user.rgw.acl 2023-10-05T19:40:28.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.410+0000 7f3f1010ce40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:28.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.410+0000 7f3f1010ce40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 : type miss (requested=0x13, cached=0x16) 2023-10-05T19:40:28.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.410+0000 7f3f1010ce40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.410+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:105 8.e 8:70176bbd:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x56036784fe30 con 0x5603676ee920 2023-10-05T19:40:28.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.410+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(105 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 [call,call out=48b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv7 ondisk = 0) v8 ==== 332+0+618 (crc 0 0 0) 0x7f3f00014040 con 0x5603676ee920 2023-10-05T19:40:28.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.410+0000 7f3f1010ce40 20 rados_obj.operate() r=0 bl.length=352 2023-10-05T19:40:28.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.410+0000 7f3f1010ce40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 info.flags=0x13 2023-10-05T19:40:28.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.410+0000 7f3f1010ce40 10 moving default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 to cache LRU end 2023-10-05T19:40:28.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.410+0000 7f3f1010ce40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:28.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.410+0000 7f3f1010ce40 10 updating xattr: name=user.rgw.acl bl.length()=123 2023-10-05T19:40:28.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.410+0000 7f3f1010ce40 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 2023-10-05T19:40:28.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.410+0000 7f3f1010ce40 20 get_obj_state: rctx=0x7fff95d1f7b0 obj=myfoo:four state=0x5603678505f8 s->prefetch_data=0 2023-10-05T19:40:28.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.410+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:106 3.6 3:607d4bcd:::93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1_four:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x560367851450 con 0x5603677aba30 2023-10-05T19:40:28.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.410+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(106 93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1_four [getxattrs out=849b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 234+0+865 (crc 0 0 0) 0x7f3f00014040 con 0x5603677aba30 2023-10-05T19:40:28.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.414+0000 7f3f1010ce40 10 manifest: total_size = 4 2023-10-05T19:40:28.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.414+0000 7f3f1010ce40 20 get_obj_state: setting s->obj_tag to 93ac49f4-ff49-4f99-811b-6453b28decd5.4219.3134915741190281984 2023-10-05T19:40:28.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.414+0000 7f3f1010ce40 20 get_obj_state: rctx=0x7fff95d1f7b0 obj=myfoo:four state=0x5603678505f8 s->prefetch_data=0 2023-10-05T19:40:28.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.414+0000 7f3f1010ce40 20 prepare_atomic_modification: state is not atomic. state=0x5603678505f8 2023-10-05T19:40:28.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.414+0000 7f3f1010ce40 20 bucket index object: default.rgw.buckets.index:.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.6 2023-10-05T19:40:28.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.414+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:107 4.2c 4:358e5801:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.6:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=182b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x560367855bb0 con 0x5603677aba30 2023-10-05T19:40:28.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.414+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(107 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.6 [stat,call,call] v28'4 uv4 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x7f3f00014040 con 0x5603677aba30 2023-10-05T19:40:28.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.414+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:108 3.6 3:607d4bcd:::93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1_four:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x560367855bb0 con 0x5603677aba30 2023-10-05T19:40:28.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.418+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(108 93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1_four [call] v28'2 uv2 ondisk = 0) v8 ==== 192+0+0 (crc 0 0 0) 0x7f3f00014040 con 0x5603677aba30 2023-10-05T19:40:28.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.418+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:109 4.2c 4:358e5801:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.6:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=259b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x560367855bb0 con 0x5603677aba30 2023-10-05T19:40:28.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.418+0000 7f3f1010ce40 10 cache get: name=default.rgw.log++bucket.sync-source-hints.myfoo : miss 2023-10-05T19:40:28.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.418+0000 7f3f1010ce40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.418+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:110 6.18 6:1df25bda:::bucket.sync-source-hints.myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x560367858e70 con 0x5603676ee920 2023-10-05T19:40:28.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.418+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(110 bucket.sync-source-hints.myfoo [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 216+0+0 (crc 0 0 0) 0x7f3f00014040 con 0x5603676ee920 2023-10-05T19:40:28.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.418+0000 7f3f1010ce40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:28.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.418+0000 7f3f1010ce40 10 cache put: name=default.rgw.log++bucket.sync-source-hints.myfoo info.flags=0x0 2023-10-05T19:40:28.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.418+0000 7f3f1010ce40 10 adding default.rgw.log++bucket.sync-source-hints.myfoo to cache LRU end 2023-10-05T19:40:28.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.418+0000 7f3f1010ce40 10 cache get: name=default.rgw.log++bucket.sync-target-hints.myfoo : miss 2023-10-05T19:40:28.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.418+0000 7f3f1010ce40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.418+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:111 6.4 6:26e4c4bc:::bucket.sync-target-hints.myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x560367855bb0 con 0x5603677aba30 2023-10-05T19:40:28.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.418+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(111 bucket.sync-target-hints.myfoo [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 216+0+0 (crc 0 0 0) 0x7f3f00014040 con 0x5603677aba30 2023-10-05T19:40:28.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.418+0000 7f3f1010ce40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:28.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.418+0000 7f3f1010ce40 10 cache put: name=default.rgw.log++bucket.sync-target-hints.myfoo info.flags=0x0 2023-10-05T19:40:28.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.418+0000 7f3f1010ce40 10 adding default.rgw.log++bucket.sync-target-hints.myfoo to cache LRU end 2023-10-05T19:40:28.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.418+0000 7f3f1010ce40 10 chain_cache_entry: cache_locator= 2023-10-05T19:40:28.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.418+0000 7f3f1010ce40 20 chain_cache_entry: couldn't find cache locator 2023-10-05T19:40:28.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.418+0000 7f3f1010ce40 20 couldn't put bucket_sync_policy cache entry, might have raced with data changes 2023-10-05T19:40:28.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.422+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:112 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94572621047520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x560367791820 con 0x5603677b0300 2023-10-05T19:40:28.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.422+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(109 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.6 [stat,call,call] v28'5 uv5 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x7f3f00014040 con 0x5603677aba30 2023-10-05T19:40:28.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.422+0000 7f3f077fe700 20 handle_completion(): completion ok for obj=four 2023-10-05T19:40:28.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.422+0000 7f3f0dc41700 1 -- 172.21.15.12:0/3341206312 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(112 notify.0 [watch unwatch cookie 94572621047520] v28'146 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3f08025020 con 0x5603677b0300 2023-10-05T19:40:28.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.422+0000 7f3f1010ce40 20 remove_watcher() i=0 2023-10-05T19:40:28.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.422+0000 7f3f1010ce40 2 removed watcher, disabling cache 2023-10-05T19:40:28.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.422+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:113 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94572621051776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x560367791820 con 0x5603677aba30 2023-10-05T19:40:28.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.426+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== osd_op_reply(113 notify.1 [watch unwatch cookie 94572621051776] v28'146 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3f00014040 con 0x5603677aba30 2023-10-05T19:40:28.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.426+0000 7f3f1010ce40 20 remove_watcher() i=1 2023-10-05T19:40:28.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.426+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:114 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94572621053984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x560367791820 con 0x5603677b4aa0 2023-10-05T19:40:28.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.430+0000 7f3f0d440700 1 -- 172.21.15.12:0/3341206312 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(114 notify.2 [watch unwatch cookie 94572621053984] v28'146 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3ef8008ae0 con 0x5603677b4aa0 2023-10-05T19:40:28.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.430+0000 7f3f1010ce40 20 remove_watcher() i=2 2023-10-05T19:40:28.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.430+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:115 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94572621060032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x560367791820 con 0x5603677b0300 2023-10-05T19:40:28.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.434+0000 7f3f0dc41700 1 -- 172.21.15.12:0/3341206312 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(115 notify.3 [watch unwatch cookie 94572621060032] v28'146 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3f08025020 con 0x5603677b0300 2023-10-05T19:40:28.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.434+0000 7f3f1010ce40 20 remove_watcher() i=3 2023-10-05T19:40:28.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.434+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:116 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94572621058768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x560367791820 con 0x5603677aba30 2023-10-05T19:40:28.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.434+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.1 v2:172.21.15.12:6808/29933 39 ==== osd_op_reply(116 notify.4 [watch unwatch cookie 94572621058768] v28'146 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3f00014040 con 0x5603677aba30 2023-10-05T19:40:28.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.434+0000 7f3f1010ce40 20 remove_watcher() i=4 2023-10-05T19:40:28.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.434+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:117 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94572621065104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x560367791820 con 0x5603676ee920 2023-10-05T19:40:28.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.438+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.0 v2:172.21.15.8:6800/27563 30 ==== osd_op_reply(117 notify.5 [watch unwatch cookie 94572621065104] v28'146 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3f00014040 con 0x5603676ee920 2023-10-05T19:40:28.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.438+0000 7f3f1010ce40 20 remove_watcher() i=5 2023-10-05T19:40:28.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.438+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:118 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94572621066992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x560367791820 con 0x5603676ee920 2023-10-05T19:40:28.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.442+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.0 v2:172.21.15.8:6800/27563 31 ==== osd_op_reply(118 notify.6 [watch unwatch cookie 94572621066992] v28'146 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3f00014040 con 0x5603676ee920 2023-10-05T19:40:28.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.442+0000 7f3f1010ce40 20 remove_watcher() i=6 2023-10-05T19:40:28.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.442+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:119 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94572621068624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x560367791820 con 0x5603676ee920 2023-10-05T19:40:28.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.446+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.0 v2:172.21.15.8:6800/27563 32 ==== osd_op_reply(119 notify.7 [watch unwatch cookie 94572621068624] v28'146 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3f00014040 con 0x5603676ee920 2023-10-05T19:40:28.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.446+0000 7f3f1010ce40 20 remove_watcher() i=7 2023-10-05T19:40:28.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.446+0000 7f3edefe5700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:28.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.446+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:120 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94572621071488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x560367791820 con 0x5603677aba30 2023-10-05T19:40:28.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.450+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 <== osd.1 v2:172.21.15.12:6808/29933 40 ==== osd_op_reply(120 data_loggenerations_metadata [watch unwatch cookie 94572621071488] v28'144 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f3f00014040 con 0x5603677aba30 2023-10-05T19:40:28.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.450+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x5603676ee920 msgr2=0x7f3f080113e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.450+0000 7f3f1010ce40 1 --2- 172.21.15.12:0/3341206312 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x5603676ee920 0x7f3f080113e0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.450+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5603677aba30 msgr2=0x5603677ade10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.450+0000 7f3f1010ce40 1 --2- 172.21.15.12:0/3341206312 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5603677aba30 0x5603677ade10 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.450+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5603677b0300 msgr2=0x5603677b26e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.450+0000 7f3f1010ce40 1 --2- 172.21.15.12:0/3341206312 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5603677b0300 0x5603677b26e0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.450+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5603677b4aa0 msgr2=0x5603677b6eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.450+0000 7f3f1010ce40 1 --2- 172.21.15.12:0/3341206312 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5603677b4aa0 0x5603677b6eb0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.450+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3ef0030c50 msgr2=0x7f3ef0033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.450+0000 7f3f1010ce40 1 --2- 172.21.15.12:0/3341206312 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3ef0030c50 0x7f3ef0033100 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8005a90 tx=0x7f3ef8015000 comp rx=0 tx=0).stop 2023-10-05T19:40:28.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.450+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5603676efc30 msgr2=0x5603677a1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.450+0000 7f3f1010ce40 1 --2- 172.21.15.12:0/3341206312 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5603676efc30 0x5603677a1ec0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3f08001560 tx=0x7f3f08014000 comp rx=0 tx=0).stop 2023-10-05T19:40:28.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.450+0000 7f3f0e442700 1 -- 172.21.15.12:0/3341206312 reap_dead start 2023-10-05T19:40:28.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.450+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 shutdown_connections 2023-10-05T19:40:28.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.450+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 >> 172.21.15.12:0/3341206312 conn(0x560367775860 msgr2=0x5603677955c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:28.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.450+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 shutdown_connections 2023-10-05T19:40:28.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.450+0000 7f3f1010ce40 1 -- 172.21.15.12:0/3341206312 wait complete. 2023-10-05T19:40:28.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.450+0000 7f3f1010ce40 1 -- 172.21.15.12:0/1926475964 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x5603677e2080 msgr2=0x5603677e4490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.450+0000 7f3f1010ce40 1 --2- 172.21.15.12:0/1926475964 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x5603677e2080 0x5603677e4490 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.450+0000 7f3f1010ce40 1 -- 172.21.15.12:0/1926475964 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5603677de930 msgr2=0x5603677e0d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.450+0000 7f3f1010ce40 1 --2- 172.21.15.12:0/1926475964 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5603677de930 0x5603677e0d10 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.450+0000 7f3f1010ce40 1 -- 172.21.15.12:0/1926475964 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5603677e9690 msgr2=0x5603677eba70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.450+0000 7f3f1010ce40 1 --2- 172.21.15.12:0/1926475964 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5603677e9690 0x5603677eba70 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.450+0000 7f3f1010ce40 1 -- 172.21.15.12:0/1926475964 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5603677e5890 msgr2=0x5603677e7c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.450+0000 7f3f1010ce40 1 --2- 172.21.15.12:0/1926475964 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5603677e5890 0x5603677e7c70 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.450+0000 7f3f1010ce40 1 -- 172.21.15.12:0/1926475964 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3e54030c50 msgr2=0x7f3e54033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.450+0000 7f3f1010ce40 1 --2- 172.21.15.12:0/1926475964 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3e54030c50 0x7f3e54033100 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x5603677c0940 tx=0x7f3f0000a9d0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.450+0000 7f3f1010ce40 1 -- 172.21.15.12:0/1926475964 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5603677ce210 msgr2=0x5603677d3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.450+0000 7f3f1010ce40 1 --2- 172.21.15.12:0/1926475964 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5603677ce210 0x5603677d3d40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3ef803b000 tx=0x7f3ef8034000 comp rx=0 tx=0).stop 2023-10-05T19:40:28.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.450+0000 7f3f0dc41700 1 -- 172.21.15.12:0/1926475964 reap_dead start 2023-10-05T19:40:28.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.450+0000 7f3f1010ce40 1 -- 172.21.15.12:0/1926475964 shutdown_connections 2023-10-05T19:40:28.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.450+0000 7f3f1010ce40 1 -- 172.21.15.12:0/1926475964 >> 172.21.15.12:0/1926475964 conn(0x5603677c95d0 msgr2=0x5603677c7670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:28.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.454+0000 7f3f1010ce40 1 -- 172.21.15.12:0/1926475964 shutdown_connections 2023-10-05T19:40:28.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.454+0000 7f3f1010ce40 1 -- 172.21.15.12:0/1926475964 wait complete. 2023-10-05T19:40:28.468 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'stats', '--bucket', 'myfoo'] 2023-10-05T19:40:28.468 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'stats', '--bucket', 'myfoo'] 2023-10-05T19:40:28.469 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket stats --bucket myfoo 2023-10-05T19:40:28.508 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:28.508 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:28.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.498+0000 7faf6bef0e40 1 Processor -- start 2023-10-05T19:40:28.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.498+0000 7faf6bef0e40 1 -- start start 2023-10-05T19:40:28.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.498+0000 7faf6bef0e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55f96c7e95f0 0x55f96c7eb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:28.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.498+0000 7faf6bef0e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55f96c72fbd0 con 0x55f96c7e5010 2023-10-05T19:40:28.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.498+0000 7faf6bef0e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x55f96c6ecee0 con 0x55f96c7e95f0 2023-10-05T19:40:28.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.498+0000 7faf69224700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55f96c7e95f0 0x55f96c7eb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:28.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.498+0000 7faf69a25700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x55f96c7e5010 0x55f96c7e53e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:35654/0 (socket says 172.21.15.12:35654) 2023-10-05T19:40:28.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.498+0000 7faf69a25700 1 -- 172.21.15.12:0/1228250300 learned_addr learned my addr 172.21.15.12:0/1228250300 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:28.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.502+0000 7faf68a23700 1 -- 172.21.15.12:0/1228250300 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 228984266 0 0) 0x55f96c72fbd0 con 0x55f96c7e5010 2023-10-05T19:40:28.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.502+0000 7faf68a23700 1 -- 172.21.15.12:0/1228250300 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf58001e10 con 0x55f96c7e5010 2023-10-05T19:40:28.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.502+0000 7faf69224700 1 -- 172.21.15.12:0/1228250300 >> v1:172.21.15.8:6789/0 conn(0x55f96c7e5010 legacy=0x55f96c7e53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.502+0000 7faf69224700 1 -- 172.21.15.12:0/1228250300 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f96c7e5c20 con 0x55f96c7e95f0 2023-10-05T19:40:28.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.502+0000 7faf69224700 1 --2- 172.21.15.12:0/1228250300 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f96c7e95f0 0x55f96c7eb9d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7faf5400a5d0 tx=0x7faf54005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=174ea203f9a5065 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:28.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.502+0000 7faf68a23700 1 -- 172.21.15.12:0/1228250300 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf54007610 con 0x55f96c7e95f0 2023-10-05T19:40:28.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.502+0000 7faf68a23700 1 -- 172.21.15.12:0/1228250300 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf54007770 con 0x55f96c7e95f0 2023-10-05T19:40:28.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.502+0000 7faf68a23700 1 -- 172.21.15.12:0/1228250300 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf54007a20 con 0x55f96c7e95f0 2023-10-05T19:40:28.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.502+0000 7faf6bef0e40 1 -- 172.21.15.12:0/1228250300 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f96c7e95f0 msgr2=0x55f96c7eb9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.502+0000 7faf6bef0e40 1 --2- 172.21.15.12:0/1228250300 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f96c7e95f0 0x55f96c7eb9d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7faf5400a5d0 tx=0x7faf54005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:28.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.502+0000 7faf6bef0e40 1 -- 172.21.15.12:0/1228250300 shutdown_connections 2023-10-05T19:40:28.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.502+0000 7faf6bef0e40 1 --2- 172.21.15.12:0/1228250300 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f96c7e95f0 0x55f96c7eb9d0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.502+0000 7faf6bef0e40 1 -- 172.21.15.12:0/1228250300 >> 172.21.15.12:0/1228250300 conn(0x55f96c758860 msgr2=0x55f96c758c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:28.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.502+0000 7faf6bef0e40 1 -- 172.21.15.12:0/1228250300 shutdown_connections 2023-10-05T19:40:28.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.502+0000 7faf6bef0e40 1 -- 172.21.15.12:0/1228250300 wait complete. 2023-10-05T19:40:28.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.502+0000 7faf6bef0e40 1 Processor -- start 2023-10-05T19:40:28.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.502+0000 7faf6bef0e40 1 -- start start 2023-10-05T19:40:28.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.502+0000 7faf6bef0e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f96c7e5010 0x55f96c804040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:28.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.502+0000 7faf6bef0e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f96c6b0e30 con 0x55f96c7e5010 2023-10-05T19:40:28.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.502+0000 7faf69a25700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f96c7e5010 0x55f96c804040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:28.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.502+0000 7faf69a25700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f96c7e5010 0x55f96c804040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58414/0 (socket says 172.21.15.12:58414) 2023-10-05T19:40:28.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.502+0000 7faf69a25700 1 -- 172.21.15.12:0/812627073 learned_addr learned my addr 172.21.15.12:0/812627073 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:28.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.506+0000 7faf69a25700 1 -- 172.21.15.12:0/812627073 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf54005cf0 con 0x55f96c7e5010 2023-10-05T19:40:28.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.506+0000 7faf69a25700 1 --2- 172.21.15.12:0/812627073 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f96c7e5010 0x55f96c804040 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7faf5c001c30 tx=0x7faf5c001810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=807dd9978abbbba3 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:28.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.506+0000 7faf637fe700 1 -- 172.21.15.12:0/812627073 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf5c00bbf0 con 0x55f96c7e5010 2023-10-05T19:40:28.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.506+0000 7faf637fe700 1 -- 172.21.15.12:0/812627073 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf5c00bd50 con 0x55f96c7e5010 2023-10-05T19:40:28.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.506+0000 7faf637fe700 1 -- 172.21.15.12:0/812627073 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf5c00e470 con 0x55f96c7e5010 2023-10-05T19:40:28.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.506+0000 7faf6bef0e40 1 -- 172.21.15.12:0/812627073 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f96c7e5010 msgr2=0x55f96c804040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.506+0000 7faf6bef0e40 1 --2- 172.21.15.12:0/812627073 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f96c7e5010 0x55f96c804040 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7faf5c001c30 tx=0x7faf5c001810 comp rx=0 tx=0).stop 2023-10-05T19:40:28.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.506+0000 7faf6bef0e40 1 -- 172.21.15.12:0/812627073 shutdown_connections 2023-10-05T19:40:28.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.506+0000 7faf6bef0e40 1 --2- 172.21.15.12:0/812627073 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f96c7e5010 0x55f96c804040 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.506+0000 7faf6bef0e40 1 -- 172.21.15.12:0/812627073 >> 172.21.15.12:0/812627073 conn(0x55f96c758860 msgr2=0x55f96c7ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:28.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.506+0000 7faf6bef0e40 1 -- 172.21.15.12:0/812627073 shutdown_connections 2023-10-05T19:40:28.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.506+0000 7faf6bef0e40 1 -- 172.21.15.12:0/812627073 wait complete. 2023-10-05T19:40:28.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.506+0000 7faf6bef0e40 1 Processor -- start 2023-10-05T19:40:28.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.506+0000 7faf6bef0e40 1 -- start start 2023-10-05T19:40:28.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.506+0000 7faf6bef0e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f96c80afa0 0x55f96c80b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:28.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.506+0000 7faf6bef0e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f96c683ed0 con 0x55f96c80afa0 2023-10-05T19:40:28.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.506+0000 7faf69a25700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f96c80afa0 0x55f96c80b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:28.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.506+0000 7faf69a25700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f96c80afa0 0x55f96c80b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58428/0 (socket says 172.21.15.12:58428) 2023-10-05T19:40:28.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.506+0000 7faf69a25700 1 -- 172.21.15.12:0/3008355714 learned_addr learned my addr 172.21.15.12:0/3008355714 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:28.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.510+0000 7faf69a25700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf58001e10 con 0x55f96c80afa0 2023-10-05T19:40:28.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.510+0000 7faf69a25700 1 --2- 172.21.15.12:0/3008355714 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f96c80afa0 0x55f96c80b370 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x55f96c800720 tx=0x7faf5c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:28.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.510+0000 7faf61ffb700 1 -- 172.21.15.12:0/3008355714 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf5c01f070 con 0x55f96c80afa0 2023-10-05T19:40:28.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.510+0000 7faf61ffb700 1 -- 172.21.15.12:0/3008355714 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf5c018640 con 0x55f96c80afa0 2023-10-05T19:40:28.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.510+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f96c80b8b0 con 0x55f96c80afa0 2023-10-05T19:40:28.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.510+0000 7faf61ffb700 1 -- 172.21.15.12:0/3008355714 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf5c018910 con 0x55f96c80afa0 2023-10-05T19:40:28.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.510+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f96c80e4c0 con 0x55f96c80afa0 2023-10-05T19:40:28.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.510+0000 7faf6bef0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.510+0000 7faf61ffb700 1 -- 172.21.15.12:0/3008355714 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7faf5c018a70 con 0x55f96c80afa0 2023-10-05T19:40:28.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.510+0000 7faf61ffb700 1 --2- 172.21.15.12:0/3008355714 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7faf4c030ca0 0x7faf4c033150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:28.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.510+0000 7faf61ffb700 1 -- 172.21.15.12:0/3008355714 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7faf5c044ca0 con 0x55f96c80afa0 2023-10-05T19:40:28.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.510+0000 7faf69224700 1 --2- 172.21.15.12:0/3008355714 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7faf4c030ca0 0x7faf4c033150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:28.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.514+0000 7faf6bef0e40 1 --2- 172.21.15.12:0/3008355714 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55f96c814e10 0x55f96c8171f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:28.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.514+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c817820 con 0x55f96c814e10 2023-10-05T19:40:28.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.514+0000 7faf6a226700 1 --2- 172.21.15.12:0/3008355714 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55f96c814e10 0x55f96c8171f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:28.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.514+0000 7faf69224700 1 --2- 172.21.15.12:0/3008355714 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7faf4c030ca0 0x7faf4c033150 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x55f96c7e6060 tx=0x7faf54014000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:28.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.514+0000 7faf6a226700 1 --2- 172.21.15.12:0/3008355714 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55f96c814e10 0x55f96c8171f0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:28.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.514+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c814e10 2023-10-05T19:40:28.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.514+0000 7faf6bef0e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:28.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.514+0000 7faf6bef0e40 20 realm 2023-10-05T19:40:28.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.514+0000 7faf6bef0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.514+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c814440 con 0x55f96c814e10 2023-10-05T19:40:28.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.514+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c814e10 2023-10-05T19:40:28.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.514+0000 7faf6bef0e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:28.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.514+0000 7faf6bef0e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:28.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.514+0000 7faf6bef0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.514+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c818300 con 0x55f96c814e10 2023-10-05T19:40:28.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.514+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c814e10 2023-10-05T19:40:28.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.514+0000 7faf6bef0e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:28.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.514+0000 7faf6bef0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.514+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c817eb0 con 0x55f96c814e10 2023-10-05T19:40:28.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.514+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c814e10 2023-10-05T19:40:28.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.514+0000 7faf6bef0e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:28.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.514+0000 7faf6bef0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.514+0000 7faf6bef0e40 1 --2- 172.21.15.12:0/3008355714 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55f96c819250 0x55f96c81b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:28.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.514+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c81bd70 con 0x55f96c819250 2023-10-05T19:40:28.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.514+0000 7faf69a25700 1 --2- 172.21.15.12:0/3008355714 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55f96c819250 0x55f96c81b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:28.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.518+0000 7faf69a25700 1 --2- 172.21.15.12:0/3008355714 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55f96c819250 0x55f96c81b630 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:28.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.518+0000 7faf69a25700 1 -- 172.21.15.12:0/3008355714 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7faf5c014070 con 0x55f96c819250 2023-10-05T19:40:28.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.518+0000 7faf6bef0e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:28.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.518+0000 7faf6bef0e40 20 searching for the correct realm 2023-10-05T19:40:28.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.518+0000 7faf6bef0e40 1 --2- 172.21.15.12:0/3008355714 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55f96c81dba0 0x55f96c81ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:28.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.518+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8204c0 con 0x55f96c81dba0 2023-10-05T19:40:28.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.518+0000 7faf69224700 1 --2- 172.21.15.12:0/3008355714 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55f96c81dba0 0x55f96c81ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:28.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.518+0000 7faf69224700 1 --2- 172.21.15.12:0/3008355714 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55f96c81dba0 0x55f96c81ff80 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:28.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.518+0000 7faf69224700 1 -- 172.21.15.12:0/3008355714 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7faf54007c20 con 0x55f96c81dba0 2023-10-05T19:40:28.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.522+0000 7faf69224700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8204c0 con 0x55f96c819250 2023-10-05T19:40:28.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.522+0000 7faf69a25700 1 -- 172.21.15.12:0/3008355714 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf5c014070 con 0x55f96c819250 2023-10-05T19:40:28.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.522+0000 7faf69a25700 1 --2- 172.21.15.12:0/3008355714 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55f96c7e5010 0x7faf5c0428d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:28.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.522+0000 7faf69a25700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8204c0 con 0x55f96c7e5010 2023-10-05T19:40:28.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.522+0000 7faf6a226700 1 --2- 172.21.15.12:0/3008355714 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55f96c7e5010 0x7faf5c0428d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:28.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.522+0000 7faf6a226700 1 --2- 172.21.15.12:0/3008355714 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55f96c7e5010 0x7faf5c0428d0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:28.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.522+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c7e5010 2023-10-05T19:40:28.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.522+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8204c0 con 0x55f96c814e10 2023-10-05T19:40:28.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.522+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c814e10 2023-10-05T19:40:28.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.522+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8204c0 con 0x55f96c814e10 2023-10-05T19:40:28.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.526+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c814e10 2023-10-05T19:40:28.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.526+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8204c0 con 0x55f96c7e5010 2023-10-05T19:40:28.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.526+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c7e5010 2023-10-05T19:40:28.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.526+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8204c0 con 0x55f96c7e5010 2023-10-05T19:40:28.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.526+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c7e5010 2023-10-05T19:40:28.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.526+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8204c0 con 0x55f96c819250 2023-10-05T19:40:28.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.526+0000 7faf69a25700 1 -- 172.21.15.12:0/3008355714 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf5c014070 con 0x55f96c819250 2023-10-05T19:40:28.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.526+0000 7faf69a25700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8204c0 con 0x55f96c81dba0 2023-10-05T19:40:28.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.526+0000 7faf69224700 1 -- 172.21.15.12:0/3008355714 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf54007c20 con 0x55f96c81dba0 2023-10-05T19:40:28.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.526+0000 7faf69224700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8204c0 con 0x55f96c814e10 2023-10-05T19:40:28.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.530+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c814e10 2023-10-05T19:40:28.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.530+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8204c0 con 0x55f96c7e5010 2023-10-05T19:40:28.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.530+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c7e5010 2023-10-05T19:40:28.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.530+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8204c0 con 0x55f96c814e10 2023-10-05T19:40:28.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.530+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c814e10 2023-10-05T19:40:28.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.530+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8204c0 con 0x55f96c819250 2023-10-05T19:40:28.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.530+0000 7faf69a25700 1 -- 172.21.15.12:0/3008355714 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf5c014070 con 0x55f96c819250 2023-10-05T19:40:28.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.530+0000 7faf69a25700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8204c0 con 0x55f96c81dba0 2023-10-05T19:40:28.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.530+0000 7faf69224700 1 -- 172.21.15.12:0/3008355714 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf54007c20 con 0x55f96c81dba0 2023-10-05T19:40:28.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.530+0000 7faf69224700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8204c0 con 0x55f96c81dba0 2023-10-05T19:40:28.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.534+0000 7faf69224700 1 -- 172.21.15.12:0/3008355714 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf54007c20 con 0x55f96c81dba0 2023-10-05T19:40:28.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.534+0000 7faf69224700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8204c0 con 0x55f96c7e5010 2023-10-05T19:40:28.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.534+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c7e5010 2023-10-05T19:40:28.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.534+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8204c0 con 0x55f96c81dba0 2023-10-05T19:40:28.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.534+0000 7faf69224700 1 -- 172.21.15.12:0/3008355714 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf54007c20 con 0x55f96c81dba0 2023-10-05T19:40:28.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.534+0000 7faf69224700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8204c0 con 0x55f96c81dba0 2023-10-05T19:40:28.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.534+0000 7faf69224700 1 -- 172.21.15.12:0/3008355714 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf54007c20 con 0x55f96c81dba0 2023-10-05T19:40:28.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.534+0000 7faf69224700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8204c0 con 0x55f96c814e10 2023-10-05T19:40:28.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.534+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c814e10 2023-10-05T19:40:28.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.534+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8204c0 con 0x55f96c814e10 2023-10-05T19:40:28.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.538+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c814e10 2023-10-05T19:40:28.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.538+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8204c0 con 0x55f96c7e5010 2023-10-05T19:40:28.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.538+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c7e5010 2023-10-05T19:40:28.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.538+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8204c0 con 0x55f96c81dba0 2023-10-05T19:40:28.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.538+0000 7faf69224700 1 -- 172.21.15.12:0/3008355714 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf54007c20 con 0x55f96c81dba0 2023-10-05T19:40:28.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.538+0000 7faf69224700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8204c0 con 0x55f96c819250 2023-10-05T19:40:28.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.538+0000 7faf69a25700 1 -- 172.21.15.12:0/3008355714 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf5c014070 con 0x55f96c819250 2023-10-05T19:40:28.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.538+0000 7faf69a25700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8204c0 con 0x55f96c814e10 2023-10-05T19:40:28.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.538+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c814e10 2023-10-05T19:40:28.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.542+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8204c0 con 0x55f96c7e5010 2023-10-05T19:40:28.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.542+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c7e5010 2023-10-05T19:40:28.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.542+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8204c0 con 0x55f96c81dba0 2023-10-05T19:40:28.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.542+0000 7faf69224700 1 -- 172.21.15.12:0/3008355714 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf54007c20 con 0x55f96c81dba0 2023-10-05T19:40:28.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.542+0000 7faf69224700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8204c0 con 0x55f96c7e5010 2023-10-05T19:40:28.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.542+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c7e5010 2023-10-05T19:40:28.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.542+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8204c0 con 0x55f96c819250 2023-10-05T19:40:28.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.542+0000 7faf69a25700 1 -- 172.21.15.12:0/3008355714 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf5c014070 con 0x55f96c819250 2023-10-05T19:40:28.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.542+0000 7faf69a25700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8204c0 con 0x55f96c81dba0 2023-10-05T19:40:28.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.546+0000 7faf69224700 1 -- 172.21.15.12:0/3008355714 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf54007c20 con 0x55f96c81dba0 2023-10-05T19:40:28.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.546+0000 7faf69224700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8204c0 con 0x55f96c7e5010 2023-10-05T19:40:28.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.546+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c7e5010 2023-10-05T19:40:28.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.546+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8204c0 con 0x55f96c814e10 2023-10-05T19:40:28.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.546+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c814e10 2023-10-05T19:40:28.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.546+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8204c0 con 0x55f96c819250 2023-10-05T19:40:28.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.546+0000 7faf69a25700 1 -- 172.21.15.12:0/3008355714 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7faf5c014070 con 0x55f96c819250 2023-10-05T19:40:28.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.546+0000 7faf6bef0e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:28.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.546+0000 7faf6bef0e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:28.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.546+0000 7faf6bef0e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:28.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.546+0000 7faf6bef0e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:28.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.546+0000 7faf6bef0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.546+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c81c9d0 con 0x55f96c814e10 2023-10-05T19:40:28.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.550+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c814e10 2023-10-05T19:40:28.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.550+0000 7faf6bef0e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:28.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.550+0000 7faf6bef0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.550+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c81c540 con 0x55f96c814e10 2023-10-05T19:40:28.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.550+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c814e10 2023-10-05T19:40:28.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.550+0000 7faf6bef0e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:28.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.550+0000 7faf6bef0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.550+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c820870 con 0x55f96c81dba0 2023-10-05T19:40:28.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.550+0000 7faf69224700 1 -- 172.21.15.12:0/3008355714 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7faf54007c20 con 0x55f96c81dba0 2023-10-05T19:40:28.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.550+0000 7faf6bef0e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:28.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.550+0000 7faf6bef0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.550+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c820c20 con 0x55f96c81dba0 2023-10-05T19:40:28.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.550+0000 7faf69224700 1 -- 172.21.15.12:0/3008355714 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7faf54007c20 con 0x55f96c81dba0 2023-10-05T19:40:28.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.550+0000 7faf6bef0e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:28.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.550+0000 7faf6bef0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.550+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c820fd0 con 0x55f96c814e10 2023-10-05T19:40:28.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.550+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c814e10 2023-10-05T19:40:28.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.550+0000 7faf6bef0e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:28.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.550+0000 7faf6bef0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.550+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8213b0 con 0x55f96c814e10 2023-10-05T19:40:28.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.550+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c814e10 2023-10-05T19:40:28.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.550+0000 7faf6bef0e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:28.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.550+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c821380 con 0x55f96c81dba0 2023-10-05T19:40:28.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.554+0000 7faf69224700 1 -- 172.21.15.12:0/3008355714 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7faf54007c20 con 0x55f96c81dba0 2023-10-05T19:40:28.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.554+0000 7faf69224700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c821380 con 0x55f96c819250 2023-10-05T19:40:28.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.554+0000 7faf69a25700 1 -- 172.21.15.12:0/3008355714 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf5c014070 con 0x55f96c819250 2023-10-05T19:40:28.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.554+0000 7faf69a25700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c821380 con 0x55f96c7e5010 2023-10-05T19:40:28.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.554+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c7e5010 2023-10-05T19:40:28.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.554+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c820fd0 con 0x55f96c814e10 2023-10-05T19:40:28.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.558+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c814e10 2023-10-05T19:40:28.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.558+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c820fd0 con 0x55f96c814e10 2023-10-05T19:40:28.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.558+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c814e10 2023-10-05T19:40:28.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.558+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c820fd0 con 0x55f96c7e5010 2023-10-05T19:40:28.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.558+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c7e5010 2023-10-05T19:40:28.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.558+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c820fd0 con 0x55f96c7e5010 2023-10-05T19:40:28.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.558+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c7e5010 2023-10-05T19:40:28.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.558+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c820fd0 con 0x55f96c819250 2023-10-05T19:40:28.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.558+0000 7faf69a25700 1 -- 172.21.15.12:0/3008355714 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf5c014070 con 0x55f96c819250 2023-10-05T19:40:28.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.558+0000 7faf69a25700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c820fd0 con 0x55f96c81dba0 2023-10-05T19:40:28.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.562+0000 7faf69224700 1 -- 172.21.15.12:0/3008355714 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf54007c20 con 0x55f96c81dba0 2023-10-05T19:40:28.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.562+0000 7faf69224700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c820fd0 con 0x55f96c814e10 2023-10-05T19:40:28.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.562+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c814e10 2023-10-05T19:40:28.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.562+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c820fd0 con 0x55f96c7e5010 2023-10-05T19:40:28.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.562+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c7e5010 2023-10-05T19:40:28.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.562+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c820fd0 con 0x55f96c814e10 2023-10-05T19:40:28.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.562+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c814e10 2023-10-05T19:40:28.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.562+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c820fd0 con 0x55f96c819250 2023-10-05T19:40:28.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.566+0000 7faf69a25700 1 -- 172.21.15.12:0/3008355714 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf5c014070 con 0x55f96c819250 2023-10-05T19:40:28.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.566+0000 7faf69a25700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c820fd0 con 0x55f96c81dba0 2023-10-05T19:40:28.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.566+0000 7faf69224700 1 -- 172.21.15.12:0/3008355714 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf54007c20 con 0x55f96c81dba0 2023-10-05T19:40:28.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.566+0000 7faf69224700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c820fd0 con 0x55f96c81dba0 2023-10-05T19:40:28.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.566+0000 7faf69224700 1 -- 172.21.15.12:0/3008355714 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf54007c20 con 0x55f96c81dba0 2023-10-05T19:40:28.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.566+0000 7faf69224700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c820fd0 con 0x55f96c7e5010 2023-10-05T19:40:28.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.566+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c7e5010 2023-10-05T19:40:28.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.566+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c820fd0 con 0x55f96c81dba0 2023-10-05T19:40:28.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.566+0000 7faf69224700 1 -- 172.21.15.12:0/3008355714 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf54007c20 con 0x55f96c81dba0 2023-10-05T19:40:28.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.566+0000 7faf69224700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c820fd0 con 0x55f96c81dba0 2023-10-05T19:40:28.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.570+0000 7faf69224700 1 -- 172.21.15.12:0/3008355714 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf54007c20 con 0x55f96c81dba0 2023-10-05T19:40:28.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.570+0000 7faf69224700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c820fd0 con 0x55f96c814e10 2023-10-05T19:40:28.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.570+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c814e10 2023-10-05T19:40:28.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.570+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c820fd0 con 0x55f96c814e10 2023-10-05T19:40:28.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.570+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c814e10 2023-10-05T19:40:28.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.570+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c820fd0 con 0x55f96c7e5010 2023-10-05T19:40:28.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.570+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c7e5010 2023-10-05T19:40:28.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.570+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c820fd0 con 0x55f96c81dba0 2023-10-05T19:40:28.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.570+0000 7faf69224700 1 -- 172.21.15.12:0/3008355714 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf54007c20 con 0x55f96c81dba0 2023-10-05T19:40:28.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.570+0000 7faf69224700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c820fd0 con 0x55f96c819250 2023-10-05T19:40:28.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.574+0000 7faf69a25700 1 -- 172.21.15.12:0/3008355714 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf5c014070 con 0x55f96c819250 2023-10-05T19:40:28.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.574+0000 7faf69a25700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c820fd0 con 0x55f96c814e10 2023-10-05T19:40:28.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.574+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c814e10 2023-10-05T19:40:28.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.574+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c820fd0 con 0x55f96c7e5010 2023-10-05T19:40:28.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.574+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c7e5010 2023-10-05T19:40:28.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.574+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c820fd0 con 0x55f96c81dba0 2023-10-05T19:40:28.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.574+0000 7faf69224700 1 -- 172.21.15.12:0/3008355714 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf54007c20 con 0x55f96c81dba0 2023-10-05T19:40:28.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.574+0000 7faf69224700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c820fd0 con 0x55f96c7e5010 2023-10-05T19:40:28.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.578+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c7e5010 2023-10-05T19:40:28.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.578+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c820fd0 con 0x55f96c819250 2023-10-05T19:40:28.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.578+0000 7faf69a25700 1 -- 172.21.15.12:0/3008355714 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf5c014070 con 0x55f96c819250 2023-10-05T19:40:28.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.578+0000 7faf69a25700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c820fd0 con 0x55f96c81dba0 2023-10-05T19:40:28.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.578+0000 7faf69224700 1 -- 172.21.15.12:0/3008355714 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf54007c20 con 0x55f96c81dba0 2023-10-05T19:40:28.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.578+0000 7faf69224700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c820fd0 con 0x55f96c7e5010 2023-10-05T19:40:28.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.578+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c7e5010 2023-10-05T19:40:28.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.578+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c820fd0 con 0x55f96c814e10 2023-10-05T19:40:28.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.582+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c814e10 2023-10-05T19:40:28.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.582+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c820fd0 con 0x55f96c819250 2023-10-05T19:40:28.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.582+0000 7faf69a25700 1 -- 172.21.15.12:0/3008355714 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7faf5c014070 con 0x55f96c819250 2023-10-05T19:40:28.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.582+0000 7faf6bef0e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:28.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.582+0000 7faf6bef0e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:28.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.582+0000 7faf6bef0e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:28.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.582+0000 7faf6bef0e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:28.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.582+0000 7faf6bef0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.582+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c821380 con 0x55f96c814e10 2023-10-05T19:40:28.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.582+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c814e10 2023-10-05T19:40:28.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.582+0000 7faf6bef0e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:28.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.582+0000 7faf6bef0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.582+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8217e0 con 0x55f96c819250 2023-10-05T19:40:28.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.582+0000 7faf69a25700 1 -- 172.21.15.12:0/3008355714 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7faf5c014070 con 0x55f96c819250 2023-10-05T19:40:28.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.582+0000 7faf6bef0e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:28.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.582+0000 7faf6bef0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.582+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8220d0 con 0x55f96c814e10 2023-10-05T19:40:28.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.582+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c814e10 2023-10-05T19:40:28.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.582+0000 7faf6bef0e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:28.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.582+0000 7faf6bef0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.582+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8220d0 con 0x55f96c81dba0 2023-10-05T19:40:28.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.586+0000 7faf69224700 1 -- 172.21.15.12:0/3008355714 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7faf54007c20 con 0x55f96c81dba0 2023-10-05T19:40:28.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.586+0000 7faf6bef0e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:28.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.586+0000 7faf6bef0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.586+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8228d0 con 0x55f96c7e5010 2023-10-05T19:40:28.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.586+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c7e5010 2023-10-05T19:40:28.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.586+0000 7faf6bef0e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:28.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.586+0000 7faf6bef0e40 20 zone default found 2023-10-05T19:40:28.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.586+0000 7faf6bef0e40 4 Realm: () 2023-10-05T19:40:28.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.586+0000 7faf6bef0e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:28.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.586+0000 7faf6bef0e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:28.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.586+0000 7faf6bef0e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:28.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.586+0000 7faf6bef0e40 20 zonegroup default 2023-10-05T19:40:28.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.586+0000 7faf6bef0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.586+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8228d0 con 0x55f96c7e5010 2023-10-05T19:40:28.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.586+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c7e5010 2023-10-05T19:40:28.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.586+0000 7faf6bef0e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:28.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.586+0000 7faf6bef0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.586+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c824250 con 0x55f96c81dba0 2023-10-05T19:40:28.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.586+0000 7faf69224700 1 -- 172.21.15.12:0/3008355714 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7faf54007c20 con 0x55f96c81dba0 2023-10-05T19:40:28.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.586+0000 7faf6bef0e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:28.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.586+0000 7faf6bef0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.586+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c824600 con 0x55f96c81dba0 2023-10-05T19:40:28.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.586+0000 7faf69224700 1 -- 172.21.15.12:0/3008355714 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7faf54007c20 con 0x55f96c81dba0 2023-10-05T19:40:28.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.586+0000 7faf6bef0e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:28.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.586+0000 7faf6bef0e40 20 started sync module instance, tier type = 2023-10-05T19:40:28.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.586+0000 7faf6bef0e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:28.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.586+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c823910 con 0x55f96c819250 2023-10-05T19:40:28.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.590+0000 7faf69a25700 1 -- 172.21.15.12:0/3008355714 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'147 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faf5c014070 con 0x55f96c819250 2023-10-05T19:40:28.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.590+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94529755683808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8249b0 con 0x55f96c819250 2023-10-05T19:40:28.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.590+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c825250 con 0x55f96c814e10 2023-10-05T19:40:28.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.594+0000 7faf69a25700 1 -- 172.21.15.12:0/3008355714 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94529755683808] v28'148 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faf5c014070 con 0x55f96c819250 2023-10-05T19:40:28.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.594+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'147 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c814e10 2023-10-05T19:40:28.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.594+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94529755696480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c823e30 con 0x55f96c814e10 2023-10-05T19:40:28.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.594+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c825d60 con 0x55f96c81dba0 2023-10-05T19:40:28.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.598+0000 7faf69224700 1 -- 172.21.15.12:0/3008355714 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'147 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faf54007c20 con 0x55f96c81dba0 2023-10-05T19:40:28.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.598+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94529755699312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c826600 con 0x55f96c81dba0 2023-10-05T19:40:28.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.598+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c827010 con 0x55f96c819250 2023-10-05T19:40:28.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.598+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94529755696480] v28'148 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c814e10 2023-10-05T19:40:28.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.598+0000 7faf69224700 1 -- 172.21.15.12:0/3008355714 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94529755699312] v28'148 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faf54007c20 con 0x55f96c81dba0 2023-10-05T19:40:28.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.598+0000 7faf69a25700 1 -- 172.21.15.12:0/3008355714 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'147 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faf5c014070 con 0x55f96c819250 2023-10-05T19:40:28.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.598+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94529755704096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c826230 con 0x55f96c819250 2023-10-05T19:40:28.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.598+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c827ad0 con 0x55f96c814e10 2023-10-05T19:40:28.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.602+0000 7faf69a25700 1 -- 172.21.15.12:0/3008355714 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94529755704096] v28'148 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faf5c014070 con 0x55f96c819250 2023-10-05T19:40:28.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.602+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'147 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c814e10 2023-10-05T19:40:28.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.602+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94529755697584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c827ad0 con 0x55f96c814e10 2023-10-05T19:40:28.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.602+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c828530 con 0x55f96c7e5010 2023-10-05T19:40:28.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.606+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94529755697584] v28'148 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c814e10 2023-10-05T19:40:28.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.606+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'147 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c7e5010 2023-10-05T19:40:28.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.606+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94529755710064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c827ad0 con 0x55f96c7e5010 2023-10-05T19:40:28.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.606+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c828f90 con 0x55f96c7e5010 2023-10-05T19:40:28.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.610+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(95 notify.5 [watch watch cookie 94529755710064] v28'148 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c7e5010 2023-10-05T19:40:28.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.610+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(96 notify.6 [create] v28'147 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c7e5010 2023-10-05T19:40:28.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.610+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94529755707408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c828f90 con 0x55f96c7e5010 2023-10-05T19:40:28.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.610+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8299a0 con 0x55f96c7e5010 2023-10-05T19:40:28.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.610+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(98 notify.7 [create] v28'147 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c7e5010 2023-10-05T19:40:28.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.610+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94529755714736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c829df0 con 0x55f96c7e5010 2023-10-05T19:40:28.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.610+0000 7faf6bef0e40 20 add_watcher() i=0 2023-10-05T19:40:28.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.610+0000 7faf6bef0e40 20 add_watcher() i=1 2023-10-05T19:40:28.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.610+0000 7faf6bef0e40 20 add_watcher() i=2 2023-10-05T19:40:28.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.610+0000 7faf6bef0e40 20 add_watcher() i=3 2023-10-05T19:40:28.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.610+0000 7faf6bef0e40 20 add_watcher() i=4 2023-10-05T19:40:28.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.610+0000 7faf6bef0e40 20 add_watcher() i=5 2023-10-05T19:40:28.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.610+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(97 notify.6 [watch watch cookie 94529755707408] v28'148 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c7e5010 2023-10-05T19:40:28.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.610+0000 7faf6bef0e40 20 add_watcher() i=6 2023-10-05T19:40:28.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.614+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94529755714736] v28'148 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c7e5010 2023-10-05T19:40:28.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.614+0000 7faf6bef0e40 20 add_watcher() i=7 2023-10-05T19:40:28.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.614+0000 7faf6bef0e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:28.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.614+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c82ce20 con 0x55f96c814e10 2023-10-05T19:40:28.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.614+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c814e10 2023-10-05T19:40:28.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.614+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94529755715840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c82a240 con 0x55f96c814e10 2023-10-05T19:40:28.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.618+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94529755715840] v28'145 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c814e10 2023-10-05T19:40:28.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.618+0000 7faf3afe5700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:28.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.618+0000 7faf6bef0e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:28.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.618+0000 7faf6bef0e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:28.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.618+0000 7faf6bef0e40 1 Processor -- start 2023-10-05T19:40:28.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.618+0000 7faf6bef0e40 1 -- start start 2023-10-05T19:40:28.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.618+0000 7faf6bef0e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f96c836f10 0x55f96c8372e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:28.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.618+0000 7faf6bef0e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f96c837820 con 0x55f96c836f10 2023-10-05T19:40:28.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.618+0000 7faf69224700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f96c836f10 0x55f96c8372e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:28.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.618+0000 7faf69224700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f96c836f10 0x55f96c8372e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58432/0 (socket says 172.21.15.12:58432) 2023-10-05T19:40:28.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.618+0000 7faf69224700 1 -- 172.21.15.12:0/92403196 learned_addr learned my addr 172.21.15.12:0/92403196 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:28.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.618+0000 7faf69224700 1 -- 172.21.15.12:0/92403196 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f96c837960 con 0x55f96c836f10 2023-10-05T19:40:28.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.622+0000 7faf69224700 1 --2- 172.21.15.12:0/92403196 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f96c836f10 0x55f96c8372e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7faf54002000 tx=0x7faf54011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dcb3d91dd4244e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:28.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.622+0000 7faf3a7e4700 1 -- 172.21.15.12:0/92403196 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf54031040 con 0x55f96c836f10 2023-10-05T19:40:28.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.622+0000 7faf3a7e4700 1 -- 172.21.15.12:0/92403196 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf54041830 con 0x55f96c836f10 2023-10-05T19:40:28.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.622+0000 7faf3a7e4700 1 -- 172.21.15.12:0/92403196 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf54025040 con 0x55f96c836f10 2023-10-05T19:40:28.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.622+0000 7faf6bef0e40 1 -- 172.21.15.12:0/92403196 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f96c836f10 msgr2=0x55f96c8372e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.622+0000 7faf6bef0e40 1 --2- 172.21.15.12:0/92403196 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f96c836f10 0x55f96c8372e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7faf54002000 tx=0x7faf54011000 comp rx=0 tx=0).stop 2023-10-05T19:40:28.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.622+0000 7faf6bef0e40 1 -- 172.21.15.12:0/92403196 shutdown_connections 2023-10-05T19:40:28.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.622+0000 7faf6bef0e40 1 --2- 172.21.15.12:0/92403196 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f96c836f10 0x55f96c8372e0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.622+0000 7faf6bef0e40 1 -- 172.21.15.12:0/92403196 >> 172.21.15.12:0/92403196 conn(0x55f96c8322d0 msgr2=0x55f96c830480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:28.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.622+0000 7faf6bef0e40 1 -- 172.21.15.12:0/92403196 shutdown_connections 2023-10-05T19:40:28.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.622+0000 7faf6bef0e40 1 -- 172.21.15.12:0/92403196 wait complete. 2023-10-05T19:40:28.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.622+0000 7faf6bef0e40 1 Processor -- start 2023-10-05T19:40:28.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.622+0000 7faf6bef0e40 1 -- start start 2023-10-05T19:40:28.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.622+0000 7faf6bef0e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f96c836f10 0x55f96c83ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:28.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.622+0000 7faf6bef0e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f96c83da20 con 0x55f96c836f10 2023-10-05T19:40:28.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.622+0000 7faf69224700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f96c836f10 0x55f96c83ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:28.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.622+0000 7faf69224700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f96c836f10 0x55f96c83ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58442/0 (socket says 172.21.15.12:58442) 2023-10-05T19:40:28.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.622+0000 7faf69224700 1 -- 172.21.15.12:0/4047969973 learned_addr learned my addr 172.21.15.12:0/4047969973 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:28.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.626+0000 7faf69224700 1 -- 172.21.15.12:0/4047969973 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f96c7e5c20 con 0x55f96c836f10 2023-10-05T19:40:28.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.626+0000 7faf69224700 1 --2- 172.21.15.12:0/4047969973 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f96c836f10 0x55f96c83ca20 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7faf5400c010 tx=0x7faf54044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:28.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.626+0000 7faf38fe1700 1 -- 172.21.15.12:0/4047969973 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf54025070 con 0x55f96c836f10 2023-10-05T19:40:28.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.626+0000 7faf38fe1700 1 -- 172.21.15.12:0/4047969973 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf5402ce40 con 0x55f96c836f10 2023-10-05T19:40:28.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.626+0000 7faf38fe1700 1 -- 172.21.15.12:0/4047969973 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf54031040 con 0x55f96c836f10 2023-10-05T19:40:28.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.626+0000 7faf6bef0e40 1 -- 172.21.15.12:0/4047969973 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f96c83d020 con 0x55f96c836f10 2023-10-05T19:40:28.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.626+0000 7faf6bef0e40 1 -- 172.21.15.12:0/4047969973 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f96c83d4c0 con 0x55f96c836f10 2023-10-05T19:40:28.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.626+0000 7faf38fe1700 1 -- 172.21.15.12:0/4047969973 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7faf54034040 con 0x55f96c836f10 2023-10-05T19:40:28.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.626+0000 7faf38fe1700 1 --2- 172.21.15.12:0/4047969973 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7faeb0030bb0 0x7faeb0033060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:28.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.626+0000 7faf38fe1700 1 -- 172.21.15.12:0/4047969973 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7faf54053ab0 con 0x55f96c836f10 2023-10-05T19:40:28.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.626+0000 7faf6bef0e40 5 note: GC not initialized 2023-10-05T19:40:28.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.626+0000 7faf6a226700 1 --2- 172.21.15.12:0/4047969973 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7faeb0030bb0 0x7faeb0033060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:28.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.630+0000 7faf6a226700 1 --2- 172.21.15.12:0/4047969973 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7faeb0030bb0 0x7faeb0033060 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x55f96c828d10 tx=0x7faf640254a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:28.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.630+0000 7faea9ffb700 20 reqs_thread_entry: start 2023-10-05T19:40:28.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.630+0000 7fae9b7fe700 20 reqs_thread_entry: start 2023-10-05T19:40:28.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.630+0000 7faf6bef0e40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:28.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.630+0000 7faf6bef0e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:28.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.630+0000 7fae99ffb700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:59 2023 2023-10-05T19:40:28.641 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:28.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.630+0000 7fae997fa700 20 reqs_thread_entry: start 2023-10-05T19:40:28.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.630+0000 7faf6bef0e40 20 get_system_obj_state: rctx=0x7fff8cd4f810 obj=default.rgw.meta:root:myfoo state=0x55f96c6bc0b0 s->prefetch_data=0 2023-10-05T19:40:28.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.630+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:102 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8a5bb0 con 0x55f96c819250 2023-10-05T19:40:28.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.634+0000 7faf69a25700 1 -- 172.21.15.12:0/3008355714 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(102 myfoo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv27 ondisk = 0) v8 ==== 233+0+139 (crc 0 0 0) 0x7faf5c014070 con 0x55f96c819250 2023-10-05T19:40:28.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.634+0000 7faf6bef0e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:28.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.634+0000 7faf6bef0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.634+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:103 8.9 8:91b71841:root::myfoo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8a8150 con 0x55f96c819250 2023-10-05T19:40:28.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.634+0000 7faf69a25700 1 -- 172.21.15.12:0/3008355714 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(103 myfoo [call,call out=48b,read 0~165 out=165b] v0'0 uv27 ondisk = 0) v8 ==== 233+0+213 (crc 0 0 0) 0x7faf5c014070 con 0x55f96c819250 2023-10-05T19:40:28.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.634+0000 7faf6bef0e40 20 rados_obj.operate() r=0 bl.length=165 2023-10-05T19:40:28.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.634+0000 7faf6bef0e40 20 get_system_obj_state: rctx=0x7fff8cd4f810 obj=default.rgw.meta:root:.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 state=0x55f96c6922b0 s->prefetch_data=0 2023-10-05T19:40:28.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.634+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:104 8.e 8:70176bbd:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8a6620 con 0x55f96c7e5010 2023-10-05T19:40:28.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.634+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(104 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 [call out=48b,getxattrs out=218b,stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 290+0+282 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c7e5010 2023-10-05T19:40:28.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.634+0000 7faf6bef0e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:28.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.634+0000 7faf6bef0e40 20 Read xattr: user.rgw.acl 2023-10-05T19:40:28.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.634+0000 7faf6bef0e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:28.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.634+0000 7faf6bef0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.634+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:105 8.e 8:70176bbd:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8a6130 con 0x55f96c7e5010 2023-10-05T19:40:28.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.634+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(105 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 [call,call out=48b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv7 ondisk = 0) v8 ==== 332+0+618 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c7e5010 2023-10-05T19:40:28.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.634+0000 7faf6bef0e40 20 rados_obj.operate() r=0 bl.length=352 2023-10-05T19:40:28.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.634+0000 7faf6bef0e40 20 couldn't put binfo cache entry, might have raced with data changes 2023-10-05T19:40:28.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.634+0000 7faf6bef0e40 20 get_system_obj_state: rctx=0x7fff8cd4f770 obj=default.rgw.meta:root:myfoo state=0x55f96c6922b0 s->prefetch_data=0 2023-10-05T19:40:28.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.634+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:106 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8a6130 con 0x55f96c819250 2023-10-05T19:40:28.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.634+0000 7faf69a25700 1 -- 172.21.15.12:0/3008355714 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(106 myfoo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv27 ondisk = 0) v8 ==== 233+0+139 (crc 0 0 0) 0x7faf5c014070 con 0x55f96c819250 2023-10-05T19:40:28.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.638+0000 7faf6bef0e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:28.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.638+0000 7faf6bef0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.638+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:107 8.9 8:91b71841:root::myfoo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8a7510 con 0x55f96c819250 2023-10-05T19:40:28.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.638+0000 7faf69a25700 1 -- 172.21.15.12:0/3008355714 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(107 myfoo [call,call out=48b,read 0~165 out=165b] v0'0 uv27 ondisk = 0) v8 ==== 233+0+213 (crc 0 0 0) 0x7faf5c014070 con 0x55f96c819250 2023-10-05T19:40:28.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.638+0000 7faf6bef0e40 20 rados_obj.operate() r=0 bl.length=165 2023-10-05T19:40:28.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.638+0000 7faf6bef0e40 20 get_system_obj_state: rctx=0x7fff8cd4f770 obj=default.rgw.meta:root:.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 state=0x55f96c6bc0b0 s->prefetch_data=0 2023-10-05T19:40:28.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.638+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:108 8.e 8:70176bbd:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8a4d00 con 0x55f96c7e5010 2023-10-05T19:40:28.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.638+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(108 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 [call out=48b,getxattrs out=218b,stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 290+0+282 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c7e5010 2023-10-05T19:40:28.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.638+0000 7faf6bef0e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:28.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.638+0000 7faf6bef0e40 20 Read xattr: user.rgw.acl 2023-10-05T19:40:28.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.638+0000 7faf6bef0e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:28.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.638+0000 7faf6bef0e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.638+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:109 8.e 8:70176bbd:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8a7fe0 con 0x55f96c7e5010 2023-10-05T19:40:28.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.638+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.0 v2:172.21.15.8:6800/27563 30 ==== osd_op_reply(109 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 [call,call out=48b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv7 ondisk = 0) v8 ==== 332+0+618 (crc 0 0 0) 0x7faf6400a5a0 con 0x55f96c7e5010 2023-10-05T19:40:28.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.638+0000 7faf6bef0e40 20 rados_obj.operate() r=0 bl.length=352 2023-10-05T19:40:28.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.638+0000 7faf6bef0e40 20 couldn't put binfo cache entry, might have raced with data changes 2023-10-05T19:40:28.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.638+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:110 4.3 4:c350083b:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8aa6f0 con 0x55f96c814e10 2023-10-05T19:40:28.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.638+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:111 4.1c 4:38d73088:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8ab160 con 0x55f96c7e5010 2023-10-05T19:40:28.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.638+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:112 4.a 4:502f3015:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8affa0 con 0x55f96c819250 2023-10-05T19:40:28.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.638+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:113 4.3e 4:7e570c43:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8b2860 con 0x55f96c819250 2023-10-05T19:40:28.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.638+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:114 4.7 4:e38d8107:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8b5330 con 0x55f96c7e5010 2023-10-05T19:40:28.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.638+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:115 4.1a 4:5add8e75:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8b7c70 con 0x55f96c819250 2023-10-05T19:40:28.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.638+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:116 4.2c 4:358e5801:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8ba640 con 0x55f96c814e10 2023-10-05T19:40:28.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.638+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:117 4.2 4:4101cfe2:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8bcf00 con 0x55f96c814e10 2023-10-05T19:40:28.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.638+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:118 4.30 4:0e5788c0:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8bfd90 con 0x55f96c81dba0 2023-10-05T19:40:28.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.638+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:119 4.19 4:99ccd5a2:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8c2c80 con 0x55f96c81dba0 2023-10-05T19:40:28.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.638+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:120 4.2d 4:b73f2ca3:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8c5b70 con 0x55f96c7e5010 2023-10-05T19:40:28.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.638+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.0 v2:172.21.15.8:6800/27563 31 ==== osd_op_reply(111 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7faf6404a040 con 0x55f96c7e5010 2023-10-05T19:40:28.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.642+0000 7faf69224700 1 -- 172.21.15.12:0/3008355714 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(119 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.9 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7faf5404e070 con 0x55f96c81dba0 2023-10-05T19:40:28.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.642+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(110 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7faf6404a040 con 0x55f96c814e10 2023-10-05T19:40:28.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.642+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(116 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.6 [call out=124b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x7faf6404a040 con 0x55f96c814e10 2023-10-05T19:40:28.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.642+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.0 v2:172.21.15.8:6800/27563 32 ==== osd_op_reply(114 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7faf6404a040 con 0x55f96c7e5010 2023-10-05T19:40:28.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.642+0000 7faf69a25700 1 -- 172.21.15.12:0/3008355714 <== osd.2 v2:172.21.15.12:6800/29934 23 ==== osd_op_reply(115 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.5 [call out=124b] v0'0 uv7 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x7faf5c054040 con 0x55f96c819250 2023-10-05T19:40:28.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.642+0000 7faf69224700 1 -- 172.21.15.12:0/3008355714 <== osd.3 v2:172.21.15.12:6816/29951 27 ==== osd_op_reply(118 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.8 [call out=124b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x7faf5404e070 con 0x55f96c81dba0 2023-10-05T19:40:28.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.642+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.0 v2:172.21.15.8:6800/27563 33 ==== osd_op_reply(120 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7faf6404a040 con 0x55f96c7e5010 2023-10-05T19:40:28.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.642+0000 7faf69a25700 1 -- 172.21.15.12:0/3008355714 <== osd.2 v2:172.21.15.12:6800/29934 24 ==== osd_op_reply(113 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.3 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7faf5c054040 con 0x55f96c819250 2023-10-05T19:40:28.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.642+0000 7faf69a25700 1 -- 172.21.15.12:0/3008355714 <== osd.2 v2:172.21.15.12:6800/29934 25 ==== osd_op_reply(112 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.2 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7faf5c054040 con 0x55f96c819250 2023-10-05T19:40:28.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.642+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(117 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1.7 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7faf6404a040 con 0x55f96c814e10 2023-10-05T19:40:28.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.642+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:121 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94529755683808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8c2c80 con 0x55f96c819250 2023-10-05T19:40:28.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.642+0000 7faf69a25700 1 -- 172.21.15.12:0/3008355714 <== osd.2 v2:172.21.15.12:6800/29934 26 ==== osd_op_reply(121 notify.0 [watch unwatch cookie 94529755683808] v28'149 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faf5c054040 con 0x55f96c819250 2023-10-05T19:40:28.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.646+0000 7faf6bef0e40 20 remove_watcher() i=0 2023-10-05T19:40:28.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.646+0000 7faf6bef0e40 2 removed watcher, disabling cache 2023-10-05T19:40:28.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.646+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:122 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94529755696480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8c2c80 con 0x55f96c814e10 2023-10-05T19:40:28.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.646+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(122 notify.1 [watch unwatch cookie 94529755696480] v28'149 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faf6404a040 con 0x55f96c814e10 2023-10-05T19:40:28.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.646+0000 7faf6bef0e40 20 remove_watcher() i=1 2023-10-05T19:40:28.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.646+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:123 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94529755699312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8c2c80 con 0x55f96c81dba0 2023-10-05T19:40:28.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.650+0000 7faf69224700 1 -- 172.21.15.12:0/3008355714 <== osd.3 v2:172.21.15.12:6816/29951 28 ==== osd_op_reply(123 notify.2 [watch unwatch cookie 94529755699312] v28'149 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faf5404e070 con 0x55f96c81dba0 2023-10-05T19:40:28.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.650+0000 7faf6bef0e40 20 remove_watcher() i=2 2023-10-05T19:40:28.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.650+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:124 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94529755704096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8c2c80 con 0x55f96c819250 2023-10-05T19:40:28.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.654+0000 7faf69a25700 1 -- 172.21.15.12:0/3008355714 <== osd.2 v2:172.21.15.12:6800/29934 27 ==== osd_op_reply(124 notify.3 [watch unwatch cookie 94529755704096] v28'149 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faf5c054040 con 0x55f96c819250 2023-10-05T19:40:28.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.654+0000 7faf6bef0e40 20 remove_watcher() i=3 2023-10-05T19:40:28.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.654+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:125 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94529755697584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8c2c80 con 0x55f96c814e10 2023-10-05T19:40:28.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.654+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(125 notify.4 [watch unwatch cookie 94529755697584] v28'149 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faf6404a040 con 0x55f96c814e10 2023-10-05T19:40:28.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.654+0000 7faf6bef0e40 20 remove_watcher() i=4 2023-10-05T19:40:28.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.654+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:126 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94529755710064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8c2c80 con 0x55f96c7e5010 2023-10-05T19:40:28.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.658+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.0 v2:172.21.15.8:6800/27563 34 ==== osd_op_reply(126 notify.5 [watch unwatch cookie 94529755710064] v28'149 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faf6404a040 con 0x55f96c7e5010 2023-10-05T19:40:28.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.658+0000 7faf6bef0e40 20 remove_watcher() i=5 2023-10-05T19:40:28.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.658+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:127 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94529755707408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8c2c80 con 0x55f96c7e5010 2023-10-05T19:40:28.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.662+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.0 v2:172.21.15.8:6800/27563 35 ==== osd_op_reply(127 notify.6 [watch unwatch cookie 94529755707408] v28'149 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faf6404a040 con 0x55f96c7e5010 2023-10-05T19:40:28.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.662+0000 7faf6bef0e40 20 remove_watcher() i=6 2023-10-05T19:40:28.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.662+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:128 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94529755714736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8c2c80 con 0x55f96c7e5010 2023-10-05T19:40:28.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.662+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.0 v2:172.21.15.8:6800/27563 36 ==== osd_op_reply(128 notify.7 [watch unwatch cookie 94529755714736] v28'149 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faf6404a040 con 0x55f96c7e5010 2023-10-05T19:40:28.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.662+0000 7faf6bef0e40 20 remove_watcher() i=7 2023-10-05T19:40:28.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.666+0000 7faf3afe5700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:28.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.666+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:129 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94529755715840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f96c8c2c80 con 0x55f96c814e10 2023-10-05T19:40:28.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.670+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== osd_op_reply(129 data_loggenerations_metadata [watch unwatch cookie 94529755715840] v28'146 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7faf6404a040 con 0x55f96c814e10 2023-10-05T19:40:28.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.670+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55f96c7e5010 msgr2=0x7faf5c0428d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.670+0000 7faf6bef0e40 1 --2- 172.21.15.12:0/3008355714 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55f96c7e5010 0x7faf5c0428d0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.670+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55f96c814e10 msgr2=0x55f96c8171f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.670+0000 7faf6bef0e40 1 --2- 172.21.15.12:0/3008355714 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55f96c814e10 0x55f96c8171f0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.670+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55f96c819250 msgr2=0x55f96c81b630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.670+0000 7faf6bef0e40 1 --2- 172.21.15.12:0/3008355714 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55f96c819250 0x55f96c81b630 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.670+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55f96c81dba0 msgr2=0x55f96c81ff80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.670+0000 7faf6bef0e40 1 --2- 172.21.15.12:0/3008355714 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55f96c81dba0 0x55f96c81ff80 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.670+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7faf4c030ca0 msgr2=0x7faf4c033150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.670+0000 7faf6bef0e40 1 --2- 172.21.15.12:0/3008355714 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7faf4c030ca0 0x7faf4c033150 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x55f96c7e6060 tx=0x7faf54014000 comp rx=0 tx=0).stop 2023-10-05T19:40:28.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.670+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f96c80afa0 msgr2=0x55f96c80b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.670+0000 7faf6bef0e40 1 --2- 172.21.15.12:0/3008355714 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f96c80afa0 0x55f96c80b370 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x55f96c800720 tx=0x7faf5c014000 comp rx=0 tx=0).stop 2023-10-05T19:40:28.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.670+0000 7faf6a226700 1 -- 172.21.15.12:0/3008355714 reap_dead start 2023-10-05T19:40:28.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.670+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 shutdown_connections 2023-10-05T19:40:28.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.670+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 >> 172.21.15.12:0/3008355714 conn(0x55f96c758860 msgr2=0x55f96c7fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:28.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.670+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 shutdown_connections 2023-10-05T19:40:28.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.670+0000 7faf6bef0e40 1 -- 172.21.15.12:0/3008355714 wait complete. 2023-10-05T19:40:28.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.670+0000 7faf6bef0e40 1 -- 172.21.15.12:0/4047969973 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7faeb0030bb0 msgr2=0x7faeb0033060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.670+0000 7faf6bef0e40 1 --2- 172.21.15.12:0/4047969973 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7faeb0030bb0 0x7faeb0033060 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x55f96c828d10 tx=0x7faf640254a0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.670+0000 7faf6bef0e40 1 -- 172.21.15.12:0/4047969973 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f96c836f10 msgr2=0x55f96c83ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.670+0000 7faf6bef0e40 1 --2- 172.21.15.12:0/4047969973 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f96c836f10 0x55f96c83ca20 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7faf5400c010 tx=0x7faf54044000 comp rx=0 tx=0).stop 2023-10-05T19:40:28.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.670+0000 7faf6bef0e40 1 -- 172.21.15.12:0/4047969973 shutdown_connections 2023-10-05T19:40:28.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.670+0000 7faf6bef0e40 1 --2- 172.21.15.12:0/4047969973 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7faeb0030bb0 0x7faeb0033060 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.670+0000 7faf6bef0e40 1 --2- 172.21.15.12:0/4047969973 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f96c836f10 0x55f96c83ca20 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.670+0000 7faf6bef0e40 1 -- 172.21.15.12:0/4047969973 >> 172.21.15.12:0/4047969973 conn(0x55f96c8322d0 msgr2=0x55f96c8303d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:28.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.670+0000 7faf6bef0e40 1 -- 172.21.15.12:0/4047969973 shutdown_connections 2023-10-05T19:40:28.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.670+0000 7faf6bef0e40 1 -- 172.21.15.12:0/4047969973 wait complete. 2023-10-05T19:40:28.686 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:40:28.686 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo", 2023-10-05T19:40:28.686 INFO:teuthology.orchestra.run.smithi012.stdout: "num_shards": 11, 2023-10-05T19:40:28.686 INFO:teuthology.orchestra.run.smithi012.stdout: "tenant": "", 2023-10-05T19:40:28.686 INFO:teuthology.orchestra.run.smithi012.stdout: "zonegroup": "1602ab00-ae0f-4921-ad63-b9cfe7bc77db", 2023-10-05T19:40:28.686 INFO:teuthology.orchestra.run.smithi012.stdout: "placement_rule": "default-placement", 2023-10-05T19:40:28.686 INFO:teuthology.orchestra.run.smithi012.stdout: "explicit_placement": { 2023-10-05T19:40:28.686 INFO:teuthology.orchestra.run.smithi012.stdout: "data_pool": "", 2023-10-05T19:40:28.686 INFO:teuthology.orchestra.run.smithi012.stdout: "data_extra_pool": "", 2023-10-05T19:40:28.686 INFO:teuthology.orchestra.run.smithi012.stdout: "index_pool": "" 2023-10-05T19:40:28.686 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:28.687 INFO:teuthology.orchestra.run.smithi012.stdout: "id": "93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1", 2023-10-05T19:40:28.687 INFO:teuthology.orchestra.run.smithi012.stdout: "marker": "93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1", 2023-10-05T19:40:28.687 INFO:teuthology.orchestra.run.smithi012.stdout: "index_type": "Normal", 2023-10-05T19:40:28.687 INFO:teuthology.orchestra.run.smithi012.stdout: "owner": "foo", 2023-10-05T19:40:28.687 INFO:teuthology.orchestra.run.smithi012.stdout: "ver": "0#1,1#1,2#1,3#1,4#1,5#3,6#3,7#1,8#3,9#1,10#1", 2023-10-05T19:40:28.687 INFO:teuthology.orchestra.run.smithi012.stdout: "master_ver": "0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0", 2023-10-05T19:40:28.687 INFO:teuthology.orchestra.run.smithi012.stdout: "mtime": "2023-10-05T19:40:27.659036Z", 2023-10-05T19:40:28.687 INFO:teuthology.orchestra.run.smithi012.stdout: "creation_time": "2023-10-05T19:40:22.857926Z", 2023-10-05T19:40:28.687 INFO:teuthology.orchestra.run.smithi012.stdout: "max_marker": "0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#", 2023-10-05T19:40:28.687 INFO:teuthology.orchestra.run.smithi012.stdout: "usage": { 2023-10-05T19:40:28.687 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw.main": { 2023-10-05T19:40:28.687 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 0, 2023-10-05T19:40:28.688 INFO:teuthology.orchestra.run.smithi012.stdout: "size_actual": 0, 2023-10-05T19:40:28.688 INFO:teuthology.orchestra.run.smithi012.stdout: "size_utilized": 0, 2023-10-05T19:40:28.688 INFO:teuthology.orchestra.run.smithi012.stdout: "size_kb": 0, 2023-10-05T19:40:28.688 INFO:teuthology.orchestra.run.smithi012.stdout: "size_kb_actual": 0, 2023-10-05T19:40:28.688 INFO:teuthology.orchestra.run.smithi012.stdout: "size_kb_utilized": 0, 2023-10-05T19:40:28.688 INFO:teuthology.orchestra.run.smithi012.stdout: "num_objects": 0 2023-10-05T19:40:28.688 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:28.688 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:28.688 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket_quota": { 2023-10-05T19:40:28.688 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:28.688 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:28.688 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:28.689 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:28.689 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:28.689 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:28.689 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:40:28.690 DEBUG:tasks.util.rgw: json result: {'bucket': 'myfoo', 'num_shards': 11, 'tenant': '', 'zonegroup': '1602ab00-ae0f-4921-ad63-b9cfe7bc77db', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1', 'marker': '93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1', 'index_type': 'Normal', 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#3,6#3,7#1,8#3,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2023-10-05T19:40:27.659036Z', 'creation_time': '2023-10-05T19:40:22.857926Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {'rgw.main': {'size': 0, 'size_actual': 0, 'size_utilized': 0, 'size_kb': 0, 'size_kb_actual': 0, 'size_kb_utilized': 0, 'num_objects': 0}}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}} 2023-10-05T19:40:28.690 INFO:tasks.util.rgw:rgwadmin: client.0 : ['log', 'list'] 2023-10-05T19:40:28.690 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'log', 'list'] 2023-10-05T19:40:28.690 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph log list 2023-10-05T19:40:28.772 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:28.772 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:28.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.762+0000 7f2a8af9ee40 1 Processor -- start 2023-10-05T19:40:28.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.762+0000 7f2a8af9ee40 1 -- start start 2023-10-05T19:40:28.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.766+0000 7f2a8af9ee40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x561ed0c408e0 0x561ed0cd4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:28.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.766+0000 7f2a8af9ee40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x561ed0c18c00 con 0x561ed0c41bf0 2023-10-05T19:40:28.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.766+0000 7f2a8af9ee40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x561ed0bd5ee0 con 0x561ed0c408e0 2023-10-05T19:40:28.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.766+0000 7f2a882d2700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x561ed0c408e0 0x561ed0cd4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:28.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.766+0000 7f2a88ad3700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x561ed0c41bf0 0x561ed0c401d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:35656/0 (socket says 172.21.15.12:35656) 2023-10-05T19:40:28.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.766+0000 7f2a88ad3700 1 -- 172.21.15.12:0/2755539872 learned_addr learned my addr 172.21.15.12:0/2755539872 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:28.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.766+0000 7f2a882d2700 1 -- 172.21.15.12:0/2755539872 >> v1:172.21.15.8:6789/0 conn(0x561ed0c41bf0 legacy=0x561ed0c401d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.766+0000 7f2a882d2700 1 -- 172.21.15.12:0/2755539872 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ed0c42080 con 0x561ed0c408e0 2023-10-05T19:40:28.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.766+0000 7f2a882d2700 1 --2- 172.21.15.12:0/2755539872 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561ed0c408e0 0x561ed0cd4f30 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00a5d0 tx=0x7f2a7c005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98bdf5d6dbb87d5e server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:28.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.766+0000 7f2a87ad1700 1 -- 172.21.15.12:0/2755539872 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2a7c0086a0 con 0x561ed0c408e0 2023-10-05T19:40:28.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.766+0000 7f2a87ad1700 1 -- 172.21.15.12:0/2755539872 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7c008800 con 0x561ed0c408e0 2023-10-05T19:40:28.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.766+0000 7f2a87ad1700 1 -- 172.21.15.12:0/2755539872 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2a7c008ad0 con 0x561ed0c408e0 2023-10-05T19:40:28.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.766+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/2755539872 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561ed0c408e0 msgr2=0x561ed0cd4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.766+0000 7f2a8af9ee40 1 --2- 172.21.15.12:0/2755539872 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561ed0c408e0 0x561ed0cd4f30 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00a5d0 tx=0x7f2a7c005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:28.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.766+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/2755539872 shutdown_connections 2023-10-05T19:40:28.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.766+0000 7f2a8af9ee40 1 --2- 172.21.15.12:0/2755539872 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561ed0c408e0 0x561ed0cd4f30 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.766+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/2755539872 >> 172.21.15.12:0/2755539872 conn(0x561ed0cc7820 msgr2=0x561ed0cc9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:28.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.766+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/2755539872 shutdown_connections 2023-10-05T19:40:28.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.766+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/2755539872 wait complete. 2023-10-05T19:40:28.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.770+0000 7f2a8af9ee40 1 Processor -- start 2023-10-05T19:40:28.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.770+0000 7f2a8af9ee40 1 -- start start 2023-10-05T19:40:28.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.770+0000 7f2a8af9ee40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561ed0c408e0 0x561ed0ced0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:28.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.770+0000 7f2a8af9ee40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ed0b99e30 con 0x561ed0c408e0 2023-10-05T19:40:28.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.770+0000 7f2a88ad3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561ed0c408e0 0x561ed0ced0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:28.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.770+0000 7f2a88ad3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561ed0c408e0 0x561ed0ced0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58468/0 (socket says 172.21.15.12:58468) 2023-10-05T19:40:28.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.770+0000 7f2a88ad3700 1 -- 172.21.15.12:0/255524594 learned_addr learned my addr 172.21.15.12:0/255524594 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:28.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.770+0000 7f2a88ad3700 1 -- 172.21.15.12:0/255524594 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a7c005cf0 con 0x561ed0c408e0 2023-10-05T19:40:28.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.770+0000 7f2a88ad3700 1 --2- 172.21.15.12:0/255524594 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561ed0c408e0 0x561ed0ced0d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2a78001c30 tx=0x7f2a780017e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e88888669547a1c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:28.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.770+0000 7f2a86acf700 1 -- 172.21.15.12:0/255524594 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2a7800bbf0 con 0x561ed0c408e0 2023-10-05T19:40:28.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.770+0000 7f2a86acf700 1 -- 172.21.15.12:0/255524594 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7800bd50 con 0x561ed0c408e0 2023-10-05T19:40:28.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.770+0000 7f2a86acf700 1 -- 172.21.15.12:0/255524594 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2a7800e470 con 0x561ed0c408e0 2023-10-05T19:40:28.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.774+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/255524594 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561ed0c408e0 msgr2=0x561ed0ced0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.774+0000 7f2a8af9ee40 1 --2- 172.21.15.12:0/255524594 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561ed0c408e0 0x561ed0ced0d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2a78001c30 tx=0x7f2a780017e0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.774+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/255524594 shutdown_connections 2023-10-05T19:40:28.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.774+0000 7f2a8af9ee40 1 --2- 172.21.15.12:0/255524594 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561ed0c408e0 0x561ed0ced0d0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.774+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/255524594 >> 172.21.15.12:0/255524594 conn(0x561ed0cc7820 msgr2=0x561ed0ce8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:28.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.774+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/255524594 shutdown_connections 2023-10-05T19:40:28.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.774+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/255524594 wait complete. 2023-10-05T19:40:28.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.774+0000 7f2a8af9ee40 1 Processor -- start 2023-10-05T19:40:28.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.774+0000 7f2a8af9ee40 1 -- start start 2023-10-05T19:40:28.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.774+0000 7f2a8af9ee40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561ed0c408e0 0x561ed0cf3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:28.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.774+0000 7f2a8af9ee40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ed0b6ced0 con 0x561ed0c408e0 2023-10-05T19:40:28.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.774+0000 7f2a88ad3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561ed0c408e0 0x561ed0cf3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:28.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.774+0000 7f2a88ad3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561ed0c408e0 0x561ed0cf3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58482/0 (socket says 172.21.15.12:58482) 2023-10-05T19:40:28.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.774+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1432187802 learned_addr learned my addr 172.21.15.12:0/1432187802 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:28.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.774+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ed0c18c00 con 0x561ed0c408e0 2023-10-05T19:40:28.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.774+0000 7f2a88ad3700 1 --2- 172.21.15.12:0/1432187802 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561ed0c408e0 0x561ed0cf3e60 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2a78001820 tx=0x7f2a78016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:28.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.778+0000 7f2a852cc700 1 -- 172.21.15.12:0/1432187802 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2a7801f070 con 0x561ed0c408e0 2023-10-05T19:40:28.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.778+0000 7f2a852cc700 1 -- 172.21.15.12:0/1432187802 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a78019760 con 0x561ed0c408e0 2023-10-05T19:40:28.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.778+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561ed0cf43a0 con 0x561ed0c408e0 2023-10-05T19:40:28.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.778+0000 7f2a852cc700 1 -- 172.21.15.12:0/1432187802 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2a780198d0 con 0x561ed0c408e0 2023-10-05T19:40:28.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.778+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561ed0cf47e0 con 0x561ed0c408e0 2023-10-05T19:40:28.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.778+0000 7f2a852cc700 1 -- 172.21.15.12:0/1432187802 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f2a78022020 con 0x561ed0c408e0 2023-10-05T19:40:28.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.778+0000 7f2a852cc700 1 --2- 172.21.15.12:0/1432187802 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2a74030c00 0x7f2a740330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:28.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.778+0000 7f2a852cc700 1 -- 172.21.15.12:0/1432187802 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f2a78044e60 con 0x561ed0c408e0 2023-10-05T19:40:28.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.778+0000 7f2a882d2700 1 --2- 172.21.15.12:0/1432187802 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2a74030c00 0x7f2a740330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:28.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.778+0000 7f2a8af9ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.778+0000 7f2a8af9ee40 1 --2- 172.21.15.12:0/1432187802 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x561ed0c41bf0 0x561ed0cff8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:28.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.778+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d00000 con 0x561ed0c41bf0 2023-10-05T19:40:28.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.778+0000 7f2a892d4700 1 --2- 172.21.15.12:0/1432187802 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x561ed0c41bf0 0x561ed0cff8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:28.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.778+0000 7f2a882d2700 1 --2- 172.21.15.12:0/1432187802 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2a74030c00 0x7f2a740330b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c005a90 tx=0x7f2a7c015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:28.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.782+0000 7f2a892d4700 1 --2- 172.21.15.12:0/1432187802 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x561ed0c41bf0 0x561ed0cff8d0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:28.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.782+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2a8000a5a0 con 0x561ed0c41bf0 2023-10-05T19:40:28.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.782+0000 7f2a8af9ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:28.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.782+0000 7f2a8af9ee40 20 realm 2023-10-05T19:40:28.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.782+0000 7f2a8af9ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.782+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0cfceb0 con 0x561ed0c41bf0 2023-10-05T19:40:28.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.782+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2a8000a5a0 con 0x561ed0c41bf0 2023-10-05T19:40:28.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.782+0000 7f2a8af9ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:28.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.782+0000 7f2a8af9ee40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:28.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.782+0000 7f2a8af9ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.782+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d00ae0 con 0x561ed0c41bf0 2023-10-05T19:40:28.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.782+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2a8000a5a0 con 0x561ed0c41bf0 2023-10-05T19:40:28.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.782+0000 7f2a8af9ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:28.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.782+0000 7f2a8af9ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.782+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d006c0 con 0x561ed0c41bf0 2023-10-05T19:40:28.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.782+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f2a8000a5a0 con 0x561ed0c41bf0 2023-10-05T19:40:28.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.782+0000 7f2a8af9ee40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:28.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.782+0000 7f2a8af9ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.782+0000 7f2a8af9ee40 1 --2- 172.21.15.12:0/1432187802 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x561ed0d01a70 0x561ed0d03e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:28.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.782+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d045c0 con 0x561ed0d01a70 2023-10-05T19:40:28.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.782+0000 7f2a88ad3700 1 --2- 172.21.15.12:0/1432187802 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x561ed0d01a70 0x561ed0d03e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:28.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.782+0000 7f2a88ad3700 1 --2- 172.21.15.12:0/1432187802 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x561ed0d01a70 0x561ed0d03e50 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:28.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.782+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1432187802 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f2a78019b10 con 0x561ed0d01a70 2023-10-05T19:40:28.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.782+0000 7f2a8af9ee40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:28.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.782+0000 7f2a8af9ee40 20 searching for the correct realm 2023-10-05T19:40:28.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.782+0000 7f2a8af9ee40 1 --2- 172.21.15.12:0/1432187802 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x561ed0d063f0 0x561ed0d087d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:28.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.782+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d08d10 con 0x561ed0d063f0 2023-10-05T19:40:28.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.782+0000 7f2a882d2700 1 --2- 172.21.15.12:0/1432187802 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x561ed0d063f0 0x561ed0d087d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:28.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.786+0000 7f2a882d2700 1 --2- 172.21.15.12:0/1432187802 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x561ed0d063f0 0x561ed0d087d0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:28.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.786+0000 7f2a882d2700 1 -- 172.21.15.12:0/1432187802 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f2a7c008ae0 con 0x561ed0d063f0 2023-10-05T19:40:28.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.786+0000 7f2a882d2700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d08d10 con 0x561ed0d01a70 2023-10-05T19:40:28.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.786+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1432187802 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a78019b10 con 0x561ed0d01a70 2023-10-05T19:40:28.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.786+0000 7f2a88ad3700 1 --2- 172.21.15.12:0/1432187802 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f2a780420d0 0x7f2a7803d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:28.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.786+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d08d10 con 0x7f2a780420d0 2023-10-05T19:40:28.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.790+0000 7f2a892d4700 1 --2- 172.21.15.12:0/1432187802 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f2a780420d0 0x7f2a7803d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:28.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.790+0000 7f2a892d4700 1 --2- 172.21.15.12:0/1432187802 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f2a780420d0 0x7f2a7803d690 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:28.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.790+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a8000a5a0 con 0x7f2a780420d0 2023-10-05T19:40:28.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.790+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d08d10 con 0x561ed0c41bf0 2023-10-05T19:40:28.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.790+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a8000a5a0 con 0x561ed0c41bf0 2023-10-05T19:40:28.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.790+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d08d10 con 0x561ed0c41bf0 2023-10-05T19:40:28.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.790+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a8000a5a0 con 0x561ed0c41bf0 2023-10-05T19:40:28.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.790+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d08d10 con 0x7f2a780420d0 2023-10-05T19:40:28.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.794+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a8000a5a0 con 0x7f2a780420d0 2023-10-05T19:40:28.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.794+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d08d10 con 0x7f2a780420d0 2023-10-05T19:40:28.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.794+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a8000a5a0 con 0x7f2a780420d0 2023-10-05T19:40:28.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.794+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d08d10 con 0x561ed0d01a70 2023-10-05T19:40:28.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.794+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1432187802 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a78019b10 con 0x561ed0d01a70 2023-10-05T19:40:28.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.794+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d08d10 con 0x561ed0d063f0 2023-10-05T19:40:28.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.794+0000 7f2a882d2700 1 -- 172.21.15.12:0/1432187802 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a7c008ae0 con 0x561ed0d063f0 2023-10-05T19:40:28.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.794+0000 7f2a882d2700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d08d10 con 0x561ed0c41bf0 2023-10-05T19:40:28.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.794+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a8000a5a0 con 0x561ed0c41bf0 2023-10-05T19:40:28.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.794+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d08d10 con 0x7f2a780420d0 2023-10-05T19:40:28.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.798+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a8000a5a0 con 0x7f2a780420d0 2023-10-05T19:40:28.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.798+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d08d10 con 0x561ed0c41bf0 2023-10-05T19:40:28.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.798+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a8000a5a0 con 0x561ed0c41bf0 2023-10-05T19:40:28.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.798+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d08d10 con 0x561ed0d01a70 2023-10-05T19:40:28.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.798+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1432187802 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a78019b10 con 0x561ed0d01a70 2023-10-05T19:40:28.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.798+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d08d10 con 0x561ed0d063f0 2023-10-05T19:40:28.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.798+0000 7f2a882d2700 1 -- 172.21.15.12:0/1432187802 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a7c008ae0 con 0x561ed0d063f0 2023-10-05T19:40:28.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.798+0000 7f2a882d2700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d08d10 con 0x561ed0d063f0 2023-10-05T19:40:28.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.798+0000 7f2a882d2700 1 -- 172.21.15.12:0/1432187802 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a7c008ae0 con 0x561ed0d063f0 2023-10-05T19:40:28.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.798+0000 7f2a882d2700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d08d10 con 0x7f2a780420d0 2023-10-05T19:40:28.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.802+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a8000a5a0 con 0x7f2a780420d0 2023-10-05T19:40:28.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.802+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d08d10 con 0x561ed0d063f0 2023-10-05T19:40:28.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.802+0000 7f2a882d2700 1 -- 172.21.15.12:0/1432187802 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a7c008ae0 con 0x561ed0d063f0 2023-10-05T19:40:28.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.802+0000 7f2a882d2700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d08d10 con 0x561ed0d063f0 2023-10-05T19:40:28.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.802+0000 7f2a882d2700 1 -- 172.21.15.12:0/1432187802 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a7c008ae0 con 0x561ed0d063f0 2023-10-05T19:40:28.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.802+0000 7f2a882d2700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d08d10 con 0x561ed0c41bf0 2023-10-05T19:40:28.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.802+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a8000a5a0 con 0x561ed0c41bf0 2023-10-05T19:40:28.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.802+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d08d10 con 0x561ed0c41bf0 2023-10-05T19:40:28.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.802+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a8000a5a0 con 0x561ed0c41bf0 2023-10-05T19:40:28.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.802+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d08d10 con 0x7f2a780420d0 2023-10-05T19:40:28.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.802+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a8000a5a0 con 0x7f2a780420d0 2023-10-05T19:40:28.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.802+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d08d10 con 0x561ed0d063f0 2023-10-05T19:40:28.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.806+0000 7f2a882d2700 1 -- 172.21.15.12:0/1432187802 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a7c008ae0 con 0x561ed0d063f0 2023-10-05T19:40:28.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.806+0000 7f2a882d2700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d08d10 con 0x561ed0d01a70 2023-10-05T19:40:28.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.806+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1432187802 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a78019b10 con 0x561ed0d01a70 2023-10-05T19:40:28.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.806+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d08d10 con 0x561ed0c41bf0 2023-10-05T19:40:28.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.806+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a8000a5a0 con 0x561ed0c41bf0 2023-10-05T19:40:28.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.806+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d08d10 con 0x7f2a780420d0 2023-10-05T19:40:28.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.806+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a8000a5a0 con 0x7f2a780420d0 2023-10-05T19:40:28.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.806+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d08d10 con 0x561ed0d063f0 2023-10-05T19:40:28.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.806+0000 7f2a882d2700 1 -- 172.21.15.12:0/1432187802 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a7c008ae0 con 0x561ed0d063f0 2023-10-05T19:40:28.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.806+0000 7f2a882d2700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d08d10 con 0x7f2a780420d0 2023-10-05T19:40:28.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.810+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a8000a5a0 con 0x7f2a780420d0 2023-10-05T19:40:28.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.810+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d08d10 con 0x561ed0d01a70 2023-10-05T19:40:28.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.810+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1432187802 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a78019b10 con 0x561ed0d01a70 2023-10-05T19:40:28.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.810+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d08d10 con 0x561ed0d063f0 2023-10-05T19:40:28.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.810+0000 7f2a882d2700 1 -- 172.21.15.12:0/1432187802 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a7c008ae0 con 0x561ed0d063f0 2023-10-05T19:40:28.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.810+0000 7f2a882d2700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d08d10 con 0x7f2a780420d0 2023-10-05T19:40:28.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.810+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a8000a5a0 con 0x7f2a780420d0 2023-10-05T19:40:28.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.810+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d08d10 con 0x561ed0c41bf0 2023-10-05T19:40:28.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.814+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f2a8000a5a0 con 0x561ed0c41bf0 2023-10-05T19:40:28.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.814+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d08d10 con 0x561ed0d01a70 2023-10-05T19:40:28.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.814+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1432187802 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f2a78019b10 con 0x561ed0d01a70 2023-10-05T19:40:28.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.814+0000 7f2a8af9ee40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:28.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.814+0000 7f2a8af9ee40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:28.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.814+0000 7f2a8af9ee40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:28.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.814+0000 7f2a8af9ee40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:28.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.814+0000 7f2a8af9ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.814+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d051a0 con 0x561ed0c41bf0 2023-10-05T19:40:28.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.814+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2a8000a5a0 con 0x561ed0c41bf0 2023-10-05T19:40:28.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.814+0000 7f2a8af9ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:28.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.814+0000 7f2a8af9ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.814+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d05af0 con 0x561ed0c41bf0 2023-10-05T19:40:28.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.814+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f2a8000a5a0 con 0x561ed0c41bf0 2023-10-05T19:40:28.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.814+0000 7f2a8af9ee40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:28.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.814+0000 7f2a8af9ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.814+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d04d60 con 0x561ed0d063f0 2023-10-05T19:40:28.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.814+0000 7f2a882d2700 1 -- 172.21.15.12:0/1432187802 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f2a7c008ae0 con 0x561ed0d063f0 2023-10-05T19:40:28.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.814+0000 7f2a8af9ee40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:28.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.814+0000 7f2a8af9ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.814+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d09640 con 0x561ed0d063f0 2023-10-05T19:40:28.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.814+0000 7f2a882d2700 1 -- 172.21.15.12:0/1432187802 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f2a7c008ae0 con 0x561ed0d063f0 2023-10-05T19:40:28.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.814+0000 7f2a8af9ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:28.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.814+0000 7f2a8af9ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.814+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d099f0 con 0x561ed0c41bf0 2023-10-05T19:40:28.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.818+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2a8000a5a0 con 0x561ed0c41bf0 2023-10-05T19:40:28.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.818+0000 7f2a8af9ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:28.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.818+0000 7f2a8af9ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.818+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d09da0 con 0x561ed0c41bf0 2023-10-05T19:40:28.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.818+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f2a8000a5a0 con 0x561ed0c41bf0 2023-10-05T19:40:28.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.818+0000 7f2a8af9ee40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:28.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.818+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d0adb0 con 0x561ed0d063f0 2023-10-05T19:40:28.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.818+0000 7f2a882d2700 1 -- 172.21.15.12:0/1432187802 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f2a7c008ae0 con 0x561ed0d063f0 2023-10-05T19:40:28.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.818+0000 7f2a882d2700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d0adb0 con 0x561ed0d01a70 2023-10-05T19:40:28.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.818+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1432187802 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a78019b10 con 0x561ed0d01a70 2023-10-05T19:40:28.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.818+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d0adb0 con 0x7f2a780420d0 2023-10-05T19:40:28.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.822+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a8000a5a0 con 0x7f2a780420d0 2023-10-05T19:40:28.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.822+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d099f0 con 0x561ed0c41bf0 2023-10-05T19:40:28.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.822+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a8000a5a0 con 0x561ed0c41bf0 2023-10-05T19:40:28.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.822+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d099f0 con 0x561ed0c41bf0 2023-10-05T19:40:28.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.822+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a8000a5a0 con 0x561ed0c41bf0 2023-10-05T19:40:28.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.822+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d099f0 con 0x7f2a780420d0 2023-10-05T19:40:28.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.822+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a8000a5a0 con 0x7f2a780420d0 2023-10-05T19:40:28.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.822+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d099f0 con 0x7f2a780420d0 2023-10-05T19:40:28.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.826+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a8000a5a0 con 0x7f2a780420d0 2023-10-05T19:40:28.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.826+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d099f0 con 0x561ed0d01a70 2023-10-05T19:40:28.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.826+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1432187802 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a78019b10 con 0x561ed0d01a70 2023-10-05T19:40:28.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.826+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d099f0 con 0x561ed0d063f0 2023-10-05T19:40:28.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.826+0000 7f2a882d2700 1 -- 172.21.15.12:0/1432187802 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a7c008ae0 con 0x561ed0d063f0 2023-10-05T19:40:28.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.826+0000 7f2a882d2700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d099f0 con 0x561ed0c41bf0 2023-10-05T19:40:28.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.826+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a8000a5a0 con 0x561ed0c41bf0 2023-10-05T19:40:28.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.826+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d099f0 con 0x7f2a780420d0 2023-10-05T19:40:28.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.830+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a8000a5a0 con 0x7f2a780420d0 2023-10-05T19:40:28.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.830+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d099f0 con 0x561ed0c41bf0 2023-10-05T19:40:28.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.830+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a8000a5a0 con 0x561ed0c41bf0 2023-10-05T19:40:28.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.830+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d099f0 con 0x561ed0d01a70 2023-10-05T19:40:28.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.830+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1432187802 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a78019b10 con 0x561ed0d01a70 2023-10-05T19:40:28.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.830+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d099f0 con 0x561ed0d063f0 2023-10-05T19:40:28.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.830+0000 7f2a882d2700 1 -- 172.21.15.12:0/1432187802 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a7c008ae0 con 0x561ed0d063f0 2023-10-05T19:40:28.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.830+0000 7f2a882d2700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d099f0 con 0x561ed0d063f0 2023-10-05T19:40:28.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.830+0000 7f2a882d2700 1 -- 172.21.15.12:0/1432187802 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a7c008ae0 con 0x561ed0d063f0 2023-10-05T19:40:28.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.830+0000 7f2a882d2700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d099f0 con 0x7f2a780420d0 2023-10-05T19:40:28.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.834+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a8000a5a0 con 0x7f2a780420d0 2023-10-05T19:40:28.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.834+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d099f0 con 0x561ed0d063f0 2023-10-05T19:40:28.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.834+0000 7f2a882d2700 1 -- 172.21.15.12:0/1432187802 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a7c008ae0 con 0x561ed0d063f0 2023-10-05T19:40:28.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.834+0000 7f2a882d2700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d099f0 con 0x561ed0d063f0 2023-10-05T19:40:28.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.834+0000 7f2a882d2700 1 -- 172.21.15.12:0/1432187802 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a7c008ae0 con 0x561ed0d063f0 2023-10-05T19:40:28.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.834+0000 7f2a882d2700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d099f0 con 0x561ed0c41bf0 2023-10-05T19:40:28.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.834+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a8000a5a0 con 0x561ed0c41bf0 2023-10-05T19:40:28.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.834+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d099f0 con 0x561ed0c41bf0 2023-10-05T19:40:28.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.838+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a8000a5a0 con 0x561ed0c41bf0 2023-10-05T19:40:28.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.838+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d099f0 con 0x7f2a780420d0 2023-10-05T19:40:28.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.838+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a8000a5a0 con 0x7f2a780420d0 2023-10-05T19:40:28.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.838+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d099f0 con 0x561ed0d063f0 2023-10-05T19:40:28.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.838+0000 7f2a882d2700 1 -- 172.21.15.12:0/1432187802 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a7c008ae0 con 0x561ed0d063f0 2023-10-05T19:40:28.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.838+0000 7f2a882d2700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d099f0 con 0x561ed0d01a70 2023-10-05T19:40:28.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.838+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1432187802 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a78048040 con 0x561ed0d01a70 2023-10-05T19:40:28.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.838+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d099f0 con 0x561ed0c41bf0 2023-10-05T19:40:28.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.842+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a8000a5a0 con 0x561ed0c41bf0 2023-10-05T19:40:28.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.842+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d099f0 con 0x7f2a780420d0 2023-10-05T19:40:28.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.842+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a8000a5a0 con 0x7f2a780420d0 2023-10-05T19:40:28.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.842+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d099f0 con 0x561ed0d063f0 2023-10-05T19:40:28.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.842+0000 7f2a882d2700 1 -- 172.21.15.12:0/1432187802 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a7c008ae0 con 0x561ed0d063f0 2023-10-05T19:40:28.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.842+0000 7f2a882d2700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d099f0 con 0x7f2a780420d0 2023-10-05T19:40:28.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.842+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a8000a5a0 con 0x7f2a780420d0 2023-10-05T19:40:28.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.842+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d099f0 con 0x561ed0d01a70 2023-10-05T19:40:28.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.842+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1432187802 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a78048040 con 0x561ed0d01a70 2023-10-05T19:40:28.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.842+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d099f0 con 0x561ed0d063f0 2023-10-05T19:40:28.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.846+0000 7f2a882d2700 1 -- 172.21.15.12:0/1432187802 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a7c008ae0 con 0x561ed0d063f0 2023-10-05T19:40:28.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.846+0000 7f2a882d2700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d099f0 con 0x7f2a780420d0 2023-10-05T19:40:28.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.846+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2a8000a5a0 con 0x7f2a780420d0 2023-10-05T19:40:28.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.846+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d099f0 con 0x561ed0c41bf0 2023-10-05T19:40:28.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.846+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f2a8000a5a0 con 0x561ed0c41bf0 2023-10-05T19:40:28.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.846+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d099f0 con 0x561ed0d01a70 2023-10-05T19:40:28.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.846+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1432187802 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f2a78048040 con 0x561ed0d01a70 2023-10-05T19:40:28.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.846+0000 7f2a8af9ee40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:28.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.846+0000 7f2a8af9ee40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:28.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.846+0000 7f2a8af9ee40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:28.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.846+0000 7f2a8af9ee40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:28.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.846+0000 7f2a8af9ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.846+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d09da0 con 0x561ed0c41bf0 2023-10-05T19:40:28.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.850+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f2a8000a5a0 con 0x561ed0c41bf0 2023-10-05T19:40:28.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.850+0000 7f2a8af9ee40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:28.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.850+0000 7f2a8af9ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.850+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d0a150 con 0x561ed0d01a70 2023-10-05T19:40:28.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.850+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1432187802 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f2a78048040 con 0x561ed0d01a70 2023-10-05T19:40:28.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.850+0000 7f2a8af9ee40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:28.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.850+0000 7f2a8af9ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.850+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d0a9f0 con 0x561ed0c41bf0 2023-10-05T19:40:28.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.850+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f2a8000a5a0 con 0x561ed0c41bf0 2023-10-05T19:40:28.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.850+0000 7f2a8af9ee40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:28.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.850+0000 7f2a8af9ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.850+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d0a9f0 con 0x561ed0d063f0 2023-10-05T19:40:28.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.850+0000 7f2a882d2700 1 -- 172.21.15.12:0/1432187802 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f2a7c008ae0 con 0x561ed0d063f0 2023-10-05T19:40:28.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.850+0000 7f2a8af9ee40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:28.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.850+0000 7f2a8af9ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.850+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d0ada0 con 0x7f2a780420d0 2023-10-05T19:40:28.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.850+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f2a8000a5a0 con 0x7f2a780420d0 2023-10-05T19:40:28.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.850+0000 7f2a8af9ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:28.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.854+0000 7f2a8af9ee40 20 zone default found 2023-10-05T19:40:28.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.854+0000 7f2a8af9ee40 4 Realm: () 2023-10-05T19:40:28.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.854+0000 7f2a8af9ee40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:28.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.854+0000 7f2a8af9ee40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:28.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.854+0000 7f2a8af9ee40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:28.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.854+0000 7f2a8af9ee40 20 zonegroup default 2023-10-05T19:40:28.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.854+0000 7f2a8af9ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.854+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d0ada0 con 0x7f2a780420d0 2023-10-05T19:40:28.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.854+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f2a8000a5a0 con 0x7f2a780420d0 2023-10-05T19:40:28.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.854+0000 7f2a8af9ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:28.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.854+0000 7f2a8af9ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.854+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d0c720 con 0x561ed0d063f0 2023-10-05T19:40:28.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.854+0000 7f2a882d2700 1 -- 172.21.15.12:0/1432187802 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f2a7c008ae0 con 0x561ed0d063f0 2023-10-05T19:40:28.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.854+0000 7f2a8af9ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:28.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.854+0000 7f2a8af9ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:28.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.854+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d0cad0 con 0x561ed0d063f0 2023-10-05T19:40:28.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.854+0000 7f2a882d2700 1 -- 172.21.15.12:0/1432187802 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f2a7c008ae0 con 0x561ed0d063f0 2023-10-05T19:40:28.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.854+0000 7f2a8af9ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:28.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.854+0000 7f2a8af9ee40 20 started sync module instance, tier type = 2023-10-05T19:40:28.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.854+0000 7f2a8af9ee40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:28.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.854+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d0bde0 con 0x561ed0d01a70 2023-10-05T19:40:28.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.858+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1432187802 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'150 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2a78048040 con 0x561ed0d01a70 2023-10-05T19:40:28.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.858+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94690352349664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d0a620 con 0x561ed0d01a70 2023-10-05T19:40:28.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.858+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d0d860 con 0x561ed0c41bf0 2023-10-05T19:40:28.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.862+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1432187802 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94690352349664] v28'151 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2a78048040 con 0x561ed0d01a70 2023-10-05T19:40:28.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.862+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'150 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2a8000a5a0 con 0x561ed0c41bf0 2023-10-05T19:40:28.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.862+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94690352353184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d0dc60 con 0x561ed0c41bf0 2023-10-05T19:40:28.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.862+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d0e500 con 0x561ed0d063f0 2023-10-05T19:40:28.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.866+0000 7f2a882d2700 1 -- 172.21.15.12:0/1432187802 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'150 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2a7c008ae0 con 0x561ed0d063f0 2023-10-05T19:40:28.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.866+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94690352356736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d0eda0 con 0x561ed0d063f0 2023-10-05T19:40:28.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.866+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d0f680 con 0x561ed0d01a70 2023-10-05T19:40:28.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.866+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94690352353184] v28'151 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2a8000a5a0 con 0x561ed0c41bf0 2023-10-05T19:40:28.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.866+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1432187802 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'150 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2a78048040 con 0x561ed0d01a70 2023-10-05T19:40:28.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.866+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94690352354288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d0f270 con 0x561ed0d01a70 2023-10-05T19:40:28.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.866+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d106d0 con 0x561ed0c41bf0 2023-10-05T19:40:28.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.870+0000 7f2a882d2700 1 -- 172.21.15.12:0/1432187802 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94690352356736] v28'151 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2a7c008ae0 con 0x561ed0d063f0 2023-10-05T19:40:28.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.870+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'150 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2a8000a5a0 con 0x561ed0c41bf0 2023-10-05T19:40:28.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.870+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94690352359600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d0f270 con 0x561ed0c41bf0 2023-10-05T19:40:28.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.870+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d11130 con 0x7f2a780420d0 2023-10-05T19:40:28.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.870+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1432187802 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94690352354288] v28'151 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2a78048040 con 0x561ed0d01a70 2023-10-05T19:40:28.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.874+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'150 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2a8000a5a0 con 0x7f2a780420d0 2023-10-05T19:40:28.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.874+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94690352364160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d11130 con 0x7f2a780420d0 2023-10-05T19:40:28.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.874+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d11b90 con 0x7f2a780420d0 2023-10-05T19:40:28.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.874+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94690352359600] v28'151 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2a8000a5a0 con 0x561ed0c41bf0 2023-10-05T19:40:28.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.874+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(95 notify.5 [watch watch cookie 94690352364160] v28'151 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2a8000a5a0 con 0x7f2a780420d0 2023-10-05T19:40:28.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.874+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(96 notify.6 [create] v28'150 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2a8000a5a0 con 0x7f2a780420d0 2023-10-05T19:40:28.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.874+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94690352368704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d11090 con 0x7f2a780420d0 2023-10-05T19:40:28.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.878+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d12080 con 0x7f2a780420d0 2023-10-05T19:40:28.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.878+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(98 notify.7 [create] v28'150 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2a8000a5a0 con 0x7f2a780420d0 2023-10-05T19:40:28.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.878+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94690352372624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d124d0 con 0x7f2a780420d0 2023-10-05T19:40:28.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.878+0000 7f2a8af9ee40 20 add_watcher() i=0 2023-10-05T19:40:28.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.878+0000 7f2a8af9ee40 20 add_watcher() i=1 2023-10-05T19:40:28.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.878+0000 7f2a8af9ee40 20 add_watcher() i=2 2023-10-05T19:40:28.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.878+0000 7f2a8af9ee40 20 add_watcher() i=3 2023-10-05T19:40:28.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.878+0000 7f2a8af9ee40 20 add_watcher() i=4 2023-10-05T19:40:28.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.878+0000 7f2a8af9ee40 20 add_watcher() i=5 2023-10-05T19:40:28.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.878+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(97 notify.6 [watch watch cookie 94690352368704] v28'151 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2a8000a5a0 con 0x7f2a780420d0 2023-10-05T19:40:28.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.878+0000 7f2a8af9ee40 20 add_watcher() i=6 2023-10-05T19:40:28.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.882+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94690352372624] v28'151 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2a8000a5a0 con 0x7f2a780420d0 2023-10-05T19:40:28.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.882+0000 7f2a8af9ee40 20 add_watcher() i=7 2023-10-05T19:40:28.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.882+0000 7f2a8af9ee40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:28.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.882+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d15500 con 0x561ed0c41bf0 2023-10-05T19:40:28.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.882+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f2a8000a5a0 con 0x561ed0c41bf0 2023-10-05T19:40:28.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.882+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94690352373728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d12920 con 0x561ed0c41bf0 2023-10-05T19:40:28.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.886+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94690352373728] v28'147 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f2a8000a5a0 con 0x561ed0c41bf0 2023-10-05T19:40:28.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.886+0000 7f2a5a7e4700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:28.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.886+0000 7f2a8af9ee40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:28.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.886+0000 7f2a8af9ee40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:28.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.886+0000 7f2a8af9ee40 1 Processor -- start 2023-10-05T19:40:28.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.886+0000 7f2a8af9ee40 1 -- start start 2023-10-05T19:40:28.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.886+0000 7f2a8af9ee40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561ed0d1f5c0 0x561ed0d1f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:28.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.886+0000 7f2a8af9ee40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ed0d1fed0 con 0x561ed0d1f5c0 2023-10-05T19:40:28.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.886+0000 7f2a882d2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561ed0d1f5c0 0x561ed0d1f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:28.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.886+0000 7f2a882d2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561ed0d1f5c0 0x561ed0d1f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58484/0 (socket says 172.21.15.12:58484) 2023-10-05T19:40:28.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.886+0000 7f2a882d2700 1 -- 172.21.15.12:0/1891027799 learned_addr learned my addr 172.21.15.12:0/1891027799 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:28.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.886+0000 7f2a882d2700 1 -- 172.21.15.12:0/1891027799 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ed0d20010 con 0x561ed0d1f5c0 2023-10-05T19:40:28.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.886+0000 7f2a882d2700 1 --2- 172.21.15.12:0/1891027799 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561ed0d1f5c0 0x561ed0d1f990 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x561ed0c42050 tx=0x7f2a7c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2254df74dae1dbb0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:28.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.886+0000 7f2a59fe3700 1 -- 172.21.15.12:0/1891027799 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2a7c0308b0 con 0x561ed0d1f5c0 2023-10-05T19:40:28.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.886+0000 7f2a59fe3700 1 -- 172.21.15.12:0/1891027799 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7c036040 con 0x561ed0d1f5c0 2023-10-05T19:40:28.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.886+0000 7f2a59fe3700 1 -- 172.21.15.12:0/1891027799 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2a7c030a90 con 0x561ed0d1f5c0 2023-10-05T19:40:28.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.890+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1891027799 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561ed0d1f5c0 msgr2=0x561ed0d1f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.890+0000 7f2a8af9ee40 1 --2- 172.21.15.12:0/1891027799 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561ed0d1f5c0 0x561ed0d1f990 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x561ed0c42050 tx=0x7f2a7c012000 comp rx=0 tx=0).stop 2023-10-05T19:40:28.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.890+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1891027799 shutdown_connections 2023-10-05T19:40:28.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.890+0000 7f2a8af9ee40 1 --2- 172.21.15.12:0/1891027799 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561ed0d1f5c0 0x561ed0d1f990 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.890+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1891027799 >> 172.21.15.12:0/1891027799 conn(0x561ed0d1a980 msgr2=0x561ed0d18b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:28.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.890+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1891027799 shutdown_connections 2023-10-05T19:40:28.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.890+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1891027799 wait complete. 2023-10-05T19:40:28.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.890+0000 7f2a8af9ee40 1 Processor -- start 2023-10-05T19:40:28.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.890+0000 7f2a8af9ee40 1 -- start start 2023-10-05T19:40:28.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.890+0000 7f2a8af9ee40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561ed0d1f5c0 0x561ed0d25050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:28.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.890+0000 7f2a8af9ee40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ed0d26050 con 0x561ed0d1f5c0 2023-10-05T19:40:28.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.890+0000 7f2a882d2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561ed0d1f5c0 0x561ed0d25050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:28.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.890+0000 7f2a882d2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561ed0d1f5c0 0x561ed0d25050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58494/0 (socket says 172.21.15.12:58494) 2023-10-05T19:40:28.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.890+0000 7f2a882d2700 1 -- 172.21.15.12:0/1035957305 learned_addr learned my addr 172.21.15.12:0/1035957305 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:28.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.890+0000 7f2a882d2700 1 -- 172.21.15.12:0/1035957305 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ed0c42080 con 0x561ed0d1f5c0 2023-10-05T19:40:28.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.890+0000 7f2a882d2700 1 --2- 172.21.15.12:0/1035957305 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561ed0d1f5c0 0x561ed0d25050 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c03b000 tx=0x7f2a7c037000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:28.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.890+0000 7f29cbfff700 1 -- 172.21.15.12:0/1035957305 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2a7c027cd0 con 0x561ed0d1f5c0 2023-10-05T19:40:28.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.890+0000 7f29cbfff700 1 -- 172.21.15.12:0/1035957305 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7c027e30 con 0x561ed0d1f5c0 2023-10-05T19:40:28.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.890+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1035957305 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561ed0d25650 con 0x561ed0d1f5c0 2023-10-05T19:40:28.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.890+0000 7f29cbfff700 1 -- 172.21.15.12:0/1035957305 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2a7c04c3f0 con 0x561ed0d1f5c0 2023-10-05T19:40:28.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.890+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1035957305 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561ed0d25af0 con 0x561ed0d1f5c0 2023-10-05T19:40:28.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.894+0000 7f29cbfff700 1 -- 172.21.15.12:0/1035957305 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f2a7c055440 con 0x561ed0d1f5c0 2023-10-05T19:40:28.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.894+0000 7f29cbfff700 1 --2- 172.21.15.12:0/1035957305 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f29d0030c50 0x7f29d0033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:28.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.894+0000 7f29cbfff700 1 -- 172.21.15.12:0/1035957305 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f2a7c04de20 con 0x561ed0d1f5c0 2023-10-05T19:40:28.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.894+0000 7f2a892d4700 1 --2- 172.21.15.12:0/1035957305 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f29d0030c50 0x7f29d0033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:28.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.894+0000 7f2a8af9ee40 5 note: GC not initialized 2023-10-05T19:40:28.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.894+0000 7f2a892d4700 1 --2- 172.21.15.12:0/1035957305 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f29d0030c50 0x7f29d0033100 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x561ed0d11150 tx=0x7f2a8000ae90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:28.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.898+0000 7f29c97fa700 20 reqs_thread_entry: start 2023-10-05T19:40:28.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.898+0000 7f2a8af9ee40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:28.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.898+0000 7f29c2ffd700 20 reqs_thread_entry: start 2023-10-05T19:40:28.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.898+0000 7f2a8af9ee40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:28.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.898+0000 7f29c17fa700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:59 2023 2023-10-05T19:40:28.908 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:28.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.898+0000 7f29c0ff9700 20 reqs_thread_entry: start 2023-10-05T19:40:28.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.898+0000 7f2a8af9ee40 1 --2- 172.21.15.12:0/1035957305 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x561ed0d8d6b0 0x561ed0d8fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:28.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.898+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1035957305 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:1 6.0 6:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d8ffd0 con 0x561ed0d8d6b0 2023-10-05T19:40:28.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.898+0000 7f2a88ad3700 1 --2- 172.21.15.12:0/1035957305 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x561ed0d8d6b0 0x561ed0d8fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:28.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.898+0000 7f2a88ad3700 1 --2- 172.21.15.12:0/1035957305 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x561ed0d8d6b0 0x561ed0d8fa90 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:28.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.902+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1035957305 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(1 [pgnls start_epoch 28 out=330b] v28'36 uv28 ondisk = 1) v8 ==== 144+0+330 (crc 0 0 0) 0x7f2a78048040 con 0x561ed0d8d6b0 2023-10-05T19:40:28.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.902+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1035957305 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:2 6.10 6:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d0f680 con 0x561ed0d8d6b0 2023-10-05T19:40:28.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.902+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1035957305 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(2 [pgnls start_epoch 28 out=131b] v28'4 uv4 ondisk = 1) v8 ==== 144+0+131 (crc 0 0 0) 0x7f2a78048040 con 0x561ed0d8d6b0 2023-10-05T19:40:28.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.902+0000 7f2a88ad3700 1 --2- 172.21.15.12:0/1035957305 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x7f2a7804e040 0x7f2a78042950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:28.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.902+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1035957305 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:3 6.8 6:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d0f680 con 0x7f2a7804e040 2023-10-05T19:40:28.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.902+0000 7f2a882d2700 1 --2- 172.21.15.12:0/1035957305 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x7f2a7804e040 0x7f2a78042950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:28.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.902+0000 7f2a882d2700 1 --2- 172.21.15.12:0/1035957305 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x7f2a7804e040 0x7f2a78042950 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:28.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.906+0000 7f2a882d2700 1 -- 172.21.15.12:0/1035957305 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(3 [pgnls start_epoch 28 out=322b] v28'25 uv21 ondisk = 1) v8 ==== 144+0+322 (crc 0 0 0) 0x7f2a7c040020 con 0x7f2a7804e040 2023-10-05T19:40:28.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.906+0000 7f2a882d2700 1 -- 172.21.15.12:0/1035957305 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:4 6.18 6:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d0eda0 con 0x561ed0d8d6b0 2023-10-05T19:40:28.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.906+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1035957305 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(4 [pgnls start_epoch 28 out=172b] v28'17 uv9 ondisk = 1) v8 ==== 144+0+172 (crc 0 0 0) 0x7f2a78048040 con 0x561ed0d8d6b0 2023-10-05T19:40:28.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.906+0000 7f2a88ad3700 1 --2- 172.21.15.12:0/1035957305 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x7f2a7800dc20 0x7f2a7804d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:28.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.906+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1035957305 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:5 6.4 6:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d0eda0 con 0x7f2a7800dc20 2023-10-05T19:40:28.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.906+0000 7f2a892d4700 1 --2- 172.21.15.12:0/1035957305 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x7f2a7800dc20 0x7f2a7804d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:28.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.906+0000 7f2a892d4700 1 --2- 172.21.15.12:0/1035957305 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x7f2a7800dc20 0x7f2a7804d740 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:28.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.910+0000 7f2a892d4700 1 -- 172.21.15.12:0/1035957305 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(5 [pgnls start_epoch 28 out=335b] v28'147 uv23 ondisk = 1) v8 ==== 144+0+335 (crc 0 0 0) 0x7f2a8000a5a0 con 0x7f2a7800dc20 2023-10-05T19:40:28.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.910+0000 7f2a892d4700 1 --2- 172.21.15.12:0/1035957305 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x7f2a80024040 0x7f2a80036950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:28.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.910+0000 7f2a892d4700 1 -- 172.21.15.12:0/1035957305 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:6 6.14 6:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d09da0 con 0x7f2a80024040 2023-10-05T19:40:28.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.910+0000 7f2a88ad3700 1 --2- 172.21.15.12:0/1035957305 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x7f2a80024040 0x7f2a80036950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:28.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.910+0000 7f2a88ad3700 1 --2- 172.21.15.12:0/1035957305 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x7f2a80024040 0x7f2a80036950 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:28.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.910+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1035957305 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=213b] v28'10 uv10 ondisk = 1) v8 ==== 144+0+213 (crc 0 0 0) 0x7f2a78048040 con 0x7f2a80024040 2023-10-05T19:40:28.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.910+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1035957305 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:7 6.c 6:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f2a78064040 con 0x7f2a7804e040 2023-10-05T19:40:28.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.910+0000 7f2a882d2700 1 -- 172.21.15.12:0/1035957305 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=254b] v28'10 uv10 ondisk = 1) v8 ==== 144+0+254 (crc 0 0 0) 0x7f2a7c040020 con 0x7f2a7804e040 2023-10-05T19:40:28.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.910+0000 7f2a882d2700 1 -- 172.21.15.12:0/1035957305 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:8 6.1c 6:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f2a78064040 con 0x7f2a7804e040 2023-10-05T19:40:28.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.910+0000 7f2a882d2700 1 -- 172.21.15.12:0/1035957305 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(8 [pgnls start_epoch 28 out=295b] v28'23 uv15 ondisk = 1) v8 ==== 144+0+295 (crc 0 0 0) 0x7f2a7c040020 con 0x7f2a7804e040 2023-10-05T19:40:28.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.914+0000 7f2a882d2700 1 -- 172.21.15.12:0/1035957305 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:9 6.2 6:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f2a78064040 con 0x7f2a7804e040 2023-10-05T19:40:28.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.914+0000 7f2a882d2700 1 -- 172.21.15.12:0/1035957305 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(9 [pgnls start_epoch 28 out=213b] v28'10 uv10 ondisk = 1) v8 ==== 144+0+213 (crc 0 0 0) 0x7f2a7c040020 con 0x7f2a7804e040 2023-10-05T19:40:28.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.914+0000 7f2a882d2700 1 -- 172.21.15.12:0/1035957305 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:10 6.12 6:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f2a78064040 con 0x7f2a80024040 2023-10-05T19:40:28.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.914+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1035957305 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(10 [pgnls start_epoch 28 out=172b] v28'17 uv9 ondisk = 1) v8 ==== 144+0+172 (crc 0 0 0) 0x7f2a78048040 con 0x7f2a80024040 2023-10-05T19:40:28.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.914+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1035957305 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:11 6.a 6:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f2a78064040 con 0x7f2a80024040 2023-10-05T19:40:28.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.914+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1035957305 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(11 [pgnls start_epoch 28 out=336b] v28'27 uv19 ondisk = 1) v8 ==== 144+0+336 (crc 0 0 0) 0x7f2a78048040 con 0x7f2a80024040 2023-10-05T19:40:28.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.914+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1035957305 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:12 6.1a 6:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f2a78064040 con 0x7f2a7804e040 2023-10-05T19:40:28.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.914+0000 7f2a882d2700 1 -- 172.21.15.12:0/1035957305 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(12 [pgnls start_epoch 28 out=131b] v28'17 uv9 ondisk = 1) v8 ==== 144+0+131 (crc 0 0 0) 0x7f2a7c040020 con 0x7f2a7804e040 2023-10-05T19:40:28.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.914+0000 7f2a882d2700 1 -- 172.21.15.12:0/1035957305 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:13 6.6 6:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f2a78064040 con 0x7f2a7804e040 2023-10-05T19:40:28.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.918+0000 7f2a882d2700 1 -- 172.21.15.12:0/1035957305 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(13 [pgnls start_epoch 28 out=254b] v28'12 uv12 ondisk = 1) v8 ==== 144+0+254 (crc 0 0 0) 0x7f2a7c040020 con 0x7f2a7804e040 2023-10-05T19:40:28.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.918+0000 7f2a882d2700 1 -- 172.21.15.12:0/1035957305 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 6.16 6:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f2a78064040 con 0x7f2a7804e040 2023-10-05T19:40:28.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.918+0000 7f2a882d2700 1 -- 172.21.15.12:0/1035957305 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(14 [pgnls start_epoch 28 out=90b] v28'17 uv9 ondisk = 1) v8 ==== 144+0+90 (crc 0 0 0) 0x7f2a7c040020 con 0x7f2a7804e040 2023-10-05T19:40:28.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.918+0000 7f2a882d2700 1 -- 172.21.15.12:0/1035957305 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 6.e 6:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f2a78064040 con 0x7f2a7800dc20 2023-10-05T19:40:28.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.918+0000 7f2a892d4700 1 -- 172.21.15.12:0/1035957305 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(15 [pgnls start_epoch 28 out=213b] v28'19 uv11 ondisk = 1) v8 ==== 144+0+213 (crc 0 0 0) 0x7f2a8000a5a0 con 0x7f2a7800dc20 2023-10-05T19:40:28.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.918+0000 7f2a892d4700 1 -- 172.21.15.12:0/1035957305 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:16 6.1e 6:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f2a78064040 con 0x7f2a7804e040 2023-10-05T19:40:28.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.918+0000 7f2a882d2700 1 -- 172.21.15.12:0/1035957305 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(16 [pgnls start_epoch 28 out=330b] v28'12 uv12 ondisk = 1) v8 ==== 144+0+330 (crc 0 0 0) 0x7f2a7c067040 con 0x7f2a7804e040 2023-10-05T19:40:28.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.918+0000 7f2a882d2700 1 -- 172.21.15.12:0/1035957305 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 6.1 6:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f2a78064040 con 0x7f2a7800dc20 2023-10-05T19:40:28.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.922+0000 7f2a892d4700 1 -- 172.21.15.12:0/1035957305 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(17 [pgnls start_epoch 28 out=254b] v28'17 uv17 ondisk = 1) v8 ==== 144+0+254 (crc 0 0 0) 0x7f2a8000a5a0 con 0x7f2a7800dc20 2023-10-05T19:40:28.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.922+0000 7f2a892d4700 1 -- 172.21.15.12:0/1035957305 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:18 6.11 6:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f2a78064040 con 0x7f2a7804e040 2023-10-05T19:40:28.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.922+0000 7f2a882d2700 1 -- 172.21.15.12:0/1035957305 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(18 [pgnls start_epoch 28 out=213b] v28'19 uv11 ondisk = 1) v8 ==== 144+0+213 (crc 0 0 0) 0x7f2a7c067040 con 0x7f2a7804e040 2023-10-05T19:40:28.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.922+0000 7f2a882d2700 1 -- 172.21.15.12:0/1035957305 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 6.9 6:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f2a78064040 con 0x7f2a7804e040 2023-10-05T19:40:28.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.922+0000 7f2a882d2700 1 -- 172.21.15.12:0/1035957305 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(19 [pgnls start_epoch 28 out=172b] v28'31 uv23 ondisk = 1) v8 ==== 144+0+172 (crc 0 0 0) 0x7f2a7c067040 con 0x7f2a7804e040 2023-10-05T19:40:28.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.922+0000 7f2a882d2700 1 -- 172.21.15.12:0/1035957305 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:20 6.19 6:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f2a78064040 con 0x7f2a7800dc20 2023-10-05T19:40:28.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.922+0000 7f2a892d4700 1 -- 172.21.15.12:0/1035957305 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=295b] v28'26 uv22 ondisk = 1) v8 ==== 144+0+295 (crc 0 0 0) 0x7f2a8000a5a0 con 0x7f2a7800dc20 2023-10-05T19:40:28.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.922+0000 7f2a892d4700 1 -- 172.21.15.12:0/1035957305 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:21 6.5 6:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f2a78064040 con 0x7f2a80024040 2023-10-05T19:40:28.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.926+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1035957305 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(21 [pgnls start_epoch 28 out=276b] v28'13 uv13 ondisk = 1) v8 ==== 144+0+276 (crc 0 0 0) 0x7f2a78048040 con 0x7f2a80024040 2023-10-05T19:40:28.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.926+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1035957305 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:22 6.15 6:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f2a78064040 con 0x7f2a80024040 2023-10-05T19:40:28.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.926+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1035957305 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=172b] v28'17 uv9 ondisk = 1) v8 ==== 144+0+172 (crc 0 0 0) 0x7f2a78048040 con 0x7f2a80024040 2023-10-05T19:40:28.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.926+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1035957305 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:23 6.d 6:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f2a78064040 con 0x7f2a7800dc20 2023-10-05T19:40:28.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.926+0000 7f2a892d4700 1 -- 172.21.15.12:0/1035957305 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(23 [pgnls start_epoch 28 out=370b] v28'33 uv32 ondisk = 1) v8 ==== 144+0+370 (crc 0 0 0) 0x7f2a8000a5a0 con 0x7f2a7800dc20 2023-10-05T19:40:28.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.926+0000 7f2a892d4700 1 -- 172.21.15.12:0/1035957305 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:24 6.1d 6:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f2a78064040 con 0x561ed0d8d6b0 2023-10-05T19:40:28.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.926+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1035957305 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(24 [pgnls start_epoch 28 out=172b] v28'28 uv12 ondisk = 1) v8 ==== 144+0+172 (crc 0 0 0) 0x7f2a78048040 con 0x561ed0d8d6b0 2023-10-05T19:40:28.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.926+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1035957305 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:25 6.3 6:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f2a78064040 con 0x7f2a80024040 2023-10-05T19:40:28.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.930+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1035957305 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(25 [pgnls start_epoch 28 out=213b] v28'43 uv19 ondisk = 1) v8 ==== 144+0+213 (crc 0 0 0) 0x7f2a78048040 con 0x7f2a80024040 2023-10-05T19:40:28.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.930+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1035957305 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 6.13 6:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f2a78064040 con 0x561ed0d8d6b0 2023-10-05T19:40:28.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.930+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1035957305 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(26 [pgnls start_epoch 28 out=172b] v28'19 uv11 ondisk = 1) v8 ==== 144+0+172 (crc 0 0 0) 0x7f2a78048040 con 0x561ed0d8d6b0 2023-10-05T19:40:28.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.930+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1035957305 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 6.b 6:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f2a78064040 con 0x7f2a7804e040 2023-10-05T19:40:28.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.930+0000 7f2a882d2700 1 -- 172.21.15.12:0/1035957305 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(27 [pgnls start_epoch 28 out=172b] v28'17 uv9 ondisk = 1) v8 ==== 144+0+172 (crc 0 0 0) 0x7f2a7c067040 con 0x7f2a7804e040 2023-10-05T19:40:28.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.930+0000 7f2a882d2700 1 -- 172.21.15.12:0/1035957305 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:28 6.1b 6:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f2a78064040 con 0x7f2a7804e040 2023-10-05T19:40:28.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.934+0000 7f2a882d2700 1 -- 172.21.15.12:0/1035957305 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(28 [pgnls start_epoch 28 out=90b] v28'14 uv5 ondisk = 1) v8 ==== 144+0+90 (crc 0 0 0) 0x7f2a7c067040 con 0x7f2a7804e040 2023-10-05T19:40:28.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.934+0000 7f2a882d2700 1 -- 172.21.15.12:0/1035957305 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:29 6.7 6:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f2a78064040 con 0x7f2a7804e040 2023-10-05T19:40:28.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.934+0000 7f2a882d2700 1 -- 172.21.15.12:0/1035957305 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(29 [pgnls start_epoch 28 out=90b] v28'37 uv13 ondisk = 1) v8 ==== 144+0+90 (crc 0 0 0) 0x7f2a7c067040 con 0x7f2a7804e040 2023-10-05T19:40:28.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.934+0000 7f2a882d2700 1 -- 172.21.15.12:0/1035957305 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:30 6.17 6:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f2a78064040 con 0x7f2a7800dc20 2023-10-05T19:40:28.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.934+0000 7f2a892d4700 1 -- 172.21.15.12:0/1035957305 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(30 [pgnls start_epoch 28 out=131b] v28'6 uv6 ondisk = 1) v8 ==== 144+0+131 (crc 0 0 0) 0x7f2a8000a5a0 con 0x7f2a7800dc20 2023-10-05T19:40:28.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.934+0000 7f2a892d4700 1 -- 172.21.15.12:0/1035957305 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:31 6.f 6:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f2a78064040 con 0x7f2a80024040 2023-10-05T19:40:28.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.934+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1035957305 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(31 [pgnls start_epoch 28 out=295b] v28'23 uv15 ondisk = 1) v8 ==== 144+0+295 (crc 0 0 0) 0x7f2a78048040 con 0x7f2a80024040 2023-10-05T19:40:28.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.934+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1035957305 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 6.1f 6:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f2a78064040 con 0x561ed0d8d6b0 2023-10-05T19:40:28.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.938+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1035957305 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(32 [pgnls start_epoch 28 out=289b] v28'22 uv17 ondisk = 1) v8 ==== 144+0+289 (crc 0 0 0) 0x7f2a78048040 con 0x561ed0d8d6b0 2023-10-05T19:40:28.949 INFO:teuthology.orchestra.run.smithi012.stdout:[ 2023-10-05T19:40:28.949 INFO:teuthology.orchestra.run.smithi012.stdout: "2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.4-myfoo4", 2023-10-05T19:40:28.949 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000078", 2023-10-05T19:40:28.949 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000070", 2023-10-05T19:40:28.949 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000104", 2023-10-05T19:40:28.949 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000026", 2023-10-05T19:40:28.949 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000028", 2023-10-05T19:40:28.949 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000040", 2023-10-05T19:40:28.950 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000015", 2023-10-05T19:40:28.950 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000069", 2023-10-05T19:40:28.950 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000095", 2023-10-05T19:40:28.950 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000003", 2023-10-05T19:40:28.950 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000047", 2023-10-05T19:40:28.950 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000035", 2023-10-05T19:40:28.950 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000037", 2023-10-05T19:40:28.950 INFO:teuthology.orchestra.run.smithi012.stdout: "2023-10-05-19--", 2023-10-05T19:40:28.950 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000024", 2023-10-05T19:40:28.950 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000000", 2023-10-05T19:40:28.950 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000031", 2023-10-05T19:40:28.951 INFO:teuthology.orchestra.run.smithi012.stdout: "data_loggenerations_metadata", 2023-10-05T19:40:28.951 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000076", 2023-10-05T19:40:28.951 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000113", 2023-10-05T19:40:28.951 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000005", 2023-10-05T19:40:28.951 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000011", 2023-10-05T19:40:28.951 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000058", 2023-10-05T19:40:28.951 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000052", 2023-10-05T19:40:28.951 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000088", 2023-10-05T19:40:28.951 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000080", 2023-10-05T19:40:28.951 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000090", 2023-10-05T19:40:28.951 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000110", 2023-10-05T19:40:28.951 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000096", 2023-10-05T19:40:28.951 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000087", 2023-10-05T19:40:28.952 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000008", 2023-10-05T19:40:28.952 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000006", 2023-10-05T19:40:28.952 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000029", 2023-10-05T19:40:28.952 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000089", 2023-10-05T19:40:28.952 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000022", 2023-10-05T19:40:28.952 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000016", 2023-10-05T19:40:28.952 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000041", 2023-10-05T19:40:28.952 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000018", 2023-10-05T19:40:28.952 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000092", 2023-10-05T19:40:28.952 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000014", 2023-10-05T19:40:28.952 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000112", 2023-10-05T19:40:28.952 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000007", 2023-10-05T19:40:28.953 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000021", 2023-10-05T19:40:28.953 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000064", 2023-10-05T19:40:28.953 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000071", 2023-10-05T19:40:28.953 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000074", 2023-10-05T19:40:28.953 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000081", 2023-10-05T19:40:28.953 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000009", 2023-10-05T19:40:28.953 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000121", 2023-10-05T19:40:28.953 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000125", 2023-10-05T19:40:28.953 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000082", 2023-10-05T19:40:28.953 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000105", 2023-10-05T19:40:28.953 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000059", 2023-10-05T19:40:28.953 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000077", 2023-10-05T19:40:28.954 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000032", 2023-10-05T19:40:28.954 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000053", 2023-10-05T19:40:28.954 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000091", 2023-10-05T19:40:28.954 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000065", 2023-10-05T19:40:28.954 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000083", 2023-10-05T19:40:28.954 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000010", 2023-10-05T19:40:28.954 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000045", 2023-10-05T19:40:28.954 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000002", 2023-10-05T19:40:28.954 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000116", 2023-10-05T19:40:28.954 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000034", 2023-10-05T19:40:28.954 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000101", 2023-10-05T19:40:28.954 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000079", 2023-10-05T19:40:28.955 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000049", 2023-10-05T19:40:28.955 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000117", 2023-10-05T19:40:28.955 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000044", 2023-10-05T19:40:28.955 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000066", 2023-10-05T19:40:28.955 INFO:teuthology.orchestra.run.smithi012.stdout: "2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.3-myfoo3", 2023-10-05T19:40:28.955 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000068", 2023-10-05T19:40:28.955 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000085", 2023-10-05T19:40:28.955 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000073", 2023-10-05T19:40:28.955 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000038", 2023-10-05T19:40:28.955 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000118", 2023-10-05T19:40:28.955 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000036", 2023-10-05T19:40:28.956 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000103", 2023-10-05T19:40:28.956 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000119", 2023-10-05T19:40:28.956 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000098", 2023-10-05T19:40:28.956 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000027", 2023-10-05T19:40:28.956 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000019", 2023-10-05T19:40:28.956 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000039", 2023-10-05T19:40:28.956 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000100", 2023-10-05T19:40:28.956 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000093", 2023-10-05T19:40:28.956 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000004", 2023-10-05T19:40:28.956 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000063", 2023-10-05T19:40:28.956 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000122", 2023-10-05T19:40:28.956 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000057", 2023-10-05T19:40:28.957 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000054", 2023-10-05T19:40:28.957 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000114", 2023-10-05T19:40:28.957 INFO:teuthology.orchestra.run.smithi012.stdout: "data_log.0", 2023-10-05T19:40:28.957 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000012", 2023-10-05T19:40:28.957 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000084", 2023-10-05T19:40:28.957 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000043", 2023-10-05T19:40:28.957 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000111", 2023-10-05T19:40:28.957 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000048", 2023-10-05T19:40:28.957 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000020", 2023-10-05T19:40:28.957 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000099", 2023-10-05T19:40:28.957 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000056", 2023-10-05T19:40:28.958 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000072", 2023-10-05T19:40:28.958 INFO:teuthology.orchestra.run.smithi012.stdout: "2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1-myfoo", 2023-10-05T19:40:28.958 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000062", 2023-10-05T19:40:28.958 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000115", 2023-10-05T19:40:28.958 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000033", 2023-10-05T19:40:28.958 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000025", 2023-10-05T19:40:28.958 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000123", 2023-10-05T19:40:28.958 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000108", 2023-10-05T19:40:28.958 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000094", 2023-10-05T19:40:28.958 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000060", 2023-10-05T19:40:28.958 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000109", 2023-10-05T19:40:28.959 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000013", 2023-10-05T19:40:28.959 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000051", 2023-10-05T19:40:28.959 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000106", 2023-10-05T19:40:28.959 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000017", 2023-10-05T19:40:28.959 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000023", 2023-10-05T19:40:28.959 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000107", 2023-10-05T19:40:28.959 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000046", 2023-10-05T19:40:28.959 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000067", 2023-10-05T19:40:28.959 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000086", 2023-10-05T19:40:28.959 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000050", 2023-10-05T19:40:28.959 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000102", 2023-10-05T19:40:28.959 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000124", 2023-10-05T19:40:28.960 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000001", 2023-10-05T19:40:28.960 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000126", 2023-10-05T19:40:28.960 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000055", 2023-10-05T19:40:28.960 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000061", 2023-10-05T19:40:28.960 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000097", 2023-10-05T19:40:28.960 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000042", 2023-10-05T19:40:28.960 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000075", 2023-10-05T19:40:28.960 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000120", 2023-10-05T19:40:28.960 INFO:teuthology.orchestra.run.smithi012.stdout: "obj_delete_at_hint.0000000030", 2023-10-05T19:40:28.960 INFO:teuthology.orchestra.run.smithi012.stdout: "2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.2-myfoo2" 2023-10-05T19:40:28.960 INFO:teuthology.orchestra.run.smithi012.stdout:] 2023-10-05T19:40:28.960 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:40:28.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.938+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:102 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94690352349664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d09da0 con 0x561ed0d01a70 2023-10-05T19:40:28.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.942+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1432187802 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(102 notify.0 [watch unwatch cookie 94690352349664] v28'152 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2a78048040 con 0x561ed0d01a70 2023-10-05T19:40:28.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.942+0000 7f2a8af9ee40 20 remove_watcher() i=0 2023-10-05T19:40:28.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.942+0000 7f2a8af9ee40 2 removed watcher, disabling cache 2023-10-05T19:40:28.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.942+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94690352353184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d0f680 con 0x561ed0c41bf0 2023-10-05T19:40:28.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.946+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(103 notify.1 [watch unwatch cookie 94690352353184] v28'152 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2a8000a5a0 con 0x561ed0c41bf0 2023-10-05T19:40:28.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.946+0000 7f2a8af9ee40 20 remove_watcher() i=1 2023-10-05T19:40:28.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.946+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:104 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94690352356736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d09da0 con 0x561ed0d063f0 2023-10-05T19:40:28.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.950+0000 7f2a882d2700 1 -- 172.21.15.12:0/1432187802 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(104 notify.2 [watch unwatch cookie 94690352356736] v28'152 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2a7c067040 con 0x561ed0d063f0 2023-10-05T19:40:28.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.950+0000 7f2a8af9ee40 20 remove_watcher() i=2 2023-10-05T19:40:28.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.950+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:105 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94690352354288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0ce3970 con 0x561ed0d01a70 2023-10-05T19:40:28.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.954+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1432187802 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(105 notify.3 [watch unwatch cookie 94690352354288] v28'152 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2a78048040 con 0x561ed0d01a70 2023-10-05T19:40:28.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.954+0000 7f2a8af9ee40 20 remove_watcher() i=3 2023-10-05T19:40:28.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.954+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:106 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94690352359600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d0d490 con 0x561ed0c41bf0 2023-10-05T19:40:28.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.954+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(106 notify.4 [watch unwatch cookie 94690352359600] v28'152 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2a8000a5a0 con 0x561ed0c41bf0 2023-10-05T19:40:28.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.954+0000 7f2a8af9ee40 20 remove_watcher() i=4 2023-10-05T19:40:28.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.954+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:107 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94690352364160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d0d490 con 0x7f2a780420d0 2023-10-05T19:40:28.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.958+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(107 notify.5 [watch unwatch cookie 94690352364160] v28'152 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2a8000a5a0 con 0x7f2a780420d0 2023-10-05T19:40:28.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.958+0000 7f2a8af9ee40 20 remove_watcher() i=5 2023-10-05T19:40:28.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.958+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:108 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94690352368704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d0d490 con 0x7f2a780420d0 2023-10-05T19:40:28.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.962+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(108 notify.6 [watch unwatch cookie 94690352368704] v28'152 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2a8000a5a0 con 0x7f2a780420d0 2023-10-05T19:40:28.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.962+0000 7f2a8af9ee40 20 remove_watcher() i=6 2023-10-05T19:40:28.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.962+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:109 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94690352372624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d0d490 con 0x7f2a780420d0 2023-10-05T19:40:28.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.966+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(109 notify.7 [watch unwatch cookie 94690352372624] v28'152 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2a8000a5a0 con 0x7f2a780420d0 2023-10-05T19:40:28.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.966+0000 7f2a8af9ee40 20 remove_watcher() i=7 2023-10-05T19:40:28.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.966+0000 7f2a5a7e4700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:28.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.966+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:110 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94690352373728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561ed0d0d490 con 0x561ed0c41bf0 2023-10-05T19:40:28.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.970+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(110 data_loggenerations_metadata [watch unwatch cookie 94690352373728] v28'148 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f2a8000a5a0 con 0x561ed0c41bf0 2023-10-05T19:40:28.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.970+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f2a780420d0 msgr2=0x7f2a7803d690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.970+0000 7f2a8af9ee40 1 --2- 172.21.15.12:0/1432187802 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f2a780420d0 0x7f2a7803d690 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.970+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x561ed0c41bf0 msgr2=0x561ed0cff8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.970+0000 7f2a8af9ee40 1 --2- 172.21.15.12:0/1432187802 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x561ed0c41bf0 0x561ed0cff8d0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.970+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x561ed0d01a70 msgr2=0x561ed0d03e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.970+0000 7f2a8af9ee40 1 --2- 172.21.15.12:0/1432187802 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x561ed0d01a70 0x561ed0d03e50 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.970+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x561ed0d063f0 msgr2=0x561ed0d087d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.970+0000 7f2a8af9ee40 1 --2- 172.21.15.12:0/1432187802 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x561ed0d063f0 0x561ed0d087d0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.970+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2a74030c00 msgr2=0x7f2a740330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.970+0000 7f2a8af9ee40 1 --2- 172.21.15.12:0/1432187802 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2a74030c00 0x7f2a740330b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c005a90 tx=0x7f2a7c015000 comp rx=0 tx=0).stop 2023-10-05T19:40:28.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.970+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561ed0c408e0 msgr2=0x561ed0cf3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.970+0000 7f2a892d4700 1 -- 172.21.15.12:0/1432187802 reap_dead start 2023-10-05T19:40:28.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.970+0000 7f2a8af9ee40 1 --2- 172.21.15.12:0/1432187802 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561ed0c408e0 0x561ed0cf3e60 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2a78001820 tx=0x7f2a78016000 comp rx=0 tx=0).stop 2023-10-05T19:40:28.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.970+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 shutdown_connections 2023-10-05T19:40:28.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.970+0000 7f2a8af9ee40 1 --2- 172.21.15.12:0/1432187802 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561ed0c408e0 0x561ed0cf3e60 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.970+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 >> 172.21.15.12:0/1432187802 conn(0x561ed0cc7820 msgr2=0x561ed0ce7710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:28.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.970+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 shutdown_connections 2023-10-05T19:40:28.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.970+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1432187802 wait complete. 2023-10-05T19:40:28.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.970+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1035957305 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x561ed0d8d6b0 msgr2=0x561ed0d8fa90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.970+0000 7f2a8af9ee40 1 --2- 172.21.15.12:0/1035957305 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x561ed0d8d6b0 0x561ed0d8fa90 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.970+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1035957305 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x7f2a7800dc20 msgr2=0x7f2a7804d740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.970+0000 7f2a8af9ee40 1 --2- 172.21.15.12:0/1035957305 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x7f2a7800dc20 0x7f2a7804d740 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.970+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1035957305 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x7f2a80024040 msgr2=0x7f2a80036950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.970+0000 7f2a8af9ee40 1 --2- 172.21.15.12:0/1035957305 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x7f2a80024040 0x7f2a80036950 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.970+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1035957305 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x7f2a7804e040 msgr2=0x7f2a78042950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.970+0000 7f2a8af9ee40 1 --2- 172.21.15.12:0/1035957305 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x7f2a7804e040 0x7f2a78042950 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.970+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1035957305 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f29d0030c50 msgr2=0x7f29d0033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.970+0000 7f2a8af9ee40 1 --2- 172.21.15.12:0/1035957305 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f29d0030c50 0x7f29d0033100 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x561ed0d11150 tx=0x7f2a8000ae90 comp rx=0 tx=0).stop 2023-10-05T19:40:28.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.970+0000 7f2a88ad3700 1 -- 172.21.15.12:0/1035957305 reap_dead start 2023-10-05T19:40:28.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.970+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1035957305 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561ed0d1f5c0 msgr2=0x561ed0d25050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:28.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.970+0000 7f2a8af9ee40 1 --2- 172.21.15.12:0/1035957305 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561ed0d1f5c0 0x561ed0d25050 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c03b000 tx=0x7f2a7c037000 comp rx=0 tx=0).stop 2023-10-05T19:40:28.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.974+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1035957305 shutdown_connections 2023-10-05T19:40:28.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.974+0000 7f2a8af9ee40 1 --2- 172.21.15.12:0/1035957305 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561ed0d1f5c0 0x561ed0d25050 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:28.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.974+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1035957305 >> 172.21.15.12:0/1035957305 conn(0x561ed0d1a980 msgr2=0x561ed0d18a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:28.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.974+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1035957305 shutdown_connections 2023-10-05T19:40:28.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:28.974+0000 7f2a8af9ee40 1 -- 172.21.15.12:0/1035957305 wait complete. 2023-10-05T19:40:28.987 DEBUG:tasks.util.rgw: json result: ['2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.4-myfoo4', 'obj_delete_at_hint.0000000078', 'obj_delete_at_hint.0000000070', 'obj_delete_at_hint.0000000104', 'obj_delete_at_hint.0000000026', 'obj_delete_at_hint.0000000028', 'obj_delete_at_hint.0000000040', 'obj_delete_at_hint.0000000015', 'obj_delete_at_hint.0000000069', 'obj_delete_at_hint.0000000095', 'obj_delete_at_hint.0000000003', 'obj_delete_at_hint.0000000047', 'obj_delete_at_hint.0000000035', 'obj_delete_at_hint.0000000037', '2023-10-05-19--', 'obj_delete_at_hint.0000000024', 'obj_delete_at_hint.0000000000', 'obj_delete_at_hint.0000000031', 'data_loggenerations_metadata', 'obj_delete_at_hint.0000000076', 'obj_delete_at_hint.0000000113', 'obj_delete_at_hint.0000000005', 'obj_delete_at_hint.0000000011', 'obj_delete_at_hint.0000000058', 'obj_delete_at_hint.0000000052', 'obj_delete_at_hint.0000000088', 'obj_delete_at_hint.0000000080', 'obj_delete_at_hint.0000000090', 'obj_delete_at_hint.0000000110', 'obj_delete_at_hint.0000000096', 'obj_delete_at_hint.0000000087', 'obj_delete_at_hint.0000000008', 'obj_delete_at_hint.0000000006', 'obj_delete_at_hint.0000000029', 'obj_delete_at_hint.0000000089', 'obj_delete_at_hint.0000000022', 'obj_delete_at_hint.0000000016', 'obj_delete_at_hint.0000000041', 'obj_delete_at_hint.0000000018', 'obj_delete_at_hint.0000000092', 'obj_delete_at_hint.0000000014', 'obj_delete_at_hint.0000000112', 'obj_delete_at_hint.0000000007', 'obj_delete_at_hint.0000000021', 'obj_delete_at_hint.0000000064', 'obj_delete_at_hint.0000000071', 'obj_delete_at_hint.0000000074', 'obj_delete_at_hint.0000000081', 'obj_delete_at_hint.0000000009', 'obj_delete_at_hint.0000000121', 'obj_delete_at_hint.0000000125', 'obj_delete_at_hint.0000000082', 'obj_delete_at_hint.0000000105', 'obj_delete_at_hint.0000000059', 'obj_delete_at_hint.0000000077', 'obj_delete_at_hint.0000000032', 'obj_delete_at_hint.0000000053', 'obj_delete_at_hint.0000000091', 'obj_delete_at_hint.0000000065', 'obj_delete_at_hint.0000000083', 'obj_delete_at_hint.0000000010', 'obj_delete_at_hint.0000000045', 'obj_delete_at_hint.0000000002', 'obj_delete_at_hint.0000000116', 'obj_delete_at_hint.0000000034', 'obj_delete_at_hint.0000000101', 'obj_delete_at_hint.0000000079', 'obj_delete_at_hint.0000000049', 'obj_delete_at_hint.0000000117', 'obj_delete_at_hint.0000000044', 'obj_delete_at_hint.0000000066', '2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.3-myfoo3', 'obj_delete_at_hint.0000000068', 'obj_delete_at_hint.0000000085', 'obj_delete_at_hint.0000000073', 'obj_delete_at_hint.0000000038', 'obj_delete_at_hint.0000000118', 'obj_delete_at_hint.0000000036', 'obj_delete_at_hint.0000000103', 'obj_delete_at_hint.0000000119', 'obj_delete_at_hint.0000000098', 'obj_delete_at_hint.0000000027', 'obj_delete_at_hint.0000000019', 'obj_delete_at_hint.0000000039', 'obj_delete_at_hint.0000000100', 'obj_delete_at_hint.0000000093', 'obj_delete_at_hint.0000000004', 'obj_delete_at_hint.0000000063', 'obj_delete_at_hint.0000000122', 'obj_delete_at_hint.0000000057', 'obj_delete_at_hint.0000000054', 'obj_delete_at_hint.0000000114', 'data_log.0', 'obj_delete_at_hint.0000000012', 'obj_delete_at_hint.0000000084', 'obj_delete_at_hint.0000000043', 'obj_delete_at_hint.0000000111', 'obj_delete_at_hint.0000000048', 'obj_delete_at_hint.0000000020', 'obj_delete_at_hint.0000000099', 'obj_delete_at_hint.0000000056', 'obj_delete_at_hint.0000000072', '2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1-myfoo', 'obj_delete_at_hint.0000000062', 'obj_delete_at_hint.0000000115', 'obj_delete_at_hint.0000000033', 'obj_delete_at_hint.0000000025', 'obj_delete_at_hint.0000000123', 'obj_delete_at_hint.0000000108', 'obj_delete_at_hint.0000000094', 'obj_delete_at_hint.0000000060', 'obj_delete_at_hint.0000000109', 'obj_delete_at_hint.0000000013', 'obj_delete_at_hint.0000000051', 'obj_delete_at_hint.0000000106', 'obj_delete_at_hint.0000000017', 'obj_delete_at_hint.0000000023', 'obj_delete_at_hint.0000000107', 'obj_delete_at_hint.0000000046', 'obj_delete_at_hint.0000000067', 'obj_delete_at_hint.0000000086', 'obj_delete_at_hint.0000000050', 'obj_delete_at_hint.0000000102', 'obj_delete_at_hint.0000000124', 'obj_delete_at_hint.0000000001', 'obj_delete_at_hint.0000000126', 'obj_delete_at_hint.0000000055', 'obj_delete_at_hint.0000000061', 'obj_delete_at_hint.0000000097', 'obj_delete_at_hint.0000000042', 'obj_delete_at_hint.0000000075', 'obj_delete_at_hint.0000000120', 'obj_delete_at_hint.0000000030', '2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.2-myfoo2'] 2023-10-05T19:40:28.988 INFO:tasks.util.rgw:rgwadmin: client.0 : ['log', 'show', '--object', '2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.4-myfoo4'] 2023-10-05T19:40:28.988 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'log', 'show', '--object', '2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.4-myfoo4'] 2023-10-05T19:40:28.988 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph log show --object 2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.4-myfoo4 2023-10-05T19:40:29.028 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:29.028 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:29.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.018+0000 7fb7c7aa4e40 1 Processor -- start 2023-10-05T19:40:29.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.018+0000 7fb7c7aa4e40 1 -- start start 2023-10-05T19:40:29.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.018+0000 7fb7c7aa4e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55a1186075f0 0x55a1186099d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.018+0000 7fb7c7aa4e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55a11854dbd0 con 0x55a118603010 2023-10-05T19:40:29.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.018+0000 7fb7c7aa4e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x55a11850aee0 con 0x55a1186075f0 2023-10-05T19:40:29.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.018+0000 7fb7c4dd8700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55a1186075f0 0x55a1186099d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.022+0000 7fb7c4dd8700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55a1186075f0 0x55a1186099d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58496/0 (socket says 172.21.15.12:58496) 2023-10-05T19:40:29.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.018+0000 7fb7c55d9700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x55a118603010 0x55a1186033e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:35668/0 (socket says 172.21.15.12:35668) 2023-10-05T19:40:29.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.022+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/1129850930 learned_addr learned my addr 172.21.15.12:0/1129850930 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:29.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.022+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/1129850930 >> v1:172.21.15.8:6789/0 conn(0x55a118603010 legacy=0x55a1186033e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.022+0000 7fb7bffff700 1 -- 172.21.15.12:0/1129850930 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 952414345 0 0) 0x55a11854dbd0 con 0x55a118603010 2023-10-05T19:40:29.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.022+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/1129850930 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a118603c20 con 0x55a1186075f0 2023-10-05T19:40:29.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.022+0000 7fb7c4dd8700 1 --2- 172.21.15.12:0/1129850930 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a1186075f0 0x55a1186099d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb7b000a650 tx=0x7fb7b0005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=854f714caff339e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.022+0000 7fb7bffff700 1 -- 172.21.15.12:0/1129850930 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb7b0008760 con 0x55a1186075f0 2023-10-05T19:40:29.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.022+0000 7fb7bffff700 1 -- 172.21.15.12:0/1129850930 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b00088c0 con 0x55a1186075f0 2023-10-05T19:40:29.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.022+0000 7fb7bffff700 1 -- 172.21.15.12:0/1129850930 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb7b0008bb0 con 0x55a1186075f0 2023-10-05T19:40:29.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.022+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/1129850930 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a1186075f0 msgr2=0x55a1186099d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.022+0000 7fb7c7aa4e40 1 --2- 172.21.15.12:0/1129850930 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a1186075f0 0x55a1186099d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb7b000a650 tx=0x7fb7b0005b10 comp rx=0 tx=0).stop 2023-10-05T19:40:29.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.022+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/1129850930 shutdown_connections 2023-10-05T19:40:29.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.022+0000 7fb7c7aa4e40 1 --2- 172.21.15.12:0/1129850930 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a1186075f0 0x55a1186099d0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.022+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/1129850930 >> 172.21.15.12:0/1129850930 conn(0x55a118576860 msgr2=0x55a118576c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:29.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.022+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/1129850930 shutdown_connections 2023-10-05T19:40:29.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.022+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/1129850930 wait complete. 2023-10-05T19:40:29.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.022+0000 7fb7c7aa4e40 1 Processor -- start 2023-10-05T19:40:29.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.022+0000 7fb7c7aa4e40 1 -- start start 2023-10-05T19:40:29.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.022+0000 7fb7c7aa4e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a118603010 0x55a118622130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.022+0000 7fb7c7aa4e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1184cee30 con 0x55a118603010 2023-10-05T19:40:29.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.022+0000 7fb7c55d9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a118603010 0x55a118622130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.022+0000 7fb7c55d9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a118603010 0x55a118622130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58510/0 (socket says 172.21.15.12:58510) 2023-10-05T19:40:29.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.022+0000 7fb7c55d9700 1 -- 172.21.15.12:0/990797422 learned_addr learned my addr 172.21.15.12:0/990797422 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:29.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.026+0000 7fb7c55d9700 1 -- 172.21.15.12:0/990797422 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b0005db0 con 0x55a118603010 2023-10-05T19:40:29.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.026+0000 7fb7c55d9700 1 --2- 172.21.15.12:0/990797422 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a118603010 0x55a118622130 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8001630 tx=0x7fb7b80069b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd68f10cb3b2c85c server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.026+0000 7fb7beffd700 1 -- 172.21.15.12:0/990797422 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb7b8006320 con 0x55a118603010 2023-10-05T19:40:29.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.026+0000 7fb7beffd700 1 -- 172.21.15.12:0/990797422 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b8013420 con 0x55a118603010 2023-10-05T19:40:29.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.026+0000 7fb7beffd700 1 -- 172.21.15.12:0/990797422 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb7b80135d0 con 0x55a118603010 2023-10-05T19:40:29.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.026+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/990797422 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a118603010 msgr2=0x55a118622130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.026+0000 7fb7c7aa4e40 1 --2- 172.21.15.12:0/990797422 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a118603010 0x55a118622130 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8001630 tx=0x7fb7b80069b0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.026+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/990797422 shutdown_connections 2023-10-05T19:40:29.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.026+0000 7fb7c7aa4e40 1 --2- 172.21.15.12:0/990797422 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a118603010 0x55a118622130 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.026+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/990797422 >> 172.21.15.12:0/990797422 conn(0x55a118576860 msgr2=0x55a11861dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:29.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.026+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/990797422 shutdown_connections 2023-10-05T19:40:29.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.026+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/990797422 wait complete. 2023-10-05T19:40:29.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.026+0000 7fb7c7aa4e40 1 Processor -- start 2023-10-05T19:40:29.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.026+0000 7fb7c7aa4e40 1 -- start start 2023-10-05T19:40:29.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.026+0000 7fb7c7aa4e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a118603010 0x55a118628ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.026+0000 7fb7c7aa4e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1184a1ed0 con 0x55a118603010 2023-10-05T19:40:29.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.026+0000 7fb7c55d9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a118603010 0x55a118628ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.026+0000 7fb7c55d9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a118603010 0x55a118628ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58518/0 (socket says 172.21.15.12:58518) 2023-10-05T19:40:29.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.026+0000 7fb7c55d9700 1 -- 172.21.15.12:0/2437350865 learned_addr learned my addr 172.21.15.12:0/2437350865 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:29.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.026+0000 7fb7c55d9700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b8006c90 con 0x55a118603010 2023-10-05T19:40:29.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.030+0000 7fb7c55d9700 1 --2- 172.21.15.12:0/2437350865 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a118603010 0x55a118628ec0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8001330 tx=0x7fb7b8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.030+0000 7fb7bd7fa700 1 -- 172.21.15.12:0/2437350865 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb7b8006320 con 0x55a118603010 2023-10-05T19:40:29.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.030+0000 7fb7bd7fa700 1 -- 172.21.15.12:0/2437350865 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b8004bc0 con 0x55a118603010 2023-10-05T19:40:29.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.030+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a118629400 con 0x55a118603010 2023-10-05T19:40:29.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.030+0000 7fb7bd7fa700 1 -- 172.21.15.12:0/2437350865 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb7b801e3f0 con 0x55a118603010 2023-10-05T19:40:29.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.030+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a118629840 con 0x55a118603010 2023-10-05T19:40:29.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.030+0000 7fb7c7aa4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.030+0000 7fb7bd7fa700 1 -- 172.21.15.12:0/2437350865 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fb7b801e5f0 con 0x55a118603010 2023-10-05T19:40:29.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.030+0000 7fb7bd7fa700 1 --2- 172.21.15.12:0/2437350865 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fb7a8030c00 0x7fb7a80330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.030+0000 7fb7bd7fa700 1 -- 172.21.15.12:0/2437350865 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7fb7b8045370 con 0x55a118603010 2023-10-05T19:40:29.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.030+0000 7fb7c7aa4e40 1 --2- 172.21.15.12:0/2437350865 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55a118632a00 0x55a118634de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.030+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a118635450 con 0x55a118632a00 2023-10-05T19:40:29.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.030+0000 7fb7c4dd8700 1 --2- 172.21.15.12:0/2437350865 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fb7a8030c00 0x7fb7a80330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.030+0000 7fb7c5dda700 1 --2- 172.21.15.12:0/2437350865 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55a118632a00 0x55a118634de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.034+0000 7fb7c4dd8700 1 --2- 172.21.15.12:0/2437350865 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fb7a8030c00 0x7fb7a80330b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x55a118604060 tx=0x7fb7b0015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.034+0000 7fb7c5dda700 1 --2- 172.21.15.12:0/2437350865 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55a118632a00 0x55a118634de0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.034+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb7c000a500 con 0x55a118632a00 2023-10-05T19:40:29.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.034+0000 7fb7c7aa4e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:29.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.034+0000 7fb7c7aa4e40 20 realm 2023-10-05T19:40:29.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.034+0000 7fb7c7aa4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.034+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a118632030 con 0x55a118632a00 2023-10-05T19:40:29.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.034+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb7c000a500 con 0x55a118632a00 2023-10-05T19:40:29.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.034+0000 7fb7c7aa4e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:29.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.034+0000 7fb7c7aa4e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:29.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.034+0000 7fb7c7aa4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.034+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a118635f30 con 0x55a118632a00 2023-10-05T19:40:29.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.034+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb7c000a500 con 0x55a118632a00 2023-10-05T19:40:29.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.034+0000 7fb7c7aa4e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:29.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.034+0000 7fb7c7aa4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.034+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a118635ae0 con 0x55a118632a00 2023-10-05T19:40:29.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.034+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fb7c000a500 con 0x55a118632a00 2023-10-05T19:40:29.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.034+0000 7fb7c7aa4e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:29.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.034+0000 7fb7c7aa4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.034+0000 7fb7c7aa4e40 1 --2- 172.21.15.12:0/2437350865 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55a118636e80 0x55a118639260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.034+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a1186399a0 con 0x55a118636e80 2023-10-05T19:40:29.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.034+0000 7fb7c55d9700 1 --2- 172.21.15.12:0/2437350865 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55a118636e80 0x55a118639260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.034+0000 7fb7c55d9700 1 --2- 172.21.15.12:0/2437350865 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55a118636e80 0x55a118639260 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.038+0000 7fb7c55d9700 1 -- 172.21.15.12:0/2437350865 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fb7b8024020 con 0x55a118636e80 2023-10-05T19:40:29.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.038+0000 7fb7c7aa4e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:29.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.038+0000 7fb7c7aa4e40 20 searching for the correct realm 2023-10-05T19:40:29.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.038+0000 7fb7c7aa4e40 1 --2- 172.21.15.12:0/2437350865 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55a11863b7d0 0x55a11863dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.038+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863e0f0 con 0x55a11863b7d0 2023-10-05T19:40:29.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.038+0000 7fb7c4dd8700 1 --2- 172.21.15.12:0/2437350865 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55a11863b7d0 0x55a11863dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.038+0000 7fb7c4dd8700 1 --2- 172.21.15.12:0/2437350865 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55a11863b7d0 0x55a11863dbb0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.038+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/2437350865 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fb7b0008d10 con 0x55a11863b7d0 2023-10-05T19:40:29.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.038+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863e0f0 con 0x55a118636e80 2023-10-05T19:40:29.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.042+0000 7fb7c55d9700 1 -- 172.21.15.12:0/2437350865 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7b8024020 con 0x55a118636e80 2023-10-05T19:40:29.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.042+0000 7fb7c55d9700 1 --2- 172.21.15.12:0/2437350865 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fb7b8011c00 0x7fb7b8010360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.042+0000 7fb7c55d9700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863e0f0 con 0x7fb7b8011c00 2023-10-05T19:40:29.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.042+0000 7fb7c5dda700 1 --2- 172.21.15.12:0/2437350865 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fb7b8011c00 0x7fb7b8010360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.042+0000 7fb7c5dda700 1 --2- 172.21.15.12:0/2437350865 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fb7b8011c00 0x7fb7b8010360 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.042+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7c0014040 con 0x7fb7b8011c00 2023-10-05T19:40:29.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.042+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863e0f0 con 0x55a118632a00 2023-10-05T19:40:29.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.042+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7c0014040 con 0x55a118632a00 2023-10-05T19:40:29.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.042+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863e0f0 con 0x55a118632a00 2023-10-05T19:40:29.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.042+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7c0014040 con 0x55a118632a00 2023-10-05T19:40:29.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.042+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863e0f0 con 0x7fb7b8011c00 2023-10-05T19:40:29.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.046+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7c0014040 con 0x7fb7b8011c00 2023-10-05T19:40:29.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.046+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863e0f0 con 0x7fb7b8011c00 2023-10-05T19:40:29.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.046+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7c0014040 con 0x7fb7b8011c00 2023-10-05T19:40:29.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.046+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863e0f0 con 0x55a118636e80 2023-10-05T19:40:29.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.046+0000 7fb7c55d9700 1 -- 172.21.15.12:0/2437350865 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7b8024020 con 0x55a118636e80 2023-10-05T19:40:29.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.046+0000 7fb7c55d9700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863e0f0 con 0x55a11863b7d0 2023-10-05T19:40:29.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.046+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/2437350865 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7b0008d10 con 0x55a11863b7d0 2023-10-05T19:40:29.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.046+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863e0f0 con 0x55a118632a00 2023-10-05T19:40:29.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.046+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7c0014040 con 0x55a118632a00 2023-10-05T19:40:29.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.046+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863e0f0 con 0x7fb7b8011c00 2023-10-05T19:40:29.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.050+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7c0014040 con 0x7fb7b8011c00 2023-10-05T19:40:29.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.050+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863e0f0 con 0x55a118632a00 2023-10-05T19:40:29.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.050+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7c0014040 con 0x55a118632a00 2023-10-05T19:40:29.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.050+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863e0f0 con 0x55a118636e80 2023-10-05T19:40:29.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.050+0000 7fb7c55d9700 1 -- 172.21.15.12:0/2437350865 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7b8024020 con 0x55a118636e80 2023-10-05T19:40:29.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.050+0000 7fb7c55d9700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863e0f0 con 0x55a11863b7d0 2023-10-05T19:40:29.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.050+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/2437350865 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7b0008d10 con 0x55a11863b7d0 2023-10-05T19:40:29.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.050+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863e0f0 con 0x55a11863b7d0 2023-10-05T19:40:29.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.050+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/2437350865 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7b0008d10 con 0x55a11863b7d0 2023-10-05T19:40:29.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.050+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863e0f0 con 0x7fb7b8011c00 2023-10-05T19:40:29.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.054+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7c0014040 con 0x7fb7b8011c00 2023-10-05T19:40:29.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.054+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863e0f0 con 0x55a11863b7d0 2023-10-05T19:40:29.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.054+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/2437350865 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7b0008d10 con 0x55a11863b7d0 2023-10-05T19:40:29.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.054+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863e0f0 con 0x55a11863b7d0 2023-10-05T19:40:29.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.054+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/2437350865 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7b0008d10 con 0x55a11863b7d0 2023-10-05T19:40:29.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.054+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863e0f0 con 0x55a118632a00 2023-10-05T19:40:29.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.054+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7c0014040 con 0x55a118632a00 2023-10-05T19:40:29.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.054+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863e0f0 con 0x55a118632a00 2023-10-05T19:40:29.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.054+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7c0014040 con 0x55a118632a00 2023-10-05T19:40:29.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.054+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863e0f0 con 0x7fb7b8011c00 2023-10-05T19:40:29.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.058+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7c0014040 con 0x7fb7b8011c00 2023-10-05T19:40:29.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.058+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863e0f0 con 0x55a11863b7d0 2023-10-05T19:40:29.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.058+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/2437350865 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7b0008d10 con 0x55a11863b7d0 2023-10-05T19:40:29.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.058+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863e0f0 con 0x55a118636e80 2023-10-05T19:40:29.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.058+0000 7fb7c55d9700 1 -- 172.21.15.12:0/2437350865 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7b8024020 con 0x55a118636e80 2023-10-05T19:40:29.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.058+0000 7fb7c55d9700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863e0f0 con 0x55a118632a00 2023-10-05T19:40:29.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.058+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7c0014040 con 0x55a118632a00 2023-10-05T19:40:29.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.058+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863e0f0 con 0x7fb7b8011c00 2023-10-05T19:40:29.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.062+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7c0014040 con 0x7fb7b8011c00 2023-10-05T19:40:29.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.062+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863e0f0 con 0x55a11863b7d0 2023-10-05T19:40:29.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.062+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/2437350865 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7b0008d10 con 0x55a11863b7d0 2023-10-05T19:40:29.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.062+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863e0f0 con 0x7fb7b8011c00 2023-10-05T19:40:29.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.062+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7c0014040 con 0x7fb7b8011c00 2023-10-05T19:40:29.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.062+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863e0f0 con 0x55a118636e80 2023-10-05T19:40:29.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.062+0000 7fb7c55d9700 1 -- 172.21.15.12:0/2437350865 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7b8024020 con 0x55a118636e80 2023-10-05T19:40:29.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.062+0000 7fb7c55d9700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863e0f0 con 0x55a11863b7d0 2023-10-05T19:40:29.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.062+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/2437350865 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7b0008d10 con 0x55a11863b7d0 2023-10-05T19:40:29.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.062+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863e0f0 con 0x7fb7b8011c00 2023-10-05T19:40:29.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.066+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7c0014040 con 0x7fb7b8011c00 2023-10-05T19:40:29.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.066+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863e0f0 con 0x55a118632a00 2023-10-05T19:40:29.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.066+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fb7c0014040 con 0x55a118632a00 2023-10-05T19:40:29.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.066+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863e0f0 con 0x55a118636e80 2023-10-05T19:40:29.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.066+0000 7fb7c55d9700 1 -- 172.21.15.12:0/2437350865 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fb7b8024020 con 0x55a118636e80 2023-10-05T19:40:29.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.066+0000 7fb7c7aa4e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:29.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.066+0000 7fb7c7aa4e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:29.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.066+0000 7fb7c7aa4e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:29.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.066+0000 7fb7c7aa4e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:29.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.066+0000 7fb7c7aa4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.066+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863a5a0 con 0x55a118632a00 2023-10-05T19:40:29.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.066+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb7c0014040 con 0x55a118632a00 2023-10-05T19:40:29.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.066+0000 7fb7c7aa4e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:29.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.066+0000 7fb7c7aa4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.066+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863aef0 con 0x55a118632a00 2023-10-05T19:40:29.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.070+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fb7c0014040 con 0x55a118632a00 2023-10-05T19:40:29.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.070+0000 7fb7c7aa4e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:29.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.070+0000 7fb7c7aa4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.070+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863a190 con 0x55a11863b7d0 2023-10-05T19:40:29.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.070+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/2437350865 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fb7b0008d10 con 0x55a11863b7d0 2023-10-05T19:40:29.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.070+0000 7fb7c7aa4e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:29.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.070+0000 7fb7c7aa4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.070+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863e9e0 con 0x55a11863b7d0 2023-10-05T19:40:29.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.070+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/2437350865 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7fb7b0008d10 con 0x55a11863b7d0 2023-10-05T19:40:29.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.070+0000 7fb7c7aa4e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:29.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.070+0000 7fb7c7aa4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.070+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863ed90 con 0x55a118632a00 2023-10-05T19:40:29.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.070+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb7c0014040 con 0x55a118632a00 2023-10-05T19:40:29.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.070+0000 7fb7c7aa4e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:29.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.070+0000 7fb7c7aa4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.070+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863f140 con 0x55a118632a00 2023-10-05T19:40:29.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.070+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fb7c0014040 con 0x55a118632a00 2023-10-05T19:40:29.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.070+0000 7fb7c7aa4e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:29.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.070+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a118640150 con 0x55a11863b7d0 2023-10-05T19:40:29.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.074+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/2437350865 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fb7b0008d10 con 0x55a11863b7d0 2023-10-05T19:40:29.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.074+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a118640150 con 0x55a118636e80 2023-10-05T19:40:29.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.074+0000 7fb7c55d9700 1 -- 172.21.15.12:0/2437350865 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7b8024020 con 0x55a118636e80 2023-10-05T19:40:29.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.074+0000 7fb7c55d9700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a118640150 con 0x7fb7b8011c00 2023-10-05T19:40:29.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.074+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7c0014040 con 0x7fb7b8011c00 2023-10-05T19:40:29.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.074+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863ed90 con 0x55a118632a00 2023-10-05T19:40:29.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.078+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7c0014040 con 0x55a118632a00 2023-10-05T19:40:29.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.078+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863ed90 con 0x55a118632a00 2023-10-05T19:40:29.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.078+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7c0014040 con 0x55a118632a00 2023-10-05T19:40:29.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.078+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863ed90 con 0x7fb7b8011c00 2023-10-05T19:40:29.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.078+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7c0014040 con 0x7fb7b8011c00 2023-10-05T19:40:29.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.078+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863ed90 con 0x7fb7b8011c00 2023-10-05T19:40:29.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.078+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7c0014040 con 0x7fb7b8011c00 2023-10-05T19:40:29.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.078+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863ed90 con 0x55a118636e80 2023-10-05T19:40:29.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.078+0000 7fb7c55d9700 1 -- 172.21.15.12:0/2437350865 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7b8024020 con 0x55a118636e80 2023-10-05T19:40:29.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.078+0000 7fb7c55d9700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863ed90 con 0x55a11863b7d0 2023-10-05T19:40:29.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.082+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/2437350865 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7b0008d10 con 0x55a11863b7d0 2023-10-05T19:40:29.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.082+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863ed90 con 0x55a118632a00 2023-10-05T19:40:29.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.082+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7c0014040 con 0x55a118632a00 2023-10-05T19:40:29.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.082+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863ed90 con 0x7fb7b8011c00 2023-10-05T19:40:29.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.082+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7c0014040 con 0x7fb7b8011c00 2023-10-05T19:40:29.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.082+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863ed90 con 0x55a118632a00 2023-10-05T19:40:29.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.082+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7c0014040 con 0x55a118632a00 2023-10-05T19:40:29.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.082+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863ed90 con 0x55a118636e80 2023-10-05T19:40:29.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.082+0000 7fb7c55d9700 1 -- 172.21.15.12:0/2437350865 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7b8024020 con 0x55a118636e80 2023-10-05T19:40:29.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.082+0000 7fb7c55d9700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863ed90 con 0x55a11863b7d0 2023-10-05T19:40:29.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.086+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/2437350865 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7b0008d10 con 0x55a11863b7d0 2023-10-05T19:40:29.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.086+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863ed90 con 0x55a11863b7d0 2023-10-05T19:40:29.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.086+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/2437350865 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7b0008d10 con 0x55a11863b7d0 2023-10-05T19:40:29.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.086+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863ed90 con 0x7fb7b8011c00 2023-10-05T19:40:29.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.086+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7c0014040 con 0x7fb7b8011c00 2023-10-05T19:40:29.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.086+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863ed90 con 0x55a11863b7d0 2023-10-05T19:40:29.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.086+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/2437350865 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7b0008d10 con 0x55a11863b7d0 2023-10-05T19:40:29.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.086+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863ed90 con 0x55a11863b7d0 2023-10-05T19:40:29.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.090+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/2437350865 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7b0008d10 con 0x55a11863b7d0 2023-10-05T19:40:29.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.090+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863ed90 con 0x55a118632a00 2023-10-05T19:40:29.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.090+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7c0014040 con 0x55a118632a00 2023-10-05T19:40:29.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.090+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863ed90 con 0x55a118632a00 2023-10-05T19:40:29.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.090+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7c0014040 con 0x55a118632a00 2023-10-05T19:40:29.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.090+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863ed90 con 0x7fb7b8011c00 2023-10-05T19:40:29.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.090+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7c0014040 con 0x7fb7b8011c00 2023-10-05T19:40:29.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.090+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863ed90 con 0x55a11863b7d0 2023-10-05T19:40:29.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.090+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/2437350865 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7b0008d10 con 0x55a11863b7d0 2023-10-05T19:40:29.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.090+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863ed90 con 0x55a118636e80 2023-10-05T19:40:29.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.094+0000 7fb7c55d9700 1 -- 172.21.15.12:0/2437350865 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7b8024020 con 0x55a118636e80 2023-10-05T19:40:29.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.094+0000 7fb7c55d9700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863ed90 con 0x55a118632a00 2023-10-05T19:40:29.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.094+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7c0014040 con 0x55a118632a00 2023-10-05T19:40:29.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.094+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863ed90 con 0x7fb7b8011c00 2023-10-05T19:40:29.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.094+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7c0014040 con 0x7fb7b8011c00 2023-10-05T19:40:29.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.094+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863ed90 con 0x55a11863b7d0 2023-10-05T19:40:29.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.094+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/2437350865 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7b0008d10 con 0x55a11863b7d0 2023-10-05T19:40:29.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.094+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863ed90 con 0x7fb7b8011c00 2023-10-05T19:40:29.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.094+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7c0014040 con 0x7fb7b8011c00 2023-10-05T19:40:29.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.094+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863ed90 con 0x55a118636e80 2023-10-05T19:40:29.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.098+0000 7fb7c55d9700 1 -- 172.21.15.12:0/2437350865 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7b8024020 con 0x55a118636e80 2023-10-05T19:40:29.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.098+0000 7fb7c55d9700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863ed90 con 0x55a11863b7d0 2023-10-05T19:40:29.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.098+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/2437350865 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7b0008d10 con 0x55a11863b7d0 2023-10-05T19:40:29.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.098+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863ed90 con 0x7fb7b8011c00 2023-10-05T19:40:29.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.098+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb7c0014040 con 0x7fb7b8011c00 2023-10-05T19:40:29.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.098+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863ed90 con 0x55a118632a00 2023-10-05T19:40:29.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.098+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fb7c0014040 con 0x55a118632a00 2023-10-05T19:40:29.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.098+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863ed90 con 0x55a118636e80 2023-10-05T19:40:29.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.102+0000 7fb7c55d9700 1 -- 172.21.15.12:0/2437350865 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fb7b8024020 con 0x55a118636e80 2023-10-05T19:40:29.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.102+0000 7fb7c7aa4e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:29.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.102+0000 7fb7c7aa4e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:29.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.102+0000 7fb7c7aa4e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:29.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.102+0000 7fb7c7aa4e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:29.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.102+0000 7fb7c7aa4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.102+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863f140 con 0x55a118632a00 2023-10-05T19:40:29.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.102+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fb7c0014040 con 0x55a118632a00 2023-10-05T19:40:29.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.102+0000 7fb7c7aa4e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:29.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.102+0000 7fb7c7aa4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.102+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863f4f0 con 0x55a118636e80 2023-10-05T19:40:29.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.102+0000 7fb7c55d9700 1 -- 172.21.15.12:0/2437350865 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fb7b8024020 con 0x55a118636e80 2023-10-05T19:40:29.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.102+0000 7fb7c7aa4e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:29.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.102+0000 7fb7c7aa4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.102+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863fd90 con 0x55a118632a00 2023-10-05T19:40:29.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.102+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fb7c0014040 con 0x55a118632a00 2023-10-05T19:40:29.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.102+0000 7fb7c7aa4e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:29.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.102+0000 7fb7c7aa4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.102+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a11863fd90 con 0x55a11863b7d0 2023-10-05T19:40:29.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.106+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/2437350865 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fb7b0008d10 con 0x55a11863b7d0 2023-10-05T19:40:29.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.106+0000 7fb7c7aa4e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:29.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.106+0000 7fb7c7aa4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.106+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a118640630 con 0x7fb7b8011c00 2023-10-05T19:40:29.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.106+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7fb7c0014040 con 0x7fb7b8011c00 2023-10-05T19:40:29.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.106+0000 7fb7c7aa4e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:29.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.106+0000 7fb7c7aa4e40 20 zone default found 2023-10-05T19:40:29.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.106+0000 7fb7c7aa4e40 4 Realm: () 2023-10-05T19:40:29.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.106+0000 7fb7c7aa4e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:29.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.106+0000 7fb7c7aa4e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:29.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.106+0000 7fb7c7aa4e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:29.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.106+0000 7fb7c7aa4e40 20 zonegroup default 2023-10-05T19:40:29.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.106+0000 7fb7c7aa4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.106+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a118640630 con 0x7fb7b8011c00 2023-10-05T19:40:29.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.106+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fb7c0014040 con 0x7fb7b8011c00 2023-10-05T19:40:29.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.106+0000 7fb7c7aa4e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:29.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.106+0000 7fb7c7aa4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.106+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a118641fb0 con 0x55a11863b7d0 2023-10-05T19:40:29.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.106+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/2437350865 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fb7b0008d10 con 0x55a11863b7d0 2023-10-05T19:40:29.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.106+0000 7fb7c7aa4e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:29.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.106+0000 7fb7c7aa4e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.106+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a118642360 con 0x55a11863b7d0 2023-10-05T19:40:29.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.106+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/2437350865 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fb7b0008d10 con 0x55a11863b7d0 2023-10-05T19:40:29.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.106+0000 7fb7c7aa4e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:29.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.106+0000 7fb7c7aa4e40 20 started sync module instance, tier type = 2023-10-05T19:40:29.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.106+0000 7fb7c7aa4e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:29.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.106+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a118641670 con 0x55a118636e80 2023-10-05T19:40:29.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.110+0000 7fb7c55d9700 1 -- 172.21.15.12:0/2437350865 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'153 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb7b8024020 con 0x55a118636e80 2023-10-05T19:40:29.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.110+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94150387306880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a118642710 con 0x55a118636e80 2023-10-05T19:40:29.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.110+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a118642fb0 con 0x55a118632a00 2023-10-05T19:40:29.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.114+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'153 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb7c0014040 con 0x55a118632a00 2023-10-05T19:40:29.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.114+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94150387313344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a118643400 con 0x55a118632a00 2023-10-05T19:40:29.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.114+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a118643fc0 con 0x55a11863b7d0 2023-10-05T19:40:29.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.114+0000 7fb7c55d9700 1 -- 172.21.15.12:0/2437350865 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94150387306880] v28'154 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb7b8024020 con 0x55a118636e80 2023-10-05T19:40:29.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.118+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94150387313344] v28'154 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb7c0014040 con 0x55a118632a00 2023-10-05T19:40:29.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.118+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/2437350865 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'153 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb7b0008d10 con 0x55a11863b7d0 2023-10-05T19:40:29.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.118+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94150387309168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a118643400 con 0x55a11863b7d0 2023-10-05T19:40:29.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.118+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a1186448b0 con 0x55a118636e80 2023-10-05T19:40:29.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.118+0000 7fb7c55d9700 1 -- 172.21.15.12:0/2437350865 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'153 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb7b8024020 con 0x55a118636e80 2023-10-05T19:40:29.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.118+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94150387319744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a118645150 con 0x55a118636e80 2023-10-05T19:40:29.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.118+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a118645cb0 con 0x55a118632a00 2023-10-05T19:40:29.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.122+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/2437350865 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94150387309168] v28'154 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb7b0008d10 con 0x55a11863b7d0 2023-10-05T19:40:29.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.122+0000 7fb7c55d9700 1 -- 172.21.15.12:0/2437350865 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94150387319744] v28'154 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb7b8024020 con 0x55a118636e80 2023-10-05T19:40:29.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.122+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'153 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb7c0014040 con 0x55a118632a00 2023-10-05T19:40:29.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.122+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94150387314448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a118644c60 con 0x55a118632a00 2023-10-05T19:40:29.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.122+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a1186465a0 con 0x7fb7b8011c00 2023-10-05T19:40:29.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.126+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'153 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb7c0014040 con 0x7fb7b8011c00 2023-10-05T19:40:29.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.126+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94150387326688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a118644c60 con 0x7fb7b8011c00 2023-10-05T19:40:29.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.126+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a118647000 con 0x7fb7b8011c00 2023-10-05T19:40:29.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.126+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94150387314448] v28'154 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb7c0014040 con 0x55a118632a00 2023-10-05T19:40:29.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.130+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(95 notify.5 [watch watch cookie 94150387326688] v28'154 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb7c0014040 con 0x7fb7b8011c00 2023-10-05T19:40:29.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.130+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(96 notify.6 [create] v28'153 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb7c0014040 con 0x7fb7b8011c00 2023-10-05T19:40:29.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.130+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94150387327792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a118646ad0 con 0x7fb7b8011c00 2023-10-05T19:40:29.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.130+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a1186474e0 con 0x7fb7b8011c00 2023-10-05T19:40:29.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.130+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(98 notify.7 [create] v28'153 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb7c0014040 con 0x7fb7b8011c00 2023-10-05T19:40:29.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.130+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94150387331056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a118647930 con 0x7fb7b8011c00 2023-10-05T19:40:29.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.130+0000 7fb7c7aa4e40 20 add_watcher() i=0 2023-10-05T19:40:29.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.130+0000 7fb7c7aa4e40 20 add_watcher() i=1 2023-10-05T19:40:29.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.130+0000 7fb7c7aa4e40 20 add_watcher() i=2 2023-10-05T19:40:29.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.130+0000 7fb7c7aa4e40 20 add_watcher() i=3 2023-10-05T19:40:29.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.130+0000 7fb7c7aa4e40 20 add_watcher() i=4 2023-10-05T19:40:29.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.130+0000 7fb7c7aa4e40 20 add_watcher() i=5 2023-10-05T19:40:29.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.130+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(97 notify.6 [watch watch cookie 94150387327792] v28'154 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb7c0014040 con 0x7fb7b8011c00 2023-10-05T19:40:29.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.130+0000 7fb7c7aa4e40 20 add_watcher() i=6 2023-10-05T19:40:29.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.134+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94150387331056] v28'154 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb7c0014040 con 0x7fb7b8011c00 2023-10-05T19:40:29.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.134+0000 7fb7c7aa4e40 20 add_watcher() i=7 2023-10-05T19:40:29.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.134+0000 7fb7c7aa4e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:29.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.134+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a118645780 con 0x55a118632a00 2023-10-05T19:40:29.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.134+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fb7c0014040 con 0x55a118632a00 2023-10-05T19:40:29.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.134+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94150387332160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a118647d10 con 0x55a118632a00 2023-10-05T19:40:29.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.138+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94150387332160] v28'149 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fb7c0014040 con 0x55a118632a00 2023-10-05T19:40:29.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.138+0000 7fb7967e4700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:29.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.138+0000 7fb7c7aa4e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:29.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.138+0000 7fb7c7aa4e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:29.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.138+0000 7fb7c7aa4e40 1 Processor -- start 2023-10-05T19:40:29.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.138+0000 7fb7c7aa4e40 1 -- start start 2023-10-05T19:40:29.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.138+0000 7fb7c7aa4e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a118654a70 0x55a118654e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.138+0000 7fb7c7aa4e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a118655380 con 0x55a118654a70 2023-10-05T19:40:29.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.138+0000 7fb7c4dd8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a118654a70 0x55a118654e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.138+0000 7fb7c4dd8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a118654a70 0x55a118654e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58532/0 (socket says 172.21.15.12:58532) 2023-10-05T19:40:29.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.138+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/251332906 learned_addr learned my addr 172.21.15.12:0/251332906 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:29.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.138+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/251332906 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1186554c0 con 0x55a118654a70 2023-10-05T19:40:29.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.138+0000 7fb7c4dd8700 1 --2- 172.21.15.12:0/251332906 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a118654a70 0x55a118654e40 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x55a118604090 tx=0x7fb7b0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6de2803eab55c1b server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.138+0000 7fb795fe3700 1 -- 172.21.15.12:0/251332906 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb7b0012e90 con 0x55a118654a70 2023-10-05T19:40:29.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.138+0000 7fb795fe3700 1 -- 172.21.15.12:0/251332906 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b0036040 con 0x55a118654a70 2023-10-05T19:40:29.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.138+0000 7fb795fe3700 1 -- 172.21.15.12:0/251332906 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb7b0030800 con 0x55a118654a70 2023-10-05T19:40:29.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.142+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/251332906 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a118654a70 msgr2=0x55a118654e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.142+0000 7fb7c7aa4e40 1 --2- 172.21.15.12:0/251332906 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a118654a70 0x55a118654e40 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x55a118604090 tx=0x7fb7b0012000 comp rx=0 tx=0).stop 2023-10-05T19:40:29.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.142+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/251332906 shutdown_connections 2023-10-05T19:40:29.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.142+0000 7fb7c7aa4e40 1 --2- 172.21.15.12:0/251332906 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a118654a70 0x55a118654e40 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.142+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/251332906 >> 172.21.15.12:0/251332906 conn(0x55a11864fe30 msgr2=0x55a11864dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:29.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.142+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/251332906 shutdown_connections 2023-10-05T19:40:29.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.142+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/251332906 wait complete. 2023-10-05T19:40:29.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.142+0000 7fb7c7aa4e40 1 Processor -- start 2023-10-05T19:40:29.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.142+0000 7fb7c7aa4e40 1 -- start start 2023-10-05T19:40:29.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.142+0000 7fb7c7aa4e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a118654a70 0x55a11865a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.142+0000 7fb7c7aa4e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a11865b550 con 0x55a118654a70 2023-10-05T19:40:29.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.142+0000 7fb7c4dd8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a118654a70 0x55a11865a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.142+0000 7fb7c4dd8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a118654a70 0x55a11865a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58538/0 (socket says 172.21.15.12:58538) 2023-10-05T19:40:29.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.142+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/4043093390 learned_addr learned my addr 172.21.15.12:0/4043093390 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:29.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.146+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/4043093390 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a118603c20 con 0x55a118654a70 2023-10-05T19:40:29.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.146+0000 7fb7c4dd8700 1 --2- 172.21.15.12:0/4043093390 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a118654a70 0x55a11865a550 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb7b00129f0 tx=0x7fb7b0037000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.146+0000 7fb707fff700 1 -- 172.21.15.12:0/4043093390 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb7b0005460 con 0x55a118654a70 2023-10-05T19:40:29.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.146+0000 7fb707fff700 1 -- 172.21.15.12:0/4043093390 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b0027b60 con 0x55a118654a70 2023-10-05T19:40:29.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.146+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/4043093390 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a11865ab50 con 0x55a118654a70 2023-10-05T19:40:29.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.146+0000 7fb707fff700 1 -- 172.21.15.12:0/4043093390 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb7b0027d50 con 0x55a118654a70 2023-10-05T19:40:29.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.146+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/4043093390 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a11865aff0 con 0x55a118654a70 2023-10-05T19:40:29.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.146+0000 7fb707fff700 1 -- 172.21.15.12:0/4043093390 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fb7b0055440 con 0x55a118654a70 2023-10-05T19:40:29.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.146+0000 7fb707fff700 1 --2- 172.21.15.12:0/4043093390 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fb70c030c50 0x7fb70c033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.146+0000 7fb707fff700 1 -- 172.21.15.12:0/4043093390 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7fb7b004dab0 con 0x55a118654a70 2023-10-05T19:40:29.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.146+0000 7fb7c5dda700 1 --2- 172.21.15.12:0/4043093390 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fb70c030c50 0x7fb70c033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.146+0000 7fb7c7aa4e40 5 note: GC not initialized 2023-10-05T19:40:29.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.150+0000 7fb7c5dda700 1 --2- 172.21.15.12:0/4043093390 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fb70c030c50 0x7fb70c033100 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x55a1186478d0 tx=0x7fb7c000ab80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.150+0000 7fb7057fa700 20 reqs_thread_entry: start 2023-10-05T19:40:29.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.150+0000 7fb6feffd700 20 reqs_thread_entry: start 2023-10-05T19:40:29.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.150+0000 7fb7c7aa4e40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:29.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.150+0000 7fb7c7aa4e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:29.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.150+0000 7fb6fcff9700 20 reqs_thread_entry: start 2023-10-05T19:40:29.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.150+0000 7fb7c7aa4e40 10 log_show_next pos 0 bl 0 off 0 eof 0 2023-10-05T19:40:29.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.150+0000 7fb7c7aa4e40 1 --2- 172.21.15.12:0/4043093390 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55a1186c1b00 0x55a1186c3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.150+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/4043093390 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:1 6.0 6:0018ba17:::2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.4-myfoo4:head [read 0~1048576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a1186c4420 con 0x55a1186c1b00 2023-10-05T19:40:29.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.150+0000 7fb6fd7fa700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:59 2023 2023-10-05T19:40:29.162 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:29.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.150+0000 7fb7c55d9700 1 --2- 172.21.15.12:0/4043093390 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55a1186c1b00 0x55a1186c3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.154+0000 7fb7c55d9700 1 --2- 172.21.15.12:0/4043093390 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55a1186c1b00 0x55a1186c3ee0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.154+0000 7fb7c55d9700 1 -- 172.21.15.12:0/4043093390 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(1 2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.4-myfoo4 [read 0~850 out=850b] v0'0 uv28 ondisk = 0) v8 ==== 208+0+850 (crc 0 0 0) 0x7fb7b8024020 con 0x55a1186c1b00 2023-10-05T19:40:29.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.154+0000 7fb7c7aa4e40 10 read 850 2023-10-05T19:40:29.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.154+0000 7fb7c7aa4e40 10 log_show_next pos 850 bl 850 off 419 eof 1 2023-10-05T19:40:29.164 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:40:29.164 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket_id": "93ac49f4-ff49-4f99-811b-6453b28decd5.4221.4", 2023-10-05T19:40:29.164 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket_owner": "foo", 2023-10-05T19:40:29.164 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo4", 2023-10-05T19:40:29.164 INFO:teuthology.orchestra.run.smithi012.stdout: "log_entries": [ 2023-10-05T19:40:29.164 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:29.165 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo4", 2023-10-05T19:40:29.165 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:40:23.375112Z", 2023-10-05T19:40:29.165 INFO:teuthology.orchestra.run.smithi012.stdout: "time_local": "2023-10-05T19:40:23.375112+0000", 2023-10-05T19:40:29.165 INFO:teuthology.orchestra.run.smithi012.stdout: "remote_addr": "172.21.0.51", 2023-10-05T19:40:29.165 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:40:29.165 INFO:teuthology.orchestra.run.smithi012.stdout: "operation": "create_bucket", 2023-10-05T19:40:29.165 INFO:teuthology.orchestra.run.smithi012.stdout: "uri": "PUT /myfoo4/ HTTP/1.1", 2023-10-05T19:40:29.165 INFO:teuthology.orchestra.run.smithi012.stdout: "http_status": "200", 2023-10-05T19:40:29.165 INFO:teuthology.orchestra.run.smithi012.stdout: "error_code": "", 2023-10-05T19:40:29.165 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:40:29.165 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:40:29.165 INFO:teuthology.orchestra.run.smithi012.stdout: "object_size": 0, 2023-10-05T19:40:29.166 INFO:teuthology.orchestra.run.smithi012.stdout: "total_time": 19, 2023-10-05T19:40:29.166 INFO:teuthology.orchestra.run.smithi012.stdout: "user_agent": "Boto/2.49.0 Python/3.8.10 Linux/5.4.0-163-generic", 2023-10-05T19:40:29.166 INFO:teuthology.orchestra.run.smithi012.stdout: "referrer": "", 2023-10-05T19:40:29.166 INFO:teuthology.orchestra.run.smithi012.stdout: "trans_id": "tx00000dd2d8159385ba3bb-00651f1127-107b-default", 2023-10-05T19:40:29.166 INFO:teuthology.orchestra.run.smithi012.stdout: "authentication_type": "Local", 2023-10-05T19:40:29.166 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key_id": "9te6NH5mcdcq0Tc5i8i1", 2023-10-05T19:40:29.166 INFO:teuthology.orchestra.run.smithi012.stdout: "temp_url": false 2023-10-05T19:40:29.166 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:29.166 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:29.166 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo4", 2023-10-05T19:40:29.166 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:40:23.599106Z", 2023-10-05T19:40:29.167 INFO:teuthology.orchestra.run.smithi012.stdout: "time_local": "2023-10-05T19:40:23.599106+0000", 2023-10-05T19:40:29.167 INFO:teuthology.orchestra.run.smithi012.stdout: "remote_addr": "172.21.0.51", 2023-10-05T19:40:29.167 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:40:29.167 INFO:teuthology.orchestra.run.smithi012.stdout: "operation": "delete_bucket", 2023-10-05T19:40:29.167 INFO:teuthology.orchestra.run.smithi012.stdout: "uri": "DELETE /myfoo4/ HTTP/1.1", 2023-10-05T19:40:29.167 INFO:teuthology.orchestra.run.smithi012.stdout: "http_status": "204", 2023-10-05T19:40:29.167 INFO:teuthology.orchestra.run.smithi012.stdout: "error_code": "NoContent", 2023-10-05T19:40:29.167 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:40:29.167 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:40:29.167 INFO:teuthology.orchestra.run.smithi012.stdout: "object_size": 0, 2023-10-05T19:40:29.167 INFO:teuthology.orchestra.run.smithi012.stdout: "total_time": 71, 2023-10-05T19:40:29.167 INFO:teuthology.orchestra.run.smithi012.stdout: "user_agent": "Boto/2.49.0 Python/3.8.10 Linux/5.4.0-163-generic", 2023-10-05T19:40:29.167 INFO:teuthology.orchestra.run.smithi012.stdout: "referrer": "", 2023-10-05T19:40:29.168 INFO:teuthology.orchestra.run.smithi012.stdout: "trans_id": "tx000005364c9e12e71da2c-00651f1127-107b-default", 2023-10-05T19:40:29.168 INFO:teuthology.orchestra.run.smithi012.stdout: "authentication_type": "Local", 2023-10-05T19:40:29.168 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key_id": "9te6NH5mcdcq0Tc5i8i1", 2023-10-05T19:40:29.168 INFO:teuthology.orchestra.run.smithi012.stdout: "temp_url": false 2023-10-05T19:40:29.168 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:29.168 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:29.168 INFO:teuthology.orchestra.run.smithi012.stdout: "log_sum": { 2023-10-05T19:40:29.168 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:40:29.168 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:40:29.168 INFO:teuthology.orchestra.run.smithi012.stdout: "total_time": 90, 2023-10-05T19:40:29.168 INFO:teuthology.orchestra.run.smithi012.stdout: "total_entries": 2 2023-10-05T19:40:29.168 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:29.169 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:40:29.169 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:40:29.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.154+0000 7fb7c7aa4e40 10 log_show_next pos 850 bl 850 off 850 eof 1 2023-10-05T19:40:29.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.154+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:102 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94150387306880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a118681130 con 0x55a118636e80 2023-10-05T19:40:29.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.158+0000 7fb7c55d9700 1 -- 172.21.15.12:0/2437350865 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(102 notify.0 [watch unwatch cookie 94150387306880] v28'155 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb7b8024020 con 0x55a118636e80 2023-10-05T19:40:29.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.158+0000 7fb7c7aa4e40 20 remove_watcher() i=0 2023-10-05T19:40:29.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.158+0000 7fb7c7aa4e40 2 removed watcher, disabling cache 2023-10-05T19:40:29.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.158+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94150387313344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a118641180 con 0x55a118632a00 2023-10-05T19:40:29.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.158+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(103 notify.1 [watch unwatch cookie 94150387313344] v28'155 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb7c0014040 con 0x55a118632a00 2023-10-05T19:40:29.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.158+0000 7fb7c7aa4e40 20 remove_watcher() i=1 2023-10-05T19:40:29.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.162+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:104 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94150387309168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a118641180 con 0x55a11863b7d0 2023-10-05T19:40:29.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.162+0000 7fb7c4dd8700 1 -- 172.21.15.12:0/2437350865 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(104 notify.2 [watch unwatch cookie 94150387309168] v28'155 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb7b0040020 con 0x55a11863b7d0 2023-10-05T19:40:29.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.162+0000 7fb7c7aa4e40 20 remove_watcher() i=2 2023-10-05T19:40:29.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.162+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:105 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94150387319744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a118641a70 con 0x55a118636e80 2023-10-05T19:40:29.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.166+0000 7fb7c55d9700 1 -- 172.21.15.12:0/2437350865 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(105 notify.3 [watch unwatch cookie 94150387319744] v28'155 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb7b8024020 con 0x55a118636e80 2023-10-05T19:40:29.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.166+0000 7fb7c7aa4e40 20 remove_watcher() i=3 2023-10-05T19:40:29.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.166+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:106 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94150387314448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a118641180 con 0x55a118632a00 2023-10-05T19:40:29.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.170+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(106 notify.4 [watch unwatch cookie 94150387314448] v28'155 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb7c0014040 con 0x55a118632a00 2023-10-05T19:40:29.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.170+0000 7fb7c7aa4e40 20 remove_watcher() i=4 2023-10-05T19:40:29.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.170+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:107 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94150387326688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a118642fb0 con 0x7fb7b8011c00 2023-10-05T19:40:29.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.174+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(107 notify.5 [watch unwatch cookie 94150387326688] v28'155 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb7c0014040 con 0x7fb7b8011c00 2023-10-05T19:40:29.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.174+0000 7fb7c7aa4e40 20 remove_watcher() i=5 2023-10-05T19:40:29.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.174+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:108 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94150387327792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a118642fb0 con 0x7fb7b8011c00 2023-10-05T19:40:29.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.174+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(108 notify.6 [watch unwatch cookie 94150387327792] v28'155 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb7c0014040 con 0x7fb7b8011c00 2023-10-05T19:40:29.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.174+0000 7fb7c7aa4e40 20 remove_watcher() i=6 2023-10-05T19:40:29.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.174+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:109 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94150387331056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a118642fb0 con 0x7fb7b8011c00 2023-10-05T19:40:29.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.178+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(109 notify.7 [watch unwatch cookie 94150387331056] v28'155 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb7c0014040 con 0x7fb7b8011c00 2023-10-05T19:40:29.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.178+0000 7fb7c7aa4e40 20 remove_watcher() i=7 2023-10-05T19:40:29.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.182+0000 7fb7967e4700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:29.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.182+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:110 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94150387332160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a118642fb0 con 0x55a118632a00 2023-10-05T19:40:29.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.182+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(110 data_loggenerations_metadata [watch unwatch cookie 94150387332160] v28'150 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fb7c0014040 con 0x55a118632a00 2023-10-05T19:40:29.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.182+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fb7b8011c00 msgr2=0x7fb7b8010360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.182+0000 7fb7c7aa4e40 1 --2- 172.21.15.12:0/2437350865 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fb7b8011c00 0x7fb7b8010360 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.182+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55a118632a00 msgr2=0x55a118634de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.182+0000 7fb7c7aa4e40 1 --2- 172.21.15.12:0/2437350865 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55a118632a00 0x55a118634de0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.182+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55a118636e80 msgr2=0x55a118639260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.182+0000 7fb7c7aa4e40 1 --2- 172.21.15.12:0/2437350865 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55a118636e80 0x55a118639260 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.182+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55a11863b7d0 msgr2=0x55a11863dbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.182+0000 7fb7c7aa4e40 1 --2- 172.21.15.12:0/2437350865 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55a11863b7d0 0x55a11863dbb0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.186+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fb7a8030c00 msgr2=0x7fb7a80330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.186+0000 7fb7c7aa4e40 1 --2- 172.21.15.12:0/2437350865 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fb7a8030c00 0x7fb7a80330b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x55a118604060 tx=0x7fb7b0015000 comp rx=0 tx=0).stop 2023-10-05T19:40:29.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.186+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a118603010 msgr2=0x55a118628ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.186+0000 7fb7c7aa4e40 1 --2- 172.21.15.12:0/2437350865 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a118603010 0x55a118628ec0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8001330 tx=0x7fb7b8014000 comp rx=0 tx=0).stop 2023-10-05T19:40:29.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.186+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 reap_dead start 2023-10-05T19:40:29.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.186+0000 7fb7c5dda700 1 -- 172.21.15.12:0/2437350865 reap_dead start 2023-10-05T19:40:29.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.186+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 shutdown_connections 2023-10-05T19:40:29.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.186+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 >> 172.21.15.12:0/2437350865 conn(0x55a118576860 msgr2=0x55a11861c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:29.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.186+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 shutdown_connections 2023-10-05T19:40:29.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.186+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/2437350865 wait complete. 2023-10-05T19:40:29.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.186+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/4043093390 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55a1186c1b00 msgr2=0x55a1186c3ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.186+0000 7fb7c7aa4e40 1 --2- 172.21.15.12:0/4043093390 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55a1186c1b00 0x55a1186c3ee0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.186+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/4043093390 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fb70c030c50 msgr2=0x7fb70c033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.186+0000 7fb7c7aa4e40 1 --2- 172.21.15.12:0/4043093390 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fb70c030c50 0x7fb70c033100 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x55a1186478d0 tx=0x7fb7c000ab80 comp rx=0 tx=0).stop 2023-10-05T19:40:29.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.186+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/4043093390 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a118654a70 msgr2=0x55a11865a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.186+0000 7fb7c7aa4e40 1 --2- 172.21.15.12:0/4043093390 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a118654a70 0x55a11865a550 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb7b00129f0 tx=0x7fb7b0037000 comp rx=0 tx=0).stop 2023-10-05T19:40:29.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.186+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/4043093390 shutdown_connections 2023-10-05T19:40:29.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.186+0000 7fb7c7aa4e40 1 --2- 172.21.15.12:0/4043093390 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55a1186c1b00 0x55a1186c3ee0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.186+0000 7fb7c7aa4e40 1 --2- 172.21.15.12:0/4043093390 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fb70c030c50 0x7fb70c033100 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.186+0000 7fb7c7aa4e40 1 --2- 172.21.15.12:0/4043093390 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a118654a70 0x55a11865a550 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.186+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/4043093390 >> 172.21.15.12:0/4043093390 conn(0x55a11864fe30 msgr2=0x55a11864df30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:29.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.186+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/4043093390 shutdown_connections 2023-10-05T19:40:29.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.186+0000 7fb7c7aa4e40 1 -- 172.21.15.12:0/4043093390 wait complete. 2023-10-05T19:40:29.202 DEBUG:tasks.util.rgw: json result: {'bucket_id': '93ac49f4-ff49-4f99-811b-6453b28decd5.4221.4', 'bucket_owner': 'foo', 'bucket': 'myfoo4', 'log_entries': [{'bucket': 'myfoo4', 'time': '2023-10-05T19:40:23.375112Z', 'time_local': '2023-10-05T19:40:23.375112+0000', 'remote_addr': '172.21.0.51', 'user': 'foo', 'operation': 'create_bucket', 'uri': 'PUT /myfoo4/ HTTP/1.1', 'http_status': '200', 'error_code': '', 'bytes_sent': 0, 'bytes_received': 0, 'object_size': 0, 'total_time': 19, 'user_agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-163-generic', 'referrer': '', 'trans_id': 'tx00000dd2d8159385ba3bb-00651f1127-107b-default', 'authentication_type': 'Local', 'access_key_id': '9te6NH5mcdcq0Tc5i8i1', 'temp_url': False}, {'bucket': 'myfoo4', 'time': '2023-10-05T19:40:23.599106Z', 'time_local': '2023-10-05T19:40:23.599106+0000', 'remote_addr': '172.21.0.51', 'user': 'foo', 'operation': 'delete_bucket', 'uri': 'DELETE /myfoo4/ HTTP/1.1', 'http_status': '204', 'error_code': 'NoContent', 'bytes_sent': 0, 'bytes_received': 0, 'object_size': 0, 'total_time': 71, 'user_agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-163-generic', 'referrer': '', 'trans_id': 'tx000005364c9e12e71da2c-00651f1127-107b-default', 'authentication_type': 'Local', 'access_key_id': '9te6NH5mcdcq0Tc5i8i1', 'temp_url': False}], 'log_sum': {'bytes_sent': 0, 'bytes_received': 0, 'total_time': 90, 'total_entries': 2}} 2023-10-05T19:40:29.202 DEBUG:tasks.radosgw_admin:checking log entry: 2023-10-05T19:40:29.202 DEBUG:tasks.radosgw_admin:checking log entry: 2023-10-05T19:40:29.202 INFO:tasks.util.rgw:rgwadmin: client.0 : ['log', 'rm', '--object', '2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.4-myfoo4'] 2023-10-05T19:40:29.202 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'log', 'rm', '--object', '2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.4-myfoo4'] 2023-10-05T19:40:29.202 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph log rm --object 2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.4-myfoo4 2023-10-05T19:40:29.239 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:29.240 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:29.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.230+0000 7f4f36f0ae40 1 Processor -- start 2023-10-05T19:40:29.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.230+0000 7f4f36f0ae40 1 -- start start 2023-10-05T19:40:29.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.230+0000 7f4f36f0ae40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55f928ad2010 0x55f928ad23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.230+0000 7f4f36f0ae40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55f928a1cbd0 con 0x55f928ad2920 2023-10-05T19:40:29.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.230+0000 7f4f36f0ae40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x55f9289d9ee0 con 0x55f928ad2010 2023-10-05T19:40:29.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.230+0000 7f4f34a3f700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55f928ad2010 0x55f928ad23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.230+0000 7f4f3423e700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x55f928ad2920 0x55f928ad33c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:35680/0 (socket says 172.21.15.12:35680) 2023-10-05T19:40:29.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.230+0000 7f4f34a3f700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55f928ad2010 0x55f928ad23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58546/0 (socket says 172.21.15.12:58546) 2023-10-05T19:40:29.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.230+0000 7f4f3423e700 1 -- 172.21.15.12:0/4216639863 learned_addr learned my addr 172.21.15.12:0/4216639863 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:29.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.234+0000 7f4f34a3f700 1 -- 172.21.15.12:0/4216639863 >> v1:172.21.15.8:6789/0 conn(0x55f928ad2920 legacy=0x55f928ad33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.234+0000 7f4f34a3f700 1 -- 172.21.15.12:0/4216639863 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f928ad3ad0 con 0x55f928ad2010 2023-10-05T19:40:29.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.234+0000 7f4f34a3f700 1 --2- 172.21.15.12:0/4216639863 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f928ad2010 0x55f928ad23e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4f2400a520 tx=0x7f4f24005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4555655cbd8eeff4 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.234+0000 7f4f33a3d700 1 -- 172.21.15.12:0/4216639863 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f2400f590 con 0x55f928ad2010 2023-10-05T19:40:29.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.234+0000 7f4f33a3d700 1 -- 172.21.15.12:0/4216639863 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f2400f6f0 con 0x55f928ad2010 2023-10-05T19:40:29.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.234+0000 7f4f33a3d700 1 -- 172.21.15.12:0/4216639863 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f2400f880 con 0x55f928ad2010 2023-10-05T19:40:29.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.234+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/4216639863 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f928ad2010 msgr2=0x55f928ad23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.234+0000 7f4f36f0ae40 1 --2- 172.21.15.12:0/4216639863 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f928ad2010 0x55f928ad23e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4f2400a520 tx=0x7f4f24005d40 comp rx=0 tx=0).stop 2023-10-05T19:40:29.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.234+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/4216639863 shutdown_connections 2023-10-05T19:40:29.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.234+0000 7f4f36f0ae40 1 --2- 172.21.15.12:0/4216639863 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f928ad2010 0x55f928ad23e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.234+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/4216639863 >> 172.21.15.12:0/4216639863 conn(0x55f928a45860 msgr2=0x55f928a45c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:29.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.234+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/4216639863 shutdown_connections 2023-10-05T19:40:29.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.234+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/4216639863 wait complete. 2023-10-05T19:40:29.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.234+0000 7f4f36f0ae40 1 Processor -- start 2023-10-05T19:40:29.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.234+0000 7f4f36f0ae40 1 -- start start 2023-10-05T19:40:29.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.234+0000 7f4f36f0ae40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f928ad2010 0x55f928af1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.234+0000 7f4f36f0ae40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f92899de30 con 0x55f928ad2010 2023-10-05T19:40:29.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.234+0000 7f4f34a3f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f928ad2010 0x55f928af1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.234+0000 7f4f34a3f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f928ad2010 0x55f928af1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58550/0 (socket says 172.21.15.12:58550) 2023-10-05T19:40:29.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.234+0000 7f4f34a3f700 1 -- 172.21.15.12:0/2614976217 learned_addr learned my addr 172.21.15.12:0/2614976217 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:29.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.238+0000 7f4f34a3f700 1 -- 172.21.15.12:0/2614976217 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f2400a0e0 con 0x55f928ad2010 2023-10-05T19:40:29.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.238+0000 7f4f34a3f700 1 --2- 172.21.15.12:0/2614976217 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f928ad2010 0x55f928af1170 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4f2400a470 tx=0x7f4f24014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e86f0f7c050aae66 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.238+0000 7f4f32a3b700 1 -- 172.21.15.12:0/2614976217 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f2400f860 con 0x55f928ad2010 2023-10-05T19:40:29.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.238+0000 7f4f32a3b700 1 -- 172.21.15.12:0/2614976217 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f2400f9e0 con 0x55f928ad2010 2023-10-05T19:40:29.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.238+0000 7f4f32a3b700 1 -- 172.21.15.12:0/2614976217 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f2400fb90 con 0x55f928ad2010 2023-10-05T19:40:29.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.238+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/2614976217 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f928ad2010 msgr2=0x55f928af1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.238+0000 7f4f36f0ae40 1 --2- 172.21.15.12:0/2614976217 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f928ad2010 0x55f928af1170 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4f2400a470 tx=0x7f4f24014000 comp rx=0 tx=0).stop 2023-10-05T19:40:29.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.238+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/2614976217 shutdown_connections 2023-10-05T19:40:29.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.238+0000 7f4f36f0ae40 1 --2- 172.21.15.12:0/2614976217 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f928ad2010 0x55f928af1170 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.238+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/2614976217 >> 172.21.15.12:0/2614976217 conn(0x55f928a45860 msgr2=0x55f928aecd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:29.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.238+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/2614976217 shutdown_connections 2023-10-05T19:40:29.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.238+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/2614976217 wait complete. 2023-10-05T19:40:29.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.238+0000 7f4f36f0ae40 1 Processor -- start 2023-10-05T19:40:29.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.238+0000 7f4f36f0ae40 1 -- start start 2023-10-05T19:40:29.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.238+0000 7f4f36f0ae40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f928ad2010 0x55f928af7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.238+0000 7f4f36f0ae40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f928970ed0 con 0x55f928ad2010 2023-10-05T19:40:29.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.242+0000 7f4f34a3f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f928ad2010 0x55f928af7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.242+0000 7f4f34a3f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f928ad2010 0x55f928af7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58552/0 (socket says 172.21.15.12:58552) 2023-10-05T19:40:29.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.242+0000 7f4f34a3f700 1 -- 172.21.15.12:0/3627952161 learned_addr learned my addr 172.21.15.12:0/3627952161 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:29.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.242+0000 7f4f34a3f700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f928ad3ad0 con 0x55f928ad2010 2023-10-05T19:40:29.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.242+0000 7f4f34a3f700 1 --2- 172.21.15.12:0/3627952161 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f928ad2010 0x55f928af7db0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4f2400a470 tx=0x7f4f2401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.242+0000 7f4f31238700 1 -- 172.21.15.12:0/3627952161 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f2402f420 con 0x55f928ad2010 2023-10-05T19:40:29.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.242+0000 7f4f31238700 1 -- 172.21.15.12:0/3627952161 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f2402f580 con 0x55f928ad2010 2023-10-05T19:40:29.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.242+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f928af82f0 con 0x55f928ad2010 2023-10-05T19:40:29.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.242+0000 7f4f31238700 1 -- 172.21.15.12:0/3627952161 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f2402f730 con 0x55f928ad2010 2023-10-05T19:40:29.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.242+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f928af8790 con 0x55f928ad2010 2023-10-05T19:40:29.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.242+0000 7f4f31238700 1 -- 172.21.15.12:0/3627952161 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f4f24010420 con 0x55f928ad2010 2023-10-05T19:40:29.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.242+0000 7f4f31238700 1 --2- 172.21.15.12:0/3627952161 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f4f18030c50 0x7f4f18033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.242+0000 7f4f36f0ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.242+0000 7f4f31238700 1 -- 172.21.15.12:0/3627952161 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f4f2402b070 con 0x55f928ad2010 2023-10-05T19:40:29.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.242+0000 7f4f3423e700 1 --2- 172.21.15.12:0/3627952161 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f4f18030c50 0x7f4f18033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.246+0000 7f4f36f0ae40 1 --2- 172.21.15.12:0/3627952161 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55f928ad2920 0x55f928b03840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.246+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b03f40 con 0x55f928ad2920 2023-10-05T19:40:29.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.246+0000 7f4f35240700 1 --2- 172.21.15.12:0/3627952161 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55f928ad2920 0x55f928b03840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.246+0000 7f4f3423e700 1 --2- 172.21.15.12:0/3627952161 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f4f18030c50 0x7f4f18033100 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c001e90 tx=0x7f4f1c0013b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.246+0000 7f4f35240700 1 --2- 172.21.15.12:0/3627952161 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55f928ad2920 0x55f928b03840 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.246+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4f2c00a500 con 0x55f928ad2920 2023-10-05T19:40:29.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.246+0000 7f4f36f0ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:29.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.246+0000 7f4f36f0ae40 20 realm 2023-10-05T19:40:29.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.246+0000 7f4f36f0ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.246+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b00ea0 con 0x55f928ad2920 2023-10-05T19:40:29.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.246+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4f2c00a500 con 0x55f928ad2920 2023-10-05T19:40:29.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.246+0000 7f4f36f0ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:29.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.246+0000 7f4f36f0ae40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:29.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.246+0000 7f4f36f0ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.246+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b04a20 con 0x55f928ad2920 2023-10-05T19:40:29.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.246+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4f2c00a500 con 0x55f928ad2920 2023-10-05T19:40:29.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.246+0000 7f4f36f0ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:29.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.246+0000 7f4f36f0ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.246+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b04630 con 0x55f928ad2920 2023-10-05T19:40:29.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.246+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f4f2c00a500 con 0x55f928ad2920 2023-10-05T19:40:29.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.246+0000 7f4f36f0ae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:29.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.250+0000 7f4f36f0ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.250+0000 7f4f36f0ae40 1 --2- 172.21.15.12:0/3627952161 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55f928b05dc0 0x55f928b081a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.250+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b086e0 con 0x55f928b05dc0 2023-10-05T19:40:29.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.250+0000 7f4f34a3f700 1 --2- 172.21.15.12:0/3627952161 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55f928b05dc0 0x55f928b081a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.250+0000 7f4f34a3f700 1 --2- 172.21.15.12:0/3627952161 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55f928b05dc0 0x55f928b081a0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.250+0000 7f4f34a3f700 1 -- 172.21.15.12:0/3627952161 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f4f24010900 con 0x55f928b05dc0 2023-10-05T19:40:29.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.250+0000 7f4f36f0ae40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:29.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.250+0000 7f4f36f0ae40 20 searching for the correct realm 2023-10-05T19:40:29.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.250+0000 7f4f36f0ae40 1 --2- 172.21.15.12:0/3627952161 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55f928b0a560 0x55f928b0c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.250+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0d020 con 0x55f928b0a560 2023-10-05T19:40:29.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.250+0000 7f4f3423e700 1 --2- 172.21.15.12:0/3627952161 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55f928b0a560 0x55f928b0c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.250+0000 7f4f3423e700 1 --2- 172.21.15.12:0/3627952161 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55f928b0a560 0x55f928b0c970 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.254+0000 7f4f3423e700 1 -- 172.21.15.12:0/3627952161 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f4f1c006070 con 0x55f928b0a560 2023-10-05T19:40:29.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.254+0000 7f4f3423e700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0d020 con 0x55f928b05dc0 2023-10-05T19:40:29.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.254+0000 7f4f34a3f700 1 -- 172.21.15.12:0/3627952161 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f24010900 con 0x55f928b05dc0 2023-10-05T19:40:29.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.254+0000 7f4f34a3f700 1 --2- 172.21.15.12:0/3627952161 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f4f2402e3c0 0x7f4f240377b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.254+0000 7f4f34a3f700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0d020 con 0x7f4f2402e3c0 2023-10-05T19:40:29.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.254+0000 7f4f35240700 1 --2- 172.21.15.12:0/3627952161 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f4f2402e3c0 0x7f4f240377b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.254+0000 7f4f35240700 1 --2- 172.21.15.12:0/3627952161 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f4f2402e3c0 0x7f4f240377b0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.258+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f2c014040 con 0x7f4f2402e3c0 2023-10-05T19:40:29.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.258+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0d020 con 0x55f928ad2920 2023-10-05T19:40:29.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.258+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f2c014040 con 0x55f928ad2920 2023-10-05T19:40:29.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.258+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0d020 con 0x55f928ad2920 2023-10-05T19:40:29.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.258+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f2c014040 con 0x55f928ad2920 2023-10-05T19:40:29.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.258+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0d020 con 0x7f4f2402e3c0 2023-10-05T19:40:29.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.258+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f2c014040 con 0x7f4f2402e3c0 2023-10-05T19:40:29.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.258+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0d020 con 0x7f4f2402e3c0 2023-10-05T19:40:29.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.258+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f2c014040 con 0x7f4f2402e3c0 2023-10-05T19:40:29.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.258+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0d020 con 0x55f928b05dc0 2023-10-05T19:40:29.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.262+0000 7f4f34a3f700 1 -- 172.21.15.12:0/3627952161 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f24010900 con 0x55f928b05dc0 2023-10-05T19:40:29.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.262+0000 7f4f34a3f700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0d020 con 0x55f928b0a560 2023-10-05T19:40:29.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.262+0000 7f4f3423e700 1 -- 172.21.15.12:0/3627952161 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f1c006070 con 0x55f928b0a560 2023-10-05T19:40:29.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.262+0000 7f4f3423e700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0d020 con 0x55f928ad2920 2023-10-05T19:40:29.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.262+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f2c014040 con 0x55f928ad2920 2023-10-05T19:40:29.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.262+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0d020 con 0x7f4f2402e3c0 2023-10-05T19:40:29.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.262+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f2c014040 con 0x7f4f2402e3c0 2023-10-05T19:40:29.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.262+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0d020 con 0x55f928ad2920 2023-10-05T19:40:29.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.262+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f2c014040 con 0x55f928ad2920 2023-10-05T19:40:29.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.262+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0d020 con 0x55f928b05dc0 2023-10-05T19:40:29.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.266+0000 7f4f34a3f700 1 -- 172.21.15.12:0/3627952161 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f24010900 con 0x55f928b05dc0 2023-10-05T19:40:29.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.266+0000 7f4f34a3f700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0d020 con 0x55f928b0a560 2023-10-05T19:40:29.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.266+0000 7f4f3423e700 1 -- 172.21.15.12:0/3627952161 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f1c006070 con 0x55f928b0a560 2023-10-05T19:40:29.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.266+0000 7f4f3423e700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0d020 con 0x55f928b0a560 2023-10-05T19:40:29.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.266+0000 7f4f3423e700 1 -- 172.21.15.12:0/3627952161 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f1c006070 con 0x55f928b0a560 2023-10-05T19:40:29.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.266+0000 7f4f3423e700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0d020 con 0x7f4f2402e3c0 2023-10-05T19:40:29.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.266+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f2c014040 con 0x7f4f2402e3c0 2023-10-05T19:40:29.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.266+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0d020 con 0x55f928b0a560 2023-10-05T19:40:29.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.266+0000 7f4f3423e700 1 -- 172.21.15.12:0/3627952161 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f1c006070 con 0x55f928b0a560 2023-10-05T19:40:29.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.266+0000 7f4f3423e700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0d020 con 0x55f928b0a560 2023-10-05T19:40:29.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.270+0000 7f4f3423e700 1 -- 172.21.15.12:0/3627952161 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f1c006070 con 0x55f928b0a560 2023-10-05T19:40:29.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.270+0000 7f4f3423e700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0d020 con 0x55f928ad2920 2023-10-05T19:40:29.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.270+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f2c014040 con 0x55f928ad2920 2023-10-05T19:40:29.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.270+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0d020 con 0x55f928ad2920 2023-10-05T19:40:29.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.270+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f2c014040 con 0x55f928ad2920 2023-10-05T19:40:29.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.270+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0d020 con 0x7f4f2402e3c0 2023-10-05T19:40:29.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.270+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f2c014040 con 0x7f4f2402e3c0 2023-10-05T19:40:29.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.270+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0d020 con 0x55f928b0a560 2023-10-05T19:40:29.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.270+0000 7f4f3423e700 1 -- 172.21.15.12:0/3627952161 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f1c006070 con 0x55f928b0a560 2023-10-05T19:40:29.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.274+0000 7f4f3423e700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0d020 con 0x55f928b05dc0 2023-10-05T19:40:29.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.274+0000 7f4f34a3f700 1 -- 172.21.15.12:0/3627952161 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f24010900 con 0x55f928b05dc0 2023-10-05T19:40:29.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.274+0000 7f4f34a3f700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0d020 con 0x55f928ad2920 2023-10-05T19:40:29.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.274+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f2c014040 con 0x55f928ad2920 2023-10-05T19:40:29.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.274+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0d020 con 0x7f4f2402e3c0 2023-10-05T19:40:29.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.274+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f2c014040 con 0x7f4f2402e3c0 2023-10-05T19:40:29.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.274+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0d020 con 0x55f928b0a560 2023-10-05T19:40:29.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.274+0000 7f4f3423e700 1 -- 172.21.15.12:0/3627952161 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f1c006070 con 0x55f928b0a560 2023-10-05T19:40:29.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.274+0000 7f4f3423e700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0d020 con 0x7f4f2402e3c0 2023-10-05T19:40:29.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.278+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f2c014040 con 0x7f4f2402e3c0 2023-10-05T19:40:29.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.278+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0d020 con 0x55f928b05dc0 2023-10-05T19:40:29.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.278+0000 7f4f34a3f700 1 -- 172.21.15.12:0/3627952161 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f24010900 con 0x55f928b05dc0 2023-10-05T19:40:29.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.278+0000 7f4f34a3f700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0d020 con 0x55f928b0a560 2023-10-05T19:40:29.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.278+0000 7f4f3423e700 1 -- 172.21.15.12:0/3627952161 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f1c006070 con 0x55f928b0a560 2023-10-05T19:40:29.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.278+0000 7f4f3423e700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0d020 con 0x7f4f2402e3c0 2023-10-05T19:40:29.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.278+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f2c014040 con 0x7f4f2402e3c0 2023-10-05T19:40:29.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.278+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0d020 con 0x55f928ad2920 2023-10-05T19:40:29.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.282+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f4f2c014040 con 0x55f928ad2920 2023-10-05T19:40:29.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.282+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0d020 con 0x55f928b05dc0 2023-10-05T19:40:29.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.282+0000 7f4f34a3f700 1 -- 172.21.15.12:0/3627952161 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f4f24010900 con 0x55f928b05dc0 2023-10-05T19:40:29.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.282+0000 7f4f36f0ae40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:29.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.282+0000 7f4f36f0ae40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:29.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.282+0000 7f4f36f0ae40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:29.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.282+0000 7f4f36f0ae40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:29.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.282+0000 7f4f36f0ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.282+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b092e0 con 0x55f928ad2920 2023-10-05T19:40:29.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.282+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4f2c014040 con 0x55f928ad2920 2023-10-05T19:40:29.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.282+0000 7f4f36f0ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:29.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.282+0000 7f4f36f0ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.282+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b08f10 con 0x55f928ad2920 2023-10-05T19:40:29.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.282+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f4f2c014040 con 0x55f928ad2920 2023-10-05T19:40:29.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.282+0000 7f4f36f0ae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:29.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.282+0000 7f4f36f0ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.286+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0d490 con 0x55f928b0a560 2023-10-05T19:40:29.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.286+0000 7f4f3423e700 1 -- 172.21.15.12:0/3627952161 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f4f1c006070 con 0x55f928b0a560 2023-10-05T19:40:29.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.286+0000 7f4f36f0ae40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:29.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.286+0000 7f4f36f0ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.286+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0d840 con 0x55f928b0a560 2023-10-05T19:40:29.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.286+0000 7f4f3423e700 1 -- 172.21.15.12:0/3627952161 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f4f1c006070 con 0x55f928b0a560 2023-10-05T19:40:29.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.286+0000 7f4f36f0ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:29.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.286+0000 7f4f36f0ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.286+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0dbf0 con 0x55f928ad2920 2023-10-05T19:40:29.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.286+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4f2c014040 con 0x55f928ad2920 2023-10-05T19:40:29.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.286+0000 7f4f36f0ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:29.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.286+0000 7f4f36f0ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.286+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b09800 con 0x55f928ad2920 2023-10-05T19:40:29.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.286+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f4f2c014040 con 0x55f928ad2920 2023-10-05T19:40:29.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.286+0000 7f4f36f0ae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:29.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.286+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b09800 con 0x55f928b0a560 2023-10-05T19:40:29.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.290+0000 7f4f3423e700 1 -- 172.21.15.12:0/3627952161 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f4f1c006070 con 0x55f928b0a560 2023-10-05T19:40:29.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.290+0000 7f4f3423e700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b09800 con 0x55f928b05dc0 2023-10-05T19:40:29.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.290+0000 7f4f34a3f700 1 -- 172.21.15.12:0/3627952161 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f24010900 con 0x55f928b05dc0 2023-10-05T19:40:29.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.290+0000 7f4f34a3f700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b09800 con 0x7f4f2402e3c0 2023-10-05T19:40:29.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.290+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f2c014040 con 0x7f4f2402e3c0 2023-10-05T19:40:29.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.290+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0dbf0 con 0x55f928ad2920 2023-10-05T19:40:29.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.294+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f2c014040 con 0x55f928ad2920 2023-10-05T19:40:29.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.294+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0dbf0 con 0x55f928ad2920 2023-10-05T19:40:29.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.294+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f2c014040 con 0x55f928ad2920 2023-10-05T19:40:29.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.294+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0dbf0 con 0x7f4f2402e3c0 2023-10-05T19:40:29.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.294+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f2c014040 con 0x7f4f2402e3c0 2023-10-05T19:40:29.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.294+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0dbf0 con 0x7f4f2402e3c0 2023-10-05T19:40:29.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.294+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f2c014040 con 0x7f4f2402e3c0 2023-10-05T19:40:29.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.294+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0dbf0 con 0x55f928b05dc0 2023-10-05T19:40:29.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.298+0000 7f4f34a3f700 1 -- 172.21.15.12:0/3627952161 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f24010900 con 0x55f928b05dc0 2023-10-05T19:40:29.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.298+0000 7f4f34a3f700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0dbf0 con 0x55f928b0a560 2023-10-05T19:40:29.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.298+0000 7f4f3423e700 1 -- 172.21.15.12:0/3627952161 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f1c006070 con 0x55f928b0a560 2023-10-05T19:40:29.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.298+0000 7f4f3423e700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0dbf0 con 0x55f928ad2920 2023-10-05T19:40:29.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.298+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f2c014040 con 0x55f928ad2920 2023-10-05T19:40:29.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.298+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0dbf0 con 0x7f4f2402e3c0 2023-10-05T19:40:29.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.298+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f2c014040 con 0x7f4f2402e3c0 2023-10-05T19:40:29.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.298+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0dbf0 con 0x55f928ad2920 2023-10-05T19:40:29.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.302+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f2c014040 con 0x55f928ad2920 2023-10-05T19:40:29.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.302+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0dbf0 con 0x55f928b05dc0 2023-10-05T19:40:29.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.302+0000 7f4f34a3f700 1 -- 172.21.15.12:0/3627952161 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f24010900 con 0x55f928b05dc0 2023-10-05T19:40:29.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.302+0000 7f4f34a3f700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0dbf0 con 0x55f928b0a560 2023-10-05T19:40:29.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.302+0000 7f4f3423e700 1 -- 172.21.15.12:0/3627952161 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f1c006070 con 0x55f928b0a560 2023-10-05T19:40:29.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.302+0000 7f4f3423e700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0dbf0 con 0x55f928b0a560 2023-10-05T19:40:29.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.302+0000 7f4f3423e700 1 -- 172.21.15.12:0/3627952161 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f1c006070 con 0x55f928b0a560 2023-10-05T19:40:29.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.302+0000 7f4f3423e700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0dbf0 con 0x7f4f2402e3c0 2023-10-05T19:40:29.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.302+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f2c02e040 con 0x7f4f2402e3c0 2023-10-05T19:40:29.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.302+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0dbf0 con 0x55f928b0a560 2023-10-05T19:40:29.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.306+0000 7f4f3423e700 1 -- 172.21.15.12:0/3627952161 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f1c006070 con 0x55f928b0a560 2023-10-05T19:40:29.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.306+0000 7f4f3423e700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0dbf0 con 0x55f928b0a560 2023-10-05T19:40:29.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.306+0000 7f4f3423e700 1 -- 172.21.15.12:0/3627952161 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f1c006070 con 0x55f928b0a560 2023-10-05T19:40:29.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.306+0000 7f4f3423e700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0dbf0 con 0x55f928ad2920 2023-10-05T19:40:29.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.306+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f2c02e040 con 0x55f928ad2920 2023-10-05T19:40:29.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.306+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0dbf0 con 0x55f928ad2920 2023-10-05T19:40:29.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.306+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f2c02e040 con 0x55f928ad2920 2023-10-05T19:40:29.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.306+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0dbf0 con 0x7f4f2402e3c0 2023-10-05T19:40:29.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.306+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f2c02e040 con 0x7f4f2402e3c0 2023-10-05T19:40:29.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.306+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0dbf0 con 0x55f928b0a560 2023-10-05T19:40:29.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.310+0000 7f4f3423e700 1 -- 172.21.15.12:0/3627952161 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f1c006070 con 0x55f928b0a560 2023-10-05T19:40:29.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.310+0000 7f4f3423e700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0dbf0 con 0x55f928b05dc0 2023-10-05T19:40:29.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.310+0000 7f4f34a3f700 1 -- 172.21.15.12:0/3627952161 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f24010900 con 0x55f928b05dc0 2023-10-05T19:40:29.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.310+0000 7f4f34a3f700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0dbf0 con 0x55f928ad2920 2023-10-05T19:40:29.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.310+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f2c02e040 con 0x55f928ad2920 2023-10-05T19:40:29.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.310+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0dbf0 con 0x7f4f2402e3c0 2023-10-05T19:40:29.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.310+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f2c02e040 con 0x7f4f2402e3c0 2023-10-05T19:40:29.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.310+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0dbf0 con 0x55f928b0a560 2023-10-05T19:40:29.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.314+0000 7f4f3423e700 1 -- 172.21.15.12:0/3627952161 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f1c006070 con 0x55f928b0a560 2023-10-05T19:40:29.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.314+0000 7f4f3423e700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0dbf0 con 0x7f4f2402e3c0 2023-10-05T19:40:29.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.314+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f2c02e040 con 0x7f4f2402e3c0 2023-10-05T19:40:29.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.314+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0dbf0 con 0x55f928b05dc0 2023-10-05T19:40:29.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.314+0000 7f4f34a3f700 1 -- 172.21.15.12:0/3627952161 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f24010900 con 0x55f928b05dc0 2023-10-05T19:40:29.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.314+0000 7f4f34a3f700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0dbf0 con 0x55f928b0a560 2023-10-05T19:40:29.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.314+0000 7f4f3423e700 1 -- 172.21.15.12:0/3627952161 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f1c006070 con 0x55f928b0a560 2023-10-05T19:40:29.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.314+0000 7f4f3423e700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0dbf0 con 0x7f4f2402e3c0 2023-10-05T19:40:29.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.318+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f2c02e040 con 0x7f4f2402e3c0 2023-10-05T19:40:29.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.318+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0dbf0 con 0x55f928ad2920 2023-10-05T19:40:29.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.318+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f4f2c02e040 con 0x55f928ad2920 2023-10-05T19:40:29.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.318+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0dbf0 con 0x55f928b05dc0 2023-10-05T19:40:29.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.318+0000 7f4f34a3f700 1 -- 172.21.15.12:0/3627952161 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f4f24010900 con 0x55f928b05dc0 2023-10-05T19:40:29.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.318+0000 7f4f36f0ae40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:29.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.318+0000 7f4f36f0ae40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:29.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.318+0000 7f4f36f0ae40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:29.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.318+0000 7f4f36f0ae40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:29.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.318+0000 7f4f36f0ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.318+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b09800 con 0x55f928ad2920 2023-10-05T19:40:29.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.318+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f4f2c02e040 con 0x55f928ad2920 2023-10-05T19:40:29.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.318+0000 7f4f36f0ae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:29.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.318+0000 7f4f36f0ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.318+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0e590 con 0x55f928b05dc0 2023-10-05T19:40:29.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.322+0000 7f4f34a3f700 1 -- 172.21.15.12:0/3627952161 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f4f24010900 con 0x55f928b05dc0 2023-10-05T19:40:29.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.322+0000 7f4f36f0ae40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:29.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.322+0000 7f4f36f0ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.322+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0ee80 con 0x55f928ad2920 2023-10-05T19:40:29.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.322+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f4f2c02e040 con 0x55f928ad2920 2023-10-05T19:40:29.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.322+0000 7f4f36f0ae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:29.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.322+0000 7f4f36f0ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.322+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0ee80 con 0x55f928b0a560 2023-10-05T19:40:29.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.322+0000 7f4f3423e700 1 -- 172.21.15.12:0/3627952161 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f4f1c006070 con 0x55f928b0a560 2023-10-05T19:40:29.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.322+0000 7f4f36f0ae40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:29.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.322+0000 7f4f36f0ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.322+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0f5f0 con 0x7f4f2402e3c0 2023-10-05T19:40:29.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.322+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f4f2c02e040 con 0x7f4f2402e3c0 2023-10-05T19:40:29.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.322+0000 7f4f36f0ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:29.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.322+0000 7f4f36f0ae40 20 zone default found 2023-10-05T19:40:29.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.322+0000 7f4f36f0ae40 4 Realm: () 2023-10-05T19:40:29.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.322+0000 7f4f36f0ae40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:29.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.322+0000 7f4f36f0ae40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:29.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.322+0000 7f4f36f0ae40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:29.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.322+0000 7f4f36f0ae40 20 zonegroup default 2023-10-05T19:40:29.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.322+0000 7f4f36f0ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.322+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0f230 con 0x7f4f2402e3c0 2023-10-05T19:40:29.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.322+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f4f2c02e040 con 0x7f4f2402e3c0 2023-10-05T19:40:29.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.322+0000 7f4f36f0ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:29.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.322+0000 7f4f36f0ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.322+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b10bb0 con 0x55f928b0a560 2023-10-05T19:40:29.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.326+0000 7f4f3423e700 1 -- 172.21.15.12:0/3627952161 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f4f1c006070 con 0x55f928b0a560 2023-10-05T19:40:29.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.326+0000 7f4f36f0ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:29.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.326+0000 7f4f36f0ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.326+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b10f60 con 0x55f928b0a560 2023-10-05T19:40:29.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.326+0000 7f4f3423e700 1 -- 172.21.15.12:0/3627952161 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f4f1c006070 con 0x55f928b0a560 2023-10-05T19:40:29.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.326+0000 7f4f36f0ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:29.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.326+0000 7f4f36f0ae40 20 started sync module instance, tier type = 2023-10-05T19:40:29.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.326+0000 7f4f36f0ae40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:29.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.326+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b10270 con 0x55f928b05dc0 2023-10-05T19:40:29.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.326+0000 7f4f34a3f700 1 -- 172.21.15.12:0/3627952161 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'156 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f24010900 con 0x55f928b05dc0 2023-10-05T19:40:29.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.330+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94528617907824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b0eab0 con 0x55f928b05dc0 2023-10-05T19:40:29.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.330+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b11d60 con 0x55f928ad2920 2023-10-05T19:40:29.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.330+0000 7f4f34a3f700 1 -- 172.21.15.12:0/3627952161 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94528617907824] v28'157 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f24010900 con 0x55f928b05dc0 2023-10-05T19:40:29.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.330+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'156 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f2c02e040 con 0x55f928ad2920 2023-10-05T19:40:29.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.330+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94528617911056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b12160 con 0x55f928ad2920 2023-10-05T19:40:29.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.330+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b12a40 con 0x55f928b0a560 2023-10-05T19:40:29.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.334+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94528617911056] v28'157 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f2c02e040 con 0x55f928ad2920 2023-10-05T19:40:29.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.334+0000 7f4f3423e700 1 -- 172.21.15.12:0/3627952161 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'156 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f1c006070 con 0x55f928b0a560 2023-10-05T19:40:29.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.334+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94528617914720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b125b0 con 0x55f928b0a560 2023-10-05T19:40:29.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.334+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b13820 con 0x55f928b05dc0 2023-10-05T19:40:29.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.338+0000 7f4f34a3f700 1 -- 172.21.15.12:0/3627952161 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'156 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f24010900 con 0x55f928b05dc0 2023-10-05T19:40:29.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.338+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94528617919568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b13d90 con 0x55f928b05dc0 2023-10-05T19:40:29.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.338+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b14910 con 0x55f928ad2920 2023-10-05T19:40:29.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.338+0000 7f4f3423e700 1 -- 172.21.15.12:0/3627952161 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94528617914720] v28'157 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f1c006070 con 0x55f928b0a560 2023-10-05T19:40:29.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.338+0000 7f4f34a3f700 1 -- 172.21.15.12:0/3627952161 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94528617919568] v28'157 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f24010900 con 0x55f928b05dc0 2023-10-05T19:40:29.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.342+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'156 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f2c02e040 con 0x55f928ad2920 2023-10-05T19:40:29.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.342+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94528617923616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b13d90 con 0x55f928ad2920 2023-10-05T19:40:29.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.342+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b14870 con 0x7f4f2402e3c0 2023-10-05T19:40:29.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.342+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'156 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f2c02e040 con 0x7f4f2402e3c0 2023-10-05T19:40:29.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.342+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94528617924720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b151b0 con 0x7f4f2402e3c0 2023-10-05T19:40:29.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.342+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b15cc0 con 0x7f4f2402e3c0 2023-10-05T19:40:29.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.342+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94528617923616] v28'157 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f2c02e040 con 0x55f928ad2920 2023-10-05T19:40:29.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.346+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(95 notify.5 [watch watch cookie 94528617924720] v28'157 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f2c02e040 con 0x7f4f2402e3c0 2023-10-05T19:40:29.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.346+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(96 notify.6 [create] v28'156 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f2c02e040 con 0x7f4f2402e3c0 2023-10-05T19:40:29.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.346+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94528617920672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b14cc0 con 0x7f4f2402e3c0 2023-10-05T19:40:29.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.346+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b163e0 con 0x7f4f2402e3c0 2023-10-05T19:40:29.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.346+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(97 notify.6 [watch watch cookie 94528617920672] v28'157 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f2c02e040 con 0x7f4f2402e3c0 2023-10-05T19:40:29.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.350+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(98 notify.7 [create] v28'156 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f2c02e040 con 0x7f4f2402e3c0 2023-10-05T19:40:29.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.350+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94528617928592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b14cc0 con 0x7f4f2402e3c0 2023-10-05T19:40:29.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.350+0000 7f4f36f0ae40 20 add_watcher() i=0 2023-10-05T19:40:29.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.350+0000 7f4f36f0ae40 20 add_watcher() i=1 2023-10-05T19:40:29.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.350+0000 7f4f36f0ae40 20 add_watcher() i=2 2023-10-05T19:40:29.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.350+0000 7f4f36f0ae40 20 add_watcher() i=3 2023-10-05T19:40:29.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.350+0000 7f4f36f0ae40 20 add_watcher() i=4 2023-10-05T19:40:29.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.350+0000 7f4f36f0ae40 20 add_watcher() i=5 2023-10-05T19:40:29.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.350+0000 7f4f36f0ae40 20 add_watcher() i=6 2023-10-05T19:40:29.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.350+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94528617928592] v28'157 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f2c02e040 con 0x7f4f2402e3c0 2023-10-05T19:40:29.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.350+0000 7f4f36f0ae40 20 add_watcher() i=7 2023-10-05T19:40:29.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.350+0000 7f4f36f0ae40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:29.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.350+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b19440 con 0x55f928ad2920 2023-10-05T19:40:29.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.354+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f4f2c02e040 con 0x55f928ad2920 2023-10-05T19:40:29.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.354+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94528617930480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b16900 con 0x55f928ad2920 2023-10-05T19:40:29.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.354+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94528617930480] v28'151 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f4f2c02e040 con 0x55f928ad2920 2023-10-05T19:40:29.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.354+0000 7f4f057f2700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:29.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.354+0000 7f4f36f0ae40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:29.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.354+0000 7f4f36f0ae40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:29.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.354+0000 7f4f36f0ae40 1 Processor -- start 2023-10-05T19:40:29.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.354+0000 7f4f36f0ae40 1 -- start start 2023-10-05T19:40:29.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.358+0000 7f4f36f0ae40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f928b23b50 0x55f928b23f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.358+0000 7f4f36f0ae40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f928b24460 con 0x55f928b23b50 2023-10-05T19:40:29.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.358+0000 7f4f3423e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f928b23b50 0x55f928b23f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.358+0000 7f4f3423e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f928b23b50 0x55f928b23f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58566/0 (socket says 172.21.15.12:58566) 2023-10-05T19:40:29.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.358+0000 7f4f3423e700 1 -- 172.21.15.12:0/2596122013 learned_addr learned my addr 172.21.15.12:0/2596122013 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:29.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.358+0000 7f4f3423e700 1 -- 172.21.15.12:0/2596122013 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f928b245a0 con 0x55f928b23b50 2023-10-05T19:40:29.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.358+0000 7f4f3423e700 1 --2- 172.21.15.12:0/2596122013 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f928b23b50 0x55f928b23f20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c00e550 tx=0x7f4f1c01e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d599c7eb6fe5499 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.358+0000 7f4f04ff1700 1 -- 172.21.15.12:0/2596122013 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f1c013040 con 0x55f928b23b50 2023-10-05T19:40:29.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.358+0000 7f4f04ff1700 1 -- 172.21.15.12:0/2596122013 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f1c022040 con 0x55f928b23b50 2023-10-05T19:40:29.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.358+0000 7f4f04ff1700 1 -- 172.21.15.12:0/2596122013 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f1c00ee90 con 0x55f928b23b50 2023-10-05T19:40:29.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.358+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/2596122013 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f928b23b50 msgr2=0x55f928b23f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.358+0000 7f4f36f0ae40 1 --2- 172.21.15.12:0/2596122013 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f928b23b50 0x55f928b23f20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c00e550 tx=0x7f4f1c01e040 comp rx=0 tx=0).stop 2023-10-05T19:40:29.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.358+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/2596122013 shutdown_connections 2023-10-05T19:40:29.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.358+0000 7f4f36f0ae40 1 --2- 172.21.15.12:0/2596122013 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f928b23b50 0x55f928b23f20 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.358+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/2596122013 >> 172.21.15.12:0/2596122013 conn(0x55f928b1ef10 msgr2=0x55f928b1d060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:29.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.358+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/2596122013 shutdown_connections 2023-10-05T19:40:29.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.358+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/2596122013 wait complete. 2023-10-05T19:40:29.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.362+0000 7f4f36f0ae40 1 Processor -- start 2023-10-05T19:40:29.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.362+0000 7f4f36f0ae40 1 -- start start 2023-10-05T19:40:29.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.362+0000 7f4f36f0ae40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f928b23b50 0x55f928b296f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.362+0000 7f4f36f0ae40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f928b2a6f0 con 0x55f928b23b50 2023-10-05T19:40:29.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.362+0000 7f4f3423e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f928b23b50 0x55f928b296f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.362+0000 7f4f3423e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f928b23b50 0x55f928b296f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58572/0 (socket says 172.21.15.12:58572) 2023-10-05T19:40:29.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.362+0000 7f4f3423e700 1 -- 172.21.15.12:0/3868146301 learned_addr learned my addr 172.21.15.12:0/3868146301 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:29.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.362+0000 7f4f3423e700 1 -- 172.21.15.12:0/3868146301 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f1c034040 con 0x55f928b23b50 2023-10-05T19:40:29.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.362+0000 7f4f3423e700 1 --2- 172.21.15.12:0/3868146301 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f928b23b50 0x55f928b296f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c00e550 tx=0x7f4f1c03d030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.362+0000 7f4e7effd700 1 -- 172.21.15.12:0/3868146301 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f1c02c640 con 0x55f928b23b50 2023-10-05T19:40:29.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.362+0000 7f4e7effd700 1 -- 172.21.15.12:0/3868146301 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f1c02c7a0 con 0x55f928b23b50 2023-10-05T19:40:29.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.362+0000 7f4e7effd700 1 -- 172.21.15.12:0/3868146301 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f1c02c910 con 0x55f928b23b50 2023-10-05T19:40:29.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.362+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3868146301 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f928b29cf0 con 0x55f928b23b50 2023-10-05T19:40:29.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.362+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3868146301 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f928b2a130 con 0x55f928b23b50 2023-10-05T19:40:29.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.366+0000 7f4e7effd700 1 -- 172.21.15.12:0/3868146301 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f4f1c04d070 con 0x55f928b23b50 2023-10-05T19:40:29.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.366+0000 7f4e7effd700 1 --2- 172.21.15.12:0/3868146301 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f4e74030c50 0x7f4e74033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.366+0000 7f4e7effd700 1 -- 172.21.15.12:0/3868146301 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f4f1c031140 con 0x55f928b23b50 2023-10-05T19:40:29.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.366+0000 7f4f35240700 1 --2- 172.21.15.12:0/3868146301 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f4e74030c50 0x7f4e74033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.366+0000 7f4f36f0ae40 5 note: GC not initialized 2023-10-05T19:40:29.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.366+0000 7f4f35240700 1 --2- 172.21.15.12:0/3868146301 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f4e74030c50 0x7f4e74033100 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x55f928b14f70 tx=0x7f4f2c00add0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.366+0000 7f4e73fff700 20 reqs_thread_entry: start 2023-10-05T19:40:29.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.370+0000 7f4f36f0ae40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:29.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.370+0000 7f4e71ffb700 20 reqs_thread_entry: start 2023-10-05T19:40:29.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.370+0000 7f4f36f0ae40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:29.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.370+0000 7f4e5ffff700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:59 2023 2023-10-05T19:40:29.379 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:29.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.370+0000 7f4f36f0ae40 1 --2- 172.21.15.12:0/3868146301 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55f928b90ba0 0x55f928b92f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.370+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3868146301 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:1 6.0 6:0018ba17:::2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.4-myfoo4:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b934c0 con 0x55f928b90ba0 2023-10-05T19:40:29.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.370+0000 7f4e5f7fe700 20 reqs_thread_entry: start 2023-10-05T19:40:29.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.370+0000 7f4f34a3f700 1 --2- 172.21.15.12:0/3868146301 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55f928b90ba0 0x55f928b92f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.370+0000 7f4f34a3f700 1 --2- 172.21.15.12:0/3868146301 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55f928b90ba0 0x55f928b92f80 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.374+0000 7f4f34a3f700 1 -- 172.21.15.12:0/3868146301 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(1 2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.4-myfoo4 [delete] v28'37 uv37 ondisk = 0) v8 ==== 208+0+0 (crc 0 0 0) 0x7f4f24010900 con 0x55f928b90ba0 2023-10-05T19:40:29.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.374+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:102 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94528617907824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b4fea0 con 0x55f928b05dc0 2023-10-05T19:40:29.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.378+0000 7f4f34a3f700 1 -- 172.21.15.12:0/3627952161 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(102 notify.0 [watch unwatch cookie 94528617907824] v28'158 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f24010900 con 0x55f928b05dc0 2023-10-05T19:40:29.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.378+0000 7f4f36f0ae40 20 remove_watcher() i=0 2023-10-05T19:40:29.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.378+0000 7f4f36f0ae40 2 removed watcher, disabling cache 2023-10-05T19:40:29.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.378+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94528617911056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b10670 con 0x55f928ad2920 2023-10-05T19:40:29.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.378+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(103 notify.1 [watch unwatch cookie 94528617911056] v28'158 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f2c02e040 con 0x55f928ad2920 2023-10-05T19:40:29.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.378+0000 7f4f36f0ae40 20 remove_watcher() i=1 2023-10-05T19:40:29.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.378+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:104 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94528617914720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b10670 con 0x55f928b0a560 2023-10-05T19:40:29.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.382+0000 7f4f3423e700 1 -- 172.21.15.12:0/3627952161 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(104 notify.2 [watch unwatch cookie 94528617914720] v28'158 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f1c055020 con 0x55f928b0a560 2023-10-05T19:40:29.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.382+0000 7f4f36f0ae40 20 remove_watcher() i=2 2023-10-05T19:40:29.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.382+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:105 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94528617919568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b12160 con 0x55f928b05dc0 2023-10-05T19:40:29.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.386+0000 7f4f34a3f700 1 -- 172.21.15.12:0/3627952161 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(105 notify.3 [watch unwatch cookie 94528617919568] v28'158 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f24010900 con 0x55f928b05dc0 2023-10-05T19:40:29.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.386+0000 7f4f36f0ae40 20 remove_watcher() i=3 2023-10-05T19:40:29.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.386+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:106 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94528617923616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b10670 con 0x55f928ad2920 2023-10-05T19:40:29.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.390+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(106 notify.4 [watch unwatch cookie 94528617923616] v28'158 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f2c02e040 con 0x55f928ad2920 2023-10-05T19:40:29.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.390+0000 7f4f36f0ae40 20 remove_watcher() i=4 2023-10-05T19:40:29.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.390+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:107 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94528617924720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b13450 con 0x7f4f2402e3c0 2023-10-05T19:40:29.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.390+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(107 notify.5 [watch unwatch cookie 94528617924720] v28'158 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f2c02e040 con 0x7f4f2402e3c0 2023-10-05T19:40:29.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.390+0000 7f4f36f0ae40 20 remove_watcher() i=5 2023-10-05T19:40:29.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.390+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:108 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94528617920672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b12160 con 0x7f4f2402e3c0 2023-10-05T19:40:29.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.394+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(108 notify.6 [watch unwatch cookie 94528617920672] v28'158 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f2c02e040 con 0x7f4f2402e3c0 2023-10-05T19:40:29.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.394+0000 7f4f36f0ae40 20 remove_watcher() i=6 2023-10-05T19:40:29.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.394+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:109 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94528617928592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b10670 con 0x7f4f2402e3c0 2023-10-05T19:40:29.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.398+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(109 notify.7 [watch unwatch cookie 94528617928592] v28'158 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f2c02e040 con 0x7f4f2402e3c0 2023-10-05T19:40:29.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.398+0000 7f4f36f0ae40 20 remove_watcher() i=7 2023-10-05T19:40:29.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.398+0000 7f4f057f2700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:29.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.398+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:110 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94528617930480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55f928b12160 con 0x55f928ad2920 2023-10-05T19:40:29.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.402+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(110 data_loggenerations_metadata [watch unwatch cookie 94528617930480] v28'152 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f4f2c02e040 con 0x55f928ad2920 2023-10-05T19:40:29.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.402+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f4f2402e3c0 msgr2=0x7f4f240377b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.402+0000 7f4f36f0ae40 1 --2- 172.21.15.12:0/3627952161 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f4f2402e3c0 0x7f4f240377b0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.402+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55f928ad2920 msgr2=0x55f928b03840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.402+0000 7f4f36f0ae40 1 --2- 172.21.15.12:0/3627952161 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55f928ad2920 0x55f928b03840 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.402+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55f928b05dc0 msgr2=0x55f928b081a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.402+0000 7f4f36f0ae40 1 --2- 172.21.15.12:0/3627952161 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55f928b05dc0 0x55f928b081a0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.402+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55f928b0a560 msgr2=0x55f928b0c970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.402+0000 7f4f36f0ae40 1 --2- 172.21.15.12:0/3627952161 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55f928b0a560 0x55f928b0c970 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.402+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f4f18030c50 msgr2=0x7f4f18033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.402+0000 7f4f36f0ae40 1 --2- 172.21.15.12:0/3627952161 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f4f18030c50 0x7f4f18033100 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c001e90 tx=0x7f4f1c0013b0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.402+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f928ad2010 msgr2=0x55f928af7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.402+0000 7f4f36f0ae40 1 --2- 172.21.15.12:0/3627952161 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f928ad2010 0x55f928af7db0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4f2400a470 tx=0x7f4f2401d000 comp rx=0 tx=0).stop 2023-10-05T19:40:29.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.402+0000 7f4f35240700 1 -- 172.21.15.12:0/3627952161 reap_dead start 2023-10-05T19:40:29.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.402+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 shutdown_connections 2023-10-05T19:40:29.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.402+0000 7f4f36f0ae40 1 --2- 172.21.15.12:0/3627952161 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f928ad2010 0x55f928af7db0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.402+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 >> 172.21.15.12:0/3627952161 conn(0x55f928a45860 msgr2=0x55f928aeb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:29.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.402+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 shutdown_connections 2023-10-05T19:40:29.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.402+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3627952161 wait complete. 2023-10-05T19:40:29.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.402+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3868146301 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55f928b90ba0 msgr2=0x55f928b92f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.402+0000 7f4f36f0ae40 1 --2- 172.21.15.12:0/3868146301 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55f928b90ba0 0x55f928b92f80 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.402+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3868146301 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f4e74030c50 msgr2=0x7f4e74033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.402+0000 7f4f36f0ae40 1 --2- 172.21.15.12:0/3868146301 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f4e74030c50 0x7f4e74033100 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x55f928b14f70 tx=0x7f4f2c00add0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.402+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3868146301 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f928b23b50 msgr2=0x55f928b296f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.402+0000 7f4f36f0ae40 1 --2- 172.21.15.12:0/3868146301 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f928b23b50 0x55f928b296f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c00e550 tx=0x7f4f1c03d030 comp rx=0 tx=0).stop 2023-10-05T19:40:29.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.406+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3868146301 shutdown_connections 2023-10-05T19:40:29.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.406+0000 7f4f36f0ae40 1 --2- 172.21.15.12:0/3868146301 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55f928b90ba0 0x55f928b92f80 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.406+0000 7f4f36f0ae40 1 --2- 172.21.15.12:0/3868146301 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f4e74030c50 0x7f4e74033100 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.406+0000 7f4f36f0ae40 1 --2- 172.21.15.12:0/3868146301 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f928b23b50 0x55f928b296f0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.406+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3868146301 >> 172.21.15.12:0/3868146301 conn(0x55f928b1ef10 msgr2=0x55f928b1cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:29.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.406+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3868146301 shutdown_connections 2023-10-05T19:40:29.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.406+0000 7f4f36f0ae40 1 -- 172.21.15.12:0/3868146301 wait complete. 2023-10-05T19:40:29.421 INFO:tasks.util.rgw:rgwadmin: client.0 : ['log', 'show', '--object', '2023-10-05-19--'] 2023-10-05T19:40:29.421 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'log', 'show', '--object', '2023-10-05-19--'] 2023-10-05T19:40:29.421 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph log show --object 2023-10-05-19-- 2023-10-05T19:40:29.460 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:29.460 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:29.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.450+0000 7f5e9769ee40 1 Processor -- start 2023-10-05T19:40:29.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.450+0000 7f5e9769ee40 1 -- start start 2023-10-05T19:40:29.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.450+0000 7f5e9769ee40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55e5b4d57e30 0x55e5b4d58200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.450+0000 7f5e9769ee40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55e5b4ca4bd0 con 0x55e5b4d58740 2023-10-05T19:40:29.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.450+0000 7f5e9769ee40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x55e5b4c61ee0 con 0x55e5b4d57e30 2023-10-05T19:40:29.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.450+0000 7f5e949d2700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x55e5b4d58740 0x55e5b4d591e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:35682/0 (socket says 172.21.15.12:35682) 2023-10-05T19:40:29.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.450+0000 7f5e949d2700 1 -- 172.21.15.12:0/362749653 learned_addr learned my addr 172.21.15.12:0/362749653 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:29.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.450+0000 7f5e951d3700 1 --2- 172.21.15.12:0/362749653 >> v2:172.21.15.8:3300/0 conn(0x55e5b4d57e30 0x55e5b4d58200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.454+0000 7f5e8ffff700 1 -- 172.21.15.12:0/362749653 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 699540183 0 0) 0x55e5b4ca4bd0 con 0x55e5b4d58740 2023-10-05T19:40:29.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.454+0000 7f5e8ffff700 1 -- 172.21.15.12:0/362749653 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e78001e10 con 0x55e5b4d58740 2023-10-05T19:40:29.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.454+0000 7f5e951d3700 1 -- 172.21.15.12:0/362749653 >> v1:172.21.15.8:6789/0 conn(0x55e5b4d58740 legacy=0x55e5b4d591e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.454+0000 7f5e951d3700 1 -- 172.21.15.12:0/362749653 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5b4d598f0 con 0x55e5b4d57e30 2023-10-05T19:40:29.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.454+0000 7f5e951d3700 1 --2- 172.21.15.12:0/362749653 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5b4d57e30 0x55e5b4d58200 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5e8800a5d0 tx=0x7f5e88005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d42ab821313bb3 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.454+0000 7f5e8ffff700 1 -- 172.21.15.12:0/362749653 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e88007610 con 0x55e5b4d57e30 2023-10-05T19:40:29.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.454+0000 7f5e8ffff700 1 -- 172.21.15.12:0/362749653 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e88007770 con 0x55e5b4d57e30 2023-10-05T19:40:29.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.454+0000 7f5e8ffff700 1 -- 172.21.15.12:0/362749653 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e88007a40 con 0x55e5b4d57e30 2023-10-05T19:40:29.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.454+0000 7f5e9769ee40 1 -- 172.21.15.12:0/362749653 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5b4d57e30 msgr2=0x55e5b4d58200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.454+0000 7f5e9769ee40 1 --2- 172.21.15.12:0/362749653 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5b4d57e30 0x55e5b4d58200 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5e8800a5d0 tx=0x7f5e88005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:29.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.454+0000 7f5e9769ee40 1 -- 172.21.15.12:0/362749653 shutdown_connections 2023-10-05T19:40:29.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.454+0000 7f5e9769ee40 1 --2- 172.21.15.12:0/362749653 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5b4d57e30 0x55e5b4d58200 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.454+0000 7f5e9769ee40 1 -- 172.21.15.12:0/362749653 >> 172.21.15.12:0/362749653 conn(0x55e5b4d53850 msgr2=0x55e5b4d55c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:29.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.454+0000 7f5e9769ee40 1 -- 172.21.15.12:0/362749653 shutdown_connections 2023-10-05T19:40:29.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.454+0000 7f5e9769ee40 1 -- 172.21.15.12:0/362749653 wait complete. 2023-10-05T19:40:29.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.458+0000 7f5e9769ee40 1 Processor -- start 2023-10-05T19:40:29.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.458+0000 7f5e9769ee40 1 -- start start 2023-10-05T19:40:29.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.458+0000 7f5e9769ee40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5b4d57e30 0x55e5b4d76ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.458+0000 7f5e9769ee40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5b4c25e30 con 0x55e5b4d57e30 2023-10-05T19:40:29.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.458+0000 7f5e951d3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5b4d57e30 0x55e5b4d76ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.458+0000 7f5e951d3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5b4d57e30 0x55e5b4d76ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58598/0 (socket says 172.21.15.12:58598) 2023-10-05T19:40:29.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.458+0000 7f5e951d3700 1 -- 172.21.15.12:0/4123148980 learned_addr learned my addr 172.21.15.12:0/4123148980 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:29.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.458+0000 7f5e951d3700 1 -- 172.21.15.12:0/4123148980 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e88005cf0 con 0x55e5b4d57e30 2023-10-05T19:40:29.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.458+0000 7f5e951d3700 1 --2- 172.21.15.12:0/4123148980 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5b4d57e30 0x55e5b4d76ce0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5e88000c00 tx=0x7f5e88014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c24235cd88af0da7 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.458+0000 7f5e8effd700 1 -- 172.21.15.12:0/4123148980 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e880078e0 con 0x55e5b4d57e30 2023-10-05T19:40:29.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.458+0000 7f5e8effd700 1 -- 172.21.15.12:0/4123148980 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e88007a40 con 0x55e5b4d57e30 2023-10-05T19:40:29.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.458+0000 7f5e8effd700 1 -- 172.21.15.12:0/4123148980 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e88007bd0 con 0x55e5b4d57e30 2023-10-05T19:40:29.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.462+0000 7f5e9769ee40 1 -- 172.21.15.12:0/4123148980 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5b4d57e30 msgr2=0x55e5b4d76ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.462+0000 7f5e9769ee40 1 --2- 172.21.15.12:0/4123148980 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5b4d57e30 0x55e5b4d76ce0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5e88000c00 tx=0x7f5e88014000 comp rx=0 tx=0).stop 2023-10-05T19:40:29.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.462+0000 7f5e9769ee40 1 -- 172.21.15.12:0/4123148980 shutdown_connections 2023-10-05T19:40:29.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.462+0000 7f5e9769ee40 1 --2- 172.21.15.12:0/4123148980 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5b4d57e30 0x55e5b4d76ce0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.462+0000 7f5e9769ee40 1 -- 172.21.15.12:0/4123148980 >> 172.21.15.12:0/4123148980 conn(0x55e5b4d53850 msgr2=0x55e5b4d728a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:29.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.462+0000 7f5e9769ee40 1 -- 172.21.15.12:0/4123148980 shutdown_connections 2023-10-05T19:40:29.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.462+0000 7f5e9769ee40 1 -- 172.21.15.12:0/4123148980 wait complete. 2023-10-05T19:40:29.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.462+0000 7f5e9769ee40 1 Processor -- start 2023-10-05T19:40:29.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.462+0000 7f5e9769ee40 1 -- start start 2023-10-05T19:40:29.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.462+0000 7f5e9769ee40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5b4d57e30 0x55e5b4d7d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.462+0000 7f5e9769ee40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5b4bf8ed0 con 0x55e5b4d57e30 2023-10-05T19:40:29.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.462+0000 7f5e951d3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5b4d57e30 0x55e5b4d7d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.462+0000 7f5e951d3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5b4d57e30 0x55e5b4d7d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58602/0 (socket says 172.21.15.12:58602) 2023-10-05T19:40:29.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.462+0000 7f5e951d3700 1 -- 172.21.15.12:0/910014983 learned_addr learned my addr 172.21.15.12:0/910014983 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:29.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.462+0000 7f5e951d3700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5b4d598f0 con 0x55e5b4d57e30 2023-10-05T19:40:29.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.462+0000 7f5e951d3700 1 --2- 172.21.15.12:0/910014983 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5b4d57e30 0x55e5b4d7d9a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5e8801f040 tx=0x7f5e8801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.462+0000 7f5e8d7fa700 1 -- 172.21.15.12:0/910014983 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e88007bb0 con 0x55e5b4d57e30 2023-10-05T19:40:29.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.466+0000 7f5e8d7fa700 1 -- 172.21.15.12:0/910014983 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e88029070 con 0x55e5b4d57e30 2023-10-05T19:40:29.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.466+0000 7f5e8d7fa700 1 -- 172.21.15.12:0/910014983 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e88012610 con 0x55e5b4d57e30 2023-10-05T19:40:29.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.466+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e5b4d7dee0 con 0x55e5b4d57e30 2023-10-05T19:40:29.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.466+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e5b4d7e320 con 0x55e5b4d57e30 2023-10-05T19:40:29.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.466+0000 7f5e8d7fa700 1 -- 172.21.15.12:0/910014983 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f5e88012770 con 0x55e5b4d57e30 2023-10-05T19:40:29.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.466+0000 7f5e8d7fa700 1 --2- 172.21.15.12:0/910014983 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5e7c030ca0 0x7f5e7c033150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.466+0000 7f5e8d7fa700 1 -- 172.21.15.12:0/910014983 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f5e8802b070 con 0x55e5b4d57e30 2023-10-05T19:40:29.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.466+0000 7f5e9769ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.466+0000 7f5e949d2700 1 --2- 172.21.15.12:0/910014983 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5e7c030ca0 0x7f5e7c033150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.466+0000 7f5e9769ee40 1 --2- 172.21.15.12:0/910014983 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55e5b4d58740 0x55e5b4d89370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.466+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d89aa0 con 0x55e5b4d58740 2023-10-05T19:40:29.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.466+0000 7f5e959d4700 1 --2- 172.21.15.12:0/910014983 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55e5b4d58740 0x55e5b4d89370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.466+0000 7f5e949d2700 1 --2- 172.21.15.12:0/910014983 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5e7c030ca0 0x7f5e7c033150 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5e800044f0 tx=0x7f5e80006690 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.466+0000 7f5e959d4700 1 --2- 172.21.15.12:0/910014983 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55e5b4d58740 0x55e5b4d89370 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.470+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5e9000a500 con 0x55e5b4d58740 2023-10-05T19:40:29.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.470+0000 7f5e9769ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:29.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.470+0000 7f5e9769ee40 20 realm 2023-10-05T19:40:29.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.470+0000 7f5e9769ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.470+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d869a0 con 0x55e5b4d58740 2023-10-05T19:40:29.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.470+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5e9000a500 con 0x55e5b4d58740 2023-10-05T19:40:29.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.470+0000 7f5e9769ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:29.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.470+0000 7f5e9769ee40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:29.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.470+0000 7f5e9769ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.470+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d8a5d0 con 0x55e5b4d58740 2023-10-05T19:40:29.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.470+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5e9000a500 con 0x55e5b4d58740 2023-10-05T19:40:29.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.470+0000 7f5e9769ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:29.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.470+0000 7f5e9769ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.470+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d8a1b0 con 0x55e5b4d58740 2023-10-05T19:40:29.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.470+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f5e9000a500 con 0x55e5b4d58740 2023-10-05T19:40:29.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.470+0000 7f5e9769ee40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:29.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.470+0000 7f5e9769ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.470+0000 7f5e9769ee40 1 --2- 172.21.15.12:0/910014983 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55e5b4d8b560 0x55e5b4d8d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.470+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d8e0b0 con 0x55e5b4d8b560 2023-10-05T19:40:29.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.470+0000 7f5e951d3700 1 --2- 172.21.15.12:0/910014983 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55e5b4d8b560 0x55e5b4d8d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.470+0000 7f5e951d3700 1 --2- 172.21.15.12:0/910014983 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55e5b4d8b560 0x55e5b4d8d940 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.470+0000 7f5e951d3700 1 -- 172.21.15.12:0/910014983 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f5e8801cc20 con 0x55e5b4d8b560 2023-10-05T19:40:29.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.470+0000 7f5e9769ee40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:29.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.470+0000 7f5e9769ee40 20 searching for the correct realm 2023-10-05T19:40:29.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.474+0000 7f5e9769ee40 1 --2- 172.21.15.12:0/910014983 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55e5b4d8fee0 0x55e5b4d922c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.474+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d92800 con 0x55e5b4d8fee0 2023-10-05T19:40:29.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.474+0000 7f5e949d2700 1 --2- 172.21.15.12:0/910014983 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55e5b4d8fee0 0x55e5b4d922c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.474+0000 7f5e949d2700 1 --2- 172.21.15.12:0/910014983 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55e5b4d8fee0 0x55e5b4d922c0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.474+0000 7f5e949d2700 1 -- 172.21.15.12:0/910014983 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f5e80004520 con 0x55e5b4d8fee0 2023-10-05T19:40:29.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.474+0000 7f5e949d2700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d92800 con 0x55e5b4d8b560 2023-10-05T19:40:29.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.474+0000 7f5e951d3700 1 -- 172.21.15.12:0/910014983 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e8801cc20 con 0x55e5b4d8b560 2023-10-05T19:40:29.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.474+0000 7f5e951d3700 1 --2- 172.21.15.12:0/910014983 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f5e88007030 0x7f5e880113b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.474+0000 7f5e951d3700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d92800 con 0x7f5e88007030 2023-10-05T19:40:29.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.474+0000 7f5e959d4700 1 --2- 172.21.15.12:0/910014983 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f5e88007030 0x7f5e880113b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.478+0000 7f5e959d4700 1 --2- 172.21.15.12:0/910014983 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f5e88007030 0x7f5e880113b0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.478+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e90014040 con 0x7f5e88007030 2023-10-05T19:40:29.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.478+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d92800 con 0x55e5b4d58740 2023-10-05T19:40:29.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.478+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e90014040 con 0x55e5b4d58740 2023-10-05T19:40:29.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.478+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d92800 con 0x55e5b4d58740 2023-10-05T19:40:29.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.478+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e90014040 con 0x55e5b4d58740 2023-10-05T19:40:29.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.478+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d92800 con 0x7f5e88007030 2023-10-05T19:40:29.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.478+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e90014040 con 0x7f5e88007030 2023-10-05T19:40:29.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.478+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d92800 con 0x7f5e88007030 2023-10-05T19:40:29.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.482+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e90014040 con 0x7f5e88007030 2023-10-05T19:40:29.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.482+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d92800 con 0x55e5b4d8b560 2023-10-05T19:40:29.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.482+0000 7f5e951d3700 1 -- 172.21.15.12:0/910014983 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e8801cc20 con 0x55e5b4d8b560 2023-10-05T19:40:29.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.482+0000 7f5e951d3700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d92800 con 0x55e5b4d8fee0 2023-10-05T19:40:29.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.482+0000 7f5e949d2700 1 -- 172.21.15.12:0/910014983 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e80004520 con 0x55e5b4d8fee0 2023-10-05T19:40:29.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.482+0000 7f5e949d2700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d92800 con 0x55e5b4d58740 2023-10-05T19:40:29.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.482+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e90014040 con 0x55e5b4d58740 2023-10-05T19:40:29.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.482+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d92800 con 0x7f5e88007030 2023-10-05T19:40:29.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.482+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e90014040 con 0x7f5e88007030 2023-10-05T19:40:29.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.482+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d92800 con 0x55e5b4d58740 2023-10-05T19:40:29.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.486+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e90014040 con 0x55e5b4d58740 2023-10-05T19:40:29.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.486+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d92800 con 0x55e5b4d8b560 2023-10-05T19:40:29.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.486+0000 7f5e951d3700 1 -- 172.21.15.12:0/910014983 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e8801cc20 con 0x55e5b4d8b560 2023-10-05T19:40:29.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.486+0000 7f5e951d3700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d92800 con 0x55e5b4d8fee0 2023-10-05T19:40:29.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.486+0000 7f5e949d2700 1 -- 172.21.15.12:0/910014983 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e80004520 con 0x55e5b4d8fee0 2023-10-05T19:40:29.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.486+0000 7f5e949d2700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d92800 con 0x55e5b4d8fee0 2023-10-05T19:40:29.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.486+0000 7f5e949d2700 1 -- 172.21.15.12:0/910014983 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e80004520 con 0x55e5b4d8fee0 2023-10-05T19:40:29.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.486+0000 7f5e949d2700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d92800 con 0x7f5e88007030 2023-10-05T19:40:29.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.486+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e90014040 con 0x7f5e88007030 2023-10-05T19:40:29.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.486+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d92800 con 0x55e5b4d8fee0 2023-10-05T19:40:29.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.490+0000 7f5e949d2700 1 -- 172.21.15.12:0/910014983 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e80004520 con 0x55e5b4d8fee0 2023-10-05T19:40:29.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.490+0000 7f5e949d2700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d92800 con 0x55e5b4d8fee0 2023-10-05T19:40:29.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.490+0000 7f5e949d2700 1 -- 172.21.15.12:0/910014983 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e80004520 con 0x55e5b4d8fee0 2023-10-05T19:40:29.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.490+0000 7f5e949d2700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d92800 con 0x55e5b4d58740 2023-10-05T19:40:29.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.490+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e90014040 con 0x55e5b4d58740 2023-10-05T19:40:29.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.490+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d92800 con 0x55e5b4d58740 2023-10-05T19:40:29.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.490+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e90014040 con 0x55e5b4d58740 2023-10-05T19:40:29.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.490+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d92800 con 0x7f5e88007030 2023-10-05T19:40:29.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.490+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e90014040 con 0x7f5e88007030 2023-10-05T19:40:29.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.490+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d92800 con 0x55e5b4d8fee0 2023-10-05T19:40:29.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.494+0000 7f5e949d2700 1 -- 172.21.15.12:0/910014983 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e80004520 con 0x55e5b4d8fee0 2023-10-05T19:40:29.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.494+0000 7f5e949d2700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d92800 con 0x55e5b4d8b560 2023-10-05T19:40:29.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.494+0000 7f5e951d3700 1 -- 172.21.15.12:0/910014983 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e8801cc20 con 0x55e5b4d8b560 2023-10-05T19:40:29.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.494+0000 7f5e951d3700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d92800 con 0x55e5b4d58740 2023-10-05T19:40:29.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.494+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e90014040 con 0x55e5b4d58740 2023-10-05T19:40:29.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.494+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d92800 con 0x7f5e88007030 2023-10-05T19:40:29.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.494+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e90014040 con 0x7f5e88007030 2023-10-05T19:40:29.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.494+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d92800 con 0x55e5b4d8fee0 2023-10-05T19:40:29.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.498+0000 7f5e949d2700 1 -- 172.21.15.12:0/910014983 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e80004520 con 0x55e5b4d8fee0 2023-10-05T19:40:29.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.498+0000 7f5e949d2700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d92800 con 0x7f5e88007030 2023-10-05T19:40:29.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.498+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e90014040 con 0x7f5e88007030 2023-10-05T19:40:29.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.498+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d92800 con 0x55e5b4d8b560 2023-10-05T19:40:29.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.498+0000 7f5e951d3700 1 -- 172.21.15.12:0/910014983 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e8801cc20 con 0x55e5b4d8b560 2023-10-05T19:40:29.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.498+0000 7f5e951d3700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d92800 con 0x55e5b4d8fee0 2023-10-05T19:40:29.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.498+0000 7f5e949d2700 1 -- 172.21.15.12:0/910014983 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e80004520 con 0x55e5b4d8fee0 2023-10-05T19:40:29.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.498+0000 7f5e949d2700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d92800 con 0x7f5e88007030 2023-10-05T19:40:29.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.502+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e90014040 con 0x7f5e88007030 2023-10-05T19:40:29.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.502+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d92800 con 0x55e5b4d58740 2023-10-05T19:40:29.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.502+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f5e90014040 con 0x55e5b4d58740 2023-10-05T19:40:29.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.502+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d92800 con 0x55e5b4d8b560 2023-10-05T19:40:29.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.502+0000 7f5e951d3700 1 -- 172.21.15.12:0/910014983 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f5e8801cc20 con 0x55e5b4d8b560 2023-10-05T19:40:29.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.502+0000 7f5e9769ee40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:29.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.502+0000 7f5e9769ee40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:29.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.502+0000 7f5e9769ee40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:29.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.502+0000 7f5e9769ee40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:29.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.502+0000 7f5e9769ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.502+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d8ec90 con 0x55e5b4d58740 2023-10-05T19:40:29.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.502+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5e90014040 con 0x55e5b4d58740 2023-10-05T19:40:29.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.502+0000 7f5e9769ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:29.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.502+0000 7f5e9769ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.502+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d8f5e0 con 0x55e5b4d58740 2023-10-05T19:40:29.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.502+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f5e90014040 con 0x55e5b4d58740 2023-10-05T19:40:29.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.502+0000 7f5e9769ee40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:29.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.502+0000 7f5e9769ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.502+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d8e850 con 0x55e5b4d8fee0 2023-10-05T19:40:29.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.506+0000 7f5e949d2700 1 -- 172.21.15.12:0/910014983 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f5e80004520 con 0x55e5b4d8fee0 2023-10-05T19:40:29.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.506+0000 7f5e9769ee40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:29.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.506+0000 7f5e9769ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.506+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d93130 con 0x55e5b4d8fee0 2023-10-05T19:40:29.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.506+0000 7f5e949d2700 1 -- 172.21.15.12:0/910014983 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f5e80004520 con 0x55e5b4d8fee0 2023-10-05T19:40:29.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.506+0000 7f5e9769ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:29.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.506+0000 7f5e9769ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.506+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d934e0 con 0x55e5b4d58740 2023-10-05T19:40:29.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.506+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5e90014040 con 0x55e5b4d58740 2023-10-05T19:40:29.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.506+0000 7f5e9769ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:29.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.506+0000 7f5e9769ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.506+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d93890 con 0x55e5b4d58740 2023-10-05T19:40:29.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.506+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f5e90014040 con 0x55e5b4d58740 2023-10-05T19:40:29.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.506+0000 7f5e9769ee40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:29.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.506+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d948a0 con 0x55e5b4d8fee0 2023-10-05T19:40:29.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.510+0000 7f5e949d2700 1 -- 172.21.15.12:0/910014983 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f5e80004520 con 0x55e5b4d8fee0 2023-10-05T19:40:29.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.510+0000 7f5e949d2700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d948a0 con 0x55e5b4d8b560 2023-10-05T19:40:29.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.510+0000 7f5e951d3700 1 -- 172.21.15.12:0/910014983 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e8801cc20 con 0x55e5b4d8b560 2023-10-05T19:40:29.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.510+0000 7f5e951d3700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d948a0 con 0x7f5e88007030 2023-10-05T19:40:29.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.510+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e90014040 con 0x7f5e88007030 2023-10-05T19:40:29.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.510+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d934e0 con 0x55e5b4d58740 2023-10-05T19:40:29.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.510+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e90014040 con 0x55e5b4d58740 2023-10-05T19:40:29.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.510+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d934e0 con 0x55e5b4d58740 2023-10-05T19:40:29.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.514+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e90014040 con 0x55e5b4d58740 2023-10-05T19:40:29.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.514+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d934e0 con 0x7f5e88007030 2023-10-05T19:40:29.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.514+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e90014040 con 0x7f5e88007030 2023-10-05T19:40:29.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.514+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d934e0 con 0x7f5e88007030 2023-10-05T19:40:29.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.514+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e90014040 con 0x7f5e88007030 2023-10-05T19:40:29.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.514+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d934e0 con 0x55e5b4d8b560 2023-10-05T19:40:29.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.514+0000 7f5e951d3700 1 -- 172.21.15.12:0/910014983 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e8801cc20 con 0x55e5b4d8b560 2023-10-05T19:40:29.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.514+0000 7f5e951d3700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d934e0 con 0x55e5b4d8fee0 2023-10-05T19:40:29.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.518+0000 7f5e949d2700 1 -- 172.21.15.12:0/910014983 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e8002d040 con 0x55e5b4d8fee0 2023-10-05T19:40:29.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.518+0000 7f5e949d2700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d934e0 con 0x55e5b4d58740 2023-10-05T19:40:29.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.518+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e90014040 con 0x55e5b4d58740 2023-10-05T19:40:29.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.518+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d934e0 con 0x7f5e88007030 2023-10-05T19:40:29.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.518+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e90014040 con 0x7f5e88007030 2023-10-05T19:40:29.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.518+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d934e0 con 0x55e5b4d58740 2023-10-05T19:40:29.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.518+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e90014040 con 0x55e5b4d58740 2023-10-05T19:40:29.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.518+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d934e0 con 0x55e5b4d8b560 2023-10-05T19:40:29.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.518+0000 7f5e951d3700 1 -- 172.21.15.12:0/910014983 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e8801cc20 con 0x55e5b4d8b560 2023-10-05T19:40:29.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.518+0000 7f5e951d3700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d934e0 con 0x55e5b4d8fee0 2023-10-05T19:40:29.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.522+0000 7f5e949d2700 1 -- 172.21.15.12:0/910014983 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e8002d040 con 0x55e5b4d8fee0 2023-10-05T19:40:29.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.522+0000 7f5e949d2700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d934e0 con 0x55e5b4d8fee0 2023-10-05T19:40:29.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.522+0000 7f5e949d2700 1 -- 172.21.15.12:0/910014983 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e8002d040 con 0x55e5b4d8fee0 2023-10-05T19:40:29.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.522+0000 7f5e949d2700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d934e0 con 0x7f5e88007030 2023-10-05T19:40:29.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.522+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e90014040 con 0x7f5e88007030 2023-10-05T19:40:29.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.522+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d934e0 con 0x55e5b4d8fee0 2023-10-05T19:40:29.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.522+0000 7f5e949d2700 1 -- 172.21.15.12:0/910014983 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e8002d040 con 0x55e5b4d8fee0 2023-10-05T19:40:29.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.522+0000 7f5e949d2700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f5e80031040 con 0x55e5b4d8fee0 2023-10-05T19:40:29.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.522+0000 7f5e949d2700 1 -- 172.21.15.12:0/910014983 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e8002d040 con 0x55e5b4d8fee0 2023-10-05T19:40:29.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.522+0000 7f5e949d2700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f5e80031040 con 0x55e5b4d58740 2023-10-05T19:40:29.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.526+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e90014040 con 0x55e5b4d58740 2023-10-05T19:40:29.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.526+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f5e80031040 con 0x55e5b4d58740 2023-10-05T19:40:29.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.526+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e90014040 con 0x55e5b4d58740 2023-10-05T19:40:29.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.526+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f5e80031040 con 0x7f5e88007030 2023-10-05T19:40:29.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.526+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e90014040 con 0x7f5e88007030 2023-10-05T19:40:29.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.526+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f5e80031040 con 0x55e5b4d8fee0 2023-10-05T19:40:29.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.526+0000 7f5e949d2700 1 -- 172.21.15.12:0/910014983 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e8002d040 con 0x55e5b4d8fee0 2023-10-05T19:40:29.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.526+0000 7f5e949d2700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f5e80031040 con 0x55e5b4d8b560 2023-10-05T19:40:29.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.530+0000 7f5e951d3700 1 -- 172.21.15.12:0/910014983 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e8801cc20 con 0x55e5b4d8b560 2023-10-05T19:40:29.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.530+0000 7f5e951d3700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f5e80031040 con 0x55e5b4d58740 2023-10-05T19:40:29.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.530+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e90014040 con 0x55e5b4d58740 2023-10-05T19:40:29.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.530+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f5e80031040 con 0x7f5e88007030 2023-10-05T19:40:29.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.530+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e90014040 con 0x7f5e88007030 2023-10-05T19:40:29.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.530+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f5e80031040 con 0x55e5b4d8fee0 2023-10-05T19:40:29.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.530+0000 7f5e949d2700 1 -- 172.21.15.12:0/910014983 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e8002d040 con 0x55e5b4d8fee0 2023-10-05T19:40:29.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.530+0000 7f5e949d2700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f5e80031040 con 0x7f5e88007030 2023-10-05T19:40:29.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.530+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e90014040 con 0x7f5e88007030 2023-10-05T19:40:29.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.530+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f5e80031040 con 0x55e5b4d8b560 2023-10-05T19:40:29.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.534+0000 7f5e951d3700 1 -- 172.21.15.12:0/910014983 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e8801cc20 con 0x55e5b4d8b560 2023-10-05T19:40:29.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.534+0000 7f5e951d3700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f5e80031040 con 0x55e5b4d8fee0 2023-10-05T19:40:29.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.534+0000 7f5e949d2700 1 -- 172.21.15.12:0/910014983 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e8002d040 con 0x55e5b4d8fee0 2023-10-05T19:40:29.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.534+0000 7f5e949d2700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f5e80031040 con 0x7f5e88007030 2023-10-05T19:40:29.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.534+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e90014040 con 0x7f5e88007030 2023-10-05T19:40:29.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.534+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f5e80031040 con 0x55e5b4d58740 2023-10-05T19:40:29.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.534+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f5e90014040 con 0x55e5b4d58740 2023-10-05T19:40:29.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.534+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f5e80031040 con 0x55e5b4d8b560 2023-10-05T19:40:29.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.538+0000 7f5e951d3700 1 -- 172.21.15.12:0/910014983 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f5e8801cc20 con 0x55e5b4d8b560 2023-10-05T19:40:29.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.538+0000 7f5e9769ee40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:29.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.538+0000 7f5e9769ee40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:29.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.538+0000 7f5e9769ee40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:29.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.538+0000 7f5e9769ee40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:29.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.538+0000 7f5e9769ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.538+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d93890 con 0x55e5b4d58740 2023-10-05T19:40:29.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.538+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f5e90014040 con 0x55e5b4d58740 2023-10-05T19:40:29.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.538+0000 7f5e9769ee40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:29.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.538+0000 7f5e9769ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.538+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d93c40 con 0x55e5b4d8b560 2023-10-05T19:40:29.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.538+0000 7f5e951d3700 1 -- 172.21.15.12:0/910014983 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f5e8801cc20 con 0x55e5b4d8b560 2023-10-05T19:40:29.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.538+0000 7f5e9769ee40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:29.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.538+0000 7f5e9769ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.538+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d92d60 con 0x55e5b4d58740 2023-10-05T19:40:29.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.538+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f5e90014040 con 0x55e5b4d58740 2023-10-05T19:40:29.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.538+0000 7f5e9769ee40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:29.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.538+0000 7f5e9769ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.538+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d92d70 con 0x55e5b4d8fee0 2023-10-05T19:40:29.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.538+0000 7f5e949d2700 1 -- 172.21.15.12:0/910014983 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f5e8002d040 con 0x55e5b4d8fee0 2023-10-05T19:40:29.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.538+0000 7f5e9769ee40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:29.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.538+0000 7f5e9769ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.538+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d94ae0 con 0x7f5e88007030 2023-10-05T19:40:29.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.542+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f5e90014040 con 0x7f5e88007030 2023-10-05T19:40:29.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.542+0000 7f5e9769ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:29.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.542+0000 7f5e9769ee40 20 zone default found 2023-10-05T19:40:29.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.542+0000 7f5e9769ee40 4 Realm: () 2023-10-05T19:40:29.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.542+0000 7f5e9769ee40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:29.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.542+0000 7f5e9769ee40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:29.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.542+0000 7f5e9769ee40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:29.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.542+0000 7f5e9769ee40 20 zonegroup default 2023-10-05T19:40:29.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.542+0000 7f5e9769ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.542+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d94ae0 con 0x7f5e88007030 2023-10-05T19:40:29.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.542+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f5e90014040 con 0x7f5e88007030 2023-10-05T19:40:29.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.542+0000 7f5e9769ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:29.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.542+0000 7f5e9769ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.542+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d96460 con 0x55e5b4d8fee0 2023-10-05T19:40:29.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.542+0000 7f5e949d2700 1 -- 172.21.15.12:0/910014983 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f5e8002d040 con 0x55e5b4d8fee0 2023-10-05T19:40:29.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.542+0000 7f5e9769ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:29.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.542+0000 7f5e9769ee40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.542+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d96810 con 0x55e5b4d8fee0 2023-10-05T19:40:29.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.542+0000 7f5e949d2700 1 -- 172.21.15.12:0/910014983 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f5e8002d040 con 0x55e5b4d8fee0 2023-10-05T19:40:29.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.542+0000 7f5e9769ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:29.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.542+0000 7f5e9769ee40 20 started sync module instance, tier type = 2023-10-05T19:40:29.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.542+0000 7f5e9769ee40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:29.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.542+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d95b20 con 0x55e5b4d8b560 2023-10-05T19:40:29.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.546+0000 7f5e951d3700 1 -- 172.21.15.12:0/910014983 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'159 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e8801cc20 con 0x55e5b4d8b560 2023-10-05T19:40:29.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.546+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94445070014000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d96bc0 con 0x55e5b4d8b560 2023-10-05T19:40:29.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.546+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d97460 con 0x55e5b4d58740 2023-10-05T19:40:29.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.550+0000 7f5e951d3700 1 -- 172.21.15.12:0/910014983 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94445070014000] v28'160 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e8801cc20 con 0x55e5b4d8b560 2023-10-05T19:40:29.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.550+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'159 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e90014040 con 0x55e5b4d58740 2023-10-05T19:40:29.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.550+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94445070020464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d960b0 con 0x55e5b4d58740 2023-10-05T19:40:29.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.550+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d98070 con 0x55e5b4d8fee0 2023-10-05T19:40:29.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.550+0000 7f5e949d2700 1 -- 172.21.15.12:0/910014983 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'159 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e8002d040 con 0x55e5b4d8fee0 2023-10-05T19:40:29.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.550+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94445070023488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d98960 con 0x55e5b4d8fee0 2023-10-05T19:40:29.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.550+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d99370 con 0x55e5b4d8b560 2023-10-05T19:40:29.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.554+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94445070020464] v28'160 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e90014040 con 0x55e5b4d58740 2023-10-05T19:40:29.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.554+0000 7f5e951d3700 1 -- 172.21.15.12:0/910014983 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'159 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e8801cc20 con 0x55e5b4d8b560 2023-10-05T19:40:29.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.554+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94445070021568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d99370 con 0x55e5b4d8b560 2023-10-05T19:40:29.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.554+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d99c10 con 0x55e5b4d58740 2023-10-05T19:40:29.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.554+0000 7f5e949d2700 1 -- 172.21.15.12:0/910014983 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94445070023488] v28'160 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e8002d040 con 0x55e5b4d8fee0 2023-10-05T19:40:29.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.558+0000 7f5e951d3700 1 -- 172.21.15.12:0/910014983 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94445070021568] v28'160 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e8801cc20 con 0x55e5b4d8b560 2023-10-05T19:40:29.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.558+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'159 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e90014040 con 0x55e5b4d58740 2023-10-05T19:40:29.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.558+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94445070028416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d997c0 con 0x55e5b4d58740 2023-10-05T19:40:29.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.558+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d9a060 con 0x7f5e88007030 2023-10-05T19:40:29.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.558+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94445070028416] v28'160 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e90014040 con 0x55e5b4d58740 2023-10-05T19:40:29.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.558+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'159 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e90014040 con 0x7f5e88007030 2023-10-05T19:40:29.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.558+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94445070030080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d99e40 con 0x7f5e88007030 2023-10-05T19:40:29.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.558+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d9a950 con 0x7f5e88007030 2023-10-05T19:40:29.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.562+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(96 notify.6 [create] v28'159 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e90014040 con 0x7f5e88007030 2023-10-05T19:40:29.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.562+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94445070033952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d9ad60 con 0x7f5e88007030 2023-10-05T19:40:29.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.562+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d9b840 con 0x7f5e88007030 2023-10-05T19:40:29.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.562+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(95 notify.5 [watch watch cookie 94445070030080] v28'160 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e90014040 con 0x7f5e88007030 2023-10-05T19:40:29.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.566+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(98 notify.7 [create] v28'159 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e90014040 con 0x7f5e88007030 2023-10-05T19:40:29.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.566+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94445070031184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d9b800 con 0x7f5e88007030 2023-10-05T19:40:29.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.566+0000 7f5e9769ee40 20 add_watcher() i=0 2023-10-05T19:40:29.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.566+0000 7f5e9769ee40 20 add_watcher() i=1 2023-10-05T19:40:29.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.566+0000 7f5e9769ee40 20 add_watcher() i=2 2023-10-05T19:40:29.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.566+0000 7f5e9769ee40 20 add_watcher() i=3 2023-10-05T19:40:29.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.566+0000 7f5e9769ee40 20 add_watcher() i=4 2023-10-05T19:40:29.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.566+0000 7f5e9769ee40 20 add_watcher() i=5 2023-10-05T19:40:29.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.566+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(97 notify.6 [watch watch cookie 94445070033952] v28'160 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e90014040 con 0x7f5e88007030 2023-10-05T19:40:29.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.566+0000 7f5e9769ee40 20 add_watcher() i=6 2023-10-05T19:40:29.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.566+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94445070031184] v28'160 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e90014040 con 0x7f5e88007030 2023-10-05T19:40:29.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.566+0000 7f5e9769ee40 20 add_watcher() i=7 2023-10-05T19:40:29.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.566+0000 7f5e9769ee40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:29.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.570+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d9e830 con 0x55e5b4d58740 2023-10-05T19:40:29.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.570+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f5e90014040 con 0x55e5b4d58740 2023-10-05T19:40:29.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.570+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94445070037776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d9bc50 con 0x55e5b4d58740 2023-10-05T19:40:29.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.570+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94445070037776] v28'153 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f5e90014040 con 0x55e5b4d58740 2023-10-05T19:40:29.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.570+0000 7f5e65feb700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:29.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.570+0000 7f5e9769ee40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:29.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.570+0000 7f5e9769ee40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:29.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.570+0000 7f5e9769ee40 1 Processor -- start 2023-10-05T19:40:29.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.570+0000 7f5e9769ee40 1 -- start start 2023-10-05T19:40:29.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.574+0000 7f5e9769ee40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5b4da8a10 0x55e5b4da8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.574+0000 7f5e9769ee40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5b4da9320 con 0x55e5b4da8a10 2023-10-05T19:40:29.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.574+0000 7f5e949d2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5b4da8a10 0x55e5b4da8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.574+0000 7f5e949d2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5b4da8a10 0x55e5b4da8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58610/0 (socket says 172.21.15.12:58610) 2023-10-05T19:40:29.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.574+0000 7f5e949d2700 1 -- 172.21.15.12:0/2454113036 learned_addr learned my addr 172.21.15.12:0/2454113036 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:29.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.574+0000 7f5e949d2700 1 -- 172.21.15.12:0/2454113036 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5b4da9460 con 0x55e5b4da8a10 2023-10-05T19:40:29.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.574+0000 7f5e949d2700 1 --2- 172.21.15.12:0/2454113036 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5b4da8a10 0x55e5b4da8de0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5e88001eb0 tx=0x7f5e80027040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86466705d713ed5d server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.574+0000 7f5e657ea700 1 -- 172.21.15.12:0/2454113036 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e80038040 con 0x55e5b4da8a10 2023-10-05T19:40:29.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.574+0000 7f5e657ea700 1 -- 172.21.15.12:0/2454113036 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e80039040 con 0x55e5b4da8a10 2023-10-05T19:40:29.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.574+0000 7f5e657ea700 1 -- 172.21.15.12:0/2454113036 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e8002a040 con 0x55e5b4da8a10 2023-10-05T19:40:29.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.574+0000 7f5e9769ee40 1 -- 172.21.15.12:0/2454113036 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5b4da8a10 msgr2=0x55e5b4da8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.574+0000 7f5e9769ee40 1 --2- 172.21.15.12:0/2454113036 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5b4da8a10 0x55e5b4da8de0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5e88001eb0 tx=0x7f5e80027040 comp rx=0 tx=0).stop 2023-10-05T19:40:29.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.574+0000 7f5e9769ee40 1 -- 172.21.15.12:0/2454113036 shutdown_connections 2023-10-05T19:40:29.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.574+0000 7f5e9769ee40 1 --2- 172.21.15.12:0/2454113036 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5b4da8a10 0x55e5b4da8de0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.574+0000 7f5e9769ee40 1 -- 172.21.15.12:0/2454113036 >> 172.21.15.12:0/2454113036 conn(0x55e5b4da3dd0 msgr2=0x55e5b4da1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:29.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.574+0000 7f5e9769ee40 1 -- 172.21.15.12:0/2454113036 shutdown_connections 2023-10-05T19:40:29.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.574+0000 7f5e9769ee40 1 -- 172.21.15.12:0/2454113036 wait complete. 2023-10-05T19:40:29.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.578+0000 7f5e9769ee40 1 Processor -- start 2023-10-05T19:40:29.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.578+0000 7f5e9769ee40 1 -- start start 2023-10-05T19:40:29.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.578+0000 7f5e9769ee40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5b4da8a10 0x55e5b4dae590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.578+0000 7f5e9769ee40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5b4daf590 con 0x55e5b4da8a10 2023-10-05T19:40:29.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.578+0000 7f5e949d2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5b4da8a10 0x55e5b4dae590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.578+0000 7f5e949d2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5b4da8a10 0x55e5b4dae590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58616/0 (socket says 172.21.15.12:58616) 2023-10-05T19:40:29.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.578+0000 7f5e949d2700 1 -- 172.21.15.12:0/1484634197 learned_addr learned my addr 172.21.15.12:0/1484634197 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:29.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.578+0000 7f5e949d2700 1 -- 172.21.15.12:0/1484634197 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e78001e10 con 0x55e5b4da8a10 2023-10-05T19:40:29.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.578+0000 7f5e949d2700 1 --2- 172.21.15.12:0/1484634197 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5b4da8a10 0x55e5b4dae590 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x55e5b4da2af0 tx=0x7f5e8003b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.578+0000 7f5ddf7fe700 1 -- 172.21.15.12:0/1484634197 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e80038030 con 0x55e5b4da8a10 2023-10-05T19:40:29.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.578+0000 7f5ddf7fe700 1 -- 172.21.15.12:0/1484634197 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e8001b040 con 0x55e5b4da8a10 2023-10-05T19:40:29.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.578+0000 7f5e9769ee40 1 -- 172.21.15.12:0/1484634197 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e5b4daeb90 con 0x55e5b4da8a10 2023-10-05T19:40:29.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.578+0000 7f5ddf7fe700 1 -- 172.21.15.12:0/1484634197 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e8002a040 con 0x55e5b4da8a10 2023-10-05T19:40:29.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.578+0000 7f5e9769ee40 1 -- 172.21.15.12:0/1484634197 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e5b4daefd0 con 0x55e5b4da8a10 2023-10-05T19:40:29.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.582+0000 7f5ddf7fe700 1 -- 172.21.15.12:0/1484634197 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f5e8004ca40 con 0x55e5b4da8a10 2023-10-05T19:40:29.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.582+0000 7f5ddf7fe700 1 --2- 172.21.15.12:0/1484634197 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5dd4030c50 0x7f5dd4033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.582+0000 7f5ddf7fe700 1 -- 172.21.15.12:0/1484634197 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f5e80056ab0 con 0x55e5b4da8a10 2023-10-05T19:40:29.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.582+0000 7f5e959d4700 1 --2- 172.21.15.12:0/1484634197 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5dd4030c50 0x7f5dd4033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.582+0000 7f5e9769ee40 5 note: GC not initialized 2023-10-05T19:40:29.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.582+0000 7f5e959d4700 1 --2- 172.21.15.12:0/1484634197 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5dd4030c50 0x7f5dd4033100 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x55e5b4d9aae0 tx=0x7f5e9000ab20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.582+0000 7f5ddcff9700 20 reqs_thread_entry: start 2023-10-05T19:40:29.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.582+0000 7f5dd27fc700 20 reqs_thread_entry: start 2023-10-05T19:40:29.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.582+0000 7f5e9769ee40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:29.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.582+0000 7f5e9769ee40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:29.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.586+0000 7f5dd0ff9700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:40:59 2023 2023-10-05T19:40:29.594 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:29.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.586+0000 7f5db7fff700 20 reqs_thread_entry: start 2023-10-05T19:40:29.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.586+0000 7f5e9769ee40 10 log_show_next pos 0 bl 0 off 0 eof 0 2023-10-05T19:40:29.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.586+0000 7f5e9769ee40 1 --2- 172.21.15.12:0/1484634197 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55e5b4e15c60 0x55e5b4e18040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.586+0000 7f5e9769ee40 1 -- 172.21.15.12:0/1484634197 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:1 6.8 6:163360d8:::2023-10-05-19--:head [read 0~1048576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4e18580 con 0x55e5b4e15c60 2023-10-05T19:40:29.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.586+0000 7f5e951d3700 1 --2- 172.21.15.12:0/1484634197 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55e5b4e15c60 0x55e5b4e18040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.586+0000 7f5e951d3700 1 --2- 172.21.15.12:0/1484634197 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55e5b4e15c60 0x55e5b4e18040 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.586+0000 7f5e951d3700 1 -- 172.21.15.12:0/1484634197 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(1 2023-10-05-19-- [read 0~657 out=657b] v0'0 uv21 ondisk = 0) v8 ==== 159+0+657 (crc 0 0 0) 0x7f5e8801cc20 con 0x55e5b4e15c60 2023-10-05T19:40:29.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.586+0000 7f5e9769ee40 10 read 657 2023-10-05T19:40:29.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.586+0000 7f5e9769ee40 10 log_show_next pos 657 bl 657 off 303 eof 1 2023-10-05T19:40:29.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.586+0000 7f5e9769ee40 10 log_show_next pos 657 bl 657 off 657 eof 1 2023-10-05T19:40:29.596 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:40:29.596 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket_id": "", 2023-10-05T19:40:29.597 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket_owner": "", 2023-10-05T19:40:29.597 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "", 2023-10-05T19:40:29.597 INFO:teuthology.orchestra.run.smithi012.stdout: "log_entries": [ 2023-10-05T19:40:29.597 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:29.597 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "", 2023-10-05T19:40:29.597 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:40:15.811286Z", 2023-10-05T19:40:29.597 INFO:teuthology.orchestra.run.smithi012.stdout: "time_local": "2023-10-05T19:40:15.811286+0000", 2023-10-05T19:40:29.597 INFO:teuthology.orchestra.run.smithi012.stdout: "remote_addr": "172.21.15.12", 2023-10-05T19:40:29.597 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "anonymous", 2023-10-05T19:40:29.597 INFO:teuthology.orchestra.run.smithi012.stdout: "operation": "list_buckets", 2023-10-05T19:40:29.597 INFO:teuthology.orchestra.run.smithi012.stdout: "uri": "GET / HTTP/1.1", 2023-10-05T19:40:29.598 INFO:teuthology.orchestra.run.smithi012.stdout: "http_status": "200", 2023-10-05T19:40:29.598 INFO:teuthology.orchestra.run.smithi012.stdout: "error_code": "", 2023-10-05T19:40:29.598 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 231, 2023-10-05T19:40:29.598 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:40:29.598 INFO:teuthology.orchestra.run.smithi012.stdout: "object_size": 0, 2023-10-05T19:40:29.598 INFO:teuthology.orchestra.run.smithi012.stdout: "total_time": 0, 2023-10-05T19:40:29.598 INFO:teuthology.orchestra.run.smithi012.stdout: "user_agent": "curl/7.68.0", 2023-10-05T19:40:29.598 INFO:teuthology.orchestra.run.smithi012.stdout: "referrer": "", 2023-10-05T19:40:29.598 INFO:teuthology.orchestra.run.smithi012.stdout: "trans_id": "tx0000023f35da06d42986b-00651f111f-107b-default", 2023-10-05T19:40:29.598 INFO:teuthology.orchestra.run.smithi012.stdout: "authentication_type": "Local", 2023-10-05T19:40:29.598 INFO:teuthology.orchestra.run.smithi012.stdout: "temp_url": false 2023-10-05T19:40:29.599 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:29.599 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:29.599 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "", 2023-10-05T19:40:29.599 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:40:23.095118Z", 2023-10-05T19:40:29.599 INFO:teuthology.orchestra.run.smithi012.stdout: "time_local": "2023-10-05T19:40:23.095118+0000", 2023-10-05T19:40:29.599 INFO:teuthology.orchestra.run.smithi012.stdout: "remote_addr": "172.21.0.51", 2023-10-05T19:40:29.599 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:40:29.599 INFO:teuthology.orchestra.run.smithi012.stdout: "operation": "list_buckets", 2023-10-05T19:40:29.599 INFO:teuthology.orchestra.run.smithi012.stdout: "uri": "GET / HTTP/1.1", 2023-10-05T19:40:29.599 INFO:teuthology.orchestra.run.smithi012.stdout: "http_status": "200", 2023-10-05T19:40:29.599 INFO:teuthology.orchestra.run.smithi012.stdout: "error_code": "", 2023-10-05T19:40:29.599 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 322, 2023-10-05T19:40:29.600 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:40:29.600 INFO:teuthology.orchestra.run.smithi012.stdout: "object_size": 0, 2023-10-05T19:40:29.600 INFO:teuthology.orchestra.run.smithi012.stdout: "total_time": 0, 2023-10-05T19:40:29.600 INFO:teuthology.orchestra.run.smithi012.stdout: "user_agent": "Boto/2.49.0 Python/3.8.10 Linux/5.4.0-163-generic", 2023-10-05T19:40:29.600 INFO:teuthology.orchestra.run.smithi012.stdout: "referrer": "", 2023-10-05T19:40:29.600 INFO:teuthology.orchestra.run.smithi012.stdout: "trans_id": "tx000004acddcaf7770ec8e-00651f1127-107b-default", 2023-10-05T19:40:29.600 INFO:teuthology.orchestra.run.smithi012.stdout: "authentication_type": "Local", 2023-10-05T19:40:29.600 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key_id": "9te6NH5mcdcq0Tc5i8i1", 2023-10-05T19:40:29.600 INFO:teuthology.orchestra.run.smithi012.stdout: "temp_url": false 2023-10-05T19:40:29.600 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:29.600 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:29.600 INFO:teuthology.orchestra.run.smithi012.stdout: "log_sum": { 2023-10-05T19:40:29.600 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 553, 2023-10-05T19:40:29.601 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:40:29.601 INFO:teuthology.orchestra.run.smithi012.stdout: "total_time": 0, 2023-10-05T19:40:29.601 INFO:teuthology.orchestra.run.smithi012.stdout: "total_entries": 2 2023-10-05T19:40:29.601 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:29.601 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:40:29.601 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:40:29.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.586+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:102 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94445070014000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4dd5290 con 0x55e5b4d8b560 2023-10-05T19:40:29.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.590+0000 7f5e951d3700 1 -- 172.21.15.12:0/910014983 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(102 notify.0 [watch unwatch cookie 94445070014000] v28'161 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e8801cc20 con 0x55e5b4d8b560 2023-10-05T19:40:29.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.590+0000 7f5e9769ee40 20 remove_watcher() i=0 2023-10-05T19:40:29.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.590+0000 7f5e9769ee40 2 removed watcher, disabling cache 2023-10-05T19:40:29.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.590+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94445070020464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d95630 con 0x55e5b4d58740 2023-10-05T19:40:29.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.590+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(103 notify.1 [watch unwatch cookie 94445070020464] v28'161 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e90014040 con 0x55e5b4d58740 2023-10-05T19:40:29.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.590+0000 7f5e9769ee40 20 remove_watcher() i=1 2023-10-05T19:40:29.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.590+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:104 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94445070023488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d95630 con 0x55e5b4d8fee0 2023-10-05T19:40:29.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.594+0000 7f5e949d2700 1 -- 172.21.15.12:0/910014983 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(104 notify.2 [watch unwatch cookie 94445070023488] v28'161 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e8003b070 con 0x55e5b4d8fee0 2023-10-05T19:40:29.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.594+0000 7f5e9769ee40 20 remove_watcher() i=2 2023-10-05T19:40:29.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.594+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:105 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94445070021568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d97b40 con 0x55e5b4d8b560 2023-10-05T19:40:29.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.598+0000 7f5e951d3700 1 -- 172.21.15.12:0/910014983 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(105 notify.3 [watch unwatch cookie 94445070021568] v28'161 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e8801cc20 con 0x55e5b4d8b560 2023-10-05T19:40:29.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.598+0000 7f5e9769ee40 20 remove_watcher() i=3 2023-10-05T19:40:29.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.598+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:106 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94445070028416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d97460 con 0x55e5b4d58740 2023-10-05T19:40:29.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.602+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(106 notify.4 [watch unwatch cookie 94445070028416] v28'161 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e90014040 con 0x55e5b4d58740 2023-10-05T19:40:29.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.602+0000 7f5e9769ee40 20 remove_watcher() i=4 2023-10-05T19:40:29.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.602+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:107 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94445070030080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d97460 con 0x7f5e88007030 2023-10-05T19:40:29.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.602+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(107 notify.5 [watch unwatch cookie 94445070030080] v28'161 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e90014040 con 0x7f5e88007030 2023-10-05T19:40:29.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.602+0000 7f5e9769ee40 20 remove_watcher() i=5 2023-10-05T19:40:29.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.602+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:108 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94445070033952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d97460 con 0x7f5e88007030 2023-10-05T19:40:29.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.606+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(108 notify.6 [watch unwatch cookie 94445070033952] v28'161 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e90014040 con 0x7f5e88007030 2023-10-05T19:40:29.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.606+0000 7f5e9769ee40 20 remove_watcher() i=6 2023-10-05T19:40:29.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.606+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:109 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94445070031184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4d97460 con 0x7f5e88007030 2023-10-05T19:40:29.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.610+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(109 notify.7 [watch unwatch cookie 94445070031184] v28'161 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e90014040 con 0x7f5e88007030 2023-10-05T19:40:29.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.610+0000 7f5e9769ee40 20 remove_watcher() i=7 2023-10-05T19:40:29.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.610+0000 7f5e65feb700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:29.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.610+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:110 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94445070037776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55e5b4dd6220 con 0x55e5b4d58740 2023-10-05T19:40:29.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.614+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(110 data_loggenerations_metadata [watch unwatch cookie 94445070037776] v28'154 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f5e90014040 con 0x55e5b4d58740 2023-10-05T19:40:29.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.614+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f5e88007030 msgr2=0x7f5e880113b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.614+0000 7f5e9769ee40 1 --2- 172.21.15.12:0/910014983 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f5e88007030 0x7f5e880113b0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.614+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55e5b4d58740 msgr2=0x55e5b4d89370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.614+0000 7f5e9769ee40 1 --2- 172.21.15.12:0/910014983 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55e5b4d58740 0x55e5b4d89370 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.614+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55e5b4d8b560 msgr2=0x55e5b4d8d940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.614+0000 7f5e9769ee40 1 --2- 172.21.15.12:0/910014983 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55e5b4d8b560 0x55e5b4d8d940 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.614+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55e5b4d8fee0 msgr2=0x55e5b4d922c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.614+0000 7f5e9769ee40 1 --2- 172.21.15.12:0/910014983 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55e5b4d8fee0 0x55e5b4d922c0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.614+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5e7c030ca0 msgr2=0x7f5e7c033150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.614+0000 7f5e9769ee40 1 --2- 172.21.15.12:0/910014983 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5e7c030ca0 0x7f5e7c033150 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5e800044f0 tx=0x7f5e80006690 comp rx=0 tx=0).stop 2023-10-05T19:40:29.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.614+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5b4d57e30 msgr2=0x55e5b4d7d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.614+0000 7f5e9769ee40 1 --2- 172.21.15.12:0/910014983 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5b4d57e30 0x55e5b4d7d9a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5e8801f040 tx=0x7f5e8801c000 comp rx=0 tx=0).stop 2023-10-05T19:40:29.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.614+0000 7f5e959d4700 1 -- 172.21.15.12:0/910014983 reap_dead start 2023-10-05T19:40:29.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.614+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 shutdown_connections 2023-10-05T19:40:29.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.614+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 >> 172.21.15.12:0/910014983 conn(0x55e5b4d53850 msgr2=0x55e5b4d71320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:29.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.614+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 shutdown_connections 2023-10-05T19:40:29.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.614+0000 7f5e9769ee40 1 -- 172.21.15.12:0/910014983 wait complete. 2023-10-05T19:40:29.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.614+0000 7f5e9769ee40 1 -- 172.21.15.12:0/1484634197 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55e5b4e15c60 msgr2=0x55e5b4e18040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.614+0000 7f5e9769ee40 1 --2- 172.21.15.12:0/1484634197 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55e5b4e15c60 0x55e5b4e18040 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.614+0000 7f5e9769ee40 1 -- 172.21.15.12:0/1484634197 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5dd4030c50 msgr2=0x7f5dd4033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.614+0000 7f5e9769ee40 1 --2- 172.21.15.12:0/1484634197 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5dd4030c50 0x7f5dd4033100 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x55e5b4d9aae0 tx=0x7f5e9000ab20 comp rx=0 tx=0).stop 2023-10-05T19:40:29.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.614+0000 7f5e9769ee40 1 -- 172.21.15.12:0/1484634197 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5b4da8a10 msgr2=0x55e5b4dae590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.614+0000 7f5e9769ee40 1 --2- 172.21.15.12:0/1484634197 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5b4da8a10 0x55e5b4dae590 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x55e5b4da2af0 tx=0x7f5e8003b000 comp rx=0 tx=0).stop 2023-10-05T19:40:29.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.614+0000 7f5e9769ee40 1 -- 172.21.15.12:0/1484634197 shutdown_connections 2023-10-05T19:40:29.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.614+0000 7f5e9769ee40 1 --2- 172.21.15.12:0/1484634197 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55e5b4e15c60 0x55e5b4e18040 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.614+0000 7f5e9769ee40 1 --2- 172.21.15.12:0/1484634197 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5dd4030c50 0x7f5dd4033100 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.614+0000 7f5e9769ee40 1 --2- 172.21.15.12:0/1484634197 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5b4da8a10 0x55e5b4dae590 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.614+0000 7f5e9769ee40 1 -- 172.21.15.12:0/1484634197 >> 172.21.15.12:0/1484634197 conn(0x55e5b4da3dd0 msgr2=0x55e5b4da1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:29.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.614+0000 7f5e9769ee40 1 -- 172.21.15.12:0/1484634197 shutdown_connections 2023-10-05T19:40:29.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.614+0000 7f5e9769ee40 1 -- 172.21.15.12:0/1484634197 wait complete. 2023-10-05T19:40:29.630 DEBUG:tasks.util.rgw: json result: {'bucket_id': '', 'bucket_owner': '', 'bucket': '', 'log_entries': [{'bucket': '', 'time': '2023-10-05T19:40:15.811286Z', 'time_local': '2023-10-05T19:40:15.811286+0000', 'remote_addr': '172.21.15.12', 'user': 'anonymous', 'operation': 'list_buckets', 'uri': 'GET / HTTP/1.1', 'http_status': '200', 'error_code': '', 'bytes_sent': 231, 'bytes_received': 0, 'object_size': 0, 'total_time': 0, 'user_agent': 'curl/7.68.0', 'referrer': '', 'trans_id': 'tx0000023f35da06d42986b-00651f111f-107b-default', 'authentication_type': 'Local', 'temp_url': False}, {'bucket': '', 'time': '2023-10-05T19:40:23.095118Z', 'time_local': '2023-10-05T19:40:23.095118+0000', 'remote_addr': '172.21.0.51', 'user': 'foo', 'operation': 'list_buckets', 'uri': 'GET / HTTP/1.1', 'http_status': '200', 'error_code': '', 'bytes_sent': 322, 'bytes_received': 0, 'object_size': 0, 'total_time': 0, 'user_agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-163-generic', 'referrer': '', 'trans_id': 'tx000004acddcaf7770ec8e-00651f1127-107b-default', 'authentication_type': 'Local', 'access_key_id': '9te6NH5mcdcq0Tc5i8i1', 'temp_url': False}], 'log_sum': {'bytes_sent': 553, 'bytes_received': 0, 'total_time': 0, 'total_entries': 2}} 2023-10-05T19:40:29.630 INFO:tasks.util.rgw:rgwadmin: client.0 : ['log', 'rm', '--object', '2023-10-05-19--'] 2023-10-05T19:40:29.630 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'log', 'rm', '--object', '2023-10-05-19--'] 2023-10-05T19:40:29.631 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph log rm --object 2023-10-05-19-- 2023-10-05T19:40:29.712 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:29.712 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:29.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.702+0000 7fc9708b6e40 1 Processor -- start 2023-10-05T19:40:29.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.702+0000 7fc9708b6e40 1 -- start start 2023-10-05T19:40:29.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.702+0000 7fc9708b6e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x559038431c20 0x559038430200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.702+0000 7fc9708b6e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x559038408bd0 con 0x559038430740 2023-10-05T19:40:29.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.702+0000 7fc9708b6e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x5590383c5ee0 con 0x559038431c20 2023-10-05T19:40:29.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.706+0000 7fc96e3eb700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x559038431c20 0x559038430200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.706+0000 7fc96dbea700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x559038430740 0x5590384c3f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:35698/0 (socket says 172.21.15.12:35698) 2023-10-05T19:40:29.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.706+0000 7fc96dbea700 1 -- 172.21.15.12:0/3510036222 learned_addr learned my addr 172.21.15.12:0/3510036222 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:29.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.706+0000 7fc96e3eb700 1 -- 172.21.15.12:0/3510036222 >> v1:172.21.15.8:6789/0 conn(0x559038430740 legacy=0x5590384c3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.706+0000 7fc96e3eb700 1 -- 172.21.15.12:0/3510036222 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590384320b0 con 0x559038431c20 2023-10-05T19:40:29.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.706+0000 7fc96e3eb700 1 --2- 172.21.15.12:0/3510036222 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559038431c20 0x559038430200 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc96800a5d0 tx=0x7fc968005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=791c3c38f4fc57f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.706+0000 7fc96d3e9700 1 -- 172.21.15.12:0/3510036222 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc9680086a0 con 0x559038431c20 2023-10-05T19:40:29.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.706+0000 7fc96d3e9700 1 -- 172.21.15.12:0/3510036222 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc968008800 con 0x559038431c20 2023-10-05T19:40:29.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.706+0000 7fc96d3e9700 1 -- 172.21.15.12:0/3510036222 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc968008ad0 con 0x559038431c20 2023-10-05T19:40:29.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.706+0000 7fc9708b6e40 1 -- 172.21.15.12:0/3510036222 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559038431c20 msgr2=0x559038430200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.706+0000 7fc9708b6e40 1 --2- 172.21.15.12:0/3510036222 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559038431c20 0x559038430200 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc96800a5d0 tx=0x7fc968005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:29.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.706+0000 7fc9708b6e40 1 -- 172.21.15.12:0/3510036222 shutdown_connections 2023-10-05T19:40:29.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.706+0000 7fc9708b6e40 1 --2- 172.21.15.12:0/3510036222 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559038431c20 0x559038430200 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.706+0000 7fc9708b6e40 1 -- 172.21.15.12:0/3510036222 >> 172.21.15.12:0/3510036222 conn(0x5590384b7850 msgr2=0x5590384b9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:29.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.706+0000 7fc9708b6e40 1 -- 172.21.15.12:0/3510036222 shutdown_connections 2023-10-05T19:40:29.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.706+0000 7fc9708b6e40 1 -- 172.21.15.12:0/3510036222 wait complete. 2023-10-05T19:40:29.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.706+0000 7fc9708b6e40 1 Processor -- start 2023-10-05T19:40:29.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.706+0000 7fc9708b6e40 1 -- start start 2023-10-05T19:40:29.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.706+0000 7fc9708b6e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559038430740 0x5590384dd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.706+0000 7fc9708b6e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559038389e30 con 0x559038430740 2023-10-05T19:40:29.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.710+0000 7fc96e3eb700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559038430740 0x5590384dd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.710+0000 7fc96e3eb700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559038430740 0x5590384dd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58642/0 (socket says 172.21.15.12:58642) 2023-10-05T19:40:29.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.710+0000 7fc96e3eb700 1 -- 172.21.15.12:0/373084945 learned_addr learned my addr 172.21.15.12:0/373084945 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:29.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.710+0000 7fc96e3eb700 1 -- 172.21.15.12:0/373084945 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc968005cf0 con 0x559038430740 2023-10-05T19:40:29.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.710+0000 7fc96e3eb700 1 --2- 172.21.15.12:0/373084945 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559038430740 0x5590384dd050 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc968005a90 tx=0x7fc968015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f4362fc2522449f server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.710+0000 7fc967fff700 1 -- 172.21.15.12:0/373084945 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc968008a40 con 0x559038430740 2023-10-05T19:40:29.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.710+0000 7fc967fff700 1 -- 172.21.15.12:0/373084945 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc968008bc0 con 0x559038430740 2023-10-05T19:40:29.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.710+0000 7fc967fff700 1 -- 172.21.15.12:0/373084945 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc968012450 con 0x559038430740 2023-10-05T19:40:29.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.710+0000 7fc9708b6e40 1 -- 172.21.15.12:0/373084945 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559038430740 msgr2=0x5590384dd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.710+0000 7fc9708b6e40 1 --2- 172.21.15.12:0/373084945 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559038430740 0x5590384dd050 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc968005a90 tx=0x7fc968015000 comp rx=0 tx=0).stop 2023-10-05T19:40:29.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.710+0000 7fc9708b6e40 1 -- 172.21.15.12:0/373084945 shutdown_connections 2023-10-05T19:40:29.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.710+0000 7fc9708b6e40 1 --2- 172.21.15.12:0/373084945 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559038430740 0x5590384dd050 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.710+0000 7fc9708b6e40 1 -- 172.21.15.12:0/373084945 >> 172.21.15.12:0/373084945 conn(0x5590384b7850 msgr2=0x5590384d8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:29.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.710+0000 7fc9708b6e40 1 -- 172.21.15.12:0/373084945 shutdown_connections 2023-10-05T19:40:29.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.710+0000 7fc9708b6e40 1 -- 172.21.15.12:0/373084945 wait complete. 2023-10-05T19:40:29.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.710+0000 7fc9708b6e40 1 Processor -- start 2023-10-05T19:40:29.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.710+0000 7fc9708b6e40 1 -- start start 2023-10-05T19:40:29.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.710+0000 7fc9708b6e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559038430740 0x5590384e3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.710+0000 7fc9708b6e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55903835ced0 con 0x559038430740 2023-10-05T19:40:29.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.710+0000 7fc96e3eb700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559038430740 0x5590384e3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.710+0000 7fc96e3eb700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559038430740 0x5590384e3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58656/0 (socket says 172.21.15.12:58656) 2023-10-05T19:40:29.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.710+0000 7fc96e3eb700 1 -- 172.21.15.12:0/1061644500 learned_addr learned my addr 172.21.15.12:0/1061644500 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:29.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.710+0000 7fc96e3eb700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590384320b0 con 0x559038430740 2023-10-05T19:40:29.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.710+0000 7fc96e3eb700 1 --2- 172.21.15.12:0/1061644500 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559038430740 0x5590384e3c60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc96800c010 tx=0x7fc96801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.714+0000 7fc9667fc700 1 -- 172.21.15.12:0/1061644500 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc968008e50 con 0x559038430740 2023-10-05T19:40:29.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.714+0000 7fc9667fc700 1 -- 172.21.15.12:0/1061644500 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc968025660 con 0x559038430740 2023-10-05T19:40:29.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.714+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5590384e41a0 con 0x559038430740 2023-10-05T19:40:29.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.714+0000 7fc9667fc700 1 -- 172.21.15.12:0/1061644500 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc968025950 con 0x559038430740 2023-10-05T19:40:29.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.714+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590384e4640 con 0x559038430740 2023-10-05T19:40:29.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.714+0000 7fc9708b6e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.714+0000 7fc9667fc700 1 -- 172.21.15.12:0/1061644500 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fc968019070 con 0x559038430740 2023-10-05T19:40:29.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.714+0000 7fc9667fc700 1 --2- 172.21.15.12:0/1061644500 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fc950030c50 0x7fc950033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.714+0000 7fc9667fc700 1 -- 172.21.15.12:0/1061644500 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7fc968046610 con 0x559038430740 2023-10-05T19:40:29.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.714+0000 7fc96dbea700 1 --2- 172.21.15.12:0/1061644500 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fc950030c50 0x7fc950033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.714+0000 7fc9708b6e40 1 --2- 172.21.15.12:0/1061644500 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x559038431c20 0x5590384ef780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.714+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384efdf0 con 0x559038431c20 2023-10-05T19:40:29.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.714+0000 7fc96ebec700 1 --2- 172.21.15.12:0/1061644500 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x559038431c20 0x5590384ef780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.714+0000 7fc96dbea700 1 --2- 172.21.15.12:0/1061644500 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fc950030c50 0x7fc950033100 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc958001e90 tx=0x7fc9580013b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.714+0000 7fc96ebec700 1 --2- 172.21.15.12:0/1061644500 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x559038431c20 0x5590384ef780 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.718+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fc96000a5a0 con 0x559038431c20 2023-10-05T19:40:29.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.718+0000 7fc9708b6e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:29.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.718+0000 7fc9708b6e40 20 realm 2023-10-05T19:40:29.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.718+0000 7fc9708b6e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.718+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384ecde0 con 0x559038431c20 2023-10-05T19:40:29.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.718+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fc96000a5a0 con 0x559038431c20 2023-10-05T19:40:29.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.718+0000 7fc9708b6e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:29.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.718+0000 7fc9708b6e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:29.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.718+0000 7fc9708b6e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.718+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f08d0 con 0x559038431c20 2023-10-05T19:40:29.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.718+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fc96000a5a0 con 0x559038431c20 2023-10-05T19:40:29.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.718+0000 7fc9708b6e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:29.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.718+0000 7fc9708b6e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.718+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f04e0 con 0x559038431c20 2023-10-05T19:40:29.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.718+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fc96000a5a0 con 0x559038431c20 2023-10-05T19:40:29.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.718+0000 7fc9708b6e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:29.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.718+0000 7fc9708b6e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.718+0000 7fc9708b6e40 1 --2- 172.21.15.12:0/1061644500 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5590384f1c70 0x5590384f4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.718+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f4590 con 0x5590384f1c70 2023-10-05T19:40:29.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.718+0000 7fc96e3eb700 1 --2- 172.21.15.12:0/1061644500 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5590384f1c70 0x5590384f4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.718+0000 7fc96e3eb700 1 --2- 172.21.15.12:0/1061644500 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5590384f1c70 0x5590384f4050 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.718+0000 7fc96e3eb700 1 -- 172.21.15.12:0/1061644500 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fc96803d040 con 0x5590384f1c70 2023-10-05T19:40:29.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.718+0000 7fc9708b6e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:29.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.718+0000 7fc9708b6e40 20 searching for the correct realm 2023-10-05T19:40:29.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.722+0000 7fc9708b6e40 1 --2- 172.21.15.12:0/1061644500 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5590384f6380 0x5590384f8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.722+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f8e40 con 0x5590384f6380 2023-10-05T19:40:29.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.722+0000 7fc96dbea700 1 --2- 172.21.15.12:0/1061644500 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5590384f6380 0x5590384f8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.722+0000 7fc96dbea700 1 --2- 172.21.15.12:0/1061644500 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5590384f6380 0x5590384f8790 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.722+0000 7fc96dbea700 1 -- 172.21.15.12:0/1061644500 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fc958006070 con 0x5590384f6380 2023-10-05T19:40:29.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.722+0000 7fc96dbea700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f8e40 con 0x5590384f1c70 2023-10-05T19:40:29.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.722+0000 7fc96e3eb700 1 -- 172.21.15.12:0/1061644500 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc96803d040 con 0x5590384f1c70 2023-10-05T19:40:29.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.722+0000 7fc96e3eb700 1 --2- 172.21.15.12:0/1061644500 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fc968007030 0x7fc9680069e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.722+0000 7fc96e3eb700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f8e40 con 0x7fc968007030 2023-10-05T19:40:29.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.722+0000 7fc96ebec700 1 --2- 172.21.15.12:0/1061644500 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fc968007030 0x7fc9680069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.726+0000 7fc96ebec700 1 --2- 172.21.15.12:0/1061644500 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fc968007030 0x7fc9680069e0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.726+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc96000a5a0 con 0x7fc968007030 2023-10-05T19:40:29.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.726+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f8e40 con 0x559038431c20 2023-10-05T19:40:29.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.726+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc96000a5a0 con 0x559038431c20 2023-10-05T19:40:29.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.726+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f8e40 con 0x559038431c20 2023-10-05T19:40:29.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.726+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc96000a5a0 con 0x559038431c20 2023-10-05T19:40:29.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.726+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f8e40 con 0x7fc968007030 2023-10-05T19:40:29.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.726+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc96000a5a0 con 0x7fc968007030 2023-10-05T19:40:29.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.726+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f8e40 con 0x7fc968007030 2023-10-05T19:40:29.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.730+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc96000a5a0 con 0x7fc968007030 2023-10-05T19:40:29.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.730+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f8e40 con 0x5590384f1c70 2023-10-05T19:40:29.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.730+0000 7fc96e3eb700 1 -- 172.21.15.12:0/1061644500 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc96803d040 con 0x5590384f1c70 2023-10-05T19:40:29.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.730+0000 7fc96e3eb700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f8e40 con 0x5590384f6380 2023-10-05T19:40:29.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.730+0000 7fc96dbea700 1 -- 172.21.15.12:0/1061644500 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc958006070 con 0x5590384f6380 2023-10-05T19:40:29.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.730+0000 7fc96dbea700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f8e40 con 0x559038431c20 2023-10-05T19:40:29.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.730+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc96000a5a0 con 0x559038431c20 2023-10-05T19:40:29.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.730+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f8e40 con 0x7fc968007030 2023-10-05T19:40:29.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.730+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc96000a5a0 con 0x7fc968007030 2023-10-05T19:40:29.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.730+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f8e40 con 0x559038431c20 2023-10-05T19:40:29.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.730+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc96000a5a0 con 0x559038431c20 2023-10-05T19:40:29.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.730+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f8e40 con 0x5590384f1c70 2023-10-05T19:40:29.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.734+0000 7fc96e3eb700 1 -- 172.21.15.12:0/1061644500 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc96803d040 con 0x5590384f1c70 2023-10-05T19:40:29.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.734+0000 7fc96e3eb700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f8e40 con 0x5590384f6380 2023-10-05T19:40:29.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.734+0000 7fc96dbea700 1 -- 172.21.15.12:0/1061644500 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc958006070 con 0x5590384f6380 2023-10-05T19:40:29.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.734+0000 7fc96dbea700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f8e40 con 0x5590384f6380 2023-10-05T19:40:29.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.734+0000 7fc96dbea700 1 -- 172.21.15.12:0/1061644500 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc958006070 con 0x5590384f6380 2023-10-05T19:40:29.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.734+0000 7fc96dbea700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f8e40 con 0x7fc968007030 2023-10-05T19:40:29.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.734+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc96000a5a0 con 0x7fc968007030 2023-10-05T19:40:29.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.734+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f8e40 con 0x5590384f6380 2023-10-05T19:40:29.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.734+0000 7fc96dbea700 1 -- 172.21.15.12:0/1061644500 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc958006070 con 0x5590384f6380 2023-10-05T19:40:29.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.734+0000 7fc96dbea700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f8e40 con 0x5590384f6380 2023-10-05T19:40:29.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.734+0000 7fc96dbea700 1 -- 172.21.15.12:0/1061644500 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc958006070 con 0x5590384f6380 2023-10-05T19:40:29.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.734+0000 7fc96dbea700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f8e40 con 0x559038431c20 2023-10-05T19:40:29.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.738+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc96000a5a0 con 0x559038431c20 2023-10-05T19:40:29.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.738+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f8e40 con 0x559038431c20 2023-10-05T19:40:29.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.738+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc96000a5a0 con 0x559038431c20 2023-10-05T19:40:29.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.738+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f8e40 con 0x7fc968007030 2023-10-05T19:40:29.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.738+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc96000a5a0 con 0x7fc968007030 2023-10-05T19:40:29.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.738+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f8e40 con 0x5590384f6380 2023-10-05T19:40:29.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.738+0000 7fc96dbea700 1 -- 172.21.15.12:0/1061644500 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc958006070 con 0x5590384f6380 2023-10-05T19:40:29.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.738+0000 7fc96dbea700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f8e40 con 0x5590384f1c70 2023-10-05T19:40:29.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.738+0000 7fc96e3eb700 1 -- 172.21.15.12:0/1061644500 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc96803d040 con 0x5590384f1c70 2023-10-05T19:40:29.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.738+0000 7fc96e3eb700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f8e40 con 0x559038431c20 2023-10-05T19:40:29.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.742+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc96000a5a0 con 0x559038431c20 2023-10-05T19:40:29.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.742+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f8e40 con 0x7fc968007030 2023-10-05T19:40:29.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.742+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc96000a5a0 con 0x7fc968007030 2023-10-05T19:40:29.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.742+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f8e40 con 0x5590384f6380 2023-10-05T19:40:29.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.742+0000 7fc96dbea700 1 -- 172.21.15.12:0/1061644500 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc958006070 con 0x5590384f6380 2023-10-05T19:40:29.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.742+0000 7fc96dbea700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f8e40 con 0x7fc968007030 2023-10-05T19:40:29.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.742+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc96000a5a0 con 0x7fc968007030 2023-10-05T19:40:29.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.742+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f8e40 con 0x5590384f1c70 2023-10-05T19:40:29.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.746+0000 7fc96e3eb700 1 -- 172.21.15.12:0/1061644500 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc96803d040 con 0x5590384f1c70 2023-10-05T19:40:29.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.746+0000 7fc96e3eb700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f8e40 con 0x5590384f6380 2023-10-05T19:40:29.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.746+0000 7fc96dbea700 1 -- 172.21.15.12:0/1061644500 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc958006070 con 0x5590384f6380 2023-10-05T19:40:29.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.746+0000 7fc96dbea700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f8e40 con 0x7fc968007030 2023-10-05T19:40:29.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.746+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc96000a5a0 con 0x7fc968007030 2023-10-05T19:40:29.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.746+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f8e40 con 0x559038431c20 2023-10-05T19:40:29.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.746+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fc96000a5a0 con 0x559038431c20 2023-10-05T19:40:29.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.746+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f8e40 con 0x5590384f1c70 2023-10-05T19:40:29.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.750+0000 7fc96e3eb700 1 -- 172.21.15.12:0/1061644500 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fc96803d040 con 0x5590384f1c70 2023-10-05T19:40:29.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.750+0000 7fc9708b6e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:29.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.750+0000 7fc9708b6e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:29.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.750+0000 7fc9708b6e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:29.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.750+0000 7fc9708b6e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:29.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.750+0000 7fc9708b6e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.750+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f50d0 con 0x559038431c20 2023-10-05T19:40:29.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.750+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fc96000a5a0 con 0x559038431c20 2023-10-05T19:40:29.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.750+0000 7fc9708b6e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:29.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.750+0000 7fc9708b6e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.750+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f4d00 con 0x559038431c20 2023-10-05T19:40:29.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.750+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fc96000a5a0 con 0x559038431c20 2023-10-05T19:40:29.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.750+0000 7fc9708b6e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:29.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.750+0000 7fc9708b6e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.750+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f92b0 con 0x5590384f6380 2023-10-05T19:40:29.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.750+0000 7fc96dbea700 1 -- 172.21.15.12:0/1061644500 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fc958006070 con 0x5590384f6380 2023-10-05T19:40:29.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.750+0000 7fc9708b6e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:29.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.750+0000 7fc9708b6e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.750+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f9660 con 0x5590384f6380 2023-10-05T19:40:29.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.750+0000 7fc96dbea700 1 -- 172.21.15.12:0/1061644500 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7fc958006070 con 0x5590384f6380 2023-10-05T19:40:29.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.750+0000 7fc9708b6e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:29.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.750+0000 7fc9708b6e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.750+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f9a10 con 0x559038431c20 2023-10-05T19:40:29.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.754+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fc96000a5a0 con 0x559038431c20 2023-10-05T19:40:29.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.754+0000 7fc9708b6e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:29.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.754+0000 7fc9708b6e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.754+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f55f0 con 0x559038431c20 2023-10-05T19:40:29.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.754+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fc96000a5a0 con 0x559038431c20 2023-10-05T19:40:29.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.754+0000 7fc9708b6e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:29.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.754+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f55f0 con 0x5590384f6380 2023-10-05T19:40:29.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.754+0000 7fc96dbea700 1 -- 172.21.15.12:0/1061644500 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fc958006070 con 0x5590384f6380 2023-10-05T19:40:29.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.754+0000 7fc96dbea700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f55f0 con 0x5590384f1c70 2023-10-05T19:40:29.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.758+0000 7fc96e3eb700 1 -- 172.21.15.12:0/1061644500 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc96803d040 con 0x5590384f1c70 2023-10-05T19:40:29.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.758+0000 7fc96e3eb700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f55f0 con 0x7fc968007030 2023-10-05T19:40:29.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.758+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc96000a5a0 con 0x7fc968007030 2023-10-05T19:40:29.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.758+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f9a10 con 0x559038431c20 2023-10-05T19:40:29.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.758+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc96000a5a0 con 0x559038431c20 2023-10-05T19:40:29.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.758+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f9a10 con 0x559038431c20 2023-10-05T19:40:29.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.758+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc96000a5a0 con 0x559038431c20 2023-10-05T19:40:29.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.758+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f9a10 con 0x7fc968007030 2023-10-05T19:40:29.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.762+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc96000a5a0 con 0x7fc968007030 2023-10-05T19:40:29.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.762+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f9a10 con 0x7fc968007030 2023-10-05T19:40:29.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.762+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc96000a5a0 con 0x7fc968007030 2023-10-05T19:40:29.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.762+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f9a10 con 0x5590384f1c70 2023-10-05T19:40:29.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.762+0000 7fc96e3eb700 1 -- 172.21.15.12:0/1061644500 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc96803d040 con 0x5590384f1c70 2023-10-05T19:40:29.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.762+0000 7fc96e3eb700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f9a10 con 0x5590384f6380 2023-10-05T19:40:29.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.762+0000 7fc96dbea700 1 -- 172.21.15.12:0/1061644500 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc958006070 con 0x5590384f6380 2023-10-05T19:40:29.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.762+0000 7fc96dbea700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f9a10 con 0x559038431c20 2023-10-05T19:40:29.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.762+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc96000a5a0 con 0x559038431c20 2023-10-05T19:40:29.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.762+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f9a10 con 0x7fc968007030 2023-10-05T19:40:29.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.766+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc96000a5a0 con 0x7fc968007030 2023-10-05T19:40:29.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.766+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f9a10 con 0x559038431c20 2023-10-05T19:40:29.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.766+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc96000a5a0 con 0x559038431c20 2023-10-05T19:40:29.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.766+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f9a10 con 0x5590384f1c70 2023-10-05T19:40:29.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.766+0000 7fc96e3eb700 1 -- 172.21.15.12:0/1061644500 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc96803d040 con 0x5590384f1c70 2023-10-05T19:40:29.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.766+0000 7fc96e3eb700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f9a10 con 0x5590384f6380 2023-10-05T19:40:29.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.766+0000 7fc96dbea700 1 -- 172.21.15.12:0/1061644500 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc958006070 con 0x5590384f6380 2023-10-05T19:40:29.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.766+0000 7fc96dbea700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f9a10 con 0x5590384f6380 2023-10-05T19:40:29.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.766+0000 7fc96dbea700 1 -- 172.21.15.12:0/1061644500 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc958006070 con 0x5590384f6380 2023-10-05T19:40:29.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.766+0000 7fc96dbea700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f9a10 con 0x7fc968007030 2023-10-05T19:40:29.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.770+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc96000a5a0 con 0x7fc968007030 2023-10-05T19:40:29.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.770+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f9a10 con 0x5590384f6380 2023-10-05T19:40:29.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.770+0000 7fc96dbea700 1 -- 172.21.15.12:0/1061644500 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc958006070 con 0x5590384f6380 2023-10-05T19:40:29.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.770+0000 7fc96dbea700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f9a10 con 0x5590384f6380 2023-10-05T19:40:29.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.770+0000 7fc96dbea700 1 -- 172.21.15.12:0/1061644500 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc958006070 con 0x5590384f6380 2023-10-05T19:40:29.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.770+0000 7fc96dbea700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f9a10 con 0x559038431c20 2023-10-05T19:40:29.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.770+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc96000a5a0 con 0x559038431c20 2023-10-05T19:40:29.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.770+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f9a10 con 0x559038431c20 2023-10-05T19:40:29.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.770+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc96000a5a0 con 0x559038431c20 2023-10-05T19:40:29.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.770+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f9a10 con 0x7fc968007030 2023-10-05T19:40:29.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.774+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc96000a5a0 con 0x7fc968007030 2023-10-05T19:40:29.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.774+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f9a10 con 0x5590384f6380 2023-10-05T19:40:29.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.774+0000 7fc96dbea700 1 -- 172.21.15.12:0/1061644500 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc958006070 con 0x5590384f6380 2023-10-05T19:40:29.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.774+0000 7fc96dbea700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f9a10 con 0x5590384f1c70 2023-10-05T19:40:29.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.774+0000 7fc96e3eb700 1 -- 172.21.15.12:0/1061644500 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc96803d040 con 0x5590384f1c70 2023-10-05T19:40:29.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.774+0000 7fc96e3eb700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f9a10 con 0x559038431c20 2023-10-05T19:40:29.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.774+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc96000a5a0 con 0x559038431c20 2023-10-05T19:40:29.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.774+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f9a10 con 0x7fc968007030 2023-10-05T19:40:29.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.778+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc96000a5a0 con 0x7fc968007030 2023-10-05T19:40:29.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.778+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f9a10 con 0x5590384f6380 2023-10-05T19:40:29.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.778+0000 7fc96dbea700 1 -- 172.21.15.12:0/1061644500 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc958006070 con 0x5590384f6380 2023-10-05T19:40:29.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.778+0000 7fc96dbea700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f9a10 con 0x7fc968007030 2023-10-05T19:40:29.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.778+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc96000a5a0 con 0x7fc968007030 2023-10-05T19:40:29.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.778+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f9a10 con 0x5590384f1c70 2023-10-05T19:40:29.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.778+0000 7fc96e3eb700 1 -- 172.21.15.12:0/1061644500 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc96803d040 con 0x5590384f1c70 2023-10-05T19:40:29.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.778+0000 7fc96e3eb700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f9a10 con 0x5590384f6380 2023-10-05T19:40:29.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.778+0000 7fc96dbea700 1 -- 172.21.15.12:0/1061644500 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc958006070 con 0x5590384f6380 2023-10-05T19:40:29.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.778+0000 7fc96dbea700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f9a10 con 0x7fc968007030 2023-10-05T19:40:29.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.782+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc96000a5a0 con 0x7fc968007030 2023-10-05T19:40:29.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.782+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f9a10 con 0x559038431c20 2023-10-05T19:40:29.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.782+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fc96000a5a0 con 0x559038431c20 2023-10-05T19:40:29.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.782+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f9a10 con 0x5590384f1c70 2023-10-05T19:40:29.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.782+0000 7fc96e3eb700 1 -- 172.21.15.12:0/1061644500 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fc96803d040 con 0x5590384f1c70 2023-10-05T19:40:29.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.782+0000 7fc9708b6e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:29.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.782+0000 7fc9708b6e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:29.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.782+0000 7fc9708b6e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:29.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.782+0000 7fc9708b6e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:29.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.782+0000 7fc9708b6e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.782+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384f55f0 con 0x559038431c20 2023-10-05T19:40:29.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.782+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fc96000a5a0 con 0x559038431c20 2023-10-05T19:40:29.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.782+0000 7fc9708b6e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:29.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.782+0000 7fc9708b6e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.782+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384fa360 con 0x5590384f1c70 2023-10-05T19:40:29.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.786+0000 7fc96e3eb700 1 -- 172.21.15.12:0/1061644500 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fc96803d040 con 0x5590384f1c70 2023-10-05T19:40:29.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.786+0000 7fc9708b6e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:29.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.786+0000 7fc9708b6e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.786+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384fac50 con 0x559038431c20 2023-10-05T19:40:29.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.786+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fc96000a5a0 con 0x559038431c20 2023-10-05T19:40:29.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.786+0000 7fc9708b6e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:29.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.786+0000 7fc9708b6e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.786+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384fac50 con 0x5590384f6380 2023-10-05T19:40:29.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.786+0000 7fc96dbea700 1 -- 172.21.15.12:0/1061644500 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fc958006070 con 0x5590384f6380 2023-10-05T19:40:29.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.786+0000 7fc9708b6e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:29.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.786+0000 7fc9708b6e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.786+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384fb3c0 con 0x7fc968007030 2023-10-05T19:40:29.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.786+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7fc96000a5a0 con 0x7fc968007030 2023-10-05T19:40:29.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.786+0000 7fc9708b6e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:29.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.786+0000 7fc9708b6e40 20 zone default found 2023-10-05T19:40:29.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.786+0000 7fc9708b6e40 4 Realm: () 2023-10-05T19:40:29.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.786+0000 7fc9708b6e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:29.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.786+0000 7fc9708b6e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:29.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.786+0000 7fc9708b6e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:29.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.786+0000 7fc9708b6e40 20 zonegroup default 2023-10-05T19:40:29.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.786+0000 7fc9708b6e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.790+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384fb090 con 0x7fc968007030 2023-10-05T19:40:29.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.790+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fc96000a5a0 con 0x7fc968007030 2023-10-05T19:40:29.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.790+0000 7fc9708b6e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:29.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.790+0000 7fc9708b6e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.790+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384fca10 con 0x5590384f6380 2023-10-05T19:40:29.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.790+0000 7fc96dbea700 1 -- 172.21.15.12:0/1061644500 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fc958006070 con 0x5590384f6380 2023-10-05T19:40:29.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.790+0000 7fc9708b6e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:29.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.790+0000 7fc9708b6e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.790+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384fcdc0 con 0x5590384f6380 2023-10-05T19:40:29.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.790+0000 7fc96dbea700 1 -- 172.21.15.12:0/1061644500 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fc958006070 con 0x5590384f6380 2023-10-05T19:40:29.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.790+0000 7fc9708b6e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:29.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.790+0000 7fc9708b6e40 20 started sync module instance, tier type = 2023-10-05T19:40:29.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.790+0000 7fc9708b6e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:29.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.790+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384fc5c0 con 0x5590384f1c70 2023-10-05T19:40:29.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.794+0000 7fc96e3eb700 1 -- 172.21.15.12:0/1061644500 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'162 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc96803d040 con 0x5590384f1c70 2023-10-05T19:40:29.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.794+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94077908402384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384fd660 con 0x5590384f1c70 2023-10-05T19:40:29.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.794+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384fdf00 con 0x559038431c20 2023-10-05T19:40:29.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.798+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'162 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc96000a5a0 con 0x559038431c20 2023-10-05T19:40:29.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.798+0000 7fc96e3eb700 1 -- 172.21.15.12:0/1061644500 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94077908402384] v28'163 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc96803d040 con 0x5590384f1c70 2023-10-05T19:40:29.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.798+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94077908406640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384fdf00 con 0x559038431c20 2023-10-05T19:40:29.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.798+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384fe910 con 0x5590384f6380 2023-10-05T19:40:29.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.802+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94077908406640] v28'163 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc96000a5a0 con 0x559038431c20 2023-10-05T19:40:29.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.802+0000 7fc96dbea700 1 -- 172.21.15.12:0/1061644500 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'162 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc958006070 con 0x5590384f6380 2023-10-05T19:40:29.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.802+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94077908408848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384fe540 con 0x5590384f6380 2023-10-05T19:40:29.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.802+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384ff6a0 con 0x5590384f1c70 2023-10-05T19:40:29.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.802+0000 7fc96e3eb700 1 -- 172.21.15.12:0/1061644500 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'162 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc96803d040 con 0x5590384f1c70 2023-10-05T19:40:29.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.802+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94077908414896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384ffb70 con 0x5590384f1c70 2023-10-05T19:40:29.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.802+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559038500700 con 0x559038431c20 2023-10-05T19:40:29.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.802+0000 7fc96dbea700 1 -- 172.21.15.12:0/1061644500 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94077908408848] v28'163 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc958006070 con 0x5590384f6380 2023-10-05T19:40:29.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.806+0000 7fc96e3eb700 1 -- 172.21.15.12:0/1061644500 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94077908414896] v28'163 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc96803d040 con 0x5590384f1c70 2023-10-05T19:40:29.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.806+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'162 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc96000a5a0 con 0x559038431c20 2023-10-05T19:40:29.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.806+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94077908413632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590385001d0 con 0x559038431c20 2023-10-05T19:40:29.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.806+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559038500ff0 con 0x7fc968007030 2023-10-05T19:40:29.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.810+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'162 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc96000a5a0 con 0x7fc968007030 2023-10-05T19:40:29.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.810+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94077908420704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590385011a0 con 0x7fc968007030 2023-10-05T19:40:29.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.810+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559038501bb0 con 0x7fc968007030 2023-10-05T19:40:29.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.810+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94077908413632] v28'163 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc96000a5a0 con 0x559038431c20 2023-10-05T19:40:29.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.810+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(96 notify.6 [create] v28'162 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc96000a5a0 con 0x7fc968007030 2023-10-05T19:40:29.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.810+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(95 notify.5 [watch watch cookie 94077908420704] v28'163 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc96000a5a0 con 0x7fc968007030 2023-10-05T19:40:29.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.810+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94077908416000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590385016c0 con 0x7fc968007030 2023-10-05T19:40:29.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.810+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559038501f60 con 0x7fc968007030 2023-10-05T19:40:29.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.814+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(97 notify.6 [watch watch cookie 94077908416000] v28'163 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc96000a5a0 con 0x7fc968007030 2023-10-05T19:40:29.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.814+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(98 notify.7 [create] v28'162 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc96000a5a0 con 0x7fc968007030 2023-10-05T19:40:29.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.814+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94077908424384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559038502000 con 0x7fc968007030 2023-10-05T19:40:29.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.814+0000 7fc9708b6e40 20 add_watcher() i=0 2023-10-05T19:40:29.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.814+0000 7fc9708b6e40 20 add_watcher() i=1 2023-10-05T19:40:29.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.814+0000 7fc9708b6e40 20 add_watcher() i=2 2023-10-05T19:40:29.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.814+0000 7fc9708b6e40 20 add_watcher() i=3 2023-10-05T19:40:29.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.814+0000 7fc9708b6e40 20 add_watcher() i=4 2023-10-05T19:40:29.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.814+0000 7fc9708b6e40 20 add_watcher() i=5 2023-10-05T19:40:29.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.814+0000 7fc9708b6e40 20 add_watcher() i=6 2023-10-05T19:40:29.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.814+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94077908424384] v28'163 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc96000a5a0 con 0x7fc968007030 2023-10-05T19:40:29.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.814+0000 7fc9708b6e40 20 add_watcher() i=7 2023-10-05T19:40:29.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.814+0000 7fc9708b6e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:29.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.814+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590385051a0 con 0x559038431c20 2023-10-05T19:40:29.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.818+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fc96000a5a0 con 0x559038431c20 2023-10-05T19:40:29.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.818+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94077908425856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x559038502720 con 0x559038431c20 2023-10-05T19:40:29.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.818+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94077908425856] v28'155 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fc96000a5a0 con 0x559038431c20 2023-10-05T19:40:29.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.818+0000 7fc93ffdf700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:29.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.818+0000 7fc9708b6e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:29.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.818+0000 7fc9708b6e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:29.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.818+0000 7fc9708b6e40 1 Processor -- start 2023-10-05T19:40:29.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.818+0000 7fc9708b6e40 1 -- start start 2023-10-05T19:40:29.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.818+0000 7fc9708b6e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55903850f980 0x55903850fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.818+0000 7fc9708b6e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559038510290 con 0x55903850f980 2023-10-05T19:40:29.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.822+0000 7fc96dbea700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55903850f980 0x55903850fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.822+0000 7fc96dbea700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55903850f980 0x55903850fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58672/0 (socket says 172.21.15.12:58672) 2023-10-05T19:40:29.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.822+0000 7fc96dbea700 1 -- 172.21.15.12:0/1982195557 learned_addr learned my addr 172.21.15.12:0/1982195557 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:29.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.822+0000 7fc96dbea700 1 -- 172.21.15.12:0/1982195557 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590385103d0 con 0x55903850f980 2023-10-05T19:40:29.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.822+0000 7fc96dbea700 1 --2- 172.21.15.12:0/1982195557 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55903850f980 0x55903850fd50 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc968006740 tx=0x7fc95801e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e88784ee903f747 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.822+0000 7fc93f7de700 1 -- 172.21.15.12:0/1982195557 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc958013040 con 0x55903850f980 2023-10-05T19:40:29.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.822+0000 7fc93f7de700 1 -- 172.21.15.12:0/1982195557 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc958022040 con 0x55903850f980 2023-10-05T19:40:29.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.822+0000 7fc93f7de700 1 -- 172.21.15.12:0/1982195557 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc95800ee90 con 0x55903850f980 2023-10-05T19:40:29.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.822+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1982195557 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55903850f980 msgr2=0x55903850fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.822+0000 7fc9708b6e40 1 --2- 172.21.15.12:0/1982195557 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55903850f980 0x55903850fd50 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc968006740 tx=0x7fc95801e040 comp rx=0 tx=0).stop 2023-10-05T19:40:29.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.822+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1982195557 shutdown_connections 2023-10-05T19:40:29.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.822+0000 7fc9708b6e40 1 --2- 172.21.15.12:0/1982195557 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55903850f980 0x55903850fd50 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.822+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1982195557 >> 172.21.15.12:0/1982195557 conn(0x55903850ad40 msgr2=0x559038508e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:29.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.822+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1982195557 shutdown_connections 2023-10-05T19:40:29.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.822+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1982195557 wait complete. 2023-10-05T19:40:29.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.822+0000 7fc9708b6e40 1 Processor -- start 2023-10-05T19:40:29.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.822+0000 7fc9708b6e40 1 -- start start 2023-10-05T19:40:29.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.822+0000 7fc9708b6e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55903850f980 0x559038515480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.822+0000 7fc9708b6e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559038516480 con 0x55903850f980 2023-10-05T19:40:29.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.826+0000 7fc96dbea700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55903850f980 0x559038515480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.826+0000 7fc96dbea700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55903850f980 0x559038515480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58676/0 (socket says 172.21.15.12:58676) 2023-10-05T19:40:29.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.826+0000 7fc96dbea700 1 -- 172.21.15.12:0/3650937924 learned_addr learned my addr 172.21.15.12:0/3650937924 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:29.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.826+0000 7fc96dbea700 1 -- 172.21.15.12:0/3650937924 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc958034040 con 0x55903850f980 2023-10-05T19:40:29.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.826+0000 7fc96dbea700 1 --2- 172.21.15.12:0/3650937924 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55903850f980 0x559038515480 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fc95800bed0 tx=0x7fc95803d030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.826+0000 7fc93dfdb700 1 -- 172.21.15.12:0/3650937924 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc95802c640 con 0x55903850f980 2023-10-05T19:40:29.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.826+0000 7fc9708b6e40 1 -- 172.21.15.12:0/3650937924 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559038515a80 con 0x55903850f980 2023-10-05T19:40:29.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.826+0000 7fc93dfdb700 1 -- 172.21.15.12:0/3650937924 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc95802c7c0 con 0x55903850f980 2023-10-05T19:40:29.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.826+0000 7fc9708b6e40 1 -- 172.21.15.12:0/3650937924 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559038515ec0 con 0x55903850f980 2023-10-05T19:40:29.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.826+0000 7fc93dfdb700 1 -- 172.21.15.12:0/3650937924 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc95802c930 con 0x55903850f980 2023-10-05T19:40:29.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.826+0000 7fc93dfdb700 1 -- 172.21.15.12:0/3650937924 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fc95804d070 con 0x55903850f980 2023-10-05T19:40:29.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.826+0000 7fc93dfdb700 1 --2- 172.21.15.12:0/3650937924 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fc8b4030bb0 0x7fc8b4033060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.826+0000 7fc93dfdb700 1 -- 172.21.15.12:0/3650937924 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7fc958032150 con 0x55903850f980 2023-10-05T19:40:29.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.826+0000 7fc96ebec700 1 --2- 172.21.15.12:0/3650937924 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fc8b4030bb0 0x7fc8b4033060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.826+0000 7fc9708b6e40 5 note: GC not initialized 2023-10-05T19:40:29.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.830+0000 7fc96ebec700 1 --2- 172.21.15.12:0/3650937924 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fc8b4030bb0 0x7fc8b4033060 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x559038502680 tx=0x7fc96000ae90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.830+0000 7fc8aeffd700 20 reqs_thread_entry: start 2023-10-05T19:40:29.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.830+0000 7fc8acff9700 20 reqs_thread_entry: start 2023-10-05T19:40:29.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.830+0000 7fc9708b6e40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:29.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.830+0000 7fc9708b6e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:29.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.830+0000 7fc8a6ffd700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:41:00 2023 2023-10-05T19:40:29.841 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:29.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.830+0000 7fc8a67fc700 20 reqs_thread_entry: start 2023-10-05T19:40:29.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.830+0000 7fc9708b6e40 1 --2- 172.21.15.12:0/3650937924 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55903857ca50 0x55903857ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.830+0000 7fc9708b6e40 1 -- 172.21.15.12:0/3650937924 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:1 6.8 6:163360d8:::2023-10-05-19--:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e28) v8 -- 0x55903857f370 con 0x55903857ca50 2023-10-05T19:40:29.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.830+0000 7fc96e3eb700 1 --2- 172.21.15.12:0/3650937924 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55903857ca50 0x55903857ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.834+0000 7fc96e3eb700 1 --2- 172.21.15.12:0/3650937924 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55903857ca50 0x55903857ee30 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.834+0000 7fc96e3eb700 1 -- 172.21.15.12:0/3650937924 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(1 2023-10-05-19-- [delete] v28'26 uv26 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fc96803d040 con 0x55903857ca50 2023-10-05T19:40:29.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.834+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:102 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94077908402384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55903853bb90 con 0x5590384f1c70 2023-10-05T19:40:29.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.838+0000 7fc96e3eb700 1 -- 172.21.15.12:0/1061644500 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(102 notify.0 [watch unwatch cookie 94077908402384] v28'164 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc96803d040 con 0x5590384f1c70 2023-10-05T19:40:29.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.838+0000 7fc9708b6e40 20 remove_watcher() i=0 2023-10-05T19:40:29.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.838+0000 7fc9708b6e40 2 removed watcher, disabling cache 2023-10-05T19:40:29.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.838+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94077908406640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384fc0d0 con 0x559038431c20 2023-10-05T19:40:29.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.842+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(103 notify.1 [watch unwatch cookie 94077908406640] v28'164 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc96000a5a0 con 0x559038431c20 2023-10-05T19:40:29.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.842+0000 7fc9708b6e40 20 remove_watcher() i=1 2023-10-05T19:40:29.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.842+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:104 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94077908408848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384fc0d0 con 0x5590384f6380 2023-10-05T19:40:29.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.842+0000 7fc96dbea700 1 -- 172.21.15.12:0/1061644500 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(104 notify.2 [watch unwatch cookie 94077908408848] v28'164 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc958055020 con 0x5590384f6380 2023-10-05T19:40:29.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.842+0000 7fc9708b6e40 20 remove_watcher() i=2 2023-10-05T19:40:29.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.842+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:105 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94077908414896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384fda10 con 0x5590384f1c70 2023-10-05T19:40:29.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.846+0000 7fc96e3eb700 1 -- 172.21.15.12:0/1061644500 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(105 notify.3 [watch unwatch cookie 94077908414896] v28'164 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc96803d040 con 0x5590384f1c70 2023-10-05T19:40:29.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.846+0000 7fc9708b6e40 20 remove_watcher() i=3 2023-10-05T19:40:29.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.846+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:106 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94077908413632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384fc0d0 con 0x559038431c20 2023-10-05T19:40:29.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.850+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(106 notify.4 [watch unwatch cookie 94077908413632] v28'164 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc96000a5a0 con 0x559038431c20 2023-10-05T19:40:29.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.850+0000 7fc9708b6e40 20 remove_watcher() i=4 2023-10-05T19:40:29.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.850+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:107 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94077908420704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384ff1b0 con 0x7fc968007030 2023-10-05T19:40:29.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.854+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(107 notify.5 [watch unwatch cookie 94077908420704] v28'164 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc96000a5a0 con 0x7fc968007030 2023-10-05T19:40:29.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.854+0000 7fc9708b6e40 20 remove_watcher() i=5 2023-10-05T19:40:29.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.854+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:108 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94077908416000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384fecc0 con 0x7fc968007030 2023-10-05T19:40:29.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.854+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(108 notify.6 [watch unwatch cookie 94077908416000] v28'164 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc96000a5a0 con 0x7fc968007030 2023-10-05T19:40:29.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.854+0000 7fc9708b6e40 20 remove_watcher() i=6 2023-10-05T19:40:29.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.854+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:109 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94077908424384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5590384fc0d0 con 0x7fc968007030 2023-10-05T19:40:29.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.858+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(109 notify.7 [watch unwatch cookie 94077908424384] v28'164 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc96000a5a0 con 0x7fc968007030 2023-10-05T19:40:29.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.858+0000 7fc9708b6e40 20 remove_watcher() i=7 2023-10-05T19:40:29.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.858+0000 7fc93ffdf700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:29.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.858+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:110 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94077908425856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55903853e800 con 0x559038431c20 2023-10-05T19:40:29.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.862+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(110 data_loggenerations_metadata [watch unwatch cookie 94077908425856] v28'156 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fc96000a5a0 con 0x559038431c20 2023-10-05T19:40:29.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.862+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fc968007030 msgr2=0x7fc9680069e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.862+0000 7fc9708b6e40 1 --2- 172.21.15.12:0/1061644500 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fc968007030 0x7fc9680069e0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.862+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x559038431c20 msgr2=0x5590384ef780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.862+0000 7fc9708b6e40 1 --2- 172.21.15.12:0/1061644500 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x559038431c20 0x5590384ef780 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.862+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5590384f1c70 msgr2=0x5590384f4050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.862+0000 7fc9708b6e40 1 --2- 172.21.15.12:0/1061644500 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5590384f1c70 0x5590384f4050 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.862+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5590384f6380 msgr2=0x5590384f8790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.862+0000 7fc9708b6e40 1 --2- 172.21.15.12:0/1061644500 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5590384f6380 0x5590384f8790 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.862+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fc950030c50 msgr2=0x7fc950033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.862+0000 7fc9708b6e40 1 --2- 172.21.15.12:0/1061644500 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fc950030c50 0x7fc950033100 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc958001e90 tx=0x7fc9580013b0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.862+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559038430740 msgr2=0x5590384e3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.862+0000 7fc9708b6e40 1 --2- 172.21.15.12:0/1061644500 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559038430740 0x5590384e3c60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc96800c010 tx=0x7fc96801c000 comp rx=0 tx=0).stop 2023-10-05T19:40:29.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.862+0000 7fc96ebec700 1 -- 172.21.15.12:0/1061644500 reap_dead start 2023-10-05T19:40:29.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.862+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 shutdown_connections 2023-10-05T19:40:29.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.862+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 >> 172.21.15.12:0/1061644500 conn(0x5590384b7850 msgr2=0x5590384d7690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:29.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.862+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 shutdown_connections 2023-10-05T19:40:29.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.862+0000 7fc9708b6e40 1 -- 172.21.15.12:0/1061644500 wait complete. 2023-10-05T19:40:29.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.866+0000 7fc9708b6e40 1 -- 172.21.15.12:0/3650937924 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55903857ca50 msgr2=0x55903857ee30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.866+0000 7fc9708b6e40 1 --2- 172.21.15.12:0/3650937924 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55903857ca50 0x55903857ee30 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.866+0000 7fc9708b6e40 1 -- 172.21.15.12:0/3650937924 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fc8b4030bb0 msgr2=0x7fc8b4033060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.866+0000 7fc9708b6e40 1 --2- 172.21.15.12:0/3650937924 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fc8b4030bb0 0x7fc8b4033060 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x559038502680 tx=0x7fc96000ae90 comp rx=0 tx=0).stop 2023-10-05T19:40:29.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.866+0000 7fc9708b6e40 1 -- 172.21.15.12:0/3650937924 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55903850f980 msgr2=0x559038515480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.866+0000 7fc9708b6e40 1 --2- 172.21.15.12:0/3650937924 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55903850f980 0x559038515480 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fc95800bed0 tx=0x7fc95803d030 comp rx=0 tx=0).stop 2023-10-05T19:40:29.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.866+0000 7fc9708b6e40 1 -- 172.21.15.12:0/3650937924 shutdown_connections 2023-10-05T19:40:29.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.866+0000 7fc9708b6e40 1 --2- 172.21.15.12:0/3650937924 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55903857ca50 0x55903857ee30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.866+0000 7fc9708b6e40 1 --2- 172.21.15.12:0/3650937924 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fc8b4030bb0 0x7fc8b4033060 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.866+0000 7fc9708b6e40 1 --2- 172.21.15.12:0/3650937924 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55903850f980 0x559038515480 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.866+0000 7fc9708b6e40 1 -- 172.21.15.12:0/3650937924 >> 172.21.15.12:0/3650937924 conn(0x55903850ad40 msgr2=0x559038508de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:29.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.866+0000 7fc9708b6e40 1 -- 172.21.15.12:0/3650937924 shutdown_connections 2023-10-05T19:40:29.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.866+0000 7fc9708b6e40 1 -- 172.21.15.12:0/3650937924 wait complete. 2023-10-05T19:40:29.881 INFO:tasks.util.rgw:rgwadmin: client.0 : ['log', 'show', '--object', '2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.3-myfoo3'] 2023-10-05T19:40:29.881 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'log', 'show', '--object', '2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.3-myfoo3'] 2023-10-05T19:40:29.881 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph log show --object 2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.3-myfoo3 2023-10-05T19:40:29.920 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:29.921 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:29.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.910+0000 7f1716209e40 1 Processor -- start 2023-10-05T19:40:29.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.910+0000 7f1716209e40 1 -- start start 2023-10-05T19:40:29.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.910+0000 7f1716209e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55b99433e3f0 0x55b9943407d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.910+0000 7f1716209e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55b994286bd0 con 0x55b994339e10 2023-10-05T19:40:29.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.910+0000 7f1716209e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x55b994243ee0 con 0x55b99433e3f0 2023-10-05T19:40:29.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.914+0000 7f171353d700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55b99433e3f0 0x55b9943407d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.914+0000 7f1713d3e700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x55b994339e10 0x55b99433a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:35702/0 (socket says 172.21.15.12:35702) 2023-10-05T19:40:29.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.914+0000 7f1713d3e700 1 -- 172.21.15.12:0/4073581660 learned_addr learned my addr 172.21.15.12:0/4073581660 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:29.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.914+0000 7f171353d700 1 -- 172.21.15.12:0/4073581660 >> v1:172.21.15.8:6789/0 conn(0x55b994339e10 legacy=0x55b99433a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.914+0000 7f171353d700 1 -- 172.21.15.12:0/4073581660 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b99433aa20 con 0x55b99433e3f0 2023-10-05T19:40:29.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.914+0000 7f171353d700 1 --2- 172.21.15.12:0/4073581660 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b99433e3f0 0x55b9943407d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f16fc00a520 tx=0x7f16fc005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db0ed01c66ca2eed server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.914+0000 7f1712d3c700 1 -- 172.21.15.12:0/4073581660 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16fc00f590 con 0x55b99433e3f0 2023-10-05T19:40:29.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.914+0000 7f1712d3c700 1 -- 172.21.15.12:0/4073581660 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16fc00f6f0 con 0x55b99433e3f0 2023-10-05T19:40:29.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.914+0000 7f1712d3c700 1 -- 172.21.15.12:0/4073581660 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16fc00f880 con 0x55b99433e3f0 2023-10-05T19:40:29.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.914+0000 7f1716209e40 1 -- 172.21.15.12:0/4073581660 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b99433e3f0 msgr2=0x55b9943407d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.914+0000 7f1716209e40 1 --2- 172.21.15.12:0/4073581660 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b99433e3f0 0x55b9943407d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f16fc00a520 tx=0x7f16fc005d40 comp rx=0 tx=0).stop 2023-10-05T19:40:29.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.914+0000 7f1716209e40 1 -- 172.21.15.12:0/4073581660 shutdown_connections 2023-10-05T19:40:29.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.914+0000 7f1716209e40 1 --2- 172.21.15.12:0/4073581660 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b99433e3f0 0x55b9943407d0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.914+0000 7f1716209e40 1 -- 172.21.15.12:0/4073581660 >> 172.21.15.12:0/4073581660 conn(0x55b994335830 msgr2=0x55b994337c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:29.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.914+0000 7f1716209e40 1 -- 172.21.15.12:0/4073581660 shutdown_connections 2023-10-05T19:40:29.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.914+0000 7f1716209e40 1 -- 172.21.15.12:0/4073581660 wait complete. 2023-10-05T19:40:29.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.914+0000 7f1716209e40 1 Processor -- start 2023-10-05T19:40:29.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.914+0000 7f1716209e40 1 -- start start 2023-10-05T19:40:29.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.914+0000 7f1716209e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b994339e10 0x55b9942b0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.914+0000 7f1716209e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b994207e30 con 0x55b994339e10 2023-10-05T19:40:29.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.914+0000 7f1713d3e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b994339e10 0x55b9942b0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.914+0000 7f1713d3e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b994339e10 0x55b9942b0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58688/0 (socket says 172.21.15.12:58688) 2023-10-05T19:40:29.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.914+0000 7f1713d3e700 1 -- 172.21.15.12:0/3012578203 learned_addr learned my addr 172.21.15.12:0/3012578203 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:29.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.918+0000 7f1713d3e700 1 -- 172.21.15.12:0/3012578203 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16fc00a0e0 con 0x55b994339e10 2023-10-05T19:40:29.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.918+0000 7f1713d3e700 1 --2- 172.21.15.12:0/3012578203 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b994339e10 0x55b9942b0170 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f170c001c30 tx=0x7f170c0017e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=909a4d214214d00e server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.918+0000 7f1711d3a700 1 -- 172.21.15.12:0/3012578203 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f170c00bbf0 con 0x55b994339e10 2023-10-05T19:40:29.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.918+0000 7f1711d3a700 1 -- 172.21.15.12:0/3012578203 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f170c00bd50 con 0x55b994339e10 2023-10-05T19:40:29.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.918+0000 7f1711d3a700 1 -- 172.21.15.12:0/3012578203 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f170c00e470 con 0x55b994339e10 2023-10-05T19:40:29.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.918+0000 7f1716209e40 1 -- 172.21.15.12:0/3012578203 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b994339e10 msgr2=0x55b9942b0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:29.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.918+0000 7f1716209e40 1 --2- 172.21.15.12:0/3012578203 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b994339e10 0x55b9942b0170 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f170c001c30 tx=0x7f170c0017e0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.918+0000 7f1716209e40 1 -- 172.21.15.12:0/3012578203 shutdown_connections 2023-10-05T19:40:29.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.918+0000 7f1716209e40 1 --2- 172.21.15.12:0/3012578203 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b994339e10 0x55b9942b0170 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:29.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.918+0000 7f1716209e40 1 -- 172.21.15.12:0/3012578203 >> 172.21.15.12:0/3012578203 conn(0x55b994335830 msgr2=0x55b994354ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:29.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.918+0000 7f1716209e40 1 -- 172.21.15.12:0/3012578203 shutdown_connections 2023-10-05T19:40:29.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.918+0000 7f1716209e40 1 -- 172.21.15.12:0/3012578203 wait complete. 2023-10-05T19:40:29.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.918+0000 7f1716209e40 1 Processor -- start 2023-10-05T19:40:29.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.918+0000 7f1716209e40 1 -- start start 2023-10-05T19:40:29.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.918+0000 7f1716209e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b994339e10 0x55b994361fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.918+0000 7f1716209e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9941daed0 con 0x55b994339e10 2023-10-05T19:40:29.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.918+0000 7f1713d3e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b994339e10 0x55b994361fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.918+0000 7f1713d3e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b994339e10 0x55b994361fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58702/0 (socket says 172.21.15.12:58702) 2023-10-05T19:40:29.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.918+0000 7f1713d3e700 1 -- 172.21.15.12:0/3229436812 learned_addr learned my addr 172.21.15.12:0/3229436812 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:29.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.918+0000 7f1713d3e700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b994286bd0 con 0x55b994339e10 2023-10-05T19:40:29.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.918+0000 7f1713d3e700 1 --2- 172.21.15.12:0/3229436812 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b994339e10 0x55b994361fb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f170c001820 tx=0x7f170c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.922+0000 7f170bfff700 1 -- 172.21.15.12:0/3229436812 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f170c01f070 con 0x55b994339e10 2023-10-05T19:40:29.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.922+0000 7f170bfff700 1 -- 172.21.15.12:0/3229436812 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f170c0197b0 con 0x55b994339e10 2023-10-05T19:40:29.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.922+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b9943624f0 con 0x55b994339e10 2023-10-05T19:40:29.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.922+0000 7f170bfff700 1 -- 172.21.15.12:0/3229436812 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f170c019960 con 0x55b994339e10 2023-10-05T19:40:29.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.922+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b994362930 con 0x55b994339e10 2023-10-05T19:40:29.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.922+0000 7f170bfff700 1 -- 172.21.15.12:0/3229436812 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f170c022020 con 0x55b994339e10 2023-10-05T19:40:29.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.922+0000 7f170bfff700 1 --2- 172.21.15.12:0/3229436812 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f16f4030c00 0x7f16f40330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.922+0000 7f170bfff700 1 -- 172.21.15.12:0/3229436812 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f170c044d20 con 0x55b994339e10 2023-10-05T19:40:29.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.922+0000 7f1716209e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.922+0000 7f171353d700 1 --2- 172.21.15.12:0/3229436812 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f16f4030c00 0x7f16f40330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.922+0000 7f1716209e40 1 --2- 172.21.15.12:0/3229436812 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55b99433e3f0 0x55b99436da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.922+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b99436e140 con 0x55b99433e3f0 2023-10-05T19:40:29.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.922+0000 7f171453f700 1 --2- 172.21.15.12:0/3229436812 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55b99433e3f0 0x55b99436da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.922+0000 7f171353d700 1 --2- 172.21.15.12:0/3229436812 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f16f4030c00 0x7f16f40330b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f16fc005d80 tx=0x7f16fc014000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.922+0000 7f171453f700 1 --2- 172.21.15.12:0/3229436812 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55b99433e3f0 0x55b99436da40 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.926+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f170400a500 con 0x55b99433e3f0 2023-10-05T19:40:29.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.926+0000 7f1716209e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:29.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.926+0000 7f1716209e40 20 realm 2023-10-05T19:40:29.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.926+0000 7f1716209e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.926+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b99436b0a0 con 0x55b99433e3f0 2023-10-05T19:40:29.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.926+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f170400a500 con 0x55b99433e3f0 2023-10-05T19:40:29.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.926+0000 7f1716209e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:29.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.926+0000 7f1716209e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:29.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.926+0000 7f1716209e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.926+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b99436ec20 con 0x55b99433e3f0 2023-10-05T19:40:29.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.926+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f170400a500 con 0x55b99433e3f0 2023-10-05T19:40:29.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.926+0000 7f1716209e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:29.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.926+0000 7f1716209e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.926+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b99436e800 con 0x55b99433e3f0 2023-10-05T19:40:29.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.926+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f170400a500 con 0x55b99433e3f0 2023-10-05T19:40:29.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.926+0000 7f1716209e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:29.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.926+0000 7f1716209e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.926+0000 7f1716209e40 1 --2- 172.21.15.12:0/3229436812 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55b99436fbb0 0x55b994371f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.926+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994372700 con 0x55b99436fbb0 2023-10-05T19:40:29.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.926+0000 7f1713d3e700 1 --2- 172.21.15.12:0/3229436812 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55b99436fbb0 0x55b994371f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.926+0000 7f1713d3e700 1 --2- 172.21.15.12:0/3229436812 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55b99436fbb0 0x55b994371f90 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.930+0000 7f1713d3e700 1 -- 172.21.15.12:0/3229436812 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f170c0173e0 con 0x55b99436fbb0 2023-10-05T19:40:29.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.930+0000 7f1716209e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:29.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.930+0000 7f1716209e40 20 searching for the correct realm 2023-10-05T19:40:29.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.930+0000 7f1716209e40 1 --2- 172.21.15.12:0/3229436812 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55b994374530 0x55b994376910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.930+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994376e50 con 0x55b994374530 2023-10-05T19:40:29.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.930+0000 7f171353d700 1 --2- 172.21.15.12:0/3229436812 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55b994374530 0x55b994376910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.930+0000 7f171353d700 1 --2- 172.21.15.12:0/3229436812 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55b994374530 0x55b994376910 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.930+0000 7f171353d700 1 -- 172.21.15.12:0/3229436812 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f16fc00fca0 con 0x55b994374530 2023-10-05T19:40:29.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.930+0000 7f171353d700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994376e50 con 0x55b99436fbb0 2023-10-05T19:40:29.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.930+0000 7f1713d3e700 1 -- 172.21.15.12:0/3229436812 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f170c0173e0 con 0x55b99436fbb0 2023-10-05T19:40:29.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.930+0000 7f1713d3e700 1 --2- 172.21.15.12:0/3229436812 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f170c016b60 0x7f170c0243d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:29.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.930+0000 7f1713d3e700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994376e50 con 0x7f170c016b60 2023-10-05T19:40:29.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.930+0000 7f171453f700 1 --2- 172.21.15.12:0/3229436812 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f170c016b60 0x7f170c0243d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:29.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.930+0000 7f171453f700 1 --2- 172.21.15.12:0/3229436812 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f170c016b60 0x7f170c0243d0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:29.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.934+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1704014040 con 0x7f170c016b60 2023-10-05T19:40:29.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.934+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994376e50 con 0x55b99433e3f0 2023-10-05T19:40:29.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.934+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1704014040 con 0x55b99433e3f0 2023-10-05T19:40:29.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.934+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994376e50 con 0x55b99433e3f0 2023-10-05T19:40:29.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.934+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1704014040 con 0x55b99433e3f0 2023-10-05T19:40:29.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.934+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994376e50 con 0x7f170c016b60 2023-10-05T19:40:29.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.934+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1704014040 con 0x7f170c016b60 2023-10-05T19:40:29.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.934+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994376e50 con 0x7f170c016b60 2023-10-05T19:40:29.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.934+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1704014040 con 0x7f170c016b60 2023-10-05T19:40:29.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.934+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994376e50 con 0x55b99436fbb0 2023-10-05T19:40:29.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.938+0000 7f1713d3e700 1 -- 172.21.15.12:0/3229436812 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f170c0173e0 con 0x55b99436fbb0 2023-10-05T19:40:29.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.938+0000 7f1713d3e700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994376e50 con 0x55b994374530 2023-10-05T19:40:29.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.938+0000 7f171353d700 1 -- 172.21.15.12:0/3229436812 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f16fc00fca0 con 0x55b994374530 2023-10-05T19:40:29.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.938+0000 7f171353d700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994376e50 con 0x55b99433e3f0 2023-10-05T19:40:29.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.938+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1704014040 con 0x55b99433e3f0 2023-10-05T19:40:29.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.938+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994376e50 con 0x7f170c016b60 2023-10-05T19:40:29.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.938+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1704014040 con 0x7f170c016b60 2023-10-05T19:40:29.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.938+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994376e50 con 0x55b99433e3f0 2023-10-05T19:40:29.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.938+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1704014040 con 0x55b99433e3f0 2023-10-05T19:40:29.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.938+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994376e50 con 0x55b99436fbb0 2023-10-05T19:40:29.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.942+0000 7f1713d3e700 1 -- 172.21.15.12:0/3229436812 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f170c0173e0 con 0x55b99436fbb0 2023-10-05T19:40:29.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.942+0000 7f1713d3e700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994376e50 con 0x55b994374530 2023-10-05T19:40:29.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.942+0000 7f171353d700 1 -- 172.21.15.12:0/3229436812 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f16fc00fca0 con 0x55b994374530 2023-10-05T19:40:29.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.942+0000 7f171353d700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994376e50 con 0x55b994374530 2023-10-05T19:40:29.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.942+0000 7f171353d700 1 -- 172.21.15.12:0/3229436812 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f16fc00fca0 con 0x55b994374530 2023-10-05T19:40:29.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.942+0000 7f171353d700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994376e50 con 0x7f170c016b60 2023-10-05T19:40:29.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.942+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1704014040 con 0x7f170c016b60 2023-10-05T19:40:29.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.942+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994376e50 con 0x55b994374530 2023-10-05T19:40:29.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.942+0000 7f171353d700 1 -- 172.21.15.12:0/3229436812 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f16fc00fca0 con 0x55b994374530 2023-10-05T19:40:29.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.942+0000 7f171353d700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994376e50 con 0x55b994374530 2023-10-05T19:40:29.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.946+0000 7f171353d700 1 -- 172.21.15.12:0/3229436812 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f16fc00fca0 con 0x55b994374530 2023-10-05T19:40:29.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.946+0000 7f171353d700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994376e50 con 0x55b99433e3f0 2023-10-05T19:40:29.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.946+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1704014040 con 0x55b99433e3f0 2023-10-05T19:40:29.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.946+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994376e50 con 0x55b99433e3f0 2023-10-05T19:40:29.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.946+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1704014040 con 0x55b99433e3f0 2023-10-05T19:40:29.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.946+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994376e50 con 0x7f170c016b60 2023-10-05T19:40:29.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.946+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1704014040 con 0x7f170c016b60 2023-10-05T19:40:29.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.946+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994376e50 con 0x55b994374530 2023-10-05T19:40:29.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.946+0000 7f171353d700 1 -- 172.21.15.12:0/3229436812 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f16fc00fca0 con 0x55b994374530 2023-10-05T19:40:29.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.946+0000 7f171353d700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994376e50 con 0x55b99436fbb0 2023-10-05T19:40:29.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.950+0000 7f1713d3e700 1 -- 172.21.15.12:0/3229436812 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f170c0173e0 con 0x55b99436fbb0 2023-10-05T19:40:29.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.950+0000 7f1713d3e700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994376e50 con 0x55b99433e3f0 2023-10-05T19:40:29.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.950+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1704014040 con 0x55b99433e3f0 2023-10-05T19:40:29.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.950+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994376e50 con 0x7f170c016b60 2023-10-05T19:40:29.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.950+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1704014040 con 0x7f170c016b60 2023-10-05T19:40:29.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.950+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994376e50 con 0x55b994374530 2023-10-05T19:40:29.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.950+0000 7f171353d700 1 -- 172.21.15.12:0/3229436812 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f16fc00fca0 con 0x55b994374530 2023-10-05T19:40:29.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.950+0000 7f171353d700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994376e50 con 0x7f170c016b60 2023-10-05T19:40:29.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.954+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1704014040 con 0x7f170c016b60 2023-10-05T19:40:29.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.954+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994376e50 con 0x55b99436fbb0 2023-10-05T19:40:29.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.954+0000 7f1713d3e700 1 -- 172.21.15.12:0/3229436812 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f170c0173e0 con 0x55b99436fbb0 2023-10-05T19:40:29.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.954+0000 7f1713d3e700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994376e50 con 0x55b994374530 2023-10-05T19:40:29.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.954+0000 7f171353d700 1 -- 172.21.15.12:0/3229436812 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f16fc00fca0 con 0x55b994374530 2023-10-05T19:40:29.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.954+0000 7f171353d700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994376e50 con 0x7f170c016b60 2023-10-05T19:40:29.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.954+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1704014040 con 0x7f170c016b60 2023-10-05T19:40:29.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.954+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994376e50 con 0x55b99433e3f0 2023-10-05T19:40:29.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.954+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f1704014040 con 0x55b99433e3f0 2023-10-05T19:40:29.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.954+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994376e50 con 0x55b99436fbb0 2023-10-05T19:40:29.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.958+0000 7f1713d3e700 1 -- 172.21.15.12:0/3229436812 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f170c0173e0 con 0x55b99436fbb0 2023-10-05T19:40:29.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.958+0000 7f1716209e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:29.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.958+0000 7f1716209e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:29.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.958+0000 7f1716209e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:29.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.958+0000 7f1716209e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:29.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.958+0000 7f1716209e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.958+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994373280 con 0x55b99433e3f0 2023-10-05T19:40:29.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.958+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1704014040 con 0x55b99433e3f0 2023-10-05T19:40:29.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.958+0000 7f1716209e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:29.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.958+0000 7f1716209e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.958+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994372e50 con 0x55b99433e3f0 2023-10-05T19:40:29.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.958+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f1704014040 con 0x55b99433e3f0 2023-10-05T19:40:29.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.958+0000 7f1716209e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:29.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.958+0000 7f1716209e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.958+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994373bb0 con 0x55b994374530 2023-10-05T19:40:29.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.958+0000 7f171353d700 1 -- 172.21.15.12:0/3229436812 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f16fc00fca0 con 0x55b994374530 2023-10-05T19:40:29.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.958+0000 7f1716209e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:29.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.958+0000 7f1716209e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.958+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994377780 con 0x55b994374530 2023-10-05T19:40:29.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.958+0000 7f171353d700 1 -- 172.21.15.12:0/3229436812 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f16fc00fca0 con 0x55b994374530 2023-10-05T19:40:29.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.958+0000 7f1716209e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:29.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.958+0000 7f1716209e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.958+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994377b30 con 0x55b99433e3f0 2023-10-05T19:40:29.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.962+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1704014040 con 0x55b99433e3f0 2023-10-05T19:40:29.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.962+0000 7f1716209e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:29.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.962+0000 7f1716209e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:29.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.962+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994377ee0 con 0x55b99433e3f0 2023-10-05T19:40:29.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.962+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f1704014040 con 0x55b99433e3f0 2023-10-05T19:40:29.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.962+0000 7f1716209e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:29.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.962+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994378ef0 con 0x55b994374530 2023-10-05T19:40:29.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.962+0000 7f171353d700 1 -- 172.21.15.12:0/3229436812 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f16fc00fca0 con 0x55b994374530 2023-10-05T19:40:29.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.962+0000 7f171353d700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994378ef0 con 0x55b99436fbb0 2023-10-05T19:40:29.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.966+0000 7f1713d3e700 1 -- 172.21.15.12:0/3229436812 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f170c0173e0 con 0x55b99436fbb0 2023-10-05T19:40:29.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.966+0000 7f1713d3e700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994378ef0 con 0x7f170c016b60 2023-10-05T19:40:29.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.966+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1704014040 con 0x7f170c016b60 2023-10-05T19:40:29.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.966+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994377b30 con 0x55b99433e3f0 2023-10-05T19:40:29.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.966+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1704014040 con 0x55b99433e3f0 2023-10-05T19:40:29.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.966+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994377b30 con 0x55b99433e3f0 2023-10-05T19:40:29.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.966+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1704014040 con 0x55b99433e3f0 2023-10-05T19:40:29.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.966+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994377b30 con 0x7f170c016b60 2023-10-05T19:40:29.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.966+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1704014040 con 0x7f170c016b60 2023-10-05T19:40:29.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.970+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994377b30 con 0x7f170c016b60 2023-10-05T19:40:29.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.970+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1704014040 con 0x7f170c016b60 2023-10-05T19:40:29.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.970+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994377b30 con 0x55b99436fbb0 2023-10-05T19:40:29.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.970+0000 7f1713d3e700 1 -- 172.21.15.12:0/3229436812 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f170c0173e0 con 0x55b99436fbb0 2023-10-05T19:40:29.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.970+0000 7f1713d3e700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994377b30 con 0x55b994374530 2023-10-05T19:40:29.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.970+0000 7f171353d700 1 -- 172.21.15.12:0/3229436812 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f16fc00fca0 con 0x55b994374530 2023-10-05T19:40:29.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.970+0000 7f171353d700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994377b30 con 0x55b99433e3f0 2023-10-05T19:40:29.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.970+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1704014040 con 0x55b99433e3f0 2023-10-05T19:40:29.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.970+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994377b30 con 0x7f170c016b60 2023-10-05T19:40:29.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.974+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1704014040 con 0x7f170c016b60 2023-10-05T19:40:29.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.974+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994377b30 con 0x55b99433e3f0 2023-10-05T19:40:29.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.974+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1704014040 con 0x55b99433e3f0 2023-10-05T19:40:29.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.974+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994377b30 con 0x55b99436fbb0 2023-10-05T19:40:29.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.974+0000 7f1713d3e700 1 -- 172.21.15.12:0/3229436812 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f170c0173e0 con 0x55b99436fbb0 2023-10-05T19:40:29.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.974+0000 7f1713d3e700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994377b30 con 0x55b994374530 2023-10-05T19:40:29.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.974+0000 7f171353d700 1 -- 172.21.15.12:0/3229436812 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f16fc00fca0 con 0x55b994374530 2023-10-05T19:40:29.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.974+0000 7f171353d700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994377b30 con 0x55b994374530 2023-10-05T19:40:29.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.974+0000 7f171353d700 1 -- 172.21.15.12:0/3229436812 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f16fc00fca0 con 0x55b994374530 2023-10-05T19:40:29.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.978+0000 7f171353d700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994377b30 con 0x7f170c016b60 2023-10-05T19:40:29.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.978+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1704014040 con 0x7f170c016b60 2023-10-05T19:40:29.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.978+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994377b30 con 0x55b994374530 2023-10-05T19:40:29.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.978+0000 7f171353d700 1 -- 172.21.15.12:0/3229436812 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f16fc00fca0 con 0x55b994374530 2023-10-05T19:40:29.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.978+0000 7f171353d700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994377b30 con 0x55b994374530 2023-10-05T19:40:29.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.978+0000 7f171353d700 1 -- 172.21.15.12:0/3229436812 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f16fc00fca0 con 0x55b994374530 2023-10-05T19:40:29.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.978+0000 7f171353d700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994377b30 con 0x55b99433e3f0 2023-10-05T19:40:29.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.978+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1704014040 con 0x55b99433e3f0 2023-10-05T19:40:29.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.978+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994377b30 con 0x55b99433e3f0 2023-10-05T19:40:29.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.982+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1704014040 con 0x55b99433e3f0 2023-10-05T19:40:29.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.982+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994377b30 con 0x7f170c016b60 2023-10-05T19:40:29.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.982+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1704014040 con 0x7f170c016b60 2023-10-05T19:40:29.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.982+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994377b30 con 0x55b994374530 2023-10-05T19:40:29.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.982+0000 7f171353d700 1 -- 172.21.15.12:0/3229436812 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f16fc00fca0 con 0x55b994374530 2023-10-05T19:40:29.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.982+0000 7f171353d700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994377b30 con 0x55b99436fbb0 2023-10-05T19:40:29.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.982+0000 7f1713d3e700 1 -- 172.21.15.12:0/3229436812 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f170c0173e0 con 0x55b99436fbb0 2023-10-05T19:40:29.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.982+0000 7f1713d3e700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994377b30 con 0x55b99433e3f0 2023-10-05T19:40:29.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.986+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1704014040 con 0x55b99433e3f0 2023-10-05T19:40:29.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.986+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994377b30 con 0x7f170c016b60 2023-10-05T19:40:29.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.986+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1704014040 con 0x7f170c016b60 2023-10-05T19:40:29.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.986+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994377b30 con 0x55b994374530 2023-10-05T19:40:29.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.986+0000 7f171353d700 1 -- 172.21.15.12:0/3229436812 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f16fc00fca0 con 0x55b994374530 2023-10-05T19:40:29.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.986+0000 7f171353d700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994377b30 con 0x7f170c016b60 2023-10-05T19:40:29.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.986+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1704014040 con 0x7f170c016b60 2023-10-05T19:40:29.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.986+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994377b30 con 0x55b99436fbb0 2023-10-05T19:40:29.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.990+0000 7f1713d3e700 1 -- 172.21.15.12:0/3229436812 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f170c0173e0 con 0x55b99436fbb0 2023-10-05T19:40:29.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.990+0000 7f1713d3e700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994377b30 con 0x55b994374530 2023-10-05T19:40:29.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.990+0000 7f171353d700 1 -- 172.21.15.12:0/3229436812 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f16fc00fca0 con 0x55b994374530 2023-10-05T19:40:29.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.990+0000 7f171353d700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994377b30 con 0x7f170c016b60 2023-10-05T19:40:30.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.990+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1704014040 con 0x7f170c016b60 2023-10-05T19:40:30.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.990+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994377b30 con 0x55b99433e3f0 2023-10-05T19:40:30.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.990+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f1704014040 con 0x55b99433e3f0 2023-10-05T19:40:30.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.990+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994377b30 con 0x55b99436fbb0 2023-10-05T19:40:30.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.990+0000 7f1713d3e700 1 -- 172.21.15.12:0/3229436812 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f170c0173e0 con 0x55b99436fbb0 2023-10-05T19:40:30.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.994+0000 7f1716209e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:30.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.994+0000 7f1716209e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:30.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.994+0000 7f1716209e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:30.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.994+0000 7f1716209e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:30.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.994+0000 7f1716209e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.994+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994377ee0 con 0x55b99433e3f0 2023-10-05T19:40:30.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.994+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f1704014040 con 0x55b99433e3f0 2023-10-05T19:40:30.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.994+0000 7f1716209e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:30.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.994+0000 7f1716209e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.994+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994378290 con 0x55b99436fbb0 2023-10-05T19:40:30.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.994+0000 7f1713d3e700 1 -- 172.21.15.12:0/3229436812 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f170c0173e0 con 0x55b99436fbb0 2023-10-05T19:40:30.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.994+0000 7f1716209e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:30.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.994+0000 7f1716209e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.994+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b9943773b0 con 0x55b99433e3f0 2023-10-05T19:40:30.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.994+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f1704014040 con 0x55b99433e3f0 2023-10-05T19:40:30.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.994+0000 7f1716209e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:30.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.994+0000 7f1716209e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.994+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b9943773c0 con 0x55b994374530 2023-10-05T19:40:30.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.994+0000 7f171353d700 1 -- 172.21.15.12:0/3229436812 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f16fc00fca0 con 0x55b994374530 2023-10-05T19:40:30.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.994+0000 7f1716209e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:30.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.994+0000 7f1716209e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.994+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994379130 con 0x7f170c016b60 2023-10-05T19:40:30.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.998+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f1704014040 con 0x7f170c016b60 2023-10-05T19:40:30.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.998+0000 7f1716209e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:30.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.998+0000 7f1716209e40 20 zone default found 2023-10-05T19:40:30.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.998+0000 7f1716209e40 4 Realm: () 2023-10-05T19:40:30.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.998+0000 7f1716209e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:30.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.998+0000 7f1716209e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:30.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.998+0000 7f1716209e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:30.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.998+0000 7f1716209e40 20 zonegroup default 2023-10-05T19:40:30.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.998+0000 7f1716209e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.998+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994379130 con 0x7f170c016b60 2023-10-05T19:40:30.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.998+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f1704014040 con 0x7f170c016b60 2023-10-05T19:40:30.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.998+0000 7f1716209e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:30.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.998+0000 7f1716209e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.998+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b99437aab0 con 0x55b994374530 2023-10-05T19:40:30.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.998+0000 7f171353d700 1 -- 172.21.15.12:0/3229436812 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f16fc00fca0 con 0x55b994374530 2023-10-05T19:40:30.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.998+0000 7f1716209e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:30.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.998+0000 7f1716209e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.998+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b99437ae60 con 0x55b994374530 2023-10-05T19:40:30.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.998+0000 7f171353d700 1 -- 172.21.15.12:0/3229436812 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f16fc00fca0 con 0x55b994374530 2023-10-05T19:40:30.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.998+0000 7f1716209e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:30.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.998+0000 7f1716209e40 20 started sync module instance, tier type = 2023-10-05T19:40:30.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.998+0000 7f1716209e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:30.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:29.998+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b99437a170 con 0x55b99436fbb0 2023-10-05T19:40:30.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.002+0000 7f1713d3e700 1 -- 172.21.15.12:0/3229436812 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'165 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f170c0173e0 con 0x55b99436fbb0 2023-10-05T19:40:30.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.002+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94255543983232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b99437b210 con 0x55b99436fbb0 2023-10-05T19:40:30.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.002+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b99437bab0 con 0x55b99433e3f0 2023-10-05T19:40:30.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.006+0000 7f1713d3e700 1 -- 172.21.15.12:0/3229436812 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94255543983232] v28'166 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f170c0173e0 con 0x55b99436fbb0 2023-10-05T19:40:30.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.006+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'165 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1704014040 con 0x55b99433e3f0 2023-10-05T19:40:30.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.006+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94255543985520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b99437bab0 con 0x55b99433e3f0 2023-10-05T19:40:30.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.006+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b99437c5d0 con 0x55b994374530 2023-10-05T19:40:30.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.010+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94255543985520] v28'166 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1704014040 con 0x55b99433e3f0 2023-10-05T19:40:30.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.010+0000 7f171353d700 1 -- 172.21.15.12:0/3229436812 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'165 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f16fc00fca0 con 0x55b994374530 2023-10-05T19:40:30.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.010+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94255543992544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b99437c980 con 0x55b994374530 2023-10-05T19:40:30.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.010+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b99437d390 con 0x55b99436fbb0 2023-10-05T19:40:30.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.010+0000 7f1713d3e700 1 -- 172.21.15.12:0/3229436812 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'165 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f170c0173e0 con 0x55b99436fbb0 2023-10-05T19:40:30.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.014+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94255543998272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b99437cfc0 con 0x55b99436fbb0 2023-10-05T19:40:30.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.014+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b99437e890 con 0x55b99433e3f0 2023-10-05T19:40:30.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.014+0000 7f171353d700 1 -- 172.21.15.12:0/3229436812 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94255543992544] v28'166 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f16fc00fca0 con 0x55b994374530 2023-10-05T19:40:30.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.014+0000 7f1713d3e700 1 -- 172.21.15.12:0/3229436812 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94255543998272] v28'166 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f170c0173e0 con 0x55b99436fbb0 2023-10-05T19:40:30.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.014+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'165 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1704014040 con 0x55b99433e3f0 2023-10-05T19:40:30.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.014+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94255543999376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b99437cfc0 con 0x55b99433e3f0 2023-10-05T19:40:30.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.014+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b99437e4d0 con 0x7f170c016b60 2023-10-05T19:40:30.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.018+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'165 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1704014040 con 0x7f170c016b60 2023-10-05T19:40:30.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.018+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94255544003456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b99437f4c0 con 0x7f170c016b60 2023-10-05T19:40:30.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.018+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b99437ffa0 con 0x7f170c016b60 2023-10-05T19:40:30.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.018+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94255543999376] v28'166 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1704014040 con 0x55b99433e3f0 2023-10-05T19:40:30.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.018+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(95 notify.5 [watch watch cookie 94255544003456] v28'166 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1704014040 con 0x7f170c016b60 2023-10-05T19:40:30.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.018+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(96 notify.6 [create] v28'165 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1704014040 con 0x7f170c016b60 2023-10-05T19:40:30.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.018+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94255543989696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b99437efd0 con 0x7f170c016b60 2023-10-05T19:40:30.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.018+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994380890 con 0x7f170c016b60 2023-10-05T19:40:30.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.022+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(98 notify.7 [create] v28'165 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1704014040 con 0x7f170c016b60 2023-10-05T19:40:30.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.022+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94255544009632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b99437efd0 con 0x7f170c016b60 2023-10-05T19:40:30.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.022+0000 7f1716209e40 20 add_watcher() i=0 2023-10-05T19:40:30.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.022+0000 7f1716209e40 20 add_watcher() i=1 2023-10-05T19:40:30.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.022+0000 7f1716209e40 20 add_watcher() i=2 2023-10-05T19:40:30.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.022+0000 7f1716209e40 20 add_watcher() i=3 2023-10-05T19:40:30.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.022+0000 7f1716209e40 20 add_watcher() i=4 2023-10-05T19:40:30.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.022+0000 7f1716209e40 20 add_watcher() i=5 2023-10-05T19:40:30.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.022+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(97 notify.6 [watch watch cookie 94255543989696] v28'166 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1704014040 con 0x7f170c016b60 2023-10-05T19:40:30.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.022+0000 7f1716209e40 20 add_watcher() i=6 2023-10-05T19:40:30.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.026+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94255544009632] v28'166 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1704014040 con 0x7f170c016b60 2023-10-05T19:40:30.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.026+0000 7f1716209e40 20 add_watcher() i=7 2023-10-05T19:40:30.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.026+0000 7f1716209e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:30.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.026+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994383820 con 0x55b99433e3f0 2023-10-05T19:40:30.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.026+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f1704014040 con 0x55b99433e3f0 2023-10-05T19:40:30.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.026+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94255544007280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994380ce0 con 0x55b99433e3f0 2023-10-05T19:40:30.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.026+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94255544007280] v28'157 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f1704014040 con 0x55b99433e3f0 2023-10-05T19:40:30.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.026+0000 7f16e57e2700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:30.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.026+0000 7f1716209e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:30.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.026+0000 7f1716209e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:30.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.030+0000 7f1716209e40 1 Processor -- start 2023-10-05T19:40:30.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.030+0000 7f1716209e40 1 -- start start 2023-10-05T19:40:30.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.030+0000 7f1716209e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b99438daf0 0x55b99438dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.030+0000 7f1716209e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b99438e400 con 0x55b99438daf0 2023-10-05T19:40:30.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.030+0000 7f171353d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b99438daf0 0x55b99438dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.030+0000 7f171353d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b99438daf0 0x55b99438dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58708/0 (socket says 172.21.15.12:58708) 2023-10-05T19:40:30.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.030+0000 7f171353d700 1 -- 172.21.15.12:0/1610775512 learned_addr learned my addr 172.21.15.12:0/1610775512 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:30.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.030+0000 7f171353d700 1 -- 172.21.15.12:0/1610775512 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b99438e540 con 0x55b99438daf0 2023-10-05T19:40:30.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.030+0000 7f171353d700 1 --2- 172.21.15.12:0/1610775512 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b99438daf0 0x55b99438dec0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f16fc003000 tx=0x7f16fc011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b8f19cf719364df server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.030+0000 7f16e4fe1700 1 -- 172.21.15.12:0/1610775512 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16fc01ccf0 con 0x55b99438daf0 2023-10-05T19:40:30.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.030+0000 7f16e4fe1700 1 -- 172.21.15.12:0/1610775512 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16fc01ce70 con 0x55b99438daf0 2023-10-05T19:40:30.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.030+0000 7f16e4fe1700 1 -- 172.21.15.12:0/1610775512 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16fc044440 con 0x55b99438daf0 2023-10-05T19:40:30.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.030+0000 7f1716209e40 1 -- 172.21.15.12:0/1610775512 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b99438daf0 msgr2=0x55b99438dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.030+0000 7f1716209e40 1 --2- 172.21.15.12:0/1610775512 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b99438daf0 0x55b99438dec0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f16fc003000 tx=0x7f16fc011000 comp rx=0 tx=0).stop 2023-10-05T19:40:30.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.030+0000 7f1716209e40 1 -- 172.21.15.12:0/1610775512 shutdown_connections 2023-10-05T19:40:30.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.030+0000 7f1716209e40 1 --2- 172.21.15.12:0/1610775512 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b99438daf0 0x55b99438dec0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.030+0000 7f1716209e40 1 -- 172.21.15.12:0/1610775512 >> 172.21.15.12:0/1610775512 conn(0x55b994388eb0 msgr2=0x55b994387060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:30.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.030+0000 7f1716209e40 1 -- 172.21.15.12:0/1610775512 shutdown_connections 2023-10-05T19:40:30.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.030+0000 7f1716209e40 1 -- 172.21.15.12:0/1610775512 wait complete. 2023-10-05T19:40:30.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.034+0000 7f1716209e40 1 Processor -- start 2023-10-05T19:40:30.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.034+0000 7f1716209e40 1 -- start start 2023-10-05T19:40:30.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.034+0000 7f1716209e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b99438daf0 0x55b994393600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.034+0000 7f1716209e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b994394600 con 0x55b99438daf0 2023-10-05T19:40:30.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.034+0000 7f171353d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b99438daf0 0x55b994393600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.034+0000 7f171353d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b99438daf0 0x55b994393600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58722/0 (socket says 172.21.15.12:58722) 2023-10-05T19:40:30.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.034+0000 7f171353d700 1 -- 172.21.15.12:0/322455623 learned_addr learned my addr 172.21.15.12:0/322455623 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:30.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.034+0000 7f171353d700 1 -- 172.21.15.12:0/322455623 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16fc02e040 con 0x55b99438daf0 2023-10-05T19:40:30.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.034+0000 7f171353d700 1 --2- 172.21.15.12:0/322455623 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b99438daf0 0x55b994393600 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f16fc00fb60 tx=0x7f16fc044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.034+0000 7f165effd700 1 -- 172.21.15.12:0/322455623 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16fc044bc0 con 0x55b99438daf0 2023-10-05T19:40:30.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.034+0000 7f165effd700 1 -- 172.21.15.12:0/322455623 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16fc044d20 con 0x55b99438daf0 2023-10-05T19:40:30.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.034+0000 7f1716209e40 1 -- 172.21.15.12:0/322455623 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b994393c00 con 0x55b99438daf0 2023-10-05T19:40:30.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.034+0000 7f165effd700 1 -- 172.21.15.12:0/322455623 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16fc057460 con 0x55b99438daf0 2023-10-05T19:40:30.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.034+0000 7f1716209e40 1 -- 172.21.15.12:0/322455623 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b9943940a0 con 0x55b99438daf0 2023-10-05T19:40:30.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.038+0000 7f165effd700 1 -- 172.21.15.12:0/322455623 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f16fc0575c0 con 0x55b99438daf0 2023-10-05T19:40:30.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.038+0000 7f165effd700 1 --2- 172.21.15.12:0/322455623 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1654030c50 0x7f1654033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.038+0000 7f165effd700 1 -- 172.21.15.12:0/322455623 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f16fc02d160 con 0x55b99438daf0 2023-10-05T19:40:30.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.038+0000 7f171453f700 1 --2- 172.21.15.12:0/322455623 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1654030c50 0x7f1654033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.038+0000 7f1716209e40 5 note: GC not initialized 2023-10-05T19:40:30.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.038+0000 7f171453f700 1 --2- 172.21.15.12:0/322455623 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1654030c50 0x7f1654033100 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x55b99437f1d0 tx=0x7f170400ab60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.038+0000 7f1653fff700 20 reqs_thread_entry: start 2023-10-05T19:40:30.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.038+0000 7f1651ffb700 20 reqs_thread_entry: start 2023-10-05T19:40:30.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.038+0000 7f1716209e40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:30.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.042+0000 7f1716209e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:30.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.042+0000 7f163ffff700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:41:00 2023 2023-10-05T19:40:30.051 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:30.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.042+0000 7f1716209e40 10 log_show_next pos 0 bl 0 off 0 eof 0 2023-10-05T19:40:30.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.042+0000 7f163f7fe700 20 reqs_thread_entry: start 2023-10-05T19:40:30.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.042+0000 7f1716209e40 1 --2- 172.21.15.12:0/322455623 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55b9943fac90 0x55b9943fd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.042+0000 7f1716209e40 1 -- 172.21.15.12:0/322455623 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:1 6.1e 6:7f266c6f:::2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.3-myfoo3:head [read 0~1048576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b9943fd5b0 con 0x55b9943fac90 2023-10-05T19:40:30.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.042+0000 7f1713d3e700 1 --2- 172.21.15.12:0/322455623 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55b9943fac90 0x55b9943fd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.042+0000 7f1713d3e700 1 --2- 172.21.15.12:0/322455623 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55b9943fac90 0x55b9943fd070 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.042+0000 7f1713d3e700 1 -- 172.21.15.12:0/322455623 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(1 2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.3-myfoo3 [read 0~850 out=850b] v0'0 uv12 ondisk = 0) v8 ==== 208+0+850 (crc 0 0 0) 0x7f170c0173e0 con 0x55b9943fac90 2023-10-05T19:40:30.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.042+0000 7f1716209e40 10 read 850 2023-10-05T19:40:30.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.042+0000 7f1716209e40 10 log_show_next pos 850 bl 850 off 419 eof 1 2023-10-05T19:40:30.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.042+0000 7f1716209e40 10 log_show_next pos 850 bl 850 off 850 eof 1 2023-10-05T19:40:30.053 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:40:30.053 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket_id": "93ac49f4-ff49-4f99-811b-6453b28decd5.4221.3", 2023-10-05T19:40:30.053 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket_owner": "foo", 2023-10-05T19:40:30.053 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo3", 2023-10-05T19:40:30.054 INFO:teuthology.orchestra.run.smithi012.stdout: "log_entries": [ 2023-10-05T19:40:30.054 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:30.054 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo3", 2023-10-05T19:40:30.054 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:40:23.355112Z", 2023-10-05T19:40:30.054 INFO:teuthology.orchestra.run.smithi012.stdout: "time_local": "2023-10-05T19:40:23.355112+0000", 2023-10-05T19:40:30.054 INFO:teuthology.orchestra.run.smithi012.stdout: "remote_addr": "172.21.0.51", 2023-10-05T19:40:30.054 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:40:30.054 INFO:teuthology.orchestra.run.smithi012.stdout: "operation": "create_bucket", 2023-10-05T19:40:30.054 INFO:teuthology.orchestra.run.smithi012.stdout: "uri": "PUT /myfoo3/ HTTP/1.1", 2023-10-05T19:40:30.054 INFO:teuthology.orchestra.run.smithi012.stdout: "http_status": "200", 2023-10-05T19:40:30.054 INFO:teuthology.orchestra.run.smithi012.stdout: "error_code": "", 2023-10-05T19:40:30.055 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:40:30.055 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:40:30.055 INFO:teuthology.orchestra.run.smithi012.stdout: "object_size": 0, 2023-10-05T19:40:30.055 INFO:teuthology.orchestra.run.smithi012.stdout: "total_time": 19, 2023-10-05T19:40:30.055 INFO:teuthology.orchestra.run.smithi012.stdout: "user_agent": "Boto/2.49.0 Python/3.8.10 Linux/5.4.0-163-generic", 2023-10-05T19:40:30.055 INFO:teuthology.orchestra.run.smithi012.stdout: "referrer": "", 2023-10-05T19:40:30.055 INFO:teuthology.orchestra.run.smithi012.stdout: "trans_id": "tx0000060f3df2951b2288a-00651f1127-107b-default", 2023-10-05T19:40:30.055 INFO:teuthology.orchestra.run.smithi012.stdout: "authentication_type": "Local", 2023-10-05T19:40:30.055 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key_id": "9te6NH5mcdcq0Tc5i8i1", 2023-10-05T19:40:30.055 INFO:teuthology.orchestra.run.smithi012.stdout: "temp_url": false 2023-10-05T19:40:30.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:30.055 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:30.056 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo3", 2023-10-05T19:40:30.056 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:40:23.523108Z", 2023-10-05T19:40:30.056 INFO:teuthology.orchestra.run.smithi012.stdout: "time_local": "2023-10-05T19:40:23.523108+0000", 2023-10-05T19:40:30.056 INFO:teuthology.orchestra.run.smithi012.stdout: "remote_addr": "172.21.0.51", 2023-10-05T19:40:30.056 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:40:30.056 INFO:teuthology.orchestra.run.smithi012.stdout: "operation": "delete_bucket", 2023-10-05T19:40:30.056 INFO:teuthology.orchestra.run.smithi012.stdout: "uri": "DELETE /myfoo3/ HTTP/1.1", 2023-10-05T19:40:30.056 INFO:teuthology.orchestra.run.smithi012.stdout: "http_status": "204", 2023-10-05T19:40:30.056 INFO:teuthology.orchestra.run.smithi012.stdout: "error_code": "NoContent", 2023-10-05T19:40:30.056 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:40:30.056 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:40:30.056 INFO:teuthology.orchestra.run.smithi012.stdout: "object_size": 0, 2023-10-05T19:40:30.057 INFO:teuthology.orchestra.run.smithi012.stdout: "total_time": 75, 2023-10-05T19:40:30.057 INFO:teuthology.orchestra.run.smithi012.stdout: "user_agent": "Boto/2.49.0 Python/3.8.10 Linux/5.4.0-163-generic", 2023-10-05T19:40:30.057 INFO:teuthology.orchestra.run.smithi012.stdout: "referrer": "", 2023-10-05T19:40:30.057 INFO:teuthology.orchestra.run.smithi012.stdout: "trans_id": "tx00000f276cf93629445fa-00651f1127-107b-default", 2023-10-05T19:40:30.057 INFO:teuthology.orchestra.run.smithi012.stdout: "authentication_type": "Local", 2023-10-05T19:40:30.057 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key_id": "9te6NH5mcdcq0Tc5i8i1", 2023-10-05T19:40:30.057 INFO:teuthology.orchestra.run.smithi012.stdout: "temp_url": false 2023-10-05T19:40:30.057 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:30.057 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:30.057 INFO:teuthology.orchestra.run.smithi012.stdout: "log_sum": { 2023-10-05T19:40:30.057 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:40:30.057 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:40:30.058 INFO:teuthology.orchestra.run.smithi012.stdout: "total_time": 94, 2023-10-05T19:40:30.058 INFO:teuthology.orchestra.run.smithi012.stdout: "total_entries": 2 2023-10-05T19:40:30.058 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:30.058 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:40:30.058 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:40:30.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.042+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:102 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94255543983232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b9943ba2c0 con 0x55b99436fbb0 2023-10-05T19:40:30.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.046+0000 7f1713d3e700 1 -- 172.21.15.12:0/3229436812 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(102 notify.0 [watch unwatch cookie 94255543983232] v28'167 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f170c0173e0 con 0x55b99436fbb0 2023-10-05T19:40:30.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.046+0000 7f1716209e40 20 remove_watcher() i=0 2023-10-05T19:40:30.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.046+0000 7f1716209e40 2 removed watcher, disabling cache 2023-10-05T19:40:30.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.046+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94255543985520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994379c80 con 0x55b99433e3f0 2023-10-05T19:40:30.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.050+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(103 notify.1 [watch unwatch cookie 94255543985520] v28'167 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1704014040 con 0x55b99433e3f0 2023-10-05T19:40:30.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.050+0000 7f1716209e40 20 remove_watcher() i=1 2023-10-05T19:40:30.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.050+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:104 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94255543992544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994379c80 con 0x55b994374530 2023-10-05T19:40:30.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.050+0000 7f171353d700 1 -- 172.21.15.12:0/3229436812 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(104 notify.2 [watch unwatch cookie 94255543992544] v28'167 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f16fc04a020 con 0x55b994374530 2023-10-05T19:40:30.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.050+0000 7f1716209e40 20 remove_watcher() i=2 2023-10-05T19:40:30.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.050+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:105 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94255543998272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994379c80 con 0x55b99436fbb0 2023-10-05T19:40:30.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.054+0000 7f1713d3e700 1 -- 172.21.15.12:0/3229436812 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(105 notify.3 [watch unwatch cookie 94255543998272] v28'167 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f170c0173e0 con 0x55b99436fbb0 2023-10-05T19:40:30.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.054+0000 7f1716209e40 20 remove_watcher() i=3 2023-10-05T19:40:30.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.054+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:106 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94255543999376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b994379c80 con 0x55b99433e3f0 2023-10-05T19:40:30.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.058+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(106 notify.4 [watch unwatch cookie 94255543999376] v28'167 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1704014040 con 0x55b99433e3f0 2023-10-05T19:40:30.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.058+0000 7f1716209e40 20 remove_watcher() i=4 2023-10-05T19:40:30.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.058+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:107 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94255544003456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b99437dc30 con 0x7f170c016b60 2023-10-05T19:40:30.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.062+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(107 notify.5 [watch unwatch cookie 94255544003456] v28'167 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1704014040 con 0x7f170c016b60 2023-10-05T19:40:30.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.062+0000 7f1716209e40 20 remove_watcher() i=5 2023-10-05T19:40:30.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.062+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:108 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94255543989696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b99437dc30 con 0x7f170c016b60 2023-10-05T19:40:30.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.062+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(108 notify.6 [watch unwatch cookie 94255543989696] v28'167 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1704014040 con 0x7f170c016b60 2023-10-05T19:40:30.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.062+0000 7f1716209e40 20 remove_watcher() i=6 2023-10-05T19:40:30.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.062+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:109 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94255544009632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b99437dc30 con 0x7f170c016b60 2023-10-05T19:40:30.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.066+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(109 notify.7 [watch unwatch cookie 94255544009632] v28'167 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1704014040 con 0x7f170c016b60 2023-10-05T19:40:30.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.066+0000 7f1716209e40 20 remove_watcher() i=7 2023-10-05T19:40:30.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.070+0000 7f16e57e2700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:30.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.070+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:110 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94255544007280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b99437dc30 con 0x55b99433e3f0 2023-10-05T19:40:30.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.070+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(110 data_loggenerations_metadata [watch unwatch cookie 94255544007280] v28'158 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f1704014040 con 0x55b99433e3f0 2023-10-05T19:40:30.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.074+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f170c016b60 msgr2=0x7f170c0243d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.074+0000 7f1716209e40 1 --2- 172.21.15.12:0/3229436812 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f170c016b60 0x7f170c0243d0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.074+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55b99433e3f0 msgr2=0x55b99436da40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.074+0000 7f1716209e40 1 --2- 172.21.15.12:0/3229436812 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55b99433e3f0 0x55b99436da40 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.074+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55b99436fbb0 msgr2=0x55b994371f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.074+0000 7f1716209e40 1 --2- 172.21.15.12:0/3229436812 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55b99436fbb0 0x55b994371f90 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.074+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55b994374530 msgr2=0x55b994376910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.074+0000 7f1716209e40 1 --2- 172.21.15.12:0/3229436812 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55b994374530 0x55b994376910 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.074+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f16f4030c00 msgr2=0x7f16f40330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.074+0000 7f1716209e40 1 --2- 172.21.15.12:0/3229436812 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f16f4030c00 0x7f16f40330b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f16fc005d80 tx=0x7f16fc014000 comp rx=0 tx=0).stop 2023-10-05T19:40:30.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.074+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b994339e10 msgr2=0x55b994361fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.074+0000 7f171453f700 1 -- 172.21.15.12:0/3229436812 reap_dead start 2023-10-05T19:40:30.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.074+0000 7f1716209e40 1 --2- 172.21.15.12:0/3229436812 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b994339e10 0x55b994361fb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f170c001820 tx=0x7f170c016000 comp rx=0 tx=0).stop 2023-10-05T19:40:30.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.074+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 shutdown_connections 2023-10-05T19:40:30.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.074+0000 7f1716209e40 1 --2- 172.21.15.12:0/3229436812 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b994339e10 0x55b994361fb0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.074+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 >> 172.21.15.12:0/3229436812 conn(0x55b994335830 msgr2=0x55b994353620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:30.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.074+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 shutdown_connections 2023-10-05T19:40:30.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.074+0000 7f1716209e40 1 -- 172.21.15.12:0/3229436812 wait complete. 2023-10-05T19:40:30.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.074+0000 7f1716209e40 1 -- 172.21.15.12:0/322455623 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55b9943fac90 msgr2=0x55b9943fd070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.074+0000 7f1716209e40 1 --2- 172.21.15.12:0/322455623 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55b9943fac90 0x55b9943fd070 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.074+0000 7f1716209e40 1 -- 172.21.15.12:0/322455623 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1654030c50 msgr2=0x7f1654033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.074+0000 7f1716209e40 1 --2- 172.21.15.12:0/322455623 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1654030c50 0x7f1654033100 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x55b99437f1d0 tx=0x7f170400ab60 comp rx=0 tx=0).stop 2023-10-05T19:40:30.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.074+0000 7f1716209e40 1 -- 172.21.15.12:0/322455623 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b99438daf0 msgr2=0x55b994393600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.074+0000 7f1716209e40 1 --2- 172.21.15.12:0/322455623 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b99438daf0 0x55b994393600 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f16fc00fb60 tx=0x7f16fc044000 comp rx=0 tx=0).stop 2023-10-05T19:40:30.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.074+0000 7f1716209e40 1 -- 172.21.15.12:0/322455623 shutdown_connections 2023-10-05T19:40:30.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.074+0000 7f1716209e40 1 --2- 172.21.15.12:0/322455623 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55b9943fac90 0x55b9943fd070 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.074+0000 7f1716209e40 1 --2- 172.21.15.12:0/322455623 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1654030c50 0x7f1654033100 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.074+0000 7f1716209e40 1 --2- 172.21.15.12:0/322455623 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b99438daf0 0x55b994393600 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.074+0000 7f1716209e40 1 -- 172.21.15.12:0/322455623 >> 172.21.15.12:0/322455623 conn(0x55b994388eb0 msgr2=0x55b994386fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:30.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.074+0000 7f1716209e40 1 -- 172.21.15.12:0/322455623 shutdown_connections 2023-10-05T19:40:30.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.074+0000 7f1716209e40 1 -- 172.21.15.12:0/322455623 wait complete. 2023-10-05T19:40:30.089 DEBUG:tasks.util.rgw: json result: {'bucket_id': '93ac49f4-ff49-4f99-811b-6453b28decd5.4221.3', 'bucket_owner': 'foo', 'bucket': 'myfoo3', 'log_entries': [{'bucket': 'myfoo3', 'time': '2023-10-05T19:40:23.355112Z', 'time_local': '2023-10-05T19:40:23.355112+0000', 'remote_addr': '172.21.0.51', 'user': 'foo', 'operation': 'create_bucket', 'uri': 'PUT /myfoo3/ HTTP/1.1', 'http_status': '200', 'error_code': '', 'bytes_sent': 0, 'bytes_received': 0, 'object_size': 0, 'total_time': 19, 'user_agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-163-generic', 'referrer': '', 'trans_id': 'tx0000060f3df2951b2288a-00651f1127-107b-default', 'authentication_type': 'Local', 'access_key_id': '9te6NH5mcdcq0Tc5i8i1', 'temp_url': False}, {'bucket': 'myfoo3', 'time': '2023-10-05T19:40:23.523108Z', 'time_local': '2023-10-05T19:40:23.523108+0000', 'remote_addr': '172.21.0.51', 'user': 'foo', 'operation': 'delete_bucket', 'uri': 'DELETE /myfoo3/ HTTP/1.1', 'http_status': '204', 'error_code': 'NoContent', 'bytes_sent': 0, 'bytes_received': 0, 'object_size': 0, 'total_time': 75, 'user_agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-163-generic', 'referrer': '', 'trans_id': 'tx00000f276cf93629445fa-00651f1127-107b-default', 'authentication_type': 'Local', 'access_key_id': '9te6NH5mcdcq0Tc5i8i1', 'temp_url': False}], 'log_sum': {'bytes_sent': 0, 'bytes_received': 0, 'total_time': 94, 'total_entries': 2}} 2023-10-05T19:40:30.089 DEBUG:tasks.radosgw_admin:checking log entry: 2023-10-05T19:40:30.089 DEBUG:tasks.radosgw_admin:checking log entry: 2023-10-05T19:40:30.089 INFO:tasks.util.rgw:rgwadmin: client.0 : ['log', 'rm', '--object', '2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.3-myfoo3'] 2023-10-05T19:40:30.090 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'log', 'rm', '--object', '2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.3-myfoo3'] 2023-10-05T19:40:30.090 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph log rm --object 2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.3-myfoo3 2023-10-05T19:40:30.129 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:30.129 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:30.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.118+0000 7f920f279e40 1 Processor -- start 2023-10-05T19:40:30.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.118+0000 7f920f279e40 1 -- start start 2023-10-05T19:40:30.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.122+0000 7f920f279e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55de7cc1a410 0x55de7cc1c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.122+0000 7f920f279e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55de7cb62bd0 con 0x55de7cc15e30 2023-10-05T19:40:30.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.122+0000 7f920f279e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x55de7cb1fee0 con 0x55de7cc1a410 2023-10-05T19:40:30.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.122+0000 7f920c5ad700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55de7cc1a410 0x55de7cc1c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.122+0000 7f920cdae700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x55de7cc15e30 0x55de7cc16200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:35718/0 (socket says 172.21.15.12:35718) 2023-10-05T19:40:30.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.122+0000 7f920cdae700 1 -- 172.21.15.12:0/1377778027 learned_addr learned my addr 172.21.15.12:0/1377778027 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:30.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.122+0000 7f920c5ad700 1 -- 172.21.15.12:0/1377778027 >> v1:172.21.15.8:6789/0 conn(0x55de7cc15e30 legacy=0x55de7cc16200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.122+0000 7f920c5ad700 1 -- 172.21.15.12:0/1377778027 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de7cc16a40 con 0x55de7cc1a410 2023-10-05T19:40:30.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.122+0000 7f920c5ad700 1 --2- 172.21.15.12:0/1377778027 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de7cc1a410 0x55de7cc1c7f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f91f400a5d0 tx=0x7f91f4005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c5b422605736ad server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.122+0000 7f920bdac700 1 -- 172.21.15.12:0/1377778027 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91f40086a0 con 0x55de7cc1a410 2023-10-05T19:40:30.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.122+0000 7f920bdac700 1 -- 172.21.15.12:0/1377778027 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f4008800 con 0x55de7cc1a410 2023-10-05T19:40:30.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.122+0000 7f920bdac700 1 -- 172.21.15.12:0/1377778027 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91f4008ad0 con 0x55de7cc1a410 2023-10-05T19:40:30.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.122+0000 7f920f279e40 1 -- 172.21.15.12:0/1377778027 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de7cc1a410 msgr2=0x55de7cc1c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.122+0000 7f920f279e40 1 --2- 172.21.15.12:0/1377778027 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de7cc1a410 0x55de7cc1c7f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f91f400a5d0 tx=0x7f91f4005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:30.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.122+0000 7f920f279e40 1 -- 172.21.15.12:0/1377778027 shutdown_connections 2023-10-05T19:40:30.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.122+0000 7f920f279e40 1 --2- 172.21.15.12:0/1377778027 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de7cc1a410 0x55de7cc1c7f0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.122+0000 7f920f279e40 1 -- 172.21.15.12:0/1377778027 >> 172.21.15.12:0/1377778027 conn(0x55de7cc11850 msgr2=0x55de7cc13c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:30.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.122+0000 7f920f279e40 1 -- 172.21.15.12:0/1377778027 shutdown_connections 2023-10-05T19:40:30.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.122+0000 7f920f279e40 1 -- 172.21.15.12:0/1377778027 wait complete. 2023-10-05T19:40:30.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.122+0000 7f920f279e40 1 Processor -- start 2023-10-05T19:40:30.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.126+0000 7f920f279e40 1 -- start start 2023-10-05T19:40:30.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.126+0000 7f920f279e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de7cc15e30 0x55de7cc34ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.126+0000 7f920f279e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de7cae3e30 con 0x55de7cc15e30 2023-10-05T19:40:30.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.126+0000 7f920cdae700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de7cc15e30 0x55de7cc34ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.126+0000 7f920cdae700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de7cc15e30 0x55de7cc34ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58738/0 (socket says 172.21.15.12:58738) 2023-10-05T19:40:30.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.126+0000 7f920cdae700 1 -- 172.21.15.12:0/3731803082 learned_addr learned my addr 172.21.15.12:0/3731803082 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:30.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.126+0000 7f920cdae700 1 -- 172.21.15.12:0/3731803082 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f4005cf0 con 0x55de7cc15e30 2023-10-05T19:40:30.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.126+0000 7f920cdae700 1 --2- 172.21.15.12:0/3731803082 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de7cc15e30 0x55de7cc34ff0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f91fc0014a0 tx=0x7f91fc006b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=260f611d5f2493b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.126+0000 7f920adaa700 1 -- 172.21.15.12:0/3731803082 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91fc006110 con 0x55de7cc15e30 2023-10-05T19:40:30.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.126+0000 7f920adaa700 1 -- 172.21.15.12:0/3731803082 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91fc006270 con 0x55de7cc15e30 2023-10-05T19:40:30.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.126+0000 7f920adaa700 1 -- 172.21.15.12:0/3731803082 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91fc004520 con 0x55de7cc15e30 2023-10-05T19:40:30.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.126+0000 7f920f279e40 1 -- 172.21.15.12:0/3731803082 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de7cc15e30 msgr2=0x55de7cc34ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.126+0000 7f920f279e40 1 --2- 172.21.15.12:0/3731803082 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de7cc15e30 0x55de7cc34ff0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f91fc0014a0 tx=0x7f91fc006b90 comp rx=0 tx=0).stop 2023-10-05T19:40:30.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.126+0000 7f920f279e40 1 -- 172.21.15.12:0/3731803082 shutdown_connections 2023-10-05T19:40:30.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.126+0000 7f920f279e40 1 --2- 172.21.15.12:0/3731803082 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de7cc15e30 0x55de7cc34ff0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.126+0000 7f920f279e40 1 -- 172.21.15.12:0/3731803082 >> 172.21.15.12:0/3731803082 conn(0x55de7cc11850 msgr2=0x55de7cc30bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:30.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.126+0000 7f920f279e40 1 -- 172.21.15.12:0/3731803082 shutdown_connections 2023-10-05T19:40:30.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.126+0000 7f920f279e40 1 -- 172.21.15.12:0/3731803082 wait complete. 2023-10-05T19:40:30.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.130+0000 7f920f279e40 1 Processor -- start 2023-10-05T19:40:30.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.130+0000 7f920f279e40 1 -- start start 2023-10-05T19:40:30.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.130+0000 7f920f279e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de7cc15e30 0x55de7cc3bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.130+0000 7f920f279e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de7cab6ed0 con 0x55de7cc15e30 2023-10-05T19:40:30.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.130+0000 7f920cdae700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de7cc15e30 0x55de7cc3bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.130+0000 7f920cdae700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de7cc15e30 0x55de7cc3bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58742/0 (socket says 172.21.15.12:58742) 2023-10-05T19:40:30.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.130+0000 7f920cdae700 1 -- 172.21.15.12:0/162702257 learned_addr learned my addr 172.21.15.12:0/162702257 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:30.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.130+0000 7f920cdae700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de7cb62bd0 con 0x55de7cc15e30 2023-10-05T19:40:30.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.130+0000 7f920cdae700 1 --2- 172.21.15.12:0/162702257 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de7cc15e30 0x55de7cc3bdc0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f91fc006bd0 tx=0x7f91fc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.130+0000 7f92095a7700 1 -- 172.21.15.12:0/162702257 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91fc006110 con 0x55de7cc15e30 2023-10-05T19:40:30.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.130+0000 7f92095a7700 1 -- 172.21.15.12:0/162702257 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91fc0063e0 con 0x55de7cc15e30 2023-10-05T19:40:30.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.130+0000 7f92095a7700 1 -- 172.21.15.12:0/162702257 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91fc013d00 con 0x55de7cc15e30 2023-10-05T19:40:30.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.130+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55de7cc3c300 con 0x55de7cc15e30 2023-10-05T19:40:30.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.130+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55de7cc3c740 con 0x55de7cc15e30 2023-10-05T19:40:30.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.134+0000 7f92095a7700 1 -- 172.21.15.12:0/162702257 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f91fc004030 con 0x55de7cc15e30 2023-10-05T19:40:30.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.134+0000 7f920f279e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.134+0000 7f92095a7700 1 --2- 172.21.15.12:0/162702257 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f91f0030c00 0x7f91f00330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.134+0000 7f92095a7700 1 -- 172.21.15.12:0/162702257 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f91fc044830 con 0x55de7cc15e30 2023-10-05T19:40:30.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.134+0000 7f920f279e40 1 --2- 172.21.15.12:0/162702257 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55de7cc1a410 0x55de7cc47880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.134+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc47f80 con 0x55de7cc1a410 2023-10-05T19:40:30.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.134+0000 7f920c5ad700 1 --2- 172.21.15.12:0/162702257 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f91f0030c00 0x7f91f00330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.134+0000 7f920d5af700 1 --2- 172.21.15.12:0/162702257 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55de7cc1a410 0x55de7cc47880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.134+0000 7f920d5af700 1 --2- 172.21.15.12:0/162702257 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55de7cc1a410 0x55de7cc47880 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.134+0000 7f920c5ad700 1 --2- 172.21.15.12:0/162702257 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f91f0030c00 0x7f91f00330b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f91f4005a90 tx=0x7f91f4015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.134+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f920400a500 con 0x55de7cc1a410 2023-10-05T19:40:30.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.134+0000 7f920f279e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:30.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.134+0000 7f920f279e40 20 realm 2023-10-05T19:40:30.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.134+0000 7f920f279e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.134+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc44ee0 con 0x55de7cc1a410 2023-10-05T19:40:30.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.138+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f920400a500 con 0x55de7cc1a410 2023-10-05T19:40:30.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.138+0000 7f920f279e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:30.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.138+0000 7f920f279e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:30.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.138+0000 7f920f279e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.138+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc48a60 con 0x55de7cc1a410 2023-10-05T19:40:30.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.138+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f920400a500 con 0x55de7cc1a410 2023-10-05T19:40:30.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.138+0000 7f920f279e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:30.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.138+0000 7f920f279e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.138+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc48670 con 0x55de7cc1a410 2023-10-05T19:40:30.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.138+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f920400a500 con 0x55de7cc1a410 2023-10-05T19:40:30.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.138+0000 7f920f279e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:30.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.138+0000 7f920f279e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.138+0000 7f920f279e40 1 --2- 172.21.15.12:0/162702257 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55de7cc49e00 0x55de7cc4c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.138+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc4c720 con 0x55de7cc49e00 2023-10-05T19:40:30.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.138+0000 7f920cdae700 1 --2- 172.21.15.12:0/162702257 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55de7cc49e00 0x55de7cc4c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.138+0000 7f920cdae700 1 --2- 172.21.15.12:0/162702257 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55de7cc49e00 0x55de7cc4c1e0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.138+0000 7f920cdae700 1 -- 172.21.15.12:0/162702257 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f91fc003a00 con 0x55de7cc49e00 2023-10-05T19:40:30.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.138+0000 7f920f279e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:30.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.138+0000 7f920f279e40 20 searching for the correct realm 2023-10-05T19:40:30.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.138+0000 7f920f279e40 1 --2- 172.21.15.12:0/162702257 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55de7cc4e5a0 0x55de7cc509b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.138+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51060 con 0x55de7cc4e5a0 2023-10-05T19:40:30.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.142+0000 7f920c5ad700 1 --2- 172.21.15.12:0/162702257 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55de7cc4e5a0 0x55de7cc509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.142+0000 7f920c5ad700 1 --2- 172.21.15.12:0/162702257 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55de7cc4e5a0 0x55de7cc509b0 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.142+0000 7f920c5ad700 1 -- 172.21.15.12:0/162702257 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f91f4008a80 con 0x55de7cc4e5a0 2023-10-05T19:40:30.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.142+0000 7f920c5ad700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51060 con 0x55de7cc49e00 2023-10-05T19:40:30.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.142+0000 7f920cdae700 1 -- 172.21.15.12:0/162702257 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f91fc02f040 con 0x55de7cc49e00 2023-10-05T19:40:30.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.142+0000 7f920cdae700 1 --2- 172.21.15.12:0/162702257 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f91fc011c00 0x7f91fc03d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.142+0000 7f920cdae700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51060 con 0x7f91fc011c00 2023-10-05T19:40:30.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.142+0000 7f920d5af700 1 --2- 172.21.15.12:0/162702257 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f91fc011c00 0x7f91fc03d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.146+0000 7f920d5af700 1 --2- 172.21.15.12:0/162702257 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f91fc011c00 0x7f91fc03d100 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.146+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9204014040 con 0x7f91fc011c00 2023-10-05T19:40:30.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.146+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51060 con 0x55de7cc1a410 2023-10-05T19:40:30.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.146+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9204014040 con 0x55de7cc1a410 2023-10-05T19:40:30.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.146+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51060 con 0x55de7cc1a410 2023-10-05T19:40:30.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.146+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9204014040 con 0x55de7cc1a410 2023-10-05T19:40:30.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.146+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51060 con 0x7f91fc011c00 2023-10-05T19:40:30.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.146+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9204014040 con 0x7f91fc011c00 2023-10-05T19:40:30.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.146+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51060 con 0x7f91fc011c00 2023-10-05T19:40:30.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.150+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9204014040 con 0x7f91fc011c00 2023-10-05T19:40:30.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.150+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51060 con 0x55de7cc49e00 2023-10-05T19:40:30.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.150+0000 7f920cdae700 1 -- 172.21.15.12:0/162702257 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f91fc02f040 con 0x55de7cc49e00 2023-10-05T19:40:30.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.150+0000 7f920cdae700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51060 con 0x55de7cc4e5a0 2023-10-05T19:40:30.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.150+0000 7f920c5ad700 1 -- 172.21.15.12:0/162702257 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f91f4008a80 con 0x55de7cc4e5a0 2023-10-05T19:40:30.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.150+0000 7f920c5ad700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51060 con 0x55de7cc1a410 2023-10-05T19:40:30.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.150+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9204014040 con 0x55de7cc1a410 2023-10-05T19:40:30.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.150+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51060 con 0x7f91fc011c00 2023-10-05T19:40:30.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.154+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9204014040 con 0x7f91fc011c00 2023-10-05T19:40:30.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.154+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51060 con 0x55de7cc1a410 2023-10-05T19:40:30.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.154+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9204014040 con 0x55de7cc1a410 2023-10-05T19:40:30.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.154+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51060 con 0x55de7cc49e00 2023-10-05T19:40:30.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.154+0000 7f920cdae700 1 -- 172.21.15.12:0/162702257 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f91fc02f040 con 0x55de7cc49e00 2023-10-05T19:40:30.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.154+0000 7f920cdae700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51060 con 0x55de7cc4e5a0 2023-10-05T19:40:30.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.154+0000 7f920c5ad700 1 -- 172.21.15.12:0/162702257 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f91f4008a80 con 0x55de7cc4e5a0 2023-10-05T19:40:30.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.154+0000 7f920c5ad700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51060 con 0x55de7cc4e5a0 2023-10-05T19:40:30.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.154+0000 7f920c5ad700 1 -- 172.21.15.12:0/162702257 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f91f4008a80 con 0x55de7cc4e5a0 2023-10-05T19:40:30.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.154+0000 7f920c5ad700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51060 con 0x7f91fc011c00 2023-10-05T19:40:30.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.158+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9204014040 con 0x7f91fc011c00 2023-10-05T19:40:30.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.158+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51060 con 0x55de7cc4e5a0 2023-10-05T19:40:30.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.158+0000 7f920c5ad700 1 -- 172.21.15.12:0/162702257 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f91f4008a80 con 0x55de7cc4e5a0 2023-10-05T19:40:30.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.158+0000 7f920c5ad700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51060 con 0x55de7cc4e5a0 2023-10-05T19:40:30.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.158+0000 7f920c5ad700 1 -- 172.21.15.12:0/162702257 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f91f4008a80 con 0x55de7cc4e5a0 2023-10-05T19:40:30.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.158+0000 7f920c5ad700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51060 con 0x55de7cc1a410 2023-10-05T19:40:30.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.158+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9204014040 con 0x55de7cc1a410 2023-10-05T19:40:30.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.158+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51060 con 0x55de7cc1a410 2023-10-05T19:40:30.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.162+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9204014040 con 0x55de7cc1a410 2023-10-05T19:40:30.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.162+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51060 con 0x7f91fc011c00 2023-10-05T19:40:30.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.162+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9204014040 con 0x7f91fc011c00 2023-10-05T19:40:30.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.162+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51060 con 0x55de7cc4e5a0 2023-10-05T19:40:30.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.162+0000 7f920c5ad700 1 -- 172.21.15.12:0/162702257 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f91f4008a80 con 0x55de7cc4e5a0 2023-10-05T19:40:30.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.162+0000 7f920c5ad700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51060 con 0x55de7cc49e00 2023-10-05T19:40:30.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.162+0000 7f920cdae700 1 -- 172.21.15.12:0/162702257 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f91fc02f040 con 0x55de7cc49e00 2023-10-05T19:40:30.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.162+0000 7f920cdae700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51060 con 0x55de7cc1a410 2023-10-05T19:40:30.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.162+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9204014040 con 0x55de7cc1a410 2023-10-05T19:40:30.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.162+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51060 con 0x7f91fc011c00 2023-10-05T19:40:30.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.166+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9204014040 con 0x7f91fc011c00 2023-10-05T19:40:30.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.166+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51060 con 0x55de7cc4e5a0 2023-10-05T19:40:30.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.166+0000 7f920c5ad700 1 -- 172.21.15.12:0/162702257 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f91f4008a80 con 0x55de7cc4e5a0 2023-10-05T19:40:30.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.166+0000 7f920c5ad700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51060 con 0x7f91fc011c00 2023-10-05T19:40:30.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.166+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9204014040 con 0x7f91fc011c00 2023-10-05T19:40:30.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.166+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51060 con 0x55de7cc49e00 2023-10-05T19:40:30.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.166+0000 7f920cdae700 1 -- 172.21.15.12:0/162702257 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f91fc02f040 con 0x55de7cc49e00 2023-10-05T19:40:30.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.166+0000 7f920cdae700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51060 con 0x55de7cc4e5a0 2023-10-05T19:40:30.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.170+0000 7f920c5ad700 1 -- 172.21.15.12:0/162702257 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f91f4008a80 con 0x55de7cc4e5a0 2023-10-05T19:40:30.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.170+0000 7f920c5ad700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51060 con 0x7f91fc011c00 2023-10-05T19:40:30.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.170+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9204014040 con 0x7f91fc011c00 2023-10-05T19:40:30.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.170+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51060 con 0x55de7cc1a410 2023-10-05T19:40:30.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.170+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f9204014040 con 0x55de7cc1a410 2023-10-05T19:40:30.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.170+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51060 con 0x55de7cc49e00 2023-10-05T19:40:30.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.170+0000 7f920cdae700 1 -- 172.21.15.12:0/162702257 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f91fc02f040 con 0x55de7cc49e00 2023-10-05T19:40:30.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.170+0000 7f920f279e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:30.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.170+0000 7f920f279e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:30.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.170+0000 7f920f279e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:30.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.170+0000 7f920f279e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:30.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.170+0000 7f920f279e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.170+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc4d320 con 0x55de7cc1a410 2023-10-05T19:40:30.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.174+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9204014040 con 0x55de7cc1a410 2023-10-05T19:40:30.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.174+0000 7f920f279e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:30.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.174+0000 7f920f279e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.174+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc4cf50 con 0x55de7cc1a410 2023-10-05T19:40:30.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.174+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f9204014040 con 0x55de7cc1a410 2023-10-05T19:40:30.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.174+0000 7f920f279e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:30.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.174+0000 7f920f279e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.174+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc514d0 con 0x55de7cc4e5a0 2023-10-05T19:40:30.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.174+0000 7f920c5ad700 1 -- 172.21.15.12:0/162702257 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f91f4008a80 con 0x55de7cc4e5a0 2023-10-05T19:40:30.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.174+0000 7f920f279e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:30.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.174+0000 7f920f279e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.174+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51880 con 0x55de7cc4e5a0 2023-10-05T19:40:30.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.174+0000 7f920c5ad700 1 -- 172.21.15.12:0/162702257 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f91f4008a80 con 0x55de7cc4e5a0 2023-10-05T19:40:30.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.174+0000 7f920f279e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:30.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.174+0000 7f920f279e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.174+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51c30 con 0x55de7cc1a410 2023-10-05T19:40:30.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.178+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9204014040 con 0x55de7cc1a410 2023-10-05T19:40:30.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.178+0000 7f920f279e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:30.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.178+0000 7f920f279e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.178+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc52040 con 0x55de7cc1a410 2023-10-05T19:40:30.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.178+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f9204014040 con 0x55de7cc1a410 2023-10-05T19:40:30.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.178+0000 7f920f279e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:30.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.178+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc52010 con 0x55de7cc4e5a0 2023-10-05T19:40:30.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.178+0000 7f920c5ad700 1 -- 172.21.15.12:0/162702257 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f91f4008a80 con 0x55de7cc4e5a0 2023-10-05T19:40:30.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.182+0000 7f920c5ad700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc52010 con 0x55de7cc49e00 2023-10-05T19:40:30.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.182+0000 7f920cdae700 1 -- 172.21.15.12:0/162702257 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f91fc02f040 con 0x55de7cc49e00 2023-10-05T19:40:30.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.182+0000 7f920cdae700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc52010 con 0x7f91fc011c00 2023-10-05T19:40:30.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.182+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9204014040 con 0x7f91fc011c00 2023-10-05T19:40:30.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.182+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51c30 con 0x55de7cc1a410 2023-10-05T19:40:30.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.182+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9204014040 con 0x55de7cc1a410 2023-10-05T19:40:30.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.182+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51c30 con 0x55de7cc1a410 2023-10-05T19:40:30.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.182+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9204014040 con 0x55de7cc1a410 2023-10-05T19:40:30.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.182+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51c30 con 0x7f91fc011c00 2023-10-05T19:40:30.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.186+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9204014040 con 0x7f91fc011c00 2023-10-05T19:40:30.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.186+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51c30 con 0x7f91fc011c00 2023-10-05T19:40:30.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.186+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9204014040 con 0x7f91fc011c00 2023-10-05T19:40:30.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.186+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51c30 con 0x55de7cc49e00 2023-10-05T19:40:30.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.186+0000 7f920cdae700 1 -- 172.21.15.12:0/162702257 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f91fc02f040 con 0x55de7cc49e00 2023-10-05T19:40:30.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.186+0000 7f920cdae700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51c30 con 0x55de7cc4e5a0 2023-10-05T19:40:30.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.186+0000 7f920c5ad700 1 -- 172.21.15.12:0/162702257 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f91f4008a80 con 0x55de7cc4e5a0 2023-10-05T19:40:30.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.186+0000 7f920c5ad700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51c30 con 0x55de7cc1a410 2023-10-05T19:40:30.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.190+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9204014040 con 0x55de7cc1a410 2023-10-05T19:40:30.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.190+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51c30 con 0x7f91fc011c00 2023-10-05T19:40:30.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.190+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9204014040 con 0x7f91fc011c00 2023-10-05T19:40:30.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.190+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51c30 con 0x55de7cc1a410 2023-10-05T19:40:30.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.190+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9204014040 con 0x55de7cc1a410 2023-10-05T19:40:30.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.190+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51c30 con 0x55de7cc49e00 2023-10-05T19:40:30.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.190+0000 7f920cdae700 1 -- 172.21.15.12:0/162702257 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f91fc02f040 con 0x55de7cc49e00 2023-10-05T19:40:30.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.190+0000 7f920cdae700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51c30 con 0x55de7cc4e5a0 2023-10-05T19:40:30.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.190+0000 7f920c5ad700 1 -- 172.21.15.12:0/162702257 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f91f4008a80 con 0x55de7cc4e5a0 2023-10-05T19:40:30.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.194+0000 7f920c5ad700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51c30 con 0x55de7cc4e5a0 2023-10-05T19:40:30.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.194+0000 7f920c5ad700 1 -- 172.21.15.12:0/162702257 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f91f4008a80 con 0x55de7cc4e5a0 2023-10-05T19:40:30.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.194+0000 7f920c5ad700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51c30 con 0x7f91fc011c00 2023-10-05T19:40:30.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.194+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9204014040 con 0x7f91fc011c00 2023-10-05T19:40:30.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.194+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51c30 con 0x55de7cc4e5a0 2023-10-05T19:40:30.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.194+0000 7f920c5ad700 1 -- 172.21.15.12:0/162702257 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f91f4008a80 con 0x55de7cc4e5a0 2023-10-05T19:40:30.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.194+0000 7f920c5ad700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51c30 con 0x55de7cc4e5a0 2023-10-05T19:40:30.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.194+0000 7f920c5ad700 1 -- 172.21.15.12:0/162702257 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f91f4008a80 con 0x55de7cc4e5a0 2023-10-05T19:40:30.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.194+0000 7f920c5ad700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51c30 con 0x55de7cc1a410 2023-10-05T19:40:30.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.198+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9204014040 con 0x55de7cc1a410 2023-10-05T19:40:30.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.198+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51c30 con 0x55de7cc1a410 2023-10-05T19:40:30.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.198+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9204014040 con 0x55de7cc1a410 2023-10-05T19:40:30.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.198+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51c30 con 0x7f91fc011c00 2023-10-05T19:40:30.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.198+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9204014040 con 0x7f91fc011c00 2023-10-05T19:40:30.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.198+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51c30 con 0x55de7cc4e5a0 2023-10-05T19:40:30.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.198+0000 7f920c5ad700 1 -- 172.21.15.12:0/162702257 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f91f4008a80 con 0x55de7cc4e5a0 2023-10-05T19:40:30.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.198+0000 7f920c5ad700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51c30 con 0x55de7cc49e00 2023-10-05T19:40:30.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.198+0000 7f920cdae700 1 -- 172.21.15.12:0/162702257 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f91fc02f040 con 0x55de7cc49e00 2023-10-05T19:40:30.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.198+0000 7f920cdae700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51c30 con 0x55de7cc1a410 2023-10-05T19:40:30.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.202+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9204014040 con 0x55de7cc1a410 2023-10-05T19:40:30.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.202+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51c30 con 0x7f91fc011c00 2023-10-05T19:40:30.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.202+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9204014040 con 0x7f91fc011c00 2023-10-05T19:40:30.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.202+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51c30 con 0x55de7cc4e5a0 2023-10-05T19:40:30.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.202+0000 7f920c5ad700 1 -- 172.21.15.12:0/162702257 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f91f4008a80 con 0x55de7cc4e5a0 2023-10-05T19:40:30.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.202+0000 7f920c5ad700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51c30 con 0x7f91fc011c00 2023-10-05T19:40:30.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.202+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9204014040 con 0x7f91fc011c00 2023-10-05T19:40:30.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.202+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51c30 con 0x55de7cc49e00 2023-10-05T19:40:30.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.206+0000 7f920cdae700 1 -- 172.21.15.12:0/162702257 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f91fc02f040 con 0x55de7cc49e00 2023-10-05T19:40:30.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.206+0000 7f920cdae700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51c30 con 0x55de7cc4e5a0 2023-10-05T19:40:30.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.206+0000 7f920c5ad700 1 -- 172.21.15.12:0/162702257 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f91f4008a80 con 0x55de7cc4e5a0 2023-10-05T19:40:30.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.206+0000 7f920c5ad700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51c30 con 0x7f91fc011c00 2023-10-05T19:40:30.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.206+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9204014040 con 0x7f91fc011c00 2023-10-05T19:40:30.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.206+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51c30 con 0x55de7cc1a410 2023-10-05T19:40:30.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.206+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f9204014040 con 0x55de7cc1a410 2023-10-05T19:40:30.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.206+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc51c30 con 0x55de7cc49e00 2023-10-05T19:40:30.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.210+0000 7f920cdae700 1 -- 172.21.15.12:0/162702257 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f91fc02f040 con 0x55de7cc49e00 2023-10-05T19:40:30.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.210+0000 7f920f279e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:30.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.210+0000 7f920f279e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:30.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.210+0000 7f920f279e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:30.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.210+0000 7f920f279e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:30.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.210+0000 7f920f279e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.210+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc52010 con 0x55de7cc1a410 2023-10-05T19:40:30.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.210+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f9204014040 con 0x55de7cc1a410 2023-10-05T19:40:30.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.210+0000 7f920f279e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:30.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.210+0000 7f920f279e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.210+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc52470 con 0x55de7cc49e00 2023-10-05T19:40:30.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.210+0000 7f920cdae700 1 -- 172.21.15.12:0/162702257 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f91fc02f040 con 0x55de7cc49e00 2023-10-05T19:40:30.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.210+0000 7f920f279e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:30.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.210+0000 7f920f279e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.210+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc52d60 con 0x55de7cc1a410 2023-10-05T19:40:30.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.210+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f9204014040 con 0x55de7cc1a410 2023-10-05T19:40:30.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.210+0000 7f920f279e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:30.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.210+0000 7f920f279e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.210+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc52d60 con 0x55de7cc4e5a0 2023-10-05T19:40:30.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.210+0000 7f920c5ad700 1 -- 172.21.15.12:0/162702257 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f91f4008a80 con 0x55de7cc4e5a0 2023-10-05T19:40:30.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.210+0000 7f920f279e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:30.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.210+0000 7f920f279e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.210+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc534d0 con 0x7f91fc011c00 2023-10-05T19:40:30.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.214+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f9204014040 con 0x7f91fc011c00 2023-10-05T19:40:30.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.214+0000 7f920f279e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:30.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.214+0000 7f920f279e40 20 zone default found 2023-10-05T19:40:30.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.214+0000 7f920f279e40 4 Realm: () 2023-10-05T19:40:30.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.214+0000 7f920f279e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:30.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.214+0000 7f920f279e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:30.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.214+0000 7f920f279e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:30.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.214+0000 7f920f279e40 20 zonegroup default 2023-10-05T19:40:30.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.214+0000 7f920f279e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.214+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc532d0 con 0x7f91fc011c00 2023-10-05T19:40:30.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.214+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f9204014040 con 0x7f91fc011c00 2023-10-05T19:40:30.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.214+0000 7f920f279e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:30.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.214+0000 7f920f279e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.214+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc54c50 con 0x55de7cc4e5a0 2023-10-05T19:40:30.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.214+0000 7f920c5ad700 1 -- 172.21.15.12:0/162702257 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f91f4008a80 con 0x55de7cc4e5a0 2023-10-05T19:40:30.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.214+0000 7f920f279e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:30.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.214+0000 7f920f279e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.214+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc55000 con 0x55de7cc4e5a0 2023-10-05T19:40:30.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.214+0000 7f920c5ad700 1 -- 172.21.15.12:0/162702257 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f91f4008a80 con 0x55de7cc4e5a0 2023-10-05T19:40:30.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.214+0000 7f920f279e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:30.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.214+0000 7f920f279e40 20 started sync module instance, tier type = 2023-10-05T19:40:30.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.214+0000 7f920f279e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:30.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.214+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc54310 con 0x55de7cc49e00 2023-10-05T19:40:30.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.218+0000 7f920cdae700 1 -- 172.21.15.12:0/162702257 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'168 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f91fc02f040 con 0x55de7cc49e00 2023-10-05T19:40:30.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.218+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94414064404240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc52990 con 0x55de7cc49e00 2023-10-05T19:40:30.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.218+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc55d90 con 0x55de7cc1a410 2023-10-05T19:40:30.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.222+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'168 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9204014040 con 0x55de7cc1a410 2023-10-05T19:40:30.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.222+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94414064409024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc56340 con 0x55de7cc1a410 2023-10-05T19:40:30.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.222+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc56ed0 con 0x55de7cc4e5a0 2023-10-05T19:40:30.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.222+0000 7f920cdae700 1 -- 172.21.15.12:0/162702257 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94414064404240] v28'169 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f91fc02f040 con 0x55de7cc49e00 2023-10-05T19:40:30.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.222+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94414064409024] v28'169 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9204014040 con 0x55de7cc1a410 2023-10-05T19:40:30.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.222+0000 7f920c5ad700 1 -- 172.21.15.12:0/162702257 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'168 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f91f4008a80 con 0x55de7cc4e5a0 2023-10-05T19:40:30.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.226+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94414064407472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc569a0 con 0x55de7cc4e5a0 2023-10-05T19:40:30.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.226+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc577c0 con 0x55de7cc49e00 2023-10-05T19:40:30.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.226+0000 7f920cdae700 1 -- 172.21.15.12:0/162702257 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'168 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f91fc02f040 con 0x55de7cc49e00 2023-10-05T19:40:30.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.226+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94414064415440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc57c90 con 0x55de7cc49e00 2023-10-05T19:40:30.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.226+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc58850 con 0x55de7cc1a410 2023-10-05T19:40:30.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.226+0000 7f920c5ad700 1 -- 172.21.15.12:0/162702257 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94414064407472] v28'169 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f91f4008a80 con 0x55de7cc4e5a0 2023-10-05T19:40:30.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.230+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'168 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9204014040 con 0x55de7cc1a410 2023-10-05T19:40:30.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.230+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94414064410128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc58810 con 0x55de7cc1a410 2023-10-05T19:40:30.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.230+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc592b0 con 0x7f91fc011c00 2023-10-05T19:40:30.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.230+0000 7f920cdae700 1 -- 172.21.15.12:0/162702257 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94414064415440] v28'169 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f91fc02f040 con 0x55de7cc49e00 2023-10-05T19:40:30.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.230+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'168 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9204014040 con 0x7f91fc011c00 2023-10-05T19:40:30.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.230+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94414064416544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc592b0 con 0x7f91fc011c00 2023-10-05T19:40:30.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.230+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc59d10 con 0x7f91fc011c00 2023-10-05T19:40:30.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.234+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94414064410128] v28'169 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9204014040 con 0x55de7cc1a410 2023-10-05T19:40:30.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.234+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(96 notify.6 [create] v28'168 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9204014040 con 0x7f91fc011c00 2023-10-05T19:40:30.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.234+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(95 notify.5 [watch watch cookie 94414064416544] v28'169 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9204014040 con 0x7f91fc011c00 2023-10-05T19:40:30.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.234+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94414064419616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc59d10 con 0x7f91fc011c00 2023-10-05T19:40:30.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.234+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc5a230 con 0x7f91fc011c00 2023-10-05T19:40:30.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.238+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(98 notify.7 [create] v28'168 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9204014040 con 0x7f91fc011c00 2023-10-05T19:40:30.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.238+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94414064422336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc5a720 con 0x7f91fc011c00 2023-10-05T19:40:30.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.238+0000 7f920f279e40 20 add_watcher() i=0 2023-10-05T19:40:30.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.238+0000 7f920f279e40 20 add_watcher() i=1 2023-10-05T19:40:30.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.238+0000 7f920f279e40 20 add_watcher() i=2 2023-10-05T19:40:30.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.238+0000 7f920f279e40 20 add_watcher() i=3 2023-10-05T19:40:30.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.238+0000 7f920f279e40 20 add_watcher() i=4 2023-10-05T19:40:30.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.238+0000 7f920f279e40 20 add_watcher() i=5 2023-10-05T19:40:30.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.238+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(97 notify.6 [watch watch cookie 94414064419616] v28'169 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9204014040 con 0x7f91fc011c00 2023-10-05T19:40:30.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.238+0000 7f920f279e40 20 add_watcher() i=6 2023-10-05T19:40:30.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.238+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94414064422336] v28'169 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9204014040 con 0x7f91fc011c00 2023-10-05T19:40:30.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.238+0000 7f920f279e40 20 add_watcher() i=7 2023-10-05T19:40:30.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.238+0000 7f920f279e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:30.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.238+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc59a20 con 0x55de7cc1a410 2023-10-05T19:40:30.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.238+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f9204014040 con 0x55de7cc1a410 2023-10-05T19:40:30.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.242+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94414064427568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc5ab70 con 0x55de7cc1a410 2023-10-05T19:40:30.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.242+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94414064427568] v28'159 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f9204014040 con 0x55de7cc1a410 2023-10-05T19:40:30.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.242+0000 7f91dd7f2700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:30.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.242+0000 7f920f279e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:30.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.242+0000 7f920f279e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:30.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.242+0000 7f920f279e40 1 Processor -- start 2023-10-05T19:40:30.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.242+0000 7f920f279e40 1 -- start start 2023-10-05T19:40:30.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.242+0000 7f920f279e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de7cb8a310 0x55de7cb8a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.242+0000 7f920f279e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de7cb8ac20 con 0x55de7cb8a310 2023-10-05T19:40:30.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.242+0000 7f920c5ad700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de7cb8a310 0x55de7cb8a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.242+0000 7f920c5ad700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de7cb8a310 0x55de7cb8a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58754/0 (socket says 172.21.15.12:58754) 2023-10-05T19:40:30.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.242+0000 7f920c5ad700 1 -- 172.21.15.12:0/1125009676 learned_addr learned my addr 172.21.15.12:0/1125009676 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:30.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.246+0000 7f920c5ad700 1 -- 172.21.15.12:0/1125009676 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de7cb8ad60 con 0x55de7cb8a310 2023-10-05T19:40:30.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.246+0000 7f920c5ad700 1 --2- 172.21.15.12:0/1125009676 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de7cb8a310 0x55de7cb8a6e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x55de7cc16eb0 tx=0x7f91f4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b47d676f5830bb0f server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.246+0000 7f91dcff1700 1 -- 172.21.15.12:0/1125009676 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91f4030930 con 0x55de7cb8a310 2023-10-05T19:40:30.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.246+0000 7f91dcff1700 1 -- 172.21.15.12:0/1125009676 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f4030ab0 con 0x55de7cb8a310 2023-10-05T19:40:30.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.246+0000 7f91dcff1700 1 -- 172.21.15.12:0/1125009676 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91f4030c60 con 0x55de7cb8a310 2023-10-05T19:40:30.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.246+0000 7f920f279e40 1 -- 172.21.15.12:0/1125009676 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de7cb8a310 msgr2=0x55de7cb8a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.246+0000 7f920f279e40 1 --2- 172.21.15.12:0/1125009676 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de7cb8a310 0x55de7cb8a6e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x55de7cc16eb0 tx=0x7f91f4012000 comp rx=0 tx=0).stop 2023-10-05T19:40:30.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.246+0000 7f920f279e40 1 -- 172.21.15.12:0/1125009676 shutdown_connections 2023-10-05T19:40:30.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.246+0000 7f920f279e40 1 --2- 172.21.15.12:0/1125009676 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de7cb8a310 0x55de7cb8a6e0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.246+0000 7f920f279e40 1 -- 172.21.15.12:0/1125009676 >> 172.21.15.12:0/1125009676 conn(0x55de7cc62ec0 msgr2=0x55de7cc61010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:30.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.246+0000 7f920f279e40 1 -- 172.21.15.12:0/1125009676 shutdown_connections 2023-10-05T19:40:30.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.246+0000 7f920f279e40 1 -- 172.21.15.12:0/1125009676 wait complete. 2023-10-05T19:40:30.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.246+0000 7f920f279e40 1 Processor -- start 2023-10-05T19:40:30.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.246+0000 7f920f279e40 1 -- start start 2023-10-05T19:40:30.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.250+0000 7f920f279e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de7cb8a310 0x55de7cc6f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.250+0000 7f920f279e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de7cc70800 con 0x55de7cb8a310 2023-10-05T19:40:30.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.250+0000 7f920c5ad700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de7cb8a310 0x55de7cc6f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.250+0000 7f920c5ad700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de7cb8a310 0x55de7cc6f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58768/0 (socket says 172.21.15.12:58768) 2023-10-05T19:40:30.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.250+0000 7f920c5ad700 1 -- 172.21.15.12:0/2422437648 learned_addr learned my addr 172.21.15.12:0/2422437648 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:30.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.250+0000 7f920c5ad700 1 -- 172.21.15.12:0/2422437648 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de7cc16a40 con 0x55de7cb8a310 2023-10-05T19:40:30.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.250+0000 7f920c5ad700 1 --2- 172.21.15.12:0/2422437648 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de7cb8a310 0x55de7cc6f800 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x55de7cc634a0 tx=0x7f91f4034000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.250+0000 7f9156ffd700 1 -- 172.21.15.12:0/2422437648 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91f4027610 con 0x55de7cb8a310 2023-10-05T19:40:30.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.250+0000 7f9156ffd700 1 -- 172.21.15.12:0/2422437648 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f4027770 con 0x55de7cb8a310 2023-10-05T19:40:30.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.250+0000 7f9156ffd700 1 -- 172.21.15.12:0/2422437648 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91f4027900 con 0x55de7cb8a310 2023-10-05T19:40:30.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.250+0000 7f920f279e40 1 -- 172.21.15.12:0/2422437648 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55de7cc6fe00 con 0x55de7cb8a310 2023-10-05T19:40:30.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.250+0000 7f920f279e40 1 -- 172.21.15.12:0/2422437648 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55de7cc701e0 con 0x55de7cb8a310 2023-10-05T19:40:30.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.250+0000 7f9156ffd700 1 -- 172.21.15.12:0/2422437648 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f91f4027a60 con 0x55de7cb8a310 2023-10-05T19:40:30.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.250+0000 7f9156ffd700 1 --2- 172.21.15.12:0/2422437648 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f914c030c50 0x7f914c033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.254+0000 7f9156ffd700 1 -- 172.21.15.12:0/2422437648 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f91f4054ab0 con 0x55de7cb8a310 2023-10-05T19:40:30.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.254+0000 7f920f279e40 5 note: GC not initialized 2023-10-05T19:40:30.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.254+0000 7f920d5af700 1 --2- 172.21.15.12:0/2422437648 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f914c030c50 0x7f914c033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.254+0000 7f920d5af700 1 --2- 172.21.15.12:0/2422437648 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f914c030c50 0x7f914c033100 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x55de7cc59aa0 tx=0x7f920400ab20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.254+0000 7f914bfff700 20 reqs_thread_entry: start 2023-10-05T19:40:30.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.254+0000 7f9149ffb700 20 reqs_thread_entry: start 2023-10-05T19:40:30.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.254+0000 7f920f279e40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:30.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.254+0000 7f920f279e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:30.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.254+0000 7f920f279e40 1 --2- 172.21.15.12:0/2422437648 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55de7ccd6dd0 0x55de7ccd91b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.254+0000 7f920f279e40 1 -- 172.21.15.12:0/2422437648 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:1 6.1e 6:7f266c6f:::2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.3-myfoo3:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7ccd96f0 con 0x55de7ccd6dd0 2023-10-05T19:40:30.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.254+0000 7f91377fe700 20 reqs_thread_entry: start 2023-10-05T19:40:30.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.254+0000 7f9137fff700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:41:00 2023 2023-10-05T19:40:30.266 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:30.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.258+0000 7f920cdae700 1 --2- 172.21.15.12:0/2422437648 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55de7ccd6dd0 0x55de7ccd91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.258+0000 7f920cdae700 1 --2- 172.21.15.12:0/2422437648 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55de7ccd6dd0 0x55de7ccd91b0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.258+0000 7f920cdae700 1 -- 172.21.15.12:0/2422437648 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(1 2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.3-myfoo3 [delete] v28'13 uv13 ondisk = 0) v8 ==== 208+0+0 (crc 0 0 0) 0x7f91fc02f040 con 0x55de7ccd6dd0 2023-10-05T19:40:30.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.258+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:102 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94414064404240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc95f10 con 0x55de7cc49e00 2023-10-05T19:40:30.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.262+0000 7f920cdae700 1 -- 172.21.15.12:0/162702257 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(102 notify.0 [watch unwatch cookie 94414064404240] v28'170 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f91fc02f040 con 0x55de7cc49e00 2023-10-05T19:40:30.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.262+0000 7f920f279e40 20 remove_watcher() i=0 2023-10-05T19:40:30.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.262+0000 7f920f279e40 2 removed watcher, disabling cache 2023-10-05T19:40:30.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.262+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94414064409024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc54710 con 0x55de7cc1a410 2023-10-05T19:40:30.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.266+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(103 notify.1 [watch unwatch cookie 94414064409024] v28'170 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9204014040 con 0x55de7cc1a410 2023-10-05T19:40:30.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.266+0000 7f920f279e40 20 remove_watcher() i=1 2023-10-05T19:40:30.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.266+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:104 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94414064407472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc54710 con 0x55de7cc4e5a0 2023-10-05T19:40:30.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.266+0000 7f920c5ad700 1 -- 172.21.15.12:0/162702257 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(104 notify.2 [watch unwatch cookie 94414064407472] v28'170 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f91f4008a80 con 0x55de7cc4e5a0 2023-10-05T19:40:30.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.266+0000 7f920f279e40 20 remove_watcher() i=2 2023-10-05T19:40:30.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.266+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:105 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94414064415440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc54710 con 0x55de7cc49e00 2023-10-05T19:40:30.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.270+0000 7f920cdae700 1 -- 172.21.15.12:0/162702257 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(105 notify.3 [watch unwatch cookie 94414064415440] v28'170 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f91fc02f040 con 0x55de7cc49e00 2023-10-05T19:40:30.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.270+0000 7f920f279e40 20 remove_watcher() i=3 2023-10-05T19:40:30.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.270+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:106 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94414064410128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc572d0 con 0x55de7cc1a410 2023-10-05T19:40:30.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.274+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(106 notify.4 [watch unwatch cookie 94414064410128] v28'170 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9204014040 con 0x55de7cc1a410 2023-10-05T19:40:30.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.274+0000 7f920f279e40 20 remove_watcher() i=4 2023-10-05T19:40:30.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.274+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:107 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94414064416544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc572d0 con 0x7f91fc011c00 2023-10-05T19:40:30.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.274+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(107 notify.5 [watch unwatch cookie 94414064416544] v28'170 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9204014040 con 0x7f91fc011c00 2023-10-05T19:40:30.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.278+0000 7f920f279e40 20 remove_watcher() i=5 2023-10-05T19:40:30.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.278+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:108 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94414064419616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc54710 con 0x7f91fc011c00 2023-10-05T19:40:30.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.278+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(108 notify.6 [watch unwatch cookie 94414064419616] v28'170 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9204014040 con 0x7f91fc011c00 2023-10-05T19:40:30.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.278+0000 7f920f279e40 20 remove_watcher() i=6 2023-10-05T19:40:30.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.278+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:109 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94414064422336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc54710 con 0x7f91fc011c00 2023-10-05T19:40:30.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.282+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(109 notify.7 [watch unwatch cookie 94414064422336] v28'170 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9204014040 con 0x7f91fc011c00 2023-10-05T19:40:30.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.282+0000 7f920f279e40 20 remove_watcher() i=7 2023-10-05T19:40:30.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.286+0000 7f91dd7f2700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:30.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.286+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:110 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94414064427568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55de7cc98b80 con 0x55de7cc1a410 2023-10-05T19:40:30.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.286+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(110 data_loggenerations_metadata [watch unwatch cookie 94414064427568] v28'160 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f9204014040 con 0x55de7cc1a410 2023-10-05T19:40:30.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.286+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f91fc011c00 msgr2=0x7f91fc03d100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.286+0000 7f920f279e40 1 --2- 172.21.15.12:0/162702257 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f91fc011c00 0x7f91fc03d100 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.286+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55de7cc1a410 msgr2=0x55de7cc47880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.286+0000 7f920f279e40 1 --2- 172.21.15.12:0/162702257 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55de7cc1a410 0x55de7cc47880 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.286+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55de7cc49e00 msgr2=0x55de7cc4c1e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.286+0000 7f920f279e40 1 --2- 172.21.15.12:0/162702257 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55de7cc49e00 0x55de7cc4c1e0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.286+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55de7cc4e5a0 msgr2=0x55de7cc509b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.286+0000 7f920f279e40 1 --2- 172.21.15.12:0/162702257 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55de7cc4e5a0 0x55de7cc509b0 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.286+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f91f0030c00 msgr2=0x7f91f00330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.286+0000 7f920f279e40 1 --2- 172.21.15.12:0/162702257 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f91f0030c00 0x7f91f00330b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f91f4005a90 tx=0x7f91f4015000 comp rx=0 tx=0).stop 2023-10-05T19:40:30.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.286+0000 7f920d5af700 1 -- 172.21.15.12:0/162702257 reap_dead start 2023-10-05T19:40:30.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.286+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de7cc15e30 msgr2=0x55de7cc3bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.286+0000 7f920f279e40 1 --2- 172.21.15.12:0/162702257 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de7cc15e30 0x55de7cc3bdc0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f91fc006bd0 tx=0x7f91fc014000 comp rx=0 tx=0).stop 2023-10-05T19:40:30.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.286+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 shutdown_connections 2023-10-05T19:40:30.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.286+0000 7f920f279e40 1 --2- 172.21.15.12:0/162702257 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de7cc15e30 0x55de7cc3bdc0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.286+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 >> 172.21.15.12:0/162702257 conn(0x55de7cc11850 msgr2=0x55de7cc2f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:30.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.286+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 shutdown_connections 2023-10-05T19:40:30.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.286+0000 7f920f279e40 1 -- 172.21.15.12:0/162702257 wait complete. 2023-10-05T19:40:30.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.290+0000 7f920f279e40 1 -- 172.21.15.12:0/2422437648 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55de7ccd6dd0 msgr2=0x55de7ccd91b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.290+0000 7f920f279e40 1 --2- 172.21.15.12:0/2422437648 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55de7ccd6dd0 0x55de7ccd91b0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.290+0000 7f920f279e40 1 -- 172.21.15.12:0/2422437648 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f914c030c50 msgr2=0x7f914c033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.290+0000 7f920f279e40 1 --2- 172.21.15.12:0/2422437648 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f914c030c50 0x7f914c033100 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x55de7cc59aa0 tx=0x7f920400ab20 comp rx=0 tx=0).stop 2023-10-05T19:40:30.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.290+0000 7f920f279e40 1 -- 172.21.15.12:0/2422437648 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de7cb8a310 msgr2=0x55de7cc6f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.290+0000 7f920f279e40 1 --2- 172.21.15.12:0/2422437648 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de7cb8a310 0x55de7cc6f800 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x55de7cc634a0 tx=0x7f91f4034000 comp rx=0 tx=0).stop 2023-10-05T19:40:30.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.290+0000 7f920f279e40 1 -- 172.21.15.12:0/2422437648 shutdown_connections 2023-10-05T19:40:30.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.290+0000 7f920f279e40 1 --2- 172.21.15.12:0/2422437648 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55de7ccd6dd0 0x55de7ccd91b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.290+0000 7f920f279e40 1 --2- 172.21.15.12:0/2422437648 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f914c030c50 0x7f914c033100 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.290+0000 7f920f279e40 1 --2- 172.21.15.12:0/2422437648 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de7cb8a310 0x55de7cc6f800 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.290+0000 7f920f279e40 1 -- 172.21.15.12:0/2422437648 >> 172.21.15.12:0/2422437648 conn(0x55de7cc62ec0 msgr2=0x55de7cc60ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:30.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.290+0000 7f920f279e40 1 -- 172.21.15.12:0/2422437648 shutdown_connections 2023-10-05T19:40:30.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.290+0000 7f920f279e40 1 -- 172.21.15.12:0/2422437648 wait complete. 2023-10-05T19:40:30.305 INFO:tasks.util.rgw:rgwadmin: client.0 : ['log', 'show', '--object', '2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1-myfoo'] 2023-10-05T19:40:30.305 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'log', 'show', '--object', '2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1-myfoo'] 2023-10-05T19:40:30.305 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph log show --object 2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1-myfoo 2023-10-05T19:40:30.388 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:30.389 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:30.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.378+0000 7f31ee7f2e40 1 Processor -- start 2023-10-05T19:40:30.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.378+0000 7f31ee7f2e40 1 -- start start 2023-10-05T19:40:30.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.378+0000 7f31ee7f2e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55bb996b7910 0x55bb9974bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.378+0000 7f31ee7f2e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55bb9968fbd0 con 0x55bb996b8c20 2023-10-05T19:40:30.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.378+0000 7f31ee7f2e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x55bb9964cee0 con 0x55bb996b7910 2023-10-05T19:40:30.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.382+0000 7f31ebb26700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55bb996b7910 0x55bb9974bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.382+0000 7f31ec327700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x55bb996b8c20 0x55bb996b7200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:35734/0 (socket says 172.21.15.12:35734) 2023-10-05T19:40:30.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.382+0000 7f31ebb26700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55bb996b7910 0x55bb9974bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58776/0 (socket says 172.21.15.12:58776) 2023-10-05T19:40:30.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.382+0000 7f31ec327700 1 -- 172.21.15.12:0/2266792334 learned_addr learned my addr 172.21.15.12:0/2266792334 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:30.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.382+0000 7f31ebb26700 1 -- 172.21.15.12:0/2266792334 >> v1:172.21.15.8:6789/0 conn(0x55bb996b8c20 legacy=0x55bb996b7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.382+0000 7f31ebb26700 1 -- 172.21.15.12:0/2266792334 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb996b90b0 con 0x55bb996b7910 2023-10-05T19:40:30.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.382+0000 7f31ebb26700 1 --2- 172.21.15.12:0/2266792334 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb996b7910 0x55bb9974bf60 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f31d400a5d0 tx=0x7f31d4005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85b43669ddf20baf server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.382+0000 7f31eb325700 1 -- 172.21.15.12:0/2266792334 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f31d40086a0 con 0x55bb996b7910 2023-10-05T19:40:30.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.382+0000 7f31eb325700 1 -- 172.21.15.12:0/2266792334 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31d4008800 con 0x55bb996b7910 2023-10-05T19:40:30.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.382+0000 7f31eb325700 1 -- 172.21.15.12:0/2266792334 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f31d4008ad0 con 0x55bb996b7910 2023-10-05T19:40:30.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.382+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/2266792334 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb996b7910 msgr2=0x55bb9974bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.382+0000 7f31ee7f2e40 1 --2- 172.21.15.12:0/2266792334 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb996b7910 0x55bb9974bf60 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f31d400a5d0 tx=0x7f31d4005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:30.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.382+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/2266792334 shutdown_connections 2023-10-05T19:40:30.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.382+0000 7f31ee7f2e40 1 --2- 172.21.15.12:0/2266792334 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb996b7910 0x55bb9974bf60 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.382+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/2266792334 >> 172.21.15.12:0/2266792334 conn(0x55bb9973e850 msgr2=0x55bb99740c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:30.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.382+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/2266792334 shutdown_connections 2023-10-05T19:40:30.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.382+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/2266792334 wait complete. 2023-10-05T19:40:30.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.386+0000 7f31ee7f2e40 1 Processor -- start 2023-10-05T19:40:30.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.386+0000 7f31ee7f2e40 1 -- start start 2023-10-05T19:40:30.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.386+0000 7f31ee7f2e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb996b7910 0x55bb99764190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.386+0000 7f31ee7f2e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb99610e30 con 0x55bb996b7910 2023-10-05T19:40:30.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.386+0000 7f31ec327700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb996b7910 0x55bb99764190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.386+0000 7f31ec327700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb996b7910 0x55bb99764190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58788/0 (socket says 172.21.15.12:58788) 2023-10-05T19:40:30.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.386+0000 7f31ec327700 1 -- 172.21.15.12:0/2797799382 learned_addr learned my addr 172.21.15.12:0/2797799382 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:30.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.386+0000 7f31ec327700 1 -- 172.21.15.12:0/2797799382 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d4005cf0 con 0x55bb996b7910 2023-10-05T19:40:30.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.386+0000 7f31ec327700 1 --2- 172.21.15.12:0/2797799382 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb996b7910 0x55bb99764190 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f31e4001c30 tx=0x7f31e40017e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25c20f4437e8dfaf server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.386+0000 7f31ea323700 1 -- 172.21.15.12:0/2797799382 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f31e400bbf0 con 0x55bb996b7910 2023-10-05T19:40:30.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.386+0000 7f31ea323700 1 -- 172.21.15.12:0/2797799382 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e400bd50 con 0x55bb996b7910 2023-10-05T19:40:30.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.386+0000 7f31ea323700 1 -- 172.21.15.12:0/2797799382 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f31e400e470 con 0x55bb996b7910 2023-10-05T19:40:30.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.390+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/2797799382 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb996b7910 msgr2=0x55bb99764190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.390+0000 7f31ee7f2e40 1 --2- 172.21.15.12:0/2797799382 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb996b7910 0x55bb99764190 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f31e4001c30 tx=0x7f31e40017e0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.390+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/2797799382 shutdown_connections 2023-10-05T19:40:30.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.390+0000 7f31ee7f2e40 1 --2- 172.21.15.12:0/2797799382 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb996b7910 0x55bb99764190 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.390+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/2797799382 >> 172.21.15.12:0/2797799382 conn(0x55bb9973e850 msgr2=0x55bb9975fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:30.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.390+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/2797799382 shutdown_connections 2023-10-05T19:40:30.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.390+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/2797799382 wait complete. 2023-10-05T19:40:30.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.390+0000 7f31ee7f2e40 1 Processor -- start 2023-10-05T19:40:30.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.390+0000 7f31ee7f2e40 1 -- start start 2023-10-05T19:40:30.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.390+0000 7f31ee7f2e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb996b7910 0x55bb9976af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.390+0000 7f31ee7f2e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb995e3ed0 con 0x55bb996b7910 2023-10-05T19:40:30.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.390+0000 7f31ec327700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb996b7910 0x55bb9976af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.390+0000 7f31ec327700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb996b7910 0x55bb9976af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58800/0 (socket says 172.21.15.12:58800) 2023-10-05T19:40:30.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.390+0000 7f31ec327700 1 -- 172.21.15.12:0/4213286051 learned_addr learned my addr 172.21.15.12:0/4213286051 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:30.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.390+0000 7f31ec327700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb9968fbd0 con 0x55bb996b7910 2023-10-05T19:40:30.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.394+0000 7f31ec327700 1 --2- 172.21.15.12:0/4213286051 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb996b7910 0x55bb9976af60 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f31e4000c00 tx=0x7f31e4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.394+0000 7f31e8b20700 1 -- 172.21.15.12:0/4213286051 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f31e401f070 con 0x55bb996b7910 2023-10-05T19:40:30.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.394+0000 7f31e8b20700 1 -- 172.21.15.12:0/4213286051 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e4019690 con 0x55bb996b7910 2023-10-05T19:40:30.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.394+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bb9976b4a0 con 0x55bb996b7910 2023-10-05T19:40:30.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.394+0000 7f31e8b20700 1 -- 172.21.15.12:0/4213286051 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f31e4019840 con 0x55bb996b7910 2023-10-05T19:40:30.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.394+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bb9976b8e0 con 0x55bb996b7910 2023-10-05T19:40:30.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.394+0000 7f31e8b20700 1 -- 172.21.15.12:0/4213286051 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f31e4022020 con 0x55bb996b7910 2023-10-05T19:40:30.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.394+0000 7f31ee7f2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.394+0000 7f31e8b20700 1 --2- 172.21.15.12:0/4213286051 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f31cc030bf0 0x7f31cc0330a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.394+0000 7f31e8b20700 1 -- 172.21.15.12:0/4213286051 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f31e4044da0 con 0x55bb996b7910 2023-10-05T19:40:30.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.394+0000 7f31ebb26700 1 --2- 172.21.15.12:0/4213286051 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f31cc030bf0 0x7f31cc0330a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.394+0000 7f31ee7f2e40 1 --2- 172.21.15.12:0/4213286051 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55bb996b8c20 0x55bb997769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.394+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb997770a0 con 0x55bb996b8c20 2023-10-05T19:40:30.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.398+0000 7f31ecb28700 1 --2- 172.21.15.12:0/4213286051 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55bb996b8c20 0x55bb997769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.398+0000 7f31ebb26700 1 --2- 172.21.15.12:0/4213286051 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f31cc030bf0 0x7f31cc0330a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f31d4005a90 tx=0x7f31d4015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.398+0000 7f31ecb28700 1 --2- 172.21.15.12:0/4213286051 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55bb996b8c20 0x55bb997769f0 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.398+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f31dc00a5a0 con 0x55bb996b8c20 2023-10-05T19:40:30.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.398+0000 7f31ee7f2e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:30.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.398+0000 7f31ee7f2e40 20 realm 2023-10-05T19:40:30.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.398+0000 7f31ee7f2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.398+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99774050 con 0x55bb996b8c20 2023-10-05T19:40:30.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.398+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f31dc00a5a0 con 0x55bb996b8c20 2023-10-05T19:40:30.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.398+0000 7f31ee7f2e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:30.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.398+0000 7f31ee7f2e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:30.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.398+0000 7f31ee7f2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.398+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99777b80 con 0x55bb996b8c20 2023-10-05T19:40:30.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.398+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f31dc00a5a0 con 0x55bb996b8c20 2023-10-05T19:40:30.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.398+0000 7f31ee7f2e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:30.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.398+0000 7f31ee7f2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.398+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99777760 con 0x55bb996b8c20 2023-10-05T19:40:30.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.398+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f31dc00a5a0 con 0x55bb996b8c20 2023-10-05T19:40:30.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.398+0000 7f31ee7f2e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:30.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.398+0000 7f31ee7f2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.398+0000 7f31ee7f2e40 1 --2- 172.21.15.12:0/4213286051 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55bb99778b10 0x55bb9977aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.398+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb9977b660 con 0x55bb99778b10 2023-10-05T19:40:30.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.402+0000 7f31ec327700 1 --2- 172.21.15.12:0/4213286051 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55bb99778b10 0x55bb9977aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.402+0000 7f31ec327700 1 --2- 172.21.15.12:0/4213286051 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55bb99778b10 0x55bb9977aef0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.402+0000 7f31ec327700 1 -- 172.21.15.12:0/4213286051 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f31e400d7f0 con 0x55bb99778b10 2023-10-05T19:40:30.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.402+0000 7f31ee7f2e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:30.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.402+0000 7f31ee7f2e40 20 searching for the correct realm 2023-10-05T19:40:30.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.402+0000 7f31ee7f2e40 1 --2- 172.21.15.12:0/4213286051 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55bb9977d490 0x55bb9977f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.402+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb9977fdb0 con 0x55bb9977d490 2023-10-05T19:40:30.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.402+0000 7f31ebb26700 1 --2- 172.21.15.12:0/4213286051 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55bb9977d490 0x55bb9977f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.402+0000 7f31ebb26700 1 --2- 172.21.15.12:0/4213286051 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55bb9977d490 0x55bb9977f870 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.406+0000 7f31ebb26700 1 -- 172.21.15.12:0/4213286051 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f31d4008ae0 con 0x55bb9977d490 2023-10-05T19:40:30.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.406+0000 7f31ebb26700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb9977fdb0 con 0x55bb99778b10 2023-10-05T19:40:30.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.406+0000 7f31ec327700 1 -- 172.21.15.12:0/4213286051 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31e400d7f0 con 0x55bb99778b10 2023-10-05T19:40:30.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.406+0000 7f31ec327700 1 --2- 172.21.15.12:0/4213286051 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f31e4016b80 0x7f31e40420e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.406+0000 7f31ec327700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb9977fdb0 con 0x7f31e4016b80 2023-10-05T19:40:30.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.406+0000 7f31ecb28700 1 --2- 172.21.15.12:0/4213286051 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f31e4016b80 0x7f31e40420e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.406+0000 7f31ecb28700 1 --2- 172.21.15.12:0/4213286051 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f31e4016b80 0x7f31e40420e0 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.406+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31dc00a5a0 con 0x7f31e4016b80 2023-10-05T19:40:30.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.406+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb9977fdb0 con 0x55bb996b8c20 2023-10-05T19:40:30.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.406+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31dc00a5a0 con 0x55bb996b8c20 2023-10-05T19:40:30.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.406+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb9977fdb0 con 0x55bb996b8c20 2023-10-05T19:40:30.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.410+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31dc00a5a0 con 0x55bb996b8c20 2023-10-05T19:40:30.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.410+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb9977fdb0 con 0x7f31e4016b80 2023-10-05T19:40:30.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.410+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31dc00a5a0 con 0x7f31e4016b80 2023-10-05T19:40:30.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.410+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb9977fdb0 con 0x7f31e4016b80 2023-10-05T19:40:30.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.410+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31dc00a5a0 con 0x7f31e4016b80 2023-10-05T19:40:30.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.410+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb9977fdb0 con 0x55bb99778b10 2023-10-05T19:40:30.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.410+0000 7f31ec327700 1 -- 172.21.15.12:0/4213286051 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31e400d7f0 con 0x55bb99778b10 2023-10-05T19:40:30.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.410+0000 7f31ec327700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb9977fdb0 con 0x55bb9977d490 2023-10-05T19:40:30.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.410+0000 7f31ebb26700 1 -- 172.21.15.12:0/4213286051 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31d4008ae0 con 0x55bb9977d490 2023-10-05T19:40:30.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.410+0000 7f31ebb26700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb9977fdb0 con 0x55bb996b8c20 2023-10-05T19:40:30.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.414+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31dc00a5a0 con 0x55bb996b8c20 2023-10-05T19:40:30.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.414+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb9977fdb0 con 0x7f31e4016b80 2023-10-05T19:40:30.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.414+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31dc00a5a0 con 0x7f31e4016b80 2023-10-05T19:40:30.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.414+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb9977fdb0 con 0x55bb996b8c20 2023-10-05T19:40:30.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.414+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31dc00a5a0 con 0x55bb996b8c20 2023-10-05T19:40:30.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.414+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb9977fdb0 con 0x55bb99778b10 2023-10-05T19:40:30.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.414+0000 7f31ec327700 1 -- 172.21.15.12:0/4213286051 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31e400d7f0 con 0x55bb99778b10 2023-10-05T19:40:30.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.414+0000 7f31ec327700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb9977fdb0 con 0x55bb9977d490 2023-10-05T19:40:30.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.414+0000 7f31ebb26700 1 -- 172.21.15.12:0/4213286051 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31d4008ae0 con 0x55bb9977d490 2023-10-05T19:40:30.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.414+0000 7f31ebb26700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb9977fdb0 con 0x55bb9977d490 2023-10-05T19:40:30.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.414+0000 7f31ebb26700 1 -- 172.21.15.12:0/4213286051 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31d4008ae0 con 0x55bb9977d490 2023-10-05T19:40:30.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.418+0000 7f31ebb26700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb9977fdb0 con 0x7f31e4016b80 2023-10-05T19:40:30.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.418+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31dc00a5a0 con 0x7f31e4016b80 2023-10-05T19:40:30.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.418+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb9977fdb0 con 0x55bb9977d490 2023-10-05T19:40:30.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.418+0000 7f31ebb26700 1 -- 172.21.15.12:0/4213286051 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31d4008ae0 con 0x55bb9977d490 2023-10-05T19:40:30.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.418+0000 7f31ebb26700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb9977fdb0 con 0x55bb9977d490 2023-10-05T19:40:30.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.418+0000 7f31ebb26700 1 -- 172.21.15.12:0/4213286051 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31d4008ae0 con 0x55bb9977d490 2023-10-05T19:40:30.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.418+0000 7f31ebb26700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb9977fdb0 con 0x55bb996b8c20 2023-10-05T19:40:30.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.418+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31dc00a5a0 con 0x55bb996b8c20 2023-10-05T19:40:30.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.418+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb9977fdb0 con 0x55bb996b8c20 2023-10-05T19:40:30.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.418+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31dc00a5a0 con 0x55bb996b8c20 2023-10-05T19:40:30.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.418+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb9977fdb0 con 0x7f31e4016b80 2023-10-05T19:40:30.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.422+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31dc00a5a0 con 0x7f31e4016b80 2023-10-05T19:40:30.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.422+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb9977fdb0 con 0x55bb9977d490 2023-10-05T19:40:30.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.422+0000 7f31ebb26700 1 -- 172.21.15.12:0/4213286051 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31d4008ae0 con 0x55bb9977d490 2023-10-05T19:40:30.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.422+0000 7f31ebb26700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb9977fdb0 con 0x55bb99778b10 2023-10-05T19:40:30.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.422+0000 7f31ec327700 1 -- 172.21.15.12:0/4213286051 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31e400d7f0 con 0x55bb99778b10 2023-10-05T19:40:30.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.422+0000 7f31ec327700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb9977fdb0 con 0x55bb996b8c20 2023-10-05T19:40:30.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.422+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31dc00a5a0 con 0x55bb996b8c20 2023-10-05T19:40:30.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.422+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb9977fdb0 con 0x7f31e4016b80 2023-10-05T19:40:30.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.426+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31dc00a5a0 con 0x7f31e4016b80 2023-10-05T19:40:30.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.426+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb9977fdb0 con 0x55bb9977d490 2023-10-05T19:40:30.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.426+0000 7f31ebb26700 1 -- 172.21.15.12:0/4213286051 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31d4008ae0 con 0x55bb9977d490 2023-10-05T19:40:30.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.426+0000 7f31ebb26700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb9977fdb0 con 0x7f31e4016b80 2023-10-05T19:40:30.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.426+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31dc00a5a0 con 0x7f31e4016b80 2023-10-05T19:40:30.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.426+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb9977fdb0 con 0x55bb99778b10 2023-10-05T19:40:30.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.426+0000 7f31ec327700 1 -- 172.21.15.12:0/4213286051 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31e400d7f0 con 0x55bb99778b10 2023-10-05T19:40:30.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.426+0000 7f31ec327700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb9977fdb0 con 0x55bb9977d490 2023-10-05T19:40:30.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.426+0000 7f31ebb26700 1 -- 172.21.15.12:0/4213286051 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31d4008ae0 con 0x55bb9977d490 2023-10-05T19:40:30.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.426+0000 7f31ebb26700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb9977fdb0 con 0x7f31e4016b80 2023-10-05T19:40:30.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.430+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31dc00a5a0 con 0x7f31e4016b80 2023-10-05T19:40:30.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.430+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb9977fdb0 con 0x55bb996b8c20 2023-10-05T19:40:30.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.430+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f31dc00a5a0 con 0x55bb996b8c20 2023-10-05T19:40:30.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.430+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb9977fdb0 con 0x55bb99778b10 2023-10-05T19:40:30.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.430+0000 7f31ec327700 1 -- 172.21.15.12:0/4213286051 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f31e400d7f0 con 0x55bb99778b10 2023-10-05T19:40:30.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.430+0000 7f31ee7f2e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:30.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.430+0000 7f31ee7f2e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:30.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.430+0000 7f31ee7f2e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:30.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.430+0000 7f31ee7f2e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:30.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.430+0000 7f31ee7f2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.430+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb9977c210 con 0x55bb996b8c20 2023-10-05T19:40:30.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.430+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f31dc00a5a0 con 0x55bb996b8c20 2023-10-05T19:40:30.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.430+0000 7f31ee7f2e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:30.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.430+0000 7f31ee7f2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.430+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb9977cb60 con 0x55bb996b8c20 2023-10-05T19:40:30.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.434+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f31dc00a5a0 con 0x55bb996b8c20 2023-10-05T19:40:30.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.434+0000 7f31ee7f2e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:30.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.434+0000 7f31ee7f2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.434+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb9977bdd0 con 0x55bb9977d490 2023-10-05T19:40:30.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.434+0000 7f31ebb26700 1 -- 172.21.15.12:0/4213286051 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f31d4008ae0 con 0x55bb9977d490 2023-10-05T19:40:30.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.434+0000 7f31ee7f2e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:30.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.434+0000 7f31ee7f2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.434+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb997806e0 con 0x55bb9977d490 2023-10-05T19:40:30.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.434+0000 7f31ebb26700 1 -- 172.21.15.12:0/4213286051 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f31d4008ae0 con 0x55bb9977d490 2023-10-05T19:40:30.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.434+0000 7f31ee7f2e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:30.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.434+0000 7f31ee7f2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.434+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99780a90 con 0x55bb996b8c20 2023-10-05T19:40:30.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.434+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f31dc00a5a0 con 0x55bb996b8c20 2023-10-05T19:40:30.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.434+0000 7f31ee7f2e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:30.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.434+0000 7f31ee7f2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.434+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99780e40 con 0x55bb996b8c20 2023-10-05T19:40:30.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.434+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f31dc00a5a0 con 0x55bb996b8c20 2023-10-05T19:40:30.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.434+0000 7f31ee7f2e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:30.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.434+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99781e50 con 0x55bb9977d490 2023-10-05T19:40:30.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.438+0000 7f31ebb26700 1 -- 172.21.15.12:0/4213286051 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f31d4008ae0 con 0x55bb9977d490 2023-10-05T19:40:30.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.438+0000 7f31ebb26700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99781e50 con 0x55bb99778b10 2023-10-05T19:40:30.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.438+0000 7f31ec327700 1 -- 172.21.15.12:0/4213286051 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31e400d7f0 con 0x55bb99778b10 2023-10-05T19:40:30.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.438+0000 7f31ec327700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99781e50 con 0x7f31e4016b80 2023-10-05T19:40:30.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.438+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31dc00a5a0 con 0x7f31e4016b80 2023-10-05T19:40:30.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.438+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99780a90 con 0x55bb996b8c20 2023-10-05T19:40:30.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.442+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31dc00a5a0 con 0x55bb996b8c20 2023-10-05T19:40:30.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.442+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99780a90 con 0x55bb996b8c20 2023-10-05T19:40:30.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.442+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31dc00a5a0 con 0x55bb996b8c20 2023-10-05T19:40:30.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.442+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99780a90 con 0x7f31e4016b80 2023-10-05T19:40:30.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.442+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31dc00a5a0 con 0x7f31e4016b80 2023-10-05T19:40:30.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.442+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99780a90 con 0x7f31e4016b80 2023-10-05T19:40:30.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.442+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31dc00a5a0 con 0x7f31e4016b80 2023-10-05T19:40:30.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.442+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99780a90 con 0x55bb99778b10 2023-10-05T19:40:30.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.446+0000 7f31ec327700 1 -- 172.21.15.12:0/4213286051 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31e400d7f0 con 0x55bb99778b10 2023-10-05T19:40:30.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.446+0000 7f31ec327700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99780a90 con 0x55bb9977d490 2023-10-05T19:40:30.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.446+0000 7f31ebb26700 1 -- 172.21.15.12:0/4213286051 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31d4008ae0 con 0x55bb9977d490 2023-10-05T19:40:30.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.446+0000 7f31ebb26700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99780a90 con 0x55bb996b8c20 2023-10-05T19:40:30.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.446+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31dc00a5a0 con 0x55bb996b8c20 2023-10-05T19:40:30.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.446+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99780a90 con 0x7f31e4016b80 2023-10-05T19:40:30.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.446+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31dc00a5a0 con 0x7f31e4016b80 2023-10-05T19:40:30.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.446+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99780a90 con 0x55bb996b8c20 2023-10-05T19:40:30.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.446+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31dc00a5a0 con 0x55bb996b8c20 2023-10-05T19:40:30.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.446+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99780a90 con 0x55bb99778b10 2023-10-05T19:40:30.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.450+0000 7f31ec327700 1 -- 172.21.15.12:0/4213286051 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31e400d7f0 con 0x55bb99778b10 2023-10-05T19:40:30.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.450+0000 7f31ec327700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99780a90 con 0x55bb9977d490 2023-10-05T19:40:30.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.450+0000 7f31ebb26700 1 -- 172.21.15.12:0/4213286051 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31d4008ae0 con 0x55bb9977d490 2023-10-05T19:40:30.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.450+0000 7f31ebb26700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99780a90 con 0x55bb9977d490 2023-10-05T19:40:30.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.450+0000 7f31ebb26700 1 -- 172.21.15.12:0/4213286051 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31d4008ae0 con 0x55bb9977d490 2023-10-05T19:40:30.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.450+0000 7f31ebb26700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99780a90 con 0x7f31e4016b80 2023-10-05T19:40:30.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.450+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31dc00a5a0 con 0x7f31e4016b80 2023-10-05T19:40:30.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.450+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99780a90 con 0x55bb9977d490 2023-10-05T19:40:30.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.450+0000 7f31ebb26700 1 -- 172.21.15.12:0/4213286051 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31d4008ae0 con 0x55bb9977d490 2023-10-05T19:40:30.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.450+0000 7f31ebb26700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99780a90 con 0x55bb9977d490 2023-10-05T19:40:30.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.454+0000 7f31ebb26700 1 -- 172.21.15.12:0/4213286051 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31d4008ae0 con 0x55bb9977d490 2023-10-05T19:40:30.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.454+0000 7f31ebb26700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99780a90 con 0x55bb996b8c20 2023-10-05T19:40:30.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.454+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31dc00a5a0 con 0x55bb996b8c20 2023-10-05T19:40:30.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.454+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99780a90 con 0x55bb996b8c20 2023-10-05T19:40:30.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.454+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31dc00a5a0 con 0x55bb996b8c20 2023-10-05T19:40:30.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.454+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99780a90 con 0x7f31e4016b80 2023-10-05T19:40:30.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.454+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31dc00a5a0 con 0x7f31e4016b80 2023-10-05T19:40:30.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.454+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99780a90 con 0x55bb9977d490 2023-10-05T19:40:30.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.458+0000 7f31ebb26700 1 -- 172.21.15.12:0/4213286051 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31d4008ae0 con 0x55bb9977d490 2023-10-05T19:40:30.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.458+0000 7f31ebb26700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99780a90 con 0x55bb99778b10 2023-10-05T19:40:30.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.458+0000 7f31ec327700 1 -- 172.21.15.12:0/4213286051 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31e400d7f0 con 0x55bb99778b10 2023-10-05T19:40:30.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.458+0000 7f31ec327700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99780a90 con 0x55bb996b8c20 2023-10-05T19:40:30.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.458+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31dc00a5a0 con 0x55bb996b8c20 2023-10-05T19:40:30.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.458+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99780a90 con 0x7f31e4016b80 2023-10-05T19:40:30.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.458+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31dc00a5a0 con 0x7f31e4016b80 2023-10-05T19:40:30.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.458+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99780a90 con 0x55bb9977d490 2023-10-05T19:40:30.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.458+0000 7f31ebb26700 1 -- 172.21.15.12:0/4213286051 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31d4008ae0 con 0x55bb9977d490 2023-10-05T19:40:30.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.458+0000 7f31ebb26700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99780a90 con 0x7f31e4016b80 2023-10-05T19:40:30.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.462+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31dc00a5a0 con 0x7f31e4016b80 2023-10-05T19:40:30.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.462+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99780a90 con 0x55bb99778b10 2023-10-05T19:40:30.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.462+0000 7f31ec327700 1 -- 172.21.15.12:0/4213286051 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31e400d7f0 con 0x55bb99778b10 2023-10-05T19:40:30.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.462+0000 7f31ec327700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99780a90 con 0x55bb9977d490 2023-10-05T19:40:30.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.462+0000 7f31ebb26700 1 -- 172.21.15.12:0/4213286051 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31d4008ae0 con 0x55bb9977d490 2023-10-05T19:40:30.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.462+0000 7f31ebb26700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99780a90 con 0x7f31e4016b80 2023-10-05T19:40:30.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.462+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f31dc00a5a0 con 0x7f31e4016b80 2023-10-05T19:40:30.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.462+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99780a90 con 0x55bb996b8c20 2023-10-05T19:40:30.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.466+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f31dc00a5a0 con 0x55bb996b8c20 2023-10-05T19:40:30.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.466+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99780a90 con 0x55bb99778b10 2023-10-05T19:40:30.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.466+0000 7f31ec327700 1 -- 172.21.15.12:0/4213286051 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f31e400d7f0 con 0x55bb99778b10 2023-10-05T19:40:30.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.466+0000 7f31ee7f2e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:30.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.466+0000 7f31ee7f2e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:30.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.466+0000 7f31ee7f2e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:30.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.466+0000 7f31ee7f2e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:30.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.466+0000 7f31ee7f2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.466+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99780e40 con 0x55bb996b8c20 2023-10-05T19:40:30.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.466+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f31dc00a5a0 con 0x55bb996b8c20 2023-10-05T19:40:30.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.466+0000 7f31ee7f2e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:30.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.466+0000 7f31ee7f2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.466+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb997811f0 con 0x55bb99778b10 2023-10-05T19:40:30.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.466+0000 7f31ec327700 1 -- 172.21.15.12:0/4213286051 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f31e400d7f0 con 0x55bb99778b10 2023-10-05T19:40:30.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.466+0000 7f31ee7f2e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:30.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.466+0000 7f31ee7f2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.466+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99781a90 con 0x55bb996b8c20 2023-10-05T19:40:30.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.466+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f31dc00a5a0 con 0x55bb996b8c20 2023-10-05T19:40:30.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.466+0000 7f31ee7f2e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:30.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.466+0000 7f31ee7f2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.466+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99781a90 con 0x55bb9977d490 2023-10-05T19:40:30.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.470+0000 7f31ebb26700 1 -- 172.21.15.12:0/4213286051 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f31d4008ae0 con 0x55bb9977d490 2023-10-05T19:40:30.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.470+0000 7f31ee7f2e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:30.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.470+0000 7f31ee7f2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.470+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99781620 con 0x7f31e4016b80 2023-10-05T19:40:30.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.470+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f31dc00a5a0 con 0x7f31e4016b80 2023-10-05T19:40:30.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.470+0000 7f31ee7f2e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:30.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.470+0000 7f31ee7f2e40 20 zone default found 2023-10-05T19:40:30.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.470+0000 7f31ee7f2e40 4 Realm: () 2023-10-05T19:40:30.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.470+0000 7f31ee7f2e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:30.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.470+0000 7f31ee7f2e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:30.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.470+0000 7f31ee7f2e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:30.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.470+0000 7f31ee7f2e40 20 zonegroup default 2023-10-05T19:40:30.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.470+0000 7f31ee7f2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.470+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99781620 con 0x7f31e4016b80 2023-10-05T19:40:30.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.470+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f31dc00a5a0 con 0x7f31e4016b80 2023-10-05T19:40:30.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.470+0000 7f31ee7f2e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:30.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.470+0000 7f31ee7f2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.470+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99781620 con 0x55bb9977d490 2023-10-05T19:40:30.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.470+0000 7f31ebb26700 1 -- 172.21.15.12:0/4213286051 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f31d4008ae0 con 0x55bb9977d490 2023-10-05T19:40:30.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.470+0000 7f31ee7f2e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:30.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.470+0000 7f31ee7f2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.470+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99783cb0 con 0x55bb9977d490 2023-10-05T19:40:30.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.474+0000 7f31ebb26700 1 -- 172.21.15.12:0/4213286051 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f31d4008ae0 con 0x55bb9977d490 2023-10-05T19:40:30.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.474+0000 7f31ee7f2e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:30.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.474+0000 7f31ee7f2e40 20 started sync module instance, tier type = 2023-10-05T19:40:30.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.474+0000 7f31ee7f2e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:30.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.474+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99783370 con 0x55bb99778b10 2023-10-05T19:40:30.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.474+0000 7f31ec327700 1 -- 172.21.15.12:0/4213286051 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'171 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f31e400d7f0 con 0x55bb99778b10 2023-10-05T19:40:30.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.474+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94264222037872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99782fa0 con 0x55bb99778b10 2023-10-05T19:40:30.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.474+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99784a40 con 0x55bb996b8c20 2023-10-05T19:40:30.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.478+0000 7f31ec327700 1 -- 172.21.15.12:0/4213286051 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94264222037872] v28'172 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f31e400d7f0 con 0x55bb99778b10 2023-10-05T19:40:30.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.478+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'171 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f31dc00a5a0 con 0x55bb996b8c20 2023-10-05T19:40:30.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.478+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94264222040448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99784ac0 con 0x55bb996b8c20 2023-10-05T19:40:30.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.478+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99785620 con 0x55bb9977d490 2023-10-05T19:40:30.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.482+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94264222040448] v28'172 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f31dc00a5a0 con 0x55bb996b8c20 2023-10-05T19:40:30.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.482+0000 7f31ebb26700 1 -- 172.21.15.12:0/4213286051 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'171 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f31d4008ae0 con 0x55bb9977d490 2023-10-05T19:40:30.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.482+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94264222041552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99784a20 con 0x55bb9977d490 2023-10-05T19:40:30.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.482+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99786400 con 0x55bb99778b10 2023-10-05T19:40:30.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.486+0000 7f31ec327700 1 -- 172.21.15.12:0/4213286051 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'171 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f31e400d7f0 con 0x55bb99778b10 2023-10-05T19:40:30.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.486+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94264222050224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99786030 con 0x55bb99778b10 2023-10-05T19:40:30.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.486+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99787810 con 0x55bb996b8c20 2023-10-05T19:40:30.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.486+0000 7f31ebb26700 1 -- 172.21.15.12:0/4213286051 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94264222041552] v28'172 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f31d4008ae0 con 0x55bb9977d490 2023-10-05T19:40:30.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.486+0000 7f31ec327700 1 -- 172.21.15.12:0/4213286051 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94264222050224] v28'172 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f31e400d7f0 con 0x55bb99778b10 2023-10-05T19:40:30.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.490+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'171 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f31dc00a5a0 con 0x55bb996b8c20 2023-10-05T19:40:30.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.490+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94264222046752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99787320 con 0x55bb996b8c20 2023-10-05T19:40:30.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.490+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99788100 con 0x7f31e4016b80 2023-10-05T19:40:30.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.490+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'171 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f31dc00a5a0 con 0x7f31e4016b80 2023-10-05T19:40:30.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.490+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94264222054784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb997882c0 con 0x7f31e4016b80 2023-10-05T19:40:30.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.490+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99788cd0 con 0x7f31e4016b80 2023-10-05T19:40:30.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.490+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94264222046752] v28'172 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f31dc00a5a0 con 0x55bb996b8c20 2023-10-05T19:40:30.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.494+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(96 notify.6 [create] v28'171 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f31dc00a5a0 con 0x7f31e4016b80 2023-10-05T19:40:30.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.494+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94264222051328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99788cd0 con 0x7f31e4016b80 2023-10-05T19:40:30.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.494+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99789570 con 0x7f31e4016b80 2023-10-05T19:40:30.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.494+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(95 notify.5 [watch watch cookie 94264222054784] v28'172 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f31dc00a5a0 con 0x7f31e4016b80 2023-10-05T19:40:30.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.494+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(97 notify.6 [watch watch cookie 94264222051328] v28'172 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f31dc00a5a0 con 0x7f31e4016b80 2023-10-05T19:40:30.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.494+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(98 notify.7 [create] v28'171 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f31dc00a5a0 con 0x7f31e4016b80 2023-10-05T19:40:30.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.494+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94264222058464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99789120 con 0x7f31e4016b80 2023-10-05T19:40:30.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.494+0000 7f31ee7f2e40 20 add_watcher() i=0 2023-10-05T19:40:30.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.494+0000 7f31ee7f2e40 20 add_watcher() i=1 2023-10-05T19:40:30.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.494+0000 7f31ee7f2e40 20 add_watcher() i=2 2023-10-05T19:40:30.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.494+0000 7f31ee7f2e40 20 add_watcher() i=3 2023-10-05T19:40:30.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.494+0000 7f31ee7f2e40 20 add_watcher() i=4 2023-10-05T19:40:30.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.494+0000 7f31ee7f2e40 20 add_watcher() i=5 2023-10-05T19:40:30.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.494+0000 7f31ee7f2e40 20 add_watcher() i=6 2023-10-05T19:40:30.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.498+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94264222058464] v28'172 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f31dc00a5a0 con 0x7f31e4016b80 2023-10-05T19:40:30.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.498+0000 7f31ee7f2e40 20 add_watcher() i=7 2023-10-05T19:40:30.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.498+0000 7f31ee7f2e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:30.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.498+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb9978c1c0 con 0x55bb996b8c20 2023-10-05T19:40:30.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.498+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f31dc00a5a0 con 0x55bb996b8c20 2023-10-05T19:40:30.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.498+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94264222059568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99789570 con 0x55bb996b8c20 2023-10-05T19:40:30.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.502+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94264222059568] v28'161 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f31dc00a5a0 con 0x55bb996b8c20 2023-10-05T19:40:30.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.502+0000 7f31bdfe3700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:30.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.502+0000 7f31ee7f2e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:30.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.502+0000 7f31ee7f2e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:30.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.502+0000 7f31ee7f2e40 1 Processor -- start 2023-10-05T19:40:30.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.502+0000 7f31ee7f2e40 1 -- start start 2023-10-05T19:40:30.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.502+0000 7f31ee7f2e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb997966b0 0x55bb99796a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.502+0000 7f31ee7f2e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb99796fc0 con 0x55bb997966b0 2023-10-05T19:40:30.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.502+0000 7f31ebb26700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb997966b0 0x55bb99796a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.502+0000 7f31ebb26700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb997966b0 0x55bb99796a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58810/0 (socket says 172.21.15.12:58810) 2023-10-05T19:40:30.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.502+0000 7f31ebb26700 1 -- 172.21.15.12:0/3766762886 learned_addr learned my addr 172.21.15.12:0/3766762886 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:30.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.506+0000 7f31ebb26700 1 -- 172.21.15.12:0/3766762886 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb99797100 con 0x55bb997966b0 2023-10-05T19:40:30.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.506+0000 7f31ebb26700 1 --2- 172.21.15.12:0/3766762886 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb997966b0 0x55bb99796a80 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x55bb996b9080 tx=0x7f31d4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baf7ec04e973956b server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.506+0000 7f31bd7e2700 1 -- 172.21.15.12:0/3766762886 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f31d40308b0 con 0x55bb997966b0 2023-10-05T19:40:30.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.506+0000 7f31bd7e2700 1 -- 172.21.15.12:0/3766762886 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31d4036040 con 0x55bb997966b0 2023-10-05T19:40:30.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.506+0000 7f31bd7e2700 1 -- 172.21.15.12:0/3766762886 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f31d4030a50 con 0x55bb997966b0 2023-10-05T19:40:30.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.506+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/3766762886 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb997966b0 msgr2=0x55bb99796a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.506+0000 7f31ee7f2e40 1 --2- 172.21.15.12:0/3766762886 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb997966b0 0x55bb99796a80 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x55bb996b9080 tx=0x7f31d4012000 comp rx=0 tx=0).stop 2023-10-05T19:40:30.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.506+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/3766762886 shutdown_connections 2023-10-05T19:40:30.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.506+0000 7f31ee7f2e40 1 --2- 172.21.15.12:0/3766762886 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb997966b0 0x55bb99796a80 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.506+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/3766762886 >> 172.21.15.12:0/3766762886 conn(0x55bb99791a70 msgr2=0x55bb9978fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:30.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.506+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/3766762886 shutdown_connections 2023-10-05T19:40:30.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.506+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/3766762886 wait complete. 2023-10-05T19:40:30.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.506+0000 7f31ee7f2e40 1 Processor -- start 2023-10-05T19:40:30.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.506+0000 7f31ee7f2e40 1 -- start start 2023-10-05T19:40:30.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.506+0000 7f31ee7f2e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb997966b0 0x55bb9979c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.506+0000 7f31ee7f2e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb9979d040 con 0x55bb997966b0 2023-10-05T19:40:30.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.506+0000 7f31ebb26700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb997966b0 0x55bb9979c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.510+0000 7f31ebb26700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb997966b0 0x55bb9979c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58816/0 (socket says 172.21.15.12:58816) 2023-10-05T19:40:30.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.510+0000 7f31ebb26700 1 -- 172.21.15.12:0/184472982 learned_addr learned my addr 172.21.15.12:0/184472982 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:30.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.510+0000 7f31ebb26700 1 -- 172.21.15.12:0/184472982 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb996b90b0 con 0x55bb997966b0 2023-10-05T19:40:30.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.510+0000 7f31ebb26700 1 --2- 172.21.15.12:0/184472982 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb997966b0 0x55bb9979c040 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f31d4012fd0 tx=0x7f31d4037000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.510+0000 7f31377fe700 1 -- 172.21.15.12:0/184472982 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f31d4027d40 con 0x55bb997966b0 2023-10-05T19:40:30.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.510+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/184472982 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bb9979c640 con 0x55bb997966b0 2023-10-05T19:40:30.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.510+0000 7f31377fe700 1 -- 172.21.15.12:0/184472982 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31d404c3f0 con 0x55bb997966b0 2023-10-05T19:40:30.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.510+0000 7f31377fe700 1 -- 172.21.15.12:0/184472982 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f31d404c5a0 con 0x55bb997966b0 2023-10-05T19:40:30.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.510+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/184472982 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bb9979ca80 con 0x55bb997966b0 2023-10-05T19:40:30.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.510+0000 7f31377fe700 1 -- 172.21.15.12:0/184472982 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f31d4055440 con 0x55bb997966b0 2023-10-05T19:40:30.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.510+0000 7f31377fe700 1 --2- 172.21.15.12:0/184472982 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f312c030c00 0x7f312c0330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.510+0000 7f31377fe700 1 -- 172.21.15.12:0/184472982 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f31d403c0d0 con 0x55bb997966b0 2023-10-05T19:40:30.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.510+0000 7f31ee7f2e40 5 note: GC not initialized 2023-10-05T19:40:30.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.510+0000 7f31ecb28700 1 --2- 172.21.15.12:0/184472982 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f312c030c00 0x7f312c0330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.514+0000 7f31ecb28700 1 --2- 172.21.15.12:0/184472982 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f312c030c00 0x7f312c0330b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x55bb9978c2d0 tx=0x7f31dc00ae20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.514+0000 7f3134ff9700 20 reqs_thread_entry: start 2023-10-05T19:40:30.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.514+0000 7f312a7fc700 20 reqs_thread_entry: start 2023-10-05T19:40:30.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.514+0000 7f31ee7f2e40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:30.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.514+0000 7f31ee7f2e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:30.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.514+0000 7f3128ff9700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:41:01 2023 2023-10-05T19:40:30.526 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:30.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.514+0000 7f31ee7f2e40 10 log_show_next pos 0 bl 0 off 0 eof 0 2023-10-05T19:40:30.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.514+0000 7f31ee7f2e40 1 --2- 172.21.15.12:0/184472982 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55bb998036d0 0x55bb99805ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.514+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/184472982 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 6.d 6:b5faf345:::2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1-myfoo:head [read 0~1048576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99805ff0 con 0x55bb998036d0 2023-10-05T19:40:30.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.514+0000 7f310ffff700 20 reqs_thread_entry: start 2023-10-05T19:40:30.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.514+0000 7f31ec327700 1 --2- 172.21.15.12:0/184472982 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55bb998036d0 0x55bb99805ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.518+0000 7f31ec327700 1 --2- 172.21.15.12:0/184472982 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55bb998036d0 0x55bb99805ab0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.518+0000 7f31ec327700 1 -- 172.21.15.12:0/184472982 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1-myfoo [read 0~3730 out=3730b] v0'0 uv32 ondisk = 0) v8 ==== 207+0+3730 (crc 0 0 0) 0x7f31e400d7f0 con 0x55bb998036d0 2023-10-05T19:40:30.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.518+0000 7f31ee7f2e40 10 read 3730 2023-10-05T19:40:30.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.518+0000 7f31ee7f2e40 10 log_show_next pos 3730 bl 3730 off 417 eof 1 2023-10-05T19:40:30.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.518+0000 7f31ee7f2e40 10 log_show_next pos 3730 bl 3730 off 922 eof 1 2023-10-05T19:40:30.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.518+0000 7f31ee7f2e40 10 log_show_next pos 3730 bl 3730 off 1442 eof 1 2023-10-05T19:40:30.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.518+0000 7f31ee7f2e40 10 log_show_next pos 3730 bl 3730 off 1947 eof 1 2023-10-05T19:40:30.529 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:40:30.529 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket_id": "93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1", 2023-10-05T19:40:30.529 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket_owner": "foo", 2023-10-05T19:40:30.529 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo", 2023-10-05T19:40:30.529 INFO:teuthology.orchestra.run.smithi012.stdout: "log_entries": [ 2023-10-05T19:40:30.529 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:30.530 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo", 2023-10-05T19:40:30.530 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:40:22.851124Z", 2023-10-05T19:40:30.530 INFO:teuthology.orchestra.run.smithi012.stdout: "time_local": "2023-10-05T19:40:22.851124+0000", 2023-10-05T19:40:30.530 INFO:teuthology.orchestra.run.smithi012.stdout: "remote_addr": "172.21.0.51", 2023-10-05T19:40:30.530 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:40:30.530 INFO:teuthology.orchestra.run.smithi012.stdout: "operation": "create_bucket", 2023-10-05T19:40:30.530 INFO:teuthology.orchestra.run.smithi012.stdout: "uri": "PUT /myfoo/ HTTP/1.1", 2023-10-05T19:40:30.530 INFO:teuthology.orchestra.run.smithi012.stdout: "http_status": "200", 2023-10-05T19:40:30.530 INFO:teuthology.orchestra.run.smithi012.stdout: "error_code": "", 2023-10-05T19:40:30.530 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:40:30.531 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:40:30.531 INFO:teuthology.orchestra.run.smithi012.stdout: "object_size": 0, 2023-10-05T19:40:30.531 INFO:teuthology.orchestra.run.smithi012.stdout: "total_time": 19, 2023-10-05T19:40:30.531 INFO:teuthology.orchestra.run.smithi012.stdout: "user_agent": "Boto/2.49.0 Python/3.8.10 Linux/5.4.0-163-generic", 2023-10-05T19:40:30.531 INFO:teuthology.orchestra.run.smithi012.stdout: "referrer": "", 2023-10-05T19:40:30.531 INFO:teuthology.orchestra.run.smithi012.stdout: "trans_id": "tx000001f2c404aa7b1584e-00651f1126-107b-default", 2023-10-05T19:40:30.531 INFO:teuthology.orchestra.run.smithi012.stdout: "authentication_type": "Local", 2023-10-05T19:40:30.531 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key_id": "9te6NH5mcdcq0Tc5i8i1", 2023-10-05T19:40:30.531 INFO:teuthology.orchestra.run.smithi012.stdout: "temp_url": false 2023-10-05T19:40:30.531 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:30.531 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:30.532 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo", 2023-10-05T19:40:30.532 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:40:24.151094Z", 2023-10-05T19:40:30.532 INFO:teuthology.orchestra.run.smithi012.stdout: "time_local": "2023-10-05T19:40:24.151094+0000", 2023-10-05T19:40:30.532 INFO:teuthology.orchestra.run.smithi012.stdout: "remote_addr": "172.21.0.51", 2023-10-05T19:40:30.532 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:40:30.532 INFO:teuthology.orchestra.run.smithi012.stdout: "operation": "put_obj", 2023-10-05T19:40:30.532 INFO:teuthology.orchestra.run.smithi012.stdout: "uri": "PUT /myfoo/f97c5d29941bfb1b2fdab0874906ab82 HTTP/1.1", 2023-10-05T19:40:30.532 INFO:teuthology.orchestra.run.smithi012.stdout: "http_status": "200", 2023-10-05T19:40:30.532 INFO:teuthology.orchestra.run.smithi012.stdout: "error_code": "", 2023-10-05T19:40:30.532 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:40:30.533 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 3, 2023-10-05T19:40:30.533 INFO:teuthology.orchestra.run.smithi012.stdout: "object_size": 3, 2023-10-05T19:40:30.533 INFO:teuthology.orchestra.run.smithi012.stdout: "total_time": 11, 2023-10-05T19:40:30.533 INFO:teuthology.orchestra.run.smithi012.stdout: "user_agent": "Boto/2.49.0 Python/3.8.10 Linux/5.4.0-163-generic", 2023-10-05T19:40:30.533 INFO:teuthology.orchestra.run.smithi012.stdout: "referrer": "", 2023-10-05T19:40:30.533 INFO:teuthology.orchestra.run.smithi012.stdout: "trans_id": "tx00000051c2922633090b7-00651f1128-107b-default", 2023-10-05T19:40:30.533 INFO:teuthology.orchestra.run.smithi012.stdout: "authentication_type": "Local", 2023-10-05T19:40:30.533 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key_id": "9te6NH5mcdcq0Tc5i8i1", 2023-10-05T19:40:30.533 INFO:teuthology.orchestra.run.smithi012.stdout: "temp_url": false 2023-10-05T19:40:30.533 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:30.533 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:30.533 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo", 2023-10-05T19:40:30.534 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:40:24.659082Z", 2023-10-05T19:40:30.534 INFO:teuthology.orchestra.run.smithi012.stdout: "time_local": "2023-10-05T19:40:24.659082+0000", 2023-10-05T19:40:30.534 INFO:teuthology.orchestra.run.smithi012.stdout: "remote_addr": "172.21.0.51", 2023-10-05T19:40:30.534 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:40:30.534 INFO:teuthology.orchestra.run.smithi012.stdout: "operation": "delete_obj", 2023-10-05T19:40:30.534 INFO:teuthology.orchestra.run.smithi012.stdout: "uri": "DELETE /myfoo/f97c5d29941bfb1b2fdab0874906ab82 HTTP/1.1", 2023-10-05T19:40:30.534 INFO:teuthology.orchestra.run.smithi012.stdout: "http_status": "204", 2023-10-05T19:40:30.534 INFO:teuthology.orchestra.run.smithi012.stdout: "error_code": "NoContent", 2023-10-05T19:40:30.534 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:40:30.534 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:40:30.535 INFO:teuthology.orchestra.run.smithi012.stdout: "object_size": 0, 2023-10-05T19:40:30.535 INFO:teuthology.orchestra.run.smithi012.stdout: "total_time": 7, 2023-10-05T19:40:30.535 INFO:teuthology.orchestra.run.smithi012.stdout: "user_agent": "Boto/2.49.0 Python/3.8.10 Linux/5.4.0-163-generic", 2023-10-05T19:40:30.535 INFO:teuthology.orchestra.run.smithi012.stdout: "referrer": "", 2023-10-05T19:40:30.535 INFO:teuthology.orchestra.run.smithi012.stdout: "trans_id": "tx000003856fbfc4c17c993-00651f1128-107b-default", 2023-10-05T19:40:30.535 INFO:teuthology.orchestra.run.smithi012.stdout: "authentication_type": "Local", 2023-10-05T19:40:30.535 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key_id": "9te6NH5mcdcq0Tc5i8i1", 2023-10-05T19:40:30.535 INFO:teuthology.orchestra.run.smithi012.stdout: "temp_url": false 2023-10-05T19:40:30.535 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:30.535 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:30.536 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo", 2023-10-05T19:40:30.536 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:40:25.123071Z", 2023-10-05T19:40:30.536 INFO:teuthology.orchestra.run.smithi012.stdout: "time_local": "2023-10-05T19:40:25.123071+0000", 2023-10-05T19:40:30.536 INFO:teuthology.orchestra.run.smithi012.stdout: "remote_addr": "172.21.0.51", 2023-10-05T19:40:30.536 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:40:30.536 INFO:teuthology.orchestra.run.smithi012.stdout: "operation": "put_obj", 2023-10-05T19:40:30.536 INFO:teuthology.orchestra.run.smithi012.stdout: "uri": "PUT /myfoo/b8a9f715dbb64fd5c56e7783c6820a61 HTTP/1.1", 2023-10-05T19:40:30.536 INFO:teuthology.orchestra.run.smithi012.stdout: "http_status": "200", 2023-10-05T19:40:30.536 INFO:teuthology.orchestra.run.smithi012.stdout: "error_code": "", 2023-10-05T19:40:30.537 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:40:30.537 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 3, 2023-10-05T19:40:30.537 INFO:teuthology.orchestra.run.smithi012.stdout: "object_size": 3, 2023-10-05T19:40:30.537 INFO:teuthology.orchestra.run.smithi012.stdout: "total_time": 7, 2023-10-05T19:40:30.537 INFO:teuthology.orchestra.run.smithi012.stdout: "user_agent": "Boto/2.49.0 Python/3.8.10 Linux/5.4.0-163-generic", 2023-10-05T19:40:30.537 INFO:teuthology.orchestra.run.smithi012.stdout: "referrer": "", 2023-10-05T19:40:30.537 INFO:teuthology.orchestra.run.smithi012.stdout: "trans_id": "tx000004fabf97b086166df-00651f1129-107b-default", 2023-10-05T19:40:30.537 INFO:teuthology.orchestra.run.smithi012.stdout: "authentication_type": "Local", 2023-10-05T19:40:30.537 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key_id": "9te6NH5mcdcq0Tc5i8i1", 2023-10-05T19:40:30.537 INFO:teuthology.orchestra.run.smithi012.stdout: "temp_url": false 2023-10-05T19:40:30.537 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:30.537 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:30.538 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo", 2023-10-05T19:40:30.538 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:40:25.131071Z", 2023-10-05T19:40:30.538 INFO:teuthology.orchestra.run.smithi012.stdout: "time_local": "2023-10-05T19:40:25.131071+0000", 2023-10-05T19:40:30.538 INFO:teuthology.orchestra.run.smithi012.stdout: "remote_addr": "172.21.0.51", 2023-10-05T19:40:30.538 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:40:30.538 INFO:teuthology.orchestra.run.smithi012.stdout: "operation": "delete_obj", 2023-10-05T19:40:30.538 INFO:teuthology.orchestra.run.smithi012.stdout: "uri": "DELETE /myfoo/b8a9f715dbb64fd5c56e7783c6820a61 HTTP/1.1", 2023-10-05T19:40:30.538 INFO:teuthology.orchestra.run.smithi012.stdout: "http_status": "204", 2023-10-05T19:40:30.538 INFO:teuthology.orchestra.run.smithi012.stdout: "error_code": "NoContent", 2023-10-05T19:40:30.538 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:40:30.538 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:40:30.538 INFO:teuthology.orchestra.run.smithi012.stdout: "object_size": 0, 2023-10-05T19:40:30.539 INFO:teuthology.orchestra.run.smithi012.stdout: "total_time": 7, 2023-10-05T19:40:30.539 INFO:teuthology.orchestra.run.smithi012.stdout: "user_agent": "Boto/2.49.0 Python/3.8.10 Linux/5.4.0-163-generic", 2023-10-05T19:40:30.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.518+0000 7f31ee7f2e40 10 log_show_next pos 3730 bl 3730 off 2467 eof 1 2023-10-05T19:40:30.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.518+0000 7f31ee7f2e40 10 log_show_next pos 3730 bl 3730 off 2888 eof 1 2023-10-05T19:40:30.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.518+0000 7f31ee7f2e40 10 log_show_next pos 3730 bl 3730 off 3315 eof 1 2023-10-05T19:40:30.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.518+0000 7f31ee7f2e40 10 log_show_next pos 3730 bl 3730 off 3730 eof 1 2023-10-05T19:40:30.540 INFO:teuthology.orchestra.run.smithi012.stdout: "referrer": "", 2023-10-05T19:40:30.540 INFO:teuthology.orchestra.run.smithi012.stdout: "trans_id": "tx00000256d8363dbe48ef2-00651f1129-107b-default", 2023-10-05T19:40:30.540 INFO:teuthology.orchestra.run.smithi012.stdout: "authentication_type": "Local", 2023-10-05T19:40:30.540 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key_id": "9te6NH5mcdcq0Tc5i8i1", 2023-10-05T19:40:30.540 INFO:teuthology.orchestra.run.smithi012.stdout: "temp_url": false 2023-10-05T19:40:30.540 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:30.540 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:30.540 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo", 2023-10-05T19:40:30.541 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:40:28.079003Z", 2023-10-05T19:40:30.541 INFO:teuthology.orchestra.run.smithi012.stdout: "time_local": "2023-10-05T19:40:28.079003+0000", 2023-10-05T19:40:30.541 INFO:teuthology.orchestra.run.smithi012.stdout: "remote_addr": "172.21.0.51", 2023-10-05T19:40:30.541 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:40:30.541 INFO:teuthology.orchestra.run.smithi012.stdout: "operation": "put_obj", 2023-10-05T19:40:30.541 INFO:teuthology.orchestra.run.smithi012.stdout: "uri": "PUT /myfoo/four HTTP/1.1", 2023-10-05T19:40:30.541 INFO:teuthology.orchestra.run.smithi012.stdout: "http_status": "200", 2023-10-05T19:40:30.541 INFO:teuthology.orchestra.run.smithi012.stdout: "error_code": "", 2023-10-05T19:40:30.541 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:40:30.541 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 4, 2023-10-05T19:40:30.541 INFO:teuthology.orchestra.run.smithi012.stdout: "object_size": 4, 2023-10-05T19:40:30.541 INFO:teuthology.orchestra.run.smithi012.stdout: "total_time": 11, 2023-10-05T19:40:30.542 INFO:teuthology.orchestra.run.smithi012.stdout: "user_agent": "Boto/2.49.0 Python/3.8.10 Linux/5.4.0-163-generic", 2023-10-05T19:40:30.542 INFO:teuthology.orchestra.run.smithi012.stdout: "referrer": "", 2023-10-05T19:40:30.542 INFO:teuthology.orchestra.run.smithi012.stdout: "trans_id": "tx000002b817541d26d5b00-00651f112c-107b-default", 2023-10-05T19:40:30.542 INFO:teuthology.orchestra.run.smithi012.stdout: "authentication_type": "Local", 2023-10-05T19:40:30.542 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key_id": "9te6NH5mcdcq0Tc5i8i1", 2023-10-05T19:40:30.542 INFO:teuthology.orchestra.run.smithi012.stdout: "temp_url": false 2023-10-05T19:40:30.542 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:30.542 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:30.542 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo", 2023-10-05T19:40:30.542 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:40:28.091003Z", 2023-10-05T19:40:30.542 INFO:teuthology.orchestra.run.smithi012.stdout: "time_local": "2023-10-05T19:40:28.091003+0000", 2023-10-05T19:40:30.542 INFO:teuthology.orchestra.run.smithi012.stdout: "remote_addr": "172.21.0.51", 2023-10-05T19:40:30.543 INFO:teuthology.orchestra.run.smithi012.stdout: "object_owner": "foo", 2023-10-05T19:40:30.543 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:40:30.543 INFO:teuthology.orchestra.run.smithi012.stdout: "operation": "get_obj", 2023-10-05T19:40:30.543 INFO:teuthology.orchestra.run.smithi012.stdout: "uri": "GET /myfoo/four HTTP/1.1", 2023-10-05T19:40:30.543 INFO:teuthology.orchestra.run.smithi012.stdout: "http_status": "200", 2023-10-05T19:40:30.543 INFO:teuthology.orchestra.run.smithi012.stdout: "error_code": "", 2023-10-05T19:40:30.543 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 4, 2023-10-05T19:40:30.543 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:40:30.543 INFO:teuthology.orchestra.run.smithi012.stdout: "object_size": 4, 2023-10-05T19:40:30.543 INFO:teuthology.orchestra.run.smithi012.stdout: "total_time": 3, 2023-10-05T19:40:30.543 INFO:teuthology.orchestra.run.smithi012.stdout: "user_agent": "Boto/2.49.0 Python/3.8.10 Linux/5.4.0-163-generic", 2023-10-05T19:40:30.544 INFO:teuthology.orchestra.run.smithi012.stdout: "referrer": "", 2023-10-05T19:40:30.544 INFO:teuthology.orchestra.run.smithi012.stdout: "trans_id": "tx00000e8bb9242c2140f94-00651f112c-107b-default", 2023-10-05T19:40:30.544 INFO:teuthology.orchestra.run.smithi012.stdout: "authentication_type": "Local", 2023-10-05T19:40:30.544 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key_id": "9te6NH5mcdcq0Tc5i8i1", 2023-10-05T19:40:30.544 INFO:teuthology.orchestra.run.smithi012.stdout: "temp_url": false 2023-10-05T19:40:30.544 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:30.544 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:30.544 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo", 2023-10-05T19:40:30.544 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:40:28.095003Z", 2023-10-05T19:40:30.544 INFO:teuthology.orchestra.run.smithi012.stdout: "time_local": "2023-10-05T19:40:28.095003+0000", 2023-10-05T19:40:30.544 INFO:teuthology.orchestra.run.smithi012.stdout: "remote_addr": "172.21.0.51", 2023-10-05T19:40:30.545 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:40:30.545 INFO:teuthology.orchestra.run.smithi012.stdout: "operation": "list_bucket", 2023-10-05T19:40:30.545 INFO:teuthology.orchestra.run.smithi012.stdout: "uri": "GET /myfoo/ HTTP/1.1", 2023-10-05T19:40:30.545 INFO:teuthology.orchestra.run.smithi012.stdout: "http_status": "200", 2023-10-05T19:40:30.545 INFO:teuthology.orchestra.run.smithi012.stdout: "error_code": "", 2023-10-05T19:40:30.545 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 515, 2023-10-05T19:40:30.545 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:40:30.545 INFO:teuthology.orchestra.run.smithi012.stdout: "object_size": 0, 2023-10-05T19:40:30.545 INFO:teuthology.orchestra.run.smithi012.stdout: "total_time": 3, 2023-10-05T19:40:30.545 INFO:teuthology.orchestra.run.smithi012.stdout: "user_agent": "Boto/2.49.0 Python/3.8.10 Linux/5.4.0-163-generic", 2023-10-05T19:40:30.545 INFO:teuthology.orchestra.run.smithi012.stdout: "referrer": "", 2023-10-05T19:40:30.545 INFO:teuthology.orchestra.run.smithi012.stdout: "trans_id": "tx0000076f53d2e61343d34-00651f112c-107b-default", 2023-10-05T19:40:30.545 INFO:teuthology.orchestra.run.smithi012.stdout: "authentication_type": "Local", 2023-10-05T19:40:30.546 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key_id": "9te6NH5mcdcq0Tc5i8i1", 2023-10-05T19:40:30.546 INFO:teuthology.orchestra.run.smithi012.stdout: "temp_url": false 2023-10-05T19:40:30.546 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:30.546 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:30.546 INFO:teuthology.orchestra.run.smithi012.stdout: "log_sum": { 2023-10-05T19:40:30.546 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 519, 2023-10-05T19:40:30.546 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 10, 2023-10-05T19:40:30.546 INFO:teuthology.orchestra.run.smithi012.stdout: "total_time": 68, 2023-10-05T19:40:30.546 INFO:teuthology.orchestra.run.smithi012.stdout: "total_entries": 8 2023-10-05T19:40:30.546 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:30.546 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:40:30.547 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:40:30.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.518+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:102 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94264222037872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb997c2d00 con 0x55bb99778b10 2023-10-05T19:40:30.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.522+0000 7f31ec327700 1 -- 172.21.15.12:0/4213286051 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(102 notify.0 [watch unwatch cookie 94264222037872] v28'173 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f31e400d7f0 con 0x55bb99778b10 2023-10-05T19:40:30.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.522+0000 7f31ee7f2e40 20 remove_watcher() i=0 2023-10-05T19:40:30.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.522+0000 7f31ee7f2e40 2 removed watcher, disabling cache 2023-10-05T19:40:30.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.522+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94264222040448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99783770 con 0x55bb996b8c20 2023-10-05T19:40:30.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.526+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(103 notify.1 [watch unwatch cookie 94264222040448] v28'173 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f31dc03e070 con 0x55bb996b8c20 2023-10-05T19:40:30.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.526+0000 7f31ee7f2e40 20 remove_watcher() i=1 2023-10-05T19:40:30.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.526+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:104 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94264222041552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99784180 con 0x55bb9977d490 2023-10-05T19:40:30.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.530+0000 7f31ebb26700 1 -- 172.21.15.12:0/4213286051 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(104 notify.2 [watch unwatch cookie 94264222041552] v28'173 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f31d4040020 con 0x55bb9977d490 2023-10-05T19:40:30.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.530+0000 7f31ee7f2e40 20 remove_watcher() i=2 2023-10-05T19:40:30.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.530+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:105 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94264222050224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99784670 con 0x55bb99778b10 2023-10-05T19:40:30.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.530+0000 7f31ec327700 1 -- 172.21.15.12:0/4213286051 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(105 notify.3 [watch unwatch cookie 94264222050224] v28'173 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f31e400d7f0 con 0x55bb99778b10 2023-10-05T19:40:30.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.530+0000 7f31ee7f2e40 20 remove_watcher() i=3 2023-10-05T19:40:30.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.530+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:106 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94264222046752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99784670 con 0x55bb996b8c20 2023-10-05T19:40:30.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.534+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(106 notify.4 [watch unwatch cookie 94264222046752] v28'173 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f31dc03e070 con 0x55bb996b8c20 2023-10-05T19:40:30.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.534+0000 7f31ee7f2e40 20 remove_watcher() i=4 2023-10-05T19:40:30.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.534+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:107 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94264222054784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99784670 con 0x7f31e4016b80 2023-10-05T19:40:30.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.538+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(107 notify.5 [watch unwatch cookie 94264222054784] v28'173 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f31dc03e070 con 0x7f31e4016b80 2023-10-05T19:40:30.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.538+0000 7f31ee7f2e40 20 remove_watcher() i=5 2023-10-05T19:40:30.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.538+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:108 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94264222051328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99784670 con 0x7f31e4016b80 2023-10-05T19:40:30.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.542+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(108 notify.6 [watch unwatch cookie 94264222051328] v28'173 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f31dc03e070 con 0x7f31e4016b80 2023-10-05T19:40:30.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.542+0000 7f31ee7f2e40 20 remove_watcher() i=6 2023-10-05T19:40:30.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.542+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:109 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94264222058464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb99784670 con 0x7f31e4016b80 2023-10-05T19:40:30.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.542+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(109 notify.7 [watch unwatch cookie 94264222058464] v28'173 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f31dc03e070 con 0x7f31e4016b80 2023-10-05T19:40:30.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.542+0000 7f31ee7f2e40 20 remove_watcher() i=7 2023-10-05T19:40:30.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.546+0000 7f31bdfe3700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:30.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.546+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:110 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94264222059568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55bb997c5480 con 0x55bb996b8c20 2023-10-05T19:40:30.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.550+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(110 data_loggenerations_metadata [watch unwatch cookie 94264222059568] v28'162 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f31dc03e070 con 0x55bb996b8c20 2023-10-05T19:40:30.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.550+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f31e4016b80 msgr2=0x7f31e40420e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.550+0000 7f31ee7f2e40 1 --2- 172.21.15.12:0/4213286051 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f31e4016b80 0x7f31e40420e0 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.550+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55bb996b8c20 msgr2=0x55bb997769f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.550+0000 7f31ee7f2e40 1 --2- 172.21.15.12:0/4213286051 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55bb996b8c20 0x55bb997769f0 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.550+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55bb99778b10 msgr2=0x55bb9977aef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.550+0000 7f31ee7f2e40 1 --2- 172.21.15.12:0/4213286051 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55bb99778b10 0x55bb9977aef0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.550+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55bb9977d490 msgr2=0x55bb9977f870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.550+0000 7f31ee7f2e40 1 --2- 172.21.15.12:0/4213286051 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55bb9977d490 0x55bb9977f870 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.550+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f31cc030bf0 msgr2=0x7f31cc0330a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.550+0000 7f31ee7f2e40 1 --2- 172.21.15.12:0/4213286051 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f31cc030bf0 0x7f31cc0330a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f31d4005a90 tx=0x7f31d4015000 comp rx=0 tx=0).stop 2023-10-05T19:40:30.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.550+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb996b7910 msgr2=0x55bb9976af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.550+0000 7f31ecb28700 1 -- 172.21.15.12:0/4213286051 reap_dead start 2023-10-05T19:40:30.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.550+0000 7f31ee7f2e40 1 --2- 172.21.15.12:0/4213286051 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb996b7910 0x55bb9976af60 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f31e4000c00 tx=0x7f31e4016000 comp rx=0 tx=0).stop 2023-10-05T19:40:30.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.550+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 shutdown_connections 2023-10-05T19:40:30.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.550+0000 7f31ee7f2e40 1 --2- 172.21.15.12:0/4213286051 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb996b7910 0x55bb9976af60 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.550+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 >> 172.21.15.12:0/4213286051 conn(0x55bb9973e850 msgr2=0x55bb9975e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:30.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.550+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 shutdown_connections 2023-10-05T19:40:30.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.550+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/4213286051 wait complete. 2023-10-05T19:40:30.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.550+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/184472982 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55bb998036d0 msgr2=0x55bb99805ab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.550+0000 7f31ee7f2e40 1 --2- 172.21.15.12:0/184472982 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55bb998036d0 0x55bb99805ab0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.550+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/184472982 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f312c030c00 msgr2=0x7f312c0330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.550+0000 7f31ee7f2e40 1 --2- 172.21.15.12:0/184472982 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f312c030c00 0x7f312c0330b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x55bb9978c2d0 tx=0x7f31dc00ae20 comp rx=0 tx=0).stop 2023-10-05T19:40:30.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.550+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/184472982 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb997966b0 msgr2=0x55bb9979c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.550+0000 7f31ee7f2e40 1 --2- 172.21.15.12:0/184472982 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb997966b0 0x55bb9979c040 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f31d4012fd0 tx=0x7f31d4037000 comp rx=0 tx=0).stop 2023-10-05T19:40:30.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.550+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/184472982 shutdown_connections 2023-10-05T19:40:30.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.550+0000 7f31ee7f2e40 1 --2- 172.21.15.12:0/184472982 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55bb998036d0 0x55bb99805ab0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.550+0000 7f31ee7f2e40 1 --2- 172.21.15.12:0/184472982 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f312c030c00 0x7f312c0330b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.550+0000 7f31ee7f2e40 1 --2- 172.21.15.12:0/184472982 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bb997966b0 0x55bb9979c040 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.550+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/184472982 >> 172.21.15.12:0/184472982 conn(0x55bb99791a70 msgr2=0x55bb9978fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:30.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.550+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/184472982 shutdown_connections 2023-10-05T19:40:30.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.550+0000 7f31ee7f2e40 1 -- 172.21.15.12:0/184472982 wait complete. 2023-10-05T19:40:30.567 DEBUG:tasks.util.rgw: json result: {'bucket_id': '93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1', 'bucket_owner': 'foo', 'bucket': 'myfoo', 'log_entries': [{'bucket': 'myfoo', 'time': '2023-10-05T19:40:22.851124Z', 'time_local': '2023-10-05T19:40:22.851124+0000', 'remote_addr': '172.21.0.51', 'user': 'foo', 'operation': 'create_bucket', 'uri': 'PUT /myfoo/ HTTP/1.1', 'http_status': '200', 'error_code': '', 'bytes_sent': 0, 'bytes_received': 0, 'object_size': 0, 'total_time': 19, 'user_agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-163-generic', 'referrer': '', 'trans_id': 'tx000001f2c404aa7b1584e-00651f1126-107b-default', 'authentication_type': 'Local', 'access_key_id': '9te6NH5mcdcq0Tc5i8i1', 'temp_url': False}, {'bucket': 'myfoo', 'time': '2023-10-05T19:40:24.151094Z', 'time_local': '2023-10-05T19:40:24.151094+0000', 'remote_addr': '172.21.0.51', 'user': 'foo', 'operation': 'put_obj', 'uri': 'PUT /myfoo/f97c5d29941bfb1b2fdab0874906ab82 HTTP/1.1', 'http_status': '200', 'error_code': '', 'bytes_sent': 0, 'bytes_received': 3, 'object_size': 3, 'total_time': 11, 'user_agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-163-generic', 'referrer': '', 'trans_id': 'tx00000051c2922633090b7-00651f1128-107b-default', 'authentication_type': 'Local', 'access_key_id': '9te6NH5mcdcq0Tc5i8i1', 'temp_url': False}, {'bucket': 'myfoo', 'time': '2023-10-05T19:40:24.659082Z', 'time_local': '2023-10-05T19:40:24.659082+0000', 'remote_addr': '172.21.0.51', 'user': 'foo', 'operation': 'delete_obj', 'uri': 'DELETE /myfoo/f97c5d29941bfb1b2fdab0874906ab82 HTTP/1.1', 'http_status': '204', 'error_code': 'NoContent', 'bytes_sent': 0, 'bytes_received': 0, 'object_size': 0, 'total_time': 7, 'user_agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-163-generic', 'referrer': '', 'trans_id': 'tx000003856fbfc4c17c993-00651f1128-107b-default', 'authentication_type': 'Local', 'access_key_id': '9te6NH5mcdcq0Tc5i8i1', 'temp_url': False}, {'bucket': 'myfoo', 'time': '2023-10-05T19:40:25.123071Z', 'time_local': '2023-10-05T19:40:25.123071+0000', 'remote_addr': '172.21.0.51', 'user': 'foo', 'operation': 'put_obj', 'uri': 'PUT /myfoo/b8a9f715dbb64fd5c56e7783c6820a61 HTTP/1.1', 'http_status': '200', 'error_code': '', 'bytes_sent': 0, 'bytes_received': 3, 'object_size': 3, 'total_time': 7, 'user_agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-163-generic', 'referrer': '', 'trans_id': 'tx000004fabf97b086166df-00651f1129-107b-default', 'authentication_type': 'Local', 'access_key_id': '9te6NH5mcdcq0Tc5i8i1', 'temp_url': False}, {'bucket': 'myfoo', 'time': '2023-10-05T19:40:25.131071Z', 'time_local': '2023-10-05T19:40:25.131071+0000', 'remote_addr': '172.21.0.51', 'user': 'foo', 'operation': 'delete_obj', 'uri': 'DELETE /myfoo/b8a9f715dbb64fd5c56e7783c6820a61 HTTP/1.1', 'http_status': '204', 'error_code': 'NoContent', 'bytes_sent': 0, 'bytes_received': 0, 'object_size': 0, 'total_time': 7, 'user_agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-163-generic', 'referrer': '', 'trans_id': 'tx00000256d8363dbe48ef2-00651f1129-107b-default', 'authentication_type': 'Local', 'access_key_id': '9te6NH5mcdcq0Tc5i8i1', 'temp_url': False}, {'bucket': 'myfoo', 'time': '2023-10-05T19:40:28.079003Z', 'time_local': '2023-10-05T19:40:28.079003+0000', 'remote_addr': '172.21.0.51', 'user': 'foo', 'operation': 'put_obj', 'uri': 'PUT /myfoo/four HTTP/1.1', 'http_status': '200', 'error_code': '', 'bytes_sent': 0, 'bytes_received': 4, 'object_size': 4, 'total_time': 11, 'user_agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-163-generic', 'referrer': '', 'trans_id': 'tx000002b817541d26d5b00-00651f112c-107b-default', 'authentication_type': 'Local', 'access_key_id': '9te6NH5mcdcq0Tc5i8i1', 'temp_url': False}, {'bucket': 'myfoo', 'time': '2023-10-05T19:40:28.091003Z', 'time_local': '2023-10-05T19:40:28.091003+0000', 'remote_addr': '172.21.0.51', 'object_owner': 'foo', 'user': 'foo', 'operation': 'get_obj', 'uri': 'GET /myfoo/four HTTP/1.1', 'http_status': '200', 'error_code': '', 'bytes_sent': 4, 'bytes_received': 0, 'object_size': 4, 'total_time': 3, 'user_agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-163-generic', 'referrer': '', 'trans_id': 'tx00000e8bb9242c2140f94-00651f112c-107b-default', 'authentication_type': 'Local', 'access_key_id': '9te6NH5mcdcq0Tc5i8i1', 'temp_url': False}, {'bucket': 'myfoo', 'time': '2023-10-05T19:40:28.095003Z', 'time_local': '2023-10-05T19:40:28.095003+0000', 'remote_addr': '172.21.0.51', 'user': 'foo', 'operation': 'list_bucket', 'uri': 'GET /myfoo/ HTTP/1.1', 'http_status': '200', 'error_code': '', 'bytes_sent': 515, 'bytes_received': 0, 'object_size': 0, 'total_time': 3, 'user_agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-163-generic', 'referrer': '', 'trans_id': 'tx0000076f53d2e61343d34-00651f112c-107b-default', 'authentication_type': 'Local', 'access_key_id': '9te6NH5mcdcq0Tc5i8i1', 'temp_url': False}], 'log_sum': {'bytes_sent': 519, 'bytes_received': 10, 'total_time': 68, 'total_entries': 8}} 2023-10-05T19:40:30.567 DEBUG:tasks.radosgw_admin:checking log entry: 2023-10-05T19:40:30.567 DEBUG:tasks.radosgw_admin:checking log entry: 2023-10-05T19:40:30.567 DEBUG:tasks.radosgw_admin:checking log entry: 2023-10-05T19:40:30.567 DEBUG:tasks.radosgw_admin:checking log entry: 2023-10-05T19:40:30.567 DEBUG:tasks.radosgw_admin:checking log entry: 2023-10-05T19:40:30.567 DEBUG:tasks.radosgw_admin:checking log entry: 2023-10-05T19:40:30.567 DEBUG:tasks.radosgw_admin:checking log entry: 2023-10-05T19:40:30.567 DEBUG:tasks.radosgw_admin:checking log entry: 2023-10-05T19:40:30.568 INFO:tasks.util.rgw:rgwadmin: client.0 : ['log', 'rm', '--object', '2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1-myfoo'] 2023-10-05T19:40:30.568 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'log', 'rm', '--object', '2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1-myfoo'] 2023-10-05T19:40:30.568 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph log rm --object 2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1-myfoo 2023-10-05T19:40:30.605 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:30.605 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:30.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.598+0000 7fc91ff5ae40 1 Processor -- start 2023-10-05T19:40:30.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.598+0000 7fc91ff5ae40 1 -- start start 2023-10-05T19:40:30.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.598+0000 7fc91ff5ae40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x561c684c25f0 0x561c684c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.598+0000 7fc91ff5ae40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x561c68408bd0 con 0x561c684be010 2023-10-05T19:40:30.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.598+0000 7fc91ff5ae40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x561c683c5ee0 con 0x561c684c25f0 2023-10-05T19:40:30.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.598+0000 7fc91d28e700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x561c684c25f0 0x561c684c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.598+0000 7fc91da8f700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x561c684be010 0x561c684be3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:35740/0 (socket says 172.21.15.12:35740) 2023-10-05T19:40:30.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.598+0000 7fc91d28e700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x561c684c25f0 0x561c684c49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58818/0 (socket says 172.21.15.12:58818) 2023-10-05T19:40:30.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.598+0000 7fc91da8f700 1 -- 172.21.15.12:0/1827808341 learned_addr learned my addr 172.21.15.12:0/1827808341 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:30.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.598+0000 7fc91d28e700 1 -- 172.21.15.12:0/1827808341 >> v1:172.21.15.8:6789/0 conn(0x561c684be010 legacy=0x561c684be3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.598+0000 7fc91d28e700 1 -- 172.21.15.12:0/1827808341 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c684bec20 con 0x561c684c25f0 2023-10-05T19:40:30.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.598+0000 7fc91d28e700 1 --2- 172.21.15.12:0/1827808341 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c684c25f0 0x561c684c49d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc91800a520 tx=0x7fc918005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0b4301468b377ef server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.598+0000 7fc91ca8d700 1 -- 172.21.15.12:0/1827808341 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc91800f590 con 0x561c684c25f0 2023-10-05T19:40:30.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.598+0000 7fc91ca8d700 1 -- 172.21.15.12:0/1827808341 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc91800f6f0 con 0x561c684c25f0 2023-10-05T19:40:30.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.598+0000 7fc91ca8d700 1 -- 172.21.15.12:0/1827808341 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc91800f880 con 0x561c684c25f0 2023-10-05T19:40:30.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.598+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/1827808341 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c684c25f0 msgr2=0x561c684c49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.598+0000 7fc91ff5ae40 1 --2- 172.21.15.12:0/1827808341 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c684c25f0 0x561c684c49d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc91800a520 tx=0x7fc918005d40 comp rx=0 tx=0).stop 2023-10-05T19:40:30.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.598+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/1827808341 shutdown_connections 2023-10-05T19:40:30.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.598+0000 7fc91ff5ae40 1 --2- 172.21.15.12:0/1827808341 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c684c25f0 0x561c684c49d0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.598+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/1827808341 >> 172.21.15.12:0/1827808341 conn(0x561c68431860 msgr2=0x561c68431c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:30.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.598+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/1827808341 shutdown_connections 2023-10-05T19:40:30.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.598+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/1827808341 wait complete. 2023-10-05T19:40:30.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.602+0000 7fc91ff5ae40 1 Processor -- start 2023-10-05T19:40:30.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.602+0000 7fc91ff5ae40 1 -- start start 2023-10-05T19:40:30.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.602+0000 7fc91ff5ae40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c684be010 0x561c684dd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.602+0000 7fc91ff5ae40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c68389e30 con 0x561c684be010 2023-10-05T19:40:30.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.602+0000 7fc91da8f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c684be010 0x561c684dd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.602+0000 7fc91da8f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c684be010 0x561c684dd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58834/0 (socket says 172.21.15.12:58834) 2023-10-05T19:40:30.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.602+0000 7fc91da8f700 1 -- 172.21.15.12:0/167491551 learned_addr learned my addr 172.21.15.12:0/167491551 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:30.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.602+0000 7fc91da8f700 1 -- 172.21.15.12:0/167491551 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc91800a0e0 con 0x561c684be010 2023-10-05T19:40:30.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.602+0000 7fc91da8f700 1 --2- 172.21.15.12:0/167491551 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c684be010 0x561c684dd180 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fc908001c30 tx=0x7fc9080017e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d04cab2771ca093 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.606+0000 7fc9177fe700 1 -- 172.21.15.12:0/167491551 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc90800bbf0 con 0x561c684be010 2023-10-05T19:40:30.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.606+0000 7fc9177fe700 1 -- 172.21.15.12:0/167491551 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc90800bd50 con 0x561c684be010 2023-10-05T19:40:30.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.606+0000 7fc9177fe700 1 -- 172.21.15.12:0/167491551 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc90800e470 con 0x561c684be010 2023-10-05T19:40:30.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.606+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/167491551 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c684be010 msgr2=0x561c684dd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.606+0000 7fc91ff5ae40 1 --2- 172.21.15.12:0/167491551 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c684be010 0x561c684dd180 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fc908001c30 tx=0x7fc9080017e0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.606+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/167491551 shutdown_connections 2023-10-05T19:40:30.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.606+0000 7fc91ff5ae40 1 --2- 172.21.15.12:0/167491551 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c684be010 0x561c684dd180 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.606+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/167491551 >> 172.21.15.12:0/167491551 conn(0x561c68431860 msgr2=0x561c684d8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:30.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.606+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/167491551 shutdown_connections 2023-10-05T19:40:30.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.606+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/167491551 wait complete. 2023-10-05T19:40:30.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.606+0000 7fc91ff5ae40 1 Processor -- start 2023-10-05T19:40:30.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.606+0000 7fc91ff5ae40 1 -- start start 2023-10-05T19:40:30.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.606+0000 7fc91ff5ae40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c684be010 0x561c684e3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.606+0000 7fc91ff5ae40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c6835ced0 con 0x561c684be010 2023-10-05T19:40:30.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.606+0000 7fc91da8f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c684be010 0x561c684e3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.606+0000 7fc91da8f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c684be010 0x561c684e3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58844/0 (socket says 172.21.15.12:58844) 2023-10-05T19:40:30.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.606+0000 7fc91da8f700 1 -- 172.21.15.12:0/3918446649 learned_addr learned my addr 172.21.15.12:0/3918446649 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:30.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.610+0000 7fc91da8f700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c68408bd0 con 0x561c684be010 2023-10-05T19:40:30.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.610+0000 7fc91da8f700 1 --2- 172.21.15.12:0/3918446649 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c684be010 0x561c684e3f50 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fc908004000 tx=0x7fc908016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.610+0000 7fc915ffb700 1 -- 172.21.15.12:0/3918446649 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc90801f070 con 0x561c684be010 2023-10-05T19:40:30.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.610+0000 7fc915ffb700 1 -- 172.21.15.12:0/3918446649 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9080197b0 con 0x561c684be010 2023-10-05T19:40:30.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.610+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561c684e4490 con 0x561c684be010 2023-10-05T19:40:30.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.610+0000 7fc915ffb700 1 -- 172.21.15.12:0/3918446649 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc908019980 con 0x561c684be010 2023-10-05T19:40:30.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.610+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561c684e48d0 con 0x561c684be010 2023-10-05T19:40:30.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.610+0000 7fc91ff5ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.610+0000 7fc915ffb700 1 -- 172.21.15.12:0/3918446649 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fc908022020 con 0x561c684be010 2023-10-05T19:40:30.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.610+0000 7fc915ffb700 1 --2- 172.21.15.12:0/3918446649 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fc900030c50 0x7fc900033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.614+0000 7fc915ffb700 1 -- 172.21.15.12:0/3918446649 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7fc908044c70 con 0x561c684be010 2023-10-05T19:40:30.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.614+0000 7fc91d28e700 1 --2- 172.21.15.12:0/3918446649 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fc900030c50 0x7fc900033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.614+0000 7fc91ff5ae40 1 --2- 172.21.15.12:0/3918446649 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x561c684c25f0 0x561c684efa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.614+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f0110 con 0x561c684c25f0 2023-10-05T19:40:30.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.614+0000 7fc91e290700 1 --2- 172.21.15.12:0/3918446649 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x561c684c25f0 0x561c684efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.614+0000 7fc91d28e700 1 --2- 172.21.15.12:0/3918446649 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fc900030c50 0x7fc900033100 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc918005d80 tx=0x7fc918014000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.614+0000 7fc91e290700 1 --2- 172.21.15.12:0/3918446649 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x561c684c25f0 0x561c684efa10 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.614+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fc91000a500 con 0x561c684c25f0 2023-10-05T19:40:30.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.614+0000 7fc91ff5ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:30.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.614+0000 7fc91ff5ae40 20 realm 2023-10-05T19:40:30.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.614+0000 7fc91ff5ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.614+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684ed070 con 0x561c684c25f0 2023-10-05T19:40:30.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.614+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fc91000a500 con 0x561c684c25f0 2023-10-05T19:40:30.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.614+0000 7fc91ff5ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:30.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.614+0000 7fc91ff5ae40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:30.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.614+0000 7fc91ff5ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.614+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f0bf0 con 0x561c684c25f0 2023-10-05T19:40:30.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.614+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fc91000a500 con 0x561c684c25f0 2023-10-05T19:40:30.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.614+0000 7fc91ff5ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:30.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.614+0000 7fc91ff5ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.614+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f0800 con 0x561c684c25f0 2023-10-05T19:40:30.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.618+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fc91000a500 con 0x561c684c25f0 2023-10-05T19:40:30.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.618+0000 7fc91ff5ae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:30.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.618+0000 7fc91ff5ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.618+0000 7fc91ff5ae40 1 --2- 172.21.15.12:0/3918446649 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x561c684f1f90 0x561c684f4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.618+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f48b0 con 0x561c684f1f90 2023-10-05T19:40:30.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.618+0000 7fc91da8f700 1 --2- 172.21.15.12:0/3918446649 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x561c684f1f90 0x561c684f4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.618+0000 7fc91da8f700 1 --2- 172.21.15.12:0/3918446649 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x561c684f1f90 0x561c684f4370 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.618+0000 7fc91da8f700 1 -- 172.21.15.12:0/3918446649 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fc9080174c0 con 0x561c684f1f90 2023-10-05T19:40:30.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.618+0000 7fc91ff5ae40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:30.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.618+0000 7fc91ff5ae40 20 searching for the correct realm 2023-10-05T19:40:30.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.618+0000 7fc91ff5ae40 1 --2- 172.21.15.12:0/3918446649 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x561c684f6730 0x561c684f8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.618+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f91f0 con 0x561c684f6730 2023-10-05T19:40:30.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.618+0000 7fc91d28e700 1 --2- 172.21.15.12:0/3918446649 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x561c684f6730 0x561c684f8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.618+0000 7fc91d28e700 1 --2- 172.21.15.12:0/3918446649 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x561c684f6730 0x561c684f8b40 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.622+0000 7fc91d28e700 1 -- 172.21.15.12:0/3918446649 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fc91800fcc0 con 0x561c684f6730 2023-10-05T19:40:30.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.622+0000 7fc91d28e700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f91f0 con 0x561c684f1f90 2023-10-05T19:40:30.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.622+0000 7fc91da8f700 1 -- 172.21.15.12:0/3918446649 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc908019b30 con 0x561c684f1f90 2023-10-05T19:40:30.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.622+0000 7fc91da8f700 1 --2- 172.21.15.12:0/3918446649 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fc9080243d0 0x7fc90801c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.622+0000 7fc91da8f700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f91f0 con 0x7fc9080243d0 2023-10-05T19:40:30.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.622+0000 7fc91e290700 1 --2- 172.21.15.12:0/3918446649 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fc9080243d0 0x7fc90801c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.622+0000 7fc91e290700 1 --2- 172.21.15.12:0/3918446649 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fc9080243d0 0x7fc90801c960 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.622+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc910014040 con 0x7fc9080243d0 2023-10-05T19:40:30.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.622+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f91f0 con 0x561c684c25f0 2023-10-05T19:40:30.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.626+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc910014040 con 0x561c684c25f0 2023-10-05T19:40:30.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.626+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f91f0 con 0x561c684c25f0 2023-10-05T19:40:30.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.626+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc910014040 con 0x561c684c25f0 2023-10-05T19:40:30.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.626+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f91f0 con 0x7fc9080243d0 2023-10-05T19:40:30.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.626+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc910014040 con 0x7fc9080243d0 2023-10-05T19:40:30.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.626+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f91f0 con 0x7fc9080243d0 2023-10-05T19:40:30.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.626+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc910014040 con 0x7fc9080243d0 2023-10-05T19:40:30.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.626+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f91f0 con 0x561c684f1f90 2023-10-05T19:40:30.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.626+0000 7fc91da8f700 1 -- 172.21.15.12:0/3918446649 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc908019b30 con 0x561c684f1f90 2023-10-05T19:40:30.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.626+0000 7fc91da8f700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f91f0 con 0x561c684f6730 2023-10-05T19:40:30.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.626+0000 7fc91d28e700 1 -- 172.21.15.12:0/3918446649 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc91800fcc0 con 0x561c684f6730 2023-10-05T19:40:30.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.626+0000 7fc91d28e700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f91f0 con 0x561c684c25f0 2023-10-05T19:40:30.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.630+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc910014040 con 0x561c684c25f0 2023-10-05T19:40:30.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.630+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f91f0 con 0x7fc9080243d0 2023-10-05T19:40:30.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.630+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc910014040 con 0x7fc9080243d0 2023-10-05T19:40:30.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.630+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f91f0 con 0x561c684c25f0 2023-10-05T19:40:30.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.630+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc910014040 con 0x561c684c25f0 2023-10-05T19:40:30.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.630+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f91f0 con 0x561c684f1f90 2023-10-05T19:40:30.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.630+0000 7fc91da8f700 1 -- 172.21.15.12:0/3918446649 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc908019b30 con 0x561c684f1f90 2023-10-05T19:40:30.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.630+0000 7fc91da8f700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f91f0 con 0x561c684f6730 2023-10-05T19:40:30.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.630+0000 7fc91d28e700 1 -- 172.21.15.12:0/3918446649 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc91800fcc0 con 0x561c684f6730 2023-10-05T19:40:30.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.630+0000 7fc91d28e700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f91f0 con 0x561c684f6730 2023-10-05T19:40:30.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.630+0000 7fc91d28e700 1 -- 172.21.15.12:0/3918446649 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc91800fcc0 con 0x561c684f6730 2023-10-05T19:40:30.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.634+0000 7fc91d28e700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f91f0 con 0x7fc9080243d0 2023-10-05T19:40:30.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.634+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc910014040 con 0x7fc9080243d0 2023-10-05T19:40:30.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.634+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f91f0 con 0x561c684f6730 2023-10-05T19:40:30.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.634+0000 7fc91d28e700 1 -- 172.21.15.12:0/3918446649 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc91800fcc0 con 0x561c684f6730 2023-10-05T19:40:30.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.634+0000 7fc91d28e700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f91f0 con 0x561c684f6730 2023-10-05T19:40:30.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.634+0000 7fc91d28e700 1 -- 172.21.15.12:0/3918446649 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc91800fcc0 con 0x561c684f6730 2023-10-05T19:40:30.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.634+0000 7fc91d28e700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f91f0 con 0x561c684c25f0 2023-10-05T19:40:30.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.634+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc910014040 con 0x561c684c25f0 2023-10-05T19:40:30.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.634+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f91f0 con 0x561c684c25f0 2023-10-05T19:40:30.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.634+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc910014040 con 0x561c684c25f0 2023-10-05T19:40:30.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.634+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f91f0 con 0x7fc9080243d0 2023-10-05T19:40:30.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.638+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc910014040 con 0x7fc9080243d0 2023-10-05T19:40:30.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.638+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f91f0 con 0x561c684f6730 2023-10-05T19:40:30.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.638+0000 7fc91d28e700 1 -- 172.21.15.12:0/3918446649 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc91800fcc0 con 0x561c684f6730 2023-10-05T19:40:30.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.638+0000 7fc91d28e700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f91f0 con 0x561c684f1f90 2023-10-05T19:40:30.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.638+0000 7fc91da8f700 1 -- 172.21.15.12:0/3918446649 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc908019b30 con 0x561c684f1f90 2023-10-05T19:40:30.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.638+0000 7fc91da8f700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f91f0 con 0x561c684c25f0 2023-10-05T19:40:30.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.638+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc910014040 con 0x561c684c25f0 2023-10-05T19:40:30.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.638+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f91f0 con 0x7fc9080243d0 2023-10-05T19:40:30.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.638+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc910014040 con 0x7fc9080243d0 2023-10-05T19:40:30.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.638+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f91f0 con 0x561c684f6730 2023-10-05T19:40:30.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.638+0000 7fc91d28e700 1 -- 172.21.15.12:0/3918446649 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc91800fcc0 con 0x561c684f6730 2023-10-05T19:40:30.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.638+0000 7fc91d28e700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f91f0 con 0x7fc9080243d0 2023-10-05T19:40:30.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.642+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc910014040 con 0x7fc9080243d0 2023-10-05T19:40:30.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.642+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f91f0 con 0x561c684f1f90 2023-10-05T19:40:30.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.642+0000 7fc91da8f700 1 -- 172.21.15.12:0/3918446649 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc908019b30 con 0x561c684f1f90 2023-10-05T19:40:30.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.642+0000 7fc91da8f700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f91f0 con 0x561c684f6730 2023-10-05T19:40:30.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.642+0000 7fc91d28e700 1 -- 172.21.15.12:0/3918446649 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc91800fcc0 con 0x561c684f6730 2023-10-05T19:40:30.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.642+0000 7fc91d28e700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f91f0 con 0x7fc9080243d0 2023-10-05T19:40:30.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.642+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc910014040 con 0x7fc9080243d0 2023-10-05T19:40:30.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.642+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f91f0 con 0x561c684c25f0 2023-10-05T19:40:30.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.642+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fc910014040 con 0x561c684c25f0 2023-10-05T19:40:30.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.642+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f91f0 con 0x561c684f1f90 2023-10-05T19:40:30.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.646+0000 7fc91da8f700 1 -- 172.21.15.12:0/3918446649 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fc908019b30 con 0x561c684f1f90 2023-10-05T19:40:30.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.646+0000 7fc91ff5ae40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:30.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.646+0000 7fc91ff5ae40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:30.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.646+0000 7fc91ff5ae40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:30.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.646+0000 7fc91ff5ae40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:30.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.646+0000 7fc91ff5ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.646+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f54b0 con 0x561c684c25f0 2023-10-05T19:40:30.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.646+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fc910014040 con 0x561c684c25f0 2023-10-05T19:40:30.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.646+0000 7fc91ff5ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:30.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.646+0000 7fc91ff5ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.646+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f50e0 con 0x561c684c25f0 2023-10-05T19:40:30.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.646+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fc910014040 con 0x561c684c25f0 2023-10-05T19:40:30.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.646+0000 7fc91ff5ae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:30.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.646+0000 7fc91ff5ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.646+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f9660 con 0x561c684f6730 2023-10-05T19:40:30.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.646+0000 7fc91d28e700 1 -- 172.21.15.12:0/3918446649 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fc91800fcc0 con 0x561c684f6730 2023-10-05T19:40:30.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.646+0000 7fc91ff5ae40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:30.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.646+0000 7fc91ff5ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.646+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f9a10 con 0x561c684f6730 2023-10-05T19:40:30.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.646+0000 7fc91d28e700 1 -- 172.21.15.12:0/3918446649 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7fc91800fcc0 con 0x561c684f6730 2023-10-05T19:40:30.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.646+0000 7fc91ff5ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:30.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.646+0000 7fc91ff5ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.646+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f9dc0 con 0x561c684c25f0 2023-10-05T19:40:30.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.646+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fc910014040 con 0x561c684c25f0 2023-10-05T19:40:30.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.646+0000 7fc91ff5ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:30.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.646+0000 7fc91ff5ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.646+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684fa1a0 con 0x561c684c25f0 2023-10-05T19:40:30.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.650+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fc910014040 con 0x561c684c25f0 2023-10-05T19:40:30.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.650+0000 7fc91ff5ae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:30.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.650+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684fa170 con 0x561c684f6730 2023-10-05T19:40:30.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.650+0000 7fc91d28e700 1 -- 172.21.15.12:0/3918446649 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fc91800fcc0 con 0x561c684f6730 2023-10-05T19:40:30.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.650+0000 7fc91d28e700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684fa170 con 0x561c684f1f90 2023-10-05T19:40:30.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.650+0000 7fc91da8f700 1 -- 172.21.15.12:0/3918446649 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc908019b30 con 0x561c684f1f90 2023-10-05T19:40:30.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.650+0000 7fc91da8f700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684fa170 con 0x7fc9080243d0 2023-10-05T19:40:30.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.650+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc910014040 con 0x7fc9080243d0 2023-10-05T19:40:30.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.650+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f9dc0 con 0x561c684c25f0 2023-10-05T19:40:30.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.654+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc910014040 con 0x561c684c25f0 2023-10-05T19:40:30.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.654+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f9dc0 con 0x561c684c25f0 2023-10-05T19:40:30.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.654+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc910014040 con 0x561c684c25f0 2023-10-05T19:40:30.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.654+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f9dc0 con 0x7fc9080243d0 2023-10-05T19:40:30.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.654+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc910014040 con 0x7fc9080243d0 2023-10-05T19:40:30.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.654+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f9dc0 con 0x7fc9080243d0 2023-10-05T19:40:30.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.654+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc910014040 con 0x7fc9080243d0 2023-10-05T19:40:30.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.654+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f9dc0 con 0x561c684f1f90 2023-10-05T19:40:30.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.654+0000 7fc91da8f700 1 -- 172.21.15.12:0/3918446649 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc908019b30 con 0x561c684f1f90 2023-10-05T19:40:30.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.654+0000 7fc91da8f700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684f9dc0 con 0x561c684f6730 2023-10-05T19:40:30.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.654+0000 7fc91d28e700 1 -- 172.21.15.12:0/3918446649 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc91800fcc0 con 0x561c684f6730 2023-10-05T19:40:30.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.658+0000 7fc91d28e700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc918039040 con 0x561c684c25f0 2023-10-05T19:40:30.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.658+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc910014040 con 0x561c684c25f0 2023-10-05T19:40:30.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.658+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc918039040 con 0x7fc9080243d0 2023-10-05T19:40:30.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.658+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc910014040 con 0x7fc9080243d0 2023-10-05T19:40:30.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.658+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc918039040 con 0x561c684c25f0 2023-10-05T19:40:30.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.658+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc910014040 con 0x561c684c25f0 2023-10-05T19:40:30.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.658+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc918039040 con 0x561c684f1f90 2023-10-05T19:40:30.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.658+0000 7fc91da8f700 1 -- 172.21.15.12:0/3918446649 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc908019b30 con 0x561c684f1f90 2023-10-05T19:40:30.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.658+0000 7fc91da8f700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc918039040 con 0x561c684f6730 2023-10-05T19:40:30.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.658+0000 7fc91d28e700 1 -- 172.21.15.12:0/3918446649 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc91800fcc0 con 0x561c684f6730 2023-10-05T19:40:30.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.658+0000 7fc91d28e700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc918039040 con 0x561c684f6730 2023-10-05T19:40:30.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.662+0000 7fc91d28e700 1 -- 172.21.15.12:0/3918446649 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc91800fcc0 con 0x561c684f6730 2023-10-05T19:40:30.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.662+0000 7fc91d28e700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc918039040 con 0x7fc9080243d0 2023-10-05T19:40:30.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.662+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc910014040 con 0x7fc9080243d0 2023-10-05T19:40:30.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.662+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc918039040 con 0x561c684f6730 2023-10-05T19:40:30.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.662+0000 7fc91d28e700 1 -- 172.21.15.12:0/3918446649 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc91800fcc0 con 0x561c684f6730 2023-10-05T19:40:30.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.662+0000 7fc91d28e700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc918039040 con 0x561c684f6730 2023-10-05T19:40:30.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.662+0000 7fc91d28e700 1 -- 172.21.15.12:0/3918446649 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc91800fcc0 con 0x561c684f6730 2023-10-05T19:40:30.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.662+0000 7fc91d28e700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc918039040 con 0x561c684c25f0 2023-10-05T19:40:30.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.662+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc910014040 con 0x561c684c25f0 2023-10-05T19:40:30.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.662+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc918039040 con 0x561c684c25f0 2023-10-05T19:40:30.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.662+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc910014040 con 0x561c684c25f0 2023-10-05T19:40:30.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.662+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc918039040 con 0x7fc9080243d0 2023-10-05T19:40:30.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.666+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc910014040 con 0x7fc9080243d0 2023-10-05T19:40:30.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.666+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc918039040 con 0x561c684f6730 2023-10-05T19:40:30.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.666+0000 7fc91d28e700 1 -- 172.21.15.12:0/3918446649 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc91800fcc0 con 0x561c684f6730 2023-10-05T19:40:30.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.666+0000 7fc91d28e700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc918039040 con 0x561c684f1f90 2023-10-05T19:40:30.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.666+0000 7fc91da8f700 1 -- 172.21.15.12:0/3918446649 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc908019b30 con 0x561c684f1f90 2023-10-05T19:40:30.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.666+0000 7fc91da8f700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc918039040 con 0x561c684c25f0 2023-10-05T19:40:30.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.666+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc910014040 con 0x561c684c25f0 2023-10-05T19:40:30.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.666+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc918039040 con 0x7fc9080243d0 2023-10-05T19:40:30.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.670+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc910014040 con 0x7fc9080243d0 2023-10-05T19:40:30.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.670+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc918039040 con 0x561c684f6730 2023-10-05T19:40:30.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.670+0000 7fc91d28e700 1 -- 172.21.15.12:0/3918446649 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc91800fcc0 con 0x561c684f6730 2023-10-05T19:40:30.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.670+0000 7fc91d28e700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc918039040 con 0x7fc9080243d0 2023-10-05T19:40:30.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.670+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc910014040 con 0x7fc9080243d0 2023-10-05T19:40:30.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.670+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc918039040 con 0x561c684f1f90 2023-10-05T19:40:30.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.670+0000 7fc91da8f700 1 -- 172.21.15.12:0/3918446649 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc908019b30 con 0x561c684f1f90 2023-10-05T19:40:30.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.670+0000 7fc91da8f700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc918039040 con 0x561c684f6730 2023-10-05T19:40:30.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.670+0000 7fc91d28e700 1 -- 172.21.15.12:0/3918446649 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc91800fcc0 con 0x561c684f6730 2023-10-05T19:40:30.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.670+0000 7fc91d28e700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc918039040 con 0x7fc9080243d0 2023-10-05T19:40:30.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.674+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc910014040 con 0x7fc9080243d0 2023-10-05T19:40:30.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.674+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc918039040 con 0x561c684c25f0 2023-10-05T19:40:30.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.674+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fc910014040 con 0x561c684c25f0 2023-10-05T19:40:30.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.674+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fc918039040 con 0x561c684f1f90 2023-10-05T19:40:30.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.674+0000 7fc91da8f700 1 -- 172.21.15.12:0/3918446649 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fc908019b30 con 0x561c684f1f90 2023-10-05T19:40:30.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.674+0000 7fc91ff5ae40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:30.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.674+0000 7fc91ff5ae40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:30.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.674+0000 7fc91ff5ae40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:30.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.674+0000 7fc91ff5ae40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:30.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.674+0000 7fc91ff5ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.674+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684fa170 con 0x561c684c25f0 2023-10-05T19:40:30.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.674+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fc910014040 con 0x561c684c25f0 2023-10-05T19:40:30.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.674+0000 7fc91ff5ae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:30.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.674+0000 7fc91ff5ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.674+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684fa5d0 con 0x561c684f1f90 2023-10-05T19:40:30.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.674+0000 7fc91da8f700 1 -- 172.21.15.12:0/3918446649 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fc908019b30 con 0x561c684f1f90 2023-10-05T19:40:30.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.678+0000 7fc91ff5ae40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:30.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.678+0000 7fc91ff5ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.678+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684faec0 con 0x561c684c25f0 2023-10-05T19:40:30.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.678+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fc910014040 con 0x561c684c25f0 2023-10-05T19:40:30.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.678+0000 7fc91ff5ae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:30.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.678+0000 7fc91ff5ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.678+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684faec0 con 0x561c684f6730 2023-10-05T19:40:30.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.678+0000 7fc91d28e700 1 -- 172.21.15.12:0/3918446649 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fc91800fcc0 con 0x561c684f6730 2023-10-05T19:40:30.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.678+0000 7fc91ff5ae40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:30.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.678+0000 7fc91ff5ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.678+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684fb630 con 0x7fc9080243d0 2023-10-05T19:40:30.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.678+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7fc910014040 con 0x7fc9080243d0 2023-10-05T19:40:30.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.678+0000 7fc91ff5ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:30.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.678+0000 7fc91ff5ae40 20 zone default found 2023-10-05T19:40:30.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.678+0000 7fc91ff5ae40 4 Realm: () 2023-10-05T19:40:30.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.678+0000 7fc91ff5ae40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:30.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.678+0000 7fc91ff5ae40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:30.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.678+0000 7fc91ff5ae40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:30.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.678+0000 7fc91ff5ae40 20 zonegroup default 2023-10-05T19:40:30.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.678+0000 7fc91ff5ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.678+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684fb430 con 0x7fc9080243d0 2023-10-05T19:40:30.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.678+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fc910014040 con 0x7fc9080243d0 2023-10-05T19:40:30.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.678+0000 7fc91ff5ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:30.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.678+0000 7fc91ff5ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.678+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684fcdb0 con 0x561c684f6730 2023-10-05T19:40:30.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.682+0000 7fc91d28e700 1 -- 172.21.15.12:0/3918446649 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fc91800fcc0 con 0x561c684f6730 2023-10-05T19:40:30.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.682+0000 7fc91ff5ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:30.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.682+0000 7fc91ff5ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.682+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684fd160 con 0x561c684f6730 2023-10-05T19:40:30.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.682+0000 7fc91d28e700 1 -- 172.21.15.12:0/3918446649 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fc91800fcc0 con 0x561c684f6730 2023-10-05T19:40:30.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.682+0000 7fc91ff5ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:30.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.682+0000 7fc91ff5ae40 20 started sync module instance, tier type = 2023-10-05T19:40:30.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.682+0000 7fc91ff5ae40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:30.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.682+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684fc470 con 0x561c684f1f90 2023-10-05T19:40:30.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.686+0000 7fc91da8f700 1 -- 172.21.15.12:0/3918446649 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'174 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc908019b30 con 0x561c684f1f90 2023-10-05T19:40:30.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.686+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94680009132144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684faaf0 con 0x561c684f1f90 2023-10-05T19:40:30.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.686+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684fdef0 con 0x561c684c25f0 2023-10-05T19:40:30.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.686+0000 7fc91da8f700 1 -- 172.21.15.12:0/3918446649 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94680009132144] v28'175 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc908019b30 con 0x561c684f1f90 2023-10-05T19:40:30.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.690+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'174 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc910014040 con 0x561c684c25f0 2023-10-05T19:40:30.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.690+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94680009135664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684fe2f0 con 0x561c684c25f0 2023-10-05T19:40:30.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.690+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684feb90 con 0x561c684f6730 2023-10-05T19:40:30.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.690+0000 7fc91d28e700 1 -- 172.21.15.12:0/3918446649 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'174 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc91800fcc0 con 0x561c684f6730 2023-10-05T19:40:30.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.690+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94680009139216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684fedd0 con 0x561c684f6730 2023-10-05T19:40:30.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.690+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684ff990 con 0x561c684f1f90 2023-10-05T19:40:30.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.694+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94680009135664] v28'175 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc910014040 con 0x561c684c25f0 2023-10-05T19:40:30.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.694+0000 7fc91da8f700 1 -- 172.21.15.12:0/3918446649 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'174 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc908019b30 con 0x561c684f1f90 2023-10-05T19:40:30.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.694+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94680009136768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684ff950 con 0x561c684f1f90 2023-10-05T19:40:30.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.694+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c685003f0 con 0x561c684c25f0 2023-10-05T19:40:30.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.694+0000 7fc91d28e700 1 -- 172.21.15.12:0/3918446649 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94680009139216] v28'175 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc91800fcc0 con 0x561c684f6730 2023-10-05T19:40:30.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.698+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'174 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc910014040 con 0x561c684c25f0 2023-10-05T19:40:30.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.698+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94680009140320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c685003f0 con 0x561c684c25f0 2023-10-05T19:40:30.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.698+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c68500c90 con 0x7fc9080243d0 2023-10-05T19:40:30.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.698+0000 7fc91da8f700 1 -- 172.21.15.12:0/3918446649 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94680009136768] v28'175 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc908019b30 con 0x561c684f1f90 2023-10-05T19:40:30.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.698+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94680009140320] v28'175 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc910014040 con 0x561c684c25f0 2023-10-05T19:40:30.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.698+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'174 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc910014040 con 0x7fc9080243d0 2023-10-05T19:40:30.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.698+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94680009146112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c68500840 con 0x7fc9080243d0 2023-10-05T19:40:30.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.698+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c68501580 con 0x7fc9080243d0 2023-10-05T19:40:30.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.702+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(95 notify.5 [watch watch cookie 94680009146112] v28'175 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc910014040 con 0x7fc9080243d0 2023-10-05T19:40:30.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.702+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(96 notify.6 [create] v28'174 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc910014040 con 0x7fc9080243d0 2023-10-05T19:40:30.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.702+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94680009147216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c685007a0 con 0x7fc9080243d0 2023-10-05T19:40:30.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.702+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c68501a70 con 0x7fc9080243d0 2023-10-05T19:40:30.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.706+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(97 notify.6 [watch watch cookie 94680009147216] v28'175 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc910014040 con 0x7fc9080243d0 2023-10-05T19:40:30.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.706+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(98 notify.7 [create] v28'174 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc910014040 con 0x7fc9080243d0 2023-10-05T19:40:30.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.706+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94680009150480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c68501950 con 0x7fc9080243d0 2023-10-05T19:40:30.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.706+0000 7fc91ff5ae40 20 add_watcher() i=0 2023-10-05T19:40:30.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.706+0000 7fc91ff5ae40 20 add_watcher() i=1 2023-10-05T19:40:30.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.706+0000 7fc91ff5ae40 20 add_watcher() i=2 2023-10-05T19:40:30.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.706+0000 7fc91ff5ae40 20 add_watcher() i=3 2023-10-05T19:40:30.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.706+0000 7fc91ff5ae40 20 add_watcher() i=4 2023-10-05T19:40:30.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.706+0000 7fc91ff5ae40 20 add_watcher() i=5 2023-10-05T19:40:30.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.706+0000 7fc91ff5ae40 20 add_watcher() i=6 2023-10-05T19:40:30.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.706+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94680009150480] v28'175 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc910014040 con 0x7fc9080243d0 2023-10-05T19:40:30.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.706+0000 7fc91ff5ae40 20 add_watcher() i=7 2023-10-05T19:40:30.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.706+0000 7fc91ff5ae40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:30.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.706+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c68504dc0 con 0x561c684c25f0 2023-10-05T19:40:30.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.706+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fc910014040 con 0x561c684c25f0 2023-10-05T19:40:30.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.706+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94680009152048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c68502280 con 0x561c684c25f0 2023-10-05T19:40:30.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.710+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94680009152048] v28'163 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fc910014040 con 0x561c684c25f0 2023-10-05T19:40:30.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.710+0000 7fc8eefe5700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:30.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.710+0000 7fc91ff5ae40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:30.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.710+0000 7fc91ff5ae40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:30.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.710+0000 7fc91ff5ae40 1 Processor -- start 2023-10-05T19:40:30.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.710+0000 7fc91ff5ae40 1 -- start start 2023-10-05T19:40:30.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.710+0000 7fc91ff5ae40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c6850f470 0x561c6850f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.710+0000 7fc91ff5ae40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c6850fd80 con 0x561c6850f470 2023-10-05T19:40:30.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.710+0000 7fc91d28e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c6850f470 0x561c6850f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.710+0000 7fc91d28e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c6850f470 0x561c6850f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58852/0 (socket says 172.21.15.12:58852) 2023-10-05T19:40:30.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.710+0000 7fc91d28e700 1 -- 172.21.15.12:0/3028026535 learned_addr learned my addr 172.21.15.12:0/3028026535 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:30.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.710+0000 7fc91d28e700 1 -- 172.21.15.12:0/3028026535 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c6850fec0 con 0x561c6850f470 2023-10-05T19:40:30.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.710+0000 7fc91d28e700 1 --2- 172.21.15.12:0/3028026535 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c6850f470 0x561c6850f840 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x561c684bf090 tx=0x7fc918011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c48299d9a27e1f6e server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.714+0000 7fc8ee7e4700 1 -- 172.21.15.12:0/3028026535 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc91801ccf0 con 0x561c6850f470 2023-10-05T19:40:30.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.714+0000 7fc8ee7e4700 1 -- 172.21.15.12:0/3028026535 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc91801ce70 con 0x561c6850f470 2023-10-05T19:40:30.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.714+0000 7fc8ee7e4700 1 -- 172.21.15.12:0/3028026535 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc918029420 con 0x561c6850f470 2023-10-05T19:40:30.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.714+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3028026535 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c6850f470 msgr2=0x561c6850f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.714+0000 7fc91ff5ae40 1 --2- 172.21.15.12:0/3028026535 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c6850f470 0x561c6850f840 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x561c684bf090 tx=0x7fc918011000 comp rx=0 tx=0).stop 2023-10-05T19:40:30.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.714+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3028026535 shutdown_connections 2023-10-05T19:40:30.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.714+0000 7fc91ff5ae40 1 --2- 172.21.15.12:0/3028026535 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c6850f470 0x561c6850f840 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.714+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3028026535 >> 172.21.15.12:0/3028026535 conn(0x561c6850a830 msgr2=0x561c68508980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:30.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.714+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3028026535 shutdown_connections 2023-10-05T19:40:30.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.714+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3028026535 wait complete. 2023-10-05T19:40:30.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.714+0000 7fc91ff5ae40 1 Processor -- start 2023-10-05T19:40:30.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.714+0000 7fc91ff5ae40 1 -- start start 2023-10-05T19:40:30.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.714+0000 7fc91ff5ae40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c6850f470 0x561c68515010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.714+0000 7fc91ff5ae40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c68516010 con 0x561c6850f470 2023-10-05T19:40:30.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.714+0000 7fc91d28e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c6850f470 0x561c68515010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.714+0000 7fc91d28e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c6850f470 0x561c68515010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58866/0 (socket says 172.21.15.12:58866) 2023-10-05T19:40:30.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.714+0000 7fc91d28e700 1 -- 172.21.15.12:0/3278179309 learned_addr learned my addr 172.21.15.12:0/3278179309 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:30.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.714+0000 7fc91d28e700 1 -- 172.21.15.12:0/3278179309 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc91802e040 con 0x561c6850f470 2023-10-05T19:40:30.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.714+0000 7fc91d28e700 1 --2- 172.21.15.12:0/3278179309 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c6850f470 0x561c68515010 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc91803c040 tx=0x7fc918042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.714+0000 7fc8ecfe1700 1 -- 172.21.15.12:0/3278179309 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc91801cc70 con 0x561c6850f470 2023-10-05T19:40:30.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.718+0000 7fc8ecfe1700 1 -- 172.21.15.12:0/3278179309 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc918050450 con 0x561c6850f470 2023-10-05T19:40:30.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.718+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3278179309 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561c68515610 con 0x561c6850f470 2023-10-05T19:40:30.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.718+0000 7fc8ecfe1700 1 -- 172.21.15.12:0/3278179309 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc918050600 con 0x561c6850f470 2023-10-05T19:40:30.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.718+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3278179309 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561c68515ab0 con 0x561c6850f470 2023-10-05T19:40:30.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.718+0000 7fc8ecfe1700 1 -- 172.21.15.12:0/3278179309 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fc91804e440 con 0x561c6850f470 2023-10-05T19:40:30.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.718+0000 7fc8ecfe1700 1 --2- 172.21.15.12:0/3278179309 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fc864030c50 0x7fc864033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.718+0000 7fc8ecfe1700 1 -- 172.21.15.12:0/3278179309 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7fc918051e20 con 0x561c6850f470 2023-10-05T19:40:30.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.718+0000 7fc91e290700 1 --2- 172.21.15.12:0/3278179309 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fc864030c50 0x7fc864033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.718+0000 7fc91ff5ae40 5 note: GC not initialized 2023-10-05T19:40:30.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.718+0000 7fc91e290700 1 --2- 172.21.15.12:0/3278179309 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fc864030c50 0x7fc864033100 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x561c68501b50 tx=0x7fc9100325c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.722+0000 7fc85dffb700 20 reqs_thread_entry: start 2023-10-05T19:40:30.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.722+0000 7fc84f7fe700 20 reqs_thread_entry: start 2023-10-05T19:40:30.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.722+0000 7fc91ff5ae40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:30.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.722+0000 7fc91ff5ae40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:30.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.722+0000 7fc84d7fa700 20 reqs_thread_entry: start 2023-10-05T19:40:30.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.722+0000 7fc91ff5ae40 1 --2- 172.21.15.12:0/3278179309 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x561c6857c480 0x561c6857e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.722+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3278179309 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 6.d 6:b5faf345:::2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1-myfoo:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c6857eda0 con 0x561c6857c480 2023-10-05T19:40:30.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.722+0000 7fc84dffb700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:41:01 2023 2023-10-05T19:40:30.731 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:30.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.722+0000 7fc91da8f700 1 --2- 172.21.15.12:0/3278179309 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x561c6857c480 0x561c6857e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.722+0000 7fc91da8f700 1 --2- 172.21.15.12:0/3278179309 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x561c6857c480 0x561c6857e860 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.726+0000 7fc91da8f700 1 -- 172.21.15.12:0/3278179309 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1-myfoo [delete] v28'34 uv34 ondisk = 0) v8 ==== 207+0+0 (crc 0 0 0) 0x7fc908019b30 con 0x561c6857c480 2023-10-05T19:40:30.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.726+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:102 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94680009132144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c6853b780 con 0x561c684f1f90 2023-10-05T19:40:30.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.726+0000 7fc91da8f700 1 -- 172.21.15.12:0/3918446649 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(102 notify.0 [watch unwatch cookie 94680009132144] v28'176 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc908019b30 con 0x561c684f1f90 2023-10-05T19:40:30.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.726+0000 7fc91ff5ae40 20 remove_watcher() i=0 2023-10-05T19:40:30.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.726+0000 7fc91ff5ae40 2 removed watcher, disabling cache 2023-10-05T19:40:30.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.726+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94680009135664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684fc870 con 0x561c684c25f0 2023-10-05T19:40:30.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.730+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(103 notify.1 [watch unwatch cookie 94680009135664] v28'176 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc910014040 con 0x561c684c25f0 2023-10-05T19:40:30.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.730+0000 7fc91ff5ae40 20 remove_watcher() i=1 2023-10-05T19:40:30.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.730+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:104 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94680009139216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684fc870 con 0x561c684f6730 2023-10-05T19:40:30.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.734+0000 7fc91d28e700 1 -- 172.21.15.12:0/3918446649 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(104 notify.2 [watch unwatch cookie 94680009139216] v28'176 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc91804b040 con 0x561c684f6730 2023-10-05T19:40:30.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.734+0000 7fc91ff5ae40 20 remove_watcher() i=2 2023-10-05T19:40:30.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.734+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:105 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94680009136768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684fe410 con 0x561c684f1f90 2023-10-05T19:40:30.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.734+0000 7fc91da8f700 1 -- 172.21.15.12:0/3918446649 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(105 notify.3 [watch unwatch cookie 94680009136768] v28'176 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc908019b30 con 0x561c684f1f90 2023-10-05T19:40:30.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.734+0000 7fc91ff5ae40 20 remove_watcher() i=3 2023-10-05T19:40:30.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.734+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:106 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94680009140320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684fe410 con 0x561c684c25f0 2023-10-05T19:40:30.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.738+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(106 notify.4 [watch unwatch cookie 94680009140320] v28'176 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc910014040 con 0x561c684c25f0 2023-10-05T19:40:30.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.738+0000 7fc91ff5ae40 20 remove_watcher() i=4 2023-10-05T19:40:30.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.738+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:107 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94680009146112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684fc870 con 0x7fc9080243d0 2023-10-05T19:40:30.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.742+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(107 notify.5 [watch unwatch cookie 94680009146112] v28'176 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc910014040 con 0x7fc9080243d0 2023-10-05T19:40:30.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.742+0000 7fc91ff5ae40 20 remove_watcher() i=5 2023-10-05T19:40:30.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.742+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:108 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94680009147216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684fc870 con 0x7fc9080243d0 2023-10-05T19:40:30.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.742+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(108 notify.6 [watch unwatch cookie 94680009147216] v28'176 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc910014040 con 0x7fc9080243d0 2023-10-05T19:40:30.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.742+0000 7fc91ff5ae40 20 remove_watcher() i=6 2023-10-05T19:40:30.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.742+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:109 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94680009150480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684fc870 con 0x7fc9080243d0 2023-10-05T19:40:30.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.746+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(109 notify.7 [watch unwatch cookie 94680009150480] v28'176 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc910014040 con 0x7fc9080243d0 2023-10-05T19:40:30.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.746+0000 7fc91ff5ae40 20 remove_watcher() i=7 2023-10-05T19:40:30.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.750+0000 7fc8eefe5700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:30.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.750+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:110 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94680009152048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561c684fc870 con 0x561c684c25f0 2023-10-05T19:40:30.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.750+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(110 data_loggenerations_metadata [watch unwatch cookie 94680009152048] v28'164 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fc910014040 con 0x561c684c25f0 2023-10-05T19:40:30.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.750+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fc9080243d0 msgr2=0x7fc90801c960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.754+0000 7fc91ff5ae40 1 --2- 172.21.15.12:0/3918446649 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fc9080243d0 0x7fc90801c960 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.754+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x561c684c25f0 msgr2=0x561c684efa10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.754+0000 7fc91ff5ae40 1 --2- 172.21.15.12:0/3918446649 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x561c684c25f0 0x561c684efa10 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.754+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x561c684f1f90 msgr2=0x561c684f4370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.754+0000 7fc91ff5ae40 1 --2- 172.21.15.12:0/3918446649 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x561c684f1f90 0x561c684f4370 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.754+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x561c684f6730 msgr2=0x561c684f8b40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.754+0000 7fc91ff5ae40 1 --2- 172.21.15.12:0/3918446649 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x561c684f6730 0x561c684f8b40 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.754+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fc900030c50 msgr2=0x7fc900033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.754+0000 7fc91ff5ae40 1 --2- 172.21.15.12:0/3918446649 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fc900030c50 0x7fc900033100 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc918005d80 tx=0x7fc918014000 comp rx=0 tx=0).stop 2023-10-05T19:40:30.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.754+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c684be010 msgr2=0x561c684e3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.754+0000 7fc91ff5ae40 1 --2- 172.21.15.12:0/3918446649 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c684be010 0x561c684e3f50 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fc908004000 tx=0x7fc908016000 comp rx=0 tx=0).stop 2023-10-05T19:40:30.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.754+0000 7fc91e290700 1 -- 172.21.15.12:0/3918446649 reap_dead start 2023-10-05T19:40:30.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.754+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 shutdown_connections 2023-10-05T19:40:30.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.754+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 >> 172.21.15.12:0/3918446649 conn(0x561c68431860 msgr2=0x561c684d77c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:30.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.754+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 shutdown_connections 2023-10-05T19:40:30.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.754+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3918446649 wait complete. 2023-10-05T19:40:30.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.754+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3278179309 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x561c6857c480 msgr2=0x561c6857e860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.754+0000 7fc91ff5ae40 1 --2- 172.21.15.12:0/3278179309 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x561c6857c480 0x561c6857e860 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.754+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3278179309 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fc864030c50 msgr2=0x7fc864033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.754+0000 7fc91ff5ae40 1 --2- 172.21.15.12:0/3278179309 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fc864030c50 0x7fc864033100 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x561c68501b50 tx=0x7fc9100325c0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.754+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3278179309 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c6850f470 msgr2=0x561c68515010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.754+0000 7fc91ff5ae40 1 --2- 172.21.15.12:0/3278179309 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c6850f470 0x561c68515010 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc91803c040 tx=0x7fc918042000 comp rx=0 tx=0).stop 2023-10-05T19:40:30.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.754+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3278179309 shutdown_connections 2023-10-05T19:40:30.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.754+0000 7fc91ff5ae40 1 --2- 172.21.15.12:0/3278179309 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x561c6857c480 0x561c6857e860 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.754+0000 7fc91ff5ae40 1 --2- 172.21.15.12:0/3278179309 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fc864030c50 0x7fc864033100 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.754+0000 7fc91ff5ae40 1 --2- 172.21.15.12:0/3278179309 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561c6850f470 0x561c68515010 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.754+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3278179309 >> 172.21.15.12:0/3278179309 conn(0x561c6850a830 msgr2=0x561c685088d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:30.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.754+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3278179309 shutdown_connections 2023-10-05T19:40:30.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.754+0000 7fc91ff5ae40 1 -- 172.21.15.12:0/3278179309 wait complete. 2023-10-05T19:40:30.769 INFO:tasks.util.rgw:rgwadmin: client.0 : ['log', 'show', '--object', '2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.2-myfoo2'] 2023-10-05T19:40:30.769 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'log', 'show', '--object', '2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.2-myfoo2'] 2023-10-05T19:40:30.770 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph log show --object 2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.2-myfoo2 2023-10-05T19:40:30.809 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:30.809 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:30.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.802+0000 7f2bb208ae40 1 Processor -- start 2023-10-05T19:40:30.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.802+0000 7f2bb208ae40 1 -- start start 2023-10-05T19:40:30.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.802+0000 7f2bb208ae40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55a6aad9cd10 0x55a6aad9f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.802+0000 7f2bb208ae40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55a6aace2bd0 con 0x55a6aad9f630 2023-10-05T19:40:30.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.802+0000 7f2bb208ae40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x55a6aac9fee0 con 0x55a6aad9cd10 2023-10-05T19:40:30.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.802+0000 7f2bafbbf700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55a6aad9cd10 0x55a6aad9f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.802+0000 7f2baf3be700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x55a6aad9f630 0x55a6aada1a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:35746/0 (socket says 172.21.15.12:35746) 2023-10-05T19:40:30.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.802+0000 7f2baf3be700 1 -- 172.21.15.12:0/3310694942 learned_addr learned my addr 172.21.15.12:0/3310694942 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:30.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.802+0000 7f2bafbbf700 1 -- 172.21.15.12:0/3310694942 >> v1:172.21.15.8:6789/0 conn(0x55a6aad9f630 legacy=0x55a6aada1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.802+0000 7f2bafbbf700 1 -- 172.21.15.12:0/3310694942 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6aada4140 con 0x55a6aad9cd10 2023-10-05T19:40:30.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.802+0000 7f2bafbbf700 1 --2- 172.21.15.12:0/3310694942 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a6aad9cd10 0x55a6aad9f0f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2ba000a520 tx=0x7f2ba0005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ef53e5571cbb09 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.802+0000 7f2baebbd700 1 -- 172.21.15.12:0/3310694942 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ba000f590 con 0x55a6aad9cd10 2023-10-05T19:40:30.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.802+0000 7f2baebbd700 1 -- 172.21.15.12:0/3310694942 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba000f6f0 con 0x55a6aad9cd10 2023-10-05T19:40:30.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.802+0000 7f2baebbd700 1 -- 172.21.15.12:0/3310694942 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ba000f880 con 0x55a6aad9cd10 2023-10-05T19:40:30.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.802+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3310694942 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a6aad9cd10 msgr2=0x55a6aad9f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.802+0000 7f2bb208ae40 1 --2- 172.21.15.12:0/3310694942 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a6aad9cd10 0x55a6aad9f0f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2ba000a520 tx=0x7f2ba0005d40 comp rx=0 tx=0).stop 2023-10-05T19:40:30.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.802+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3310694942 shutdown_connections 2023-10-05T19:40:30.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.802+0000 7f2bb208ae40 1 --2- 172.21.15.12:0/3310694942 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a6aad9cd10 0x55a6aad9f0f0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.802+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3310694942 >> 172.21.15.12:0/3310694942 conn(0x55a6aad0b870 msgr2=0x55a6aad0dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:30.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.802+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3310694942 shutdown_connections 2023-10-05T19:40:30.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.802+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3310694942 wait complete. 2023-10-05T19:40:30.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.806+0000 7f2bb208ae40 1 Processor -- start 2023-10-05T19:40:30.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.806+0000 7f2bb208ae40 1 -- start start 2023-10-05T19:40:30.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.806+0000 7f2bb208ae40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a6aad9cd10 0x55a6aadbb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.806+0000 7f2bb208ae40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6aac63e30 con 0x55a6aad9cd10 2023-10-05T19:40:30.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.806+0000 7f2bafbbf700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a6aad9cd10 0x55a6aadbb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.806+0000 7f2bafbbf700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a6aad9cd10 0x55a6aadbb500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58886/0 (socket says 172.21.15.12:58886) 2023-10-05T19:40:30.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.806+0000 7f2bafbbf700 1 -- 172.21.15.12:0/3802280781 learned_addr learned my addr 172.21.15.12:0/3802280781 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:30.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.806+0000 7f2bafbbf700 1 -- 172.21.15.12:0/3802280781 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba000a0e0 con 0x55a6aad9cd10 2023-10-05T19:40:30.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.806+0000 7f2bafbbf700 1 --2- 172.21.15.12:0/3802280781 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a6aad9cd10 0x55a6aadbb500 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2ba000a470 tx=0x7f2ba0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f8dcd23ef306ae server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.806+0000 7f2badbbb700 1 -- 172.21.15.12:0/3802280781 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ba000f860 con 0x55a6aad9cd10 2023-10-05T19:40:30.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.806+0000 7f2badbbb700 1 -- 172.21.15.12:0/3802280781 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba000f9e0 con 0x55a6aad9cd10 2023-10-05T19:40:30.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.806+0000 7f2badbbb700 1 -- 172.21.15.12:0/3802280781 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ba000fb90 con 0x55a6aad9cd10 2023-10-05T19:40:30.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.806+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3802280781 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a6aad9cd10 msgr2=0x55a6aadbb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.806+0000 7f2bb208ae40 1 --2- 172.21.15.12:0/3802280781 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a6aad9cd10 0x55a6aadbb500 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2ba000a470 tx=0x7f2ba0014000 comp rx=0 tx=0).stop 2023-10-05T19:40:30.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.806+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3802280781 shutdown_connections 2023-10-05T19:40:30.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.806+0000 7f2bb208ae40 1 --2- 172.21.15.12:0/3802280781 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a6aad9cd10 0x55a6aadbb500 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.806+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3802280781 >> 172.21.15.12:0/3802280781 conn(0x55a6aad0b870 msgr2=0x55a6aadb70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:30.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.806+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3802280781 shutdown_connections 2023-10-05T19:40:30.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.810+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3802280781 wait complete. 2023-10-05T19:40:30.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.810+0000 7f2bb208ae40 1 Processor -- start 2023-10-05T19:40:30.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.810+0000 7f2bb208ae40 1 -- start start 2023-10-05T19:40:30.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.810+0000 7f2bb208ae40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a6aad9cd10 0x55a6aadc2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.810+0000 7f2bb208ae40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6aac36ed0 con 0x55a6aad9cd10 2023-10-05T19:40:30.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.810+0000 7f2bafbbf700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a6aad9cd10 0x55a6aadc2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.810+0000 7f2bafbbf700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a6aad9cd10 0x55a6aadc2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58892/0 (socket says 172.21.15.12:58892) 2023-10-05T19:40:30.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.810+0000 7f2bafbbf700 1 -- 172.21.15.12:0/3359460706 learned_addr learned my addr 172.21.15.12:0/3359460706 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:30.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.810+0000 7f2bafbbf700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6aada4140 con 0x55a6aad9cd10 2023-10-05T19:40:30.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.810+0000 7f2bafbbf700 1 --2- 172.21.15.12:0/3359460706 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a6aad9cd10 0x55a6aadc2140 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2ba000a470 tx=0x7f2ba001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.810+0000 7f2b97fff700 1 -- 172.21.15.12:0/3359460706 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ba002f420 con 0x55a6aad9cd10 2023-10-05T19:40:30.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.810+0000 7f2b97fff700 1 -- 172.21.15.12:0/3359460706 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba002f580 con 0x55a6aad9cd10 2023-10-05T19:40:30.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.810+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a6aadc2680 con 0x55a6aad9cd10 2023-10-05T19:40:30.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.810+0000 7f2b97fff700 1 -- 172.21.15.12:0/3359460706 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ba002f730 con 0x55a6aad9cd10 2023-10-05T19:40:30.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.810+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a6aadc2b20 con 0x55a6aad9cd10 2023-10-05T19:40:30.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.814+0000 7f2b97fff700 1 -- 172.21.15.12:0/3359460706 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f2ba0010420 con 0x55a6aad9cd10 2023-10-05T19:40:30.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.814+0000 7f2bb208ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.814+0000 7f2b97fff700 1 --2- 172.21.15.12:0/3359460706 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2b9c030c00 0x7f2b9c0330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.814+0000 7f2b97fff700 1 -- 172.21.15.12:0/3359460706 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f2ba002b070 con 0x55a6aad9cd10 2023-10-05T19:40:30.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.814+0000 7f2bb208ae40 1 --2- 172.21.15.12:0/3359460706 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55a6aad9f630 0x55a6aadcdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.814+0000 7f2baf3be700 1 --2- 172.21.15.12:0/3359460706 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2b9c030c00 0x7f2b9c0330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.814+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadce270 con 0x55a6aad9f630 2023-10-05T19:40:30.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.814+0000 7f2bb03c0700 1 --2- 172.21.15.12:0/3359460706 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55a6aad9f630 0x55a6aadcdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.814+0000 7f2baf3be700 1 --2- 172.21.15.12:0/3359460706 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2b9c030c00 0x7f2b9c0330b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4001df0 tx=0x7f2ba4001310 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.814+0000 7f2bb03c0700 1 --2- 172.21.15.12:0/3359460706 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55a6aad9f630 0x55a6aadcdc00 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.814+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2ba800a500 con 0x55a6aad9f630 2023-10-05T19:40:30.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.814+0000 7f2bb208ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:30.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.814+0000 7f2bb208ae40 20 realm 2023-10-05T19:40:30.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.814+0000 7f2bb208ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.814+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadcb260 con 0x55a6aad9f630 2023-10-05T19:40:30.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.818+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2ba800a500 con 0x55a6aad9f630 2023-10-05T19:40:30.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.818+0000 7f2bb208ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:30.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.818+0000 7f2bb208ae40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:30.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.818+0000 7f2bb208ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.818+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadced50 con 0x55a6aad9f630 2023-10-05T19:40:30.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.818+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2ba800a500 con 0x55a6aad9f630 2023-10-05T19:40:30.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.818+0000 7f2bb208ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:30.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.818+0000 7f2bb208ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.818+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadce930 con 0x55a6aad9f630 2023-10-05T19:40:30.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.818+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f2ba800a500 con 0x55a6aad9f630 2023-10-05T19:40:30.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.818+0000 7f2bb208ae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:30.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.818+0000 7f2bb208ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.818+0000 7f2bb208ae40 1 --2- 172.21.15.12:0/3359460706 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55a6aadcfce0 0x55a6aadd20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.818+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd2830 con 0x55a6aadcfce0 2023-10-05T19:40:30.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.818+0000 7f2bafbbf700 1 --2- 172.21.15.12:0/3359460706 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55a6aadcfce0 0x55a6aadd20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.818+0000 7f2bafbbf700 1 --2- 172.21.15.12:0/3359460706 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55a6aadcfce0 0x55a6aadd20c0 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.818+0000 7f2bafbbf700 1 -- 172.21.15.12:0/3359460706 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f2ba002e7e0 con 0x55a6aadcfce0 2023-10-05T19:40:30.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.818+0000 7f2bb208ae40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:30.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.818+0000 7f2bb208ae40 20 searching for the correct realm 2023-10-05T19:40:30.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.818+0000 7f2bb208ae40 1 --2- 172.21.15.12:0/3359460706 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55a6aadd4660 0x55a6aadd6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.818+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd6f80 con 0x55a6aadd4660 2023-10-05T19:40:30.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.818+0000 7f2baf3be700 1 --2- 172.21.15.12:0/3359460706 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55a6aadd4660 0x55a6aadd6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.818+0000 7f2baf3be700 1 --2- 172.21.15.12:0/3359460706 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55a6aadd4660 0x55a6aadd6a40 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.822+0000 7f2baf3be700 1 -- 172.21.15.12:0/3359460706 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f2ba4006070 con 0x55a6aadd4660 2023-10-05T19:40:30.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.822+0000 7f2baf3be700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd6f80 con 0x55a6aadcfce0 2023-10-05T19:40:30.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.822+0000 7f2bafbbf700 1 -- 172.21.15.12:0/3359460706 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba002e7e0 con 0x55a6aadcfce0 2023-10-05T19:40:30.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.822+0000 7f2bafbbf700 1 --2- 172.21.15.12:0/3359460706 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f2ba001abe0 0x7f2ba002f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.822+0000 7f2bafbbf700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd6f80 con 0x7f2ba001abe0 2023-10-05T19:40:30.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.822+0000 7f2bb03c0700 1 --2- 172.21.15.12:0/3359460706 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f2ba001abe0 0x7f2ba002f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.822+0000 7f2bb03c0700 1 --2- 172.21.15.12:0/3359460706 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f2ba001abe0 0x7f2ba002f9f0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.826+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba8014040 con 0x7f2ba001abe0 2023-10-05T19:40:30.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.826+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd6f80 con 0x55a6aad9f630 2023-10-05T19:40:30.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.826+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba8014040 con 0x55a6aad9f630 2023-10-05T19:40:30.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.826+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd6f80 con 0x55a6aad9f630 2023-10-05T19:40:30.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.826+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba8014040 con 0x55a6aad9f630 2023-10-05T19:40:30.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.826+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd6f80 con 0x7f2ba001abe0 2023-10-05T19:40:30.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.826+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba8014040 con 0x7f2ba001abe0 2023-10-05T19:40:30.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.826+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd6f80 con 0x7f2ba001abe0 2023-10-05T19:40:30.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.826+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba8014040 con 0x7f2ba001abe0 2023-10-05T19:40:30.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.826+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd6f80 con 0x55a6aadcfce0 2023-10-05T19:40:30.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.830+0000 7f2bafbbf700 1 -- 172.21.15.12:0/3359460706 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba002e7e0 con 0x55a6aadcfce0 2023-10-05T19:40:30.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.830+0000 7f2bafbbf700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd6f80 con 0x55a6aadd4660 2023-10-05T19:40:30.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.830+0000 7f2baf3be700 1 -- 172.21.15.12:0/3359460706 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba4006070 con 0x55a6aadd4660 2023-10-05T19:40:30.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.830+0000 7f2baf3be700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd6f80 con 0x55a6aad9f630 2023-10-05T19:40:30.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.830+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba8014040 con 0x55a6aad9f630 2023-10-05T19:40:30.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.830+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd6f80 con 0x7f2ba001abe0 2023-10-05T19:40:30.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.830+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba8014040 con 0x7f2ba001abe0 2023-10-05T19:40:30.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.830+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd6f80 con 0x55a6aad9f630 2023-10-05T19:40:30.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.830+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba8014040 con 0x55a6aad9f630 2023-10-05T19:40:30.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.830+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd6f80 con 0x55a6aadcfce0 2023-10-05T19:40:30.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.830+0000 7f2bafbbf700 1 -- 172.21.15.12:0/3359460706 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba002e7e0 con 0x55a6aadcfce0 2023-10-05T19:40:30.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.830+0000 7f2bafbbf700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd6f80 con 0x55a6aadd4660 2023-10-05T19:40:30.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.834+0000 7f2baf3be700 1 -- 172.21.15.12:0/3359460706 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba4006070 con 0x55a6aadd4660 2023-10-05T19:40:30.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.834+0000 7f2baf3be700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd6f80 con 0x55a6aadd4660 2023-10-05T19:40:30.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.834+0000 7f2baf3be700 1 -- 172.21.15.12:0/3359460706 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba4006070 con 0x55a6aadd4660 2023-10-05T19:40:30.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.834+0000 7f2baf3be700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd6f80 con 0x7f2ba001abe0 2023-10-05T19:40:30.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.834+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba8014040 con 0x7f2ba001abe0 2023-10-05T19:40:30.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.834+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd6f80 con 0x55a6aadd4660 2023-10-05T19:40:30.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.834+0000 7f2baf3be700 1 -- 172.21.15.12:0/3359460706 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba4006070 con 0x55a6aadd4660 2023-10-05T19:40:30.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.834+0000 7f2baf3be700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd6f80 con 0x55a6aadd4660 2023-10-05T19:40:30.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.838+0000 7f2baf3be700 1 -- 172.21.15.12:0/3359460706 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba4006070 con 0x55a6aadd4660 2023-10-05T19:40:30.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.838+0000 7f2baf3be700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd6f80 con 0x55a6aad9f630 2023-10-05T19:40:30.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.838+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba8014040 con 0x55a6aad9f630 2023-10-05T19:40:30.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.838+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd6f80 con 0x55a6aad9f630 2023-10-05T19:40:30.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.838+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba8014040 con 0x55a6aad9f630 2023-10-05T19:40:30.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.838+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd6f80 con 0x7f2ba001abe0 2023-10-05T19:40:30.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.838+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba8014040 con 0x7f2ba001abe0 2023-10-05T19:40:30.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.838+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd6f80 con 0x55a6aadd4660 2023-10-05T19:40:30.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.838+0000 7f2baf3be700 1 -- 172.21.15.12:0/3359460706 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba4006070 con 0x55a6aadd4660 2023-10-05T19:40:30.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.838+0000 7f2baf3be700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd6f80 con 0x55a6aadcfce0 2023-10-05T19:40:30.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.838+0000 7f2bafbbf700 1 -- 172.21.15.12:0/3359460706 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba002e7e0 con 0x55a6aadcfce0 2023-10-05T19:40:30.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.838+0000 7f2bafbbf700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd6f80 con 0x55a6aad9f630 2023-10-05T19:40:30.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.842+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba8014040 con 0x55a6aad9f630 2023-10-05T19:40:30.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.842+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd6f80 con 0x7f2ba001abe0 2023-10-05T19:40:30.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.842+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba8014040 con 0x7f2ba001abe0 2023-10-05T19:40:30.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.842+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd6f80 con 0x55a6aadd4660 2023-10-05T19:40:30.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.842+0000 7f2baf3be700 1 -- 172.21.15.12:0/3359460706 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba4006070 con 0x55a6aadd4660 2023-10-05T19:40:30.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.842+0000 7f2baf3be700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd6f80 con 0x7f2ba001abe0 2023-10-05T19:40:30.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.846+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba8014040 con 0x7f2ba001abe0 2023-10-05T19:40:30.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.846+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd6f80 con 0x55a6aadcfce0 2023-10-05T19:40:30.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.846+0000 7f2bafbbf700 1 -- 172.21.15.12:0/3359460706 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba002e7e0 con 0x55a6aadcfce0 2023-10-05T19:40:30.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.846+0000 7f2bafbbf700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd6f80 con 0x55a6aadd4660 2023-10-05T19:40:30.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.846+0000 7f2baf3be700 1 -- 172.21.15.12:0/3359460706 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba4006070 con 0x55a6aadd4660 2023-10-05T19:40:30.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.846+0000 7f2baf3be700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd6f80 con 0x7f2ba001abe0 2023-10-05T19:40:30.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.846+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba8014040 con 0x7f2ba001abe0 2023-10-05T19:40:30.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.846+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd6f80 con 0x55a6aad9f630 2023-10-05T19:40:30.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.846+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f2ba8014040 con 0x55a6aad9f630 2023-10-05T19:40:30.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.850+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd6f80 con 0x55a6aadcfce0 2023-10-05T19:40:30.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.850+0000 7f2bafbbf700 1 -- 172.21.15.12:0/3359460706 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f2ba002e7e0 con 0x55a6aadcfce0 2023-10-05T19:40:30.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.850+0000 7f2bb208ae40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:30.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.850+0000 7f2bb208ae40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:30.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.850+0000 7f2bb208ae40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:30.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.850+0000 7f2bb208ae40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:30.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.850+0000 7f2bb208ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.850+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd33e0 con 0x55a6aad9f630 2023-10-05T19:40:30.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.850+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2ba8014040 con 0x55a6aad9f630 2023-10-05T19:40:30.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.850+0000 7f2bb208ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:30.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.850+0000 7f2bb208ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.850+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd3d30 con 0x55a6aad9f630 2023-10-05T19:40:30.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.850+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f2ba8014040 con 0x55a6aad9f630 2023-10-05T19:40:30.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.850+0000 7f2bb208ae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:30.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.850+0000 7f2bb208ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.850+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd2fa0 con 0x55a6aadd4660 2023-10-05T19:40:30.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.850+0000 7f2baf3be700 1 -- 172.21.15.12:0/3359460706 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f2ba4006070 con 0x55a6aadd4660 2023-10-05T19:40:30.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.850+0000 7f2bb208ae40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:30.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.850+0000 7f2bb208ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.850+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd78b0 con 0x55a6aadd4660 2023-10-05T19:40:30.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.854+0000 7f2baf3be700 1 -- 172.21.15.12:0/3359460706 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f2ba4006070 con 0x55a6aadd4660 2023-10-05T19:40:30.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.854+0000 7f2bb208ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:30.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.854+0000 7f2bb208ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.854+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd7c60 con 0x55a6aad9f630 2023-10-05T19:40:30.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.854+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2ba8014040 con 0x55a6aad9f630 2023-10-05T19:40:30.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.854+0000 7f2bb208ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:30.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.854+0000 7f2bb208ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.854+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd8010 con 0x55a6aad9f630 2023-10-05T19:40:30.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.854+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f2ba8014040 con 0x55a6aad9f630 2023-10-05T19:40:30.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.854+0000 7f2bb208ae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:30.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.854+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd9020 con 0x55a6aadd4660 2023-10-05T19:40:30.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.858+0000 7f2baf3be700 1 -- 172.21.15.12:0/3359460706 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f2ba4006070 con 0x55a6aadd4660 2023-10-05T19:40:30.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.858+0000 7f2baf3be700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd9020 con 0x55a6aadcfce0 2023-10-05T19:40:30.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.858+0000 7f2bafbbf700 1 -- 172.21.15.12:0/3359460706 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba002e7e0 con 0x55a6aadcfce0 2023-10-05T19:40:30.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.858+0000 7f2bafbbf700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd9020 con 0x7f2ba001abe0 2023-10-05T19:40:30.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.858+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba8014040 con 0x7f2ba001abe0 2023-10-05T19:40:30.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.858+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd7c60 con 0x55a6aad9f630 2023-10-05T19:40:30.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.858+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba8014040 con 0x55a6aad9f630 2023-10-05T19:40:30.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.858+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd7c60 con 0x55a6aad9f630 2023-10-05T19:40:30.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.862+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba8014040 con 0x55a6aad9f630 2023-10-05T19:40:30.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.862+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd7c60 con 0x7f2ba001abe0 2023-10-05T19:40:30.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.862+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba8014040 con 0x7f2ba001abe0 2023-10-05T19:40:30.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.862+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd7c60 con 0x7f2ba001abe0 2023-10-05T19:40:30.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.862+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba8014040 con 0x7f2ba001abe0 2023-10-05T19:40:30.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.862+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd7c60 con 0x55a6aadcfce0 2023-10-05T19:40:30.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.862+0000 7f2bafbbf700 1 -- 172.21.15.12:0/3359460706 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba002e7e0 con 0x55a6aadcfce0 2023-10-05T19:40:30.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.862+0000 7f2bafbbf700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd7c60 con 0x55a6aadd4660 2023-10-05T19:40:30.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.862+0000 7f2baf3be700 1 -- 172.21.15.12:0/3359460706 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba4006070 con 0x55a6aadd4660 2023-10-05T19:40:30.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.862+0000 7f2baf3be700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd7c60 con 0x55a6aad9f630 2023-10-05T19:40:30.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.866+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba8014040 con 0x55a6aad9f630 2023-10-05T19:40:30.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.866+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd7c60 con 0x7f2ba001abe0 2023-10-05T19:40:30.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.866+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba8014040 con 0x7f2ba001abe0 2023-10-05T19:40:30.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.866+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd7c60 con 0x55a6aad9f630 2023-10-05T19:40:30.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.866+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba8014040 con 0x55a6aad9f630 2023-10-05T19:40:30.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.866+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd7c60 con 0x55a6aadcfce0 2023-10-05T19:40:30.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.866+0000 7f2bafbbf700 1 -- 172.21.15.12:0/3359460706 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba002e7e0 con 0x55a6aadcfce0 2023-10-05T19:40:30.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.866+0000 7f2bafbbf700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd7c60 con 0x55a6aadd4660 2023-10-05T19:40:30.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.870+0000 7f2baf3be700 1 -- 172.21.15.12:0/3359460706 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba4006070 con 0x55a6aadd4660 2023-10-05T19:40:30.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.870+0000 7f2baf3be700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd7c60 con 0x55a6aadd4660 2023-10-05T19:40:30.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.870+0000 7f2baf3be700 1 -- 172.21.15.12:0/3359460706 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba4006070 con 0x55a6aadd4660 2023-10-05T19:40:30.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.870+0000 7f2baf3be700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd7c60 con 0x7f2ba001abe0 2023-10-05T19:40:30.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.870+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba8014040 con 0x7f2ba001abe0 2023-10-05T19:40:30.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.870+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd7c60 con 0x55a6aadd4660 2023-10-05T19:40:30.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.870+0000 7f2baf3be700 1 -- 172.21.15.12:0/3359460706 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba4006070 con 0x55a6aadd4660 2023-10-05T19:40:30.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.870+0000 7f2baf3be700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd7c60 con 0x55a6aadd4660 2023-10-05T19:40:30.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.870+0000 7f2baf3be700 1 -- 172.21.15.12:0/3359460706 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba4006070 con 0x55a6aadd4660 2023-10-05T19:40:30.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.870+0000 7f2baf3be700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd7c60 con 0x55a6aad9f630 2023-10-05T19:40:30.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.874+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba8014040 con 0x55a6aad9f630 2023-10-05T19:40:30.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.874+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd7c60 con 0x55a6aad9f630 2023-10-05T19:40:30.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.874+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba8014040 con 0x55a6aad9f630 2023-10-05T19:40:30.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.874+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd7c60 con 0x7f2ba001abe0 2023-10-05T19:40:30.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.874+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba8014040 con 0x7f2ba001abe0 2023-10-05T19:40:30.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.874+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd7c60 con 0x55a6aadd4660 2023-10-05T19:40:30.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.874+0000 7f2baf3be700 1 -- 172.21.15.12:0/3359460706 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba4006070 con 0x55a6aadd4660 2023-10-05T19:40:30.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.874+0000 7f2baf3be700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd7c60 con 0x55a6aadcfce0 2023-10-05T19:40:30.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.878+0000 7f2bafbbf700 1 -- 172.21.15.12:0/3359460706 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba002e7e0 con 0x55a6aadcfce0 2023-10-05T19:40:30.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.878+0000 7f2bafbbf700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd7c60 con 0x55a6aad9f630 2023-10-05T19:40:30.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.878+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba8014040 con 0x55a6aad9f630 2023-10-05T19:40:30.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.878+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd7c60 con 0x7f2ba001abe0 2023-10-05T19:40:30.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.878+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba8014040 con 0x7f2ba001abe0 2023-10-05T19:40:30.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.878+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd7c60 con 0x55a6aadd4660 2023-10-05T19:40:30.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.878+0000 7f2baf3be700 1 -- 172.21.15.12:0/3359460706 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba4006070 con 0x55a6aadd4660 2023-10-05T19:40:30.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.878+0000 7f2baf3be700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd7c60 con 0x7f2ba001abe0 2023-10-05T19:40:30.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.882+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba8014040 con 0x7f2ba001abe0 2023-10-05T19:40:30.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.882+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd7c60 con 0x55a6aadcfce0 2023-10-05T19:40:30.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.882+0000 7f2bafbbf700 1 -- 172.21.15.12:0/3359460706 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba002e7e0 con 0x55a6aadcfce0 2023-10-05T19:40:30.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.882+0000 7f2bafbbf700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd7c60 con 0x55a6aadd4660 2023-10-05T19:40:30.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.882+0000 7f2baf3be700 1 -- 172.21.15.12:0/3359460706 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba4006070 con 0x55a6aadd4660 2023-10-05T19:40:30.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.882+0000 7f2baf3be700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd7c60 con 0x7f2ba001abe0 2023-10-05T19:40:30.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.882+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2ba8014040 con 0x7f2ba001abe0 2023-10-05T19:40:30.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.882+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd7c60 con 0x55a6aad9f630 2023-10-05T19:40:30.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.886+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f2ba8014040 con 0x55a6aad9f630 2023-10-05T19:40:30.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.886+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd7c60 con 0x55a6aadcfce0 2023-10-05T19:40:30.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.886+0000 7f2bafbbf700 1 -- 172.21.15.12:0/3359460706 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f2ba002e7e0 con 0x55a6aadcfce0 2023-10-05T19:40:30.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.886+0000 7f2bb208ae40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:30.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.886+0000 7f2bb208ae40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:30.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.886+0000 7f2bb208ae40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:30.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.886+0000 7f2bb208ae40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:30.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.886+0000 7f2bb208ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.886+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd8010 con 0x55a6aad9f630 2023-10-05T19:40:30.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.886+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f2ba8014040 con 0x55a6aad9f630 2023-10-05T19:40:30.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.886+0000 7f2bb208ae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:30.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.886+0000 7f2bb208ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.886+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd83c0 con 0x55a6aadcfce0 2023-10-05T19:40:30.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.886+0000 7f2bafbbf700 1 -- 172.21.15.12:0/3359460706 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f2ba002e7e0 con 0x55a6aadcfce0 2023-10-05T19:40:30.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.886+0000 7f2bb208ae40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:30.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.886+0000 7f2bb208ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.886+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd8c60 con 0x55a6aad9f630 2023-10-05T19:40:30.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.886+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f2ba8014040 con 0x55a6aad9f630 2023-10-05T19:40:30.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.886+0000 7f2bb208ae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:30.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.886+0000 7f2bb208ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.886+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd8c60 con 0x55a6aadd4660 2023-10-05T19:40:30.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.890+0000 7f2baf3be700 1 -- 172.21.15.12:0/3359460706 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f2ba4006070 con 0x55a6aadd4660 2023-10-05T19:40:30.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.890+0000 7f2bb208ae40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:30.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.890+0000 7f2bb208ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.890+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd87f0 con 0x7f2ba001abe0 2023-10-05T19:40:30.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.890+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f2ba8014040 con 0x7f2ba001abe0 2023-10-05T19:40:30.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.890+0000 7f2bb208ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:30.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.890+0000 7f2bb208ae40 20 zone default found 2023-10-05T19:40:30.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.890+0000 7f2bb208ae40 4 Realm: () 2023-10-05T19:40:30.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.890+0000 7f2bb208ae40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:30.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.890+0000 7f2bb208ae40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:30.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.890+0000 7f2bb208ae40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:30.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.890+0000 7f2bb208ae40 20 zonegroup default 2023-10-05T19:40:30.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.890+0000 7f2bb208ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.890+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd87f0 con 0x7f2ba001abe0 2023-10-05T19:40:30.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.890+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f2ba8014040 con 0x7f2ba001abe0 2023-10-05T19:40:30.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.890+0000 7f2bb208ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:30.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.890+0000 7f2bb208ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.890+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadd87f0 con 0x55a6aadd4660 2023-10-05T19:40:30.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.890+0000 7f2baf3be700 1 -- 172.21.15.12:0/3359460706 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f2ba4006070 con 0x55a6aadd4660 2023-10-05T19:40:30.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.890+0000 7f2bb208ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:30.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.890+0000 7f2bb208ae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:30.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.890+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aaddae80 con 0x55a6aadd4660 2023-10-05T19:40:30.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.894+0000 7f2baf3be700 1 -- 172.21.15.12:0/3359460706 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f2ba4006070 con 0x55a6aadd4660 2023-10-05T19:40:30.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.894+0000 7f2bb208ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:30.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.894+0000 7f2bb208ae40 20 started sync module instance, tier type = 2023-10-05T19:40:30.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.894+0000 7f2bb208ae40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:30.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.894+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadda540 con 0x55a6aadcfce0 2023-10-05T19:40:30.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.894+0000 7f2bafbbf700 1 -- 172.21.15.12:0/3359460706 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'177 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2ba002e7e0 con 0x55a6aadcfce0 2023-10-05T19:40:30.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.894+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94174319585600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadda170 con 0x55a6aadcfce0 2023-10-05T19:40:30.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.894+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aaddbc10 con 0x55a6aad9f630 2023-10-05T19:40:30.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.898+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'177 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2ba8014040 con 0x55a6aad9f630 2023-10-05T19:40:30.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.898+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94174319589440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aaddc200 con 0x55a6aad9f630 2023-10-05T19:40:30.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.898+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aaddcd60 con 0x55a6aadd4660 2023-10-05T19:40:30.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.898+0000 7f2bafbbf700 1 -- 172.21.15.12:0/3359460706 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94174319585600] v28'178 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2ba002e7e0 con 0x55a6aadcfce0 2023-10-05T19:40:30.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.902+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94174319589440] v28'178 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2ba8014040 con 0x55a6aad9f630 2023-10-05T19:40:30.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.902+0000 7f2baf3be700 1 -- 172.21.15.12:0/3359460706 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'177 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2ba4006070 con 0x55a6aadd4660 2023-10-05T19:40:30.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.902+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94174319587888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aaddbc90 con 0x55a6aadd4660 2023-10-05T19:40:30.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.902+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aaddd600 con 0x55a6aadcfce0 2023-10-05T19:40:30.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.906+0000 7f2baf3be700 1 -- 172.21.15.12:0/3359460706 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94174319587888] v28'178 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2ba4006070 con 0x55a6aadd4660 2023-10-05T19:40:30.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.906+0000 7f2bafbbf700 1 -- 172.21.15.12:0/3359460706 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'177 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2ba002e7e0 con 0x55a6aadcfce0 2023-10-05T19:40:30.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.906+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94174319595792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aaddc950 con 0x55a6aadcfce0 2023-10-05T19:40:30.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.906+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadde1b0 con 0x55a6aad9f630 2023-10-05T19:40:30.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.906+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'177 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2ba8014040 con 0x55a6aad9f630 2023-10-05T19:40:30.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.906+0000 7f2bafbbf700 1 -- 172.21.15.12:0/3359460706 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94174319595792] v28'178 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2ba002e7e0 con 0x55a6aadcfce0 2023-10-05T19:40:30.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.906+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94174319598720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aaddc950 con 0x55a6aad9f630 2023-10-05T19:40:30.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.910+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aaddef90 con 0x7f2ba001abe0 2023-10-05T19:40:30.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.910+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94174319598720] v28'178 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2ba8014040 con 0x55a6aad9f630 2023-10-05T19:40:30.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.910+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'177 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2ba8014040 con 0x7f2ba001abe0 2023-10-05T19:40:30.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.910+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94174319600896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aaddc950 con 0x7f2ba001abe0 2023-10-05T19:40:30.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.910+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aaddf4e0 con 0x7f2ba001abe0 2023-10-05T19:40:30.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.914+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(96 notify.6 [create] v28'177 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2ba8014040 con 0x7f2ba001abe0 2023-10-05T19:40:30.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.914+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94174319603632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aaddf400 con 0x7f2ba001abe0 2023-10-05T19:40:30.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.914+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aade0430 con 0x7f2ba001abe0 2023-10-05T19:40:30.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.914+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(95 notify.5 [watch watch cookie 94174319600896] v28'178 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2ba8014040 con 0x7f2ba001abe0 2023-10-05T19:40:30.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.914+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(97 notify.6 [watch watch cookie 94174319603632] v28'178 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2ba8014040 con 0x7f2ba001abe0 2023-10-05T19:40:30.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.918+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(98 notify.7 [create] v28'177 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2ba8014040 con 0x7f2ba001abe0 2023-10-05T19:40:30.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.918+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94174319602000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aaddff40 con 0x7f2ba001abe0 2023-10-05T19:40:30.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.918+0000 7f2bb208ae40 20 add_watcher() i=0 2023-10-05T19:40:30.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.918+0000 7f2bb208ae40 20 add_watcher() i=1 2023-10-05T19:40:30.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.918+0000 7f2bb208ae40 20 add_watcher() i=2 2023-10-05T19:40:30.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.918+0000 7f2bb208ae40 20 add_watcher() i=3 2023-10-05T19:40:30.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.918+0000 7f2bb208ae40 20 add_watcher() i=4 2023-10-05T19:40:30.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.918+0000 7f2bb208ae40 20 add_watcher() i=5 2023-10-05T19:40:30.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.918+0000 7f2bb208ae40 20 add_watcher() i=6 2023-10-05T19:40:30.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.918+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94174319602000] v28'178 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2ba8014040 con 0x7f2ba001abe0 2023-10-05T19:40:30.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.918+0000 7f2bb208ae40 20 add_watcher() i=7 2023-10-05T19:40:30.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.918+0000 7f2bb208ae40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:30.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.918+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aade3460 con 0x55a6aad9f630 2023-10-05T19:40:30.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.918+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f2ba8014040 con 0x55a6aad9f630 2023-10-05T19:40:30.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.918+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94174319607616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aade0880 con 0x55a6aad9f630 2023-10-05T19:40:30.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.922+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94174319607616] v28'165 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f2ba8014040 con 0x55a6aad9f630 2023-10-05T19:40:30.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.922+0000 7f2b817e2700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:30.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.922+0000 7f2bb208ae40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:30.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.922+0000 7f2bb208ae40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:30.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.922+0000 7f2bb208ae40 1 Processor -- start 2023-10-05T19:40:30.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.922+0000 7f2bb208ae40 1 -- start start 2023-10-05T19:40:30.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.922+0000 7f2bb208ae40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a6aaded540 0x55a6aaded910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.922+0000 7f2bb208ae40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6aadede50 con 0x55a6aaded540 2023-10-05T19:40:30.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.922+0000 7f2baf3be700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a6aaded540 0x55a6aaded910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.922+0000 7f2baf3be700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a6aaded540 0x55a6aaded910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58900/0 (socket says 172.21.15.12:58900) 2023-10-05T19:40:30.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.922+0000 7f2baf3be700 1 -- 172.21.15.12:0/2343806459 learned_addr learned my addr 172.21.15.12:0/2343806459 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:30.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.922+0000 7f2baf3be700 1 -- 172.21.15.12:0/2343806459 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6aadedf90 con 0x55a6aaded540 2023-10-05T19:40:30.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.922+0000 7f2baf3be700 1 --2- 172.21.15.12:0/2343806459 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a6aaded540 0x55a6aaded910 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2ba002ea30 tx=0x7f2ba401e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b24b969437f02e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.926+0000 7f2b80fe1700 1 -- 172.21.15.12:0/2343806459 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ba4013040 con 0x55a6aaded540 2023-10-05T19:40:30.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.926+0000 7f2b80fe1700 1 -- 172.21.15.12:0/2343806459 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba4022040 con 0x55a6aaded540 2023-10-05T19:40:30.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.926+0000 7f2b80fe1700 1 -- 172.21.15.12:0/2343806459 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ba400e940 con 0x55a6aaded540 2023-10-05T19:40:30.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.926+0000 7f2bb208ae40 1 -- 172.21.15.12:0/2343806459 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a6aaded540 msgr2=0x55a6aaded910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.926+0000 7f2bb208ae40 1 --2- 172.21.15.12:0/2343806459 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a6aaded540 0x55a6aaded910 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2ba002ea30 tx=0x7f2ba401e040 comp rx=0 tx=0).stop 2023-10-05T19:40:30.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.926+0000 7f2bb208ae40 1 -- 172.21.15.12:0/2343806459 shutdown_connections 2023-10-05T19:40:30.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.926+0000 7f2bb208ae40 1 --2- 172.21.15.12:0/2343806459 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a6aaded540 0x55a6aaded910 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.926+0000 7f2bb208ae40 1 -- 172.21.15.12:0/2343806459 >> 172.21.15.12:0/2343806459 conn(0x55a6aade8900 msgr2=0x55a6aade6ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:30.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.926+0000 7f2bb208ae40 1 -- 172.21.15.12:0/2343806459 shutdown_connections 2023-10-05T19:40:30.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.926+0000 7f2bb208ae40 1 -- 172.21.15.12:0/2343806459 wait complete. 2023-10-05T19:40:30.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.926+0000 7f2bb208ae40 1 Processor -- start 2023-10-05T19:40:30.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.926+0000 7f2bb208ae40 1 -- start start 2023-10-05T19:40:30.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.926+0000 7f2bb208ae40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a6aaded540 0x55a6aadf3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.926+0000 7f2bb208ae40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6aadf4010 con 0x55a6aaded540 2023-10-05T19:40:30.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.926+0000 7f2baf3be700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a6aaded540 0x55a6aadf3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.926+0000 7f2baf3be700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a6aaded540 0x55a6aadf3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58912/0 (socket says 172.21.15.12:58912) 2023-10-05T19:40:30.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.926+0000 7f2baf3be700 1 -- 172.21.15.12:0/3999809178 learned_addr learned my addr 172.21.15.12:0/3999809178 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:30.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.926+0000 7f2baf3be700 1 -- 172.21.15.12:0/3999809178 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba4034040 con 0x55a6aaded540 2023-10-05T19:40:30.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.930+0000 7f2baf3be700 1 --2- 172.21.15.12:0/3999809178 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a6aaded540 0x55a6aadf3010 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f2ba400e4d0 tx=0x7f2ba403d030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.930+0000 7f2afaffd700 1 -- 172.21.15.12:0/3999809178 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ba402c640 con 0x55a6aaded540 2023-10-05T19:40:30.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.930+0000 7f2afaffd700 1 -- 172.21.15.12:0/3999809178 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba402c7c0 con 0x55a6aaded540 2023-10-05T19:40:30.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.930+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3999809178 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a6aadf3610 con 0x55a6aaded540 2023-10-05T19:40:30.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.930+0000 7f2afaffd700 1 -- 172.21.15.12:0/3999809178 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ba402c970 con 0x55a6aaded540 2023-10-05T19:40:30.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.930+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3999809178 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a6aadf3a50 con 0x55a6aaded540 2023-10-05T19:40:30.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.930+0000 7f2afaffd700 1 -- 172.21.15.12:0/3999809178 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f2ba404d070 con 0x55a6aaded540 2023-10-05T19:40:30.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.930+0000 7f2afaffd700 1 --2- 172.21.15.12:0/3999809178 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2af0030c50 0x7f2af0033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.930+0000 7f2afaffd700 1 -- 172.21.15.12:0/3999809178 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f2ba4032150 con 0x55a6aaded540 2023-10-05T19:40:30.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.930+0000 7f2bb208ae40 5 note: GC not initialized 2023-10-05T19:40:30.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.930+0000 7f2bb03c0700 1 --2- 172.21.15.12:0/3999809178 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2af0030c50 0x7f2af0033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.930+0000 7f2bb03c0700 1 --2- 172.21.15.12:0/3999809178 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2af0030c50 0x7f2af0033100 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x55a6aaddfaa0 tx=0x7f2ba800ab80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.934+0000 7f2aeffff700 20 reqs_thread_entry: start 2023-10-05T19:40:30.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.934+0000 7f2aedffb700 20 reqs_thread_entry: start 2023-10-05T19:40:30.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.934+0000 7f2bb208ae40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:30.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.934+0000 7f2bb208ae40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:30.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.934+0000 7f2adbfff700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:41:01 2023 2023-10-05T19:40:30.945 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:30.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.934+0000 7f2adb7fe700 20 reqs_thread_entry: start 2023-10-05T19:40:30.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.934+0000 7f2bb208ae40 10 log_show_next pos 0 bl 0 off 0 eof 0 2023-10-05T19:40:30.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.934+0000 7f2bb208ae40 1 --2- 172.21.15.12:0/3999809178 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55a6aae5a6a0 0x55a6aae5ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:30.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.934+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3999809178 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:1 6.1f 6:ffc55689:::2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.2-myfoo2:head [read 0~1048576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aae5cfc0 con 0x55a6aae5a6a0 2023-10-05T19:40:30.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.934+0000 7f2bafbbf700 1 --2- 172.21.15.12:0/3999809178 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55a6aae5a6a0 0x55a6aae5ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:30.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.934+0000 7f2bafbbf700 1 --2- 172.21.15.12:0/3999809178 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55a6aae5a6a0 0x55a6aae5ca80 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:30.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.938+0000 7f2bafbbf700 1 -- 172.21.15.12:0/3999809178 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(1 2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.2-myfoo2 [read 0~850 out=850b] v0'0 uv17 ondisk = 0) v8 ==== 208+0+850 (crc 0 0 0) 0x7f2ba002e7e0 con 0x55a6aae5a6a0 2023-10-05T19:40:30.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.938+0000 7f2bb208ae40 10 read 850 2023-10-05T19:40:30.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.938+0000 7f2bb208ae40 10 log_show_next pos 850 bl 850 off 419 eof 1 2023-10-05T19:40:30.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.938+0000 7f2bb208ae40 10 log_show_next pos 850 bl 850 off 850 eof 1 2023-10-05T19:40:30.947 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:40:30.948 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket_id": "93ac49f4-ff49-4f99-811b-6453b28decd5.4221.2", 2023-10-05T19:40:30.948 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket_owner": "foo", 2023-10-05T19:40:30.948 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo2", 2023-10-05T19:40:30.948 INFO:teuthology.orchestra.run.smithi012.stdout: "log_entries": [ 2023-10-05T19:40:30.948 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:30.948 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo2", 2023-10-05T19:40:30.948 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:40:23.335113Z", 2023-10-05T19:40:30.948 INFO:teuthology.orchestra.run.smithi012.stdout: "time_local": "2023-10-05T19:40:23.335113+0000", 2023-10-05T19:40:30.948 INFO:teuthology.orchestra.run.smithi012.stdout: "remote_addr": "172.21.0.51", 2023-10-05T19:40:30.948 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:40:30.948 INFO:teuthology.orchestra.run.smithi012.stdout: "operation": "create_bucket", 2023-10-05T19:40:30.949 INFO:teuthology.orchestra.run.smithi012.stdout: "uri": "PUT /myfoo2/ HTTP/1.1", 2023-10-05T19:40:30.949 INFO:teuthology.orchestra.run.smithi012.stdout: "http_status": "200", 2023-10-05T19:40:30.949 INFO:teuthology.orchestra.run.smithi012.stdout: "error_code": "", 2023-10-05T19:40:30.949 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:40:30.949 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:40:30.949 INFO:teuthology.orchestra.run.smithi012.stdout: "object_size": 0, 2023-10-05T19:40:30.949 INFO:teuthology.orchestra.run.smithi012.stdout: "total_time": 19, 2023-10-05T19:40:30.949 INFO:teuthology.orchestra.run.smithi012.stdout: "user_agent": "Boto/2.49.0 Python/3.8.10 Linux/5.4.0-163-generic", 2023-10-05T19:40:30.949 INFO:teuthology.orchestra.run.smithi012.stdout: "referrer": "", 2023-10-05T19:40:30.949 INFO:teuthology.orchestra.run.smithi012.stdout: "trans_id": "tx000001d5f9806e86de6cd-00651f1127-107b-default", 2023-10-05T19:40:30.949 INFO:teuthology.orchestra.run.smithi012.stdout: "authentication_type": "Local", 2023-10-05T19:40:30.949 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key_id": "9te6NH5mcdcq0Tc5i8i1", 2023-10-05T19:40:30.950 INFO:teuthology.orchestra.run.smithi012.stdout: "temp_url": false 2023-10-05T19:40:30.950 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:30.950 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:30.950 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo2", 2023-10-05T19:40:30.950 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:40:23.439110Z", 2023-10-05T19:40:30.950 INFO:teuthology.orchestra.run.smithi012.stdout: "time_local": "2023-10-05T19:40:23.439110+0000", 2023-10-05T19:40:30.950 INFO:teuthology.orchestra.run.smithi012.stdout: "remote_addr": "172.21.0.51", 2023-10-05T19:40:30.950 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:40:30.950 INFO:teuthology.orchestra.run.smithi012.stdout: "operation": "delete_bucket", 2023-10-05T19:40:30.950 INFO:teuthology.orchestra.run.smithi012.stdout: "uri": "DELETE /myfoo2/ HTTP/1.1", 2023-10-05T19:40:30.950 INFO:teuthology.orchestra.run.smithi012.stdout: "http_status": "204", 2023-10-05T19:40:30.950 INFO:teuthology.orchestra.run.smithi012.stdout: "error_code": "NoContent", 2023-10-05T19:40:30.951 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:40:30.951 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:40:30.951 INFO:teuthology.orchestra.run.smithi012.stdout: "object_size": 0, 2023-10-05T19:40:30.951 INFO:teuthology.orchestra.run.smithi012.stdout: "total_time": 79, 2023-10-05T19:40:30.951 INFO:teuthology.orchestra.run.smithi012.stdout: "user_agent": "Boto/2.49.0 Python/3.8.10 Linux/5.4.0-163-generic", 2023-10-05T19:40:30.951 INFO:teuthology.orchestra.run.smithi012.stdout: "referrer": "", 2023-10-05T19:40:30.951 INFO:teuthology.orchestra.run.smithi012.stdout: "trans_id": "tx000000979af3281d3c777-00651f1127-107b-default", 2023-10-05T19:40:30.951 INFO:teuthology.orchestra.run.smithi012.stdout: "authentication_type": "Local", 2023-10-05T19:40:30.951 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key_id": "9te6NH5mcdcq0Tc5i8i1", 2023-10-05T19:40:30.951 INFO:teuthology.orchestra.run.smithi012.stdout: "temp_url": false 2023-10-05T19:40:30.951 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:30.951 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:30.952 INFO:teuthology.orchestra.run.smithi012.stdout: "log_sum": { 2023-10-05T19:40:30.952 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:40:30.952 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:40:30.952 INFO:teuthology.orchestra.run.smithi012.stdout: "total_time": 98, 2023-10-05T19:40:30.952 INFO:teuthology.orchestra.run.smithi012.stdout: "total_entries": 2 2023-10-05T19:40:30.952 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:30.952 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:40:30.952 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:40:30.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.938+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:102 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94174319585600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aae19cd0 con 0x55a6aadcfce0 2023-10-05T19:40:30.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.942+0000 7f2bafbbf700 1 -- 172.21.15.12:0/3359460706 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(102 notify.0 [watch unwatch cookie 94174319585600] v28'179 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2ba002e7e0 con 0x55a6aadcfce0 2023-10-05T19:40:30.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.942+0000 7f2bb208ae40 20 remove_watcher() i=0 2023-10-05T19:40:30.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.942+0000 7f2bb208ae40 2 removed watcher, disabling cache 2023-10-05T19:40:30.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.942+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94174319589440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadda8f0 con 0x55a6aad9f630 2023-10-05T19:40:30.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.942+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(103 notify.1 [watch unwatch cookie 94174319589440] v28'179 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2ba8014040 con 0x55a6aad9f630 2023-10-05T19:40:30.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.942+0000 7f2bb208ae40 20 remove_watcher() i=1 2023-10-05T19:40:30.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.942+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:104 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94174319587888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aaddb840 con 0x55a6aadd4660 2023-10-05T19:40:30.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.946+0000 7f2baf3be700 1 -- 172.21.15.12:0/3359460706 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(104 notify.2 [watch unwatch cookie 94174319587888] v28'179 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2ba4055020 con 0x55a6aadd4660 2023-10-05T19:40:30.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.946+0000 7f2bb208ae40 20 remove_watcher() i=2 2023-10-05T19:40:30.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.946+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:105 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94174319595792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aaddb840 con 0x55a6aadcfce0 2023-10-05T19:40:30.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.950+0000 7f2bafbbf700 1 -- 172.21.15.12:0/3359460706 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(105 notify.3 [watch unwatch cookie 94174319595792] v28'179 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2ba002e7e0 con 0x55a6aadcfce0 2023-10-05T19:40:30.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.950+0000 7f2bb208ae40 20 remove_watcher() i=3 2023-10-05T19:40:30.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.950+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:106 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94174319598720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aaddb840 con 0x55a6aad9f630 2023-10-05T19:40:30.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.954+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(106 notify.4 [watch unwatch cookie 94174319598720] v28'179 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2ba8014040 con 0x55a6aad9f630 2023-10-05T19:40:30.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.954+0000 7f2bb208ae40 20 remove_watcher() i=4 2023-10-05T19:40:30.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.954+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:107 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94174319600896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadddc80 con 0x7f2ba001abe0 2023-10-05T19:40:30.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.954+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(107 notify.5 [watch unwatch cookie 94174319600896] v28'179 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2ba8014040 con 0x7f2ba001abe0 2023-10-05T19:40:30.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.954+0000 7f2bb208ae40 20 remove_watcher() i=5 2023-10-05T19:40:30.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.954+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:108 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94174319603632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aaddb840 con 0x7f2ba001abe0 2023-10-05T19:40:30.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.958+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(108 notify.6 [watch unwatch cookie 94174319603632] v28'179 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2ba8014040 con 0x7f2ba001abe0 2023-10-05T19:40:30.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.958+0000 7f2bb208ae40 20 remove_watcher() i=6 2023-10-05T19:40:30.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.958+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:109 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94174319602000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadde500 con 0x7f2ba001abe0 2023-10-05T19:40:30.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.962+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(109 notify.7 [watch unwatch cookie 94174319602000] v28'179 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2ba8014040 con 0x7f2ba001abe0 2023-10-05T19:40:30.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.962+0000 7f2bb208ae40 20 remove_watcher() i=7 2023-10-05T19:40:30.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.966+0000 7f2b817e2700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:30.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.966+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:110 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94174319607616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55a6aadddc80 con 0x55a6aad9f630 2023-10-05T19:40:30.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.966+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(110 data_loggenerations_metadata [watch unwatch cookie 94174319607616] v28'166 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f2ba8014040 con 0x55a6aad9f630 2023-10-05T19:40:30.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.966+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f2ba001abe0 msgr2=0x7f2ba002f9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.966+0000 7f2bb208ae40 1 --2- 172.21.15.12:0/3359460706 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f2ba001abe0 0x7f2ba002f9f0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.966+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55a6aad9f630 msgr2=0x55a6aadcdc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.966+0000 7f2bb208ae40 1 --2- 172.21.15.12:0/3359460706 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55a6aad9f630 0x55a6aadcdc00 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.966+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55a6aadcfce0 msgr2=0x55a6aadd20c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.966+0000 7f2bb208ae40 1 --2- 172.21.15.12:0/3359460706 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55a6aadcfce0 0x55a6aadd20c0 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.970+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55a6aadd4660 msgr2=0x55a6aadd6a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.970+0000 7f2bb208ae40 1 --2- 172.21.15.12:0/3359460706 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55a6aadd4660 0x55a6aadd6a40 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.970+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2b9c030c00 msgr2=0x7f2b9c0330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.970+0000 7f2bb208ae40 1 --2- 172.21.15.12:0/3359460706 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2b9c030c00 0x7f2b9c0330b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4001df0 tx=0x7f2ba4001310 comp rx=0 tx=0).stop 2023-10-05T19:40:30.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.970+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a6aad9cd10 msgr2=0x55a6aadc2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.970+0000 7f2bb03c0700 1 -- 172.21.15.12:0/3359460706 reap_dead start 2023-10-05T19:40:30.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.970+0000 7f2bb208ae40 1 --2- 172.21.15.12:0/3359460706 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a6aad9cd10 0x55a6aadc2140 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2ba000a470 tx=0x7f2ba001d000 comp rx=0 tx=0).stop 2023-10-05T19:40:30.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.970+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 shutdown_connections 2023-10-05T19:40:30.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.970+0000 7f2bb208ae40 1 --2- 172.21.15.12:0/3359460706 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a6aad9cd10 0x55a6aadc2140 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.970+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 >> 172.21.15.12:0/3359460706 conn(0x55a6aad0b870 msgr2=0x55a6aadb5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:30.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.970+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 shutdown_connections 2023-10-05T19:40:30.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.970+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3359460706 wait complete. 2023-10-05T19:40:30.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.970+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3999809178 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55a6aae5a6a0 msgr2=0x55a6aae5ca80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.970+0000 7f2bb208ae40 1 --2- 172.21.15.12:0/3999809178 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55a6aae5a6a0 0x55a6aae5ca80 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.970+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3999809178 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2af0030c50 msgr2=0x7f2af0033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.970+0000 7f2bb208ae40 1 --2- 172.21.15.12:0/3999809178 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2af0030c50 0x7f2af0033100 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x55a6aaddfaa0 tx=0x7f2ba800ab80 comp rx=0 tx=0).stop 2023-10-05T19:40:30.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.970+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3999809178 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a6aaded540 msgr2=0x55a6aadf3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:30.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.970+0000 7f2bb208ae40 1 --2- 172.21.15.12:0/3999809178 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a6aaded540 0x55a6aadf3010 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f2ba400e4d0 tx=0x7f2ba403d030 comp rx=0 tx=0).stop 2023-10-05T19:40:30.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.970+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3999809178 shutdown_connections 2023-10-05T19:40:30.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.970+0000 7f2bb208ae40 1 --2- 172.21.15.12:0/3999809178 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55a6aae5a6a0 0x55a6aae5ca80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.970+0000 7f2bb208ae40 1 --2- 172.21.15.12:0/3999809178 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2af0030c50 0x7f2af0033100 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.970+0000 7f2bb208ae40 1 --2- 172.21.15.12:0/3999809178 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a6aaded540 0x55a6aadf3010 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:30.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.970+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3999809178 >> 172.21.15.12:0/3999809178 conn(0x55a6aade8900 msgr2=0x55a6aade6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:30.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.970+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3999809178 shutdown_connections 2023-10-05T19:40:30.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:30.970+0000 7f2bb208ae40 1 -- 172.21.15.12:0/3999809178 wait complete. 2023-10-05T19:40:30.985 DEBUG:tasks.util.rgw: json result: {'bucket_id': '93ac49f4-ff49-4f99-811b-6453b28decd5.4221.2', 'bucket_owner': 'foo', 'bucket': 'myfoo2', 'log_entries': [{'bucket': 'myfoo2', 'time': '2023-10-05T19:40:23.335113Z', 'time_local': '2023-10-05T19:40:23.335113+0000', 'remote_addr': '172.21.0.51', 'user': 'foo', 'operation': 'create_bucket', 'uri': 'PUT /myfoo2/ HTTP/1.1', 'http_status': '200', 'error_code': '', 'bytes_sent': 0, 'bytes_received': 0, 'object_size': 0, 'total_time': 19, 'user_agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-163-generic', 'referrer': '', 'trans_id': 'tx000001d5f9806e86de6cd-00651f1127-107b-default', 'authentication_type': 'Local', 'access_key_id': '9te6NH5mcdcq0Tc5i8i1', 'temp_url': False}, {'bucket': 'myfoo2', 'time': '2023-10-05T19:40:23.439110Z', 'time_local': '2023-10-05T19:40:23.439110+0000', 'remote_addr': '172.21.0.51', 'user': 'foo', 'operation': 'delete_bucket', 'uri': 'DELETE /myfoo2/ HTTP/1.1', 'http_status': '204', 'error_code': 'NoContent', 'bytes_sent': 0, 'bytes_received': 0, 'object_size': 0, 'total_time': 79, 'user_agent': 'Boto/2.49.0 Python/3.8.10 Linux/5.4.0-163-generic', 'referrer': '', 'trans_id': 'tx000000979af3281d3c777-00651f1127-107b-default', 'authentication_type': 'Local', 'access_key_id': '9te6NH5mcdcq0Tc5i8i1', 'temp_url': False}], 'log_sum': {'bytes_sent': 0, 'bytes_received': 0, 'total_time': 98, 'total_entries': 2}} 2023-10-05T19:40:30.985 DEBUG:tasks.radosgw_admin:checking log entry: 2023-10-05T19:40:30.986 DEBUG:tasks.radosgw_admin:checking log entry: 2023-10-05T19:40:30.986 INFO:tasks.util.rgw:rgwadmin: client.0 : ['log', 'rm', '--object', '2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.2-myfoo2'] 2023-10-05T19:40:30.986 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'log', 'rm', '--object', '2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.2-myfoo2'] 2023-10-05T19:40:30.986 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph log rm --object 2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.2-myfoo2 2023-10-05T19:40:31.024 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:31.024 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:31.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.014+0000 7fdaed1a2e40 1 Processor -- start 2023-10-05T19:40:31.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.014+0000 7fdaed1a2e40 1 -- start start 2023-10-05T19:40:31.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.014+0000 7fdaed1a2e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55b77f9b9010 0x55b77f9b93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:31.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.014+0000 7fdaed1a2e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55b77f903bd0 con 0x55b77f9b9920 2023-10-05T19:40:31.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.014+0000 7fdaed1a2e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x55b77f8c0ee0 con 0x55b77f9b9010 2023-10-05T19:40:31.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.014+0000 7fdaeacd7700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55b77f9b9010 0x55b77f9b93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:31.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.014+0000 7fdaea4d6700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x55b77f9b9920 0x55b77f9ba3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:35750/0 (socket says 172.21.15.12:35750) 2023-10-05T19:40:31.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.018+0000 7fdaeacd7700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55b77f9b9010 0x55b77f9b93e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58928/0 (socket says 172.21.15.12:58928) 2023-10-05T19:40:31.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.018+0000 7fdaea4d6700 1 -- 172.21.15.12:0/1439040405 learned_addr learned my addr 172.21.15.12:0/1439040405 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:31.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.018+0000 7fdaeacd7700 1 -- 172.21.15.12:0/1439040405 >> v1:172.21.15.8:6789/0 conn(0x55b77f9b9920 legacy=0x55b77f9ba3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:31.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.018+0000 7fdaeacd7700 1 -- 172.21.15.12:0/1439040405 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b77f9baad0 con 0x55b77f9b9010 2023-10-05T19:40:31.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.018+0000 7fdaeacd7700 1 --2- 172.21.15.12:0/1439040405 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b77f9b9010 0x55b77f9b93e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fdae400a520 tx=0x7fdae4005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d7ff3a780ac8b6c server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:31.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.018+0000 7fdae9cd5700 1 -- 172.21.15.12:0/1439040405 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdae400f590 con 0x55b77f9b9010 2023-10-05T19:40:31.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.018+0000 7fdae9cd5700 1 -- 172.21.15.12:0/1439040405 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae400f6f0 con 0x55b77f9b9010 2023-10-05T19:40:31.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.018+0000 7fdae9cd5700 1 -- 172.21.15.12:0/1439040405 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdae400f880 con 0x55b77f9b9010 2023-10-05T19:40:31.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.018+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/1439040405 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b77f9b9010 msgr2=0x55b77f9b93e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:31.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.018+0000 7fdaed1a2e40 1 --2- 172.21.15.12:0/1439040405 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b77f9b9010 0x55b77f9b93e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fdae400a520 tx=0x7fdae4005d40 comp rx=0 tx=0).stop 2023-10-05T19:40:31.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.018+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/1439040405 shutdown_connections 2023-10-05T19:40:31.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.018+0000 7fdaed1a2e40 1 --2- 172.21.15.12:0/1439040405 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b77f9b9010 0x55b77f9b93e0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:31.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.018+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/1439040405 >> 172.21.15.12:0/1439040405 conn(0x55b77f92c860 msgr2=0x55b77f92cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:31.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.018+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/1439040405 shutdown_connections 2023-10-05T19:40:31.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.018+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/1439040405 wait complete. 2023-10-05T19:40:31.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.018+0000 7fdaed1a2e40 1 Processor -- start 2023-10-05T19:40:31.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.018+0000 7fdaed1a2e40 1 -- start start 2023-10-05T19:40:31.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.018+0000 7fdaed1a2e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b77f9b9010 0x55b77f9d8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:31.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.018+0000 7fdaed1a2e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b77f884e30 con 0x55b77f9b9010 2023-10-05T19:40:31.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.018+0000 7fdaeacd7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b77f9b9010 0x55b77f9d8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:31.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.018+0000 7fdaeacd7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b77f9b9010 0x55b77f9d8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58944/0 (socket says 172.21.15.12:58944) 2023-10-05T19:40:31.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.018+0000 7fdaeacd7700 1 -- 172.21.15.12:0/677837938 learned_addr learned my addr 172.21.15.12:0/677837938 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:31.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.022+0000 7fdaeacd7700 1 -- 172.21.15.12:0/677837938 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae400a0e0 con 0x55b77f9b9010 2023-10-05T19:40:31.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.022+0000 7fdaeacd7700 1 --2- 172.21.15.12:0/677837938 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b77f9b9010 0x55b77f9d8170 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fdae4005b20 tx=0x7fdae4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55e6b285e20b8b72 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:31.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.022+0000 7fdae8cd3700 1 -- 172.21.15.12:0/677837938 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdae400f860 con 0x55b77f9b9010 2023-10-05T19:40:31.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.022+0000 7fdae8cd3700 1 -- 172.21.15.12:0/677837938 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae400f9c0 con 0x55b77f9b9010 2023-10-05T19:40:31.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.022+0000 7fdae8cd3700 1 -- 172.21.15.12:0/677837938 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdae400fb30 con 0x55b77f9b9010 2023-10-05T19:40:31.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.022+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/677837938 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b77f9b9010 msgr2=0x55b77f9d8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:31.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.022+0000 7fdaed1a2e40 1 --2- 172.21.15.12:0/677837938 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b77f9b9010 0x55b77f9d8170 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fdae4005b20 tx=0x7fdae4014000 comp rx=0 tx=0).stop 2023-10-05T19:40:31.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.022+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/677837938 shutdown_connections 2023-10-05T19:40:31.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.022+0000 7fdaed1a2e40 1 --2- 172.21.15.12:0/677837938 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b77f9b9010 0x55b77f9d8170 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:31.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.022+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/677837938 >> 172.21.15.12:0/677837938 conn(0x55b77f92c860 msgr2=0x55b77f9d3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:31.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.022+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/677837938 shutdown_connections 2023-10-05T19:40:31.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.022+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/677837938 wait complete. 2023-10-05T19:40:31.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.022+0000 7fdaed1a2e40 1 Processor -- start 2023-10-05T19:40:31.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.022+0000 7fdaed1a2e40 1 -- start start 2023-10-05T19:40:31.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.026+0000 7fdaed1a2e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b77f9b9010 0x55b77f9dedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:31.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.026+0000 7fdaed1a2e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b77f857ed0 con 0x55b77f9b9010 2023-10-05T19:40:31.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.026+0000 7fdaeacd7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b77f9b9010 0x55b77f9dedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:31.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.026+0000 7fdaeacd7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b77f9b9010 0x55b77f9dedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58950/0 (socket says 172.21.15.12:58950) 2023-10-05T19:40:31.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.026+0000 7fdaeacd7700 1 -- 172.21.15.12:0/2109803331 learned_addr learned my addr 172.21.15.12:0/2109803331 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:31.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.026+0000 7fdaeacd7700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b77f9baad0 con 0x55b77f9b9010 2023-10-05T19:40:31.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.026+0000 7fdaeacd7700 1 --2- 172.21.15.12:0/2109803331 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b77f9b9010 0x55b77f9dedd0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fdae400a520 tx=0x7fdae401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:31.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.026+0000 7fdae2ffd700 1 -- 172.21.15.12:0/2109803331 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdae402a070 con 0x55b77f9b9010 2023-10-05T19:40:31.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.026+0000 7fdae2ffd700 1 -- 172.21.15.12:0/2109803331 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae4010420 con 0x55b77f9b9010 2023-10-05T19:40:31.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.026+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b77f9df310 con 0x55b77f9b9010 2023-10-05T19:40:31.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.026+0000 7fdae2ffd700 1 -- 172.21.15.12:0/2109803331 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdae40105b0 con 0x55b77f9b9010 2023-10-05T19:40:31.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.026+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b77f9df6f0 con 0x55b77f9b9010 2023-10-05T19:40:31.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.030+0000 7fdae2ffd700 1 -- 172.21.15.12:0/2109803331 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fdae4010820 con 0x55b77f9b9010 2023-10-05T19:40:31.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.030+0000 7fdaed1a2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:31.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.030+0000 7fdae2ffd700 1 --2- 172.21.15.12:0/2109803331 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fdacc030c40 0x7fdacc0330f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:31.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.030+0000 7fdae2ffd700 1 -- 172.21.15.12:0/2109803331 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7fdae40463d0 con 0x55b77f9b9010 2023-10-05T19:40:31.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.030+0000 7fdaed1a2e40 1 --2- 172.21.15.12:0/2109803331 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55b77f9b9920 0x55b77f9ea890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:31.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.030+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9eaf90 con 0x55b77f9b9920 2023-10-05T19:40:31.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.030+0000 7fdaea4d6700 1 --2- 172.21.15.12:0/2109803331 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fdacc030c40 0x7fdacc0330f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:31.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.030+0000 7fdaeb4d8700 1 --2- 172.21.15.12:0/2109803331 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55b77f9b9920 0x55b77f9ea890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:31.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.030+0000 7fdaea4d6700 1 --2- 172.21.15.12:0/2109803331 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fdacc030c40 0x7fdacc0330f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdad4004170 tx=0x7fdad4001380 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:31.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.030+0000 7fdaeb4d8700 1 --2- 172.21.15.12:0/2109803331 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55b77f9b9920 0x55b77f9ea890 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:31.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.030+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fdadc00a500 con 0x55b77f9b9920 2023-10-05T19:40:31.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.030+0000 7fdaed1a2e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:31.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.030+0000 7fdaed1a2e40 20 realm 2023-10-05T19:40:31.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.030+0000 7fdaed1a2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:31.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.030+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9e7ef0 con 0x55b77f9b9920 2023-10-05T19:40:31.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.030+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fdadc00a500 con 0x55b77f9b9920 2023-10-05T19:40:31.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.030+0000 7fdaed1a2e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:31.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.030+0000 7fdaed1a2e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:31.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.030+0000 7fdaed1a2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:31.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.030+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9eba70 con 0x55b77f9b9920 2023-10-05T19:40:31.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.034+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fdadc00a500 con 0x55b77f9b9920 2023-10-05T19:40:31.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.034+0000 7fdaed1a2e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:31.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.034+0000 7fdaed1a2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:31.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.034+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9eb680 con 0x55b77f9b9920 2023-10-05T19:40:31.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.034+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fdadc00a500 con 0x55b77f9b9920 2023-10-05T19:40:31.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.034+0000 7fdaed1a2e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:31.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.034+0000 7fdaed1a2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:31.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.034+0000 7fdaed1a2e40 1 --2- 172.21.15.12:0/2109803331 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55b77f9ece10 0x55b77f9ef1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:31.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.034+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9ef730 con 0x55b77f9ece10 2023-10-05T19:40:31.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.034+0000 7fdaeacd7700 1 --2- 172.21.15.12:0/2109803331 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55b77f9ece10 0x55b77f9ef1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:31.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.034+0000 7fdaeacd7700 1 --2- 172.21.15.12:0/2109803331 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55b77f9ece10 0x55b77f9ef1f0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:31.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.034+0000 7fdaeacd7700 1 -- 172.21.15.12:0/2109803331 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fdae4034070 con 0x55b77f9ece10 2023-10-05T19:40:31.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.034+0000 7fdaed1a2e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:31.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.034+0000 7fdaed1a2e40 20 searching for the correct realm 2023-10-05T19:40:31.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.034+0000 7fdaed1a2e40 1 --2- 172.21.15.12:0/2109803331 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55b77f9f15b0 0x55b77f9f39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:31.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.034+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4070 con 0x55b77f9f15b0 2023-10-05T19:40:31.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.034+0000 7fdaea4d6700 1 --2- 172.21.15.12:0/2109803331 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55b77f9f15b0 0x55b77f9f39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:31.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.034+0000 7fdaea4d6700 1 --2- 172.21.15.12:0/2109803331 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55b77f9f15b0 0x55b77f9f39c0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:31.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.038+0000 7fdaea4d6700 1 -- 172.21.15.12:0/2109803331 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fdad4006070 con 0x55b77f9f15b0 2023-10-05T19:40:31.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.038+0000 7fdaea4d6700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4070 con 0x55b77f9ece10 2023-10-05T19:40:31.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.038+0000 7fdaeacd7700 1 -- 172.21.15.12:0/2109803331 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdae403f040 con 0x55b77f9ece10 2023-10-05T19:40:31.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.038+0000 7fdaeacd7700 1 --2- 172.21.15.12:0/2109803331 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fdae401b740 0x7fdae401a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:31.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.038+0000 7fdaeacd7700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4070 con 0x7fdae401b740 2023-10-05T19:40:31.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.038+0000 7fdaeb4d8700 1 --2- 172.21.15.12:0/2109803331 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fdae401b740 0x7fdae401a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:31.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.038+0000 7fdaeb4d8700 1 --2- 172.21.15.12:0/2109803331 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fdae401b740 0x7fdae401a5f0 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:31.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.038+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdadc014040 con 0x7fdae401b740 2023-10-05T19:40:31.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.042+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4070 con 0x55b77f9b9920 2023-10-05T19:40:31.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.042+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdadc014040 con 0x55b77f9b9920 2023-10-05T19:40:31.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.042+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4070 con 0x55b77f9b9920 2023-10-05T19:40:31.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.042+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdadc014040 con 0x55b77f9b9920 2023-10-05T19:40:31.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.042+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4070 con 0x7fdae401b740 2023-10-05T19:40:31.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.042+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdadc014040 con 0x7fdae401b740 2023-10-05T19:40:31.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.042+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4070 con 0x7fdae401b740 2023-10-05T19:40:31.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.042+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdadc014040 con 0x7fdae401b740 2023-10-05T19:40:31.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.042+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4070 con 0x55b77f9ece10 2023-10-05T19:40:31.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.046+0000 7fdaeacd7700 1 -- 172.21.15.12:0/2109803331 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdae403f040 con 0x55b77f9ece10 2023-10-05T19:40:31.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.046+0000 7fdaeacd7700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4070 con 0x55b77f9f15b0 2023-10-05T19:40:31.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.046+0000 7fdaea4d6700 1 -- 172.21.15.12:0/2109803331 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdad4006070 con 0x55b77f9f15b0 2023-10-05T19:40:31.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.046+0000 7fdaea4d6700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4070 con 0x55b77f9b9920 2023-10-05T19:40:31.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.046+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdadc014040 con 0x55b77f9b9920 2023-10-05T19:40:31.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.046+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4070 con 0x7fdae401b740 2023-10-05T19:40:31.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.046+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdadc014040 con 0x7fdae401b740 2023-10-05T19:40:31.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.046+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4070 con 0x55b77f9b9920 2023-10-05T19:40:31.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.046+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdadc014040 con 0x55b77f9b9920 2023-10-05T19:40:31.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.046+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4070 con 0x55b77f9ece10 2023-10-05T19:40:31.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.050+0000 7fdaeacd7700 1 -- 172.21.15.12:0/2109803331 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdae403f040 con 0x55b77f9ece10 2023-10-05T19:40:31.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.050+0000 7fdaeacd7700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4070 con 0x55b77f9f15b0 2023-10-05T19:40:31.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.050+0000 7fdaea4d6700 1 -- 172.21.15.12:0/2109803331 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdad4006070 con 0x55b77f9f15b0 2023-10-05T19:40:31.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.050+0000 7fdaea4d6700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4070 con 0x55b77f9f15b0 2023-10-05T19:40:31.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.050+0000 7fdaea4d6700 1 -- 172.21.15.12:0/2109803331 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdad4006070 con 0x55b77f9f15b0 2023-10-05T19:40:31.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.050+0000 7fdaea4d6700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4070 con 0x7fdae401b740 2023-10-05T19:40:31.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.050+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdadc014040 con 0x7fdae401b740 2023-10-05T19:40:31.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.050+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4070 con 0x55b77f9f15b0 2023-10-05T19:40:31.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.054+0000 7fdaea4d6700 1 -- 172.21.15.12:0/2109803331 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdad4006070 con 0x55b77f9f15b0 2023-10-05T19:40:31.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.054+0000 7fdaea4d6700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4070 con 0x55b77f9f15b0 2023-10-05T19:40:31.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.054+0000 7fdaea4d6700 1 -- 172.21.15.12:0/2109803331 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdad4006070 con 0x55b77f9f15b0 2023-10-05T19:40:31.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.054+0000 7fdaea4d6700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4070 con 0x55b77f9b9920 2023-10-05T19:40:31.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.054+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdadc014040 con 0x55b77f9b9920 2023-10-05T19:40:31.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.054+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4070 con 0x55b77f9b9920 2023-10-05T19:40:31.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.054+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdadc014040 con 0x55b77f9b9920 2023-10-05T19:40:31.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.054+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4070 con 0x7fdae401b740 2023-10-05T19:40:31.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.054+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdadc014040 con 0x7fdae401b740 2023-10-05T19:40:31.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.054+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4070 con 0x55b77f9f15b0 2023-10-05T19:40:31.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.058+0000 7fdaea4d6700 1 -- 172.21.15.12:0/2109803331 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdad4006070 con 0x55b77f9f15b0 2023-10-05T19:40:31.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.058+0000 7fdaea4d6700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4070 con 0x55b77f9ece10 2023-10-05T19:40:31.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.058+0000 7fdaeacd7700 1 -- 172.21.15.12:0/2109803331 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdae403f040 con 0x55b77f9ece10 2023-10-05T19:40:31.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.058+0000 7fdaeacd7700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4070 con 0x55b77f9b9920 2023-10-05T19:40:31.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.058+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdadc014040 con 0x55b77f9b9920 2023-10-05T19:40:31.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.058+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4070 con 0x7fdae401b740 2023-10-05T19:40:31.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.058+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdadc014040 con 0x7fdae401b740 2023-10-05T19:40:31.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.058+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4070 con 0x55b77f9f15b0 2023-10-05T19:40:31.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.062+0000 7fdaea4d6700 1 -- 172.21.15.12:0/2109803331 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdad4006070 con 0x55b77f9f15b0 2023-10-05T19:40:31.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.062+0000 7fdaea4d6700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4070 con 0x7fdae401b740 2023-10-05T19:40:31.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.062+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdadc014040 con 0x7fdae401b740 2023-10-05T19:40:31.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.062+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4070 con 0x55b77f9ece10 2023-10-05T19:40:31.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.062+0000 7fdaeacd7700 1 -- 172.21.15.12:0/2109803331 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdae403f040 con 0x55b77f9ece10 2023-10-05T19:40:31.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.062+0000 7fdaeacd7700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4070 con 0x55b77f9f15b0 2023-10-05T19:40:31.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.062+0000 7fdaea4d6700 1 -- 172.21.15.12:0/2109803331 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdad4006070 con 0x55b77f9f15b0 2023-10-05T19:40:31.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.062+0000 7fdaea4d6700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4070 con 0x7fdae401b740 2023-10-05T19:40:31.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.066+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdadc014040 con 0x7fdae401b740 2023-10-05T19:40:31.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.066+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4070 con 0x55b77f9b9920 2023-10-05T19:40:31.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.066+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fdadc014040 con 0x55b77f9b9920 2023-10-05T19:40:31.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.066+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4070 con 0x55b77f9ece10 2023-10-05T19:40:31.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.066+0000 7fdaeacd7700 1 -- 172.21.15.12:0/2109803331 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fdae403f040 con 0x55b77f9ece10 2023-10-05T19:40:31.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.066+0000 7fdaed1a2e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:31.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.066+0000 7fdaed1a2e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:31.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.066+0000 7fdaed1a2e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:31.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.066+0000 7fdaed1a2e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:31.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.066+0000 7fdaed1a2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:31.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.066+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f0330 con 0x55b77f9b9920 2023-10-05T19:40:31.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.066+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fdadc014040 con 0x55b77f9b9920 2023-10-05T19:40:31.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.066+0000 7fdaed1a2e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:31.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.066+0000 7fdaed1a2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:31.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.066+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9eff60 con 0x55b77f9b9920 2023-10-05T19:40:31.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.066+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fdadc014040 con 0x55b77f9b9920 2023-10-05T19:40:31.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.066+0000 7fdaed1a2e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:31.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.066+0000 7fdaed1a2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:31.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.066+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f44e0 con 0x55b77f9f15b0 2023-10-05T19:40:31.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.070+0000 7fdaea4d6700 1 -- 172.21.15.12:0/2109803331 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fdad4006070 con 0x55b77f9f15b0 2023-10-05T19:40:31.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.070+0000 7fdaed1a2e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:31.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.070+0000 7fdaed1a2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:31.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.070+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4890 con 0x55b77f9f15b0 2023-10-05T19:40:31.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.070+0000 7fdaea4d6700 1 -- 172.21.15.12:0/2109803331 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7fdad4006070 con 0x55b77f9f15b0 2023-10-05T19:40:31.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.070+0000 7fdaed1a2e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:31.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.070+0000 7fdaed1a2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:31.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.070+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4c40 con 0x55b77f9b9920 2023-10-05T19:40:31.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.070+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fdadc014040 con 0x55b77f9b9920 2023-10-05T19:40:31.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.070+0000 7fdaed1a2e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:31.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.070+0000 7fdaed1a2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:31.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.070+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f0820 con 0x55b77f9b9920 2023-10-05T19:40:31.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.070+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fdadc014040 con 0x55b77f9b9920 2023-10-05T19:40:31.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.070+0000 7fdaed1a2e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:31.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.070+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f07f0 con 0x55b77f9f15b0 2023-10-05T19:40:31.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.074+0000 7fdaea4d6700 1 -- 172.21.15.12:0/2109803331 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fdad4006070 con 0x55b77f9f15b0 2023-10-05T19:40:31.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.074+0000 7fdaea4d6700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f07f0 con 0x55b77f9ece10 2023-10-05T19:40:31.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.074+0000 7fdaeacd7700 1 -- 172.21.15.12:0/2109803331 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdae403f040 con 0x55b77f9ece10 2023-10-05T19:40:31.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.074+0000 7fdaeacd7700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f07f0 con 0x7fdae401b740 2023-10-05T19:40:31.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.074+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdadc014040 con 0x7fdae401b740 2023-10-05T19:40:31.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.074+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4c40 con 0x55b77f9b9920 2023-10-05T19:40:31.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.078+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdadc014040 con 0x55b77f9b9920 2023-10-05T19:40:31.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.078+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4c40 con 0x55b77f9b9920 2023-10-05T19:40:31.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.078+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdadc014040 con 0x55b77f9b9920 2023-10-05T19:40:31.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.078+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4c40 con 0x7fdae401b740 2023-10-05T19:40:31.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.078+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdadc014040 con 0x7fdae401b740 2023-10-05T19:40:31.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.078+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4c40 con 0x7fdae401b740 2023-10-05T19:40:31.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.078+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdadc014040 con 0x7fdae401b740 2023-10-05T19:40:31.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.078+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4c40 con 0x55b77f9ece10 2023-10-05T19:40:31.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.078+0000 7fdaeacd7700 1 -- 172.21.15.12:0/2109803331 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdae403f040 con 0x55b77f9ece10 2023-10-05T19:40:31.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.078+0000 7fdaeacd7700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4c40 con 0x55b77f9f15b0 2023-10-05T19:40:31.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.082+0000 7fdaea4d6700 1 -- 172.21.15.12:0/2109803331 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdad4006070 con 0x55b77f9f15b0 2023-10-05T19:40:31.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.082+0000 7fdaea4d6700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4c40 con 0x55b77f9b9920 2023-10-05T19:40:31.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.082+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdadc014040 con 0x55b77f9b9920 2023-10-05T19:40:31.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.082+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4c40 con 0x7fdae401b740 2023-10-05T19:40:31.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.082+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdadc014040 con 0x7fdae401b740 2023-10-05T19:40:31.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.082+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4c40 con 0x55b77f9b9920 2023-10-05T19:40:31.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.082+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdadc014040 con 0x55b77f9b9920 2023-10-05T19:40:31.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.082+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4c40 con 0x55b77f9ece10 2023-10-05T19:40:31.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.086+0000 7fdaeacd7700 1 -- 172.21.15.12:0/2109803331 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdae403f040 con 0x55b77f9ece10 2023-10-05T19:40:31.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.086+0000 7fdaeacd7700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4c40 con 0x55b77f9f15b0 2023-10-05T19:40:31.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.086+0000 7fdaea4d6700 1 -- 172.21.15.12:0/2109803331 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdad4006070 con 0x55b77f9f15b0 2023-10-05T19:40:31.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.086+0000 7fdaea4d6700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4c40 con 0x55b77f9f15b0 2023-10-05T19:40:31.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.086+0000 7fdaea4d6700 1 -- 172.21.15.12:0/2109803331 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdad4006070 con 0x55b77f9f15b0 2023-10-05T19:40:31.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.086+0000 7fdaea4d6700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4c40 con 0x7fdae401b740 2023-10-05T19:40:31.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.086+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdadc014040 con 0x7fdae401b740 2023-10-05T19:40:31.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.086+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4c40 con 0x55b77f9f15b0 2023-10-05T19:40:31.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.090+0000 7fdaea4d6700 1 -- 172.21.15.12:0/2109803331 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdad4006070 con 0x55b77f9f15b0 2023-10-05T19:40:31.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.090+0000 7fdaea4d6700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4c40 con 0x55b77f9f15b0 2023-10-05T19:40:31.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.090+0000 7fdaea4d6700 1 -- 172.21.15.12:0/2109803331 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdad4006070 con 0x55b77f9f15b0 2023-10-05T19:40:31.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.090+0000 7fdaea4d6700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4c40 con 0x55b77f9b9920 2023-10-05T19:40:31.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.090+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdadc014040 con 0x55b77f9b9920 2023-10-05T19:40:31.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.090+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4c40 con 0x55b77f9b9920 2023-10-05T19:40:31.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.090+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdadc014040 con 0x55b77f9b9920 2023-10-05T19:40:31.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.090+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4c40 con 0x7fdae401b740 2023-10-05T19:40:31.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.094+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdadc014040 con 0x7fdae401b740 2023-10-05T19:40:31.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.094+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4c40 con 0x55b77f9f15b0 2023-10-05T19:40:31.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.094+0000 7fdaea4d6700 1 -- 172.21.15.12:0/2109803331 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdad4006070 con 0x55b77f9f15b0 2023-10-05T19:40:31.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.094+0000 7fdaea4d6700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4c40 con 0x55b77f9ece10 2023-10-05T19:40:31.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.094+0000 7fdaeacd7700 1 -- 172.21.15.12:0/2109803331 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdae403f040 con 0x55b77f9ece10 2023-10-05T19:40:31.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.094+0000 7fdaeacd7700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4c40 con 0x55b77f9b9920 2023-10-05T19:40:31.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.094+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdadc014040 con 0x55b77f9b9920 2023-10-05T19:40:31.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.094+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4c40 con 0x7fdae401b740 2023-10-05T19:40:31.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.094+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdadc014040 con 0x7fdae401b740 2023-10-05T19:40:31.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.094+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4c40 con 0x55b77f9f15b0 2023-10-05T19:40:31.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.098+0000 7fdaea4d6700 1 -- 172.21.15.12:0/2109803331 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdad4006070 con 0x55b77f9f15b0 2023-10-05T19:40:31.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.098+0000 7fdaea4d6700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4c40 con 0x7fdae401b740 2023-10-05T19:40:31.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.098+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdadc014040 con 0x7fdae401b740 2023-10-05T19:40:31.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.098+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4c40 con 0x55b77f9ece10 2023-10-05T19:40:31.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.098+0000 7fdaeacd7700 1 -- 172.21.15.12:0/2109803331 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdae403f040 con 0x55b77f9ece10 2023-10-05T19:40:31.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.098+0000 7fdaeacd7700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4c40 con 0x55b77f9f15b0 2023-10-05T19:40:31.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.098+0000 7fdaea4d6700 1 -- 172.21.15.12:0/2109803331 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdad4006070 con 0x55b77f9f15b0 2023-10-05T19:40:31.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.098+0000 7fdaea4d6700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4c40 con 0x7fdae401b740 2023-10-05T19:40:31.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.102+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdadc014040 con 0x7fdae401b740 2023-10-05T19:40:31.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.102+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4c40 con 0x55b77f9b9920 2023-10-05T19:40:31.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.102+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fdadc014040 con 0x55b77f9b9920 2023-10-05T19:40:31.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.102+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f4c40 con 0x55b77f9ece10 2023-10-05T19:40:31.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.102+0000 7fdaeacd7700 1 -- 172.21.15.12:0/2109803331 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fdae403f040 con 0x55b77f9ece10 2023-10-05T19:40:31.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.102+0000 7fdaed1a2e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:31.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.102+0000 7fdaed1a2e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:31.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.102+0000 7fdaed1a2e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:31.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.102+0000 7fdaed1a2e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:31.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.102+0000 7fdaed1a2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:31.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.102+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f07f0 con 0x55b77f9b9920 2023-10-05T19:40:31.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.102+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fdadc014040 con 0x55b77f9b9920 2023-10-05T19:40:31.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.102+0000 7fdaed1a2e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:31.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.102+0000 7fdaed1a2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:31.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.102+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f5580 con 0x55b77f9ece10 2023-10-05T19:40:31.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.106+0000 7fdaeacd7700 1 -- 172.21.15.12:0/2109803331 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fdae403f040 con 0x55b77f9ece10 2023-10-05T19:40:31.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.106+0000 7fdaed1a2e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:31.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.106+0000 7fdaed1a2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:31.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.106+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f5e20 con 0x55b77f9b9920 2023-10-05T19:40:31.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.106+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fdadc014040 con 0x55b77f9b9920 2023-10-05T19:40:31.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.106+0000 7fdaed1a2e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:31.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.106+0000 7fdaed1a2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:31.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.106+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f5e20 con 0x55b77f9f15b0 2023-10-05T19:40:31.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.106+0000 7fdaea4d6700 1 -- 172.21.15.12:0/2109803331 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fdad4006070 con 0x55b77f9f15b0 2023-10-05T19:40:31.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.106+0000 7fdaed1a2e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:31.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.106+0000 7fdaed1a2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:31.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.106+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f5a30 con 0x7fdae401b740 2023-10-05T19:40:31.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.106+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7fdadc014040 con 0x7fdae401b740 2023-10-05T19:40:31.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.106+0000 7fdaed1a2e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:31.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.106+0000 7fdaed1a2e40 20 zone default found 2023-10-05T19:40:31.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.106+0000 7fdaed1a2e40 4 Realm: () 2023-10-05T19:40:31.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.106+0000 7fdaed1a2e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:31.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.106+0000 7fdaed1a2e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:31.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.106+0000 7fdaed1a2e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:31.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.106+0000 7fdaed1a2e40 20 zonegroup default 2023-10-05T19:40:31.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.106+0000 7fdaed1a2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:31.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.106+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f5a30 con 0x7fdae401b740 2023-10-05T19:40:31.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.110+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fdadc014040 con 0x7fdae401b740 2023-10-05T19:40:31.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.110+0000 7fdaed1a2e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:31.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.110+0000 7fdaed1a2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:31.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.110+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f5a30 con 0x55b77f9f15b0 2023-10-05T19:40:31.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.110+0000 7fdaea4d6700 1 -- 172.21.15.12:0/2109803331 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fdad4006070 con 0x55b77f9f15b0 2023-10-05T19:40:31.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.110+0000 7fdaed1a2e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:31.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.110+0000 7fdaed1a2e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:31.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.110+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f8040 con 0x55b77f9f15b0 2023-10-05T19:40:31.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.110+0000 7fdaea4d6700 1 -- 172.21.15.12:0/2109803331 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fdad4006070 con 0x55b77f9f15b0 2023-10-05T19:40:31.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.110+0000 7fdaed1a2e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:31.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.110+0000 7fdaed1a2e40 20 started sync module instance, tier type = 2023-10-05T19:40:31.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.110+0000 7fdaed1a2e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:31.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.110+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f7700 con 0x55b77f9ece10 2023-10-05T19:40:31.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.114+0000 7fdaeacd7700 1 -- 172.21.15.12:0/2109803331 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'180 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdae403f040 con 0x55b77f9ece10 2023-10-05T19:40:31.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.114+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94246608534272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f7330 con 0x55b77f9ece10 2023-10-05T19:40:31.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.114+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f8dd0 con 0x55b77f9b9920 2023-10-05T19:40:31.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.118+0000 7fdaeacd7700 1 -- 172.21.15.12:0/2109803331 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94246608534272] v28'181 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdae403f040 con 0x55b77f9ece10 2023-10-05T19:40:31.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.118+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'180 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdadc014040 con 0x55b77f9b9920 2023-10-05T19:40:31.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.118+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94246608536560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f8d30 con 0x55b77f9b9920 2023-10-05T19:40:31.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.118+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f9610 con 0x55b77f9f15b0 2023-10-05T19:40:31.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.122+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94246608536560] v28'181 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdadc014040 con 0x55b77f9b9920 2023-10-05T19:40:31.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.122+0000 7fdaea4d6700 1 -- 172.21.15.12:0/2109803331 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'180 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdad4006070 con 0x55b77f9f15b0 2023-10-05T19:40:31.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.122+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94246608537952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f9a10 con 0x55b77f9f15b0 2023-10-05T19:40:31.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.122+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9fa2b0 con 0x55b77f9ece10 2023-10-05T19:40:31.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.122+0000 7fdaea4d6700 1 -- 172.21.15.12:0/2109803331 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94246608537952] v28'181 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdad4006070 con 0x55b77f9f15b0 2023-10-05T19:40:31.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.122+0000 7fdaeacd7700 1 -- 172.21.15.12:0/2109803331 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'180 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdae403f040 con 0x55b77f9ece10 2023-10-05T19:40:31.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.122+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94246608543168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9fa700 con 0x55b77f9ece10 2023-10-05T19:40:31.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.122+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9fafa0 con 0x55b77f9b9920 2023-10-05T19:40:31.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.126+0000 7fdaeacd7700 1 -- 172.21.15.12:0/2109803331 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94246608543168] v28'181 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdae403f040 con 0x55b77f9ece10 2023-10-05T19:40:31.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.126+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'180 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdadc014040 con 0x55b77f9b9920 2023-10-05T19:40:31.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.126+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94246608544832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9fad80 con 0x55b77f9b9920 2023-10-05T19:40:31.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.126+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9fb820 con 0x7fdae401b740 2023-10-05T19:40:31.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.130+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94246608544832] v28'181 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdadc014040 con 0x55b77f9b9920 2023-10-05T19:40:31.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.130+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'180 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdadc014040 con 0x7fdae401b740 2023-10-05T19:40:31.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.130+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94246608549216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9fad80 con 0x7fdae401b740 2023-10-05T19:40:31.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.130+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9fc280 con 0x7fdae401b740 2023-10-05T19:40:31.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.134+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(95 notify.5 [watch watch cookie 94246608549216] v28'181 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdadc014040 con 0x7fdae401b740 2023-10-05T19:40:31.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.134+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(96 notify.6 [create] v28'180 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdadc014040 con 0x7fdae401b740 2023-10-05T19:40:31.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.134+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94246608546496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9fc280 con 0x7fdae401b740 2023-10-05T19:40:31.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.134+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9fcc90 con 0x7fdae401b740 2023-10-05T19:40:31.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.134+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(97 notify.6 [watch watch cookie 94246608546496] v28'181 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdadc014040 con 0x7fdae401b740 2023-10-05T19:40:31.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.134+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(98 notify.7 [create] v28'180 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdadc014040 con 0x7fdae401b740 2023-10-05T19:40:31.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.134+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94246608551872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9fcc90 con 0x7fdae401b740 2023-10-05T19:40:31.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.134+0000 7fdaed1a2e40 20 add_watcher() i=0 2023-10-05T19:40:31.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.134+0000 7fdaed1a2e40 20 add_watcher() i=1 2023-10-05T19:40:31.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.134+0000 7fdaed1a2e40 20 add_watcher() i=2 2023-10-05T19:40:31.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.134+0000 7fdaed1a2e40 20 add_watcher() i=3 2023-10-05T19:40:31.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.134+0000 7fdaed1a2e40 20 add_watcher() i=4 2023-10-05T19:40:31.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.134+0000 7fdaed1a2e40 20 add_watcher() i=5 2023-10-05T19:40:31.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.134+0000 7fdaed1a2e40 20 add_watcher() i=6 2023-10-05T19:40:31.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.138+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94246608551872] v28'181 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdadc014040 con 0x7fdae401b740 2023-10-05T19:40:31.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.138+0000 7fdaed1a2e40 20 add_watcher() i=7 2023-10-05T19:40:31.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.138+0000 7fdaed1a2e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:31.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.138+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9ffd60 con 0x55b77f9b9920 2023-10-05T19:40:31.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.138+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fdadc014040 con 0x55b77f9b9920 2023-10-05T19:40:31.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.138+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94246608553888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9fd0e0 con 0x55b77f9b9920 2023-10-05T19:40:31.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.142+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94246608553888] v28'167 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fdadc014040 con 0x55b77f9b9920 2023-10-05T19:40:31.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.142+0000 7fda37fff700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:31.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.142+0000 7fdaed1a2e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:31.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.142+0000 7fdaed1a2e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:31.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.142+0000 7fdaed1a2e40 1 Processor -- start 2023-10-05T19:40:31.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.142+0000 7fdaed1a2e40 1 -- start start 2023-10-05T19:40:31.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.142+0000 7fdaed1a2e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b77fa0a5d0 0x55b77fa0a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:31.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.142+0000 7fdaed1a2e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b77fa0aee0 con 0x55b77fa0a5d0 2023-10-05T19:40:31.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.142+0000 7fdaea4d6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b77fa0a5d0 0x55b77fa0a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:31.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.142+0000 7fdaea4d6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b77fa0a5d0 0x55b77fa0a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58964/0 (socket says 172.21.15.12:58964) 2023-10-05T19:40:31.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.142+0000 7fdaea4d6700 1 -- 172.21.15.12:0/2528014951 learned_addr learned my addr 172.21.15.12:0/2528014951 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:31.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.142+0000 7fdaea4d6700 1 -- 172.21.15.12:0/2528014951 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b77fa0b020 con 0x55b77fa0a5d0 2023-10-05T19:40:31.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.142+0000 7fdaea4d6700 1 --2- 172.21.15.12:0/2528014951 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b77fa0a5d0 0x55b77fa0a9a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fdad4000c00 tx=0x7fdad401e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dfe6da45da90b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:31.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.146+0000 7fda377fe700 1 -- 172.21.15.12:0/2528014951 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdad4013040 con 0x55b77fa0a5d0 2023-10-05T19:40:31.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.146+0000 7fda377fe700 1 -- 172.21.15.12:0/2528014951 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdad4022040 con 0x55b77fa0a5d0 2023-10-05T19:40:31.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.146+0000 7fda377fe700 1 -- 172.21.15.12:0/2528014951 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdad400e950 con 0x55b77fa0a5d0 2023-10-05T19:40:31.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.146+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2528014951 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b77fa0a5d0 msgr2=0x55b77fa0a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:31.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.146+0000 7fdaed1a2e40 1 --2- 172.21.15.12:0/2528014951 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b77fa0a5d0 0x55b77fa0a9a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fdad4000c00 tx=0x7fdad401e040 comp rx=0 tx=0).stop 2023-10-05T19:40:31.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.146+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2528014951 shutdown_connections 2023-10-05T19:40:31.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.146+0000 7fdaed1a2e40 1 --2- 172.21.15.12:0/2528014951 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b77fa0a5d0 0x55b77fa0a9a0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:31.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.146+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2528014951 >> 172.21.15.12:0/2528014951 conn(0x55b77fa05990 msgr2=0x55b77fa03ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:31.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.146+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2528014951 shutdown_connections 2023-10-05T19:40:31.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.146+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2528014951 wait complete. 2023-10-05T19:40:31.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.146+0000 7fdaed1a2e40 1 Processor -- start 2023-10-05T19:40:31.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.146+0000 7fdaed1a2e40 1 -- start start 2023-10-05T19:40:31.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.146+0000 7fdaed1a2e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b77fa0a5d0 0x55b77fa101c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:31.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.146+0000 7fdaed1a2e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b77fa111c0 con 0x55b77fa0a5d0 2023-10-05T19:40:31.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.146+0000 7fdaea4d6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b77fa0a5d0 0x55b77fa101c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:31.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.146+0000 7fdaea4d6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b77fa0a5d0 0x55b77fa101c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58966/0 (socket says 172.21.15.12:58966) 2023-10-05T19:40:31.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.146+0000 7fdaea4d6700 1 -- 172.21.15.12:0/494213962 learned_addr learned my addr 172.21.15.12:0/494213962 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:31.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.150+0000 7fdaea4d6700 1 -- 172.21.15.12:0/494213962 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad4034040 con 0x55b77fa0a5d0 2023-10-05T19:40:31.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.150+0000 7fdaea4d6700 1 --2- 172.21.15.12:0/494213962 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b77fa0a5d0 0x55b77fa101c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fdad400efd0 tx=0x7fdad403d030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:31.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.150+0000 7fda35ffb700 1 -- 172.21.15.12:0/494213962 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdad404f430 con 0x55b77fa0a5d0 2023-10-05T19:40:31.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.150+0000 7fda35ffb700 1 -- 172.21.15.12:0/494213962 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdad404f590 con 0x55b77fa0a5d0 2023-10-05T19:40:31.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.150+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/494213962 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b77fa107c0 con 0x55b77fa0a5d0 2023-10-05T19:40:31.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.150+0000 7fda35ffb700 1 -- 172.21.15.12:0/494213962 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdad404f780 con 0x55b77fa0a5d0 2023-10-05T19:40:31.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.150+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/494213962 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b77fa10c00 con 0x55b77fa0a5d0 2023-10-05T19:40:31.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.150+0000 7fda35ffb700 1 -- 172.21.15.12:0/494213962 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fdad404d040 con 0x55b77fa0a5d0 2023-10-05T19:40:31.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.150+0000 7fda35ffb700 1 --2- 172.21.15.12:0/494213962 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fda28030c00 0x7fda280330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:31.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.150+0000 7fda35ffb700 1 -- 172.21.15.12:0/494213962 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7fdad402f2a0 con 0x55b77fa0a5d0 2023-10-05T19:40:31.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.150+0000 7fdaeb4d8700 1 --2- 172.21.15.12:0/494213962 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fda28030c00 0x7fda280330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:31.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.150+0000 7fdaed1a2e40 5 note: GC not initialized 2023-10-05T19:40:31.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.154+0000 7fdaeb4d8700 1 --2- 172.21.15.12:0/494213962 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fda28030c00 0x7fda280330b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x55b77f9ffe60 tx=0x7fdadc00ab80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:31.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.154+0000 7fda2effd700 20 reqs_thread_entry: start 2023-10-05T19:40:31.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.154+0000 7fda2cff9700 20 reqs_thread_entry: start 2023-10-05T19:40:31.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.154+0000 7fdaed1a2e40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:31.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.154+0000 7fdaed1a2e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:31.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.154+0000 7fda1affd700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:41:01 2023 2023-10-05T19:40:31.166 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:31.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.154+0000 7fda1a7fc700 20 reqs_thread_entry: start 2023-10-05T19:40:31.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.154+0000 7fdaed1a2e40 1 --2- 172.21.15.12:0/494213962 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55b77fa77710 0x55b77fa79af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:31.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.154+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/494213962 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:1 6.1f 6:ffc55689:::2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.2-myfoo2:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77fa7a030 con 0x55b77fa77710 2023-10-05T19:40:31.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.154+0000 7fdaeacd7700 1 --2- 172.21.15.12:0/494213962 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55b77fa77710 0x55b77fa79af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:31.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.158+0000 7fdaeacd7700 1 --2- 172.21.15.12:0/494213962 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55b77fa77710 0x55b77fa79af0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:31.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.158+0000 7fdaeacd7700 1 -- 172.21.15.12:0/494213962 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(1 2023-10-05-19-93ac49f4-ff49-4f99-811b-6453b28decd5.4221.2-myfoo2 [delete] v28'23 uv23 ondisk = 0) v8 ==== 208+0+0 (crc 0 0 0) 0x7fdae403f040 con 0x55b77fa77710 2023-10-05T19:40:31.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.158+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:102 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94246608534272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77fa36a10 con 0x55b77f9ece10 2023-10-05T19:40:31.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.162+0000 7fdaeacd7700 1 -- 172.21.15.12:0/2109803331 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(102 notify.0 [watch unwatch cookie 94246608534272] v28'182 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdae403f040 con 0x55b77f9ece10 2023-10-05T19:40:31.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.162+0000 7fdaed1a2e40 20 remove_watcher() i=0 2023-10-05T19:40:31.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.162+0000 7fdaed1a2e40 2 removed watcher, disabling cache 2023-10-05T19:40:31.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.162+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94246608536560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f7ab0 con 0x55b77f9b9920 2023-10-05T19:40:31.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.166+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(103 notify.1 [watch unwatch cookie 94246608536560] v28'182 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdadc014040 con 0x55b77f9b9920 2023-10-05T19:40:31.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.166+0000 7fdaed1a2e40 20 remove_watcher() i=1 2023-10-05T19:40:31.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.166+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:104 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94246608537952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f83f0 con 0x55b77f9f15b0 2023-10-05T19:40:31.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.166+0000 7fdaea4d6700 1 -- 172.21.15.12:0/2109803331 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(104 notify.2 [watch unwatch cookie 94246608537952] v28'182 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdad4006070 con 0x55b77f9f15b0 2023-10-05T19:40:31.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.170+0000 7fdaed1a2e40 20 remove_watcher() i=2 2023-10-05T19:40:31.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.170+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:105 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94246608543168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f83f0 con 0x55b77f9ece10 2023-10-05T19:40:31.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.170+0000 7fdaeacd7700 1 -- 172.21.15.12:0/2109803331 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(105 notify.3 [watch unwatch cookie 94246608543168] v28'182 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdae403f040 con 0x55b77f9ece10 2023-10-05T19:40:31.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.170+0000 7fdaed1a2e40 20 remove_watcher() i=3 2023-10-05T19:40:31.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.170+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:106 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94246608544832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f83f0 con 0x55b77f9b9920 2023-10-05T19:40:31.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.174+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(106 notify.4 [watch unwatch cookie 94246608544832] v28'182 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdadc014040 con 0x55b77f9b9920 2023-10-05T19:40:31.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.174+0000 7fdaed1a2e40 20 remove_watcher() i=4 2023-10-05T19:40:31.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.174+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:107 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94246608549216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9fa440 con 0x7fdae401b740 2023-10-05T19:40:31.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.178+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(107 notify.5 [watch unwatch cookie 94246608549216] v28'182 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdadc014040 con 0x7fdae401b740 2023-10-05T19:40:31.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.178+0000 7fdaed1a2e40 20 remove_watcher() i=5 2023-10-05T19:40:31.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.178+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:108 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94246608546496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9f83f0 con 0x7fdae401b740 2023-10-05T19:40:31.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.178+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(108 notify.6 [watch unwatch cookie 94246608546496] v28'182 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdadc014040 con 0x7fdae401b740 2023-10-05T19:40:31.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.178+0000 7fdaed1a2e40 20 remove_watcher() i=6 2023-10-05T19:40:31.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.178+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:109 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94246608551872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9fa440 con 0x7fdae401b740 2023-10-05T19:40:31.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.182+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(109 notify.7 [watch unwatch cookie 94246608551872] v28'182 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdadc014040 con 0x7fdae401b740 2023-10-05T19:40:31.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.182+0000 7fdaed1a2e40 20 remove_watcher() i=7 2023-10-05T19:40:31.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.186+0000 7fda37fff700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:31.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.186+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:110 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94246608553888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x55b77f9faac0 con 0x55b77f9b9920 2023-10-05T19:40:31.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.186+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(110 data_loggenerations_metadata [watch unwatch cookie 94246608553888] v28'168 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fdadc014040 con 0x55b77f9b9920 2023-10-05T19:40:31.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.186+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fdae401b740 msgr2=0x7fdae401a5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:31.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.186+0000 7fdaed1a2e40 1 --2- 172.21.15.12:0/2109803331 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fdae401b740 0x7fdae401a5f0 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:31.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.186+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55b77f9b9920 msgr2=0x55b77f9ea890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:31.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.186+0000 7fdaed1a2e40 1 --2- 172.21.15.12:0/2109803331 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55b77f9b9920 0x55b77f9ea890 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:31.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.186+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55b77f9ece10 msgr2=0x55b77f9ef1f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:31.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.186+0000 7fdaed1a2e40 1 --2- 172.21.15.12:0/2109803331 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55b77f9ece10 0x55b77f9ef1f0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:31.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.186+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55b77f9f15b0 msgr2=0x55b77f9f39c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:31.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.186+0000 7fdaed1a2e40 1 --2- 172.21.15.12:0/2109803331 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55b77f9f15b0 0x55b77f9f39c0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:31.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.186+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fdacc030c40 msgr2=0x7fdacc0330f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:31.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.186+0000 7fdaed1a2e40 1 --2- 172.21.15.12:0/2109803331 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fdacc030c40 0x7fdacc0330f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdad4004170 tx=0x7fdad4001380 comp rx=0 tx=0).stop 2023-10-05T19:40:31.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.186+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b77f9b9010 msgr2=0x55b77f9dedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:31.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.186+0000 7fdaeb4d8700 1 -- 172.21.15.12:0/2109803331 reap_dead start 2023-10-05T19:40:31.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.186+0000 7fdaed1a2e40 1 --2- 172.21.15.12:0/2109803331 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b77f9b9010 0x55b77f9dedd0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fdae400a520 tx=0x7fdae401d000 comp rx=0 tx=0).stop 2023-10-05T19:40:31.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.190+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 shutdown_connections 2023-10-05T19:40:31.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.190+0000 7fdaed1a2e40 1 --2- 172.21.15.12:0/2109803331 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b77f9b9010 0x55b77f9dedd0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:31.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.190+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 >> 172.21.15.12:0/2109803331 conn(0x55b77f92c860 msgr2=0x55b77f9d27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:31.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.190+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 shutdown_connections 2023-10-05T19:40:31.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.190+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/2109803331 wait complete. 2023-10-05T19:40:31.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.190+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/494213962 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55b77fa77710 msgr2=0x55b77fa79af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:31.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.190+0000 7fdaed1a2e40 1 --2- 172.21.15.12:0/494213962 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55b77fa77710 0x55b77fa79af0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:31.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.190+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/494213962 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fda28030c00 msgr2=0x7fda280330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:31.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.190+0000 7fdaed1a2e40 1 --2- 172.21.15.12:0/494213962 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fda28030c00 0x7fda280330b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x55b77f9ffe60 tx=0x7fdadc00ab80 comp rx=0 tx=0).stop 2023-10-05T19:40:31.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.190+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/494213962 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b77fa0a5d0 msgr2=0x55b77fa101c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:31.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.190+0000 7fdaed1a2e40 1 --2- 172.21.15.12:0/494213962 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b77fa0a5d0 0x55b77fa101c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fdad400efd0 tx=0x7fdad403d030 comp rx=0 tx=0).stop 2023-10-05T19:40:31.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.190+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/494213962 shutdown_connections 2023-10-05T19:40:31.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.190+0000 7fdaed1a2e40 1 --2- 172.21.15.12:0/494213962 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55b77fa77710 0x55b77fa79af0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:31.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.190+0000 7fdaed1a2e40 1 --2- 172.21.15.12:0/494213962 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fda28030c00 0x7fda280330b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:31.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.190+0000 7fdaed1a2e40 1 --2- 172.21.15.12:0/494213962 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b77fa0a5d0 0x55b77fa101c0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:31.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.190+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/494213962 >> 172.21.15.12:0/494213962 conn(0x55b77fa05990 msgr2=0x55b77fa03a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:31.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.190+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/494213962 shutdown_connections 2023-10-05T19:40:31.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.190+0000 7fdaed1a2e40 1 -- 172.21.15.12:0/494213962 wait complete. 2023-10-05T19:40:31.206 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'suspend', '--uid', 'foo'] 2023-10-05T19:40:31.206 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'suspend', '--uid', 'foo'] 2023-10-05T19:40:31.206 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user suspend --uid foo 2023-10-05T19:40:31.288 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:31.288 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:31.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.278+0000 7f6d96433e40 1 Processor -- start 2023-10-05T19:40:31.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.278+0000 7f6d96433e40 1 -- start start 2023-10-05T19:40:31.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.278+0000 7f6d96433e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x5623cc7d2410 0x5623cc7d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:31.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.278+0000 7f6d96433e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5623cc71ebd0 con 0x5623cc7d27e0 2023-10-05T19:40:31.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.278+0000 7f6d96433e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x5623cc6dbee0 con 0x5623cc7d2410 2023-10-05T19:40:31.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.282+0000 7f6d93f68700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x5623cc7d2410 0x5623cc7d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:31.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.282+0000 7f6d93767700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x5623cc7d27e0 0x5623cc7d1500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:35756/0 (socket says 172.21.15.12:35756) 2023-10-05T19:40:31.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.282+0000 7f6d93f68700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x5623cc7d2410 0x5623cc7d0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58968/0 (socket says 172.21.15.12:58968) 2023-10-05T19:40:31.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.282+0000 7f6d93767700 1 -- 172.21.15.12:0/1926223527 learned_addr learned my addr 172.21.15.12:0/1926223527 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:31.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.282+0000 7f6d93f68700 1 -- 172.21.15.12:0/1926223527 >> v1:172.21.15.8:6789/0 conn(0x5623cc7d27e0 legacy=0x5623cc7d1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:31.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.282+0000 7f6d93f68700 1 -- 172.21.15.12:0/1926223527 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623cc7d1cd0 con 0x5623cc7d2410 2023-10-05T19:40:31.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.282+0000 7f6d93f68700 1 --2- 172.21.15.12:0/1926223527 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623cc7d2410 0x5623cc7d0fc0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f6d8400a5d0 tx=0x7f6d84005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91606a8bdc66d122 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:31.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.282+0000 7f6d92f66700 1 -- 172.21.15.12:0/1926223527 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d840086a0 con 0x5623cc7d2410 2023-10-05T19:40:31.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.282+0000 7f6d92f66700 1 -- 172.21.15.12:0/1926223527 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d84008800 con 0x5623cc7d2410 2023-10-05T19:40:31.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.282+0000 7f6d92f66700 1 -- 172.21.15.12:0/1926223527 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d84008ad0 con 0x5623cc7d2410 2023-10-05T19:40:31.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.282+0000 7f6d96433e40 1 -- 172.21.15.12:0/1926223527 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623cc7d2410 msgr2=0x5623cc7d0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:31.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.282+0000 7f6d96433e40 1 --2- 172.21.15.12:0/1926223527 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623cc7d2410 0x5623cc7d0fc0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f6d8400a5d0 tx=0x7f6d84005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:31.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.282+0000 7f6d96433e40 1 -- 172.21.15.12:0/1926223527 shutdown_connections 2023-10-05T19:40:31.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.282+0000 7f6d96433e40 1 --2- 172.21.15.12:0/1926223527 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623cc7d2410 0x5623cc7d0fc0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:31.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.282+0000 7f6d96433e40 1 -- 172.21.15.12:0/1926223527 >> 172.21.15.12:0/1926223527 conn(0x5623cc7cc720 msgr2=0x5623cc7ceb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:31.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.282+0000 7f6d96433e40 1 -- 172.21.15.12:0/1926223527 shutdown_connections 2023-10-05T19:40:31.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.282+0000 7f6d96433e40 1 -- 172.21.15.12:0/1926223527 wait complete. 2023-10-05T19:40:31.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.282+0000 7f6d96433e40 1 Processor -- start 2023-10-05T19:40:31.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.282+0000 7f6d96433e40 1 -- start start 2023-10-05T19:40:31.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.286+0000 7f6d96433e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623cc7d2410 0x5623cc7f3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:31.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.286+0000 7f6d96433e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623cc69fe30 con 0x5623cc7d2410 2023-10-05T19:40:31.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.286+0000 7f6d93f68700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623cc7d2410 0x5623cc7f3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:31.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.286+0000 7f6d93f68700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623cc7d2410 0x5623cc7f3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58976/0 (socket says 172.21.15.12:58976) 2023-10-05T19:40:31.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.286+0000 7f6d93f68700 1 -- 172.21.15.12:0/4113760687 learned_addr learned my addr 172.21.15.12:0/4113760687 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:31.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.286+0000 7f6d93f68700 1 -- 172.21.15.12:0/4113760687 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d84005cf0 con 0x5623cc7d2410 2023-10-05T19:40:31.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.286+0000 7f6d93f68700 1 --2- 172.21.15.12:0/4113760687 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623cc7d2410 0x5623cc7f3150 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6d84000c00 tx=0x7f6d84015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f4353683d26607e server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:31.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.286+0000 7f6d91f64700 1 -- 172.21.15.12:0/4113760687 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d84008970 con 0x5623cc7d2410 2023-10-05T19:40:31.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.286+0000 7f6d91f64700 1 -- 172.21.15.12:0/4113760687 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d84008af0 con 0x5623cc7d2410 2023-10-05T19:40:31.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.286+0000 7f6d91f64700 1 -- 172.21.15.12:0/4113760687 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d84008de0 con 0x5623cc7d2410 2023-10-05T19:40:31.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.286+0000 7f6d96433e40 1 -- 172.21.15.12:0/4113760687 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623cc7d2410 msgr2=0x5623cc7f3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:31.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.286+0000 7f6d96433e40 1 --2- 172.21.15.12:0/4113760687 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623cc7d2410 0x5623cc7f3150 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6d84000c00 tx=0x7f6d84015000 comp rx=0 tx=0).stop 2023-10-05T19:40:31.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.286+0000 7f6d96433e40 1 -- 172.21.15.12:0/4113760687 shutdown_connections 2023-10-05T19:40:31.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.286+0000 7f6d96433e40 1 --2- 172.21.15.12:0/4113760687 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623cc7d2410 0x5623cc7f3150 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:31.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.286+0000 7f6d96433e40 1 -- 172.21.15.12:0/4113760687 >> 172.21.15.12:0/4113760687 conn(0x5623cc7cc720 msgr2=0x5623cc7eed10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:31.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.286+0000 7f6d96433e40 1 -- 172.21.15.12:0/4113760687 shutdown_connections 2023-10-05T19:40:31.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.286+0000 7f6d96433e40 1 -- 172.21.15.12:0/4113760687 wait complete. 2023-10-05T19:40:31.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.286+0000 7f6d96433e40 1 Processor -- start 2023-10-05T19:40:31.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.286+0000 7f6d96433e40 1 -- start start 2023-10-05T19:40:31.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.286+0000 7f6d96433e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623cc7d2410 0x5623cc7f9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:31.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.286+0000 7f6d96433e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623cc672ed0 con 0x5623cc7d2410 2023-10-05T19:40:31.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.290+0000 7f6d93f68700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623cc7d2410 0x5623cc7f9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:31.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.290+0000 7f6d93f68700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623cc7d2410 0x5623cc7f9d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58984/0 (socket says 172.21.15.12:58984) 2023-10-05T19:40:31.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.290+0000 7f6d93f68700 1 -- 172.21.15.12:0/668887359 learned_addr learned my addr 172.21.15.12:0/668887359 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:31.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.290+0000 7f6d93f68700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623cc7d1cd0 con 0x5623cc7d2410 2023-10-05T19:40:31.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.290+0000 7f6d93f68700 1 --2- 172.21.15.12:0/668887359 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623cc7d2410 0x5623cc7f9d60 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6d84000c00 tx=0x7f6d84003050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:31.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.290+0000 7f6d8bfff700 1 -- 172.21.15.12:0/668887359 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d84012b70 con 0x5623cc7d2410 2023-10-05T19:40:31.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.290+0000 7f6d8bfff700 1 -- 172.21.15.12:0/668887359 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8402a070 con 0x5623cc7d2410 2023-10-05T19:40:31.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.290+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5623cc7fa2a0 con 0x5623cc7d2410 2023-10-05T19:40:31.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.290+0000 7f6d8bfff700 1 -- 172.21.15.12:0/668887359 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d84012e50 con 0x5623cc7d2410 2023-10-05T19:40:31.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.290+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5623cc7fa740 con 0x5623cc7d2410 2023-10-05T19:40:31.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.290+0000 7f6d8bfff700 1 -- 172.21.15.12:0/668887359 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f6d84019070 con 0x5623cc7d2410 2023-10-05T19:40:31.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.294+0000 7f6d8bfff700 1 --2- 172.21.15.12:0/668887359 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f6d74030bf0 0x7f6d740330a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:31.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.294+0000 7f6d8bfff700 1 -- 172.21.15.12:0/668887359 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f6d8402c070 con 0x5623cc7d2410 2023-10-05T19:40:31.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.294+0000 7f6d93767700 1 --2- 172.21.15.12:0/668887359 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f6d74030bf0 0x7f6d740330a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:31.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.294+0000 7f6d96433e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:31.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.294+0000 7f6d96433e40 1 --2- 172.21.15.12:0/668887359 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5623cc7d27e0 0x5623cc8057c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:31.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.294+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc805ef0 con 0x5623cc7d27e0 2023-10-05T19:40:31.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.294+0000 7f6d94769700 1 --2- 172.21.15.12:0/668887359 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5623cc7d27e0 0x5623cc8057c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:31.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.294+0000 7f6d93767700 1 --2- 172.21.15.12:0/668887359 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f6d74030bf0 0x7f6d740330a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c001e90 tx=0x7f6d7c0013b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:31.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.294+0000 7f6d94769700 1 --2- 172.21.15.12:0/668887359 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5623cc7d27e0 0x5623cc8057c0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:31.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.294+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x5623cc7d27e0 2023-10-05T19:40:31.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.294+0000 7f6d96433e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:31.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.294+0000 7f6d96433e40 20 realm 2023-10-05T19:40:31.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.294+0000 7f6d96433e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:31.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.294+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc802df0 con 0x5623cc7d27e0 2023-10-05T19:40:31.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.294+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x5623cc7d27e0 2023-10-05T19:40:31.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.294+0000 7f6d96433e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:31.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.294+0000 7f6d96433e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:31.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.294+0000 7f6d96433e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:31.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.294+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc806a20 con 0x5623cc7d27e0 2023-10-05T19:40:31.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.294+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x5623cc7d27e0 2023-10-05T19:40:31.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.294+0000 7f6d96433e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:31.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.294+0000 7f6d96433e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:31.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.298+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc806630 con 0x5623cc7d27e0 2023-10-05T19:40:31.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.298+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x5623cc7d27e0 2023-10-05T19:40:31.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.298+0000 7f6d96433e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:31.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.298+0000 7f6d96433e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:31.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.298+0000 7f6d96433e40 1 --2- 172.21.15.12:0/668887359 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5623cc807dc0 0x5623cc80a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:31.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.298+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80a6e0 con 0x5623cc807dc0 2023-10-05T19:40:31.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.298+0000 7f6d93f68700 1 --2- 172.21.15.12:0/668887359 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5623cc807dc0 0x5623cc80a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:31.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.298+0000 7f6d93f68700 1 --2- 172.21.15.12:0/668887359 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5623cc807dc0 0x5623cc80a1a0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:31.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.298+0000 7f6d93f68700 1 -- 172.21.15.12:0/668887359 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f6d84022020 con 0x5623cc807dc0 2023-10-05T19:40:31.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.298+0000 7f6d96433e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:31.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.298+0000 7f6d96433e40 20 searching for the correct realm 2023-10-05T19:40:31.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.298+0000 7f6d96433e40 1 --2- 172.21.15.12:0/668887359 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5623cc80c560 0x5623cc80e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:31.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.298+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80f020 con 0x5623cc80c560 2023-10-05T19:40:31.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.298+0000 7f6d93767700 1 --2- 172.21.15.12:0/668887359 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5623cc80c560 0x5623cc80e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:31.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.298+0000 7f6d93767700 1 --2- 172.21.15.12:0/668887359 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5623cc80c560 0x5623cc80e970 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:31.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.302+0000 7f6d93767700 1 -- 172.21.15.12:0/668887359 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f6d7c006070 con 0x5623cc80c560 2023-10-05T19:40:31.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.302+0000 7f6d93767700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80f020 con 0x5623cc807dc0 2023-10-05T19:40:31.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.302+0000 7f6d93f68700 1 -- 172.21.15.12:0/668887359 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d84022020 con 0x5623cc807dc0 2023-10-05T19:40:31.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.302+0000 7f6d93f68700 1 --2- 172.21.15.12:0/668887359 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f6d84007030 0x7f6d84026880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:31.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.302+0000 7f6d93f68700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80f020 con 0x7f6d84007030 2023-10-05T19:40:31.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.302+0000 7f6d94769700 1 --2- 172.21.15.12:0/668887359 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f6d84007030 0x7f6d84026880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:31.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.306+0000 7f6d94769700 1 --2- 172.21.15.12:0/668887359 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f6d84007030 0x7f6d84026880 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:31.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.306+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x7f6d84007030 2023-10-05T19:40:31.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.306+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80f020 con 0x5623cc7d27e0 2023-10-05T19:40:31.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.306+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x5623cc7d27e0 2023-10-05T19:40:31.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.306+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80f020 con 0x5623cc7d27e0 2023-10-05T19:40:31.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.306+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x5623cc7d27e0 2023-10-05T19:40:31.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.306+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80f020 con 0x7f6d84007030 2023-10-05T19:40:31.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.306+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x7f6d84007030 2023-10-05T19:40:31.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.306+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80f020 con 0x7f6d84007030 2023-10-05T19:40:31.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.310+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x7f6d84007030 2023-10-05T19:40:31.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.310+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80f020 con 0x5623cc807dc0 2023-10-05T19:40:31.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.310+0000 7f6d93f68700 1 -- 172.21.15.12:0/668887359 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d84022020 con 0x5623cc807dc0 2023-10-05T19:40:31.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.310+0000 7f6d93f68700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80f020 con 0x5623cc80c560 2023-10-05T19:40:31.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.310+0000 7f6d93767700 1 -- 172.21.15.12:0/668887359 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d7c006070 con 0x5623cc80c560 2023-10-05T19:40:31.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.310+0000 7f6d93767700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80f020 con 0x5623cc7d27e0 2023-10-05T19:40:31.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.310+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x5623cc7d27e0 2023-10-05T19:40:31.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.310+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80f020 con 0x7f6d84007030 2023-10-05T19:40:31.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.314+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x7f6d84007030 2023-10-05T19:40:31.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.314+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80f020 con 0x5623cc7d27e0 2023-10-05T19:40:31.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.314+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x5623cc7d27e0 2023-10-05T19:40:31.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.314+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80f020 con 0x5623cc807dc0 2023-10-05T19:40:31.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.314+0000 7f6d93f68700 1 -- 172.21.15.12:0/668887359 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d84022020 con 0x5623cc807dc0 2023-10-05T19:40:31.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.314+0000 7f6d93f68700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80f020 con 0x5623cc80c560 2023-10-05T19:40:31.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.314+0000 7f6d93767700 1 -- 172.21.15.12:0/668887359 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d7c006070 con 0x5623cc80c560 2023-10-05T19:40:31.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.314+0000 7f6d93767700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80f020 con 0x5623cc80c560 2023-10-05T19:40:31.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.314+0000 7f6d93767700 1 -- 172.21.15.12:0/668887359 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d7c006070 con 0x5623cc80c560 2023-10-05T19:40:31.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.314+0000 7f6d93767700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80f020 con 0x7f6d84007030 2023-10-05T19:40:31.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.318+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x7f6d84007030 2023-10-05T19:40:31.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.318+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80f020 con 0x5623cc80c560 2023-10-05T19:40:31.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.318+0000 7f6d93767700 1 -- 172.21.15.12:0/668887359 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d7c006070 con 0x5623cc80c560 2023-10-05T19:40:31.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.318+0000 7f6d93767700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80f020 con 0x5623cc80c560 2023-10-05T19:40:31.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.318+0000 7f6d93767700 1 -- 172.21.15.12:0/668887359 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d7c006070 con 0x5623cc80c560 2023-10-05T19:40:31.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.318+0000 7f6d93767700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80f020 con 0x5623cc7d27e0 2023-10-05T19:40:31.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.318+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x5623cc7d27e0 2023-10-05T19:40:31.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.318+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80f020 con 0x5623cc7d27e0 2023-10-05T19:40:31.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.318+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x5623cc7d27e0 2023-10-05T19:40:31.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.318+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80f020 con 0x7f6d84007030 2023-10-05T19:40:31.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.322+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x7f6d84007030 2023-10-05T19:40:31.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.322+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80f020 con 0x5623cc80c560 2023-10-05T19:40:31.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.322+0000 7f6d93767700 1 -- 172.21.15.12:0/668887359 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d7c006070 con 0x5623cc80c560 2023-10-05T19:40:31.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.322+0000 7f6d93767700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80f020 con 0x5623cc807dc0 2023-10-05T19:40:31.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.322+0000 7f6d93f68700 1 -- 172.21.15.12:0/668887359 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d84022020 con 0x5623cc807dc0 2023-10-05T19:40:31.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.322+0000 7f6d93f68700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80f020 con 0x5623cc7d27e0 2023-10-05T19:40:31.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.322+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x5623cc7d27e0 2023-10-05T19:40:31.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.322+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80f020 con 0x7f6d84007030 2023-10-05T19:40:31.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.326+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x7f6d84007030 2023-10-05T19:40:31.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.326+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80f020 con 0x5623cc80c560 2023-10-05T19:40:31.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.326+0000 7f6d93767700 1 -- 172.21.15.12:0/668887359 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d7c006070 con 0x5623cc80c560 2023-10-05T19:40:31.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.326+0000 7f6d93767700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80f020 con 0x7f6d84007030 2023-10-05T19:40:31.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.326+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x7f6d84007030 2023-10-05T19:40:31.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.326+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80f020 con 0x5623cc807dc0 2023-10-05T19:40:31.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.326+0000 7f6d93f68700 1 -- 172.21.15.12:0/668887359 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d84022020 con 0x5623cc807dc0 2023-10-05T19:40:31.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.326+0000 7f6d93f68700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80f020 con 0x5623cc80c560 2023-10-05T19:40:31.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.326+0000 7f6d93767700 1 -- 172.21.15.12:0/668887359 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d7c006070 con 0x5623cc80c560 2023-10-05T19:40:31.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.330+0000 7f6d93767700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80f020 con 0x7f6d84007030 2023-10-05T19:40:31.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.330+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x7f6d84007030 2023-10-05T19:40:31.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.330+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80f020 con 0x5623cc7d27e0 2023-10-05T19:40:31.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.330+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x5623cc7d27e0 2023-10-05T19:40:31.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.330+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80f020 con 0x5623cc807dc0 2023-10-05T19:40:31.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.330+0000 7f6d93f68700 1 -- 172.21.15.12:0/668887359 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f6d84022020 con 0x5623cc807dc0 2023-10-05T19:40:31.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.330+0000 7f6d96433e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:31.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.330+0000 7f6d96433e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:31.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.330+0000 7f6d96433e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:31.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.330+0000 7f6d96433e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:31.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.330+0000 7f6d96433e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:31.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.330+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80b2e0 con 0x5623cc7d27e0 2023-10-05T19:40:31.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.330+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x5623cc7d27e0 2023-10-05T19:40:31.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.330+0000 7f6d96433e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:31.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.330+0000 7f6d96433e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:31.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.330+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80af10 con 0x5623cc7d27e0 2023-10-05T19:40:31.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.334+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x5623cc7d27e0 2023-10-05T19:40:31.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.334+0000 7f6d96433e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:31.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.334+0000 7f6d96433e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:31.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.334+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80f490 con 0x5623cc80c560 2023-10-05T19:40:31.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.334+0000 7f6d93767700 1 -- 172.21.15.12:0/668887359 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f6d7c006070 con 0x5623cc80c560 2023-10-05T19:40:31.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.334+0000 7f6d96433e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:31.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.334+0000 7f6d96433e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:31.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.334+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80f840 con 0x5623cc80c560 2023-10-05T19:40:31.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.334+0000 7f6d93767700 1 -- 172.21.15.12:0/668887359 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f6d7c006070 con 0x5623cc80c560 2023-10-05T19:40:31.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.334+0000 7f6d96433e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:31.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.334+0000 7f6d96433e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:31.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.334+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80fbf0 con 0x5623cc7d27e0 2023-10-05T19:40:31.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.334+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x5623cc7d27e0 2023-10-05T19:40:31.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.334+0000 7f6d96433e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:31.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.334+0000 7f6d96433e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:31.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.334+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80b800 con 0x5623cc7d27e0 2023-10-05T19:40:31.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.334+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x5623cc7d27e0 2023-10-05T19:40:31.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.334+0000 7f6d96433e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:31.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.334+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80b800 con 0x5623cc80c560 2023-10-05T19:40:31.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.338+0000 7f6d93767700 1 -- 172.21.15.12:0/668887359 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f6d7c006070 con 0x5623cc80c560 2023-10-05T19:40:31.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.338+0000 7f6d93767700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80b800 con 0x5623cc807dc0 2023-10-05T19:40:31.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.338+0000 7f6d93f68700 1 -- 172.21.15.12:0/668887359 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d84022020 con 0x5623cc807dc0 2023-10-05T19:40:31.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.338+0000 7f6d93f68700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80b800 con 0x7f6d84007030 2023-10-05T19:40:31.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.342+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x7f6d84007030 2023-10-05T19:40:31.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.342+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80fbf0 con 0x5623cc7d27e0 2023-10-05T19:40:31.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.342+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x5623cc7d27e0 2023-10-05T19:40:31.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.342+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80fbf0 con 0x5623cc7d27e0 2023-10-05T19:40:31.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.342+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x5623cc7d27e0 2023-10-05T19:40:31.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.342+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80fbf0 con 0x7f6d84007030 2023-10-05T19:40:31.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.342+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x7f6d84007030 2023-10-05T19:40:31.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.342+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80fbf0 con 0x7f6d84007030 2023-10-05T19:40:31.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.342+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x7f6d84007030 2023-10-05T19:40:31.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.342+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80fbf0 con 0x5623cc807dc0 2023-10-05T19:40:31.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.346+0000 7f6d93f68700 1 -- 172.21.15.12:0/668887359 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d84022020 con 0x5623cc807dc0 2023-10-05T19:40:31.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.346+0000 7f6d93f68700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80fbf0 con 0x5623cc80c560 2023-10-05T19:40:31.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.346+0000 7f6d93767700 1 -- 172.21.15.12:0/668887359 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d7c006070 con 0x5623cc80c560 2023-10-05T19:40:31.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.346+0000 7f6d93767700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80fbf0 con 0x5623cc7d27e0 2023-10-05T19:40:31.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.346+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x5623cc7d27e0 2023-10-05T19:40:31.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.346+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80fbf0 con 0x7f6d84007030 2023-10-05T19:40:31.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.346+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x7f6d84007030 2023-10-05T19:40:31.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.346+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80fbf0 con 0x5623cc7d27e0 2023-10-05T19:40:31.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.346+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x5623cc7d27e0 2023-10-05T19:40:31.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.346+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80fbf0 con 0x5623cc807dc0 2023-10-05T19:40:31.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.350+0000 7f6d93f68700 1 -- 172.21.15.12:0/668887359 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d84022020 con 0x5623cc807dc0 2023-10-05T19:40:31.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.350+0000 7f6d93f68700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80fbf0 con 0x5623cc80c560 2023-10-05T19:40:31.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.350+0000 7f6d93767700 1 -- 172.21.15.12:0/668887359 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d7c006070 con 0x5623cc80c560 2023-10-05T19:40:31.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.350+0000 7f6d93767700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80fbf0 con 0x5623cc80c560 2023-10-05T19:40:31.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.350+0000 7f6d93767700 1 -- 172.21.15.12:0/668887359 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d7c006070 con 0x5623cc80c560 2023-10-05T19:40:31.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.350+0000 7f6d93767700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80fbf0 con 0x7f6d84007030 2023-10-05T19:40:31.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.350+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x7f6d84007030 2023-10-05T19:40:31.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.350+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80fbf0 con 0x5623cc80c560 2023-10-05T19:40:31.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.354+0000 7f6d93767700 1 -- 172.21.15.12:0/668887359 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d7c006070 con 0x5623cc80c560 2023-10-05T19:40:31.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.354+0000 7f6d93767700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80fbf0 con 0x5623cc80c560 2023-10-05T19:40:31.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.354+0000 7f6d93767700 1 -- 172.21.15.12:0/668887359 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d7c006070 con 0x5623cc80c560 2023-10-05T19:40:31.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.354+0000 7f6d93767700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80fbf0 con 0x5623cc7d27e0 2023-10-05T19:40:31.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.354+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x5623cc7d27e0 2023-10-05T19:40:31.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.354+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80fbf0 con 0x5623cc7d27e0 2023-10-05T19:40:31.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.354+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x5623cc7d27e0 2023-10-05T19:40:31.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.354+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80fbf0 con 0x7f6d84007030 2023-10-05T19:40:31.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.354+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x7f6d84007030 2023-10-05T19:40:31.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.354+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80fbf0 con 0x5623cc80c560 2023-10-05T19:40:31.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.358+0000 7f6d93767700 1 -- 172.21.15.12:0/668887359 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d7c006070 con 0x5623cc80c560 2023-10-05T19:40:31.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.358+0000 7f6d93767700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80fbf0 con 0x5623cc807dc0 2023-10-05T19:40:31.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.358+0000 7f6d93f68700 1 -- 172.21.15.12:0/668887359 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d84022020 con 0x5623cc807dc0 2023-10-05T19:40:31.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.358+0000 7f6d93f68700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80fbf0 con 0x5623cc7d27e0 2023-10-05T19:40:31.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.358+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x5623cc7d27e0 2023-10-05T19:40:31.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.358+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80fbf0 con 0x7f6d84007030 2023-10-05T19:40:31.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.358+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x7f6d84007030 2023-10-05T19:40:31.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.358+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80fbf0 con 0x5623cc80c560 2023-10-05T19:40:31.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.358+0000 7f6d93767700 1 -- 172.21.15.12:0/668887359 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d7c006070 con 0x5623cc80c560 2023-10-05T19:40:31.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.358+0000 7f6d93767700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80fbf0 con 0x7f6d84007030 2023-10-05T19:40:31.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.362+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x7f6d84007030 2023-10-05T19:40:31.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.362+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80fbf0 con 0x5623cc807dc0 2023-10-05T19:40:31.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.362+0000 7f6d93f68700 1 -- 172.21.15.12:0/668887359 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d84022020 con 0x5623cc807dc0 2023-10-05T19:40:31.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.362+0000 7f6d93f68700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80fbf0 con 0x5623cc80c560 2023-10-05T19:40:31.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.362+0000 7f6d93767700 1 -- 172.21.15.12:0/668887359 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d7c006070 con 0x5623cc80c560 2023-10-05T19:40:31.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.362+0000 7f6d93767700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80fbf0 con 0x7f6d84007030 2023-10-05T19:40:31.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.362+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x7f6d84007030 2023-10-05T19:40:31.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.362+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80fbf0 con 0x5623cc7d27e0 2023-10-05T19:40:31.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.366+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x5623cc7d27e0 2023-10-05T19:40:31.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.366+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80fbf0 con 0x5623cc807dc0 2023-10-05T19:40:31.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.366+0000 7f6d93f68700 1 -- 172.21.15.12:0/668887359 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f6d84022020 con 0x5623cc807dc0 2023-10-05T19:40:31.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.366+0000 7f6d96433e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:31.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.366+0000 7f6d96433e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:31.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.366+0000 7f6d96433e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:31.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.366+0000 7f6d96433e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:31.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.366+0000 7f6d96433e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:31.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.366+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc80b800 con 0x5623cc7d27e0 2023-10-05T19:40:31.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.366+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x5623cc7d27e0 2023-10-05T19:40:31.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.366+0000 7f6d96433e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:31.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.366+0000 7f6d96433e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:31.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.366+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc810540 con 0x5623cc807dc0 2023-10-05T19:40:31.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.366+0000 7f6d93f68700 1 -- 172.21.15.12:0/668887359 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f6d84022020 con 0x5623cc807dc0 2023-10-05T19:40:31.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.366+0000 7f6d96433e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:31.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.366+0000 7f6d96433e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:31.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.366+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc810e30 con 0x5623cc7d27e0 2023-10-05T19:40:31.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.370+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x5623cc7d27e0 2023-10-05T19:40:31.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.370+0000 7f6d96433e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:31.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.370+0000 7f6d96433e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:31.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.370+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc810e30 con 0x5623cc80c560 2023-10-05T19:40:31.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.370+0000 7f6d93767700 1 -- 172.21.15.12:0/668887359 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f6d7c006070 con 0x5623cc80c560 2023-10-05T19:40:31.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.370+0000 7f6d96433e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:31.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.370+0000 7f6d96433e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:31.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.370+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc8115a0 con 0x7f6d84007030 2023-10-05T19:40:31.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.370+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x7f6d84007030 2023-10-05T19:40:31.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.370+0000 7f6d96433e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:31.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.370+0000 7f6d96433e40 20 zone default found 2023-10-05T19:40:31.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.370+0000 7f6d96433e40 4 Realm: () 2023-10-05T19:40:31.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.370+0000 7f6d96433e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:31.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.370+0000 7f6d96433e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:31.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.370+0000 7f6d96433e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:31.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.370+0000 7f6d96433e40 20 zonegroup default 2023-10-05T19:40:31.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.370+0000 7f6d96433e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:31.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.370+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc8111e0 con 0x7f6d84007030 2023-10-05T19:40:31.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.370+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x7f6d84007030 2023-10-05T19:40:31.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.370+0000 7f6d96433e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:31.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.370+0000 7f6d96433e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:31.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.370+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc812b60 con 0x5623cc80c560 2023-10-05T19:40:31.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.374+0000 7f6d93767700 1 -- 172.21.15.12:0/668887359 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f6d7c006070 con 0x5623cc80c560 2023-10-05T19:40:31.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.374+0000 7f6d96433e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:31.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.374+0000 7f6d96433e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:31.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.374+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc812f10 con 0x5623cc80c560 2023-10-05T19:40:31.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.374+0000 7f6d93767700 1 -- 172.21.15.12:0/668887359 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f6d7c006070 con 0x5623cc80c560 2023-10-05T19:40:31.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.374+0000 7f6d96433e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:31.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.374+0000 7f6d96433e40 20 started sync module instance, tier type = 2023-10-05T19:40:31.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.374+0000 7f6d96433e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:31.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.374+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc812220 con 0x5623cc807dc0 2023-10-05T19:40:31.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.378+0000 7f6d93f68700 1 -- 172.21.15.12:0/668887359 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'183 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6d84022020 con 0x5623cc807dc0 2023-10-05T19:40:31.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.378+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94711754852672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc8132c0 con 0x5623cc807dc0 2023-10-05T19:40:31.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.378+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc813b60 con 0x5623cc7d27e0 2023-10-05T19:40:31.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.378+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'183 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x5623cc7d27e0 2023-10-05T19:40:31.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.378+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94711754863296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc813c40 con 0x5623cc7d27e0 2023-10-05T19:40:31.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.378+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc8147d0 con 0x5623cc80c560 2023-10-05T19:40:31.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.382+0000 7f6d93f68700 1 -- 172.21.15.12:0/668887359 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94711754852672] v28'184 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6d84022020 con 0x5623cc807dc0 2023-10-05T19:40:31.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.382+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94711754863296] v28'184 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x5623cc7d27e0 2023-10-05T19:40:31.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.382+0000 7f6d93767700 1 -- 172.21.15.12:0/668887359 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'183 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6d7c006070 con 0x5623cc80c560 2023-10-05T19:40:31.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.382+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94711754864400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc812620 con 0x5623cc80c560 2023-10-05T19:40:31.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.382+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc8150c0 con 0x5623cc807dc0 2023-10-05T19:40:31.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.386+0000 7f6d93767700 1 -- 172.21.15.12:0/668887359 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94711754864400] v28'184 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6d7c006070 con 0x5623cc80c560 2023-10-05T19:40:31.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.386+0000 7f6d93f68700 1 -- 172.21.15.12:0/668887359 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'183 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6d84022020 con 0x5623cc807dc0 2023-10-05T19:40:31.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.386+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94711754867360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc8150c0 con 0x5623cc807dc0 2023-10-05T19:40:31.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.386+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc815c00 con 0x5623cc7d27e0 2023-10-05T19:40:31.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.390+0000 7f6d93f68700 1 -- 172.21.15.12:0/668887359 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94711754867360] v28'184 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6d84022020 con 0x5623cc807dc0 2023-10-05T19:40:31.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.390+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'183 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x5623cc7d27e0 2023-10-05T19:40:31.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.390+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94711754872528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc8150c0 con 0x5623cc7d27e0 2023-10-05T19:40:31.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.390+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc816660 con 0x7f6d84007030 2023-10-05T19:40:31.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.390+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94711754872528] v28'184 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x5623cc7d27e0 2023-10-05T19:40:31.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.394+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'183 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x7f6d84007030 2023-10-05T19:40:31.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.394+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94711754870272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc816660 con 0x7f6d84007030 2023-10-05T19:40:31.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.394+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc816660 con 0x7f6d84007030 2023-10-05T19:40:31.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.394+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(95 notify.5 [watch watch cookie 94711754870272] v28'184 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x7f6d84007030 2023-10-05T19:40:31.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.394+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(96 notify.6 [create] v28'183 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x7f6d84007030 2023-10-05T19:40:31.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.394+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94711754875808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc817070 con 0x7f6d84007030 2023-10-05T19:40:31.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.394+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc817a80 con 0x7f6d84007030 2023-10-05T19:40:31.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.398+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(97 notify.6 [watch watch cookie 94711754875808] v28'184 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x7f6d84007030 2023-10-05T19:40:31.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.398+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(98 notify.7 [create] v28'183 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x7f6d84007030 2023-10-05T19:40:31.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.398+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94711754878112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc817a80 con 0x7f6d84007030 2023-10-05T19:40:31.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.398+0000 7f6d96433e40 20 add_watcher() i=0 2023-10-05T19:40:31.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.398+0000 7f6d96433e40 20 add_watcher() i=1 2023-10-05T19:40:31.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.398+0000 7f6d96433e40 20 add_watcher() i=2 2023-10-05T19:40:31.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.398+0000 7f6d96433e40 20 add_watcher() i=3 2023-10-05T19:40:31.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.398+0000 7f6d96433e40 20 add_watcher() i=4 2023-10-05T19:40:31.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.398+0000 7f6d96433e40 20 add_watcher() i=5 2023-10-05T19:40:31.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.398+0000 7f6d96433e40 20 add_watcher() i=6 2023-10-05T19:40:31.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.402+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94711754878112] v28'184 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x7f6d84007030 2023-10-05T19:40:31.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.402+0000 7f6d96433e40 20 add_watcher() i=7 2023-10-05T19:40:31.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.402+0000 7f6d96433e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:31.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.402+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc81ac20 con 0x5623cc7d27e0 2023-10-05T19:40:31.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.402+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x5623cc7d27e0 2023-10-05T19:40:31.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.402+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94711754880768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc818150 con 0x5623cc7d27e0 2023-10-05T19:40:31.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.402+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94711754880768] v28'169 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x5623cc7d27e0 2023-10-05T19:40:31.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.402+0000 7f6d657e2700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:31.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.406+0000 7f6d96433e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:31.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.406+0000 7f6d96433e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:31.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.406+0000 7f6d96433e40 1 Processor -- start 2023-10-05T19:40:31.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.406+0000 7f6d96433e40 1 -- start start 2023-10-05T19:40:31.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.406+0000 7f6d96433e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623cc8253b0 0x5623cc825780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:31.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.406+0000 7f6d96433e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623cc825cc0 con 0x5623cc8253b0 2023-10-05T19:40:31.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.406+0000 7f6d93767700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623cc8253b0 0x5623cc825780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:31.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.406+0000 7f6d93767700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623cc8253b0 0x5623cc825780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58986/0 (socket says 172.21.15.12:58986) 2023-10-05T19:40:31.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.406+0000 7f6d93767700 1 -- 172.21.15.12:0/1928803714 learned_addr learned my addr 172.21.15.12:0/1928803714 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:31.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.406+0000 7f6d93767700 1 -- 172.21.15.12:0/1928803714 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623cc825e00 con 0x5623cc8253b0 2023-10-05T19:40:31.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.406+0000 7f6d93767700 1 --2- 172.21.15.12:0/1928803714 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623cc8253b0 0x5623cc825780 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f6d84005500 tx=0x7f6d7c01e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b9493716090928 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:31.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.406+0000 7f6d64fe1700 1 -- 172.21.15.12:0/1928803714 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d7c013040 con 0x5623cc8253b0 2023-10-05T19:40:31.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.406+0000 7f6d64fe1700 1 -- 172.21.15.12:0/1928803714 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d7c022040 con 0x5623cc8253b0 2023-10-05T19:40:31.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.406+0000 7f6d64fe1700 1 -- 172.21.15.12:0/1928803714 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d7c00ea90 con 0x5623cc8253b0 2023-10-05T19:40:31.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.410+0000 7f6d96433e40 1 -- 172.21.15.12:0/1928803714 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623cc8253b0 msgr2=0x5623cc825780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:31.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.410+0000 7f6d96433e40 1 --2- 172.21.15.12:0/1928803714 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623cc8253b0 0x5623cc825780 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f6d84005500 tx=0x7f6d7c01e040 comp rx=0 tx=0).stop 2023-10-05T19:40:31.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.410+0000 7f6d96433e40 1 -- 172.21.15.12:0/1928803714 shutdown_connections 2023-10-05T19:40:31.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.410+0000 7f6d96433e40 1 --2- 172.21.15.12:0/1928803714 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623cc8253b0 0x5623cc825780 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:31.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.410+0000 7f6d96433e40 1 -- 172.21.15.12:0/1928803714 >> 172.21.15.12:0/1928803714 conn(0x5623cc820770 msgr2=0x5623cc81e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:31.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.410+0000 7f6d96433e40 1 -- 172.21.15.12:0/1928803714 shutdown_connections 2023-10-05T19:40:31.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.410+0000 7f6d96433e40 1 -- 172.21.15.12:0/1928803714 wait complete. 2023-10-05T19:40:31.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.410+0000 7f6d96433e40 1 Processor -- start 2023-10-05T19:40:31.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.410+0000 7f6d96433e40 1 -- start start 2023-10-05T19:40:31.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.410+0000 7f6d96433e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623cc8253b0 0x5623cc82af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:31.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.410+0000 7f6d96433e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623cc82bf40 con 0x5623cc8253b0 2023-10-05T19:40:31.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.410+0000 7f6d93767700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623cc8253b0 0x5623cc82af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:31.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.410+0000 7f6d93767700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623cc8253b0 0x5623cc82af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58994/0 (socket says 172.21.15.12:58994) 2023-10-05T19:40:31.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.410+0000 7f6d93767700 1 -- 172.21.15.12:0/3806492544 learned_addr learned my addr 172.21.15.12:0/3806492544 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:31.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.410+0000 7f6d93767700 1 -- 172.21.15.12:0/3806492544 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d7c034040 con 0x5623cc8253b0 2023-10-05T19:40:31.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.410+0000 7f6d93767700 1 --2- 172.21.15.12:0/3806492544 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623cc8253b0 0x5623cc82af40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c049040 tx=0x7f6d7c03d030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:31.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.410+0000 7f6cdeffd700 1 -- 172.21.15.12:0/3806492544 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d7c02c640 con 0x5623cc8253b0 2023-10-05T19:40:31.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.410+0000 7f6cdeffd700 1 -- 172.21.15.12:0/3806492544 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d7c02c7a0 con 0x5623cc8253b0 2023-10-05T19:40:31.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.410+0000 7f6d96433e40 1 -- 172.21.15.12:0/3806492544 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5623cc82b540 con 0x5623cc8253b0 2023-10-05T19:40:31.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.410+0000 7f6cdeffd700 1 -- 172.21.15.12:0/3806492544 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d7c02c910 con 0x5623cc8253b0 2023-10-05T19:40:31.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.410+0000 7f6d96433e40 1 -- 172.21.15.12:0/3806492544 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5623cc82b980 con 0x5623cc8253b0 2023-10-05T19:40:31.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.414+0000 7f6cdeffd700 1 -- 172.21.15.12:0/3806492544 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f6d7c02ca70 con 0x5623cc8253b0 2023-10-05T19:40:31.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.414+0000 7f6cdeffd700 1 --2- 172.21.15.12:0/3806492544 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f6cd4030c50 0x7f6cd4033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:31.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.414+0000 7f6cdeffd700 1 -- 172.21.15.12:0/3806492544 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7f6d7c031140 con 0x5623cc8253b0 2023-10-05T19:40:31.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.414+0000 7f6d94769700 1 --2- 172.21.15.12:0/3806492544 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f6cd4030c50 0x7f6cd4033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:31.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.414+0000 7f6d96433e40 5 note: GC not initialized 2023-10-05T19:40:31.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.414+0000 7f6d94769700 1 --2- 172.21.15.12:0/3806492544 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f6cd4030c50 0x7f6cd4033100 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x5623cc8180b0 tx=0x7f6d8c00aea0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:31.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.414+0000 7f6cd3fff700 20 reqs_thread_entry: start 2023-10-05T19:40:31.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.418+0000 7f6cd1ffb700 20 reqs_thread_entry: start 2023-10-05T19:40:31.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.418+0000 7f6d96433e40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:31.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.418+0000 7f6d96433e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:31.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.418+0000 7f6cbffff700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:41:01 2023 2023-10-05T19:40:31.968 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:31.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.418+0000 7f6cbf7fe700 20 reqs_thread_entry: start 2023-10-05T19:40:31.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.418+0000 7f6d96433e40 20 get_system_obj_state: rctx=0x7fff8d856e48 obj=default.rgw.meta:users.uid:foo state=0x5623cc6ab0b0 s->prefetch_data=0 2023-10-05T19:40:31.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.418+0000 7f6d96433e40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2023-10-05T19:40:31.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.418+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc893710 con 0x5623cc7d27e0 2023-10-05T19:40:31.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.418+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(102 foo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x5623cc7d27e0 2023-10-05T19:40:31.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.418+0000 7f6d96433e40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x16 2023-10-05T19:40:31.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.418+0000 7f6d96433e40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:31.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.418+0000 7f6d96433e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:31.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.418+0000 7f6d96433e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:31.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.418+0000 7f6d96433e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:31.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.418+0000 7f6d96433e40 10 cache get: name=default.rgw.meta+users.uid+foo : type miss (requested=0x13, cached=0x16) 2023-10-05T19:40:31.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.418+0000 7f6d96433e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:31.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.418+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc8961a0 con 0x5623cc7d27e0 2023-10-05T19:40:31.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.418+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(103 foo [call,call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv11 ondisk = 0) v8 ==== 273+0+480 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x5623cc7d27e0 2023-10-05T19:40:31.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.418+0000 7f6d96433e40 20 rados_obj.operate() r=0 bl.length=357 2023-10-05T19:40:31.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.418+0000 7f6d96433e40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2023-10-05T19:40:31.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.418+0000 7f6d96433e40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:31.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.418+0000 7f6d96433e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:31.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.418+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:104 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc8961a0 con 0x5623cc807dc0 2023-10-05T19:40:31.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.418+0000 7f6d93f68700 1 -- 172.21.15.12:0/668887359 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(104 foo.buckets [call out=183b] v0'0 uv26 ondisk = 0) v8 ==== 155+0+183 (crc 0 0 0) 0x7f6d84022020 con 0x5623cc807dc0 2023-10-05T19:40:31.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.418+0000 7f6d96433e40 20 disabling bucket name=myfoo 2023-10-05T19:40:31.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.418+0000 7f6d96433e40 20 get_system_obj_state: rctx=0x7fff8d856620 obj=default.rgw.meta:root:myfoo state=0x5623cc6ab0b0 s->prefetch_data=0 2023-10-05T19:40:31.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.418+0000 7f6d96433e40 10 cache get: name=default.rgw.meta+root+myfoo : miss 2023-10-05T19:40:31.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.418+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:105 8.9 8:91b71841:root::myfoo:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc8949a0 con 0x5623cc807dc0 2023-10-05T19:40:31.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.422+0000 7f6d93f68700 1 -- 172.21.15.12:0/668887359 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(105 myfoo [getxattrs out=75b,stat out=16b] v0'0 uv27 ondisk = 0) v8 ==== 191+0+91 (crc 0 0 0) 0x7f6d84022020 con 0x5623cc807dc0 2023-10-05T19:40:31.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.422+0000 7f6d96433e40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x6 2023-10-05T19:40:31.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.422+0000 7f6d96433e40 10 adding default.rgw.meta+root+myfoo to cache LRU end 2023-10-05T19:40:31.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.422+0000 7f6d96433e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:31.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.422+0000 7f6d96433e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:31.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.422+0000 7f6d96433e40 10 cache get: name=default.rgw.meta+root+myfoo : type miss (requested=0x1, cached=0x6) 2023-10-05T19:40:31.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.422+0000 7f6d96433e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:31.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.422+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:106 8.9 8:91b71841:root::myfoo:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc8949a0 con 0x5623cc807dc0 2023-10-05T19:40:31.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.422+0000 7f6d93f68700 1 -- 172.21.15.12:0/668887359 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(106 myfoo [read 0~165 out=165b] v0'0 uv27 ondisk = 0) v8 ==== 149+0+165 (crc 0 0 0) 0x7f6d84022020 con 0x5623cc807dc0 2023-10-05T19:40:31.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.422+0000 7f6d96433e40 20 rados_obj.operate() r=0 bl.length=165 2023-10-05T19:40:31.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.422+0000 7f6d96433e40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x1 2023-10-05T19:40:31.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.422+0000 7f6d96433e40 10 moving default.rgw.meta+root+myfoo to cache LRU end 2023-10-05T19:40:31.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.422+0000 7f6d96433e40 20 get_system_obj_state: rctx=0x7fff8d856620 obj=default.rgw.meta:root:.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 state=0x5623cc6812b0 s->prefetch_data=0 2023-10-05T19:40:31.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.422+0000 7f6d96433e40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 : miss 2023-10-05T19:40:31.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.422+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:107 8.e 8:70176bbd:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc896e70 con 0x7f6d84007030 2023-10-05T19:40:31.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.422+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(107 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 [call out=48b,getxattrs out=218b,stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 290+0+282 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x7f6d84007030 2023-10-05T19:40:31.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.422+0000 7f6d96433e40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 info.flags=0x16 2023-10-05T19:40:31.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.422+0000 7f6d96433e40 10 adding default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 to cache LRU end 2023-10-05T19:40:31.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.422+0000 7f6d96433e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:31.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.422+0000 7f6d96433e40 10 updating xattr: name=user.rgw.acl bl.length()=123 2023-10-05T19:40:31.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.422+0000 7f6d96433e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:31.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.422+0000 7f6d96433e40 20 Read xattr: user.rgw.acl 2023-10-05T19:40:31.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.422+0000 7f6d96433e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:31.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.422+0000 7f6d96433e40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 : type miss (requested=0x13, cached=0x16) 2023-10-05T19:40:31.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.422+0000 7f6d96433e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:31.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.422+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:108 8.e 8:70176bbd:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc8999c0 con 0x7f6d84007030 2023-10-05T19:40:31.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.422+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(108 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 [call,call out=48b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv7 ondisk = 0) v8 ==== 332+0+618 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x7f6d84007030 2023-10-05T19:40:31.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.422+0000 7f6d96433e40 20 rados_obj.operate() r=0 bl.length=352 2023-10-05T19:40:31.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.422+0000 7f6d96433e40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 info.flags=0x13 2023-10-05T19:40:31.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.422+0000 7f6d96433e40 10 moving default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 to cache LRU end 2023-10-05T19:40:31.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.422+0000 7f6d96433e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:31.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.422+0000 7f6d96433e40 10 updating xattr: name=user.rgw.acl bl.length()=123 2023-10-05T19:40:31.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.422+0000 7f6d96433e40 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 2023-10-05T19:40:31.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.422+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:109 8.e 8:70176bbd:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~352 in=352b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc89eea0 con 0x7f6d84007030 2023-10-05T19:40:31.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.426+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(109 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 [delete,create,call,call,writefull 0~352,setxattr (123)] v28'8 uv8 ondisk = 0) v8 ==== 416+0+0 (crc 0 0 0) 0x7f6d8c00a5a0 con 0x7f6d84007030 2023-10-05T19:40:31.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.426+0000 7f6d96433e40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 info.flags=0x17 2023-10-05T19:40:31.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.426+0000 7f6d96433e40 10 moving default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 to cache LRU end 2023-10-05T19:40:31.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.426+0000 7f6d96433e40 10 updating xattr: name=user.rgw.acl bl.length()=123 2023-10-05T19:40:31.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.426+0000 7f6d96433e40 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1, ofs0, ns] 2023-10-05T19:40:31.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.426+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:110 7.e 7:7759931f:::notify.3:head [notify cookie 94711755435456 in=737b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc899220 con 0x5623cc807dc0 2023-10-05T19:40:31.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.430+0000 7f6d93f68700 1 -- 172.21.15.12:0/668887359 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== watch-notify(notify (1) cookie 94711754867360 notify 120259084307 ret 0) v3 ==== 767+0+0 (crc 0 0 0) 0x5623cc7fa740 con 0x5623cc807dc0 2023-10-05T19:40:31.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.430+0000 7f6d91f64700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084307 cookie 94711754867360 notifier 4591 bl.length()=725 2023-10-05T19:40:31.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.430+0000 7f6d91f64700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 info.flags=0x17 2023-10-05T19:40:31.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.430+0000 7f6d91f64700 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 to cache LRU end 2023-10-05T19:40:31.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.430+0000 7f6d91f64700 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2023-10-05T19:40:31.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.430+0000 7f6d91f64700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:111 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6d78003d90 con 0x5623cc807dc0 2023-10-05T19:40:31.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.430+0000 7f6d93f68700 1 -- 172.21.15.12:0/668887359 <== osd.2 v2:172.21.15.12:6800/29934 23 ==== osd_op_reply(110 notify.3 [notify cookie 94711755435456 out=8b] v0'0 uv183 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f6d84022020 con 0x5623cc807dc0 2023-10-05T19:40:31.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.430+0000 7f6d93f68700 1 -- 172.21.15.12:0/668887359 <== osd.2 v2:172.21.15.12:6800/29934 24 ==== osd_op_reply(111 notify.3 [notify-ack] v0'0 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6d84022020 con 0x5623cc807dc0 2023-10-05T19:40:31.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.430+0000 7f6d93f68700 1 -- 172.21.15.12:0/668887359 <== osd.2 v2:172.21.15.12:6800/29934 25 ==== watch-notify(notify_complete (2) cookie 94711755435456 notify 120259084307 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5623cc7fa2a0 con 0x5623cc807dc0 2023-10-05T19:40:31.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.430+0000 7f6d96433e40 20 handle_bi_update(): bucket=:myfoo[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1]): orig_sources= new_sources= 2023-10-05T19:40:31.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.430+0000 7f6d96433e40 20 handle_bi_update(): bucket=:myfoo[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1]): potential sources added=[] removed=[] 2023-10-05T19:40:31.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.430+0000 7f6d96433e40 20 handle_bi_update(): bucket=:myfoo[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1]): orig_dests= new_dests= 2023-10-05T19:40:31.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.430+0000 7f6d96433e40 20 handle_bi_update(): bucket=:myfoo[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1]): potential dests added=[] removed=[] 2023-10-05T19:40:31.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.430+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:112 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.set in=58b,writefull 0~357 in=357b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc89edc0 con 0x5623cc7d27e0 2023-10-05T19:40:31.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.430+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(112 foo [delete,create,call,writefull 0~357] v28'12 uv12 ondisk = 0) v8 ==== 273+0+0 (crc 0 0 0) 0x7f6d8c049040 con 0x5623cc7d27e0 2023-10-05T19:40:31.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.430+0000 7f6d96433e40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2023-10-05T19:40:31.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.430+0000 7f6d96433e40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:31.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.430+0000 7f6d96433e40 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2023-10-05T19:40:31.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.430+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:113 7.d 7:b4812045:::notify.4:head [notify cookie 94711755435456 in=545b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc895630 con 0x5623cc7d27e0 2023-10-05T19:40:31.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.434+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== watch-notify(notify (1) cookie 94711754872528 notify 120259084302 ret 0) v3 ==== 575+0+0 (crc 0 0 0) 0x7f6d8c02f040 con 0x5623cc7d27e0 2023-10-05T19:40:31.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.434+0000 7f6d91763700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084302 cookie 94711754872528 notifier 4591 bl.length()=533 2023-10-05T19:40:31.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.434+0000 7f6d91763700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2023-10-05T19:40:31.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.434+0000 7f6d91763700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:31.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.434+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(113 notify.4 [notify cookie 94711755435456 out=8b] v0'0 uv183 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f6d8c049040 con 0x5623cc7d27e0 2023-10-05T19:40:31.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.434+0000 7f6d91763700 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:114 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7f6ce4003440 con 0x5623cc7d27e0 2023-10-05T19:40:31.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.434+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== watch-notify(notify_complete (2) cookie 94711755435456 notify 120259084302 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f6d8c039da0 con 0x5623cc7d27e0 2023-10-05T19:40:31.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.434+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.1 v2:172.21.15.12:6808/29933 39 ==== osd_op_reply(114 notify.4 [notify-ack] v0'0 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6d8c049040 con 0x5623cc7d27e0 2023-10-05T19:40:31.976 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:40:31.977 INFO:teuthology.orchestra.run.smithi012.stdout: "user_id": "foo", 2023-10-05T19:40:31.977 INFO:teuthology.orchestra.run.smithi012.stdout: "display_name": "Foo", 2023-10-05T19:40:31.977 INFO:teuthology.orchestra.run.smithi012.stdout: "email": "foo@foo.com", 2023-10-05T19:40:31.977 INFO:teuthology.orchestra.run.smithi012.stdout: "suspended": 1, 2023-10-05T19:40:31.977 INFO:teuthology.orchestra.run.smithi012.stdout: "max_buckets": 4, 2023-10-05T19:40:31.977 INFO:teuthology.orchestra.run.smithi012.stdout: "subusers": [], 2023-10-05T19:40:31.977 INFO:teuthology.orchestra.run.smithi012.stdout: "keys": [ 2023-10-05T19:40:31.977 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:31.977 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:40:31.977 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2023-10-05T19:40:31.977 INFO:teuthology.orchestra.run.smithi012.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2023-10-05T19:40:31.977 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:31.978 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:31.978 INFO:teuthology.orchestra.run.smithi012.stdout: "swift_keys": [], 2023-10-05T19:40:31.978 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": [], 2023-10-05T19:40:31.978 INFO:teuthology.orchestra.run.smithi012.stdout: "op_mask": "read, write, delete", 2023-10-05T19:40:31.978 INFO:teuthology.orchestra.run.smithi012.stdout: "default_placement": "", 2023-10-05T19:40:31.978 INFO:teuthology.orchestra.run.smithi012.stdout: "default_storage_class": "", 2023-10-05T19:40:31.978 INFO:teuthology.orchestra.run.smithi012.stdout: "placement_tags": [], 2023-10-05T19:40:31.978 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket_quota": { 2023-10-05T19:40:31.978 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:31.978 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:31.978 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:31.978 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:31.978 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:31.979 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:31.979 INFO:teuthology.orchestra.run.smithi012.stdout: "user_quota": { 2023-10-05T19:40:31.979 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:31.979 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:31.979 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:31.979 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:31.979 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:31.979 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:31.979 INFO:teuthology.orchestra.run.smithi012.stdout: "temp_url_keys": [], 2023-10-05T19:40:31.979 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "rgw", 2023-10-05T19:40:31.979 INFO:teuthology.orchestra.run.smithi012.stdout: "mfa_ids": [] 2023-10-05T19:40:31.979 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:40:31.980 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:40:31.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.434+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:115 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94711754852672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc7e9a30 con 0x5623cc807dc0 2023-10-05T19:40:31.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.438+0000 7f6d93f68700 1 -- 172.21.15.12:0/668887359 <== osd.2 v2:172.21.15.12:6800/29934 26 ==== osd_op_reply(115 notify.0 [watch unwatch cookie 94711754852672] v28'185 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6d84022020 con 0x5623cc807dc0 2023-10-05T19:40:31.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.438+0000 7f6d96433e40 20 remove_watcher() i=0 2023-10-05T19:40:31.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.438+0000 7f6d96433e40 2 removed watcher, disabling cache 2023-10-05T19:40:31.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.438+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:116 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94711754863296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc810940 con 0x5623cc7d27e0 2023-10-05T19:40:31.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.438+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.1 v2:172.21.15.12:6808/29933 40 ==== osd_op_reply(116 notify.1 [watch unwatch cookie 94711754863296] v28'185 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6d8c049040 con 0x5623cc7d27e0 2023-10-05T19:40:31.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.438+0000 7f6d96433e40 20 remove_watcher() i=1 2023-10-05T19:40:31.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.438+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:117 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94711754864400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc810940 con 0x5623cc80c560 2023-10-05T19:40:31.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.442+0000 7f6d93767700 1 -- 172.21.15.12:0/668887359 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(117 notify.2 [watch unwatch cookie 94711754864400] v28'185 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6d7c054020 con 0x5623cc80c560 2023-10-05T19:40:31.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.442+0000 7f6d96433e40 20 remove_watcher() i=2 2023-10-05T19:40:31.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.442+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:118 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94711754867360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc810940 con 0x5623cc807dc0 2023-10-05T19:40:31.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.446+0000 7f6d93f68700 1 -- 172.21.15.12:0/668887359 <== osd.2 v2:172.21.15.12:6800/29934 27 ==== osd_op_reply(118 notify.3 [watch unwatch cookie 94711754867360] v28'185 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6d84022020 con 0x5623cc807dc0 2023-10-05T19:40:31.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.446+0000 7f6d96433e40 20 remove_watcher() i=3 2023-10-05T19:40:31.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.446+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:119 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94711754872528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc810940 con 0x5623cc7d27e0 2023-10-05T19:40:31.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.450+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.1 v2:172.21.15.12:6808/29933 41 ==== osd_op_reply(119 notify.4 [watch unwatch cookie 94711754872528] v28'185 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6d8c049040 con 0x5623cc7d27e0 2023-10-05T19:40:31.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.450+0000 7f6d96433e40 20 remove_watcher() i=4 2023-10-05T19:40:31.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.450+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:120 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94711754870272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc810940 con 0x7f6d84007030 2023-10-05T19:40:31.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.450+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.0 v2:172.21.15.8:6800/27563 30 ==== osd_op_reply(120 notify.5 [watch unwatch cookie 94711754870272] v28'185 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6d8c049040 con 0x7f6d84007030 2023-10-05T19:40:31.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.450+0000 7f6d96433e40 20 remove_watcher() i=5 2023-10-05T19:40:31.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.450+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:121 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94711754875808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc810940 con 0x7f6d84007030 2023-10-05T19:40:31.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.454+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.0 v2:172.21.15.8:6800/27563 31 ==== osd_op_reply(121 notify.6 [watch unwatch cookie 94711754875808] v28'185 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6d8c049040 con 0x7f6d84007030 2023-10-05T19:40:31.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.454+0000 7f6d96433e40 20 remove_watcher() i=6 2023-10-05T19:40:31.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.454+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:122 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94711754878112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc8163a0 con 0x7f6d84007030 2023-10-05T19:40:31.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.458+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.0 v2:172.21.15.8:6800/27563 32 ==== osd_op_reply(122 notify.7 [watch unwatch cookie 94711754878112] v28'185 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6d8c049040 con 0x7f6d84007030 2023-10-05T19:40:31.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.458+0000 7f6d96433e40 20 remove_watcher() i=7 2023-10-05T19:40:31.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.458+0000 7f6d657e2700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:31.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.458+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:123 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94711754880768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5623cc810940 con 0x5623cc7d27e0 2023-10-05T19:40:31.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.462+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 <== osd.1 v2:172.21.15.12:6808/29933 42 ==== osd_op_reply(123 data_loggenerations_metadata [watch unwatch cookie 94711754880768] v28'170 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f6d8c049040 con 0x5623cc7d27e0 2023-10-05T19:40:31.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.462+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f6d84007030 msgr2=0x7f6d84026880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:31.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.462+0000 7f6d96433e40 1 --2- 172.21.15.12:0/668887359 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f6d84007030 0x7f6d84026880 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:31.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.462+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5623cc7d27e0 msgr2=0x5623cc8057c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:31.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.462+0000 7f6d96433e40 1 --2- 172.21.15.12:0/668887359 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5623cc7d27e0 0x5623cc8057c0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:31.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.462+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5623cc807dc0 msgr2=0x5623cc80a1a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:31.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.462+0000 7f6d96433e40 1 --2- 172.21.15.12:0/668887359 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5623cc807dc0 0x5623cc80a1a0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:31.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.462+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5623cc80c560 msgr2=0x5623cc80e970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:31.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.462+0000 7f6d96433e40 1 --2- 172.21.15.12:0/668887359 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5623cc80c560 0x5623cc80e970 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:31.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.462+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f6d74030bf0 msgr2=0x7f6d740330a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:31.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.462+0000 7f6d96433e40 1 --2- 172.21.15.12:0/668887359 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f6d74030bf0 0x7f6d740330a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c001e90 tx=0x7f6d7c0013b0 comp rx=0 tx=0).stop 2023-10-05T19:40:31.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.462+0000 7f6d94769700 1 -- 172.21.15.12:0/668887359 reap_dead start 2023-10-05T19:40:31.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.462+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623cc7d2410 msgr2=0x5623cc7f9d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:31.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.462+0000 7f6d96433e40 1 --2- 172.21.15.12:0/668887359 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623cc7d2410 0x5623cc7f9d60 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6d84000c00 tx=0x7f6d84003050 comp rx=0 tx=0).stop 2023-10-05T19:40:31.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.462+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 shutdown_connections 2023-10-05T19:40:31.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.462+0000 7f6d96433e40 1 --2- 172.21.15.12:0/668887359 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623cc7d2410 0x5623cc7f9d60 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:31.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.462+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 >> 172.21.15.12:0/668887359 conn(0x5623cc7cc720 msgr2=0x5623cc7ed790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:31.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.462+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 shutdown_connections 2023-10-05T19:40:31.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.462+0000 7f6d96433e40 1 -- 172.21.15.12:0/668887359 wait complete. 2023-10-05T19:40:31.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.462+0000 7f6d96433e40 1 -- 172.21.15.12:0/3806492544 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f6cd4030c50 msgr2=0x7f6cd4033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:31.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.462+0000 7f6d96433e40 1 --2- 172.21.15.12:0/3806492544 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f6cd4030c50 0x7f6cd4033100 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x5623cc8180b0 tx=0x7f6d8c00aea0 comp rx=0 tx=0).stop 2023-10-05T19:40:31.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.462+0000 7f6d96433e40 1 -- 172.21.15.12:0/3806492544 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623cc8253b0 msgr2=0x5623cc82af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:31.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.462+0000 7f6d96433e40 1 --2- 172.21.15.12:0/3806492544 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623cc8253b0 0x5623cc82af40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c049040 tx=0x7f6d7c03d030 comp rx=0 tx=0).stop 2023-10-05T19:40:31.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.462+0000 7f6d96433e40 1 -- 172.21.15.12:0/3806492544 shutdown_connections 2023-10-05T19:40:31.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.462+0000 7f6d96433e40 1 --2- 172.21.15.12:0/3806492544 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f6cd4030c50 0x7f6cd4033100 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:31.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.462+0000 7f6d96433e40 1 --2- 172.21.15.12:0/3806492544 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5623cc8253b0 0x5623cc82af40 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:31.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.462+0000 7f6d96433e40 1 -- 172.21.15.12:0/3806492544 >> 172.21.15.12:0/3806492544 conn(0x5623cc820770 msgr2=0x5623cc81e810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:31.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.466+0000 7f6d96433e40 1 -- 172.21.15.12:0/3806492544 shutdown_connections 2023-10-05T19:40:31.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:31.466+0000 7f6d96433e40 1 -- 172.21.15.12:0/3806492544 wait complete. 2023-10-05T19:40:31.989 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:40:31.991 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'enable', '--uid', 'foo'] 2023-10-05T19:40:31.991 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'enable', '--uid', 'foo'] 2023-10-05T19:40:31.991 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user enable --uid foo 2023-10-05T19:40:32.032 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:32.032 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:32.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.022+0000 7fa6b5903e40 1 Processor -- start 2023-10-05T19:40:32.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.022+0000 7fa6b5903e40 1 -- start start 2023-10-05T19:40:32.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.022+0000 7fa6b5903e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x5618567827e0 0x5618567816d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:32.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.022+0000 7fa6b5903e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5618566cebd0 con 0x561856782410 2023-10-05T19:40:32.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.022+0000 7fa6b5903e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x56185668bee0 con 0x5618567827e0 2023-10-05T19:40:32.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.022+0000 7fa6b2c37700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x5618567827e0 0x5618567816d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:32.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.022+0000 7fa6b3438700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x561856782410 0x561856780fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:35758/0 (socket says 172.21.15.12:35758) 2023-10-05T19:40:32.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.022+0000 7fa6b3438700 1 -- 172.21.15.12:0/11665037 learned_addr learned my addr 172.21.15.12:0/11665037 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:32.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.026+0000 7fa6b2436700 1 -- 172.21.15.12:0/11665037 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2576855357 0 0) 0x5618566cebd0 con 0x561856782410 2023-10-05T19:40:32.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.026+0000 7fa6b2436700 1 -- 172.21.15.12:0/11665037 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa69c001e10 con 0x561856782410 2023-10-05T19:40:32.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.026+0000 7fa6b2c37700 1 -- 172.21.15.12:0/11665037 >> v1:172.21.15.8:6789/0 conn(0x561856782410 legacy=0x561856780fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:32.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.026+0000 7fa6b2c37700 1 -- 172.21.15.12:0/11665037 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561856781cd0 con 0x5618567827e0 2023-10-05T19:40:32.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.026+0000 7fa6b2c37700 1 --2- 172.21.15.12:0/11665037 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5618567827e0 0x5618567816d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa6a800a5d0 tx=0x7fa6a8005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=920e84b8a61f2831 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:32.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.026+0000 7fa6b2436700 1 -- 172.21.15.12:0/11665037 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6a8007610 con 0x5618567827e0 2023-10-05T19:40:32.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.026+0000 7fa6b2436700 1 -- 172.21.15.12:0/11665037 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6a8007770 con 0x5618567827e0 2023-10-05T19:40:32.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.026+0000 7fa6b2436700 1 -- 172.21.15.12:0/11665037 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6a8007a40 con 0x5618567827e0 2023-10-05T19:40:32.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.026+0000 7fa6b5903e40 1 -- 172.21.15.12:0/11665037 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5618567827e0 msgr2=0x5618567816d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:32.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.026+0000 7fa6b5903e40 1 --2- 172.21.15.12:0/11665037 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5618567827e0 0x5618567816d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa6a800a5d0 tx=0x7fa6a8005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:32.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.026+0000 7fa6b5903e40 1 -- 172.21.15.12:0/11665037 shutdown_connections 2023-10-05T19:40:32.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.026+0000 7fa6b5903e40 1 --2- 172.21.15.12:0/11665037 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5618567827e0 0x5618567816d0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:32.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.026+0000 7fa6b5903e40 1 -- 172.21.15.12:0/11665037 >> 172.21.15.12:0/11665037 conn(0x56185677c720 msgr2=0x56185677eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:32.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.026+0000 7fa6b5903e40 1 -- 172.21.15.12:0/11665037 shutdown_connections 2023-10-05T19:40:32.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.026+0000 7fa6b5903e40 1 -- 172.21.15.12:0/11665037 wait complete. 2023-10-05T19:40:32.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.026+0000 7fa6b5903e40 1 Processor -- start 2023-10-05T19:40:32.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.026+0000 7fa6b5903e40 1 -- start start 2023-10-05T19:40:32.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.026+0000 7fa6b5903e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561856782410 0x5618567a2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:32.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.026+0000 7fa6b5903e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56185664fe30 con 0x561856782410 2023-10-05T19:40:32.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.026+0000 7fa6b3438700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561856782410 0x5618567a2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:32.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.030+0000 7fa6b3438700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561856782410 0x5618567a2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:59008/0 (socket says 172.21.15.12:59008) 2023-10-05T19:40:32.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.030+0000 7fa6b3438700 1 -- 172.21.15.12:0/1137288744 learned_addr learned my addr 172.21.15.12:0/1137288744 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:32.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.030+0000 7fa6b3438700 1 -- 172.21.15.12:0/1137288744 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a8005cf0 con 0x561856782410 2023-10-05T19:40:32.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.030+0000 7fa6b3438700 1 --2- 172.21.15.12:0/1137288744 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561856782410 0x5618567a2f80 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac006e70 tx=0x7fa6ac006a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a91120ef22d08cb2 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:32.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.030+0000 7fa6b1434700 1 -- 172.21.15.12:0/1137288744 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6ac006110 con 0x561856782410 2023-10-05T19:40:32.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.030+0000 7fa6b1434700 1 -- 172.21.15.12:0/1137288744 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6ac006290 con 0x561856782410 2023-10-05T19:40:32.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.030+0000 7fa6b1434700 1 -- 172.21.15.12:0/1137288744 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6ac004520 con 0x561856782410 2023-10-05T19:40:32.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.030+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1137288744 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561856782410 msgr2=0x5618567a2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:32.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.030+0000 7fa6b5903e40 1 --2- 172.21.15.12:0/1137288744 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561856782410 0x5618567a2f80 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac006e70 tx=0x7fa6ac006a20 comp rx=0 tx=0).stop 2023-10-05T19:40:32.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.030+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1137288744 shutdown_connections 2023-10-05T19:40:32.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.030+0000 7fa6b5903e40 1 --2- 172.21.15.12:0/1137288744 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561856782410 0x5618567a2f80 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:32.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.030+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1137288744 >> 172.21.15.12:0/1137288744 conn(0x56185677c720 msgr2=0x56185679eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:32.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.030+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1137288744 shutdown_connections 2023-10-05T19:40:32.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.030+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1137288744 wait complete. 2023-10-05T19:40:32.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.030+0000 7fa6b5903e40 1 Processor -- start 2023-10-05T19:40:32.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.030+0000 7fa6b5903e40 1 -- start start 2023-10-05T19:40:32.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.030+0000 7fa6b5903e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561856782410 0x5618567a9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:32.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.030+0000 7fa6b5903e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x561856622ed0 con 0x561856782410 2023-10-05T19:40:32.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.030+0000 7fa6b3438700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561856782410 0x5618567a9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:32.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.030+0000 7fa6b3438700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561856782410 0x5618567a9d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:59022/0 (socket says 172.21.15.12:59022) 2023-10-05T19:40:32.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.030+0000 7fa6b3438700 1 -- 172.21.15.12:0/1228551133 learned_addr learned my addr 172.21.15.12:0/1228551133 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:32.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.030+0000 7fa6b3438700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa69c001e10 con 0x561856782410 2023-10-05T19:40:32.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.030+0000 7fa6b3438700 1 --2- 172.21.15.12:0/1228551133 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561856782410 0x5618567a9d10 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x56185679f660 tx=0x7fa6ac014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:32.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.034+0000 7fa6a37fe700 1 -- 172.21.15.12:0/1228551133 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6ac006400 con 0x561856782410 2023-10-05T19:40:32.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.034+0000 7fa6a37fe700 1 -- 172.21.15.12:0/1228551133 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6ac004e50 con 0x561856782410 2023-10-05T19:40:32.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.034+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5618567aa250 con 0x561856782410 2023-10-05T19:40:32.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.034+0000 7fa6a37fe700 1 -- 172.21.15.12:0/1228551133 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6ac013b10 con 0x561856782410 2023-10-05T19:40:32.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.034+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5618567aa690 con 0x561856782410 2023-10-05T19:40:32.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.034+0000 7fa6a37fe700 1 -- 172.21.15.12:0/1228551133 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fa6ac01a070 con 0x561856782410 2023-10-05T19:40:32.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.034+0000 7fa6a37fe700 1 --2- 172.21.15.12:0/1228551133 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fa698030c50 0x7fa698033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:32.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.034+0000 7fa6a37fe700 1 -- 172.21.15.12:0/1228551133 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7fa6ac044bc0 con 0x561856782410 2023-10-05T19:40:32.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.034+0000 7fa6b2c37700 1 --2- 172.21.15.12:0/1228551133 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fa698030c50 0x7fa698033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:32.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.034+0000 7fa6b5903e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:32.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.034+0000 7fa6b5903e40 1 --2- 172.21.15.12:0/1228551133 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5618567827e0 0x5618567b57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:32.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.034+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567b5ee0 con 0x5618567827e0 2023-10-05T19:40:32.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.034+0000 7fa6b3c39700 1 --2- 172.21.15.12:0/1228551133 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5618567827e0 0x5618567b57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:32.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.034+0000 7fa6b2c37700 1 --2- 172.21.15.12:0/1228551133 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fa698030c50 0x7fa698033100 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa6a800afd0 tx=0x7fa6a8014000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:32.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.034+0000 7fa6b3c39700 1 --2- 172.21.15.12:0/1228551133 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5618567827e0 0x5618567b57b0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:32.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.038+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa6a400a500 con 0x5618567827e0 2023-10-05T19:40:32.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.038+0000 7fa6b5903e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:32.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.038+0000 7fa6b5903e40 20 realm 2023-10-05T19:40:32.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.038+0000 7fa6b5903e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:32.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.038+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567b2d90 con 0x5618567827e0 2023-10-05T19:40:32.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.038+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa6a400a500 con 0x5618567827e0 2023-10-05T19:40:32.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.038+0000 7fa6b5903e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:32.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.038+0000 7fa6b5903e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:32.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.038+0000 7fa6b5903e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:32.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.038+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567b69c0 con 0x5618567827e0 2023-10-05T19:40:32.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.038+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa6a400a500 con 0x5618567827e0 2023-10-05T19:40:32.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.038+0000 7fa6b5903e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:32.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.038+0000 7fa6b5903e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:32.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.038+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567b65d0 con 0x5618567827e0 2023-10-05T19:40:32.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.038+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fa6a400a500 con 0x5618567827e0 2023-10-05T19:40:32.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.038+0000 7fa6b5903e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:32.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.038+0000 7fa6b5903e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:32.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.038+0000 7fa6b5903e40 1 --2- 172.21.15.12:0/1228551133 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5618567b7d60 0x5618567ba140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:32.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.038+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567ba680 con 0x5618567b7d60 2023-10-05T19:40:32.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.038+0000 7fa6b3438700 1 --2- 172.21.15.12:0/1228551133 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5618567b7d60 0x5618567ba140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:32.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.038+0000 7fa6b3438700 1 --2- 172.21.15.12:0/1228551133 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5618567b7d60 0x5618567ba140 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:32.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.038+0000 7fa6b3438700 1 -- 172.21.15.12:0/1228551133 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fa6ac025020 con 0x5618567b7d60 2023-10-05T19:40:32.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.038+0000 7fa6b5903e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:32.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.038+0000 7fa6b5903e40 20 searching for the correct realm 2023-10-05T19:40:32.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.038+0000 7fa6b5903e40 1 --2- 172.21.15.12:0/1228551133 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5618567bc500 0x5618567be910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:32.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.038+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567befc0 con 0x5618567bc500 2023-10-05T19:40:32.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.042+0000 7fa6b2c37700 1 --2- 172.21.15.12:0/1228551133 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5618567bc500 0x5618567be910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:32.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.042+0000 7fa6b2c37700 1 --2- 172.21.15.12:0/1228551133 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5618567bc500 0x5618567be910 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:32.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.042+0000 7fa6b2c37700 1 -- 172.21.15.12:0/1228551133 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fa6a8007a50 con 0x5618567bc500 2023-10-05T19:40:32.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.042+0000 7fa6b2c37700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567befc0 con 0x5618567b7d60 2023-10-05T19:40:32.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.042+0000 7fa6b3438700 1 -- 172.21.15.12:0/1228551133 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6ac025020 con 0x5618567b7d60 2023-10-05T19:40:32.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.042+0000 7fa6b3438700 1 --2- 172.21.15.12:0/1228551133 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fa6ac01ec10 0x7fa6ac0113e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:32.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.042+0000 7fa6b3438700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567befc0 con 0x7fa6ac01ec10 2023-10-05T19:40:32.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.042+0000 7fa6b3c39700 1 --2- 172.21.15.12:0/1228551133 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fa6ac01ec10 0x7fa6ac0113e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:32.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.046+0000 7fa6b3c39700 1 --2- 172.21.15.12:0/1228551133 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fa6ac01ec10 0x7fa6ac0113e0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:32.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.046+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a4014040 con 0x7fa6ac01ec10 2023-10-05T19:40:32.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.046+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567befc0 con 0x5618567827e0 2023-10-05T19:40:32.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.046+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a4014040 con 0x5618567827e0 2023-10-05T19:40:32.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.046+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567befc0 con 0x5618567827e0 2023-10-05T19:40:32.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.046+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a4014040 con 0x5618567827e0 2023-10-05T19:40:32.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.046+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567befc0 con 0x7fa6ac01ec10 2023-10-05T19:40:32.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.046+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(11 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a4014040 con 0x7fa6ac01ec10 2023-10-05T19:40:32.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.046+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567befc0 con 0x7fa6ac01ec10 2023-10-05T19:40:32.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.050+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a4014040 con 0x7fa6ac01ec10 2023-10-05T19:40:32.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.050+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567befc0 con 0x5618567b7d60 2023-10-05T19:40:32.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.050+0000 7fa6b3438700 1 -- 172.21.15.12:0/1228551133 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6ac025020 con 0x5618567b7d60 2023-10-05T19:40:32.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.050+0000 7fa6b3438700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567befc0 con 0x5618567bc500 2023-10-05T19:40:32.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.050+0000 7fa6b2c37700 1 -- 172.21.15.12:0/1228551133 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(14 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a8007a50 con 0x5618567bc500 2023-10-05T19:40:32.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.050+0000 7fa6b2c37700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567befc0 con 0x5618567827e0 2023-10-05T19:40:32.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.050+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a4014040 con 0x5618567827e0 2023-10-05T19:40:32.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.050+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567befc0 con 0x7fa6ac01ec10 2023-10-05T19:40:32.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.050+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(16 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a4014040 con 0x7fa6ac01ec10 2023-10-05T19:40:32.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.050+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567befc0 con 0x5618567827e0 2023-10-05T19:40:32.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.050+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(17 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a4014040 con 0x5618567827e0 2023-10-05T19:40:32.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.050+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567befc0 con 0x5618567b7d60 2023-10-05T19:40:32.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.054+0000 7fa6b3438700 1 -- 172.21.15.12:0/1228551133 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(18 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6ac025020 con 0x5618567b7d60 2023-10-05T19:40:32.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.054+0000 7fa6b3438700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567befc0 con 0x5618567bc500 2023-10-05T19:40:32.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.054+0000 7fa6b2c37700 1 -- 172.21.15.12:0/1228551133 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(19 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a8007a50 con 0x5618567bc500 2023-10-05T19:40:32.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.054+0000 7fa6b2c37700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567befc0 con 0x5618567bc500 2023-10-05T19:40:32.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.054+0000 7fa6b2c37700 1 -- 172.21.15.12:0/1228551133 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a8007a50 con 0x5618567bc500 2023-10-05T19:40:32.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.054+0000 7fa6b2c37700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567befc0 con 0x7fa6ac01ec10 2023-10-05T19:40:32.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.054+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(21 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a4014040 con 0x7fa6ac01ec10 2023-10-05T19:40:32.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.054+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567befc0 con 0x5618567bc500 2023-10-05T19:40:32.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.054+0000 7fa6b2c37700 1 -- 172.21.15.12:0/1228551133 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(22 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a8007a50 con 0x5618567bc500 2023-10-05T19:40:32.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.054+0000 7fa6b2c37700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567befc0 con 0x5618567bc500 2023-10-05T19:40:32.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.058+0000 7fa6b2c37700 1 -- 172.21.15.12:0/1228551133 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(23 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a8007a50 con 0x5618567bc500 2023-10-05T19:40:32.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.058+0000 7fa6b2c37700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567befc0 con 0x5618567827e0 2023-10-05T19:40:32.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.058+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(24 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a4014040 con 0x5618567827e0 2023-10-05T19:40:32.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.058+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567befc0 con 0x5618567827e0 2023-10-05T19:40:32.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.058+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(25 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a4014040 con 0x5618567827e0 2023-10-05T19:40:32.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.058+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567befc0 con 0x7fa6ac01ec10 2023-10-05T19:40:32.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.058+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(26 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a4014040 con 0x7fa6ac01ec10 2023-10-05T19:40:32.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.058+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567befc0 con 0x5618567bc500 2023-10-05T19:40:32.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.058+0000 7fa6b2c37700 1 -- 172.21.15.12:0/1228551133 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(27 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a8007a50 con 0x5618567bc500 2023-10-05T19:40:32.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.058+0000 7fa6b2c37700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567befc0 con 0x5618567b7d60 2023-10-05T19:40:32.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.058+0000 7fa6b3438700 1 -- 172.21.15.12:0/1228551133 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(28 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6ac025020 con 0x5618567b7d60 2023-10-05T19:40:32.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.058+0000 7fa6b3438700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567befc0 con 0x5618567827e0 2023-10-05T19:40:32.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.062+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(29 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a4014040 con 0x5618567827e0 2023-10-05T19:40:32.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.062+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567befc0 con 0x7fa6ac01ec10 2023-10-05T19:40:32.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.062+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a4014040 con 0x7fa6ac01ec10 2023-10-05T19:40:32.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.062+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567befc0 con 0x5618567bc500 2023-10-05T19:40:32.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.062+0000 7fa6b2c37700 1 -- 172.21.15.12:0/1228551133 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a8007a50 con 0x5618567bc500 2023-10-05T19:40:32.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.062+0000 7fa6b2c37700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567befc0 con 0x7fa6ac01ec10 2023-10-05T19:40:32.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.062+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a4014040 con 0x7fa6ac01ec10 2023-10-05T19:40:32.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.062+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567befc0 con 0x5618567b7d60 2023-10-05T19:40:32.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.066+0000 7fa6b3438700 1 -- 172.21.15.12:0/1228551133 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(33 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6ac025020 con 0x5618567b7d60 2023-10-05T19:40:32.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.066+0000 7fa6b3438700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567befc0 con 0x5618567bc500 2023-10-05T19:40:32.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.066+0000 7fa6b2c37700 1 -- 172.21.15.12:0/1228551133 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(34 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a8007a50 con 0x5618567bc500 2023-10-05T19:40:32.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.066+0000 7fa6b2c37700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567befc0 con 0x7fa6ac01ec10 2023-10-05T19:40:32.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.066+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(35 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a4014040 con 0x7fa6ac01ec10 2023-10-05T19:40:32.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.066+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567befc0 con 0x5618567827e0 2023-10-05T19:40:32.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.066+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fa6a4014040 con 0x5618567827e0 2023-10-05T19:40:32.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.066+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567befc0 con 0x5618567b7d60 2023-10-05T19:40:32.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.070+0000 7fa6b3438700 1 -- 172.21.15.12:0/1228551133 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(37 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fa6ac025020 con 0x5618567b7d60 2023-10-05T19:40:32.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.070+0000 7fa6b5903e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:32.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.070+0000 7fa6b5903e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:32.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.070+0000 7fa6b5903e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:32.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.070+0000 7fa6b5903e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:32.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.070+0000 7fa6b5903e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:32.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.070+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567bb250 con 0x5618567827e0 2023-10-05T19:40:32.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.070+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa6a4014040 con 0x5618567827e0 2023-10-05T19:40:32.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.070+0000 7fa6b5903e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:32.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.070+0000 7fa6b5903e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:32.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.070+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567bae80 con 0x5618567827e0 2023-10-05T19:40:32.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.070+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fa6a4014040 con 0x5618567827e0 2023-10-05T19:40:32.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.070+0000 7fa6b5903e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:32.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.070+0000 7fa6b5903e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:32.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.070+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567bf430 con 0x5618567bc500 2023-10-05T19:40:32.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.070+0000 7fa6b2c37700 1 -- 172.21.15.12:0/1228551133 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fa6a8007a50 con 0x5618567bc500 2023-10-05T19:40:32.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.070+0000 7fa6b5903e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:32.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.070+0000 7fa6b5903e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:32.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.070+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567bf7e0 con 0x5618567bc500 2023-10-05T19:40:32.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.070+0000 7fa6b2c37700 1 -- 172.21.15.12:0/1228551133 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7fa6a8007a50 con 0x5618567bc500 2023-10-05T19:40:32.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.070+0000 7fa6b5903e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:32.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.070+0000 7fa6b5903e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:32.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.070+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567bfb90 con 0x5618567827e0 2023-10-05T19:40:32.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.070+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa6a4014040 con 0x5618567827e0 2023-10-05T19:40:32.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.070+0000 7fa6b5903e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:32.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.070+0000 7fa6b5903e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:32.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.070+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567bffa0 con 0x5618567827e0 2023-10-05T19:40:32.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.074+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fa6a4014040 con 0x5618567827e0 2023-10-05T19:40:32.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.074+0000 7fa6b5903e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:32.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.074+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567bff70 con 0x5618567bc500 2023-10-05T19:40:32.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.074+0000 7fa6b2c37700 1 -- 172.21.15.12:0/1228551133 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(44 [pgnls start_epoch 28 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fa6a8007a50 con 0x5618567bc500 2023-10-05T19:40:32.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.074+0000 7fa6b2c37700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567bff70 con 0x5618567b7d60 2023-10-05T19:40:32.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.078+0000 7fa6b3438700 1 -- 172.21.15.12:0/1228551133 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6ac025020 con 0x5618567b7d60 2023-10-05T19:40:32.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.078+0000 7fa6b3438700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567bff70 con 0x7fa6ac01ec10 2023-10-05T19:40:32.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.078+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(46 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a4014040 con 0x7fa6ac01ec10 2023-10-05T19:40:32.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.078+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567bfb90 con 0x5618567827e0 2023-10-05T19:40:32.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.078+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(47 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a4014040 con 0x5618567827e0 2023-10-05T19:40:32.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.078+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567bfb90 con 0x5618567827e0 2023-10-05T19:40:32.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.078+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(48 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a4014040 con 0x5618567827e0 2023-10-05T19:40:32.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.078+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567bfb90 con 0x7fa6ac01ec10 2023-10-05T19:40:32.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.078+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a4014040 con 0x7fa6ac01ec10 2023-10-05T19:40:32.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.078+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567bfb90 con 0x7fa6ac01ec10 2023-10-05T19:40:32.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.082+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(50 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a4014040 con 0x7fa6ac01ec10 2023-10-05T19:40:32.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.082+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567bfb90 con 0x5618567b7d60 2023-10-05T19:40:32.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.082+0000 7fa6b3438700 1 -- 172.21.15.12:0/1228551133 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(51 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6ac025020 con 0x5618567b7d60 2023-10-05T19:40:32.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.082+0000 7fa6b3438700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567bfb90 con 0x5618567bc500 2023-10-05T19:40:32.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.082+0000 7fa6b2c37700 1 -- 172.21.15.12:0/1228551133 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(52 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a8007a50 con 0x5618567bc500 2023-10-05T19:40:32.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.082+0000 7fa6b2c37700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567bfb90 con 0x5618567827e0 2023-10-05T19:40:32.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.082+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(53 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a4014040 con 0x5618567827e0 2023-10-05T19:40:32.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.082+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567bfb90 con 0x7fa6ac01ec10 2023-10-05T19:40:32.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.082+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(54 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a4014040 con 0x7fa6ac01ec10 2023-10-05T19:40:32.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.082+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567bfb90 con 0x5618567827e0 2023-10-05T19:40:32.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.086+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(55 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a4014040 con 0x5618567827e0 2023-10-05T19:40:32.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.086+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567bfb90 con 0x5618567b7d60 2023-10-05T19:40:32.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.086+0000 7fa6b3438700 1 -- 172.21.15.12:0/1228551133 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(56 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6ac025020 con 0x5618567b7d60 2023-10-05T19:40:32.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.086+0000 7fa6b3438700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567bfb90 con 0x5618567bc500 2023-10-05T19:40:32.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.086+0000 7fa6b2c37700 1 -- 172.21.15.12:0/1228551133 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(57 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a8007a50 con 0x5618567bc500 2023-10-05T19:40:32.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.086+0000 7fa6b2c37700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567bfb90 con 0x5618567bc500 2023-10-05T19:40:32.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.086+0000 7fa6b2c37700 1 -- 172.21.15.12:0/1228551133 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(58 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a8007a50 con 0x5618567bc500 2023-10-05T19:40:32.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.086+0000 7fa6b2c37700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567bfb90 con 0x7fa6ac01ec10 2023-10-05T19:40:32.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.086+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(59 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a4014040 con 0x7fa6ac01ec10 2023-10-05T19:40:32.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.086+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567bfb90 con 0x5618567bc500 2023-10-05T19:40:32.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.090+0000 7fa6b2c37700 1 -- 172.21.15.12:0/1228551133 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(60 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a8007a50 con 0x5618567bc500 2023-10-05T19:40:32.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.090+0000 7fa6b2c37700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567bfb90 con 0x5618567bc500 2023-10-05T19:40:32.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.090+0000 7fa6b2c37700 1 -- 172.21.15.12:0/1228551133 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(61 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a8007a50 con 0x5618567bc500 2023-10-05T19:40:32.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.090+0000 7fa6b2c37700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567bfb90 con 0x5618567827e0 2023-10-05T19:40:32.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.090+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a4014040 con 0x5618567827e0 2023-10-05T19:40:32.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.090+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567bfb90 con 0x5618567827e0 2023-10-05T19:40:32.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.090+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(63 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a4014040 con 0x5618567827e0 2023-10-05T19:40:32.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.090+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567bfb90 con 0x7fa6ac01ec10 2023-10-05T19:40:32.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.090+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(64 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a4014040 con 0x7fa6ac01ec10 2023-10-05T19:40:32.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.090+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567bfb90 con 0x5618567bc500 2023-10-05T19:40:32.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.094+0000 7fa6b2c37700 1 -- 172.21.15.12:0/1228551133 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(65 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a8007a50 con 0x5618567bc500 2023-10-05T19:40:32.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.094+0000 7fa6b2c37700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567bfb90 con 0x5618567b7d60 2023-10-05T19:40:32.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.094+0000 7fa6b3438700 1 -- 172.21.15.12:0/1228551133 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(66 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6ac025020 con 0x5618567b7d60 2023-10-05T19:40:32.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.094+0000 7fa6b3438700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567bfb90 con 0x5618567827e0 2023-10-05T19:40:32.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.094+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(67 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a4014040 con 0x5618567827e0 2023-10-05T19:40:32.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.094+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567bfb90 con 0x7fa6ac01ec10 2023-10-05T19:40:32.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.094+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a4014040 con 0x7fa6ac01ec10 2023-10-05T19:40:32.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.094+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567bfb90 con 0x5618567bc500 2023-10-05T19:40:32.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.098+0000 7fa6b2c37700 1 -- 172.21.15.12:0/1228551133 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(69 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a8007a50 con 0x5618567bc500 2023-10-05T19:40:32.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.098+0000 7fa6b2c37700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567bfb90 con 0x7fa6ac01ec10 2023-10-05T19:40:32.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.098+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(70 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a4014040 con 0x7fa6ac01ec10 2023-10-05T19:40:32.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.098+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567bfb90 con 0x5618567b7d60 2023-10-05T19:40:32.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.098+0000 7fa6b3438700 1 -- 172.21.15.12:0/1228551133 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(71 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6ac025020 con 0x5618567b7d60 2023-10-05T19:40:32.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.098+0000 7fa6b3438700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567bfb90 con 0x5618567bc500 2023-10-05T19:40:32.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.098+0000 7fa6b2c37700 1 -- 172.21.15.12:0/1228551133 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(72 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a8007a50 con 0x5618567bc500 2023-10-05T19:40:32.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.098+0000 7fa6b2c37700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567bfb90 con 0x7fa6ac01ec10 2023-10-05T19:40:32.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.102+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 [pgnls start_epoch 28 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6a4014040 con 0x7fa6ac01ec10 2023-10-05T19:40:32.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.102+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567bfb90 con 0x5618567827e0 2023-10-05T19:40:32.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.102+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 [pgnls start_epoch 28 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fa6a4014040 con 0x5618567827e0 2023-10-05T19:40:32.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.102+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 28 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567bfb90 con 0x5618567b7d60 2023-10-05T19:40:32.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.102+0000 7fa6b3438700 1 -- 172.21.15.12:0/1228551133 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(75 [pgnls start_epoch 28 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fa6ac025020 con 0x5618567b7d60 2023-10-05T19:40:32.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.102+0000 7fa6b5903e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:32.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.102+0000 7fa6b5903e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:32.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.102+0000 7fa6b5903e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:32.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.102+0000 7fa6b5903e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:32.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.102+0000 7fa6b5903e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:32.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.102+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567bff70 con 0x5618567827e0 2023-10-05T19:40:32.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.102+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fa6a4014040 con 0x5618567827e0 2023-10-05T19:40:32.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.102+0000 7fa6b5903e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:32.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.102+0000 7fa6b5903e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:32.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.102+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567c03d0 con 0x5618567b7d60 2023-10-05T19:40:32.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.106+0000 7fa6b3438700 1 -- 172.21.15.12:0/1228551133 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fa6ac025020 con 0x5618567b7d60 2023-10-05T19:40:32.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.106+0000 7fa6b5903e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:32.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.106+0000 7fa6b5903e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:32.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.106+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567c0cc0 con 0x5618567827e0 2023-10-05T19:40:32.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.106+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fa6a4014040 con 0x5618567827e0 2023-10-05T19:40:32.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.106+0000 7fa6b5903e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:32.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.106+0000 7fa6b5903e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:32.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.106+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567c0cc0 con 0x5618567bc500 2023-10-05T19:40:32.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.106+0000 7fa6b2c37700 1 -- 172.21.15.12:0/1228551133 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fa6a8007a50 con 0x5618567bc500 2023-10-05T19:40:32.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.106+0000 7fa6b5903e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:32.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.106+0000 7fa6b5903e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:32.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.106+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567c1430 con 0x7fa6ac01ec10 2023-10-05T19:40:32.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.106+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7fa6a4014040 con 0x7fa6ac01ec10 2023-10-05T19:40:32.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.106+0000 7fa6b5903e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:32.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.106+0000 7fa6b5903e40 20 zone default found 2023-10-05T19:40:32.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.106+0000 7fa6b5903e40 4 Realm: () 2023-10-05T19:40:32.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.106+0000 7fa6b5903e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:32.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.106+0000 7fa6b5903e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:32.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.106+0000 7fa6b5903e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:32.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.106+0000 7fa6b5903e40 20 zonegroup default 2023-10-05T19:40:32.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.106+0000 7fa6b5903e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:32.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.106+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567c1170 con 0x7fa6ac01ec10 2023-10-05T19:40:32.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.110+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fa6a4014040 con 0x7fa6ac01ec10 2023-10-05T19:40:32.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.110+0000 7fa6b5903e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:32.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.110+0000 7fa6b5903e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:32.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.110+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567c2af0 con 0x5618567bc500 2023-10-05T19:40:32.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.110+0000 7fa6b2c37700 1 -- 172.21.15.12:0/1228551133 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fa6a8007a50 con 0x5618567bc500 2023-10-05T19:40:32.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.110+0000 7fa6b5903e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:32.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.110+0000 7fa6b5903e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:32.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.110+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567c2ea0 con 0x5618567bc500 2023-10-05T19:40:32.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.110+0000 7fa6b2c37700 1 -- 172.21.15.12:0/1228551133 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fa6a8007a50 con 0x5618567bc500 2023-10-05T19:40:32.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.110+0000 7fa6b5903e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:32.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.110+0000 7fa6b5903e40 20 started sync module instance, tier type = 2023-10-05T19:40:32.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.110+0000 7fa6b5903e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:32.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.110+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567c21b0 con 0x5618567b7d60 2023-10-05T19:40:32.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.114+0000 7fa6b3438700 1 -- 172.21.15.12:0/1228551133 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(84 notify.0 [create] v28'186 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6ac025020 con 0x5618567b7d60 2023-10-05T19:40:32.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.114+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94662530180528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567c08f0 con 0x5618567b7d60 2023-10-05T19:40:32.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.114+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567c3c70 con 0x5618567827e0 2023-10-05T19:40:32.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.118+0000 7fa6b3438700 1 -- 172.21.15.12:0/1228551133 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(85 notify.0 [watch watch cookie 94662530180528] v28'187 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6ac025020 con 0x5618567b7d60 2023-10-05T19:40:32.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.118+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(86 notify.1 [create] v28'186 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6a4014040 con 0x5618567827e0 2023-10-05T19:40:32.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.118+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94662530183760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567c4070 con 0x5618567827e0 2023-10-05T19:40:32.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.118+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567c4910 con 0x5618567bc500 2023-10-05T19:40:32.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.118+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(87 notify.1 [watch watch cookie 94662530183760] v28'187 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6a4014040 con 0x5618567827e0 2023-10-05T19:40:32.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.122+0000 7fa6b2c37700 1 -- 172.21.15.12:0/1228551133 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(88 notify.2 [create] v28'186 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6a8007a50 con 0x5618567bc500 2023-10-05T19:40:32.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.122+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94662530187376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567c44c0 con 0x5618567bc500 2023-10-05T19:40:32.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.122+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567c56f0 con 0x5618567b7d60 2023-10-05T19:40:32.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.122+0000 7fa6b2c37700 1 -- 172.21.15.12:0/1228551133 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(89 notify.2 [watch watch cookie 94662530187376] v28'187 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6a8007a50 con 0x5618567bc500 2023-10-05T19:40:32.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.122+0000 7fa6b3438700 1 -- 172.21.15.12:0/1228551133 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(90 notify.3 [create] v28'186 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6ac025020 con 0x5618567b7d60 2023-10-05T19:40:32.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.122+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94662530190608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567c5650 con 0x5618567b7d60 2023-10-05T19:40:32.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.122+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567c61b0 con 0x5618567827e0 2023-10-05T19:40:32.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.126+0000 7fa6b3438700 1 -- 172.21.15.12:0/1228551133 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(91 notify.3 [watch watch cookie 94662530190608] v28'187 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6ac025020 con 0x5618567b7d60 2023-10-05T19:40:32.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.126+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(92 notify.4 [create] v28'186 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6a4014040 con 0x5618567827e0 2023-10-05T19:40:32.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.126+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94662530194560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567c5650 con 0x5618567827e0 2023-10-05T19:40:32.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.126+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567c6c10 con 0x7fa6ac01ec10 2023-10-05T19:40:32.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.130+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(93 notify.4 [watch watch cookie 94662530194560] v28'187 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6a4014040 con 0x5618567827e0 2023-10-05T19:40:32.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.130+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(94 notify.5 [create] v28'186 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6a4014040 con 0x7fa6ac01ec10 2023-10-05T19:40:32.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.130+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94662530192272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567c6c10 con 0x7fa6ac01ec10 2023-10-05T19:40:32.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.130+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567c7620 con 0x7fa6ac01ec10 2023-10-05T19:40:32.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.130+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(95 notify.5 [watch watch cookie 94662530192272] v28'187 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6a4014040 con 0x7fa6ac01ec10 2023-10-05T19:40:32.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.130+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(96 notify.6 [create] v28'186 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6a4014040 con 0x7fa6ac01ec10 2023-10-05T19:40:32.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.130+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94662530197568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567c7620 con 0x7fa6ac01ec10 2023-10-05T19:40:32.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.130+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567c8030 con 0x7fa6ac01ec10 2023-10-05T19:40:32.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.134+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(98 notify.7 [create] v28'186 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6a4014040 con 0x7fa6ac01ec10 2023-10-05T19:40:32.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.134+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94662530202432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567c8480 con 0x7fa6ac01ec10 2023-10-05T19:40:32.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.134+0000 7fa6b5903e40 20 add_watcher() i=0 2023-10-05T19:40:32.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.134+0000 7fa6b5903e40 20 add_watcher() i=1 2023-10-05T19:40:32.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.134+0000 7fa6b5903e40 20 add_watcher() i=2 2023-10-05T19:40:32.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.134+0000 7fa6b5903e40 20 add_watcher() i=3 2023-10-05T19:40:32.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.134+0000 7fa6b5903e40 20 add_watcher() i=4 2023-10-05T19:40:32.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.134+0000 7fa6b5903e40 20 add_watcher() i=5 2023-10-05T19:40:32.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.134+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(97 notify.6 [watch watch cookie 94662530197568] v28'187 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6a4014040 con 0x7fa6ac01ec10 2023-10-05T19:40:32.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.134+0000 7fa6b5903e40 20 add_watcher() i=6 2023-10-05T19:40:32.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.138+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(99 notify.7 [watch watch cookie 94662530202432] v28'187 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6a4014040 con 0x7fa6ac01ec10 2023-10-05T19:40:32.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.138+0000 7fa6b5903e40 20 add_watcher() i=7 2023-10-05T19:40:32.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.138+0000 7fa6b5903e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:32.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.138+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567cb4b0 con 0x5618567827e0 2023-10-05T19:40:32.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.138+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fa6a4014040 con 0x5618567827e0 2023-10-05T19:40:32.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.138+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94662530203536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567c88d0 con 0x5618567827e0 2023-10-05T19:40:32.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.138+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94662530203536] v28'171 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fa6a4014040 con 0x5618567827e0 2023-10-05T19:40:32.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.138+0000 7fa5fffff700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:32.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.142+0000 7fa6b5903e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:32.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.142+0000 7fa6b5903e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:32.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.142+0000 7fa6b5903e40 1 Processor -- start 2023-10-05T19:40:32.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.142+0000 7fa6b5903e40 1 -- start start 2023-10-05T19:40:32.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.142+0000 7fa6b5903e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5618567d5940 0x5618567d5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:32.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.142+0000 7fa6b5903e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618567d6250 con 0x5618567d5940 2023-10-05T19:40:32.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.142+0000 7fa6b2c37700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5618567d5940 0x5618567d5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:32.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.142+0000 7fa6b2c37700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5618567d5940 0x5618567d5d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:59026/0 (socket says 172.21.15.12:59026) 2023-10-05T19:40:32.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.142+0000 7fa6b2c37700 1 -- 172.21.15.12:0/2483547494 learned_addr learned my addr 172.21.15.12:0/2483547494 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:32.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.142+0000 7fa6b2c37700 1 -- 172.21.15.12:0/2483547494 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618567d6390 con 0x5618567d5940 2023-10-05T19:40:32.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.142+0000 7fa6b2c37700 1 --2- 172.21.15.12:0/2483547494 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5618567d5940 0x5618567d5d10 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x561856781ca0 tx=0x7fa6a8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c2ca360dcb6fed server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:32.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.142+0000 7fa5ff7fe700 1 -- 172.21.15.12:0/2483547494 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6a8025040 con 0x5618567d5940 2023-10-05T19:40:32.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.142+0000 7fa5ff7fe700 1 -- 172.21.15.12:0/2483547494 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6a801f940 con 0x5618567d5940 2023-10-05T19:40:32.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.142+0000 7fa5ff7fe700 1 -- 172.21.15.12:0/2483547494 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6a801d890 con 0x5618567d5940 2023-10-05T19:40:32.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.142+0000 7fa6b5903e40 1 -- 172.21.15.12:0/2483547494 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5618567d5940 msgr2=0x5618567d5d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:32.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.142+0000 7fa6b5903e40 1 --2- 172.21.15.12:0/2483547494 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5618567d5940 0x5618567d5d10 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x561856781ca0 tx=0x7fa6a8011000 comp rx=0 tx=0).stop 2023-10-05T19:40:32.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.142+0000 7fa6b5903e40 1 -- 172.21.15.12:0/2483547494 shutdown_connections 2023-10-05T19:40:32.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.142+0000 7fa6b5903e40 1 --2- 172.21.15.12:0/2483547494 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5618567d5940 0x5618567d5d10 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:32.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.142+0000 7fa6b5903e40 1 -- 172.21.15.12:0/2483547494 >> 172.21.15.12:0/2483547494 conn(0x5618567d0d00 msgr2=0x5618567cee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:32.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.142+0000 7fa6b5903e40 1 -- 172.21.15.12:0/2483547494 shutdown_connections 2023-10-05T19:40:32.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.142+0000 7fa6b5903e40 1 -- 172.21.15.12:0/2483547494 wait complete. 2023-10-05T19:40:32.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.146+0000 7fa6b5903e40 1 Processor -- start 2023-10-05T19:40:32.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.146+0000 7fa6b5903e40 1 -- start start 2023-10-05T19:40:32.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.146+0000 7fa6b5903e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5618567d5940 0x5618567db630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:32.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.146+0000 7fa6b5903e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618567dc630 con 0x5618567d5940 2023-10-05T19:40:32.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.146+0000 7fa6b2c37700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5618567d5940 0x5618567db630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:32.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.146+0000 7fa6b2c37700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5618567d5940 0x5618567db630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:59034/0 (socket says 172.21.15.12:59034) 2023-10-05T19:40:32.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.146+0000 7fa6b2c37700 1 -- 172.21.15.12:0/2780479019 learned_addr learned my addr 172.21.15.12:0/2780479019 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:32.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.146+0000 7fa6b2c37700 1 -- 172.21.15.12:0/2780479019 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561856781cd0 con 0x5618567d5940 2023-10-05T19:40:32.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.146+0000 7fa6b2c37700 1 --2- 172.21.15.12:0/2780479019 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5618567d5940 0x5618567db630 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa6a80054c0 tx=0x7fa6a803b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:32.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.146+0000 7fa5fdffb700 1 -- 172.21.15.12:0/2780479019 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6a804b070 con 0x5618567d5940 2023-10-05T19:40:32.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.146+0000 7fa5fdffb700 1 -- 172.21.15.12:0/2780479019 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6a8032bd0 con 0x5618567d5940 2023-10-05T19:40:32.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.146+0000 7fa6b5903e40 1 -- 172.21.15.12:0/2780479019 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5618567dbc30 con 0x5618567d5940 2023-10-05T19:40:32.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.146+0000 7fa5fdffb700 1 -- 172.21.15.12:0/2780479019 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6a8025040 con 0x5618567d5940 2023-10-05T19:40:32.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.146+0000 7fa6b5903e40 1 -- 172.21.15.12:0/2780479019 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5618567dc070 con 0x5618567d5940 2023-10-05T19:40:32.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.146+0000 7fa5fdffb700 1 -- 172.21.15.12:0/2780479019 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fa6a8050410 con 0x5618567d5940 2023-10-05T19:40:32.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.150+0000 7fa5fdffb700 1 --2- 172.21.15.12:0/2780479019 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fa5f0030c50 0x7fa5f0033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:32.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.150+0000 7fa5fdffb700 1 -- 172.21.15.12:0/2780479019 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5702+0+0 (secure 0 0 0) 0x7fa6a8052720 con 0x5618567d5940 2023-10-05T19:40:32.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.150+0000 7fa6b5903e40 5 note: GC not initialized 2023-10-05T19:40:32.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.150+0000 7fa6b3c39700 1 --2- 172.21.15.12:0/2780479019 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fa5f0030c50 0x7fa5f0033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:32.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.150+0000 7fa6b3c39700 1 --2- 172.21.15.12:0/2780479019 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fa5f0030c50 0x7fa5f0033100 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x5618567c74a0 tx=0x7fa6a400ab80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:32.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.150+0000 7fa5f6ffd700 20 reqs_thread_entry: start 2023-10-05T19:40:32.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.150+0000 7fa5f4ff9700 20 reqs_thread_entry: start 2023-10-05T19:40:32.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.150+0000 7fa6b5903e40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:32.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.150+0000 7fa6b5903e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:32.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.154+0000 7fa5e2ffd700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:41:02 2023 2023-10-05T19:40:32.462 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:32.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.154+0000 7fa5e27fc700 20 reqs_thread_entry: start 2023-10-05T19:40:32.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.154+0000 7fa6b5903e40 20 get_system_obj_state: rctx=0x7fff82a18098 obj=default.rgw.meta:users.uid:foo state=0x56185665b0b0 s->prefetch_data=0 2023-10-05T19:40:32.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.154+0000 7fa6b5903e40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2023-10-05T19:40:32.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.154+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561856843d40 con 0x5618567827e0 2023-10-05T19:40:32.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.154+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(102 foo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7fa6a4014040 con 0x5618567827e0 2023-10-05T19:40:32.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.154+0000 7fa6b5903e40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x16 2023-10-05T19:40:32.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.154+0000 7fa6b5903e40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:32.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.154+0000 7fa6b5903e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:32.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.154+0000 7fa6b5903e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:32.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.154+0000 7fa6b5903e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:32.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.154+0000 7fa6b5903e40 10 cache get: name=default.rgw.meta+users.uid+foo : type miss (requested=0x13, cached=0x16) 2023-10-05T19:40:32.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.154+0000 7fa6b5903e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:32.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.154+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618568467d0 con 0x5618567827e0 2023-10-05T19:40:32.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.154+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(103 foo [call,call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv12 ondisk = 0) v8 ==== 273+0+480 (crc 0 0 0) 0x7fa6a4014040 con 0x5618567827e0 2023-10-05T19:40:32.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.154+0000 7fa6b5903e40 20 rados_obj.operate() r=0 bl.length=357 2023-10-05T19:40:32.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.154+0000 7fa6b5903e40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2023-10-05T19:40:32.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.154+0000 7fa6b5903e40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:32.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.154+0000 7fa6b5903e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:32.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.154+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:104 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618568467d0 con 0x5618567b7d60 2023-10-05T19:40:32.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.154+0000 7fa6b3438700 1 -- 172.21.15.12:0/1228551133 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(104 foo.buckets [call out=183b] v0'0 uv26 ondisk = 0) v8 ==== 155+0+183 (crc 0 0 0) 0x7fa6ac025020 con 0x5618567b7d60 2023-10-05T19:40:32.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.154+0000 7fa6b5903e40 20 enabling bucket name=myfoo 2023-10-05T19:40:32.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.154+0000 7fa6b5903e40 20 get_system_obj_state: rctx=0x7fff82a17870 obj=default.rgw.meta:root:myfoo state=0x56185665b0b0 s->prefetch_data=0 2023-10-05T19:40:32.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.154+0000 7fa6b5903e40 10 cache get: name=default.rgw.meta+root+myfoo : miss 2023-10-05T19:40:32.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.154+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:105 8.9 8:91b71841:root::myfoo:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561856844fd0 con 0x5618567b7d60 2023-10-05T19:40:32.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.154+0000 7fa6b3438700 1 -- 172.21.15.12:0/1228551133 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(105 myfoo [getxattrs out=75b,stat out=16b] v0'0 uv27 ondisk = 0) v8 ==== 191+0+91 (crc 0 0 0) 0x7fa6ac025020 con 0x5618567b7d60 2023-10-05T19:40:32.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.154+0000 7fa6b5903e40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x6 2023-10-05T19:40:32.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.154+0000 7fa6b5903e40 10 adding default.rgw.meta+root+myfoo to cache LRU end 2023-10-05T19:40:32.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.154+0000 7fa6b5903e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:32.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.154+0000 7fa6b5903e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:32.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.154+0000 7fa6b5903e40 10 cache get: name=default.rgw.meta+root+myfoo : type miss (requested=0x1, cached=0x6) 2023-10-05T19:40:32.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.154+0000 7fa6b5903e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:32.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.154+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:106 8.9 8:91b71841:root::myfoo:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561856844fd0 con 0x5618567b7d60 2023-10-05T19:40:32.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.158+0000 7fa6b3438700 1 -- 172.21.15.12:0/1228551133 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(106 myfoo [read 0~165 out=165b] v0'0 uv27 ondisk = 0) v8 ==== 149+0+165 (crc 0 0 0) 0x7fa6ac025020 con 0x5618567b7d60 2023-10-05T19:40:32.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.158+0000 7fa6b5903e40 20 rados_obj.operate() r=0 bl.length=165 2023-10-05T19:40:32.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.158+0000 7fa6b5903e40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x1 2023-10-05T19:40:32.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.158+0000 7fa6b5903e40 10 moving default.rgw.meta+root+myfoo to cache LRU end 2023-10-05T19:40:32.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.158+0000 7fa6b5903e40 20 get_system_obj_state: rctx=0x7fff82a17870 obj=default.rgw.meta:root:.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 state=0x5618566312b0 s->prefetch_data=0 2023-10-05T19:40:32.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.158+0000 7fa6b5903e40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 : miss 2023-10-05T19:40:32.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.158+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:107 8.e 8:70176bbd:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618568474f0 con 0x7fa6ac01ec10 2023-10-05T19:40:32.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.158+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(107 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 [call out=48b,getxattrs out=218b,stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 290+0+282 (crc 0 0 0) 0x7fa6a4014040 con 0x7fa6ac01ec10 2023-10-05T19:40:32.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.158+0000 7fa6b5903e40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 info.flags=0x16 2023-10-05T19:40:32.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.158+0000 7fa6b5903e40 10 adding default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 to cache LRU end 2023-10-05T19:40:32.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.158+0000 7fa6b5903e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:32.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.158+0000 7fa6b5903e40 10 updating xattr: name=user.rgw.acl bl.length()=123 2023-10-05T19:40:32.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.158+0000 7fa6b5903e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:32.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.158+0000 7fa6b5903e40 20 Read xattr: user.rgw.acl 2023-10-05T19:40:32.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.158+0000 7fa6b5903e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:32.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.158+0000 7fa6b5903e40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 : type miss (requested=0x13, cached=0x16) 2023-10-05T19:40:32.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.158+0000 7fa6b5903e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:32.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.158+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:108 8.e 8:70176bbd:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561856849cf0 con 0x7fa6ac01ec10 2023-10-05T19:40:32.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.158+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(108 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 [call,call out=48b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv8 ondisk = 0) v8 ==== 332+0+618 (crc 0 0 0) 0x7fa6a4014040 con 0x7fa6ac01ec10 2023-10-05T19:40:32.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.158+0000 7fa6b5903e40 20 rados_obj.operate() r=0 bl.length=352 2023-10-05T19:40:32.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.158+0000 7fa6b5903e40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 info.flags=0x13 2023-10-05T19:40:32.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.158+0000 7fa6b5903e40 10 moving default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 to cache LRU end 2023-10-05T19:40:32.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.158+0000 7fa6b5903e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:32.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.158+0000 7fa6b5903e40 10 updating xattr: name=user.rgw.acl bl.length()=123 2023-10-05T19:40:32.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.158+0000 7fa6b5903e40 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 2023-10-05T19:40:32.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.158+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:109 8.e 8:70176bbd:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~352 in=352b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x56185684f470 con 0x7fa6ac01ec10 2023-10-05T19:40:32.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.162+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(109 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 [delete,create,call,call,writefull 0~352,setxattr (123)] v28'9 uv9 ondisk = 0) v8 ==== 416+0+0 (crc 0 0 0) 0x7fa6a4014040 con 0x7fa6ac01ec10 2023-10-05T19:40:32.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.162+0000 7fa6b5903e40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 info.flags=0x17 2023-10-05T19:40:32.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.162+0000 7fa6b5903e40 10 moving default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 to cache LRU end 2023-10-05T19:40:32.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.162+0000 7fa6b5903e40 10 updating xattr: name=user.rgw.acl bl.length()=123 2023-10-05T19:40:32.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.162+0000 7fa6b5903e40 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1, ofs0, ns] 2023-10-05T19:40:32.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.162+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:110 7.e 7:7759931f:::notify.3:head [notify cookie 94662530726704 in=737b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x56185684b430 con 0x5618567b7d60 2023-10-05T19:40:32.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.162+0000 7fa6b3438700 1 -- 172.21.15.12:0/1228551133 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== watch-notify(notify (1) cookie 94662530190608 notify 120259084308 ret 0) v3 ==== 767+0+0 (crc 0 0 0) 0x5618567aa690 con 0x5618567b7d60 2023-10-05T19:40:32.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.162+0000 7fa6b1434700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084308 cookie 94662530190608 notifier 4597 bl.length()=725 2023-10-05T19:40:32.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.162+0000 7fa6b1434700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 info.flags=0x17 2023-10-05T19:40:32.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.162+0000 7fa6b1434700 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1 to cache LRU end 2023-10-05T19:40:32.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.162+0000 7fa6b1434700 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2023-10-05T19:40:32.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.162+0000 7fa6b1434700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:111 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fa690003d90 con 0x5618567b7d60 2023-10-05T19:40:32.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.166+0000 7fa6b3438700 1 -- 172.21.15.12:0/1228551133 <== osd.2 v2:172.21.15.12:6800/29934 23 ==== osd_op_reply(110 notify.3 [notify cookie 94662530726704 out=8b] v0'0 uv186 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fa6ac025020 con 0x5618567b7d60 2023-10-05T19:40:32.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.166+0000 7fa6b3438700 1 -- 172.21.15.12:0/1228551133 <== osd.2 v2:172.21.15.12:6800/29934 24 ==== osd_op_reply(111 notify.3 [notify-ack] v0'0 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6ac025020 con 0x5618567b7d60 2023-10-05T19:40:32.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.166+0000 7fa6b3438700 1 -- 172.21.15.12:0/1228551133 <== osd.2 v2:172.21.15.12:6800/29934 25 ==== watch-notify(notify_complete (2) cookie 94662530726704 notify 120259084308 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7fa6ac02e070 con 0x5618567b7d60 2023-10-05T19:40:32.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.166+0000 7fa6b5903e40 20 handle_bi_update(): bucket=:myfoo[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1]): orig_sources= new_sources= 2023-10-05T19:40:32.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.166+0000 7fa6b5903e40 20 handle_bi_update(): bucket=:myfoo[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1]): potential sources added=[] removed=[] 2023-10-05T19:40:32.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.166+0000 7fa6b5903e40 20 handle_bi_update(): bucket=:myfoo[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1]): orig_dests= new_dests= 2023-10-05T19:40:32.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.166+0000 7fa6b5903e40 20 handle_bi_update(): bucket=:myfoo[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1]): potential dests added=[] removed=[] 2023-10-05T19:40:32.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.166+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:112 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.set in=58b,writefull 0~357 in=357b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x561856847ac0 con 0x5618567827e0 2023-10-05T19:40:32.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.170+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(112 foo [delete,create,call,writefull 0~357] v28'13 uv13 ondisk = 0) v8 ==== 273+0+0 (crc 0 0 0) 0x7fa6a4014040 con 0x5618567827e0 2023-10-05T19:40:32.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.170+0000 7fa6b5903e40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2023-10-05T19:40:32.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.170+0000 7fa6b5903e40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:32.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.170+0000 7fa6b5903e40 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2023-10-05T19:40:32.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.170+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:113 7.d 7:b4812045:::notify.4:head [notify cookie 94662530726592 in=545b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x561856849410 con 0x5618567827e0 2023-10-05T19:40:32.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.170+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== watch-notify(notify (1) cookie 94662530194560 notify 120259084303 ret 0) v3 ==== 575+0+0 (crc 0 0 0) 0x7fa6a4033630 con 0x5618567827e0 2023-10-05T19:40:32.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.170+0000 7fa6b0c33700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 120259084303 cookie 94662530194560 notifier 4597 bl.length()=533 2023-10-05T19:40:32.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.170+0000 7fa6b0c33700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2023-10-05T19:40:32.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.170+0000 7fa6b0c33700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:32.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.170+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(113 notify.4 [notify cookie 94662530726592 out=8b] v0'0 uv186 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fa6a4014040 con 0x5618567827e0 2023-10-05T19:40:32.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.170+0000 7fa6b0c33700 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:114 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28) v8 -- 0x7fa604003440 con 0x5618567827e0 2023-10-05T19:40:32.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.170+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== osd_op_reply(114 notify.4 [notify-ack] v0'0 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6a4014040 con 0x5618567827e0 2023-10-05T19:40:32.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.170+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.1 v2:172.21.15.12:6808/29933 39 ==== watch-notify(notify_complete (2) cookie 94662530726592 notify 120259084303 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7fa6a40330d0 con 0x5618567827e0 2023-10-05T19:40:32.470 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:40:32.471 INFO:teuthology.orchestra.run.smithi012.stdout: "user_id": "foo", 2023-10-05T19:40:32.471 INFO:teuthology.orchestra.run.smithi012.stdout: "display_name": "Foo", 2023-10-05T19:40:32.471 INFO:teuthology.orchestra.run.smithi012.stdout: "email": "foo@foo.com", 2023-10-05T19:40:32.471 INFO:teuthology.orchestra.run.smithi012.stdout: "suspended": 0, 2023-10-05T19:40:32.471 INFO:teuthology.orchestra.run.smithi012.stdout: "max_buckets": 4, 2023-10-05T19:40:32.471 INFO:teuthology.orchestra.run.smithi012.stdout: "subusers": [], 2023-10-05T19:40:32.471 INFO:teuthology.orchestra.run.smithi012.stdout: "keys": [ 2023-10-05T19:40:32.471 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:32.471 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:40:32.471 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2023-10-05T19:40:32.471 INFO:teuthology.orchestra.run.smithi012.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2023-10-05T19:40:32.472 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:32.472 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:32.472 INFO:teuthology.orchestra.run.smithi012.stdout: "swift_keys": [], 2023-10-05T19:40:32.472 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": [], 2023-10-05T19:40:32.472 INFO:teuthology.orchestra.run.smithi012.stdout: "op_mask": "read, write, delete", 2023-10-05T19:40:32.472 INFO:teuthology.orchestra.run.smithi012.stdout: "default_placement": "", 2023-10-05T19:40:32.472 INFO:teuthology.orchestra.run.smithi012.stdout: "default_storage_class": "", 2023-10-05T19:40:32.472 INFO:teuthology.orchestra.run.smithi012.stdout: "placement_tags": [], 2023-10-05T19:40:32.472 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket_quota": { 2023-10-05T19:40:32.472 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:32.472 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:32.472 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:32.472 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:32.473 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:32.473 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:32.473 INFO:teuthology.orchestra.run.smithi012.stdout: "user_quota": { 2023-10-05T19:40:32.473 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:32.473 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:32.473 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:32.473 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:32.473 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:32.473 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:32.473 INFO:teuthology.orchestra.run.smithi012.stdout: "temp_url_keys": [], 2023-10-05T19:40:32.473 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "rgw", 2023-10-05T19:40:32.473 INFO:teuthology.orchestra.run.smithi012.stdout: "mfa_ids": [] 2023-10-05T19:40:32.474 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:40:32.474 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:40:32.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.170+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:115 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94662530180528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567998a0 con 0x5618567b7d60 2023-10-05T19:40:32.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.174+0000 7fa6b3438700 1 -- 172.21.15.12:0/1228551133 <== osd.2 v2:172.21.15.12:6800/29934 26 ==== osd_op_reply(115 notify.0 [watch unwatch cookie 94662530180528] v28'188 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6ac025020 con 0x5618567b7d60 2023-10-05T19:40:32.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.174+0000 7fa6b5903e40 20 remove_watcher() i=0 2023-10-05T19:40:32.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.174+0000 7fa6b5903e40 2 removed watcher, disabling cache 2023-10-05T19:40:32.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.174+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:116 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94662530183760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567998a0 con 0x5618567827e0 2023-10-05T19:40:32.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.178+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.1 v2:172.21.15.12:6808/29933 40 ==== osd_op_reply(116 notify.1 [watch unwatch cookie 94662530183760] v28'188 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6a4014040 con 0x5618567827e0 2023-10-05T19:40:32.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.178+0000 7fa6b5903e40 20 remove_watcher() i=1 2023-10-05T19:40:32.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.178+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:117 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94662530187376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567998a0 con 0x5618567bc500 2023-10-05T19:40:32.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.178+0000 7fa6b2c37700 1 -- 172.21.15.12:0/1228551133 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(117 notify.2 [watch unwatch cookie 94662530187376] v28'188 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6a8007a50 con 0x5618567bc500 2023-10-05T19:40:32.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.178+0000 7fa6b5903e40 20 remove_watcher() i=2 2023-10-05T19:40:32.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.178+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:118 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94662530190608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567998a0 con 0x5618567b7d60 2023-10-05T19:40:32.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.182+0000 7fa6b3438700 1 -- 172.21.15.12:0/1228551133 <== osd.2 v2:172.21.15.12:6800/29934 27 ==== osd_op_reply(118 notify.3 [watch unwatch cookie 94662530190608] v28'188 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6ac025020 con 0x5618567b7d60 2023-10-05T19:40:32.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.182+0000 7fa6b5903e40 20 remove_watcher() i=3 2023-10-05T19:40:32.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.182+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:119 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94662530194560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567998a0 con 0x5618567827e0 2023-10-05T19:40:32.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.186+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.1 v2:172.21.15.12:6808/29933 41 ==== osd_op_reply(119 notify.4 [watch unwatch cookie 94662530194560] v28'188 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6a4014040 con 0x5618567827e0 2023-10-05T19:40:32.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.186+0000 7fa6b5903e40 20 remove_watcher() i=4 2023-10-05T19:40:32.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.186+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:120 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94662530192272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567998a0 con 0x7fa6ac01ec10 2023-10-05T19:40:32.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.186+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.0 v2:172.21.15.8:6800/27563 30 ==== osd_op_reply(120 notify.5 [watch unwatch cookie 94662530192272] v28'188 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6a4014040 con 0x7fa6ac01ec10 2023-10-05T19:40:32.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.186+0000 7fa6b5903e40 20 remove_watcher() i=5 2023-10-05T19:40:32.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.186+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:121 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94662530197568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567998a0 con 0x7fa6ac01ec10 2023-10-05T19:40:32.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.190+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.0 v2:172.21.15.8:6800/27563 31 ==== osd_op_reply(121 notify.6 [watch unwatch cookie 94662530197568] v28'188 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6a4014040 con 0x7fa6ac01ec10 2023-10-05T19:40:32.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.190+0000 7fa6b5903e40 20 remove_watcher() i=6 2023-10-05T19:40:32.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.190+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:122 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94662530202432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567998a0 con 0x7fa6ac01ec10 2023-10-05T19:40:32.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.194+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.0 v2:172.21.15.8:6800/27563 32 ==== osd_op_reply(122 notify.7 [watch unwatch cookie 94662530202432] v28'188 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6a4014040 con 0x7fa6ac01ec10 2023-10-05T19:40:32.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.194+0000 7fa6b5903e40 20 remove_watcher() i=7 2023-10-05T19:40:32.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.194+0000 7fa5fffff700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:32.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.194+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:123 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94662530203536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28) v8 -- 0x5618567998a0 con 0x5618567827e0 2023-10-05T19:40:32.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.198+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 <== osd.1 v2:172.21.15.12:6808/29933 42 ==== osd_op_reply(123 data_loggenerations_metadata [watch unwatch cookie 94662530203536] v28'172 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fa6a4014040 con 0x5618567827e0 2023-10-05T19:40:32.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.198+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fa6ac01ec10 msgr2=0x7fa6ac0113e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:32.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.198+0000 7fa6b5903e40 1 --2- 172.21.15.12:0/1228551133 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fa6ac01ec10 0x7fa6ac0113e0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:32.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.198+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5618567827e0 msgr2=0x5618567b57b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:32.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.198+0000 7fa6b5903e40 1 --2- 172.21.15.12:0/1228551133 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5618567827e0 0x5618567b57b0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:32.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.198+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5618567b7d60 msgr2=0x5618567ba140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:32.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.198+0000 7fa6b5903e40 1 --2- 172.21.15.12:0/1228551133 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5618567b7d60 0x5618567ba140 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:32.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.198+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5618567bc500 msgr2=0x5618567be910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:32.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.198+0000 7fa6b5903e40 1 --2- 172.21.15.12:0/1228551133 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5618567bc500 0x5618567be910 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:32.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.198+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fa698030c50 msgr2=0x7fa698033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:32.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.198+0000 7fa6b5903e40 1 --2- 172.21.15.12:0/1228551133 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fa698030c50 0x7fa698033100 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa6a800afd0 tx=0x7fa6a8014000 comp rx=0 tx=0).stop 2023-10-05T19:40:32.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.198+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561856782410 msgr2=0x5618567a9d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:32.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.198+0000 7fa6b3c39700 1 -- 172.21.15.12:0/1228551133 reap_dead start 2023-10-05T19:40:32.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.198+0000 7fa6b5903e40 1 --2- 172.21.15.12:0/1228551133 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561856782410 0x5618567a9d10 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x56185679f660 tx=0x7fa6ac014000 comp rx=0 tx=0).stop 2023-10-05T19:40:32.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.198+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 shutdown_connections 2023-10-05T19:40:32.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.198+0000 7fa6b5903e40 1 --2- 172.21.15.12:0/1228551133 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561856782410 0x5618567a9d10 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:32.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.198+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 >> 172.21.15.12:0/1228551133 conn(0x56185677c720 msgr2=0x56185679d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:32.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.198+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 shutdown_connections 2023-10-05T19:40:32.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.198+0000 7fa6b5903e40 1 -- 172.21.15.12:0/1228551133 wait complete. 2023-10-05T19:40:32.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.198+0000 7fa6b5903e40 1 -- 172.21.15.12:0/2780479019 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fa5f0030c50 msgr2=0x7fa5f0033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:32.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.198+0000 7fa6b5903e40 1 --2- 172.21.15.12:0/2780479019 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fa5f0030c50 0x7fa5f0033100 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x5618567c74a0 tx=0x7fa6a400ab80 comp rx=0 tx=0).stop 2023-10-05T19:40:32.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.198+0000 7fa6b5903e40 1 -- 172.21.15.12:0/2780479019 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5618567d5940 msgr2=0x5618567db630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:32.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.198+0000 7fa6b5903e40 1 --2- 172.21.15.12:0/2780479019 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5618567d5940 0x5618567db630 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa6a80054c0 tx=0x7fa6a803b000 comp rx=0 tx=0).stop 2023-10-05T19:40:32.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.198+0000 7fa6b5903e40 1 -- 172.21.15.12:0/2780479019 shutdown_connections 2023-10-05T19:40:32.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.198+0000 7fa6b5903e40 1 --2- 172.21.15.12:0/2780479019 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fa5f0030c50 0x7fa5f0033100 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:32.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.198+0000 7fa6b5903e40 1 --2- 172.21.15.12:0/2780479019 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5618567d5940 0x5618567db630 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:32.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.198+0000 7fa6b5903e40 1 -- 172.21.15.12:0/2780479019 >> 172.21.15.12:0/2780479019 conn(0x5618567d0d00 msgr2=0x5618567ceda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:32.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.198+0000 7fa6b5903e40 1 -- 172.21.15.12:0/2780479019 shutdown_connections 2023-10-05T19:40:32.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:32.198+0000 7fa6b5903e40 1 -- 172.21.15.12:0/2780479019 wait complete. 2023-10-05T19:40:32.483 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:40:32.496 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=3 bytes_in=0 failed=False 2023-10-05T19:40:32.766 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=30000000 bytes_in=0 failed=False 2023-10-05T19:40:32.780 INFO:tasks.radosgw_admin:RL: delete_obj myfoo foo bytes_out=0 bytes_in=0 failed=False 2023-10-05T19:40:47.781 INFO:tasks.util.rgw:rgwadmin: client.0 : ['gc', 'list', '--include-all'] 2023-10-05T19:40:47.781 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'gc', 'list', '--include-all'] 2023-10-05T19:40:47.782 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph gc list --include-all 2023-10-05T19:40:47.822 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:47.822 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:47.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.815+0000 7f269912fe40 1 Processor -- start 2023-10-05T19:40:47.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.815+0000 7f269912fe40 1 -- start start 2023-10-05T19:40:47.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.815+0000 7f269912fe40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x563738529910 0x5637385bdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:47.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.815+0000 7f269912fe40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x563738501bd0 con 0x56373852ac20 2023-10-05T19:40:47.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.815+0000 7f269912fe40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x5637384beee0 con 0x563738529910 2023-10-05T19:40:47.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.815+0000 7f2696463700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x563738529910 0x5637385bdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:47.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.815+0000 7f2696c64700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x56373852ac20 0x563738529200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:60186/0 (socket says 172.21.15.12:60186) 2023-10-05T19:40:47.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.815+0000 7f2696463700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x563738529910 0x5637385bdf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58304/0 (socket says 172.21.15.12:58304) 2023-10-05T19:40:47.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.815+0000 7f2696c64700 1 -- 172.21.15.12:0/2338894782 learned_addr learned my addr 172.21.15.12:0/2338894782 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:47.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.815+0000 7f2696463700 1 -- 172.21.15.12:0/2338894782 >> v1:172.21.15.8:6789/0 conn(0x56373852ac20 legacy=0x563738529200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:47.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.815+0000 7f2695c62700 1 -- 172.21.15.12:0/2338894782 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1127275042 0 0) 0x563738501bd0 con 0x56373852ac20 2023-10-05T19:40:47.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.815+0000 7f2696463700 1 -- 172.21.15.12:0/2338894782 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56373852b0b0 con 0x563738529910 2023-10-05T19:40:47.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.815+0000 7f2696463700 1 --2- 172.21.15.12:0/2338894782 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563738529910 0x5637385bdf60 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f268c00a5d0 tx=0x7f268c005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65fe0d45fc3c5919 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:47.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.815+0000 7f2695c62700 1 -- 172.21.15.12:0/2338894782 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f268c0086a0 con 0x563738529910 2023-10-05T19:40:47.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.819+0000 7f2695c62700 1 -- 172.21.15.12:0/2338894782 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f268c008800 con 0x563738529910 2023-10-05T19:40:47.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.819+0000 7f2695c62700 1 -- 172.21.15.12:0/2338894782 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f268c008ad0 con 0x563738529910 2023-10-05T19:40:47.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.819+0000 7f269912fe40 1 -- 172.21.15.12:0/2338894782 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563738529910 msgr2=0x5637385bdf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:47.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.819+0000 7f269912fe40 1 --2- 172.21.15.12:0/2338894782 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563738529910 0x5637385bdf60 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f268c00a5d0 tx=0x7f268c005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:47.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.819+0000 7f269912fe40 1 -- 172.21.15.12:0/2338894782 shutdown_connections 2023-10-05T19:40:47.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.819+0000 7f269912fe40 1 --2- 172.21.15.12:0/2338894782 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563738529910 0x5637385bdf60 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:47.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.819+0000 7f269912fe40 1 -- 172.21.15.12:0/2338894782 >> 172.21.15.12:0/2338894782 conn(0x5637385b0850 msgr2=0x5637385b2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:47.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.819+0000 7f269912fe40 1 -- 172.21.15.12:0/2338894782 shutdown_connections 2023-10-05T19:40:47.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.819+0000 7f269912fe40 1 -- 172.21.15.12:0/2338894782 wait complete. 2023-10-05T19:40:47.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.819+0000 7f269912fe40 1 Processor -- start 2023-10-05T19:40:47.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.819+0000 7f269912fe40 1 -- start start 2023-10-05T19:40:47.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.819+0000 7f269912fe40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563738529910 0x5637385d5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:47.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.819+0000 7f269912fe40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x563738482e30 con 0x563738529910 2023-10-05T19:40:47.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.819+0000 7f2696c64700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563738529910 0x5637385d5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:47.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.819+0000 7f2696c64700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563738529910 0x5637385d5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58314/0 (socket says 172.21.15.12:58314) 2023-10-05T19:40:47.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.819+0000 7f2696c64700 1 -- 172.21.15.12:0/94821600 learned_addr learned my addr 172.21.15.12:0/94821600 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:47.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.819+0000 7f2696c64700 1 -- 172.21.15.12:0/94821600 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f268c005cf0 con 0x563738529910 2023-10-05T19:40:47.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.819+0000 7f2696c64700 1 --2- 172.21.15.12:0/94821600 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563738529910 0x5637385d5ec0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f2688001dc0 tx=0x7f2688001790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd392804920341cc server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:47.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.819+0000 7f2694c60700 1 -- 172.21.15.12:0/94821600 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f268800bce0 con 0x563738529910 2023-10-05T19:40:47.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.819+0000 7f2694c60700 1 -- 172.21.15.12:0/94821600 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f268800be40 con 0x563738529910 2023-10-05T19:40:47.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.819+0000 7f2694c60700 1 -- 172.21.15.12:0/94821600 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f268800e5d0 con 0x563738529910 2023-10-05T19:40:47.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.823+0000 7f269912fe40 1 -- 172.21.15.12:0/94821600 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563738529910 msgr2=0x5637385d5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:47.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.823+0000 7f269912fe40 1 --2- 172.21.15.12:0/94821600 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563738529910 0x5637385d5ec0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f2688001dc0 tx=0x7f2688001790 comp rx=0 tx=0).stop 2023-10-05T19:40:47.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.823+0000 7f269912fe40 1 -- 172.21.15.12:0/94821600 shutdown_connections 2023-10-05T19:40:47.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.823+0000 7f269912fe40 1 --2- 172.21.15.12:0/94821600 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563738529910 0x5637385d5ec0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:47.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.823+0000 7f269912fe40 1 -- 172.21.15.12:0/94821600 >> 172.21.15.12:0/94821600 conn(0x5637385b0850 msgr2=0x5637385d1a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:47.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.823+0000 7f269912fe40 1 -- 172.21.15.12:0/94821600 shutdown_connections 2023-10-05T19:40:47.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.823+0000 7f269912fe40 1 -- 172.21.15.12:0/94821600 wait complete. 2023-10-05T19:40:47.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.823+0000 7f269912fe40 1 Processor -- start 2023-10-05T19:40:47.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.823+0000 7f269912fe40 1 -- start start 2023-10-05T19:40:47.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.823+0000 7f269912fe40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563738529910 0x5637385dcca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:47.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.823+0000 7f269912fe40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x563738455ed0 con 0x563738529910 2023-10-05T19:40:47.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.823+0000 7f2696c64700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563738529910 0x5637385dcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:47.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.823+0000 7f2696c64700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563738529910 0x5637385dcca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58320/0 (socket says 172.21.15.12:58320) 2023-10-05T19:40:47.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.823+0000 7f2696c64700 1 -- 172.21.15.12:0/3953564550 learned_addr learned my addr 172.21.15.12:0/3953564550 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:47.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.823+0000 7f2696c64700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2688001a70 con 0x563738529910 2023-10-05T19:40:47.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.823+0000 7f2696c64700 1 --2- 172.21.15.12:0/3953564550 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563738529910 0x5637385dcca0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f26880017d0 tx=0x7f2688016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:47.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.823+0000 7f2686ffd700 1 -- 172.21.15.12:0/3953564550 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f268801f070 con 0x563738529910 2023-10-05T19:40:47.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.823+0000 7f2686ffd700 1 -- 172.21.15.12:0/3953564550 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f268801a4d0 con 0x563738529910 2023-10-05T19:40:47.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.823+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5637385dd1e0 con 0x563738529910 2023-10-05T19:40:47.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.823+0000 7f2686ffd700 1 -- 172.21.15.12:0/3953564550 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f268801a7a0 con 0x563738529910 2023-10-05T19:40:47.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.823+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5637385dd620 con 0x563738529910 2023-10-05T19:40:47.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.827+0000 7f269912fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:47.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.827+0000 7f2686ffd700 1 -- 172.21.15.12:0/3953564550 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f268801a900 con 0x563738529910 2023-10-05T19:40:47.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.827+0000 7f2686ffd700 1 --2- 172.21.15.12:0/3953564550 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f267c030c00 0x7f267c0330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:47.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.827+0000 7f2686ffd700 1 -- 172.21.15.12:0/3953564550 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7f26880452b0 con 0x563738529910 2023-10-05T19:40:47.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.827+0000 7f2696463700 1 --2- 172.21.15.12:0/3953564550 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f267c030c00 0x7f267c0330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:47.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.827+0000 7f269912fe40 1 --2- 172.21.15.12:0/3953564550 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x56373852ac20 0x5637385e8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:47.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.827+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385e8de0 con 0x56373852ac20 2023-10-05T19:40:47.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.827+0000 7f2697465700 1 --2- 172.21.15.12:0/3953564550 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x56373852ac20 0x5637385e8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:47.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.827+0000 7f2696463700 1 --2- 172.21.15.12:0/3953564550 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f267c030c00 0x7f267c0330b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f268c005a90 tx=0x7f268c015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:47.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.827+0000 7f2697465700 1 --2- 172.21.15.12:0/3953564550 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x56373852ac20 0x5637385e8730 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:47.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.827+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f269000a5a0 con 0x56373852ac20 2023-10-05T19:40:47.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.831+0000 7f269912fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:47.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.831+0000 7f269912fe40 20 realm 2023-10-05T19:40:47.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.831+0000 7f269912fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:47.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.831+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385e5d90 con 0x56373852ac20 2023-10-05T19:40:47.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.831+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f269000a5a0 con 0x56373852ac20 2023-10-05T19:40:47.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.831+0000 7f269912fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:47.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.831+0000 7f269912fe40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:47.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.831+0000 7f269912fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:47.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.831+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385e98c0 con 0x56373852ac20 2023-10-05T19:40:47.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.831+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f269000a5a0 con 0x56373852ac20 2023-10-05T19:40:47.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.831+0000 7f269912fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:47.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.831+0000 7f269912fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:47.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.831+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385e94a0 con 0x56373852ac20 2023-10-05T19:40:47.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.831+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f269000a5a0 con 0x56373852ac20 2023-10-05T19:40:47.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.831+0000 7f269912fe40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:47.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.831+0000 7f269912fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:47.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.831+0000 7f269912fe40 1 --2- 172.21.15.12:0/3953564550 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5637385ea850 0x5637385ecc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:47.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.831+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385ed3a0 con 0x5637385ea850 2023-10-05T19:40:47.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.831+0000 7f2696c64700 1 --2- 172.21.15.12:0/3953564550 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5637385ea850 0x5637385ecc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:47.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.831+0000 7f2696c64700 1 --2- 172.21.15.12:0/3953564550 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5637385ea850 0x5637385ecc30 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:47.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.831+0000 7f2696c64700 1 -- 172.21.15.12:0/3953564550 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f2688014070 con 0x5637385ea850 2023-10-05T19:40:47.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.831+0000 7f269912fe40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:47.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.831+0000 7f269912fe40 20 searching for the correct realm 2023-10-05T19:40:47.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.831+0000 7f269912fe40 1 --2- 172.21.15.12:0/3953564550 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5637385ef1d0 0x5637385f15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:47.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.831+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f1af0 con 0x5637385ef1d0 2023-10-05T19:40:47.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.835+0000 7f2696463700 1 --2- 172.21.15.12:0/3953564550 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5637385ef1d0 0x5637385f15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:47.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.835+0000 7f2696463700 1 --2- 172.21.15.12:0/3953564550 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5637385ef1d0 0x5637385f15b0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:47.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.835+0000 7f2696463700 1 -- 172.21.15.12:0/3953564550 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f268c012450 con 0x5637385ef1d0 2023-10-05T19:40:47.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.835+0000 7f2696463700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f1af0 con 0x5637385ea850 2023-10-05T19:40:47.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.839+0000 7f2696c64700 1 -- 172.21.15.12:0/3953564550 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2688014070 con 0x5637385ea850 2023-10-05T19:40:47.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.839+0000 7f2696c64700 1 --2- 172.21.15.12:0/3953564550 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f268802d020 0x7f2688027120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:47.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.839+0000 7f2696c64700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f1af0 con 0x7f268802d020 2023-10-05T19:40:47.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.839+0000 7f2697465700 1 --2- 172.21.15.12:0/3953564550 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f268802d020 0x7f2688027120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:47.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.839+0000 7f2697465700 1 --2- 172.21.15.12:0/3953564550 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f268802d020 0x7f2688027120 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:47.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.839+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f269000a5a0 con 0x7f268802d020 2023-10-05T19:40:47.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.839+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f1af0 con 0x56373852ac20 2023-10-05T19:40:47.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.839+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f269000a5a0 con 0x56373852ac20 2023-10-05T19:40:47.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.839+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f1af0 con 0x56373852ac20 2023-10-05T19:40:47.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.839+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f269000a5a0 con 0x56373852ac20 2023-10-05T19:40:47.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.839+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f1af0 con 0x56373852ac20 2023-10-05T19:40:47.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.843+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(11 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f269000a5a0 con 0x56373852ac20 2023-10-05T19:40:47.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.843+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f1af0 con 0x7f268802d020 2023-10-05T19:40:47.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.843+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f269000a5a0 con 0x7f268802d020 2023-10-05T19:40:47.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.843+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f1af0 con 0x5637385ea850 2023-10-05T19:40:47.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.843+0000 7f2696c64700 1 -- 172.21.15.12:0/3953564550 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2688014070 con 0x5637385ea850 2023-10-05T19:40:47.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.843+0000 7f2696c64700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f1af0 con 0x5637385ea850 2023-10-05T19:40:47.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.843+0000 7f2696c64700 1 -- 172.21.15.12:0/3953564550 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2688014070 con 0x5637385ea850 2023-10-05T19:40:47.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.843+0000 7f2696c64700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f1af0 con 0x56373852ac20 2023-10-05T19:40:47.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.843+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f269000a5a0 con 0x56373852ac20 2023-10-05T19:40:47.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.847+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f1af0 con 0x7f268802d020 2023-10-05T19:40:47.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.847+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f269000a5a0 con 0x7f268802d020 2023-10-05T19:40:47.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.847+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f1af0 con 0x56373852ac20 2023-10-05T19:40:47.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.847+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f269000a5a0 con 0x56373852ac20 2023-10-05T19:40:47.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.847+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f1af0 con 0x5637385ea850 2023-10-05T19:40:47.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.847+0000 7f2696c64700 1 -- 172.21.15.12:0/3953564550 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(18 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2688014070 con 0x5637385ea850 2023-10-05T19:40:47.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.847+0000 7f2696c64700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f1af0 con 0x5637385ef1d0 2023-10-05T19:40:47.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.847+0000 7f2696463700 1 -- 172.21.15.12:0/3953564550 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f268c012450 con 0x5637385ef1d0 2023-10-05T19:40:47.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.847+0000 7f2696463700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f1af0 con 0x5637385ef1d0 2023-10-05T19:40:47.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.851+0000 7f2696463700 1 -- 172.21.15.12:0/3953564550 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f268c012450 con 0x5637385ef1d0 2023-10-05T19:40:47.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.851+0000 7f2696463700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f1af0 con 0x7f268802d020 2023-10-05T19:40:47.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.851+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f269000a5a0 con 0x7f268802d020 2023-10-05T19:40:47.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.851+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f1af0 con 0x5637385ef1d0 2023-10-05T19:40:47.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.851+0000 7f2696463700 1 -- 172.21.15.12:0/3953564550 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f268c012450 con 0x5637385ef1d0 2023-10-05T19:40:47.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.851+0000 7f2696463700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f1af0 con 0x5637385ef1d0 2023-10-05T19:40:47.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.851+0000 7f2696463700 1 -- 172.21.15.12:0/3953564550 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f268c012450 con 0x5637385ef1d0 2023-10-05T19:40:47.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.851+0000 7f2696463700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f1af0 con 0x56373852ac20 2023-10-05T19:40:47.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.851+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f269000a5a0 con 0x56373852ac20 2023-10-05T19:40:47.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.851+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f1af0 con 0x56373852ac20 2023-10-05T19:40:47.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.855+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f269000a5a0 con 0x56373852ac20 2023-10-05T19:40:47.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.855+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f1af0 con 0x7f268802d020 2023-10-05T19:40:47.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.855+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f269000a5a0 con 0x7f268802d020 2023-10-05T19:40:47.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.855+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f1af0 con 0x5637385ef1d0 2023-10-05T19:40:47.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.855+0000 7f2696463700 1 -- 172.21.15.12:0/3953564550 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f268c012450 con 0x5637385ef1d0 2023-10-05T19:40:47.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.855+0000 7f2696463700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f1af0 con 0x5637385ea850 2023-10-05T19:40:47.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.855+0000 7f2696c64700 1 -- 172.21.15.12:0/3953564550 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2688014070 con 0x5637385ea850 2023-10-05T19:40:47.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.855+0000 7f2696c64700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f1af0 con 0x56373852ac20 2023-10-05T19:40:47.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.859+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f269000a5a0 con 0x56373852ac20 2023-10-05T19:40:47.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.859+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f1af0 con 0x7f268802d020 2023-10-05T19:40:47.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.859+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f269000a5a0 con 0x7f268802d020 2023-10-05T19:40:47.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.859+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f1af0 con 0x5637385ef1d0 2023-10-05T19:40:47.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.859+0000 7f2696463700 1 -- 172.21.15.12:0/3953564550 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f268c012450 con 0x5637385ef1d0 2023-10-05T19:40:47.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.859+0000 7f2696463700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f1af0 con 0x7f268802d020 2023-10-05T19:40:47.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.859+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f269000a5a0 con 0x7f268802d020 2023-10-05T19:40:47.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.859+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f1af0 con 0x5637385ea850 2023-10-05T19:40:47.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.863+0000 7f2696c64700 1 -- 172.21.15.12:0/3953564550 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2688014070 con 0x5637385ea850 2023-10-05T19:40:47.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.863+0000 7f2696c64700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f1af0 con 0x5637385ef1d0 2023-10-05T19:40:47.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.863+0000 7f2696463700 1 -- 172.21.15.12:0/3953564550 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f268c012450 con 0x5637385ef1d0 2023-10-05T19:40:47.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.863+0000 7f2696463700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f1af0 con 0x7f268802d020 2023-10-05T19:40:47.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.863+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f269000a5a0 con 0x7f268802d020 2023-10-05T19:40:47.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.863+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f1af0 con 0x56373852ac20 2023-10-05T19:40:47.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.863+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f269000a5a0 con 0x56373852ac20 2023-10-05T19:40:47.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.863+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f1af0 con 0x5637385ea850 2023-10-05T19:40:47.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.863+0000 7f2696c64700 1 -- 172.21.15.12:0/3953564550 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(37 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f2688014070 con 0x5637385ea850 2023-10-05T19:40:47.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.863+0000 7f269912fe40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:47.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.863+0000 7f269912fe40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:47.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.863+0000 7f269912fe40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:47.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.863+0000 7f269912fe40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:47.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.863+0000 7f269912fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:47.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.867+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385edf50 con 0x56373852ac20 2023-10-05T19:40:47.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.867+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f269000a5a0 con 0x56373852ac20 2023-10-05T19:40:47.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.867+0000 7f269912fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:47.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.867+0000 7f269912fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:47.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.867+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385ee8a0 con 0x56373852ac20 2023-10-05T19:40:47.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.867+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f269000a5a0 con 0x56373852ac20 2023-10-05T19:40:47.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.867+0000 7f269912fe40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:47.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.867+0000 7f269912fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:47.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.867+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385edb10 con 0x5637385ef1d0 2023-10-05T19:40:47.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.867+0000 7f2696463700 1 -- 172.21.15.12:0/3953564550 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f268c012450 con 0x5637385ef1d0 2023-10-05T19:40:47.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.867+0000 7f269912fe40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:47.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.867+0000 7f269912fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:47.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.867+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f2420 con 0x5637385ef1d0 2023-10-05T19:40:47.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.867+0000 7f2696463700 1 -- 172.21.15.12:0/3953564550 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f268c012450 con 0x5637385ef1d0 2023-10-05T19:40:47.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.867+0000 7f269912fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:47.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.867+0000 7f269912fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:47.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.867+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f27d0 con 0x56373852ac20 2023-10-05T19:40:47.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.871+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f269000a5a0 con 0x56373852ac20 2023-10-05T19:40:47.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.871+0000 7f269912fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:47.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.871+0000 7f269912fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:47.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.871+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f2b80 con 0x56373852ac20 2023-10-05T19:40:47.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.871+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f269000a5a0 con 0x56373852ac20 2023-10-05T19:40:47.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.871+0000 7f269912fe40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:47.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.871+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f3b90 con 0x5637385ef1d0 2023-10-05T19:40:47.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.871+0000 7f2696463700 1 -- 172.21.15.12:0/3953564550 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(44 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f268c012450 con 0x5637385ef1d0 2023-10-05T19:40:47.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.871+0000 7f2696463700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f3b90 con 0x5637385ea850 2023-10-05T19:40:47.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.875+0000 7f2696c64700 1 -- 172.21.15.12:0/3953564550 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(45 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2688014070 con 0x5637385ea850 2023-10-05T19:40:47.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.875+0000 7f2696c64700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f3b90 con 0x7f268802d020 2023-10-05T19:40:47.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.875+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(46 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f269000a5a0 con 0x7f268802d020 2023-10-05T19:40:47.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.875+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f27d0 con 0x56373852ac20 2023-10-05T19:40:47.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.875+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(47 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f269000a5a0 con 0x56373852ac20 2023-10-05T19:40:47.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.875+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f27d0 con 0x56373852ac20 2023-10-05T19:40:47.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.875+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(48 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f269000a5a0 con 0x56373852ac20 2023-10-05T19:40:47.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.875+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f27d0 con 0x56373852ac20 2023-10-05T19:40:47.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.879+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(49 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f269000a5a0 con 0x56373852ac20 2023-10-05T19:40:47.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.879+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f27d0 con 0x7f268802d020 2023-10-05T19:40:47.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.879+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(50 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f269000a5a0 con 0x7f268802d020 2023-10-05T19:40:47.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.879+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f27d0 con 0x5637385ea850 2023-10-05T19:40:47.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.879+0000 7f2696c64700 1 -- 172.21.15.12:0/3953564550 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(51 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2688014070 con 0x5637385ea850 2023-10-05T19:40:47.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.879+0000 7f2696c64700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f27d0 con 0x5637385ea850 2023-10-05T19:40:47.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.879+0000 7f2696c64700 1 -- 172.21.15.12:0/3953564550 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(52 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2688014070 con 0x5637385ea850 2023-10-05T19:40:47.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.879+0000 7f2696c64700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f27d0 con 0x56373852ac20 2023-10-05T19:40:47.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.879+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(53 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f269000a5a0 con 0x56373852ac20 2023-10-05T19:40:47.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.879+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f27d0 con 0x7f268802d020 2023-10-05T19:40:47.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.883+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(54 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f269000a5a0 con 0x7f268802d020 2023-10-05T19:40:47.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.883+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f27d0 con 0x56373852ac20 2023-10-05T19:40:47.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.883+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(55 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f269000a5a0 con 0x56373852ac20 2023-10-05T19:40:47.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.883+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f27d0 con 0x5637385ea850 2023-10-05T19:40:47.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.883+0000 7f2696c64700 1 -- 172.21.15.12:0/3953564550 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(56 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2688014070 con 0x5637385ea850 2023-10-05T19:40:47.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.883+0000 7f2696c64700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f27d0 con 0x5637385ef1d0 2023-10-05T19:40:47.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.883+0000 7f2696463700 1 -- 172.21.15.12:0/3953564550 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(57 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f268c012450 con 0x5637385ef1d0 2023-10-05T19:40:47.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.883+0000 7f2696463700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f27d0 con 0x5637385ef1d0 2023-10-05T19:40:47.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.887+0000 7f2696463700 1 -- 172.21.15.12:0/3953564550 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(58 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f268c012450 con 0x5637385ef1d0 2023-10-05T19:40:47.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.887+0000 7f2696463700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f27d0 con 0x7f268802d020 2023-10-05T19:40:47.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.887+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(59 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f269000a5a0 con 0x7f268802d020 2023-10-05T19:40:47.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.887+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f27d0 con 0x5637385ef1d0 2023-10-05T19:40:47.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.887+0000 7f2696463700 1 -- 172.21.15.12:0/3953564550 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(60 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f268c012450 con 0x5637385ef1d0 2023-10-05T19:40:47.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.887+0000 7f2696463700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f27d0 con 0x5637385ef1d0 2023-10-05T19:40:47.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.887+0000 7f2696463700 1 -- 172.21.15.12:0/3953564550 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(61 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f268c012450 con 0x5637385ef1d0 2023-10-05T19:40:47.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.887+0000 7f2696463700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f27d0 con 0x56373852ac20 2023-10-05T19:40:47.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.887+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(62 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f269000a5a0 con 0x56373852ac20 2023-10-05T19:40:47.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.887+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f27d0 con 0x56373852ac20 2023-10-05T19:40:47.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.891+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(63 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f269000a5a0 con 0x56373852ac20 2023-10-05T19:40:47.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.891+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f27d0 con 0x7f268802d020 2023-10-05T19:40:47.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.891+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(64 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f269000a5a0 con 0x7f268802d020 2023-10-05T19:40:47.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.891+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f27d0 con 0x5637385ef1d0 2023-10-05T19:40:47.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.891+0000 7f2696463700 1 -- 172.21.15.12:0/3953564550 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(65 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f268c012450 con 0x5637385ef1d0 2023-10-05T19:40:47.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.891+0000 7f2696463700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f27d0 con 0x5637385ea850 2023-10-05T19:40:47.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.891+0000 7f2696c64700 1 -- 172.21.15.12:0/3953564550 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(66 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2688014070 con 0x5637385ea850 2023-10-05T19:40:47.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.891+0000 7f2696c64700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f27d0 con 0x56373852ac20 2023-10-05T19:40:47.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.891+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(67 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f269000a5a0 con 0x56373852ac20 2023-10-05T19:40:47.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.891+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f27d0 con 0x7f268802d020 2023-10-05T19:40:47.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.895+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(68 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f269000a5a0 con 0x7f268802d020 2023-10-05T19:40:47.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.895+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f27d0 con 0x5637385ef1d0 2023-10-05T19:40:47.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.895+0000 7f2696463700 1 -- 172.21.15.12:0/3953564550 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(69 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f268c012450 con 0x5637385ef1d0 2023-10-05T19:40:47.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.895+0000 7f2696463700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f27d0 con 0x7f268802d020 2023-10-05T19:40:47.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.895+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(70 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f269000a5a0 con 0x7f268802d020 2023-10-05T19:40:47.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.895+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f27d0 con 0x5637385ea850 2023-10-05T19:40:47.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.895+0000 7f2696c64700 1 -- 172.21.15.12:0/3953564550 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(71 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2688014070 con 0x5637385ea850 2023-10-05T19:40:47.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.895+0000 7f2696c64700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f27d0 con 0x5637385ef1d0 2023-10-05T19:40:47.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.899+0000 7f2696463700 1 -- 172.21.15.12:0/3953564550 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(72 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f268c012450 con 0x5637385ef1d0 2023-10-05T19:40:47.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.899+0000 7f2696463700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f27d0 con 0x7f268802d020 2023-10-05T19:40:47.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.899+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(73 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f269000a5a0 con 0x7f268802d020 2023-10-05T19:40:47.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.899+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f27d0 con 0x56373852ac20 2023-10-05T19:40:47.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.899+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(74 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f269000a5a0 con 0x56373852ac20 2023-10-05T19:40:47.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.899+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f27d0 con 0x5637385ea850 2023-10-05T19:40:47.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.899+0000 7f2696c64700 1 -- 172.21.15.12:0/3953564550 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(75 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f2688014070 con 0x5637385ea850 2023-10-05T19:40:47.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.899+0000 7f269912fe40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:47.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.899+0000 7f269912fe40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:47.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.899+0000 7f269912fe40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:47.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.899+0000 7f269912fe40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:47.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.899+0000 7f269912fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:47.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.899+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f2b80 con 0x56373852ac20 2023-10-05T19:40:47.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.903+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f269000a5a0 con 0x56373852ac20 2023-10-05T19:40:47.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.903+0000 7f269912fe40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:47.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.903+0000 7f269912fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:47.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.903+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f2f30 con 0x5637385ea850 2023-10-05T19:40:47.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.903+0000 7f2696c64700 1 -- 172.21.15.12:0/3953564550 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f2688014070 con 0x5637385ea850 2023-10-05T19:40:47.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.903+0000 7f269912fe40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:47.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.903+0000 7f269912fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:47.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.903+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f2050 con 0x56373852ac20 2023-10-05T19:40:47.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.903+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f269000a5a0 con 0x56373852ac20 2023-10-05T19:40:47.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.903+0000 7f269912fe40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:47.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.903+0000 7f269912fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:47.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.903+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f3850 con 0x5637385ef1d0 2023-10-05T19:40:47.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.903+0000 7f2696463700 1 -- 172.21.15.12:0/3953564550 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f268c012450 con 0x5637385ef1d0 2023-10-05T19:40:47.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.903+0000 7f269912fe40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:47.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.903+0000 7f269912fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:47.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.903+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f3c00 con 0x7f268802d020 2023-10-05T19:40:47.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.903+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f269000a5a0 con 0x7f268802d020 2023-10-05T19:40:47.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.903+0000 7f269912fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:47.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.903+0000 7f269912fe40 20 zone default found 2023-10-05T19:40:47.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.903+0000 7f269912fe40 4 Realm: () 2023-10-05T19:40:47.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.903+0000 7f269912fe40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:47.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.903+0000 7f269912fe40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:47.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.903+0000 7f269912fe40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:47.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.903+0000 7f269912fe40 20 zonegroup default 2023-10-05T19:40:47.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.903+0000 7f269912fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:47.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.903+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f3c00 con 0x7f268802d020 2023-10-05T19:40:47.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.907+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f269000a5a0 con 0x7f268802d020 2023-10-05T19:40:47.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.907+0000 7f269912fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:47.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.907+0000 7f269912fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:47.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.907+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f5580 con 0x5637385ef1d0 2023-10-05T19:40:47.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.907+0000 7f2696463700 1 -- 172.21.15.12:0/3953564550 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f268c012450 con 0x5637385ef1d0 2023-10-05T19:40:47.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.907+0000 7f269912fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:47.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.907+0000 7f269912fe40 20 rados->read ofs=0 len=0 2023-10-05T19:40:47.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.907+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f5930 con 0x5637385ef1d0 2023-10-05T19:40:47.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.907+0000 7f2696463700 1 -- 172.21.15.12:0/3953564550 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f268c012450 con 0x5637385ef1d0 2023-10-05T19:40:47.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.907+0000 7f269912fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:47.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.907+0000 7f269912fe40 20 started sync module instance, tier type = 2023-10-05T19:40:47.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.907+0000 7f269912fe40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:47.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.907+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f4c40 con 0x5637385ea850 2023-10-05T19:40:47.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.911+0000 7f2696c64700 1 -- 172.21.15.12:0/3953564550 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(84 notify.0 [create] v30'189 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2688014070 con 0x5637385ea850 2023-10-05T19:40:47.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.911+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94795168960576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f3400 con 0x5637385ea850 2023-10-05T19:40:47.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.911+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f6750 con 0x56373852ac20 2023-10-05T19:40:47.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.915+0000 7f2696c64700 1 -- 172.21.15.12:0/3953564550 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(85 notify.0 [watch watch cookie 94795168960576] v30'190 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2688014070 con 0x5637385ea850 2023-10-05T19:40:47.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.915+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(86 notify.1 [create] v30'189 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f269000a5a0 con 0x56373852ac20 2023-10-05T19:40:47.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.915+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94795168965216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f5e40 con 0x56373852ac20 2023-10-05T19:40:47.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.915+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f7530 con 0x5637385ef1d0 2023-10-05T19:40:47.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.919+0000 7f2696463700 1 -- 172.21.15.12:0/3953564550 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(88 notify.2 [create] v30'189 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f268c012450 con 0x5637385ef1d0 2023-10-05T19:40:47.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.919+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(87 notify.1 [watch watch cookie 94795168965216] v30'190 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f269000a5a0 con 0x56373852ac20 2023-10-05T19:40:47.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.919+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94795168970976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f7160 con 0x5637385ef1d0 2023-10-05T19:40:47.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.919+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f8460 con 0x5637385ea850 2023-10-05T19:40:47.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.919+0000 7f2696c64700 1 -- 172.21.15.12:0/3953564550 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(90 notify.3 [create] v30'189 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2688014070 con 0x5637385ea850 2023-10-05T19:40:47.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.919+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94795168974944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f91a0 con 0x5637385ea850 2023-10-05T19:40:47.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.919+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f9a40 con 0x56373852ac20 2023-10-05T19:40:47.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.923+0000 7f2696463700 1 -- 172.21.15.12:0/3953564550 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(89 notify.2 [watch watch cookie 94795168970976] v30'190 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f268c012450 con 0x5637385ef1d0 2023-10-05T19:40:47.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.923+0000 7f2696c64700 1 -- 172.21.15.12:0/3953564550 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(91 notify.3 [watch watch cookie 94795168974944] v30'190 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2688014070 con 0x5637385ea850 2023-10-05T19:40:47.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.923+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(92 notify.4 [create] v30'189 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f269000a5a0 con 0x56373852ac20 2023-10-05T19:40:47.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.923+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94795168976048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f95f0 con 0x56373852ac20 2023-10-05T19:40:47.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.923+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f9ed0 con 0x7f268802d020 2023-10-05T19:40:47.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.927+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(94 notify.5 [create] v30'189 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f269000a5a0 con 0x7f268802d020 2023-10-05T19:40:47.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.927+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94795168978976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385fa2b0 con 0x7f268802d020 2023-10-05T19:40:47.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.927+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385fae10 con 0x7f268802d020 2023-10-05T19:40:47.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.927+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(93 notify.4 [watch watch cookie 94795168976048] v30'190 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f269000a5a0 con 0x56373852ac20 2023-10-05T19:40:47.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.927+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(95 notify.5 [watch watch cookie 94795168978976] v30'190 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f269000a5a0 con 0x7f268802d020 2023-10-05T19:40:47.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.931+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(96 notify.6 [create] v30'189 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f269000a5a0 con 0x7f268802d020 2023-10-05T19:40:47.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.931+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94795168977152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385fa8e0 con 0x7f268802d020 2023-10-05T19:40:47.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.931+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385fb2f0 con 0x7f268802d020 2023-10-05T19:40:47.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.931+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(97 notify.6 [watch watch cookie 94795168977152] v30'190 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f269000a5a0 con 0x7f268802d020 2023-10-05T19:40:47.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.931+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(98 notify.7 [create] v30'189 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f269000a5a0 con 0x7f268802d020 2023-10-05T19:40:47.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.931+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94795168980288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385fb2f0 con 0x7f268802d020 2023-10-05T19:40:47.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.931+0000 7f269912fe40 20 add_watcher() i=0 2023-10-05T19:40:47.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.931+0000 7f269912fe40 20 add_watcher() i=1 2023-10-05T19:40:47.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.931+0000 7f269912fe40 20 add_watcher() i=2 2023-10-05T19:40:47.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.931+0000 7f269912fe40 20 add_watcher() i=3 2023-10-05T19:40:47.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.931+0000 7f269912fe40 20 add_watcher() i=4 2023-10-05T19:40:47.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.931+0000 7f269912fe40 20 add_watcher() i=5 2023-10-05T19:40:47.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.931+0000 7f269912fe40 20 add_watcher() i=6 2023-10-05T19:40:47.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.935+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(99 notify.7 [watch watch cookie 94795168980288] v30'190 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f269000a5a0 con 0x7f268802d020 2023-10-05T19:40:47.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.935+0000 7f269912fe40 20 add_watcher() i=7 2023-10-05T19:40:47.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.935+0000 7f269912fe40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:47.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.935+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385fe4c0 con 0x56373852ac20 2023-10-05T19:40:47.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.935+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f269000a5a0 con 0x56373852ac20 2023-10-05T19:40:47.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.935+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94795168984944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385fb8b0 con 0x56373852ac20 2023-10-05T19:40:47.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.939+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94795168984944] v30'173 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f269000a5a0 con 0x56373852ac20 2023-10-05T19:40:47.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.939+0000 7f2667fe7700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:47.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.939+0000 7f269912fe40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:47.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.939+0000 7f269912fe40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:47.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.939+0000 7f269912fe40 1 Processor -- start 2023-10-05T19:40:47.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.939+0000 7f269912fe40 1 -- start start 2023-10-05T19:40:47.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.939+0000 7f269912fe40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563738608970 0x563738608d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:47.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.939+0000 7f269912fe40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x563738609280 con 0x563738608970 2023-10-05T19:40:47.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.939+0000 7f2696463700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563738608970 0x563738608d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:47.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.939+0000 7f2696463700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563738608970 0x563738608d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58336/0 (socket says 172.21.15.12:58336) 2023-10-05T19:40:47.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.939+0000 7f2696463700 1 -- 172.21.15.12:0/2764953317 learned_addr learned my addr 172.21.15.12:0/2764953317 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:47.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.939+0000 7f2696463700 1 -- 172.21.15.12:0/2764953317 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637386093c0 con 0x563738608970 2023-10-05T19:40:47.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.939+0000 7f2696463700 1 --2- 172.21.15.12:0/2764953317 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563738608970 0x563738608d40 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x56373852b080 tx=0x7f268c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9d226c20097e9ff server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:47.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.939+0000 7f26677e6700 1 -- 172.21.15.12:0/2764953317 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f268c030ce0 con 0x563738608970 2023-10-05T19:40:47.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.939+0000 7f26677e6700 1 -- 172.21.15.12:0/2764953317 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f268c030e60 con 0x563738608970 2023-10-05T19:40:47.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.939+0000 7f26677e6700 1 -- 172.21.15.12:0/2764953317 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f268c012b50 con 0x563738608970 2023-10-05T19:40:47.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.943+0000 7f269912fe40 1 -- 172.21.15.12:0/2764953317 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563738608970 msgr2=0x563738608d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:47.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.943+0000 7f269912fe40 1 --2- 172.21.15.12:0/2764953317 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563738608970 0x563738608d40 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x56373852b080 tx=0x7f268c012000 comp rx=0 tx=0).stop 2023-10-05T19:40:47.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.943+0000 7f269912fe40 1 -- 172.21.15.12:0/2764953317 shutdown_connections 2023-10-05T19:40:47.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.943+0000 7f269912fe40 1 --2- 172.21.15.12:0/2764953317 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563738608970 0x563738608d40 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:47.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.943+0000 7f269912fe40 1 -- 172.21.15.12:0/2764953317 >> 172.21.15.12:0/2764953317 conn(0x563738603d30 msgr2=0x563738601ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:47.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.943+0000 7f269912fe40 1 -- 172.21.15.12:0/2764953317 shutdown_connections 2023-10-05T19:40:47.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.943+0000 7f269912fe40 1 -- 172.21.15.12:0/2764953317 wait complete. 2023-10-05T19:40:47.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.943+0000 7f269912fe40 1 Processor -- start 2023-10-05T19:40:47.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.943+0000 7f269912fe40 1 -- start start 2023-10-05T19:40:47.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.943+0000 7f269912fe40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563738608970 0x56373860e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:47.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.943+0000 7f269912fe40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56373860f450 con 0x563738608970 2023-10-05T19:40:47.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.943+0000 7f2696463700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563738608970 0x56373860e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:47.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.943+0000 7f2696463700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563738608970 0x56373860e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58350/0 (socket says 172.21.15.12:58350) 2023-10-05T19:40:47.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.943+0000 7f2696463700 1 -- 172.21.15.12:0/1458559896 learned_addr learned my addr 172.21.15.12:0/1458559896 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:47.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.943+0000 7f2696463700 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56373852b0b0 con 0x563738608970 2023-10-05T19:40:47.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.943+0000 7f2696463700 1 --2- 172.21.15.12:0/1458559896 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563738608970 0x56373860e450 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x563738602a50 tx=0x7f268c032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:47.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.943+0000 7f2665fe3700 1 -- 172.21.15.12:0/1458559896 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f268c028c80 con 0x563738608970 2023-10-05T19:40:47.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.943+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56373860ea50 con 0x563738608970 2023-10-05T19:40:47.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.943+0000 7f2665fe3700 1 -- 172.21.15.12:0/1458559896 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f268c044070 con 0x563738608970 2023-10-05T19:40:47.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.943+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56373860ee90 con 0x563738608970 2023-10-05T19:40:47.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.943+0000 7f2665fe3700 1 -- 172.21.15.12:0/1458559896 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f268c028e20 con 0x563738608970 2023-10-05T19:40:47.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.947+0000 7f2665fe3700 1 -- 172.21.15.12:0/1458559896 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f268c036040 con 0x563738608970 2023-10-05T19:40:47.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.947+0000 7f2665fe3700 1 --2- 172.21.15.12:0/1458559896 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f25dc030c00 0x7f25dc0330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:47.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.947+0000 7f2665fe3700 1 -- 172.21.15.12:0/1458559896 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7f268c033040 con 0x563738608970 2023-10-05T19:40:47.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.947+0000 7f2697465700 1 --2- 172.21.15.12:0/1458559896 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f25dc030c00 0x7f25dc0330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:47.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.947+0000 7f269912fe40 1 --2- 172.21.15.12:0/1458559896 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x563738619010 0x56373861b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:47.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.947+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56373861bac0 con 0x563738619010 2023-10-05T19:40:47.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.947+0000 7f2696c64700 1 --2- 172.21.15.12:0/1458559896 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x563738619010 0x56373861b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:47.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.947+0000 7f2697465700 1 --2- 172.21.15.12:0/1458559896 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f25dc030c00 0x7f25dc0330b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x5637385fe590 tx=0x7f269000aed0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:47.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.947+0000 7f2696c64700 1 --2- 172.21.15.12:0/1458559896 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x563738619010 0x56373861b420 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:47.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.951+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v30'27 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f2688014070 con 0x563738619010 2023-10-05T19:40:47.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.951+0000 7f269912fe40 1 --2- 172.21.15.12:0/1458559896 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x56373861c820 0x56373861ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:47.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.951+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56373861f2c0 con 0x56373861c820 2023-10-05T19:40:47.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.951+0000 7f2696463700 1 --2- 172.21.15.12:0/1458559896 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x56373861c820 0x56373861ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:47.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.951+0000 7f2696463700 1 --2- 172.21.15.12:0/1458559896 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x56373861c820 0x56373861ec30 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:47.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.955+0000 7f2696463700 1 -- 172.21.15.12:0/1458559896 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v30'18 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f268c012450 con 0x56373861c820 2023-10-05T19:40:47.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.955+0000 7f269912fe40 1 --2- 172.21.15.12:0/1458559896 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x56373861ff90 0x563738622370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:47.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.955+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738622aa0 con 0x56373861ff90 2023-10-05T19:40:47.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.955+0000 7f2697465700 1 --2- 172.21.15.12:0/1458559896 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x56373861ff90 0x563738622370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:47.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.955+0000 7f2697465700 1 --2- 172.21.15.12:0/1458559896 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x56373861ff90 0x563738622370 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:47.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.955+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v30'18 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:47.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.955+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637386230f0 con 0x563738619010 2023-10-05T19:40:47.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.959+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v30'174 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f2688014070 con 0x563738619010 2023-10-05T19:40:47.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.959+0000 7f269912fe40 1 --2- 172.21.15.12:0/1458559896 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x563738623d90 0x563738626170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:47.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.959+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738626820 con 0x563738623d90 2023-10-05T19:40:47.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.959+0000 7f2696c64700 1 --2- 172.21.15.12:0/1458559896 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x563738623d90 0x563738626170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:47.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.959+0000 7f2696c64700 1 --2- 172.21.15.12:0/1458559896 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x563738623d90 0x563738626170 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:47.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.963+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v30'44 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f2688014070 con 0x563738623d90 2023-10-05T19:40:47.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.963+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738626d10 con 0x56373861ff90 2023-10-05T19:40:47.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.967+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v30'32 uv8 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:47.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.967+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738627330 con 0x563738619010 2023-10-05T19:40:47.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.971+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v30'20 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f2688014070 con 0x563738619010 2023-10-05T19:40:47.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.971+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738627330 con 0x56373861c820 2023-10-05T19:40:47.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.971+0000 7f2696463700 1 -- 172.21.15.12:0/1458559896 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v30'29 uv8 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f268c012450 con 0x56373861c820 2023-10-05T19:40:47.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.971+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738627950 con 0x563738623d90 2023-10-05T19:40:47.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.975+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v30'18 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f2688014070 con 0x563738623d90 2023-10-05T19:40:47.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.975+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738627950 con 0x56373861ff90 2023-10-05T19:40:47.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.979+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v30'24 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:47.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.979+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738627e40 con 0x563738623d90 2023-10-05T19:40:47.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.979+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v30'45 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f2688014070 con 0x563738623d90 2023-10-05T19:40:47.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.979+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738627e40 con 0x56373861c820 2023-10-05T19:40:47.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.983+0000 7f2696463700 1 -- 172.21.15.12:0/1458559896 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v30'21 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f268c012450 con 0x56373861c820 2023-10-05T19:40:47.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.983+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738614a80 con 0x563738623d90 2023-10-05T19:40:47.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.987+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v30'18 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f2688014070 con 0x563738623d90 2023-10-05T19:40:47.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.987+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738614a80 con 0x56373861c820 2023-10-05T19:40:47.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.987+0000 7f2696463700 1 -- 172.21.15.12:0/1458559896 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v30'38 uv12 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f268c012450 con 0x56373861c820 2023-10-05T19:40:47.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.987+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738628960 con 0x563738619010 2023-10-05T19:40:47.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.991+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v30'175 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f2688014070 con 0x563738619010 2023-10-05T19:40:47.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.991+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738628960 con 0x56373861ff90 2023-10-05T19:40:48.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.995+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v30'38 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:48.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.995+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738628960 con 0x563738623d90 2023-10-05T19:40:48.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.995+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v30'46 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f2688014070 con 0x563738623d90 2023-10-05T19:40:48.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.995+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738628960 con 0x56373861ff90 2023-10-05T19:40:48.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.999+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v30'18 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:48.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:47.999+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738628960 con 0x563738619010 2023-10-05T19:40:48.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.003+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v30'38 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f2688014070 con 0x563738619010 2023-10-05T19:40:48.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.003+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738628960 con 0x563738619010 2023-10-05T19:40:48.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.007+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v30'176 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f2688014070 con 0x563738619010 2023-10-05T19:40:48.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.007+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738628960 con 0x56373861ff90 2023-10-05T19:40:48.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.007+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v30'39 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:48.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.007+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738628960 con 0x563738623d90 2023-10-05T19:40:48.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.011+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v30'28 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f2688014070 con 0x563738623d90 2023-10-05T19:40:48.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.011+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738628960 con 0x56373861ff90 2023-10-05T19:40:48.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.015+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v30'21 uv20 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:48.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.015+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738628960 con 0x56373861ff90 2023-10-05T19:40:48.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.015+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v30'27 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:48.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.015+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738628960 con 0x56373861ff90 2023-10-05T19:40:48.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.019+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v30'15 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:48.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.019+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738628960 con 0x56373861ff90 2023-10-05T19:40:48.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.023+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v30'33 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:48.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.023+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738628960 con 0x56373861c820 2023-10-05T19:40:48.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.027+0000 7f2696463700 1 -- 172.21.15.12:0/1458559896 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v30'24 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f268c012450 con 0x56373861c820 2023-10-05T19:40:48.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.027+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738628960 con 0x56373861ff90 2023-10-05T19:40:48.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.027+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v30'18 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:48.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.027+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738628960 con 0x56373861c820 2023-10-05T19:40:48.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.031+0000 7f2696463700 1 -- 172.21.15.12:0/1458559896 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v30'30 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f268c012450 con 0x56373861c820 2023-10-05T19:40:48.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.031+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738628960 con 0x56373861ff90 2023-10-05T19:40:48.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.035+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v30'40 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:48.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.035+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738628960 con 0x56373861c820 2023-10-05T19:40:48.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.035+0000 7f2696463700 1 -- 172.21.15.12:0/1458559896 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v30'39 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f268c012450 con 0x56373861c820 2023-10-05T19:40:48.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.035+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738628960 con 0x563738623d90 2023-10-05T19:40:48.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.039+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v30'24 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f2688014070 con 0x563738623d90 2023-10-05T19:40:48.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.043+0000 7f25d6ffd700 20 reqs_thread_entry: start 2023-10-05T19:40:48.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.043+0000 7f25d4ff9700 20 reqs_thread_entry: start 2023-10-05T19:40:48.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.043+0000 7f269912fe40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:48.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.043+0000 7f269912fe40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:48.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.043+0000 7f25c6ffd700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:41:18 2023 2023-10-05T19:40:48.052 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:48.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.043+0000 7f25c67fc700 20 reqs_thread_entry: start 2023-10-05T19:40:48.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.043+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:33 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x563738619010 2023-10-05T19:40:48.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.043+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(33 gc.0 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f2688014070 con 0x563738619010 2023-10-05T19:40:48.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.047+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:34 6.19 6:9a6dd8c0:gc::gc.0:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738686ba0 con 0x563738619010 2023-10-05T19:40:48.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.047+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(34 gc.0 [call out=24b] v0'0 uv5 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7f2688014070 con 0x563738619010 2023-10-05T19:40:48.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.047+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:35 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x563738619010 2023-10-05T19:40:48.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.047+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(35 gc.0 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f2688014070 con 0x563738619010 2023-10-05T19:40:48.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.047+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:36 6.18 6:1e85421f:gc::gc.1:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x56373861c820 2023-10-05T19:40:48.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.047+0000 7f2696463700 1 -- 172.21.15.12:0/1458559896 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(36 gc.1 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f268c012450 con 0x56373861c820 2023-10-05T19:40:48.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.047+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:37 6.18 6:1e85421f:gc::gc.1:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738686ba0 con 0x56373861c820 2023-10-05T19:40:48.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.047+0000 7f2696463700 1 -- 172.21.15.12:0/1458559896 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(37 gc.1 [call out=24b] v0'0 uv5 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7f268c012450 con 0x56373861c820 2023-10-05T19:40:48.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.047+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:38 6.18 6:1e85421f:gc::gc.1:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x56373861c820 2023-10-05T19:40:48.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.051+0000 7f2696463700 1 -- 172.21.15.12:0/1458559896 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(38 gc.1 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f268c012450 con 0x56373861c820 2023-10-05T19:40:48.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.051+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:39 6.b 6:d242335b:gc::gc.2:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x56373861ff90 2023-10-05T19:40:48.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.051+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(39 gc.2 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:48.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.051+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 6.b 6:d242335b:gc::gc.2:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738686ba0 con 0x56373861ff90 2023-10-05T19:40:48.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.051+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(40 gc.2 [call out=24b] v0'0 uv3 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:48.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.051+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 6.b 6:d242335b:gc::gc.2:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x56373861ff90 2023-10-05T19:40:48.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.051+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(41 gc.2 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:48.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.051+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 6.4 6:2166d8e1:gc::gc.3:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x563738619010 2023-10-05T19:40:48.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.051+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(42 gc.3 [call out=15b] v0'0 uv13 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f2688014070 con 0x563738619010 2023-10-05T19:40:48.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.051+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 6.4 6:2166d8e1:gc::gc.3:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738686ba0 con 0x563738619010 2023-10-05T19:40:48.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.051+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(43 gc.3 [call out=24b] v0'0 uv13 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7f2688014070 con 0x563738619010 2023-10-05T19:40:48.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.051+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:44 6.4 6:2166d8e1:gc::gc.3:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x563738619010 2023-10-05T19:40:48.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.055+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(44 gc.3 [call out=15b] v0'0 uv13 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f2688014070 con 0x563738619010 2023-10-05T19:40:48.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.055+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 6.3 6:c60da066:gc::gc.4:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x563738623d90 2023-10-05T19:40:48.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.055+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 gc.4 [call out=15b] v0'0 uv9 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f2688014070 con 0x563738623d90 2023-10-05T19:40:48.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.055+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:46 6.3 6:c60da066:gc::gc.4:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738686ba0 con 0x563738623d90 2023-10-05T19:40:48.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.055+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(46 gc.4 [call out=24b] v0'0 uv9 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7f2688014070 con 0x563738623d90 2023-10-05T19:40:48.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.055+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:47 6.3 6:c60da066:gc::gc.4:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x563738623d90 2023-10-05T19:40:48.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.055+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(47 gc.4 [call out=15b] v0'0 uv9 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f2688014070 con 0x563738623d90 2023-10-05T19:40:48.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.055+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:48 6.9 6:954be8d2:gc::gc.5:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x56373861ff90 2023-10-05T19:40:48.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.055+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(48 gc.5 [call out=15b] v0'0 uv8 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:48.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.055+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:49 6.9 6:954be8d2:gc::gc.5:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738686ba0 con 0x56373861ff90 2023-10-05T19:40:48.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.059+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(49 gc.5 [call out=24b] v0'0 uv8 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:48.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.059+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:50 6.9 6:954be8d2:gc::gc.5:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x56373861ff90 2023-10-05T19:40:48.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.059+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(50 gc.5 [call out=15b] v0'0 uv8 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:48.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.059+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:51 6.e 6:748722ff:gc::gc.6:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x563738619010 2023-10-05T19:40:48.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.059+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(51 gc.6 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f2688014070 con 0x563738619010 2023-10-05T19:40:48.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.059+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:52 6.e 6:748722ff:gc::gc.6:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738686ba0 con 0x563738619010 2023-10-05T19:40:48.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.059+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(52 gc.6 [call out=24b] v0'0 uv5 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7f2688014070 con 0x563738619010 2023-10-05T19:40:48.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.059+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 6.e 6:748722ff:gc::gc.6:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x563738619010 2023-10-05T19:40:48.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.059+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(53 gc.6 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f2688014070 con 0x563738619010 2023-10-05T19:40:48.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.059+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 6.1d 6:beaa4be4:gc::gc.7:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x56373861c820 2023-10-05T19:40:48.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.063+0000 7f2696463700 1 -- 172.21.15.12:0/1458559896 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(54 gc.7 [call out=15b] v0'0 uv8 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f268c012450 con 0x56373861c820 2023-10-05T19:40:48.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.063+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:55 6.1d 6:beaa4be4:gc::gc.7:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738686ba0 con 0x56373861c820 2023-10-05T19:40:48.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.063+0000 7f2696463700 1 -- 172.21.15.12:0/1458559896 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(55 gc.7 [call out=24b] v0'0 uv8 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7f268c012450 con 0x56373861c820 2023-10-05T19:40:48.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.063+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:56 6.1d 6:beaa4be4:gc::gc.7:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x56373861c820 2023-10-05T19:40:48.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.063+0000 7f2696463700 1 -- 172.21.15.12:0/1458559896 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(56 gc.7 [call out=15b] v0'0 uv8 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f268c012450 con 0x56373861c820 2023-10-05T19:40:48.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.063+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:57 6.12 6:4f78e9a2:gc::gc.8:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x563738623d90 2023-10-05T19:40:48.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.063+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(57 gc.8 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f2688014070 con 0x563738623d90 2023-10-05T19:40:48.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.063+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:58 6.12 6:4f78e9a2:gc::gc.8:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738686ba0 con 0x563738623d90 2023-10-05T19:40:48.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.063+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(58 gc.8 [call out=24b] v0'0 uv3 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7f2688014070 con 0x563738623d90 2023-10-05T19:40:48.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.063+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:59 6.12 6:4f78e9a2:gc::gc.8:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x563738623d90 2023-10-05T19:40:48.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.067+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(59 gc.8 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f2688014070 con 0x563738623d90 2023-10-05T19:40:48.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.067+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 6.1c 6:3e426232:gc::gc.9:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x56373861ff90 2023-10-05T19:40:48.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.067+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(60 gc.9 [call out=15b] v0'0 uv9 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:48.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.067+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 6.1c 6:3e426232:gc::gc.9:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738686ba0 con 0x56373861ff90 2023-10-05T19:40:48.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.067+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(61 gc.9 [call out=24b] v0'0 uv9 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:48.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.067+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:62 6.1c 6:3e426232:gc::gc.9:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x56373861ff90 2023-10-05T19:40:48.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.067+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(62 gc.9 [call out=15b] v0'0 uv9 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:48.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.067+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:63 6.3 6:c31f41d7:gc::gc.10:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x563738623d90 2023-10-05T19:40:48.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.071+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(63 gc.10 [call out=15b] v0'0 uv11 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f2688014070 con 0x563738623d90 2023-10-05T19:40:48.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.071+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:64 6.3 6:c31f41d7:gc::gc.10:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738686ba0 con 0x563738623d90 2023-10-05T19:40:48.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.071+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(64 gc.10 [call out=24b] v0'0 uv11 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f2688014070 con 0x563738623d90 2023-10-05T19:40:48.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.071+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:65 6.3 6:c31f41d7:gc::gc.10:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x563738623d90 2023-10-05T19:40:48.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.071+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(65 gc.10 [call out=15b] v0'0 uv11 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f2688014070 con 0x563738623d90 2023-10-05T19:40:48.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.071+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:66 6.13 6:c8333aab:gc::gc.11:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x56373861c820 2023-10-05T19:40:48.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.071+0000 7f2696463700 1 -- 172.21.15.12:0/1458559896 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(66 gc.11 [call out=15b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f268c012450 con 0x56373861c820 2023-10-05T19:40:48.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.071+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:67 6.13 6:c8333aab:gc::gc.11:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738686ba0 con 0x56373861c820 2023-10-05T19:40:48.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.071+0000 7f2696463700 1 -- 172.21.15.12:0/1458559896 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(67 gc.11 [call out=24b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f268c012450 con 0x56373861c820 2023-10-05T19:40:48.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.071+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 6.13 6:c8333aab:gc::gc.11:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x56373861c820 2023-10-05T19:40:48.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.075+0000 7f2696463700 1 -- 172.21.15.12:0/1458559896 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 gc.11 [call out=15b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f268c012450 con 0x56373861c820 2023-10-05T19:40:48.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.075+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:69 6.15 6:ab569f81:gc::gc.12:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x563738623d90 2023-10-05T19:40:48.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.075+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(69 gc.12 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f2688014070 con 0x563738623d90 2023-10-05T19:40:48.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.075+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:70 6.15 6:ab569f81:gc::gc.12:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738686ba0 con 0x563738623d90 2023-10-05T19:40:48.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.075+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(70 gc.12 [call out=24b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f2688014070 con 0x563738623d90 2023-10-05T19:40:48.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.075+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 6.15 6:ab569f81:gc::gc.12:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x563738623d90 2023-10-05T19:40:48.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.075+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(71 gc.12 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f2688014070 con 0x563738623d90 2023-10-05T19:40:48.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.075+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:72 6.0 6:0667bb13:gc::gc.13:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x56373861c820 2023-10-05T19:40:48.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.075+0000 7f2696463700 1 -- 172.21.15.12:0/1458559896 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(72 gc.13 [call out=15b] v0'0 uv12 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f268c012450 con 0x56373861c820 2023-10-05T19:40:48.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.075+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 6.0 6:0667bb13:gc::gc.13:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738686ba0 con 0x56373861c820 2023-10-05T19:40:48.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.079+0000 7f2696463700 1 -- 172.21.15.12:0/1458559896 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 gc.13 [call out=24b] v0'0 uv12 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f268c012450 con 0x56373861c820 2023-10-05T19:40:48.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.079+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:74 6.0 6:0667bb13:gc::gc.13:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x56373861c820 2023-10-05T19:40:48.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.079+0000 7f2696463700 1 -- 172.21.15.12:0/1458559896 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(74 gc.13 [call out=15b] v0'0 uv12 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f268c012450 con 0x56373861c820 2023-10-05T19:40:48.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.079+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:75 6.4 6:227f63b5:gc::gc.14:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x563738619010 2023-10-05T19:40:48.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.079+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(75 gc.14 [call out=15b] v0'0 uv15 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f2688014070 con 0x563738619010 2023-10-05T19:40:48.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.079+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738686ba0 con 0x563738619010 2023-10-05T19:40:48.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.079+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(76 gc.14 [call out=24b] v0'0 uv15 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f2688014070 con 0x563738619010 2023-10-05T19:40:48.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.079+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:77 6.4 6:227f63b5:gc::gc.14:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x563738619010 2023-10-05T19:40:48.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.079+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(77 gc.14 [call out=15b] v0'0 uv15 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f2688014070 con 0x563738619010 2023-10-05T19:40:48.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.079+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:78 6.7 6:e07a6548:gc::gc.15:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x56373861ff90 2023-10-05T19:40:48.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.083+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(78 gc.15 [call out=15b] v0'0 uv11 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:48.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.083+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 6.7 6:e07a6548:gc::gc.15:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738686ba0 con 0x56373861ff90 2023-10-05T19:40:48.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.083+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(79 gc.15 [call out=24b] v0'0 uv11 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:48.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.083+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:80 6.7 6:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x56373861ff90 2023-10-05T19:40:48.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.083+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(80 gc.15 [call out=15b] v0'0 uv11 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:48.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.083+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:81 6.3 6:c623b173:gc::gc.16:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x563738623d90 2023-10-05T19:40:48.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.083+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(81 gc.16 [call out=15b] v0'0 uv13 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f2688014070 con 0x563738623d90 2023-10-05T19:40:48.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.083+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:82 6.3 6:c623b173:gc::gc.16:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738686ba0 con 0x563738623d90 2023-10-05T19:40:48.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.083+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(82 gc.16 [call out=24b] v0'0 uv13 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f2688014070 con 0x563738623d90 2023-10-05T19:40:48.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.083+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:83 6.3 6:c623b173:gc::gc.16:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x563738623d90 2023-10-05T19:40:48.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.087+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(83 gc.16 [call out=15b] v0'0 uv13 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f2688014070 con 0x563738623d90 2023-10-05T19:40:48.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.087+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:84 6.16 6:6849fd06:gc::gc.17:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x56373861ff90 2023-10-05T19:40:48.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.087+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(84 gc.17 [call out=15b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:48.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.087+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:85 6.16 6:6849fd06:gc::gc.17:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738686ba0 con 0x56373861ff90 2023-10-05T19:40:48.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.087+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(85 gc.17 [call out=24b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:48.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.087+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:86 6.16 6:6849fd06:gc::gc.17:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x56373861ff90 2023-10-05T19:40:48.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.087+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 27 ==== osd_op_reply(86 gc.17 [call out=15b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:48.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.087+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 6.d 6:b019ea83:gc::gc.18:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x563738619010 2023-10-05T19:40:48.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.087+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(87 gc.18 [call out=15b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f2688014070 con 0x563738619010 2023-10-05T19:40:48.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.087+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:88 6.d 6:b019ea83:gc::gc.18:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738686ba0 con 0x563738619010 2023-10-05T19:40:48.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.091+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(88 gc.18 [call out=24b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f2688014070 con 0x563738619010 2023-10-05T19:40:48.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.091+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:89 6.d 6:b019ea83:gc::gc.18:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x563738619010 2023-10-05T19:40:48.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.091+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(89 gc.18 [call out=15b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f2688014070 con 0x563738619010 2023-10-05T19:40:48.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.091+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:90 6.4 6:24c8e1ae:gc::gc.19:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x563738619010 2023-10-05T19:40:48.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.091+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(90 gc.19 [call out=15b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f2688014070 con 0x563738619010 2023-10-05T19:40:48.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.091+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:91 6.4 6:24c8e1ae:gc::gc.19:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738686ba0 con 0x563738619010 2023-10-05T19:40:48.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.091+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(91 gc.19 [call out=24b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f2688014070 con 0x563738619010 2023-10-05T19:40:48.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.091+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 6.4 6:24c8e1ae:gc::gc.19:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x563738619010 2023-10-05T19:40:48.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.091+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(92 gc.19 [call out=15b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f2688014070 con 0x563738619010 2023-10-05T19:40:48.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.091+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:93 6.7 6:e622f76a:gc::gc.20:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x56373861ff90 2023-10-05T19:40:48.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.091+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 28 ==== osd_op_reply(93 gc.20 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:48.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.091+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:94 6.7 6:e622f76a:gc::gc.20:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738686ba0 con 0x56373861ff90 2023-10-05T19:40:48.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.091+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 29 ==== osd_op_reply(94 gc.20 [call out=24b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:48.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.091+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:95 6.7 6:e622f76a:gc::gc.20:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x56373861ff90 2023-10-05T19:40:48.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.095+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 30 ==== osd_op_reply(95 gc.20 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:48.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.095+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:96 6.a 6:5127e642:gc::gc.21:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x563738623d90 2023-10-05T19:40:48.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.095+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.2 v2:172.21.15.12:6800/29934 23 ==== osd_op_reply(96 gc.21 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f2688014070 con 0x563738623d90 2023-10-05T19:40:48.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.095+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:97 6.a 6:5127e642:gc::gc.21:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738686ba0 con 0x563738623d90 2023-10-05T19:40:48.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.095+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.2 v2:172.21.15.12:6800/29934 24 ==== osd_op_reply(97 gc.21 [call out=24b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f2688014070 con 0x563738623d90 2023-10-05T19:40:48.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.095+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:98 6.a 6:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x563738623d90 2023-10-05T19:40:48.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.095+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.2 v2:172.21.15.12:6800/29934 25 ==== osd_op_reply(98 gc.21 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f2688014070 con 0x563738623d90 2023-10-05T19:40:48.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.095+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:99 6.11 6:8eb99c53:gc::gc.22:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x56373861ff90 2023-10-05T19:40:48.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.095+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 31 ==== osd_op_reply(99 gc.22 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:48.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.095+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:100 6.11 6:8eb99c53:gc::gc.22:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738686ba0 con 0x56373861ff90 2023-10-05T19:40:48.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.095+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 32 ==== osd_op_reply(100 gc.22 [call out=24b] v0'0 uv20 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:48.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.095+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:101 6.11 6:8eb99c53:gc::gc.22:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x56373861ff90 2023-10-05T19:40:48.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.099+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 33 ==== osd_op_reply(101 gc.22 [call out=1702b] v0'0 uv20 ondisk = 0) v8 ==== 149+0+1702 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:48.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.099+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:102 6.8 6:17d7e8ed:gc::gc.23:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x56373861ff90 2023-10-05T19:40:48.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.099+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 34 ==== osd_op_reply(102 gc.23 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:48.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.099+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:103 6.8 6:17d7e8ed:gc::gc.23:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738686ba0 con 0x56373861ff90 2023-10-05T19:40:48.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.099+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 35 ==== osd_op_reply(103 gc.23 [call out=24b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:48.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.099+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:104 6.8 6:17d7e8ed:gc::gc.23:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x56373861ff90 2023-10-05T19:40:48.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.099+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 36 ==== osd_op_reply(104 gc.23 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:48.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.099+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:105 6.1b 6:db9d130d:gc::gc.24:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x56373861ff90 2023-10-05T19:40:48.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.099+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 37 ==== osd_op_reply(105 gc.24 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:48.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.099+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:106 6.1b 6:db9d130d:gc::gc.24:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738686ba0 con 0x56373861ff90 2023-10-05T19:40:48.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.103+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 38 ==== osd_op_reply(106 gc.24 [call out=24b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:48.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.103+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:107 6.1b 6:db9d130d:gc::gc.24:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x56373861ff90 2023-10-05T19:40:48.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.103+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 39 ==== osd_op_reply(107 gc.24 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:48.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.103+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:108 6.9 6:921ecdb1:gc::gc.25:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x56373861ff90 2023-10-05T19:40:48.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.103+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 40 ==== osd_op_reply(108 gc.25 [call out=15b] v0'0 uv4 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:48.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.103+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:109 6.9 6:921ecdb1:gc::gc.25:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738686ba0 con 0x56373861ff90 2023-10-05T19:40:48.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.103+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 41 ==== osd_op_reply(109 gc.25 [call out=24b] v0'0 uv4 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:48.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.103+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:110 6.9 6:921ecdb1:gc::gc.25:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x56373861ff90 2023-10-05T19:40:48.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.107+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 42 ==== osd_op_reply(110 gc.25 [call out=15b] v0'0 uv4 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:48.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.107+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:111 6.1f 6:fde4dd55:gc::gc.26:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x56373861c820 2023-10-05T19:40:48.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.107+0000 7f2696463700 1 -- 172.21.15.12:0/1458559896 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(111 gc.26 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f268c012450 con 0x56373861c820 2023-10-05T19:40:48.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.107+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:112 6.1f 6:fde4dd55:gc::gc.26:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738686ba0 con 0x56373861c820 2023-10-05T19:40:48.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.107+0000 7f2696463700 1 -- 172.21.15.12:0/1458559896 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(112 gc.26 [call out=24b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f268c012450 con 0x56373861c820 2023-10-05T19:40:48.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.107+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:113 6.1f 6:fde4dd55:gc::gc.26:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x56373861c820 2023-10-05T19:40:48.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.107+0000 7f2696463700 1 -- 172.21.15.12:0/1458559896 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(113 gc.26 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f268c012450 con 0x56373861c820 2023-10-05T19:40:48.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.107+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:114 6.1a 6:5909e671:gc::gc.27:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x56373861ff90 2023-10-05T19:40:48.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.107+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 43 ==== osd_op_reply(114 gc.27 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:48.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.107+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:115 6.1a 6:5909e671:gc::gc.27:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738686ba0 con 0x56373861ff90 2023-10-05T19:40:48.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.111+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 44 ==== osd_op_reply(115 gc.27 [call out=24b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:48.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.111+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:116 6.1a 6:5909e671:gc::gc.27:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x56373861ff90 2023-10-05T19:40:48.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.111+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 45 ==== osd_op_reply(116 gc.27 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:48.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.111+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:117 6.1d 6:b88e3271:gc::gc.28:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x56373861c820 2023-10-05T19:40:48.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.111+0000 7f2696463700 1 -- 172.21.15.12:0/1458559896 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(117 gc.28 [call out=15b] v0'0 uv4 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f268c012450 con 0x56373861c820 2023-10-05T19:40:48.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.111+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:118 6.1d 6:b88e3271:gc::gc.28:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738686ba0 con 0x56373861c820 2023-10-05T19:40:48.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.111+0000 7f2696463700 1 -- 172.21.15.12:0/1458559896 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(118 gc.28 [call out=24b] v0'0 uv4 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f268c012450 con 0x56373861c820 2023-10-05T19:40:48.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.111+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:119 6.1d 6:b88e3271:gc::gc.28:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x56373861c820 2023-10-05T19:40:48.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.111+0000 7f2696463700 1 -- 172.21.15.12:0/1458559896 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(119 gc.28 [call out=15b] v0'0 uv4 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f268c012450 con 0x56373861c820 2023-10-05T19:40:48.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.111+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:120 6.7 6:e47c9322:gc::gc.29:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x56373861ff90 2023-10-05T19:40:48.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.115+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 46 ==== osd_op_reply(120 gc.29 [call out=15b] v0'0 uv9 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:48.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.115+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:121 6.7 6:e47c9322:gc::gc.29:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738686ba0 con 0x56373861ff90 2023-10-05T19:40:48.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.115+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 47 ==== osd_op_reply(121 gc.29 [call out=24b] v0'0 uv9 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:48.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.115+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:122 6.7 6:e47c9322:gc::gc.29:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x56373861ff90 2023-10-05T19:40:48.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.115+0000 7f2697465700 1 -- 172.21.15.12:0/1458559896 <== osd.3 v2:172.21.15.12:6816/29951 48 ==== osd_op_reply(122 gc.29 [call out=15b] v0'0 uv9 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f269000a5a0 con 0x56373861ff90 2023-10-05T19:40:48.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.115+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:123 6.0 6:013d6c33:gc::gc.30:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x56373861c820 2023-10-05T19:40:48.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.115+0000 7f2696463700 1 -- 172.21.15.12:0/1458559896 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(123 gc.30 [call out=15b] v0'0 uv6 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f268c012450 con 0x56373861c820 2023-10-05T19:40:48.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.115+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:124 6.0 6:013d6c33:gc::gc.30:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738686ba0 con 0x56373861c820 2023-10-05T19:40:48.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.115+0000 7f2696463700 1 -- 172.21.15.12:0/1458559896 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(124 gc.30 [call out=24b] v0'0 uv6 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f268c012450 con 0x56373861c820 2023-10-05T19:40:48.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.115+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:125 6.0 6:013d6c33:gc::gc.30:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x56373861c820 2023-10-05T19:40:48.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.119+0000 7f2696463700 1 -- 172.21.15.12:0/1458559896 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(125 gc.30 [call out=15b] v0'0 uv6 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f268c012450 con 0x56373861c820 2023-10-05T19:40:48.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.119+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:126 6.f 6:f24cb4a8:gc::gc.31:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x563738623d90 2023-10-05T19:40:48.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.119+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.2 v2:172.21.15.12:6800/29934 26 ==== osd_op_reply(126 gc.31 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f2688014070 con 0x563738623d90 2023-10-05T19:40:48.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.119+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:127 6.f 6:f24cb4a8:gc::gc.31:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738686ba0 con 0x563738623d90 2023-10-05T19:40:48.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.119+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.2 v2:172.21.15.12:6800/29934 27 ==== osd_op_reply(127 gc.31 [call out=24b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f2688014070 con 0x563738623d90 2023-10-05T19:40:48.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.119+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:128 6.f 6:f24cb4a8:gc::gc.31:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738687bb0 con 0x563738623d90 2023-10-05T19:40:48.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.119+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 <== osd.2 v2:172.21.15.12:6800/29934 28 ==== osd_op_reply(128 gc.31 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f2688014070 con 0x563738623d90 2023-10-05T19:40:48.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.123+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:102 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94795168960576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385cc4b0 con 0x5637385ea850 2023-10-05T19:40:48.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.123+0000 7f2696c64700 1 -- 172.21.15.12:0/3953564550 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(102 notify.0 [watch unwatch cookie 94795168960576] v30'191 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2688014070 con 0x5637385ea850 2023-10-05T19:40:48.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.123+0000 7f269912fe40 20 remove_watcher() i=0 2023-10-05T19:40:48.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.123+0000 7f269912fe40 2 removed watcher, disabling cache 2023-10-05T19:40:48.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.123+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94795168965216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f5040 con 0x56373852ac20 2023-10-05T19:40:48.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.127+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(103 notify.1 [watch unwatch cookie 94795168965216] v30'191 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f269000a5a0 con 0x56373852ac20 2023-10-05T19:40:48.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.127+0000 7f269912fe40 20 remove_watcher() i=1 2023-10-05T19:40:48.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.127+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:104 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94795168970976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f5040 con 0x5637385ef1d0 2023-10-05T19:40:48.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.131+0000 7f2696463700 1 -- 172.21.15.12:0/3953564550 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(104 notify.2 [watch unwatch cookie 94795168970976] v30'191 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f268c012450 con 0x5637385ef1d0 2023-10-05T19:40:48.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.131+0000 7f269912fe40 20 remove_watcher() i=2 2023-10-05T19:40:48.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.131+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:105 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94795168974944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f78e0 con 0x5637385ea850 2023-10-05T19:40:48.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.131+0000 7f2696c64700 1 -- 172.21.15.12:0/3953564550 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(105 notify.3 [watch unwatch cookie 94795168974944] v30'191 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2688014070 con 0x5637385ea850 2023-10-05T19:40:48.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.131+0000 7f269912fe40 20 remove_watcher() i=3 2023-10-05T19:40:48.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.131+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:106 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94795168976048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f5040 con 0x56373852ac20 2023-10-05T19:40:48.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.135+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(106 notify.4 [watch unwatch cookie 94795168976048] v30'191 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f269000a5a0 con 0x56373852ac20 2023-10-05T19:40:48.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.135+0000 7f269912fe40 20 remove_watcher() i=4 2023-10-05T19:40:48.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.135+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:107 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94795168978976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f5040 con 0x7f268802d020 2023-10-05T19:40:48.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.139+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(107 notify.5 [watch unwatch cookie 94795168978976] v30'191 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f269000a5a0 con 0x7f268802d020 2023-10-05T19:40:48.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.139+0000 7f269912fe40 20 remove_watcher() i=5 2023-10-05T19:40:48.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.139+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:108 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94795168977152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f5040 con 0x7f268802d020 2023-10-05T19:40:48.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.143+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(108 notify.6 [watch unwatch cookie 94795168977152] v30'191 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f269000a5a0 con 0x7f268802d020 2023-10-05T19:40:48.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.143+0000 7f269912fe40 20 remove_watcher() i=6 2023-10-05T19:40:48.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.143+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:109 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94795168980288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637385f6260 con 0x7f268802d020 2023-10-05T19:40:48.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.143+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(109 notify.7 [watch unwatch cookie 94795168980288] v30'191 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f269000a5a0 con 0x7f268802d020 2023-10-05T19:40:48.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.143+0000 7f269912fe40 20 remove_watcher() i=7 2023-10-05T19:40:48.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.147+0000 7f2667fe7700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:48.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.147+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:110 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94795168984944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563738646820 con 0x56373852ac20 2023-10-05T19:40:48.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.151+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(110 data_loggenerations_metadata [watch unwatch cookie 94795168984944] v30'177 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f269000a5a0 con 0x56373852ac20 2023-10-05T19:40:48.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.151+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f268802d020 msgr2=0x7f2688027120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:48.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.151+0000 7f269912fe40 1 --2- 172.21.15.12:0/3953564550 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f268802d020 0x7f2688027120 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:48.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.151+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x56373852ac20 msgr2=0x5637385e8730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:48.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.151+0000 7f269912fe40 1 --2- 172.21.15.12:0/3953564550 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x56373852ac20 0x5637385e8730 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:48.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.151+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5637385ea850 msgr2=0x5637385ecc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:48.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.151+0000 7f269912fe40 1 --2- 172.21.15.12:0/3953564550 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5637385ea850 0x5637385ecc30 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:48.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.151+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5637385ef1d0 msgr2=0x5637385f15b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:48.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.151+0000 7f269912fe40 1 --2- 172.21.15.12:0/3953564550 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5637385ef1d0 0x5637385f15b0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:48.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.151+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f267c030c00 msgr2=0x7f267c0330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:48.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.151+0000 7f269912fe40 1 --2- 172.21.15.12:0/3953564550 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f267c030c00 0x7f267c0330b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f268c005a90 tx=0x7f268c015000 comp rx=0 tx=0).stop 2023-10-05T19:40:48.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.151+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563738529910 msgr2=0x5637385dcca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:48.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.151+0000 7f269912fe40 1 --2- 172.21.15.12:0/3953564550 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563738529910 0x5637385dcca0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f26880017d0 tx=0x7f2688016000 comp rx=0 tx=0).stop 2023-10-05T19:40:48.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.151+0000 7f2697465700 1 -- 172.21.15.12:0/3953564550 reap_dead start 2023-10-05T19:40:48.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.151+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 shutdown_connections 2023-10-05T19:40:48.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.151+0000 7f269912fe40 1 --2- 172.21.15.12:0/3953564550 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563738529910 0x5637385dcca0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:48.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.151+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 >> 172.21.15.12:0/3953564550 conn(0x5637385b0850 msgr2=0x5637385d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:48.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.151+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 shutdown_connections 2023-10-05T19:40:48.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.151+0000 7f269912fe40 1 -- 172.21.15.12:0/3953564550 wait complete. 2023-10-05T19:40:48.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.151+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x56373861c820 msgr2=0x56373861ec30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:48.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.151+0000 7f269912fe40 1 --2- 172.21.15.12:0/1458559896 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x56373861c820 0x56373861ec30 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:48.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.151+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x563738619010 msgr2=0x56373861b420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:48.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.151+0000 7f269912fe40 1 --2- 172.21.15.12:0/1458559896 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x563738619010 0x56373861b420 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:48.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.151+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x563738623d90 msgr2=0x563738626170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:48.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.151+0000 7f269912fe40 1 --2- 172.21.15.12:0/1458559896 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x563738623d90 0x563738626170 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:48.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.151+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x56373861ff90 msgr2=0x563738622370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:48.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.151+0000 7f269912fe40 1 --2- 172.21.15.12:0/1458559896 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x56373861ff90 0x563738622370 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:48.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.151+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f25dc030c00 msgr2=0x7f25dc0330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:48.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.151+0000 7f269912fe40 1 --2- 172.21.15.12:0/1458559896 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f25dc030c00 0x7f25dc0330b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x5637385fe590 tx=0x7f269000aed0 comp rx=0 tx=0).stop 2023-10-05T19:40:48.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.151+0000 7f2696c64700 1 -- 172.21.15.12:0/1458559896 reap_dead start 2023-10-05T19:40:48.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.151+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563738608970 msgr2=0x56373860e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:48.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.151+0000 7f269912fe40 1 --2- 172.21.15.12:0/1458559896 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563738608970 0x56373860e450 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x563738602a50 tx=0x7f268c032000 comp rx=0 tx=0).stop 2023-10-05T19:40:48.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.151+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 shutdown_connections 2023-10-05T19:40:48.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.151+0000 7f269912fe40 1 --2- 172.21.15.12:0/1458559896 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563738608970 0x56373860e450 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:48.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.151+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 >> 172.21.15.12:0/1458559896 conn(0x563738603d30 msgr2=0x563738601e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:48.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.151+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 shutdown_connections 2023-10-05T19:40:48.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.151+0000 7f269912fe40 1 -- 172.21.15.12:0/1458559896 wait complete. 2023-10-05T19:40:48.163 INFO:teuthology.orchestra.run.smithi012.stdout:[ 2023-10-05T19:40:48.164 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:48.164 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "93ac49f4-ff49-4f99-811b-6453b28decd5.4219.17885726856970346626\u0000", 2023-10-05T19:40:48.164 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:40:47.773897+0000", 2023-10-05T19:40:48.164 INFO:teuthology.orchestra.run.smithi012.stdout: "objs": [ 2023-10-05T19:40:48.164 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:48.164 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": "default.rgw.buckets.data", 2023-10-05T19:40:48.164 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1__shadow_.XR8N8VQAMDt_LCHQhT8BOCUfm_xM1JH_1", 2023-10-05T19:40:48.164 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2023-10-05T19:40:48.164 INFO:teuthology.orchestra.run.smithi012.stdout: "instance": "" 2023-10-05T19:40:48.164 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:48.165 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:48.165 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": "default.rgw.buckets.data", 2023-10-05T19:40:48.165 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1__shadow_.XR8N8VQAMDt_LCHQhT8BOCUfm_xM1JH_2", 2023-10-05T19:40:48.165 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2023-10-05T19:40:48.165 INFO:teuthology.orchestra.run.smithi012.stdout: "instance": "" 2023-10-05T19:40:48.165 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:48.165 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:48.165 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": "default.rgw.buckets.data", 2023-10-05T19:40:48.165 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1__shadow_.XR8N8VQAMDt_LCHQhT8BOCUfm_xM1JH_3", 2023-10-05T19:40:48.165 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2023-10-05T19:40:48.165 INFO:teuthology.orchestra.run.smithi012.stdout: "instance": "" 2023-10-05T19:40:48.166 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:48.166 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:48.166 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": "default.rgw.buckets.data", 2023-10-05T19:40:48.166 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1__shadow_.XR8N8VQAMDt_LCHQhT8BOCUfm_xM1JH_4", 2023-10-05T19:40:48.166 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2023-10-05T19:40:48.166 INFO:teuthology.orchestra.run.smithi012.stdout: "instance": "" 2023-10-05T19:40:48.166 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:48.166 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:48.166 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": "default.rgw.buckets.data", 2023-10-05T19:40:48.166 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1__shadow_.XR8N8VQAMDt_LCHQhT8BOCUfm_xM1JH_5", 2023-10-05T19:40:48.166 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2023-10-05T19:40:48.166 INFO:teuthology.orchestra.run.smithi012.stdout: "instance": "" 2023-10-05T19:40:48.167 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:48.167 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:48.167 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": "default.rgw.buckets.data", 2023-10-05T19:40:48.167 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1__shadow_.XR8N8VQAMDt_LCHQhT8BOCUfm_xM1JH_6", 2023-10-05T19:40:48.167 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2023-10-05T19:40:48.167 INFO:teuthology.orchestra.run.smithi012.stdout: "instance": "" 2023-10-05T19:40:48.167 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:48.167 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:48.167 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": "default.rgw.buckets.data", 2023-10-05T19:40:48.167 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1__shadow_.XR8N8VQAMDt_LCHQhT8BOCUfm_xM1JH_7", 2023-10-05T19:40:48.167 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2023-10-05T19:40:48.167 INFO:teuthology.orchestra.run.smithi012.stdout: "instance": "" 2023-10-05T19:40:48.168 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:48.168 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2023-10-05T19:40:48.168 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:48.168 INFO:teuthology.orchestra.run.smithi012.stdout:] 2023-10-05T19:40:48.169 DEBUG:tasks.util.rgw: json result: [{'tag': '93ac49f4-ff49-4f99-811b-6453b28decd5.4219.17885726856970346626\x00', 'time': '2023-10-05T19:40:47.773897+0000', 'objs': [{'pool': 'default.rgw.buckets.data', 'oid': '93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1__shadow_.XR8N8VQAMDt_LCHQhT8BOCUfm_xM1JH_1', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': '93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1__shadow_.XR8N8VQAMDt_LCHQhT8BOCUfm_xM1JH_2', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': '93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1__shadow_.XR8N8VQAMDt_LCHQhT8BOCUfm_xM1JH_3', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': '93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1__shadow_.XR8N8VQAMDt_LCHQhT8BOCUfm_xM1JH_4', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': '93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1__shadow_.XR8N8VQAMDt_LCHQhT8BOCUfm_xM1JH_5', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': '93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1__shadow_.XR8N8VQAMDt_LCHQhT8BOCUfm_xM1JH_6', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': '93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1__shadow_.XR8N8VQAMDt_LCHQhT8BOCUfm_xM1JH_7', 'key': '', 'instance': ''}]}] 2023-10-05T19:40:48.169 INFO:tasks.util.rgw:rgwadmin: client.0 : ['gc', 'process'] 2023-10-05T19:40:48.169 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'gc', 'process'] 2023-10-05T19:40:48.169 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph gc process 2023-10-05T19:40:48.249 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:48.249 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:48.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.243+0000 7fd6a7412e40 1 Processor -- start 2023-10-05T19:40:48.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.243+0000 7fd6a7412e40 1 -- start start 2023-10-05T19:40:48.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.243+0000 7fd6a7412e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x559803ff8fe0 0x559803ff93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:48.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.243+0000 7fd6a7412e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x559803f43c00 con 0x559803ff98f0 2023-10-05T19:40:48.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.243+0000 7fd6a7412e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x559803f00ee0 con 0x559803ff8fe0 2023-10-05T19:40:48.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.243+0000 7fd6a4f47700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x559803ff8fe0 0x559803ff93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:48.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.243+0000 7fd6a4746700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x559803ff98f0 0x559803ffa390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:60192/0 (socket says 172.21.15.12:60192) 2023-10-05T19:40:48.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.243+0000 7fd6a4746700 1 -- 172.21.15.12:0/494695954 learned_addr learned my addr 172.21.15.12:0/494695954 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:48.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.243+0000 7fd6a4f47700 1 -- 172.21.15.12:0/494695954 >> v1:172.21.15.8:6789/0 conn(0x559803ff98f0 legacy=0x559803ffa390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:48.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.243+0000 7fd6a3f45700 1 -- 172.21.15.12:0/494695954 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4142544329 0 0) 0x559803f43c00 con 0x559803ff98f0 2023-10-05T19:40:48.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.243+0000 7fd6a4f47700 1 -- 172.21.15.12:0/494695954 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559803ffaaa0 con 0x559803ff8fe0 2023-10-05T19:40:48.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.243+0000 7fd6a4f47700 1 --2- 172.21.15.12:0/494695954 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559803ff8fe0 0x559803ff93b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd69400a5d0 tx=0x7fd694005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea22b97df2f8dab9 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:48.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.243+0000 7fd6a3f45700 1 -- 172.21.15.12:0/494695954 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd6940086a0 con 0x559803ff8fe0 2023-10-05T19:40:48.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.243+0000 7fd6a3f45700 1 -- 172.21.15.12:0/494695954 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd694008800 con 0x559803ff8fe0 2023-10-05T19:40:48.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.247+0000 7fd6a3f45700 1 -- 172.21.15.12:0/494695954 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd694008ad0 con 0x559803ff8fe0 2023-10-05T19:40:48.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.247+0000 7fd6a7412e40 1 -- 172.21.15.12:0/494695954 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559803ff8fe0 msgr2=0x559803ff93b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:48.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.247+0000 7fd6a7412e40 1 --2- 172.21.15.12:0/494695954 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559803ff8fe0 0x559803ff93b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd69400a5d0 tx=0x7fd694005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:48.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.247+0000 7fd6a7412e40 1 -- 172.21.15.12:0/494695954 shutdown_connections 2023-10-05T19:40:48.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.247+0000 7fd6a7412e40 1 --2- 172.21.15.12:0/494695954 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559803ff8fe0 0x559803ff93b0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:48.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.247+0000 7fd6a7412e40 1 -- 172.21.15.12:0/494695954 >> 172.21.15.12:0/494695954 conn(0x559803f6c830 msgr2=0x559803f6cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:48.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.247+0000 7fd6a7412e40 1 -- 172.21.15.12:0/494695954 shutdown_connections 2023-10-05T19:40:48.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.247+0000 7fd6a7412e40 1 -- 172.21.15.12:0/494695954 wait complete. 2023-10-05T19:40:48.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.247+0000 7fd6a7412e40 1 Processor -- start 2023-10-05T19:40:48.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.247+0000 7fd6a7412e40 1 -- start start 2023-10-05T19:40:48.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.247+0000 7fd6a7412e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559803ff8fe0 0x559804017ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:48.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.247+0000 7fd6a7412e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559803ec4e30 con 0x559803ff8fe0 2023-10-05T19:40:48.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.247+0000 7fd6a4f47700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559803ff8fe0 0x559804017ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:48.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.247+0000 7fd6a4f47700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559803ff8fe0 0x559804017ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58372/0 (socket says 172.21.15.12:58372) 2023-10-05T19:40:48.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.247+0000 7fd6a4f47700 1 -- 172.21.15.12:0/4054191748 learned_addr learned my addr 172.21.15.12:0/4054191748 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:48.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.247+0000 7fd6a4f47700 1 -- 172.21.15.12:0/4054191748 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd694005cf0 con 0x559803ff8fe0 2023-10-05T19:40:48.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.247+0000 7fd6a4f47700 1 --2- 172.21.15.12:0/4054191748 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559803ff8fe0 0x559804017ef0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd694005cc0 tx=0x7fd694015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=458d7902ffd2d1cf server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:48.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.247+0000 7fd6a2f43700 1 -- 172.21.15.12:0/4054191748 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd694008970 con 0x559803ff8fe0 2023-10-05T19:40:48.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.247+0000 7fd6a2f43700 1 -- 172.21.15.12:0/4054191748 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd694008ad0 con 0x559803ff8fe0 2023-10-05T19:40:48.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.247+0000 7fd6a2f43700 1 -- 172.21.15.12:0/4054191748 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd694008dc0 con 0x559803ff8fe0 2023-10-05T19:40:48.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.251+0000 7fd6a7412e40 1 -- 172.21.15.12:0/4054191748 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559803ff8fe0 msgr2=0x559804017ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:48.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.251+0000 7fd6a7412e40 1 --2- 172.21.15.12:0/4054191748 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559803ff8fe0 0x559804017ef0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd694005cc0 tx=0x7fd694015000 comp rx=0 tx=0).stop 2023-10-05T19:40:48.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.251+0000 7fd6a7412e40 1 -- 172.21.15.12:0/4054191748 shutdown_connections 2023-10-05T19:40:48.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.251+0000 7fd6a7412e40 1 --2- 172.21.15.12:0/4054191748 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559803ff8fe0 0x559804017ef0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:48.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.251+0000 7fd6a7412e40 1 -- 172.21.15.12:0/4054191748 >> 172.21.15.12:0/4054191748 conn(0x559803f6c830 msgr2=0x559804013ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:48.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.251+0000 7fd6a7412e40 1 -- 172.21.15.12:0/4054191748 shutdown_connections 2023-10-05T19:40:48.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.251+0000 7fd6a7412e40 1 -- 172.21.15.12:0/4054191748 wait complete. 2023-10-05T19:40:48.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.251+0000 7fd6a7412e40 1 Processor -- start 2023-10-05T19:40:48.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.251+0000 7fd6a7412e40 1 -- start start 2023-10-05T19:40:48.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.251+0000 7fd6a7412e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559803ff8fe0 0x55980401eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:48.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.251+0000 7fd6a7412e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559803e97ed0 con 0x559803ff8fe0 2023-10-05T19:40:48.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.251+0000 7fd6a4f47700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559803ff8fe0 0x55980401eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:48.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.251+0000 7fd6a4f47700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559803ff8fe0 0x55980401eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58382/0 (socket says 172.21.15.12:58382) 2023-10-05T19:40:48.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.251+0000 7fd6a4f47700 1 -- 172.21.15.12:0/1548698618 learned_addr learned my addr 172.21.15.12:0/1548698618 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:48.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.255+0000 7fd6a4f47700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559803ffaaa0 con 0x559803ff8fe0 2023-10-05T19:40:48.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.255+0000 7fd6a4f47700 1 --2- 172.21.15.12:0/1548698618 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559803ff8fe0 0x55980401eb00 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd69401e040 tx=0x7fd69401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:48.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.255+0000 7fd6a1740700 1 -- 172.21.15.12:0/1548698618 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd6940086a0 con 0x559803ff8fe0 2023-10-05T19:40:48.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.255+0000 7fd6a1740700 1 -- 172.21.15.12:0/1548698618 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd69402a070 con 0x559803ff8fe0 2023-10-05T19:40:48.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.255+0000 7fd6a1740700 1 -- 172.21.15.12:0/1548698618 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd694012cd0 con 0x559803ff8fe0 2023-10-05T19:40:48.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.255+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55980401f040 con 0x559803ff8fe0 2023-10-05T19:40:48.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.255+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55980401f4e0 con 0x559803ff8fe0 2023-10-05T19:40:48.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.255+0000 7fd6a7412e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:48.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.255+0000 7fd6a1740700 1 -- 172.21.15.12:0/1548698618 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fd694019070 con 0x559803ff8fe0 2023-10-05T19:40:48.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.255+0000 7fd6a1740700 1 --2- 172.21.15.12:0/1548698618 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd684030c50 0x7fd684033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:48.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.255+0000 7fd6a1740700 1 -- 172.21.15.12:0/1548698618 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7fd69402c070 con 0x559803ff8fe0 2023-10-05T19:40:48.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.255+0000 7fd6a4746700 1 --2- 172.21.15.12:0/1548698618 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd684030c50 0x7fd684033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:48.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.259+0000 7fd6a7412e40 1 --2- 172.21.15.12:0/1548698618 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x559803ff98f0 0x55980402a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:48.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.259+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55980402ac60 con 0x559803ff98f0 2023-10-05T19:40:48.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.259+0000 7fd6a5748700 1 --2- 172.21.15.12:0/1548698618 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x559803ff98f0 0x55980402a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:48.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.259+0000 7fd6a4746700 1 --2- 172.21.15.12:0/1548698618 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd684030c50 0x7fd684033100 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd68c0011e0 tx=0x7fd68c0013a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:48.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.259+0000 7fd6a5748700 1 --2- 172.21.15.12:0/1548698618 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x559803ff98f0 0x55980402a5f0 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:48.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.259+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559803ff98f0 2023-10-05T19:40:48.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.259+0000 7fd6a7412e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:48.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.259+0000 7fd6a7412e40 20 realm 2023-10-05T19:40:48.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.259+0000 7fd6a7412e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:48.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.259+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804027c50 con 0x559803ff98f0 2023-10-05T19:40:48.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.259+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559803ff98f0 2023-10-05T19:40:48.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.259+0000 7fd6a7412e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:48.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.259+0000 7fd6a7412e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:48.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.259+0000 7fd6a7412e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:48.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.259+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55980402b740 con 0x559803ff98f0 2023-10-05T19:40:48.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.259+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559803ff98f0 2023-10-05T19:40:48.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.259+0000 7fd6a7412e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:48.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.259+0000 7fd6a7412e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:48.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.259+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55980402b350 con 0x559803ff98f0 2023-10-05T19:40:48.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.259+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559803ff98f0 2023-10-05T19:40:48.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.259+0000 7fd6a7412e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:48.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.259+0000 7fd6a7412e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:48.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.259+0000 7fd6a7412e40 1 --2- 172.21.15.12:0/1548698618 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55980402cae0 0x55980402eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:48.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.259+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55980402f400 con 0x55980402cae0 2023-10-05T19:40:48.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.259+0000 7fd6a4f47700 1 --2- 172.21.15.12:0/1548698618 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55980402cae0 0x55980402eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:48.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.263+0000 7fd6a4f47700 1 --2- 172.21.15.12:0/1548698618 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55980402cae0 0x55980402eec0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:48.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.263+0000 7fd6a4f47700 1 -- 172.21.15.12:0/1548698618 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fd694042040 con 0x55980402cae0 2023-10-05T19:40:48.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.263+0000 7fd6a7412e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:48.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.263+0000 7fd6a7412e40 20 searching for the correct realm 2023-10-05T19:40:48.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.263+0000 7fd6a7412e40 1 --2- 172.21.15.12:0/1548698618 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x559804031280 0x559804033690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:48.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.263+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804033d40 con 0x559804031280 2023-10-05T19:40:48.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.263+0000 7fd6a4746700 1 --2- 172.21.15.12:0/1548698618 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x559804031280 0x559804033690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:48.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.263+0000 7fd6a4746700 1 --2- 172.21.15.12:0/1548698618 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x559804031280 0x559804033690 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:48.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.267+0000 7fd6a4746700 1 -- 172.21.15.12:0/1548698618 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fd68c006070 con 0x559804031280 2023-10-05T19:40:48.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.267+0000 7fd6a4746700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804033d40 con 0x55980402cae0 2023-10-05T19:40:48.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.267+0000 7fd6a4f47700 1 -- 172.21.15.12:0/1548698618 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd694042040 con 0x55980402cae0 2023-10-05T19:40:48.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.267+0000 7fd6a4f47700 1 --2- 172.21.15.12:0/1548698618 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fd694007030 0x7fd6940115e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:48.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.267+0000 7fd6a4f47700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804033d40 con 0x7fd694007030 2023-10-05T19:40:48.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.267+0000 7fd6a5748700 1 --2- 172.21.15.12:0/1548698618 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fd694007030 0x7fd6940115e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:48.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.267+0000 7fd6a5748700 1 --2- 172.21.15.12:0/1548698618 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fd694007030 0x7fd6940115e0 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:48.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.267+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd69c00a5a0 con 0x7fd694007030 2023-10-05T19:40:48.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.271+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804033d40 con 0x559803ff98f0 2023-10-05T19:40:48.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.271+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559803ff98f0 2023-10-05T19:40:48.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.271+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804033d40 con 0x559803ff98f0 2023-10-05T19:40:48.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.271+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559803ff98f0 2023-10-05T19:40:48.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.271+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804033d40 con 0x559803ff98f0 2023-10-05T19:40:48.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.271+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(11 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559803ff98f0 2023-10-05T19:40:48.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.271+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804033d40 con 0x7fd694007030 2023-10-05T19:40:48.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.271+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd69c00a5a0 con 0x7fd694007030 2023-10-05T19:40:48.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.271+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804033d40 con 0x55980402cae0 2023-10-05T19:40:48.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.271+0000 7fd6a4f47700 1 -- 172.21.15.12:0/1548698618 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd694042040 con 0x55980402cae0 2023-10-05T19:40:48.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.271+0000 7fd6a4f47700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804033d40 con 0x55980402cae0 2023-10-05T19:40:48.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.275+0000 7fd6a4f47700 1 -- 172.21.15.12:0/1548698618 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd694042040 con 0x55980402cae0 2023-10-05T19:40:48.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.275+0000 7fd6a4f47700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804033d40 con 0x559803ff98f0 2023-10-05T19:40:48.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.275+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559803ff98f0 2023-10-05T19:40:48.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.275+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804033d40 con 0x7fd694007030 2023-10-05T19:40:48.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.275+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd69c00a5a0 con 0x7fd694007030 2023-10-05T19:40:48.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.275+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804033d40 con 0x559803ff98f0 2023-10-05T19:40:48.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.275+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559803ff98f0 2023-10-05T19:40:48.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.275+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804033d40 con 0x55980402cae0 2023-10-05T19:40:48.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.275+0000 7fd6a4f47700 1 -- 172.21.15.12:0/1548698618 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(18 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd694042040 con 0x55980402cae0 2023-10-05T19:40:48.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.275+0000 7fd6a4f47700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804033d40 con 0x559804031280 2023-10-05T19:40:48.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.279+0000 7fd6a4746700 1 -- 172.21.15.12:0/1548698618 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd68c006070 con 0x559804031280 2023-10-05T19:40:48.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.279+0000 7fd6a4746700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804033d40 con 0x559804031280 2023-10-05T19:40:48.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.279+0000 7fd6a4746700 1 -- 172.21.15.12:0/1548698618 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd68c006070 con 0x559804031280 2023-10-05T19:40:48.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.279+0000 7fd6a4746700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804033d40 con 0x7fd694007030 2023-10-05T19:40:48.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.279+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd69c00a5a0 con 0x7fd694007030 2023-10-05T19:40:48.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.279+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804033d40 con 0x559804031280 2023-10-05T19:40:48.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.279+0000 7fd6a4746700 1 -- 172.21.15.12:0/1548698618 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd68c006070 con 0x559804031280 2023-10-05T19:40:48.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.279+0000 7fd6a4746700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804033d40 con 0x559804031280 2023-10-05T19:40:48.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.283+0000 7fd6a4746700 1 -- 172.21.15.12:0/1548698618 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd68c006070 con 0x559804031280 2023-10-05T19:40:48.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.283+0000 7fd6a4746700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804033d40 con 0x559803ff98f0 2023-10-05T19:40:48.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.283+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559803ff98f0 2023-10-05T19:40:48.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.283+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804033d40 con 0x559803ff98f0 2023-10-05T19:40:48.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.283+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559803ff98f0 2023-10-05T19:40:48.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.283+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804033d40 con 0x7fd694007030 2023-10-05T19:40:48.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.283+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd69c00a5a0 con 0x7fd694007030 2023-10-05T19:40:48.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.283+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804033d40 con 0x559804031280 2023-10-05T19:40:48.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.283+0000 7fd6a4746700 1 -- 172.21.15.12:0/1548698618 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd68c006070 con 0x559804031280 2023-10-05T19:40:48.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.283+0000 7fd6a4746700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804033d40 con 0x55980402cae0 2023-10-05T19:40:48.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.287+0000 7fd6a4f47700 1 -- 172.21.15.12:0/1548698618 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd694042040 con 0x55980402cae0 2023-10-05T19:40:48.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.287+0000 7fd6a4f47700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804033d40 con 0x559803ff98f0 2023-10-05T19:40:48.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.287+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559803ff98f0 2023-10-05T19:40:48.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.287+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804033d40 con 0x7fd694007030 2023-10-05T19:40:48.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.287+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd69c00a5a0 con 0x7fd694007030 2023-10-05T19:40:48.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.287+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804033d40 con 0x559804031280 2023-10-05T19:40:48.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.287+0000 7fd6a4746700 1 -- 172.21.15.12:0/1548698618 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd68c006070 con 0x559804031280 2023-10-05T19:40:48.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.287+0000 7fd6a4746700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804033d40 con 0x7fd694007030 2023-10-05T19:40:48.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.287+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd69c00a5a0 con 0x7fd694007030 2023-10-05T19:40:48.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.291+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804033d40 con 0x55980402cae0 2023-10-05T19:40:48.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.291+0000 7fd6a4f47700 1 -- 172.21.15.12:0/1548698618 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd694042040 con 0x55980402cae0 2023-10-05T19:40:48.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.291+0000 7fd6a4f47700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804033d40 con 0x559804031280 2023-10-05T19:40:48.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.291+0000 7fd6a4746700 1 -- 172.21.15.12:0/1548698618 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd68c006070 con 0x559804031280 2023-10-05T19:40:48.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.291+0000 7fd6a4746700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804033d40 con 0x7fd694007030 2023-10-05T19:40:48.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.291+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd69c00a5a0 con 0x7fd694007030 2023-10-05T19:40:48.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.291+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804033d40 con 0x559803ff98f0 2023-10-05T19:40:48.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.291+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559803ff98f0 2023-10-05T19:40:48.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.291+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804033d40 con 0x55980402cae0 2023-10-05T19:40:48.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.295+0000 7fd6a4f47700 1 -- 172.21.15.12:0/1548698618 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(37 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fd694042040 con 0x55980402cae0 2023-10-05T19:40:48.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.295+0000 7fd6a7412e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:48.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.295+0000 7fd6a7412e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:48.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.295+0000 7fd6a7412e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:48.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.295+0000 7fd6a7412e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:48.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.295+0000 7fd6a7412e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:48.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.295+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804030030 con 0x559803ff98f0 2023-10-05T19:40:48.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.295+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559803ff98f0 2023-10-05T19:40:48.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.295+0000 7fd6a7412e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:48.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.295+0000 7fd6a7412e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:48.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.295+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55980402fc60 con 0x559803ff98f0 2023-10-05T19:40:48.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.295+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559803ff98f0 2023-10-05T19:40:48.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.295+0000 7fd6a7412e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:48.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.295+0000 7fd6a7412e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:48.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.295+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040341b0 con 0x559804031280 2023-10-05T19:40:48.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.295+0000 7fd6a4746700 1 -- 172.21.15.12:0/1548698618 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fd68c006070 con 0x559804031280 2023-10-05T19:40:48.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.295+0000 7fd6a7412e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:48.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.295+0000 7fd6a7412e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:48.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.295+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804034560 con 0x559804031280 2023-10-05T19:40:48.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.295+0000 7fd6a4746700 1 -- 172.21.15.12:0/1548698618 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7fd68c006070 con 0x559804031280 2023-10-05T19:40:48.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.295+0000 7fd6a7412e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:48.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.295+0000 7fd6a7412e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:48.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.295+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804034910 con 0x559803ff98f0 2023-10-05T19:40:48.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.299+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559803ff98f0 2023-10-05T19:40:48.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.299+0000 7fd6a7412e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:48.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.299+0000 7fd6a7412e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:48.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.299+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804030550 con 0x559803ff98f0 2023-10-05T19:40:48.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.299+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559803ff98f0 2023-10-05T19:40:48.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.299+0000 7fd6a7412e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:48.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.299+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804030550 con 0x559804031280 2023-10-05T19:40:48.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.303+0000 7fd6a4746700 1 -- 172.21.15.12:0/1548698618 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(44 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fd68c006070 con 0x559804031280 2023-10-05T19:40:48.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.303+0000 7fd6a4746700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804030550 con 0x55980402cae0 2023-10-05T19:40:48.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.303+0000 7fd6a4f47700 1 -- 172.21.15.12:0/1548698618 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(45 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd694042040 con 0x55980402cae0 2023-10-05T19:40:48.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.303+0000 7fd6a4f47700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804030550 con 0x7fd694007030 2023-10-05T19:40:48.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.303+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(46 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd69c00a5a0 con 0x7fd694007030 2023-10-05T19:40:48.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.303+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804034910 con 0x559803ff98f0 2023-10-05T19:40:48.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.303+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(47 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559803ff98f0 2023-10-05T19:40:48.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.303+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804034910 con 0x559803ff98f0 2023-10-05T19:40:48.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.303+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(48 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559803ff98f0 2023-10-05T19:40:48.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.303+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804034910 con 0x559803ff98f0 2023-10-05T19:40:48.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.307+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(49 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559803ff98f0 2023-10-05T19:40:48.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.307+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804034910 con 0x7fd694007030 2023-10-05T19:40:48.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.307+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(50 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd69c00a5a0 con 0x7fd694007030 2023-10-05T19:40:48.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.307+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804034910 con 0x55980402cae0 2023-10-05T19:40:48.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.307+0000 7fd6a4f47700 1 -- 172.21.15.12:0/1548698618 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(51 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd694042040 con 0x55980402cae0 2023-10-05T19:40:48.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.307+0000 7fd6a4f47700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804034910 con 0x55980402cae0 2023-10-05T19:40:48.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.307+0000 7fd6a4f47700 1 -- 172.21.15.12:0/1548698618 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(52 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd694042040 con 0x55980402cae0 2023-10-05T19:40:48.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.307+0000 7fd6a4f47700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804034910 con 0x559803ff98f0 2023-10-05T19:40:48.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.307+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(53 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559803ff98f0 2023-10-05T19:40:48.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.311+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804034910 con 0x7fd694007030 2023-10-05T19:40:48.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.311+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(54 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd69c00a5a0 con 0x7fd694007030 2023-10-05T19:40:48.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.311+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804034910 con 0x559803ff98f0 2023-10-05T19:40:48.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.311+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(55 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559803ff98f0 2023-10-05T19:40:48.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.311+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804034910 con 0x55980402cae0 2023-10-05T19:40:48.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.311+0000 7fd6a4f47700 1 -- 172.21.15.12:0/1548698618 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(56 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd694042040 con 0x55980402cae0 2023-10-05T19:40:48.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.311+0000 7fd6a4f47700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804034910 con 0x559804031280 2023-10-05T19:40:48.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.311+0000 7fd6a4746700 1 -- 172.21.15.12:0/1548698618 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(57 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd68c006070 con 0x559804031280 2023-10-05T19:40:48.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.311+0000 7fd6a4746700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804034910 con 0x559804031280 2023-10-05T19:40:48.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.315+0000 7fd6a4746700 1 -- 172.21.15.12:0/1548698618 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(58 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd68c006070 con 0x559804031280 2023-10-05T19:40:48.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.315+0000 7fd6a4746700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804034910 con 0x7fd694007030 2023-10-05T19:40:48.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.315+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(59 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd69c00a5a0 con 0x7fd694007030 2023-10-05T19:40:48.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.315+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804034910 con 0x559804031280 2023-10-05T19:40:48.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.315+0000 7fd6a4746700 1 -- 172.21.15.12:0/1548698618 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(60 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd68c006070 con 0x559804031280 2023-10-05T19:40:48.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.315+0000 7fd6a4746700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804034910 con 0x559804031280 2023-10-05T19:40:48.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.315+0000 7fd6a4746700 1 -- 172.21.15.12:0/1548698618 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(61 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd68c006070 con 0x559804031280 2023-10-05T19:40:48.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.315+0000 7fd6a4746700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804034910 con 0x559803ff98f0 2023-10-05T19:40:48.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.315+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(62 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559803ff98f0 2023-10-05T19:40:48.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.319+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804034910 con 0x559803ff98f0 2023-10-05T19:40:48.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.319+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(63 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559803ff98f0 2023-10-05T19:40:48.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.319+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804034910 con 0x7fd694007030 2023-10-05T19:40:48.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.319+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(64 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd69c00a5a0 con 0x7fd694007030 2023-10-05T19:40:48.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.319+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804034910 con 0x559804031280 2023-10-05T19:40:48.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.319+0000 7fd6a4746700 1 -- 172.21.15.12:0/1548698618 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(65 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd68c006070 con 0x559804031280 2023-10-05T19:40:48.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.319+0000 7fd6a4746700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804034910 con 0x55980402cae0 2023-10-05T19:40:48.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.319+0000 7fd6a4f47700 1 -- 172.21.15.12:0/1548698618 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(66 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd694042040 con 0x55980402cae0 2023-10-05T19:40:48.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.319+0000 7fd6a4f47700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804034910 con 0x559803ff98f0 2023-10-05T19:40:48.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.323+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(67 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559803ff98f0 2023-10-05T19:40:48.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.323+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804034910 con 0x7fd694007030 2023-10-05T19:40:48.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.323+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(68 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd69c00a5a0 con 0x7fd694007030 2023-10-05T19:40:48.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.323+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804034910 con 0x559804031280 2023-10-05T19:40:48.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.323+0000 7fd6a4746700 1 -- 172.21.15.12:0/1548698618 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(69 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd68c006070 con 0x559804031280 2023-10-05T19:40:48.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.323+0000 7fd6a4746700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804034910 con 0x7fd694007030 2023-10-05T19:40:48.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.323+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(70 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd69c00a5a0 con 0x7fd694007030 2023-10-05T19:40:48.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.323+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804034910 con 0x55980402cae0 2023-10-05T19:40:48.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.327+0000 7fd6a4f47700 1 -- 172.21.15.12:0/1548698618 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(71 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd694042040 con 0x55980402cae0 2023-10-05T19:40:48.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.327+0000 7fd6a4f47700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804034910 con 0x559804031280 2023-10-05T19:40:48.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.327+0000 7fd6a4746700 1 -- 172.21.15.12:0/1548698618 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(72 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd68c006070 con 0x559804031280 2023-10-05T19:40:48.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.327+0000 7fd6a4746700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804034910 con 0x7fd694007030 2023-10-05T19:40:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.327+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(73 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd69c00a5a0 con 0x7fd694007030 2023-10-05T19:40:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.327+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804034910 con 0x559803ff98f0 2023-10-05T19:40:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.327+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(74 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559803ff98f0 2023-10-05T19:40:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.327+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804034910 con 0x55980402cae0 2023-10-05T19:40:48.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.331+0000 7fd6a4f47700 1 -- 172.21.15.12:0/1548698618 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(75 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fd694042040 con 0x55980402cae0 2023-10-05T19:40:48.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.331+0000 7fd6a7412e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:48.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.331+0000 7fd6a7412e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:48.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.331+0000 7fd6a7412e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:48.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.331+0000 7fd6a7412e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:48.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.331+0000 7fd6a7412e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:48.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.331+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804030550 con 0x559803ff98f0 2023-10-05T19:40:48.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.331+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559803ff98f0 2023-10-05T19:40:48.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.331+0000 7fd6a7412e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:48.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.331+0000 7fd6a7412e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:48.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.331+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040352b0 con 0x55980402cae0 2023-10-05T19:40:48.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.331+0000 7fd6a4f47700 1 -- 172.21.15.12:0/1548698618 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fd694042040 con 0x55980402cae0 2023-10-05T19:40:48.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.331+0000 7fd6a7412e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:48.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.331+0000 7fd6a7412e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:48.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.331+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804035ba0 con 0x559803ff98f0 2023-10-05T19:40:48.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.331+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559803ff98f0 2023-10-05T19:40:48.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.331+0000 7fd6a7412e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:48.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.331+0000 7fd6a7412e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:48.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.331+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804035ba0 con 0x559804031280 2023-10-05T19:40:48.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.335+0000 7fd6a4746700 1 -- 172.21.15.12:0/1548698618 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fd68c006070 con 0x559804031280 2023-10-05T19:40:48.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.335+0000 7fd6a7412e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:48.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.335+0000 7fd6a7412e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:48.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.335+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804036310 con 0x7fd694007030 2023-10-05T19:40:48.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.335+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x7fd694007030 2023-10-05T19:40:48.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.335+0000 7fd6a7412e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:48.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.335+0000 7fd6a7412e40 20 zone default found 2023-10-05T19:40:48.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.335+0000 7fd6a7412e40 4 Realm: () 2023-10-05T19:40:48.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.335+0000 7fd6a7412e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:48.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.335+0000 7fd6a7412e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:48.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.335+0000 7fd6a7412e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:48.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.335+0000 7fd6a7412e40 20 zonegroup default 2023-10-05T19:40:48.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.335+0000 7fd6a7412e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:48.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.335+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804036310 con 0x7fd694007030 2023-10-05T19:40:48.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.335+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x7fd694007030 2023-10-05T19:40:48.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.335+0000 7fd6a7412e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:48.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.335+0000 7fd6a7412e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:48.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.335+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804037c90 con 0x559804031280 2023-10-05T19:40:48.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.335+0000 7fd6a4746700 1 -- 172.21.15.12:0/1548698618 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fd68c006070 con 0x559804031280 2023-10-05T19:40:48.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.335+0000 7fd6a7412e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:48.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.335+0000 7fd6a7412e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:48.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.335+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804038040 con 0x559804031280 2023-10-05T19:40:48.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.339+0000 7fd6a4746700 1 -- 172.21.15.12:0/1548698618 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fd68c006070 con 0x559804031280 2023-10-05T19:40:48.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.339+0000 7fd6a7412e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:48.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.339+0000 7fd6a7412e40 20 started sync module instance, tier type = 2023-10-05T19:40:48.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.339+0000 7fd6a7412e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:48.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.339+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804037350 con 0x55980402cae0 2023-10-05T19:40:48.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.339+0000 7fd6a4f47700 1 -- 172.21.15.12:0/1548698618 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(84 notify.0 [create] v30'192 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd694042040 con 0x55980402cae0 2023-10-05T19:40:48.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.339+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94111390717616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040383f0 con 0x55980402cae0 2023-10-05T19:40:48.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.339+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804038c90 con 0x559803ff98f0 2023-10-05T19:40:48.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.343+0000 7fd6a4f47700 1 -- 172.21.15.12:0/1548698618 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(85 notify.0 [watch watch cookie 94111390717616] v30'193 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd694042040 con 0x55980402cae0 2023-10-05T19:40:48.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.343+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(86 notify.1 [create] v30'192 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559803ff98f0 2023-10-05T19:40:48.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.343+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94111390729200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804038d30 con 0x559803ff98f0 2023-10-05T19:40:48.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.343+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040395d0 con 0x559804031280 2023-10-05T19:40:48.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.347+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(87 notify.1 [watch watch cookie 94111390729200] v30'193 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559803ff98f0 2023-10-05T19:40:48.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.347+0000 7fd6a4746700 1 -- 172.21.15.12:0/1548698618 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(88 notify.2 [create] v30'192 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd68c006070 con 0x559804031280 2023-10-05T19:40:48.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.347+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94111390730592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804039980 con 0x559804031280 2023-10-05T19:40:48.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.347+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55980403a220 con 0x55980402cae0 2023-10-05T19:40:48.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.351+0000 7fd6a4f47700 1 -- 172.21.15.12:0/1548698618 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(90 notify.3 [create] v30'192 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd694042040 con 0x55980402cae0 2023-10-05T19:40:48.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.351+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94111390735952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55980403a7d0 con 0x55980402cae0 2023-10-05T19:40:48.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.351+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55980403b360 con 0x559803ff98f0 2023-10-05T19:40:48.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.351+0000 7fd6a4746700 1 -- 172.21.15.12:0/1548698618 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(89 notify.2 [watch watch cookie 94111390730592] v30'193 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd68c006070 con 0x559804031280 2023-10-05T19:40:48.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.351+0000 7fd6a4f47700 1 -- 172.21.15.12:0/1548698618 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(91 notify.3 [watch watch cookie 94111390735952] v30'193 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd694042040 con 0x55980402cae0 2023-10-05T19:40:48.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.351+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(92 notify.4 [create] v30'192 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559803ff98f0 2023-10-05T19:40:48.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.351+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94111390731696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55980403ae30 con 0x559803ff98f0 2023-10-05T19:40:48.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.351+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55980403bc50 con 0x7fd694007030 2023-10-05T19:40:48.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.355+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(93 notify.4 [watch watch cookie 94111390731696] v30'193 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559803ff98f0 2023-10-05T19:40:48.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.355+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(94 notify.5 [create] v30'192 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x7fd694007030 2023-10-05T19:40:48.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.355+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94111390737056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55980403ae30 con 0x7fd694007030 2023-10-05T19:40:48.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.355+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55980403c060 con 0x7fd694007030 2023-10-05T19:40:48.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.359+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(95 notify.5 [watch watch cookie 94111390737056] v30'193 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x7fd694007030 2023-10-05T19:40:48.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.359+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(96 notify.6 [create] v30'192 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x7fd694007030 2023-10-05T19:40:48.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.359+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94111390741568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55980403c020 con 0x7fd694007030 2023-10-05T19:40:48.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.359+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55980403cac0 con 0x7fd694007030 2023-10-05T19:40:48.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.363+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(97 notify.6 [watch watch cookie 94111390741568] v30'193 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x7fd694007030 2023-10-05T19:40:48.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.363+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(98 notify.7 [create] v30'192 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x7fd694007030 2023-10-05T19:40:48.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.363+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94111390743904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55980403cac0 con 0x7fd694007030 2023-10-05T19:40:48.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.363+0000 7fd6a7412e40 20 add_watcher() i=0 2023-10-05T19:40:48.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.363+0000 7fd6a7412e40 20 add_watcher() i=1 2023-10-05T19:40:48.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.363+0000 7fd6a7412e40 20 add_watcher() i=2 2023-10-05T19:40:48.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.363+0000 7fd6a7412e40 20 add_watcher() i=3 2023-10-05T19:40:48.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.363+0000 7fd6a7412e40 20 add_watcher() i=4 2023-10-05T19:40:48.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.363+0000 7fd6a7412e40 20 add_watcher() i=5 2023-10-05T19:40:48.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.363+0000 7fd6a7412e40 20 add_watcher() i=6 2023-10-05T19:40:48.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.363+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(99 notify.7 [watch watch cookie 94111390743904] v30'193 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x7fd694007030 2023-10-05T19:40:48.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.363+0000 7fd6a7412e40 20 add_watcher() i=7 2023-10-05T19:40:48.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.363+0000 7fd6a7412e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:48.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.363+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55980403fc60 con 0x559803ff98f0 2023-10-05T19:40:48.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.367+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559803ff98f0 2023-10-05T19:40:48.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.367+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94111390746432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55980403d1f0 con 0x559803ff98f0 2023-10-05T19:40:48.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.367+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94111390746432] v30'178 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559803ff98f0 2023-10-05T19:40:48.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.367+0000 7fd6767e4700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:48.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.367+0000 7fd6a7412e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:48.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.367+0000 7fd6a7412e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:48.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.367+0000 7fd6a7412e40 1 Processor -- start 2023-10-05T19:40:48.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.367+0000 7fd6a7412e40 1 -- start start 2023-10-05T19:40:48.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.371+0000 7fd6a7412e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55980404a550 0x55980404a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:48.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.371+0000 7fd6a7412e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55980404ae60 con 0x55980404a550 2023-10-05T19:40:48.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.371+0000 7fd6a4746700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55980404a550 0x55980404a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:48.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.371+0000 7fd6a4746700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55980404a550 0x55980404a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58386/0 (socket says 172.21.15.12:58386) 2023-10-05T19:40:48.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.371+0000 7fd6a4746700 1 -- 172.21.15.12:0/3657724624 learned_addr learned my addr 172.21.15.12:0/3657724624 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:48.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.371+0000 7fd6a4746700 1 -- 172.21.15.12:0/3657724624 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55980404afa0 con 0x55980404a550 2023-10-05T19:40:48.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.371+0000 7fd6a4746700 1 --2- 172.21.15.12:0/3657724624 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55980404a550 0x55980404a920 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd68c000fb0 tx=0x7fd68c01e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=834c38b6f5105c1f server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:48.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.371+0000 7fd675fe3700 1 -- 172.21.15.12:0/3657724624 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd68c030040 con 0x55980404a550 2023-10-05T19:40:48.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.371+0000 7fd675fe3700 1 -- 172.21.15.12:0/3657724624 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd68c00ed90 con 0x55980404a550 2023-10-05T19:40:48.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.371+0000 7fd675fe3700 1 -- 172.21.15.12:0/3657724624 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd68c021040 con 0x55980404a550 2023-10-05T19:40:48.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.371+0000 7fd6a7412e40 1 -- 172.21.15.12:0/3657724624 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55980404a550 msgr2=0x55980404a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:48.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.371+0000 7fd6a7412e40 1 --2- 172.21.15.12:0/3657724624 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55980404a550 0x55980404a920 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd68c000fb0 tx=0x7fd68c01e040 comp rx=0 tx=0).stop 2023-10-05T19:40:48.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.371+0000 7fd6a7412e40 1 -- 172.21.15.12:0/3657724624 shutdown_connections 2023-10-05T19:40:48.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.371+0000 7fd6a7412e40 1 --2- 172.21.15.12:0/3657724624 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55980404a550 0x55980404a920 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:48.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.371+0000 7fd6a7412e40 1 -- 172.21.15.12:0/3657724624 >> 172.21.15.12:0/3657724624 conn(0x559804045910 msgr2=0x559804043a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:48.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.371+0000 7fd6a7412e40 1 -- 172.21.15.12:0/3657724624 shutdown_connections 2023-10-05T19:40:48.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.371+0000 7fd6a7412e40 1 -- 172.21.15.12:0/3657724624 wait complete. 2023-10-05T19:40:48.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.371+0000 7fd6a7412e40 1 Processor -- start 2023-10-05T19:40:48.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.375+0000 7fd6a7412e40 1 -- start start 2023-10-05T19:40:48.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.375+0000 7fd6a7412e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55980404a550 0x5598040500e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:48.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.375+0000 7fd6a7412e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598040510e0 con 0x55980404a550 2023-10-05T19:40:48.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.375+0000 7fd6a4746700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55980404a550 0x5598040500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:48.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.375+0000 7fd6a4746700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55980404a550 0x5598040500e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58390/0 (socket says 172.21.15.12:58390) 2023-10-05T19:40:48.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.375+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 learned_addr learned my addr 172.21.15.12:0/2042177289 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:48.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.375+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd68c033040 con 0x55980404a550 2023-10-05T19:40:48.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.375+0000 7fd6a4746700 1 --2- 172.21.15.12:0/2042177289 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55980404a550 0x5598040500e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x559804028080 tx=0x7fd68c03c030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:48.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.375+0000 7fd5e7fff700 1 -- 172.21.15.12:0/2042177289 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd68c030070 con 0x55980404a550 2023-10-05T19:40:48.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.375+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5598040506e0 con 0x55980404a550 2023-10-05T19:40:48.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.375+0000 7fd5e7fff700 1 -- 172.21.15.12:0/2042177289 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd68c02b730 con 0x55980404a550 2023-10-05T19:40:48.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.375+0000 7fd5e7fff700 1 -- 172.21.15.12:0/2042177289 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd68c021040 con 0x55980404a550 2023-10-05T19:40:48.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.375+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559804050ac0 con 0x55980404a550 2023-10-05T19:40:48.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.375+0000 7fd5e7fff700 1 -- 172.21.15.12:0/2042177289 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fd68c02b990 con 0x55980404a550 2023-10-05T19:40:48.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.379+0000 7fd5e7fff700 1 --2- 172.21.15.12:0/2042177289 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd5ec030b70 0x7fd5ec033020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:48.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.379+0000 7fd5e7fff700 1 -- 172.21.15.12:0/2042177289 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7fd68c04fbe0 con 0x55980404a550 2023-10-05T19:40:48.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.379+0000 7fd6a5748700 1 --2- 172.21.15.12:0/2042177289 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd5ec030b70 0x7fd5ec033020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:48.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.379+0000 7fd6a7412e40 1 --2- 172.21.15.12:0/2042177289 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55980405ad00 0x55980405d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:48.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.379+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55980405d7b0 con 0x55980405ad00 2023-10-05T19:40:48.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.379+0000 7fd6a4f47700 1 --2- 172.21.15.12:0/2042177289 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55980405ad00 0x55980405d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:48.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.379+0000 7fd6a5748700 1 --2- 172.21.15.12:0/2042177289 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd5ec030b70 0x7fd5ec033020 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x55980403d150 tx=0x7fd69c00aed0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:48.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.379+0000 7fd6a4f47700 1 --2- 172.21.15.12:0/2042177289 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55980405ad00 0x55980405d110 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:48.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.379+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v30'28 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fd694042040 con 0x55980405ad00 2023-10-05T19:40:48.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.379+0000 7fd6a7412e40 1 --2- 172.21.15.12:0/2042177289 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55980405e480 0x559804060860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:48.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.383+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804060f90 con 0x55980405e480 2023-10-05T19:40:48.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.383+0000 7fd6a4746700 1 --2- 172.21.15.12:0/2042177289 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55980405e480 0x559804060860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:48.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.383+0000 7fd6a4746700 1 --2- 172.21.15.12:0/2042177289 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55980405e480 0x559804060860 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:48.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.383+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v30'19 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fd68c054070 con 0x55980405e480 2023-10-05T19:40:48.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.383+0000 7fd6a7412e40 1 --2- 172.21.15.12:0/2042177289 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x559804061c60 0x559804064040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:48.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.383+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804064770 con 0x559804061c60 2023-10-05T19:40:48.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.383+0000 7fd6a5748700 1 --2- 172.21.15.12:0/2042177289 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x559804061c60 0x559804064040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:48.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.387+0000 7fd6a5748700 1 --2- 172.21.15.12:0/2042177289 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x559804061c60 0x559804064040 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:48.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.387+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v30'19 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.387+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804064dc0 con 0x55980405ad00 2023-10-05T19:40:48.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.391+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v30'179 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fd694042040 con 0x55980405ad00 2023-10-05T19:40:48.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.391+0000 7fd6a7412e40 1 --2- 172.21.15.12:0/2042177289 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x559804065a60 0x559804067e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:48.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.391+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040684a0 con 0x559804065a60 2023-10-05T19:40:48.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.391+0000 7fd6a4f47700 1 --2- 172.21.15.12:0/2042177289 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x559804065a60 0x559804067e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:48.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.391+0000 7fd6a4f47700 1 --2- 172.21.15.12:0/2042177289 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x559804065a60 0x559804067e40 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:48.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.395+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v30'47 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.395+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804068af0 con 0x559804061c60 2023-10-05T19:40:48.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.395+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v30'34 uv8 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.395+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804069110 con 0x55980405ad00 2023-10-05T19:40:48.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.399+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v30'21 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fd694042040 con 0x55980405ad00 2023-10-05T19:40:48.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.399+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804069110 con 0x55980405e480 2023-10-05T19:40:48.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.403+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v30'31 uv8 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.403+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804069650 con 0x559804065a60 2023-10-05T19:40:48.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.403+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v30'19 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.403+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040696e0 con 0x559804061c60 2023-10-05T19:40:48.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.407+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v30'25 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.407+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804069bd0 con 0x559804065a60 2023-10-05T19:40:48.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.411+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v30'48 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.411+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804069bd0 con 0x55980405e480 2023-10-05T19:40:48.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.411+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v30'22 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.411+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804069bd0 con 0x559804065a60 2023-10-05T19:40:48.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.415+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v30'19 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.415+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804069b10 con 0x55980405e480 2023-10-05T19:40:48.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.419+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v30'40 uv12 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.419+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55980406a000 con 0x55980405ad00 2023-10-05T19:40:48.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.423+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v30'180 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fd694042040 con 0x55980405ad00 2023-10-05T19:40:48.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.423+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55980406a000 con 0x559804061c60 2023-10-05T19:40:48.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.423+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v30'41 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.423+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55980406a000 con 0x559804065a60 2023-10-05T19:40:48.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.427+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v30'49 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.427+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55980406a000 con 0x559804061c60 2023-10-05T19:40:48.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.431+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v30'19 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.431+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55980406a000 con 0x55980405ad00 2023-10-05T19:40:48.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.435+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v30'39 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fd694042040 con 0x55980405ad00 2023-10-05T19:40:48.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.435+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804069f70 con 0x55980405ad00 2023-10-05T19:40:48.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.435+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v30'181 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fd694042040 con 0x55980405ad00 2023-10-05T19:40:48.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.435+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804069f70 con 0x559804061c60 2023-10-05T19:40:48.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.439+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v30'42 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.439+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804069fa0 con 0x559804065a60 2023-10-05T19:40:48.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.443+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v30'29 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.443+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55980406a030 con 0x559804061c60 2023-10-05T19:40:48.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.443+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v30'22 uv20 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.443+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55980406a0c0 con 0x559804061c60 2023-10-05T19:40:48.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.447+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v30'28 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.447+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55980406a0c0 con 0x559804061c60 2023-10-05T19:40:48.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.447+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v30'16 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.451+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55980406a0c0 con 0x559804061c60 2023-10-05T19:40:48.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.451+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v30'35 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.451+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55980406a0c0 con 0x55980405e480 2023-10-05T19:40:48.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.455+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v30'25 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.455+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55980406a0c0 con 0x559804061c60 2023-10-05T19:40:48.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.455+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v30'19 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.455+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55980406a0c0 con 0x55980405e480 2023-10-05T19:40:48.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.459+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v30'32 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.459+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55980406a0c0 con 0x559804061c60 2023-10-05T19:40:48.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.463+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v30'43 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.463+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55980406a0c0 con 0x55980405e480 2023-10-05T19:40:48.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.463+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v30'41 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.463+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55980406a0c0 con 0x559804065a60 2023-10-05T19:40:48.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.467+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v30'25 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.471+0000 7fd5e57fa700 20 reqs_thread_entry: start 2023-10-05T19:40:48.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.471+0000 7fd5deffd700 20 reqs_thread_entry: start 2023-10-05T19:40:48.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.471+0000 7fd6a7412e40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:48.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.471+0000 7fd6a7412e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:48.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.471+0000 7fd5dd7fa700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:41:18 2023 2023-10-05T19:40:48.481 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:48.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.471+0000 7fd5dcff9700 20 reqs_thread_entry: start 2023-10-05T19:40:48.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.471+0000 7fd6a7412e40 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=1 2023-10-05T19:40:48.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.471+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:33 6.d 6:b019ea83:gc::gc.18:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c9c70 con 0x55980405ad00 2023-10-05T19:40:48.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.475+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(33 gc.18 [call] v30'40 uv40 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fd694042040 con 0x55980405ad00 2023-10-05T19:40:48.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.475+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:34 6.d 6:b019ea83:gc::gc.18:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c9c70 con 0x55980405ad00 2023-10-05T19:40:48.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.475+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(34 gc.18 [call out=15b] v0'0 uv40 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd694042040 con 0x55980405ad00 2023-10-05T19:40:48.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.475+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.475+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:35 6.d 6:b019ea83:gc::gc.18:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c8c60 con 0x55980405ad00 2023-10-05T19:40:48.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.475+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(35 gc.18 [call out=24b] v0'0 uv40 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fd694042040 con 0x55980405ad00 2023-10-05T19:40:48.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.475+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 6.d 6:b019ea83:gc::gc.18:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c9c70 con 0x55980405ad00 2023-10-05T19:40:48.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.479+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(36 gc.18 [call out=15b] v0'0 uv40 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd694042040 con 0x55980405ad00 2023-10-05T19:40:48.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.479+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2023-10-05T19:40:48.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.479+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:37 6.d 6:b019ea83:gc::gc.18:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c9c70 con 0x55980405ad00 2023-10-05T19:40:48.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.479+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(37 gc.18 [call out=15b] v0'0 uv40 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd694042040 con 0x55980405ad00 2023-10-05T19:40:48.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.479+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.479+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 6.d 6:b019ea83:gc::gc.18:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c9c70 con 0x55980405ad00 2023-10-05T19:40:48.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.483+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(38 gc.18 [call] v30'41 uv41 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fd694042040 con 0x55980405ad00 2023-10-05T19:40:48.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.483+0000 7fd6a7412e40 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=1 2023-10-05T19:40:48.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.483+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 6.4 6:24c8e1ae:gc::gc.19:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c9c70 con 0x55980405ad00 2023-10-05T19:40:48.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.483+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(39 gc.19 [call] v30'182 uv182 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fd694042040 con 0x55980405ad00 2023-10-05T19:40:48.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.483+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:40 6.4 6:24c8e1ae:gc::gc.19:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c9c70 con 0x55980405ad00 2023-10-05T19:40:48.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.483+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(40 gc.19 [call out=15b] v0'0 uv182 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd694042040 con 0x55980405ad00 2023-10-05T19:40:48.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.483+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.483+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:41 6.4 6:24c8e1ae:gc::gc.19:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c8c60 con 0x55980405ad00 2023-10-05T19:40:48.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.487+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(41 gc.19 [call out=24b] v0'0 uv182 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fd694042040 con 0x55980405ad00 2023-10-05T19:40:48.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.487+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 6.4 6:24c8e1ae:gc::gc.19:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c9c70 con 0x55980405ad00 2023-10-05T19:40:48.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.487+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(42 gc.19 [call out=15b] v0'0 uv182 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd694042040 con 0x55980405ad00 2023-10-05T19:40:48.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.487+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2023-10-05T19:40:48.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.487+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 6.4 6:24c8e1ae:gc::gc.19:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c9c70 con 0x55980405ad00 2023-10-05T19:40:48.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.487+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(43 gc.19 [call out=15b] v0'0 uv182 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd694042040 con 0x55980405ad00 2023-10-05T19:40:48.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.487+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.487+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:44 6.4 6:24c8e1ae:gc::gc.19:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c9c70 con 0x55980405ad00 2023-10-05T19:40:48.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.491+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(44 gc.19 [call] v30'183 uv183 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fd694042040 con 0x55980405ad00 2023-10-05T19:40:48.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.491+0000 7fd6a7412e40 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=1 2023-10-05T19:40:48.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.491+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:45 6.7 6:e622f76a:gc::gc.20:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c9c70 con 0x559804061c60 2023-10-05T19:40:48.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.491+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(45 gc.20 [call] v30'44 uv44 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.491+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:46 6.7 6:e622f76a:gc::gc.20:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c9c70 con 0x559804061c60 2023-10-05T19:40:48.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.495+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(46 gc.20 [call out=15b] v0'0 uv44 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.495+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.495+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:47 6.7 6:e622f76a:gc::gc.20:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c8c60 con 0x559804061c60 2023-10-05T19:40:48.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.495+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(47 gc.20 [call out=24b] v0'0 uv44 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.495+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:48 6.7 6:e622f76a:gc::gc.20:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c9c70 con 0x559804061c60 2023-10-05T19:40:48.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.495+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(48 gc.20 [call out=15b] v0'0 uv44 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.495+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2023-10-05T19:40:48.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.495+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:49 6.7 6:e622f76a:gc::gc.20:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c9c70 con 0x559804061c60 2023-10-05T19:40:48.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.495+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(49 gc.20 [call out=15b] v0'0 uv44 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.495+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.495+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:50 6.7 6:e622f76a:gc::gc.20:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c9c70 con 0x559804061c60 2023-10-05T19:40:48.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.499+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(50 gc.20 [call] v30'45 uv45 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.499+0000 7fd6a7412e40 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=1 2023-10-05T19:40:48.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.499+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 6.a 6:5127e642:gc::gc.21:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c9c70 con 0x559804065a60 2023-10-05T19:40:48.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.499+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(51 gc.21 [call] v30'30 uv30 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.499+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:52 6.a 6:5127e642:gc::gc.21:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c9c70 con 0x559804065a60 2023-10-05T19:40:48.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.503+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(52 gc.21 [call out=15b] v0'0 uv30 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.503+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.503+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:53 6.a 6:5127e642:gc::gc.21:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c8c60 con 0x559804065a60 2023-10-05T19:40:48.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.503+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(53 gc.21 [call out=24b] v0'0 uv30 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.503+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:54 6.a 6:5127e642:gc::gc.21:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c9c70 con 0x559804065a60 2023-10-05T19:40:48.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.503+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(54 gc.21 [call out=15b] v0'0 uv30 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.503+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2023-10-05T19:40:48.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.503+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:55 6.a 6:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c9c70 con 0x559804065a60 2023-10-05T19:40:48.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.503+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(55 gc.21 [call out=15b] v0'0 uv30 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.503+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.503+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 6.a 6:5127e642:gc::gc.21:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c9c70 con 0x559804065a60 2023-10-05T19:40:48.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.507+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(56 gc.21 [call] v30'31 uv31 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.507+0000 7fd6a7412e40 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=1 2023-10-05T19:40:48.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.507+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 6.11 6:8eb99c53:gc::gc.22:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c9c70 con 0x559804061c60 2023-10-05T19:40:48.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.507+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(57 gc.22 [call] v30'23 uv23 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.507+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 6.11 6:8eb99c53:gc::gc.22:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c9c70 con 0x559804061c60 2023-10-05T19:40:48.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.507+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(58 gc.22 [call out=15b] v0'0 uv23 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.507+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.507+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:59 6.11 6:8eb99c53:gc::gc.22:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c8c60 con 0x559804061c60 2023-10-05T19:40:48.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.511+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(59 gc.22 [call out=24b] v0'0 uv23 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.511+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 6.11 6:8eb99c53:gc::gc.22:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c9c70 con 0x559804061c60 2023-10-05T19:40:48.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.511+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(60 gc.22 [call out=15b] v0'0 uv23 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.511+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2023-10-05T19:40:48.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.511+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 6.11 6:8eb99c53:gc::gc.22:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c9c70 con 0x559804061c60 2023-10-05T19:40:48.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.511+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(61 gc.22 [call out=1702b] v0'0 uv23 ondisk = 0) v8 ==== 149+0+1702 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.511+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2023-10-05T19:40:48.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.511+0000 7fd6a7412e40 20 garbage collection: RGWGC::process iterating over entry tag='93ac49f4-ff49-4f99-811b-6453b28decd5.4219.17885726856970346626', time=2023-10-05T19:40:47.773897+0000, chain.objs.size()=7 2023-10-05T19:40:48.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.511+0000 7fd6a7412e40 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1__shadow_.XR8N8VQAMDt_LCHQhT8BOCUfm_xM1JH_1 2023-10-05T19:40:48.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.511+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:62 3.2c 3:35992e54:::93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1__shadow_.XR8N8VQAMDt_LCHQhT8BOCUfm_xM1JH_1:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c9c70 con 0x55980405e480 2023-10-05T19:40:48.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.511+0000 7fd6a7412e40 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1__shadow_.XR8N8VQAMDt_LCHQhT8BOCUfm_xM1JH_2 2023-10-05T19:40:48.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.511+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:63 3.30 3:0d1af92a:::93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1__shadow_.XR8N8VQAMDt_LCHQhT8BOCUfm_xM1JH_2:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040cc530 con 0x559804061c60 2023-10-05T19:40:48.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.511+0000 7fd6a7412e40 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1__shadow_.XR8N8VQAMDt_LCHQhT8BOCUfm_xM1JH_3 2023-10-05T19:40:48.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.511+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:64 3.2f 3:f4c4df70:::93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1__shadow_.XR8N8VQAMDt_LCHQhT8BOCUfm_xM1JH_3:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040cee50 con 0x559804061c60 2023-10-05T19:40:48.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.511+0000 7fd6a7412e40 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1__shadow_.XR8N8VQAMDt_LCHQhT8BOCUfm_xM1JH_4 2023-10-05T19:40:48.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.511+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:65 3.27 3:e5f45d23:::93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1__shadow_.XR8N8VQAMDt_LCHQhT8BOCUfm_xM1JH_4:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040cc530 con 0x55980405e480 2023-10-05T19:40:48.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.511+0000 7fd6a7412e40 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1__shadow_.XR8N8VQAMDt_LCHQhT8BOCUfm_xM1JH_5 2023-10-05T19:40:48.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.511+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:66 3.7 3:e328cb0c:::93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1__shadow_.XR8N8VQAMDt_LCHQhT8BOCUfm_xM1JH_5:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040d3c80 con 0x559804061c60 2023-10-05T19:40:48.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.511+0000 7fd6a7412e40 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1__shadow_.XR8N8VQAMDt_LCHQhT8BOCUfm_xM1JH_6 2023-10-05T19:40:48.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.511+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:67 3.2d 3:b5dd9c99:::93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1__shadow_.XR8N8VQAMDt_LCHQhT8BOCUfm_xM1JH_6:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040d6870 con 0x559804061c60 2023-10-05T19:40:48.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.511+0000 7fd6a7412e40 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1__shadow_.XR8N8VQAMDt_LCHQhT8BOCUfm_xM1JH_7 2023-10-05T19:40:48.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.511+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:68 3.3c 3:3c10e6b3:::93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1__shadow_.XR8N8VQAMDt_LCHQhT8BOCUfm_xM1JH_7:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040d9150 con 0x559804065a60 2023-10-05T19:40:48.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.519+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(64 93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1__shadow_.XR8N8VQAMDt_LCHQhT8BOCUfm_xM1JH_3 [call] v30'2 uv2 ondisk = 0) v8 ==== 230+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.519+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(65 93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1__shadow_.XR8N8VQAMDt_LCHQhT8BOCUfm_xM1JH_4 [call] v30'2 uv2 ondisk = 0) v8 ==== 230+0+0 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.519+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(68 93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1__shadow_.XR8N8VQAMDt_LCHQhT8BOCUfm_xM1JH_7 [call] v30'2 uv2 ondisk = 0) v8 ==== 230+0+0 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.519+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(63 93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1__shadow_.XR8N8VQAMDt_LCHQhT8BOCUfm_xM1JH_2 [call] v30'2 uv2 ondisk = 0) v8 ==== 230+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.519+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(66 93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1__shadow_.XR8N8VQAMDt_LCHQhT8BOCUfm_xM1JH_5 [call] v30'2 uv2 ondisk = 0) v8 ==== 230+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.519+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(62 93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1__shadow_.XR8N8VQAMDt_LCHQhT8BOCUfm_xM1JH_1 [call] v30'2 uv2 ondisk = 0) v8 ==== 230+0+0 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.519+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 27 ==== osd_op_reply(67 93ac49f4-ff49-4f99-811b-6453b28decd5.4221.1__shadow_.XR8N8VQAMDt_LCHQhT8BOCUfm_xM1JH_6 [call] v30'2 uv2 ondisk = 0) v8 ==== 230+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.519+0000 7fd6a7412e40 5 garbage collection: RGWGC::process removing entries, marker: 2023-10-05T19:40:48.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.519+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 6.11 6:8eb99c53:gc::gc.22:head [call rgw_gc.rgw_gc_queue_remove_entries in=47b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.523+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 28 ==== osd_op_reply(69 gc.22 [call] v30'24 uv24 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.523+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:70 6.11 6:8eb99c53:gc::gc.22:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.523+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 29 ==== osd_op_reply(70 gc.22 [call] v30'25 uv25 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.523+0000 7fd6a7412e40 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=1 2023-10-05T19:40:48.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.523+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:71 6.8 6:17d7e8ed:gc::gc.23:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.527+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 30 ==== osd_op_reply(71 gc.23 [call] v30'29 uv29 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.527+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 6.8 6:17d7e8ed:gc::gc.23:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.527+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 31 ==== osd_op_reply(72 gc.23 [call out=15b] v0'0 uv29 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.527+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.527+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:73 6.8 6:17d7e8ed:gc::gc.23:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040d5690 con 0x559804061c60 2023-10-05T19:40:48.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.527+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 32 ==== osd_op_reply(73 gc.23 [call out=24b] v0'0 uv29 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.527+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:74 6.8 6:17d7e8ed:gc::gc.23:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.527+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 33 ==== osd_op_reply(74 gc.23 [call out=15b] v0'0 uv29 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.527+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2023-10-05T19:40:48.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.527+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:75 6.8 6:17d7e8ed:gc::gc.23:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.527+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 34 ==== osd_op_reply(75 gc.23 [call out=15b] v0'0 uv29 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.527+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.527+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:76 6.8 6:17d7e8ed:gc::gc.23:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.531+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 35 ==== osd_op_reply(76 gc.23 [call] v30'30 uv30 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.531+0000 7fd6a7412e40 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=1 2023-10-05T19:40:48.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.531+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:77 6.1b 6:db9d130d:gc::gc.24:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.531+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 36 ==== osd_op_reply(77 gc.24 [call] v30'17 uv17 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.531+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:78 6.1b 6:db9d130d:gc::gc.24:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.531+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 37 ==== osd_op_reply(78 gc.24 [call out=15b] v0'0 uv17 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.531+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.531+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 6.1b 6:db9d130d:gc::gc.24:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040d55d0 con 0x559804061c60 2023-10-05T19:40:48.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.531+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 38 ==== osd_op_reply(79 gc.24 [call out=24b] v0'0 uv17 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.531+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:80 6.1b 6:db9d130d:gc::gc.24:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.531+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 39 ==== osd_op_reply(80 gc.24 [call out=15b] v0'0 uv17 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.531+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2023-10-05T19:40:48.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.531+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:81 6.1b 6:db9d130d:gc::gc.24:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.531+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 40 ==== osd_op_reply(81 gc.24 [call out=15b] v0'0 uv17 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.531+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.531+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.1b 6:db9d130d:gc::gc.24:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.535+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 41 ==== osd_op_reply(82 gc.24 [call] v30'18 uv18 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.535+0000 7fd6a7412e40 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=1 2023-10-05T19:40:48.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.535+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.9 6:921ecdb1:gc::gc.25:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.535+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 42 ==== osd_op_reply(83 gc.25 [call] v30'36 uv36 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.535+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:84 6.9 6:921ecdb1:gc::gc.25:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.539+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 43 ==== osd_op_reply(84 gc.25 [call out=15b] v0'0 uv36 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.539+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.539+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:85 6.9 6:921ecdb1:gc::gc.25:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040d55d0 con 0x559804061c60 2023-10-05T19:40:48.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.539+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 44 ==== osd_op_reply(85 gc.25 [call out=24b] v0'0 uv36 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.539+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:86 6.9 6:921ecdb1:gc::gc.25:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.539+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 45 ==== osd_op_reply(86 gc.25 [call out=15b] v0'0 uv36 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.539+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2023-10-05T19:40:48.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.539+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.539+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 46 ==== osd_op_reply(87 gc.25 [call out=15b] v0'0 uv36 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.539+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.539+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 6.9 6:921ecdb1:gc::gc.25:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.539+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 47 ==== osd_op_reply(88 gc.25 [call] v30'37 uv37 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.539+0000 7fd6a7412e40 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=1 2023-10-05T19:40:48.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.539+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:89 6.1f 6:fde4dd55:gc::gc.26:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405e480 2023-10-05T19:40:48.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.543+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(89 gc.26 [call] v30'26 uv26 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.543+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:90 6.1f 6:fde4dd55:gc::gc.26:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405e480 2023-10-05T19:40:48.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.543+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(90 gc.26 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.543+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.543+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:91 6.1f 6:fde4dd55:gc::gc.26:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040d55d0 con 0x55980405e480 2023-10-05T19:40:48.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.547+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(91 gc.26 [call out=24b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.547+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:92 6.1f 6:fde4dd55:gc::gc.26:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405e480 2023-10-05T19:40:48.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.547+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(92 gc.26 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.547+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2023-10-05T19:40:48.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.547+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:93 6.1f 6:fde4dd55:gc::gc.26:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405e480 2023-10-05T19:40:48.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.547+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(93 gc.26 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.547+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.547+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 6.1f 6:fde4dd55:gc::gc.26:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405e480 2023-10-05T19:40:48.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.551+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(94 gc.26 [call] v30'27 uv27 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.551+0000 7fd6a7412e40 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=1 2023-10-05T19:40:48.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.551+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:95 6.1a 6:5909e671:gc::gc.27:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.551+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 48 ==== osd_op_reply(95 gc.27 [call] v30'20 uv20 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.551+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:96 6.1a 6:5909e671:gc::gc.27:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.551+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 49 ==== osd_op_reply(96 gc.27 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.551+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.551+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:97 6.1a 6:5909e671:gc::gc.27:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040d55d0 con 0x559804061c60 2023-10-05T19:40:48.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.555+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 50 ==== osd_op_reply(97 gc.27 [call out=24b] v0'0 uv20 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.555+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:98 6.1a 6:5909e671:gc::gc.27:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.555+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 51 ==== osd_op_reply(98 gc.27 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.555+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2023-10-05T19:40:48.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.555+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:99 6.1a 6:5909e671:gc::gc.27:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.555+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 52 ==== osd_op_reply(99 gc.27 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.555+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.555+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:100 6.1a 6:5909e671:gc::gc.27:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.555+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 53 ==== osd_op_reply(100 gc.27 [call] v30'21 uv21 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.555+0000 7fd6a7412e40 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=1 2023-10-05T19:40:48.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.555+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:101 6.1d 6:b88e3271:gc::gc.28:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405e480 2023-10-05T19:40:48.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.559+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(101 gc.28 [call] v30'33 uv33 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.559+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:102 6.1d 6:b88e3271:gc::gc.28:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405e480 2023-10-05T19:40:48.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.559+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(102 gc.28 [call out=15b] v0'0 uv33 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.559+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.559+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:103 6.1d 6:b88e3271:gc::gc.28:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405e480 2023-10-05T19:40:48.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.563+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(103 gc.28 [call out=24b] v0'0 uv33 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.563+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:104 6.1d 6:b88e3271:gc::gc.28:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405e480 2023-10-05T19:40:48.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.563+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(104 gc.28 [call out=15b] v0'0 uv33 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.563+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2023-10-05T19:40:48.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.563+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:105 6.1d 6:b88e3271:gc::gc.28:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405e480 2023-10-05T19:40:48.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.563+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(105 gc.28 [call out=15b] v0'0 uv33 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.563+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.563+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:106 6.1d 6:b88e3271:gc::gc.28:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405e480 2023-10-05T19:40:48.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.567+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(106 gc.28 [call] v30'34 uv34 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.567+0000 7fd6a7412e40 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=1 2023-10-05T19:40:48.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.567+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:107 6.7 6:e47c9322:gc::gc.29:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.571+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 54 ==== osd_op_reply(107 gc.29 [call] v30'46 uv46 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.571+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:108 6.7 6:e47c9322:gc::gc.29:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.571+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 55 ==== osd_op_reply(108 gc.29 [call out=15b] v0'0 uv46 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.571+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.571+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:109 6.7 6:e47c9322:gc::gc.29:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040d55d0 con 0x559804061c60 2023-10-05T19:40:48.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.571+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 56 ==== osd_op_reply(109 gc.29 [call out=24b] v0'0 uv46 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.571+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:110 6.7 6:e47c9322:gc::gc.29:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.571+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 57 ==== osd_op_reply(110 gc.29 [call out=15b] v0'0 uv46 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.571+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2023-10-05T19:40:48.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.571+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:111 6.7 6:e47c9322:gc::gc.29:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.571+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 58 ==== osd_op_reply(111 gc.29 [call out=15b] v0'0 uv46 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.571+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.571+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:112 6.7 6:e47c9322:gc::gc.29:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.575+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 59 ==== osd_op_reply(112 gc.29 [call] v30'47 uv47 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.575+0000 7fd6a7412e40 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=1 2023-10-05T19:40:48.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.575+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:113 6.0 6:013d6c33:gc::gc.30:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405e480 2023-10-05T19:40:48.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.579+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(113 gc.30 [call] v30'42 uv42 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.579+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:114 6.0 6:013d6c33:gc::gc.30:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405e480 2023-10-05T19:40:48.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.579+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(114 gc.30 [call out=15b] v0'0 uv42 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.579+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.579+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:115 6.0 6:013d6c33:gc::gc.30:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405e480 2023-10-05T19:40:48.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.579+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(115 gc.30 [call out=24b] v0'0 uv42 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.579+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:116 6.0 6:013d6c33:gc::gc.30:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405e480 2023-10-05T19:40:48.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.583+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(116 gc.30 [call out=15b] v0'0 uv42 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.583+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2023-10-05T19:40:48.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.583+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:117 6.0 6:013d6c33:gc::gc.30:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405e480 2023-10-05T19:40:48.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.583+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(117 gc.30 [call out=15b] v0'0 uv42 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.583+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.583+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:118 6.0 6:013d6c33:gc::gc.30:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405e480 2023-10-05T19:40:48.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.587+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(118 gc.30 [call] v30'43 uv43 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.587+0000 7fd6a7412e40 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=1 2023-10-05T19:40:48.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.587+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:119 6.f 6:f24cb4a8:gc::gc.31:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804065a60 2023-10-05T19:40:48.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.587+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(119 gc.31 [call] v30'26 uv26 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.587+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:120 6.f 6:f24cb4a8:gc::gc.31:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804065a60 2023-10-05T19:40:48.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.591+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(120 gc.31 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.591+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.591+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:121 6.f 6:f24cb4a8:gc::gc.31:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804065a60 2023-10-05T19:40:48.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.591+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(121 gc.31 [call out=24b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.591+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:122 6.f 6:f24cb4a8:gc::gc.31:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804065a60 2023-10-05T19:40:48.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.591+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(122 gc.31 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.591+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2023-10-05T19:40:48.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.591+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:123 6.f 6:f24cb4a8:gc::gc.31:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804065a60 2023-10-05T19:40:48.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.591+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(123 gc.31 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.591+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.591+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:124 6.f 6:f24cb4a8:gc::gc.31:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804065a60 2023-10-05T19:40:48.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.595+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(124 gc.31 [call] v30'27 uv27 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.595+0000 7fd6a7412e40 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=1 2023-10-05T19:40:48.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.595+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:125 6.19 6:9a6dd8c0:gc::gc.0:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405ad00 2023-10-05T19:40:48.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.599+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(125 gc.0 [call] v30'29 uv29 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7fd694042040 con 0x55980405ad00 2023-10-05T19:40:48.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.599+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:126 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405ad00 2023-10-05T19:40:48.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.599+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(126 gc.0 [call out=15b] v0'0 uv29 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fd694042040 con 0x55980405ad00 2023-10-05T19:40:48.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.599+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.599+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:127 6.19 6:9a6dd8c0:gc::gc.0:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405ad00 2023-10-05T19:40:48.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.599+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(127 gc.0 [call out=24b] v0'0 uv29 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7fd694042040 con 0x55980405ad00 2023-10-05T19:40:48.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.599+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:128 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405ad00 2023-10-05T19:40:48.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.599+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(128 gc.0 [call out=15b] v0'0 uv29 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fd694042040 con 0x55980405ad00 2023-10-05T19:40:48.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.599+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2023-10-05T19:40:48.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.599+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:129 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405ad00 2023-10-05T19:40:48.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.603+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(129 gc.0 [call out=15b] v0'0 uv29 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fd694042040 con 0x55980405ad00 2023-10-05T19:40:48.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.603+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.603+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:130 6.19 6:9a6dd8c0:gc::gc.0:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405ad00 2023-10-05T19:40:48.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.603+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(130 gc.0 [call] v30'30 uv30 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7fd694042040 con 0x55980405ad00 2023-10-05T19:40:48.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.603+0000 7fd6a7412e40 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=1 2023-10-05T19:40:48.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.603+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:131 6.18 6:1e85421f:gc::gc.1:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405e480 2023-10-05T19:40:48.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.607+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(131 gc.1 [call] v30'20 uv20 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.607+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:132 6.18 6:1e85421f:gc::gc.1:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405e480 2023-10-05T19:40:48.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.607+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(132 gc.1 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.607+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.607+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:133 6.18 6:1e85421f:gc::gc.1:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040d5360 con 0x55980405e480 2023-10-05T19:40:48.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.607+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 30 ==== osd_op_reply(133 gc.1 [call out=24b] v0'0 uv20 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.607+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:134 6.18 6:1e85421f:gc::gc.1:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405e480 2023-10-05T19:40:48.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.611+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 31 ==== osd_op_reply(134 gc.1 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.611+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2023-10-05T19:40:48.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.611+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:135 6.18 6:1e85421f:gc::gc.1:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405e480 2023-10-05T19:40:48.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.611+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 32 ==== osd_op_reply(135 gc.1 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.611+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.611+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:136 6.18 6:1e85421f:gc::gc.1:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405e480 2023-10-05T19:40:48.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.615+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 33 ==== osd_op_reply(136 gc.1 [call] v30'21 uv21 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.615+0000 7fd6a7412e40 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=1 2023-10-05T19:40:48.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.615+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:137 6.b 6:d242335b:gc::gc.2:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.615+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 60 ==== osd_op_reply(137 gc.2 [call] v30'20 uv20 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.615+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:138 6.b 6:d242335b:gc::gc.2:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.619+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 61 ==== osd_op_reply(138 gc.2 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.619+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.619+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:139 6.b 6:d242335b:gc::gc.2:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040d5360 con 0x559804061c60 2023-10-05T19:40:48.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.619+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 62 ==== osd_op_reply(139 gc.2 [call out=24b] v0'0 uv20 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.619+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:140 6.b 6:d242335b:gc::gc.2:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.619+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 63 ==== osd_op_reply(140 gc.2 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.619+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2023-10-05T19:40:48.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.619+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:141 6.b 6:d242335b:gc::gc.2:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.619+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 64 ==== osd_op_reply(141 gc.2 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.619+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.619+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:142 6.b 6:d242335b:gc::gc.2:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.623+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 65 ==== osd_op_reply(142 gc.2 [call] v30'21 uv21 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.623+0000 7fd6a7412e40 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=1 2023-10-05T19:40:48.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.623+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:143 6.4 6:2166d8e1:gc::gc.3:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405ad00 2023-10-05T19:40:48.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.627+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(143 gc.3 [call] v30'184 uv184 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7fd694042040 con 0x55980405ad00 2023-10-05T19:40:48.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.627+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:144 6.4 6:2166d8e1:gc::gc.3:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405ad00 2023-10-05T19:40:48.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.627+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(144 gc.3 [call out=15b] v0'0 uv184 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fd694042040 con 0x55980405ad00 2023-10-05T19:40:48.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.627+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.627+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:145 6.4 6:2166d8e1:gc::gc.3:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040d5360 con 0x55980405ad00 2023-10-05T19:40:48.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.627+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(145 gc.3 [call out=24b] v0'0 uv184 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7fd694042040 con 0x55980405ad00 2023-10-05T19:40:48.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.627+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:146 6.4 6:2166d8e1:gc::gc.3:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405ad00 2023-10-05T19:40:48.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.627+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(146 gc.3 [call out=15b] v0'0 uv184 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fd694042040 con 0x55980405ad00 2023-10-05T19:40:48.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.627+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2023-10-05T19:40:48.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.627+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:147 6.4 6:2166d8e1:gc::gc.3:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405ad00 2023-10-05T19:40:48.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.627+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(147 gc.3 [call out=15b] v0'0 uv184 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fd694042040 con 0x55980405ad00 2023-10-05T19:40:48.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.627+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.627+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:148 6.4 6:2166d8e1:gc::gc.3:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405ad00 2023-10-05T19:40:48.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.631+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(148 gc.3 [call] v30'185 uv185 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7fd694042040 con 0x55980405ad00 2023-10-05T19:40:48.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.631+0000 7fd6a7412e40 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=1 2023-10-05T19:40:48.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.631+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:149 6.3 6:c60da066:gc::gc.4:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804065a60 2023-10-05T19:40:48.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.631+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(149 gc.4 [call] v30'50 uv50 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.631+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:150 6.3 6:c60da066:gc::gc.4:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804065a60 2023-10-05T19:40:48.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.631+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(150 gc.4 [call out=15b] v0'0 uv50 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.631+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.631+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:151 6.3 6:c60da066:gc::gc.4:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040d5360 con 0x559804065a60 2023-10-05T19:40:48.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.635+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 23 ==== osd_op_reply(151 gc.4 [call out=24b] v0'0 uv50 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.635+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:152 6.3 6:c60da066:gc::gc.4:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804065a60 2023-10-05T19:40:48.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.635+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 24 ==== osd_op_reply(152 gc.4 [call out=15b] v0'0 uv50 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.635+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2023-10-05T19:40:48.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.635+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:153 6.3 6:c60da066:gc::gc.4:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804065a60 2023-10-05T19:40:48.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.635+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 25 ==== osd_op_reply(153 gc.4 [call out=15b] v0'0 uv50 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.635+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.635+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:154 6.3 6:c60da066:gc::gc.4:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804065a60 2023-10-05T19:40:48.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.639+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 26 ==== osd_op_reply(154 gc.4 [call] v30'51 uv51 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.639+0000 7fd6a7412e40 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=1 2023-10-05T19:40:48.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.639+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:155 6.9 6:954be8d2:gc::gc.5:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.639+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 66 ==== osd_op_reply(155 gc.5 [call] v30'38 uv38 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.639+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:156 6.9 6:954be8d2:gc::gc.5:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.639+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 67 ==== osd_op_reply(156 gc.5 [call out=15b] v0'0 uv38 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.639+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.639+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:157 6.9 6:954be8d2:gc::gc.5:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040d5360 con 0x559804061c60 2023-10-05T19:40:48.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.643+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 68 ==== osd_op_reply(157 gc.5 [call out=24b] v0'0 uv38 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.643+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:158 6.9 6:954be8d2:gc::gc.5:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.643+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 69 ==== osd_op_reply(158 gc.5 [call out=15b] v0'0 uv38 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.643+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2023-10-05T19:40:48.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.643+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:159 6.9 6:954be8d2:gc::gc.5:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.643+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 70 ==== osd_op_reply(159 gc.5 [call out=15b] v0'0 uv38 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.643+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.643+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:160 6.9 6:954be8d2:gc::gc.5:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.647+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 71 ==== osd_op_reply(160 gc.5 [call] v30'39 uv39 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.647+0000 7fd6a7412e40 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=1 2023-10-05T19:40:48.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.647+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:161 6.e 6:748722ff:gc::gc.6:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405ad00 2023-10-05T19:40:48.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.647+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(161 gc.6 [call] v30'22 uv22 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7fd694042040 con 0x55980405ad00 2023-10-05T19:40:48.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.647+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:162 6.e 6:748722ff:gc::gc.6:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405ad00 2023-10-05T19:40:48.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.651+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(162 gc.6 [call out=15b] v0'0 uv22 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fd694042040 con 0x55980405ad00 2023-10-05T19:40:48.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.651+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.651+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:163 6.e 6:748722ff:gc::gc.6:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405ad00 2023-10-05T19:40:48.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.651+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(163 gc.6 [call out=24b] v0'0 uv22 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7fd694042040 con 0x55980405ad00 2023-10-05T19:40:48.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.651+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:164 6.e 6:748722ff:gc::gc.6:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405ad00 2023-10-05T19:40:48.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.651+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(164 gc.6 [call out=15b] v0'0 uv22 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fd694042040 con 0x55980405ad00 2023-10-05T19:40:48.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.651+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2023-10-05T19:40:48.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.651+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:165 6.e 6:748722ff:gc::gc.6:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405ad00 2023-10-05T19:40:48.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.651+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(165 gc.6 [call out=15b] v0'0 uv22 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fd694042040 con 0x55980405ad00 2023-10-05T19:40:48.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.651+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.651+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:166 6.e 6:748722ff:gc::gc.6:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405ad00 2023-10-05T19:40:48.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.655+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(166 gc.6 [call] v30'23 uv23 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7fd694042040 con 0x55980405ad00 2023-10-05T19:40:48.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.655+0000 7fd6a7412e40 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=1 2023-10-05T19:40:48.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.655+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:167 6.1d 6:beaa4be4:gc::gc.7:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405e480 2023-10-05T19:40:48.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.655+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 34 ==== osd_op_reply(167 gc.7 [call] v30'35 uv35 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.655+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:168 6.1d 6:beaa4be4:gc::gc.7:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405e480 2023-10-05T19:40:48.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.655+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 35 ==== osd_op_reply(168 gc.7 [call out=15b] v0'0 uv35 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.655+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.655+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:169 6.1d 6:beaa4be4:gc::gc.7:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405e480 2023-10-05T19:40:48.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.659+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 36 ==== osd_op_reply(169 gc.7 [call out=24b] v0'0 uv35 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.659+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:170 6.1d 6:beaa4be4:gc::gc.7:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405e480 2023-10-05T19:40:48.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.659+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 37 ==== osd_op_reply(170 gc.7 [call out=15b] v0'0 uv35 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.659+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2023-10-05T19:40:48.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.659+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:171 6.1d 6:beaa4be4:gc::gc.7:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405e480 2023-10-05T19:40:48.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.659+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 38 ==== osd_op_reply(171 gc.7 [call out=15b] v0'0 uv35 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.659+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.659+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:172 6.1d 6:beaa4be4:gc::gc.7:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405e480 2023-10-05T19:40:48.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.663+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 39 ==== osd_op_reply(172 gc.7 [call] v30'36 uv36 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.663+0000 7fd6a7412e40 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=1 2023-10-05T19:40:48.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.663+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:173 6.12 6:4f78e9a2:gc::gc.8:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804065a60 2023-10-05T19:40:48.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.663+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 27 ==== osd_op_reply(173 gc.8 [call] v30'20 uv20 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.663+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:174 6.12 6:4f78e9a2:gc::gc.8:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804065a60 2023-10-05T19:40:48.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.667+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 28 ==== osd_op_reply(174 gc.8 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.667+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.667+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:175 6.12 6:4f78e9a2:gc::gc.8:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804065a60 2023-10-05T19:40:48.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.667+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 29 ==== osd_op_reply(175 gc.8 [call out=24b] v0'0 uv20 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.667+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:176 6.12 6:4f78e9a2:gc::gc.8:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804065a60 2023-10-05T19:40:48.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.667+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 30 ==== osd_op_reply(176 gc.8 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.667+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2023-10-05T19:40:48.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.667+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:177 6.12 6:4f78e9a2:gc::gc.8:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804065a60 2023-10-05T19:40:48.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.667+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 31 ==== osd_op_reply(177 gc.8 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.667+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.667+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:178 6.12 6:4f78e9a2:gc::gc.8:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804065a60 2023-10-05T19:40:48.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.671+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 32 ==== osd_op_reply(178 gc.8 [call] v30'21 uv21 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.671+0000 7fd6a7412e40 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=1 2023-10-05T19:40:48.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.671+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:179 6.1c 6:3e426232:gc::gc.9:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.671+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 72 ==== osd_op_reply(179 gc.9 [call] v30'26 uv26 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.671+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:180 6.1c 6:3e426232:gc::gc.9:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.671+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 73 ==== osd_op_reply(180 gc.9 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.671+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.671+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:181 6.1c 6:3e426232:gc::gc.9:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.675+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 74 ==== osd_op_reply(181 gc.9 [call out=24b] v0'0 uv26 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.675+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:182 6.1c 6:3e426232:gc::gc.9:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.675+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 75 ==== osd_op_reply(182 gc.9 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.675+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2023-10-05T19:40:48.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.675+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:183 6.1c 6:3e426232:gc::gc.9:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.675+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 76 ==== osd_op_reply(183 gc.9 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.675+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.675+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:184 6.1c 6:3e426232:gc::gc.9:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.679+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 77 ==== osd_op_reply(184 gc.9 [call] v30'27 uv27 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.679+0000 7fd6a7412e40 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=1 2023-10-05T19:40:48.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.679+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:185 6.3 6:c31f41d7:gc::gc.10:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804065a60 2023-10-05T19:40:48.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.679+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 33 ==== osd_op_reply(185 gc.10 [call] v30'52 uv52 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.679+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:186 6.3 6:c31f41d7:gc::gc.10:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804065a60 2023-10-05T19:40:48.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.679+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 34 ==== osd_op_reply(186 gc.10 [call out=15b] v0'0 uv52 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.679+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.679+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:187 6.3 6:c31f41d7:gc::gc.10:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804065a60 2023-10-05T19:40:48.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.679+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 35 ==== osd_op_reply(187 gc.10 [call out=24b] v0'0 uv52 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.679+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:188 6.3 6:c31f41d7:gc::gc.10:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804065a60 2023-10-05T19:40:48.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.683+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 36 ==== osd_op_reply(188 gc.10 [call out=15b] v0'0 uv52 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.683+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2023-10-05T19:40:48.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.683+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:189 6.3 6:c31f41d7:gc::gc.10:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804065a60 2023-10-05T19:40:48.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.683+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 37 ==== osd_op_reply(189 gc.10 [call out=15b] v0'0 uv52 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.683+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.683+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:190 6.3 6:c31f41d7:gc::gc.10:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804065a60 2023-10-05T19:40:48.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.683+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 38 ==== osd_op_reply(190 gc.10 [call] v30'53 uv53 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.683+0000 7fd6a7412e40 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=1 2023-10-05T19:40:48.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.683+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:191 6.13 6:c8333aab:gc::gc.11:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405e480 2023-10-05T19:40:48.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.687+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 40 ==== osd_op_reply(191 gc.11 [call] v30'23 uv23 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.687+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:192 6.13 6:c8333aab:gc::gc.11:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405e480 2023-10-05T19:40:48.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.687+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 41 ==== osd_op_reply(192 gc.11 [call out=15b] v0'0 uv23 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.687+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.687+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:193 6.13 6:c8333aab:gc::gc.11:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405e480 2023-10-05T19:40:48.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.691+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 42 ==== osd_op_reply(193 gc.11 [call out=24b] v0'0 uv23 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.691+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:194 6.13 6:c8333aab:gc::gc.11:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405e480 2023-10-05T19:40:48.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.691+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 43 ==== osd_op_reply(194 gc.11 [call out=15b] v0'0 uv23 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.691+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2023-10-05T19:40:48.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.691+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:195 6.13 6:c8333aab:gc::gc.11:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405e480 2023-10-05T19:40:48.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.691+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 44 ==== osd_op_reply(195 gc.11 [call out=15b] v0'0 uv23 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.691+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.691+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:196 6.13 6:c8333aab:gc::gc.11:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405e480 2023-10-05T19:40:48.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.695+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 45 ==== osd_op_reply(196 gc.11 [call] v30'24 uv24 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.695+0000 7fd6a7412e40 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=1 2023-10-05T19:40:48.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.695+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:197 6.15 6:ab569f81:gc::gc.12:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804065a60 2023-10-05T19:40:48.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.699+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 39 ==== osd_op_reply(197 gc.12 [call] v30'20 uv20 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.699+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:198 6.15 6:ab569f81:gc::gc.12:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804065a60 2023-10-05T19:40:48.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.699+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 40 ==== osd_op_reply(198 gc.12 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.699+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.699+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:199 6.15 6:ab569f81:gc::gc.12:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804065a60 2023-10-05T19:40:48.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.699+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 41 ==== osd_op_reply(199 gc.12 [call out=24b] v0'0 uv20 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.699+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:200 6.15 6:ab569f81:gc::gc.12:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804065a60 2023-10-05T19:40:48.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.699+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 42 ==== osd_op_reply(200 gc.12 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.699+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2023-10-05T19:40:48.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.699+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:201 6.15 6:ab569f81:gc::gc.12:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804065a60 2023-10-05T19:40:48.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.699+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 43 ==== osd_op_reply(201 gc.12 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.699+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.699+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:202 6.15 6:ab569f81:gc::gc.12:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804065a60 2023-10-05T19:40:48.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.703+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 44 ==== osd_op_reply(202 gc.12 [call] v30'21 uv21 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.703+0000 7fd6a7412e40 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=1 2023-10-05T19:40:48.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.703+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:203 6.0 6:0667bb13:gc::gc.13:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405e480 2023-10-05T19:40:48.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.707+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 46 ==== osd_op_reply(203 gc.13 [call] v30'44 uv44 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.707+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:204 6.0 6:0667bb13:gc::gc.13:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405e480 2023-10-05T19:40:48.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.707+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 47 ==== osd_op_reply(204 gc.13 [call out=15b] v0'0 uv44 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.707+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.707+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:205 6.0 6:0667bb13:gc::gc.13:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405e480 2023-10-05T19:40:48.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.707+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 48 ==== osd_op_reply(205 gc.13 [call out=24b] v0'0 uv44 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.707+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:206 6.0 6:0667bb13:gc::gc.13:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405e480 2023-10-05T19:40:48.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.707+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 49 ==== osd_op_reply(206 gc.13 [call out=15b] v0'0 uv44 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.707+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2023-10-05T19:40:48.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.707+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:207 6.0 6:0667bb13:gc::gc.13:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405e480 2023-10-05T19:40:48.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.711+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 50 ==== osd_op_reply(207 gc.13 [call out=15b] v0'0 uv44 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.711+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.711+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:208 6.0 6:0667bb13:gc::gc.13:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405e480 2023-10-05T19:40:48.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.711+0000 7fd6a4746700 1 -- 172.21.15.12:0/2042177289 <== osd.0 v2:172.21.15.8:6800/27563 51 ==== osd_op_reply(208 gc.13 [call] v30'45 uv45 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fd68c05f040 con 0x55980405e480 2023-10-05T19:40:48.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.711+0000 7fd6a7412e40 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=1 2023-10-05T19:40:48.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.711+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:209 6.4 6:227f63b5:gc::gc.14:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405ad00 2023-10-05T19:40:48.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.715+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(209 gc.14 [call] v30'186 uv186 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fd694042040 con 0x55980405ad00 2023-10-05T19:40:48.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.715+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:210 6.4 6:227f63b5:gc::gc.14:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405ad00 2023-10-05T19:40:48.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.715+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== osd_op_reply(210 gc.14 [call out=15b] v0'0 uv186 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd694042040 con 0x55980405ad00 2023-10-05T19:40:48.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.715+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.715+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:211 6.4 6:227f63b5:gc::gc.14:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405ad00 2023-10-05T19:40:48.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.715+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.1 v2:172.21.15.12:6808/29933 39 ==== osd_op_reply(211 gc.14 [call out=24b] v0'0 uv186 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fd694042040 con 0x55980405ad00 2023-10-05T19:40:48.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.715+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:212 6.4 6:227f63b5:gc::gc.14:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405ad00 2023-10-05T19:40:48.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.715+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.1 v2:172.21.15.12:6808/29933 40 ==== osd_op_reply(212 gc.14 [call out=15b] v0'0 uv186 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd694042040 con 0x55980405ad00 2023-10-05T19:40:48.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.715+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2023-10-05T19:40:48.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.715+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:213 6.4 6:227f63b5:gc::gc.14:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405ad00 2023-10-05T19:40:48.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.715+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.1 v2:172.21.15.12:6808/29933 41 ==== osd_op_reply(213 gc.14 [call out=15b] v0'0 uv186 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd694042040 con 0x55980405ad00 2023-10-05T19:40:48.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.715+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.715+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:214 6.4 6:227f63b5:gc::gc.14:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x55980405ad00 2023-10-05T19:40:48.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.719+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.1 v2:172.21.15.12:6808/29933 42 ==== osd_op_reply(214 gc.14 [call] v30'187 uv187 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fd694042040 con 0x55980405ad00 2023-10-05T19:40:48.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.719+0000 7fd6a7412e40 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=1 2023-10-05T19:40:48.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.719+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:215 6.7 6:e07a6548:gc::gc.15:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.719+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 78 ==== osd_op_reply(215 gc.15 [call] v30'48 uv48 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.719+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:216 6.7 6:e07a6548:gc::gc.15:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.723+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 79 ==== osd_op_reply(216 gc.15 [call out=15b] v0'0 uv48 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.723+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.723+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:217 6.7 6:e07a6548:gc::gc.15:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.723+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 80 ==== osd_op_reply(217 gc.15 [call out=24b] v0'0 uv48 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.723+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:218 6.7 6:e07a6548:gc::gc.15:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.723+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 81 ==== osd_op_reply(218 gc.15 [call out=15b] v0'0 uv48 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.723+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2023-10-05T19:40:48.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.723+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:219 6.7 6:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.723+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 82 ==== osd_op_reply(219 gc.15 [call out=15b] v0'0 uv48 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.723+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.723+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:220 6.7 6:e07a6548:gc::gc.15:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.727+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 83 ==== osd_op_reply(220 gc.15 [call] v30'49 uv49 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.727+0000 7fd6a7412e40 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=1 2023-10-05T19:40:48.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.727+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:221 6.3 6:c623b173:gc::gc.16:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804065a60 2023-10-05T19:40:48.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.727+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 45 ==== osd_op_reply(221 gc.16 [call] v30'54 uv54 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.727+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:222 6.3 6:c623b173:gc::gc.16:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804065a60 2023-10-05T19:40:48.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.727+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 46 ==== osd_op_reply(222 gc.16 [call out=15b] v0'0 uv54 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.727+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.727+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:223 6.3 6:c623b173:gc::gc.16:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804065a60 2023-10-05T19:40:48.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.727+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 47 ==== osd_op_reply(223 gc.16 [call out=24b] v0'0 uv54 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.727+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:224 6.3 6:c623b173:gc::gc.16:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804065a60 2023-10-05T19:40:48.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.731+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 48 ==== osd_op_reply(224 gc.16 [call out=15b] v0'0 uv54 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.731+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2023-10-05T19:40:48.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.731+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:225 6.3 6:c623b173:gc::gc.16:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804065a60 2023-10-05T19:40:48.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.731+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 49 ==== osd_op_reply(225 gc.16 [call out=15b] v0'0 uv54 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.731+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.731+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:226 6.3 6:c623b173:gc::gc.16:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804065a60 2023-10-05T19:40:48.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.731+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 <== osd.2 v2:172.21.15.12:6800/29934 50 ==== osd_op_reply(226 gc.16 [call] v30'55 uv55 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fd694042040 con 0x559804065a60 2023-10-05T19:40:48.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.731+0000 7fd6a7412e40 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=1 2023-10-05T19:40:48.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.731+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:227 6.16 6:6849fd06:gc::gc.17:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.735+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 84 ==== osd_op_reply(227 gc.17 [call] v30'20 uv20 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.735+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:228 6.16 6:6849fd06:gc::gc.17:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.735+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 85 ==== osd_op_reply(228 gc.17 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.735+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.735+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:229 6.16 6:6849fd06:gc::gc.17:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.735+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 86 ==== osd_op_reply(229 gc.17 [call out=24b] v0'0 uv20 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.735+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:230 6.16 6:6849fd06:gc::gc.17:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.735+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 87 ==== osd_op_reply(230 gc.17 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.735+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2023-10-05T19:40:48.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.735+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:231 6.16 6:6849fd06:gc::gc.17:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.735+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 88 ==== osd_op_reply(231 gc.17 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.735+0000 7fd6a7412e40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2023-10-05T19:40:48.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.735+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:232 6.16 6:6849fd06:gc::gc.17:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7c50 con 0x559804061c60 2023-10-05T19:40:48.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.739+0000 7fd6a5748700 1 -- 172.21.15.12:0/2042177289 <== osd.3 v2:172.21.15.12:6816/29951 89 ==== osd_op_reply(232 gc.17 [call] v30'21 uv21 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559804061c60 2023-10-05T19:40:48.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.739+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:102 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94111390717616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040c7760 con 0x55980402cae0 2023-10-05T19:40:48.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.743+0000 7fd6a4f47700 1 -- 172.21.15.12:0/1548698618 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(102 notify.0 [watch unwatch cookie 94111390717616] v30'194 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd694042040 con 0x55980402cae0 2023-10-05T19:40:48.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.743+0000 7fd6a7412e40 20 remove_watcher() i=0 2023-10-05T19:40:48.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.743+0000 7fd6a7412e40 2 removed watcher, disabling cache 2023-10-05T19:40:48.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.743+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94111390729200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040356b0 con 0x559803ff98f0 2023-10-05T19:40:48.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.743+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(103 notify.1 [watch unwatch cookie 94111390729200] v30'194 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559803ff98f0 2023-10-05T19:40:48.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.743+0000 7fd6a7412e40 20 remove_watcher() i=1 2023-10-05T19:40:48.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.743+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:104 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94111390730592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040356b0 con 0x559804031280 2023-10-05T19:40:48.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.747+0000 7fd6a4746700 1 -- 172.21.15.12:0/1548698618 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(104 notify.2 [watch unwatch cookie 94111390730592] v30'194 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd68c05f040 con 0x559804031280 2023-10-05T19:40:48.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.747+0000 7fd6a7412e40 20 remove_watcher() i=2 2023-10-05T19:40:48.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.747+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:105 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94111390735952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804038960 con 0x55980402cae0 2023-10-05T19:40:48.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.751+0000 7fd6a4f47700 1 -- 172.21.15.12:0/1548698618 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(105 notify.3 [watch unwatch cookie 94111390735952] v30'194 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd694042040 con 0x55980402cae0 2023-10-05T19:40:48.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.751+0000 7fd6a7412e40 20 remove_watcher() i=3 2023-10-05T19:40:48.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.751+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:106 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94111390731696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804039e50 con 0x559803ff98f0 2023-10-05T19:40:48.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.751+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(106 notify.4 [watch unwatch cookie 94111390731696] v30'194 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559803ff98f0 2023-10-05T19:40:48.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.751+0000 7fd6a7412e40 20 remove_watcher() i=4 2023-10-05T19:40:48.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.751+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:107 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94111390737056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804039e50 con 0x7fd694007030 2023-10-05T19:40:48.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.755+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(107 notify.5 [watch unwatch cookie 94111390737056] v30'194 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x7fd694007030 2023-10-05T19:40:48.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.755+0000 7fd6a7412e40 20 remove_watcher() i=5 2023-10-05T19:40:48.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.755+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:108 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94111390741568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040383f0 con 0x7fd694007030 2023-10-05T19:40:48.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.759+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(108 notify.6 [watch unwatch cookie 94111390741568] v30'194 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x7fd694007030 2023-10-05T19:40:48.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.759+0000 7fd6a7412e40 20 remove_watcher() i=6 2023-10-05T19:40:48.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.759+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:109 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94111390743904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5598040383f0 con 0x7fd694007030 2023-10-05T19:40:48.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.763+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(109 notify.7 [watch unwatch cookie 94111390743904] v30'194 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x7fd694007030 2023-10-05T19:40:48.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.763+0000 7fd6a7412e40 20 remove_watcher() i=7 2023-10-05T19:40:48.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.763+0000 7fd6767e4700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:48.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.763+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:110 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94111390746432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x559804089870 con 0x559803ff98f0 2023-10-05T19:40:48.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.767+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(110 data_loggenerations_metadata [watch unwatch cookie 94111390746432] v30'188 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fd69c00a5a0 con 0x559803ff98f0 2023-10-05T19:40:48.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.767+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fd694007030 msgr2=0x7fd6940115e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:48.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.767+0000 7fd6a7412e40 1 --2- 172.21.15.12:0/1548698618 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fd694007030 0x7fd6940115e0 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:48.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.767+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x559803ff98f0 msgr2=0x55980402a5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:48.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.767+0000 7fd6a7412e40 1 --2- 172.21.15.12:0/1548698618 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x559803ff98f0 0x55980402a5f0 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:48.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.767+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55980402cae0 msgr2=0x55980402eec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:48.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.767+0000 7fd6a7412e40 1 --2- 172.21.15.12:0/1548698618 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55980402cae0 0x55980402eec0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:48.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.767+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x559804031280 msgr2=0x559804033690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:48.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.767+0000 7fd6a7412e40 1 --2- 172.21.15.12:0/1548698618 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x559804031280 0x559804033690 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:48.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.767+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd684030c50 msgr2=0x7fd684033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:48.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.767+0000 7fd6a7412e40 1 --2- 172.21.15.12:0/1548698618 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd684030c50 0x7fd684033100 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd68c0011e0 tx=0x7fd68c0013a0 comp rx=0 tx=0).stop 2023-10-05T19:40:48.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.767+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559803ff8fe0 msgr2=0x55980401eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:48.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.767+0000 7fd6a5748700 1 -- 172.21.15.12:0/1548698618 reap_dead start 2023-10-05T19:40:48.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.767+0000 7fd6a7412e40 1 --2- 172.21.15.12:0/1548698618 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559803ff8fe0 0x55980401eb00 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd69401e040 tx=0x7fd69401c000 comp rx=0 tx=0).stop 2023-10-05T19:40:48.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.767+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 shutdown_connections 2023-10-05T19:40:48.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.767+0000 7fd6a7412e40 1 --2- 172.21.15.12:0/1548698618 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559803ff8fe0 0x55980401eb00 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:48.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.767+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 >> 172.21.15.12:0/1548698618 conn(0x559803f6c830 msgr2=0x559804012530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:48.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.767+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 shutdown_connections 2023-10-05T19:40:48.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.767+0000 7fd6a7412e40 1 -- 172.21.15.12:0/1548698618 wait complete. 2023-10-05T19:40:48.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.767+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55980405e480 msgr2=0x559804060860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:48.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.767+0000 7fd6a7412e40 1 --2- 172.21.15.12:0/2042177289 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55980405e480 0x559804060860 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:48.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.767+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55980405ad00 msgr2=0x55980405d110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:48.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.767+0000 7fd6a7412e40 1 --2- 172.21.15.12:0/2042177289 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55980405ad00 0x55980405d110 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:48.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.767+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x559804065a60 msgr2=0x559804067e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:48.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.767+0000 7fd6a7412e40 1 --2- 172.21.15.12:0/2042177289 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x559804065a60 0x559804067e40 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:48.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.767+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x559804061c60 msgr2=0x559804064040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:48.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.767+0000 7fd6a7412e40 1 --2- 172.21.15.12:0/2042177289 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x559804061c60 0x559804064040 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:48.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.767+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd5ec030b70 msgr2=0x7fd5ec033020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:48.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.771+0000 7fd6a7412e40 1 --2- 172.21.15.12:0/2042177289 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd5ec030b70 0x7fd5ec033020 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x55980403d150 tx=0x7fd69c00aed0 comp rx=0 tx=0).stop 2023-10-05T19:40:48.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.771+0000 7fd6a4f47700 1 -- 172.21.15.12:0/2042177289 reap_dead start 2023-10-05T19:40:48.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.771+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55980404a550 msgr2=0x5598040500e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:48.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.771+0000 7fd6a7412e40 1 --2- 172.21.15.12:0/2042177289 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55980404a550 0x5598040500e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x559804028080 tx=0x7fd68c03c030 comp rx=0 tx=0).stop 2023-10-05T19:40:48.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.771+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 shutdown_connections 2023-10-05T19:40:48.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.771+0000 7fd6a7412e40 1 --2- 172.21.15.12:0/2042177289 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55980404a550 0x5598040500e0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:48.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.771+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 >> 172.21.15.12:0/2042177289 conn(0x559804045910 msgr2=0x5598040439b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:48.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.771+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 shutdown_connections 2023-10-05T19:40:48.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.771+0000 7fd6a7412e40 1 -- 172.21.15.12:0/2042177289 wait complete. 2023-10-05T19:40:48.783 INFO:tasks.util.rgw:rgwadmin: client.0 : ['gc', 'list', '--include-all'] 2023-10-05T19:40:48.783 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'gc', 'list', '--include-all'] 2023-10-05T19:40:48.784 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph gc list --include-all 2023-10-05T19:40:48.818 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:48.819 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:48.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.811+0000 7f0bab403e40 1 Processor -- start 2023-10-05T19:40:48.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.811+0000 7f0bab403e40 1 -- start start 2023-10-05T19:40:48.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.811+0000 7f0bab403e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55d5629423d0 0x55d5629ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:48.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.811+0000 7f0bab403e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55d56291abd0 con 0x55d5629cef00 2023-10-05T19:40:48.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.811+0000 7f0bab403e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x55d5628d7ee0 con 0x55d5629423d0 2023-10-05T19:40:48.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.811+0000 7f0ba8f38700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55d5629423d0 0x55d5629ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:48.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.811+0000 7f0ba8737700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x55d5629cef00 0x55d5629d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:60208/0 (socket says 172.21.15.12:60208) 2023-10-05T19:40:48.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.811+0000 7f0ba8737700 1 -- 172.21.15.12:0/1335734157 learned_addr learned my addr 172.21.15.12:0/1335734157 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:48.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.811+0000 7f0ba7f36700 1 -- 172.21.15.12:0/1335734157 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 201567698 0 0) 0x55d56291abd0 con 0x55d5629cef00 2023-10-05T19:40:48.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.811+0000 7f0ba7f36700 1 -- 172.21.15.12:0/1335734157 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b90001e10 con 0x55d5629cef00 2023-10-05T19:40:48.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.815+0000 7f0ba8f38700 1 -- 172.21.15.12:0/1335734157 >> v1:172.21.15.8:6789/0 conn(0x55d5629cef00 legacy=0x55d5629d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:48.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.815+0000 7f0ba8f38700 1 -- 172.21.15.12:0/1335734157 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5629d2a30 con 0x55d5629423d0 2023-10-05T19:40:48.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.815+0000 7f0ba8f38700 1 --2- 172.21.15.12:0/1335734157 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d5629423d0 0x55d5629ce9c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c00a5d0 tx=0x7f0b9c005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e29beb6cca8e4d71 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:48.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.815+0000 7f0ba7f36700 1 -- 172.21.15.12:0/1335734157 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b9c007610 con 0x55d5629423d0 2023-10-05T19:40:48.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.815+0000 7f0ba7f36700 1 -- 172.21.15.12:0/1335734157 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b9c007770 con 0x55d5629423d0 2023-10-05T19:40:48.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.815+0000 7f0ba7f36700 1 -- 172.21.15.12:0/1335734157 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b9c007a40 con 0x55d5629423d0 2023-10-05T19:40:48.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.815+0000 7f0bab403e40 1 -- 172.21.15.12:0/1335734157 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d5629423d0 msgr2=0x55d5629ce9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:48.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.815+0000 7f0bab403e40 1 --2- 172.21.15.12:0/1335734157 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d5629423d0 0x55d5629ce9c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c00a5d0 tx=0x7f0b9c005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:48.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.815+0000 7f0bab403e40 1 -- 172.21.15.12:0/1335734157 shutdown_connections 2023-10-05T19:40:48.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.815+0000 7f0bab403e40 1 --2- 172.21.15.12:0/1335734157 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d5629423d0 0x55d5629ce9c0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:48.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.815+0000 7f0bab403e40 1 -- 172.21.15.12:0/1335734157 >> 172.21.15.12:0/1335734157 conn(0x55d5629c8720 msgr2=0x55d5629cab30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:48.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.815+0000 7f0bab403e40 1 -- 172.21.15.12:0/1335734157 shutdown_connections 2023-10-05T19:40:48.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.815+0000 7f0bab403e40 1 -- 172.21.15.12:0/1335734157 wait complete. 2023-10-05T19:40:48.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.815+0000 7f0bab403e40 1 Processor -- start 2023-10-05T19:40:48.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.819+0000 7f0bab403e40 1 -- start start 2023-10-05T19:40:48.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.819+0000 7f0bab403e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d5629423d0 0x55d5629f3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:48.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.819+0000 7f0bab403e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d56289be30 con 0x55d5629423d0 2023-10-05T19:40:48.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.819+0000 7f0ba8f38700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d5629423d0 0x55d5629f3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:48.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.819+0000 7f0ba8f38700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d5629423d0 0x55d5629f3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58410/0 (socket says 172.21.15.12:58410) 2023-10-05T19:40:48.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.819+0000 7f0ba8f38700 1 -- 172.21.15.12:0/3196361020 learned_addr learned my addr 172.21.15.12:0/3196361020 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:48.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.819+0000 7f0ba8f38700 1 -- 172.21.15.12:0/3196361020 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b9c005cf0 con 0x55d5629423d0 2023-10-05T19:40:48.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.819+0000 7f0ba8f38700 1 --2- 172.21.15.12:0/3196361020 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d5629423d0 0x55d5629f3350 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c000c00 tx=0x7f0b9c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14cadbf0d1303da5 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:48.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.819+0000 7f0ba6f34700 1 -- 172.21.15.12:0/3196361020 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b9c0078e0 con 0x55d5629423d0 2023-10-05T19:40:48.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.819+0000 7f0ba6f34700 1 -- 172.21.15.12:0/3196361020 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b9c007a40 con 0x55d5629423d0 2023-10-05T19:40:48.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.819+0000 7f0ba6f34700 1 -- 172.21.15.12:0/3196361020 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b9c007bd0 con 0x55d5629423d0 2023-10-05T19:40:48.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.819+0000 7f0bab403e40 1 -- 172.21.15.12:0/3196361020 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d5629423d0 msgr2=0x55d5629f3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:48.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.819+0000 7f0bab403e40 1 --2- 172.21.15.12:0/3196361020 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d5629423d0 0x55d5629f3350 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c000c00 tx=0x7f0b9c014000 comp rx=0 tx=0).stop 2023-10-05T19:40:48.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.819+0000 7f0bab403e40 1 -- 172.21.15.12:0/3196361020 shutdown_connections 2023-10-05T19:40:48.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.819+0000 7f0bab403e40 1 --2- 172.21.15.12:0/3196361020 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d5629423d0 0x55d5629f3350 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:48.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.819+0000 7f0bab403e40 1 -- 172.21.15.12:0/3196361020 >> 172.21.15.12:0/3196361020 conn(0x55d5629c8720 msgr2=0x55d5629eef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:48.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.819+0000 7f0bab403e40 1 -- 172.21.15.12:0/3196361020 shutdown_connections 2023-10-05T19:40:48.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.819+0000 7f0bab403e40 1 -- 172.21.15.12:0/3196361020 wait complete. 2023-10-05T19:40:48.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.823+0000 7f0bab403e40 1 Processor -- start 2023-10-05T19:40:48.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.823+0000 7f0bab403e40 1 -- start start 2023-10-05T19:40:48.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.823+0000 7f0bab403e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d5629423d0 0x55d5629fa080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:48.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.823+0000 7f0bab403e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d56286eed0 con 0x55d5629423d0 2023-10-05T19:40:48.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.823+0000 7f0ba8f38700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d5629423d0 0x55d5629fa080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:48.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.823+0000 7f0ba8f38700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d5629423d0 0x55d5629fa080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58418/0 (socket says 172.21.15.12:58418) 2023-10-05T19:40:48.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.823+0000 7f0ba8f38700 1 -- 172.21.15.12:0/1699685380 learned_addr learned my addr 172.21.15.12:0/1699685380 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:48.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.823+0000 7f0ba8f38700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5629d2a30 con 0x55d5629423d0 2023-10-05T19:40:48.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.823+0000 7f0ba8f38700 1 --2- 172.21.15.12:0/1699685380 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d5629423d0 0x55d5629fa080 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c01f040 tx=0x7f0b9c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:48.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.823+0000 7f0ba5731700 1 -- 172.21.15.12:0/1699685380 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b9c0078e0 con 0x55d5629423d0 2023-10-05T19:40:48.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.823+0000 7f0ba5731700 1 -- 172.21.15.12:0/1699685380 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b9c026650 con 0x55d5629423d0 2023-10-05T19:40:48.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.823+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d5629fa5c0 con 0x55d5629423d0 2023-10-05T19:40:48.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.823+0000 7f0ba5731700 1 -- 172.21.15.12:0/1699685380 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b9c026820 con 0x55d5629423d0 2023-10-05T19:40:48.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.823+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d5629faa00 con 0x55d5629423d0 2023-10-05T19:40:48.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.827+0000 7f0ba5731700 1 -- 172.21.15.12:0/1699685380 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f0b9c0269d0 con 0x55d5629423d0 2023-10-05T19:40:48.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.827+0000 7f0bab403e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:48.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.827+0000 7f0ba5731700 1 --2- 172.21.15.12:0/1699685380 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f0b94030c40 0x7f0b940330f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:48.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.827+0000 7f0ba5731700 1 -- 172.21.15.12:0/1699685380 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7f0b9c046bd0 con 0x55d5629423d0 2023-10-05T19:40:48.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.827+0000 7f0ba8737700 1 --2- 172.21.15.12:0/1699685380 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f0b94030c40 0x7f0b940330f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:48.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.827+0000 7f0bab403e40 1 --2- 172.21.15.12:0/1699685380 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55d5629cef00 0x55d562a05b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:48.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.827+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a061c0 con 0x55d5629cef00 2023-10-05T19:40:48.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.827+0000 7f0ba9739700 1 --2- 172.21.15.12:0/1699685380 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55d5629cef00 0x55d562a05b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:48.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.827+0000 7f0ba8737700 1 --2- 172.21.15.12:0/1699685380 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f0b94030c40 0x7f0b940330f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0b980044f0 tx=0x7f0b98006690 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:48.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.827+0000 7f0ba9739700 1 --2- 172.21.15.12:0/1699685380 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55d5629cef00 0x55d562a05b10 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:48.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.831+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f0ba000a500 con 0x55d5629cef00 2023-10-05T19:40:48.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.831+0000 7f0bab403e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:48.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.831+0000 7f0bab403e40 20 realm 2023-10-05T19:40:48.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.831+0000 7f0bab403e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:48.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.831+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a03170 con 0x55d5629cef00 2023-10-05T19:40:48.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.831+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f0ba000a500 con 0x55d5629cef00 2023-10-05T19:40:48.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.831+0000 7f0bab403e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:48.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.831+0000 7f0bab403e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:48.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.831+0000 7f0bab403e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:48.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.831+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a06ca0 con 0x55d5629cef00 2023-10-05T19:40:48.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.831+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f0ba000a500 con 0x55d5629cef00 2023-10-05T19:40:48.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.831+0000 7f0bab403e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:48.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.831+0000 7f0bab403e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:48.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.831+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a06880 con 0x55d5629cef00 2023-10-05T19:40:48.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.831+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f0ba000a500 con 0x55d5629cef00 2023-10-05T19:40:48.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.831+0000 7f0bab403e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:48.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.831+0000 7f0bab403e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:48.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.831+0000 7f0bab403e40 1 --2- 172.21.15.12:0/1699685380 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55d562a07c30 0x55d562a0a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:48.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.831+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0a780 con 0x55d562a07c30 2023-10-05T19:40:48.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.831+0000 7f0ba8f38700 1 --2- 172.21.15.12:0/1699685380 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55d562a07c30 0x55d562a0a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:48.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.831+0000 7f0ba8f38700 1 --2- 172.21.15.12:0/1699685380 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55d562a07c30 0x55d562a0a010 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:48.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.831+0000 7f0ba8f38700 1 -- 172.21.15.12:0/1699685380 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a07c30 2023-10-05T19:40:48.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.831+0000 7f0bab403e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:48.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.831+0000 7f0bab403e40 20 searching for the correct realm 2023-10-05T19:40:48.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.831+0000 7f0bab403e40 1 --2- 172.21.15.12:0/1699685380 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55d562a0c5b0 0x55d562a0e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:48.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.831+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0eed0 con 0x55d562a0c5b0 2023-10-05T19:40:48.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.835+0000 7f0ba8737700 1 --2- 172.21.15.12:0/1699685380 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55d562a0c5b0 0x55d562a0e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:48.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.835+0000 7f0ba8737700 1 --2- 172.21.15.12:0/1699685380 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55d562a0c5b0 0x55d562a0e990 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:48.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.835+0000 7f0ba8737700 1 -- 172.21.15.12:0/1699685380 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f0b98004520 con 0x55d562a0c5b0 2023-10-05T19:40:48.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.835+0000 7f0ba8737700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0eed0 con 0x55d562a07c30 2023-10-05T19:40:48.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.835+0000 7f0ba8f38700 1 -- 172.21.15.12:0/1699685380 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a07c30 2023-10-05T19:40:48.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.835+0000 7f0ba8f38700 1 --2- 172.21.15.12:0/1699685380 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f0b9c007030 0x7f0b9c046400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:48.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.835+0000 7f0ba8f38700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0eed0 con 0x7f0b9c007030 2023-10-05T19:40:48.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.839+0000 7f0ba9739700 1 --2- 172.21.15.12:0/1699685380 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f0b9c007030 0x7f0b9c046400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:48.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.839+0000 7f0ba9739700 1 --2- 172.21.15.12:0/1699685380 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f0b9c007030 0x7f0b9c046400 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:48.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.839+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ba0014040 con 0x7f0b9c007030 2023-10-05T19:40:48.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.839+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0eed0 con 0x55d5629cef00 2023-10-05T19:40:48.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.839+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ba0014040 con 0x55d5629cef00 2023-10-05T19:40:48.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.839+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0eed0 con 0x55d5629cef00 2023-10-05T19:40:48.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.839+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ba0014040 con 0x55d5629cef00 2023-10-05T19:40:48.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.839+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0eed0 con 0x55d5629cef00 2023-10-05T19:40:48.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.839+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(11 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ba0014040 con 0x55d5629cef00 2023-10-05T19:40:48.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.839+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0eed0 con 0x7f0b9c007030 2023-10-05T19:40:48.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.843+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ba0014040 con 0x7f0b9c007030 2023-10-05T19:40:48.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.843+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0eed0 con 0x55d562a07c30 2023-10-05T19:40:48.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.843+0000 7f0ba8f38700 1 -- 172.21.15.12:0/1699685380 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a07c30 2023-10-05T19:40:48.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.843+0000 7f0ba8f38700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0eed0 con 0x55d562a07c30 2023-10-05T19:40:48.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.843+0000 7f0ba8f38700 1 -- 172.21.15.12:0/1699685380 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a07c30 2023-10-05T19:40:48.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.843+0000 7f0ba8f38700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0eed0 con 0x55d5629cef00 2023-10-05T19:40:48.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.843+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ba0014040 con 0x55d5629cef00 2023-10-05T19:40:48.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.843+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0eed0 con 0x7f0b9c007030 2023-10-05T19:40:48.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.843+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ba0014040 con 0x7f0b9c007030 2023-10-05T19:40:48.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.847+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0eed0 con 0x55d5629cef00 2023-10-05T19:40:48.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.847+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ba0014040 con 0x55d5629cef00 2023-10-05T19:40:48.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.847+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0eed0 con 0x55d562a07c30 2023-10-05T19:40:48.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.847+0000 7f0ba8f38700 1 -- 172.21.15.12:0/1699685380 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(18 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a07c30 2023-10-05T19:40:48.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.847+0000 7f0ba8f38700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0eed0 con 0x55d562a0c5b0 2023-10-05T19:40:48.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.847+0000 7f0ba8737700 1 -- 172.21.15.12:0/1699685380 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0b98004520 con 0x55d562a0c5b0 2023-10-05T19:40:48.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.847+0000 7f0ba8737700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0eed0 con 0x55d562a0c5b0 2023-10-05T19:40:48.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.847+0000 7f0ba8737700 1 -- 172.21.15.12:0/1699685380 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0b98004520 con 0x55d562a0c5b0 2023-10-05T19:40:48.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.847+0000 7f0ba8737700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0eed0 con 0x7f0b9c007030 2023-10-05T19:40:48.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.851+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ba0014040 con 0x7f0b9c007030 2023-10-05T19:40:48.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.851+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0eed0 con 0x55d562a0c5b0 2023-10-05T19:40:48.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.851+0000 7f0ba8737700 1 -- 172.21.15.12:0/1699685380 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0b9801d040 con 0x55d562a0c5b0 2023-10-05T19:40:48.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.851+0000 7f0ba8737700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0eed0 con 0x55d562a0c5b0 2023-10-05T19:40:48.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.851+0000 7f0ba8737700 1 -- 172.21.15.12:0/1699685380 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0b9801d040 con 0x55d562a0c5b0 2023-10-05T19:40:48.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.851+0000 7f0ba8737700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0eed0 con 0x55d5629cef00 2023-10-05T19:40:48.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.851+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ba0014040 con 0x55d5629cef00 2023-10-05T19:40:48.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.851+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0eed0 con 0x55d5629cef00 2023-10-05T19:40:48.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.851+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ba0014040 con 0x55d5629cef00 2023-10-05T19:40:48.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.851+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0eed0 con 0x7f0b9c007030 2023-10-05T19:40:48.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.855+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ba0014040 con 0x7f0b9c007030 2023-10-05T19:40:48.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.855+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0eed0 con 0x55d562a0c5b0 2023-10-05T19:40:48.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.855+0000 7f0ba8737700 1 -- 172.21.15.12:0/1699685380 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0b9801d040 con 0x55d562a0c5b0 2023-10-05T19:40:48.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.855+0000 7f0ba8737700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0eed0 con 0x55d562a07c30 2023-10-05T19:40:48.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.855+0000 7f0ba8f38700 1 -- 172.21.15.12:0/1699685380 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a07c30 2023-10-05T19:40:48.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.855+0000 7f0ba8f38700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0eed0 con 0x55d5629cef00 2023-10-05T19:40:48.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.855+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ba0014040 con 0x55d5629cef00 2023-10-05T19:40:48.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.855+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0eed0 con 0x7f0b9c007030 2023-10-05T19:40:48.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.855+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ba0014040 con 0x7f0b9c007030 2023-10-05T19:40:48.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.855+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0eed0 con 0x55d562a0c5b0 2023-10-05T19:40:48.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.859+0000 7f0ba8737700 1 -- 172.21.15.12:0/1699685380 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0b9801d040 con 0x55d562a0c5b0 2023-10-05T19:40:48.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.859+0000 7f0ba8737700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0eed0 con 0x7f0b9c007030 2023-10-05T19:40:48.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.859+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ba0014040 con 0x7f0b9c007030 2023-10-05T19:40:48.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.859+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0eed0 con 0x55d562a07c30 2023-10-05T19:40:48.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.859+0000 7f0ba8f38700 1 -- 172.21.15.12:0/1699685380 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a07c30 2023-10-05T19:40:48.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.859+0000 7f0ba8f38700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0eed0 con 0x55d562a0c5b0 2023-10-05T19:40:48.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.859+0000 7f0ba8737700 1 -- 172.21.15.12:0/1699685380 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0b9801d040 con 0x55d562a0c5b0 2023-10-05T19:40:48.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.859+0000 7f0ba8737700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0eed0 con 0x7f0b9c007030 2023-10-05T19:40:48.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.863+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ba0014040 con 0x7f0b9c007030 2023-10-05T19:40:48.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.863+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0eed0 con 0x55d5629cef00 2023-10-05T19:40:48.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.863+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f0ba0014040 con 0x55d5629cef00 2023-10-05T19:40:48.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.863+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0eed0 con 0x55d562a07c30 2023-10-05T19:40:48.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.863+0000 7f0ba8f38700 1 -- 172.21.15.12:0/1699685380 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(37 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a07c30 2023-10-05T19:40:48.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.863+0000 7f0bab403e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:48.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.863+0000 7f0bab403e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:48.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.863+0000 7f0bab403e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:48.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.863+0000 7f0bab403e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:48.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.863+0000 7f0bab403e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:48.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.863+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0b330 con 0x55d5629cef00 2023-10-05T19:40:48.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.863+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f0ba0014040 con 0x55d5629cef00 2023-10-05T19:40:48.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.863+0000 7f0bab403e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:48.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.863+0000 7f0bab403e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:48.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.863+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0bc80 con 0x55d5629cef00 2023-10-05T19:40:48.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.863+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f0ba0014040 con 0x55d5629cef00 2023-10-05T19:40:48.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.863+0000 7f0bab403e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:48.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.863+0000 7f0bab403e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:48.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.863+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0aef0 con 0x55d562a0c5b0 2023-10-05T19:40:48.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.867+0000 7f0ba8737700 1 -- 172.21.15.12:0/1699685380 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f0b9801d040 con 0x55d562a0c5b0 2023-10-05T19:40:48.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.867+0000 7f0bab403e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:48.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.867+0000 7f0bab403e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:48.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.867+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0f800 con 0x55d562a0c5b0 2023-10-05T19:40:48.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.867+0000 7f0ba8737700 1 -- 172.21.15.12:0/1699685380 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f0b9801d040 con 0x55d562a0c5b0 2023-10-05T19:40:48.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.867+0000 7f0bab403e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:48.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.867+0000 7f0bab403e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:48.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.867+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0fbb0 con 0x55d5629cef00 2023-10-05T19:40:48.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.867+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f0ba0014040 con 0x55d5629cef00 2023-10-05T19:40:48.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.867+0000 7f0bab403e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:48.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.867+0000 7f0bab403e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:48.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.867+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0ff60 con 0x55d5629cef00 2023-10-05T19:40:48.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.867+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f0ba0014040 con 0x55d5629cef00 2023-10-05T19:40:48.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.867+0000 7f0bab403e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:48.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.867+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a10f70 con 0x55d562a0c5b0 2023-10-05T19:40:48.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.871+0000 7f0ba8737700 1 -- 172.21.15.12:0/1699685380 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(44 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f0b9801d040 con 0x55d562a0c5b0 2023-10-05T19:40:48.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.871+0000 7f0ba8737700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a10f70 con 0x55d562a07c30 2023-10-05T19:40:48.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.871+0000 7f0ba8f38700 1 -- 172.21.15.12:0/1699685380 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(45 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a07c30 2023-10-05T19:40:48.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.871+0000 7f0ba8f38700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a10f70 con 0x7f0b9c007030 2023-10-05T19:40:48.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.871+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(46 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ba0014040 con 0x7f0b9c007030 2023-10-05T19:40:48.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.871+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0fbb0 con 0x55d5629cef00 2023-10-05T19:40:48.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.875+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(47 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ba0014040 con 0x55d5629cef00 2023-10-05T19:40:48.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.875+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0fbb0 con 0x55d5629cef00 2023-10-05T19:40:48.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.875+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(48 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ba0014040 con 0x55d5629cef00 2023-10-05T19:40:48.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.875+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0fbb0 con 0x55d5629cef00 2023-10-05T19:40:48.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.875+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(49 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ba0014040 con 0x55d5629cef00 2023-10-05T19:40:48.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.875+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0fbb0 con 0x7f0b9c007030 2023-10-05T19:40:48.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.875+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(50 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ba0014040 con 0x7f0b9c007030 2023-10-05T19:40:48.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.875+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0fbb0 con 0x55d562a07c30 2023-10-05T19:40:48.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.875+0000 7f0ba8f38700 1 -- 172.21.15.12:0/1699685380 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(51 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a07c30 2023-10-05T19:40:48.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.875+0000 7f0ba8f38700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0fbb0 con 0x55d562a07c30 2023-10-05T19:40:48.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.879+0000 7f0ba8f38700 1 -- 172.21.15.12:0/1699685380 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(52 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a07c30 2023-10-05T19:40:48.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.879+0000 7f0ba8f38700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0fbb0 con 0x55d5629cef00 2023-10-05T19:40:48.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.879+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(53 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ba0014040 con 0x55d5629cef00 2023-10-05T19:40:48.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.879+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0fbb0 con 0x7f0b9c007030 2023-10-05T19:40:48.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.879+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(54 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ba0014040 con 0x7f0b9c007030 2023-10-05T19:40:48.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.879+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0fbb0 con 0x55d5629cef00 2023-10-05T19:40:48.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.879+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(55 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ba0014040 con 0x55d5629cef00 2023-10-05T19:40:48.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.879+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0fbb0 con 0x55d562a07c30 2023-10-05T19:40:48.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.879+0000 7f0ba8f38700 1 -- 172.21.15.12:0/1699685380 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(56 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a07c30 2023-10-05T19:40:48.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.879+0000 7f0ba8f38700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0fbb0 con 0x55d562a0c5b0 2023-10-05T19:40:48.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.883+0000 7f0ba8737700 1 -- 172.21.15.12:0/1699685380 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(57 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0b9801d040 con 0x55d562a0c5b0 2023-10-05T19:40:48.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.883+0000 7f0ba8737700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0fbb0 con 0x55d562a0c5b0 2023-10-05T19:40:48.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.883+0000 7f0ba8737700 1 -- 172.21.15.12:0/1699685380 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(58 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0b9801d040 con 0x55d562a0c5b0 2023-10-05T19:40:48.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.883+0000 7f0ba8737700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0fbb0 con 0x7f0b9c007030 2023-10-05T19:40:48.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.883+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(59 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ba0014040 con 0x7f0b9c007030 2023-10-05T19:40:48.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.883+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0fbb0 con 0x55d562a0c5b0 2023-10-05T19:40:48.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.883+0000 7f0ba8737700 1 -- 172.21.15.12:0/1699685380 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(60 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0b9801d040 con 0x55d562a0c5b0 2023-10-05T19:40:48.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.883+0000 7f0ba8737700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0fbb0 con 0x55d562a0c5b0 2023-10-05T19:40:48.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.887+0000 7f0ba8737700 1 -- 172.21.15.12:0/1699685380 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(61 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0b9801d040 con 0x55d562a0c5b0 2023-10-05T19:40:48.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.887+0000 7f0ba8737700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0fbb0 con 0x55d5629cef00 2023-10-05T19:40:48.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.887+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(62 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ba0014040 con 0x55d5629cef00 2023-10-05T19:40:48.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.887+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0fbb0 con 0x55d5629cef00 2023-10-05T19:40:48.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.887+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(63 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ba0014040 con 0x55d5629cef00 2023-10-05T19:40:48.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.887+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0fbb0 con 0x7f0b9c007030 2023-10-05T19:40:48.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.887+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(64 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ba0014040 con 0x7f0b9c007030 2023-10-05T19:40:48.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.887+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0fbb0 con 0x55d562a0c5b0 2023-10-05T19:40:48.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.891+0000 7f0ba8737700 1 -- 172.21.15.12:0/1699685380 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(65 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0b9801d040 con 0x55d562a0c5b0 2023-10-05T19:40:48.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.891+0000 7f0ba8737700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0fbb0 con 0x55d562a07c30 2023-10-05T19:40:48.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.891+0000 7f0ba8f38700 1 -- 172.21.15.12:0/1699685380 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(66 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a07c30 2023-10-05T19:40:48.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.891+0000 7f0ba8f38700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0fbb0 con 0x55d5629cef00 2023-10-05T19:40:48.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.891+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(67 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ba0014040 con 0x55d5629cef00 2023-10-05T19:40:48.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.891+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0fbb0 con 0x7f0b9c007030 2023-10-05T19:40:48.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.891+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(68 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ba0014040 con 0x7f0b9c007030 2023-10-05T19:40:48.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.891+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0fbb0 con 0x55d562a0c5b0 2023-10-05T19:40:48.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.895+0000 7f0ba8737700 1 -- 172.21.15.12:0/1699685380 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(69 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0b9801d040 con 0x55d562a0c5b0 2023-10-05T19:40:48.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.895+0000 7f0ba8737700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0fbb0 con 0x7f0b9c007030 2023-10-05T19:40:48.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.895+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(70 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ba0014040 con 0x7f0b9c007030 2023-10-05T19:40:48.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.895+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0fbb0 con 0x55d562a07c30 2023-10-05T19:40:48.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.895+0000 7f0ba8f38700 1 -- 172.21.15.12:0/1699685380 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(71 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a07c30 2023-10-05T19:40:48.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.895+0000 7f0ba8f38700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0fbb0 con 0x55d562a0c5b0 2023-10-05T19:40:48.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.895+0000 7f0ba8737700 1 -- 172.21.15.12:0/1699685380 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(72 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0b9801d040 con 0x55d562a0c5b0 2023-10-05T19:40:48.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.895+0000 7f0ba8737700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0fbb0 con 0x7f0b9c007030 2023-10-05T19:40:48.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.895+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(73 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ba0014040 con 0x7f0b9c007030 2023-10-05T19:40:48.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.895+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0fbb0 con 0x55d5629cef00 2023-10-05T19:40:48.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.899+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(74 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f0ba0014040 con 0x55d5629cef00 2023-10-05T19:40:48.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.899+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0fbb0 con 0x55d562a07c30 2023-10-05T19:40:48.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.899+0000 7f0ba8f38700 1 -- 172.21.15.12:0/1699685380 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(75 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a07c30 2023-10-05T19:40:48.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.899+0000 7f0bab403e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:48.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.899+0000 7f0bab403e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:48.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.899+0000 7f0bab403e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:48.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.899+0000 7f0bab403e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:48.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.899+0000 7f0bab403e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:48.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.899+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a0ff60 con 0x55d5629cef00 2023-10-05T19:40:48.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.899+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f0ba0014040 con 0x55d5629cef00 2023-10-05T19:40:48.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.899+0000 7f0bab403e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:48.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.899+0000 7f0bab403e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:48.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.899+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a10310 con 0x55d562a07c30 2023-10-05T19:40:48.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.899+0000 7f0ba8f38700 1 -- 172.21.15.12:0/1699685380 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a07c30 2023-10-05T19:40:48.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.899+0000 7f0bab403e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:48.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.899+0000 7f0bab403e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:48.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.899+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a10bb0 con 0x55d5629cef00 2023-10-05T19:40:48.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.903+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f0ba0014040 con 0x55d5629cef00 2023-10-05T19:40:48.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.903+0000 7f0bab403e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:48.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.903+0000 7f0bab403e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:48.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.903+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a10bb0 con 0x55d562a0c5b0 2023-10-05T19:40:48.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.903+0000 7f0ba8737700 1 -- 172.21.15.12:0/1699685380 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f0b9801d040 con 0x55d562a0c5b0 2023-10-05T19:40:48.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.903+0000 7f0bab403e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:48.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.903+0000 7f0bab403e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:48.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.903+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a11450 con 0x7f0b9c007030 2023-10-05T19:40:48.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.903+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f0ba0014040 con 0x7f0b9c007030 2023-10-05T19:40:48.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.903+0000 7f0bab403e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:48.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.903+0000 7f0bab403e40 20 zone default found 2023-10-05T19:40:48.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.903+0000 7f0bab403e40 4 Realm: () 2023-10-05T19:40:48.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.903+0000 7f0bab403e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:48.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.903+0000 7f0bab403e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:48.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.903+0000 7f0bab403e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:48.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.903+0000 7f0bab403e40 20 zonegroup default 2023-10-05T19:40:48.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.903+0000 7f0bab403e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:48.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.903+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a11450 con 0x7f0b9c007030 2023-10-05T19:40:48.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.903+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f0ba0014040 con 0x7f0b9c007030 2023-10-05T19:40:48.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.903+0000 7f0bab403e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:48.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.903+0000 7f0bab403e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:48.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.903+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a12dd0 con 0x55d562a0c5b0 2023-10-05T19:40:48.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.907+0000 7f0ba8737700 1 -- 172.21.15.12:0/1699685380 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f0b9801d040 con 0x55d562a0c5b0 2023-10-05T19:40:48.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.907+0000 7f0bab403e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:48.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.907+0000 7f0bab403e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:48.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.907+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a13180 con 0x55d562a0c5b0 2023-10-05T19:40:48.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.907+0000 7f0ba8737700 1 -- 172.21.15.12:0/1699685380 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f0b9801d040 con 0x55d562a0c5b0 2023-10-05T19:40:48.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.907+0000 7f0bab403e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:48.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.907+0000 7f0bab403e40 20 started sync module instance, tier type = 2023-10-05T19:40:48.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.907+0000 7f0bab403e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:48.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.907+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a12490 con 0x55d562a07c30 2023-10-05T19:40:48.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.911+0000 7f0ba8f38700 1 -- 172.21.15.12:0/1699685380 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(84 notify.0 [create] v30'195 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a07c30 2023-10-05T19:40:48.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.911+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94374971121568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a13530 con 0x55d562a07c30 2023-10-05T19:40:48.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.911+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a13dd0 con 0x55d5629cef00 2023-10-05T19:40:48.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.911+0000 7f0ba8f38700 1 -- 172.21.15.12:0/1699685380 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(85 notify.0 [watch watch cookie 94374971121568] v30'196 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a07c30 2023-10-05T19:40:48.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.911+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(86 notify.1 [create] v30'195 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0ba0014040 con 0x55d5629cef00 2023-10-05T19:40:48.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.911+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94374971123904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a13dd0 con 0x55d5629cef00 2023-10-05T19:40:48.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.911+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a14870 con 0x55d562a0c5b0 2023-10-05T19:40:48.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.915+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(87 notify.1 [watch watch cookie 94374971123904] v30'196 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0ba0014040 con 0x55d5629cef00 2023-10-05T19:40:48.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.915+0000 7f0ba8737700 1 -- 172.21.15.12:0/1699685380 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(88 notify.2 [create] v30'195 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0b9801d040 con 0x55d562a0c5b0 2023-10-05T19:40:48.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.915+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94374971128032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a13d30 con 0x55d562a0c5b0 2023-10-05T19:40:48.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.915+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a15650 con 0x55d562a07c30 2023-10-05T19:40:48.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.919+0000 7f0ba8737700 1 -- 172.21.15.12:0/1699685380 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(89 notify.2 [watch watch cookie 94374971128032] v30'196 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0b9801d040 con 0x55d562a0c5b0 2023-10-05T19:40:48.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.919+0000 7f0ba8f38700 1 -- 172.21.15.12:0/1699685380 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(90 notify.3 [create] v30'195 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a07c30 2023-10-05T19:40:48.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.919+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94374971133328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a156d0 con 0x55d562a07c30 2023-10-05T19:40:48.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.919+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a16150 con 0x55d5629cef00 2023-10-05T19:40:48.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.923+0000 7f0ba8f38700 1 -- 172.21.15.12:0/1699685380 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(91 notify.3 [watch watch cookie 94374971133328] v30'196 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a07c30 2023-10-05T19:40:48.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.923+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(92 notify.4 [create] v30'195 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0ba0014040 con 0x55d5629cef00 2023-10-05T19:40:48.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.923+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94374971137056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a156d0 con 0x55d5629cef00 2023-10-05T19:40:48.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.923+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a16bb0 con 0x7f0b9c007030 2023-10-05T19:40:48.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.923+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(94 notify.5 [create] v30'195 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0ba0014040 con 0x7f0b9c007030 2023-10-05T19:40:48.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.923+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94374971139776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a17000 con 0x7f0b9c007030 2023-10-05T19:40:48.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.923+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a17a50 con 0x7f0b9c007030 2023-10-05T19:40:48.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.927+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(93 notify.4 [watch watch cookie 94374971137056] v30'196 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0ba0014040 con 0x55d5629cef00 2023-10-05T19:40:48.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.927+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(96 notify.6 [create] v30'195 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0ba0014040 con 0x7f0b9c007030 2023-10-05T19:40:48.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.927+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94374971134432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a17a10 con 0x7f0b9c007030 2023-10-05T19:40:48.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.927+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a18420 con 0x7f0b9c007030 2023-10-05T19:40:48.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.927+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(95 notify.5 [watch watch cookie 94374971139776] v30'196 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0ba0014040 con 0x7f0b9c007030 2023-10-05T19:40:48.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.931+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(97 notify.6 [watch watch cookie 94374971134432] v30'196 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0ba0014040 con 0x7f0b9c007030 2023-10-05T19:40:48.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.931+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(98 notify.7 [create] v30'195 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0ba0014040 con 0x7f0b9c007030 2023-10-05T19:40:48.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.931+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94374971143456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a17970 con 0x7f0b9c007030 2023-10-05T19:40:48.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.931+0000 7f0bab403e40 20 add_watcher() i=0 2023-10-05T19:40:48.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.931+0000 7f0bab403e40 20 add_watcher() i=1 2023-10-05T19:40:48.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.931+0000 7f0bab403e40 20 add_watcher() i=2 2023-10-05T19:40:48.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.931+0000 7f0bab403e40 20 add_watcher() i=3 2023-10-05T19:40:48.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.931+0000 7f0bab403e40 20 add_watcher() i=4 2023-10-05T19:40:48.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.931+0000 7f0bab403e40 20 add_watcher() i=5 2023-10-05T19:40:48.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.931+0000 7f0bab403e40 20 add_watcher() i=6 2023-10-05T19:40:48.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.931+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(99 notify.7 [watch watch cookie 94374971143456] v30'196 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0ba0014040 con 0x7f0b9c007030 2023-10-05T19:40:48.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.931+0000 7f0bab403e40 20 add_watcher() i=7 2023-10-05T19:40:48.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.931+0000 7f0bab403e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:48.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.931+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a16c30 con 0x55d5629cef00 2023-10-05T19:40:48.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.935+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f0ba0014040 con 0x55d5629cef00 2023-10-05T19:40:48.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.935+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94374971146032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a18870 con 0x55d5629cef00 2023-10-05T19:40:48.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.935+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94374971146032] v30'189 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f0ba0014040 con 0x55d5629cef00 2023-10-05T19:40:48.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.935+0000 7f0b7a7e4700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:48.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.935+0000 7f0bab403e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:48.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.935+0000 7f0bab403e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:48.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.935+0000 7f0bab403e40 1 Processor -- start 2023-10-05T19:40:48.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.935+0000 7f0bab403e40 1 -- start start 2023-10-05T19:40:48.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.935+0000 7f0bab403e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d562a25500 0x55d562a258d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:48.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.935+0000 7f0bab403e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d562a25e10 con 0x55d562a25500 2023-10-05T19:40:48.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.939+0000 7f0ba8737700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d562a25500 0x55d562a258d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:48.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.939+0000 7f0ba8737700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d562a25500 0x55d562a258d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58422/0 (socket says 172.21.15.12:58422) 2023-10-05T19:40:48.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.939+0000 7f0ba8737700 1 -- 172.21.15.12:0/1447139502 learned_addr learned my addr 172.21.15.12:0/1447139502 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:48.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.939+0000 7f0ba8737700 1 -- 172.21.15.12:0/1447139502 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d562a25f50 con 0x55d562a25500 2023-10-05T19:40:48.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.939+0000 7f0ba8737700 1 --2- 172.21.15.12:0/1447139502 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d562a25500 0x55d562a258d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f0b98001330 tx=0x7f0b98010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2080f73de3cc94ea server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:48.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.939+0000 7f0b79fe3700 1 -- 172.21.15.12:0/1447139502 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b9802a040 con 0x55d562a25500 2023-10-05T19:40:48.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.939+0000 7f0b79fe3700 1 -- 172.21.15.12:0/1447139502 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b9801b040 con 0x55d562a25500 2023-10-05T19:40:48.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.939+0000 7f0b79fe3700 1 -- 172.21.15.12:0/1447139502 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b9801a040 con 0x55d562a25500 2023-10-05T19:40:48.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.939+0000 7f0bab403e40 1 -- 172.21.15.12:0/1447139502 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d562a25500 msgr2=0x55d562a258d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:48.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.939+0000 7f0bab403e40 1 --2- 172.21.15.12:0/1447139502 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d562a25500 0x55d562a258d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f0b98001330 tx=0x7f0b98010000 comp rx=0 tx=0).stop 2023-10-05T19:40:48.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.939+0000 7f0bab403e40 1 -- 172.21.15.12:0/1447139502 shutdown_connections 2023-10-05T19:40:48.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.939+0000 7f0bab403e40 1 --2- 172.21.15.12:0/1447139502 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d562a25500 0x55d562a258d0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:48.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.939+0000 7f0bab403e40 1 -- 172.21.15.12:0/1447139502 >> 172.21.15.12:0/1447139502 conn(0x55d562a208c0 msgr2=0x55d562a1ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:48.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.939+0000 7f0bab403e40 1 -- 172.21.15.12:0/1447139502 shutdown_connections 2023-10-05T19:40:48.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.939+0000 7f0bab403e40 1 -- 172.21.15.12:0/1447139502 wait complete. 2023-10-05T19:40:48.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.939+0000 7f0bab403e40 1 Processor -- start 2023-10-05T19:40:48.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.939+0000 7f0bab403e40 1 -- start start 2023-10-05T19:40:48.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.943+0000 7f0bab403e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d562a25500 0x55d562a2af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:48.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.943+0000 7f0bab403e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d562a2bf80 con 0x55d562a25500 2023-10-05T19:40:48.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.943+0000 7f0ba8737700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d562a25500 0x55d562a2af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:48.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.943+0000 7f0ba8737700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d562a25500 0x55d562a2af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58434/0 (socket says 172.21.15.12:58434) 2023-10-05T19:40:48.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.943+0000 7f0ba8737700 1 -- 172.21.15.12:0/324485355 learned_addr learned my addr 172.21.15.12:0/324485355 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:48.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.943+0000 7f0ba8737700 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b90001e10 con 0x55d562a25500 2023-10-05T19:40:48.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.943+0000 7f0ba8737700 1 --2- 172.21.15.12:0/324485355 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d562a25500 0x55d562a2af80 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f0b98001330 tx=0x7f0b98038070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:48.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.943+0000 7f0aebfff700 1 -- 172.21.15.12:0/324485355 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b9802a030 con 0x55d562a25500 2023-10-05T19:40:48.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.943+0000 7f0aebfff700 1 -- 172.21.15.12:0/324485355 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b98033e60 con 0x55d562a25500 2023-10-05T19:40:48.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.943+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d562a2b580 con 0x55d562a25500 2023-10-05T19:40:48.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.943+0000 7f0aebfff700 1 -- 172.21.15.12:0/324485355 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b9801a070 con 0x55d562a25500 2023-10-05T19:40:48.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.943+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d562a2b960 con 0x55d562a25500 2023-10-05T19:40:48.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.943+0000 7f0aebfff700 1 -- 172.21.15.12:0/324485355 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f0b98026d50 con 0x55d562a25500 2023-10-05T19:40:48.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.947+0000 7f0aebfff700 1 --2- 172.21.15.12:0/324485355 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f0af0030bf0 0x7f0af00330a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:48.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.947+0000 7f0aebfff700 1 -- 172.21.15.12:0/324485355 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7f0b9804dbd0 con 0x55d562a25500 2023-10-05T19:40:48.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.947+0000 7f0ba9739700 1 --2- 172.21.15.12:0/324485355 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f0af0030bf0 0x7f0af00330a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:48.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.947+0000 7f0bab403e40 1 --2- 172.21.15.12:0/324485355 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55d562a35ba0 0x55d562a37fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:48.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.947+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a38650 con 0x55d562a35ba0 2023-10-05T19:40:48.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.947+0000 7f0ba8f38700 1 --2- 172.21.15.12:0/324485355 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55d562a35ba0 0x55d562a37fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:48.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.947+0000 7f0ba9739700 1 --2- 172.21.15.12:0/324485355 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f0af0030bf0 0x7f0af00330a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x55d562a16ca0 tx=0x7f0ba00335d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:48.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.947+0000 7f0ba8f38700 1 --2- 172.21.15.12:0/324485355 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55d562a35ba0 0x55d562a37fb0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:48.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.951+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v30'31 uv30 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a35ba0 2023-10-05T19:40:48.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.951+0000 7f0bab403e40 1 --2- 172.21.15.12:0/324485355 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55d562a39320 0x55d562a3b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:48.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.951+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a3be30 con 0x55d562a39320 2023-10-05T19:40:48.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.951+0000 7f0ba8737700 1 --2- 172.21.15.12:0/324485355 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55d562a39320 0x55d562a3b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:48.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.951+0000 7f0ba8737700 1 --2- 172.21.15.12:0/324485355 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55d562a39320 0x55d562a3b700 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:48.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.955+0000 7f0ba8737700 1 -- 172.21.15.12:0/324485355 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v30'22 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f0b98052070 con 0x55d562a39320 2023-10-05T19:40:48.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.955+0000 7f0bab403e40 1 --2- 172.21.15.12:0/324485355 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55d562a3cb00 0x55d562a3eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:48.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.955+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a3f610 con 0x55d562a3cb00 2023-10-05T19:40:48.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.955+0000 7f0ba9739700 1 --2- 172.21.15.12:0/324485355 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55d562a3cb00 0x55d562a3eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:48.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.955+0000 7f0ba9739700 1 --2- 172.21.15.12:0/324485355 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55d562a3cb00 0x55d562a3eee0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:48.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.955+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v30'22 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:48.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.955+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a3fc60 con 0x55d562a35ba0 2023-10-05T19:40:48.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.959+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v30'190 uv185 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a35ba0 2023-10-05T19:40:48.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.959+0000 7f0bab403e40 1 --2- 172.21.15.12:0/324485355 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55d562a40900 0x55d562a42ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:48.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.959+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a43340 con 0x55d562a40900 2023-10-05T19:40:48.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.959+0000 7f0ba8f38700 1 --2- 172.21.15.12:0/324485355 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55d562a40900 0x55d562a42ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:48.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.959+0000 7f0ba8f38700 1 --2- 172.21.15.12:0/324485355 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55d562a40900 0x55d562a42ce0 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:48.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.963+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v30'56 uv51 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a40900 2023-10-05T19:40:48.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.963+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a43990 con 0x55d562a3cb00 2023-10-05T19:40:48.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.967+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v30'40 uv39 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:48.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.967+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a43fb0 con 0x55d562a35ba0 2023-10-05T19:40:48.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.967+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v30'24 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a35ba0 2023-10-05T19:40:48.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.967+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a43fb0 con 0x55d562a39320 2023-10-05T19:40:48.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.971+0000 7f0ba8737700 1 -- 172.21.15.12:0/324485355 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v30'37 uv36 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f0b98052070 con 0x55d562a39320 2023-10-05T19:40:48.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.971+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a312a0 con 0x55d562a40900 2023-10-05T19:40:48.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.975+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v30'22 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a40900 2023-10-05T19:40:48.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.975+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a312a0 con 0x55d562a3cb00 2023-10-05T19:40:48.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.975+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v30'28 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:48.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.975+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a31950 con 0x55d562a40900 2023-10-05T19:40:48.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.979+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v30'57 uv53 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a40900 2023-10-05T19:40:48.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.979+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a31950 con 0x55d562a39320 2023-10-05T19:40:48.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.983+0000 7f0ba8737700 1 -- 172.21.15.12:0/324485355 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v30'25 uv24 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0b98052070 con 0x55d562a39320 2023-10-05T19:40:48.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.983+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a31950 con 0x55d562a40900 2023-10-05T19:40:48.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.983+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v30'22 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a40900 2023-10-05T19:40:48.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.987+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a31950 con 0x55d562a39320 2023-10-05T19:40:48.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.987+0000 7f0ba8737700 1 -- 172.21.15.12:0/324485355 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v30'46 uv45 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0b98052070 con 0x55d562a39320 2023-10-05T19:40:48.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.987+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a319a0 con 0x55d562a35ba0 2023-10-05T19:40:48.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.991+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v30'191 uv187 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a35ba0 2023-10-05T19:40:48.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.991+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a319a0 con 0x55d562a3cb00 2023-10-05T19:40:49.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.995+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v30'50 uv49 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:49.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.995+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a45230 con 0x55d562a40900 2023-10-05T19:40:49.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.999+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v30'58 uv55 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a40900 2023-10-05T19:40:49.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.999+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a45720 con 0x55d562a3cb00 2023-10-05T19:40:49.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.999+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v30'22 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:49.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:48.999+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a454b0 con 0x55d562a35ba0 2023-10-05T19:40:49.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.003+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v30'42 uv41 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a35ba0 2023-10-05T19:40:49.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.003+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a454b0 con 0x55d562a35ba0 2023-10-05T19:40:49.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.007+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v30'192 uv183 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a35ba0 2023-10-05T19:40:49.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.007+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a454b0 con 0x55d562a3cb00 2023-10-05T19:40:49.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.011+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v30'51 uv45 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:49.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.011+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a454b0 con 0x55d562a40900 2023-10-05T19:40:49.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.011+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v30'32 uv31 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a40900 2023-10-05T19:40:49.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.011+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a454b0 con 0x55d562a3cb00 2023-10-05T19:40:49.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.015+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v30'26 uv25 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:49.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.015+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a454b0 con 0x55d562a3cb00 2023-10-05T19:40:49.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.015+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v30'31 uv30 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:49.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.015+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a454b0 con 0x55d562a3cb00 2023-10-05T19:40:49.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.019+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v30'19 uv18 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:49.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.019+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a454b0 con 0x55d562a3cb00 2023-10-05T19:40:49.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.023+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v30'41 uv37 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:49.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.023+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a454b0 con 0x55d562a39320 2023-10-05T19:40:49.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.027+0000 7f0ba8737700 1 -- 172.21.15.12:0/324485355 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v30'28 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0b98052070 con 0x55d562a39320 2023-10-05T19:40:49.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.027+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a454b0 con 0x55d562a3cb00 2023-10-05T19:40:49.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.027+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v30'22 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:49.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.027+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a454b0 con 0x55d562a39320 2023-10-05T19:40:49.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.031+0000 7f0ba8737700 1 -- 172.21.15.12:0/324485355 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v30'38 uv34 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0b98052070 con 0x55d562a39320 2023-10-05T19:40:49.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.031+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a454b0 con 0x55d562a3cb00 2023-10-05T19:40:49.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.035+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v30'52 uv47 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:49.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.035+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a454b0 con 0x55d562a39320 2023-10-05T19:40:49.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.039+0000 7f0ba8737700 1 -- 172.21.15.12:0/324485355 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v30'47 uv43 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0b98052070 con 0x55d562a39320 2023-10-05T19:40:49.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.039+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a454b0 con 0x55d562a40900 2023-10-05T19:40:49.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.039+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v30'28 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a40900 2023-10-05T19:40:49.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.043+0000 7f0ae97fa700 20 reqs_thread_entry: start 2023-10-05T19:40:49.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.043+0000 7f0ae2ffd700 20 reqs_thread_entry: start 2023-10-05T19:40:49.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.043+0000 7f0bab403e40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:49.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.047+0000 7f0bab403e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:49.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.047+0000 7f0ae17fa700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:41:19 2023 2023-10-05T19:40:49.053 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:49.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.047+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:33 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a35ba0 2023-10-05T19:40:49.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.047+0000 7f0ae0ff9700 20 reqs_thread_entry: start 2023-10-05T19:40:49.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.047+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(33 gc.0 [call out=15b] v0'0 uv30 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a35ba0 2023-10-05T19:40:49.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.047+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:34 6.19 6:9a6dd8c0:gc::gc.0:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa3960 con 0x55d562a35ba0 2023-10-05T19:40:49.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.047+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(34 gc.0 [call out=24b] v0'0 uv30 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a35ba0 2023-10-05T19:40:49.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.047+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:35 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a35ba0 2023-10-05T19:40:49.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.047+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(35 gc.0 [call out=15b] v0'0 uv30 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a35ba0 2023-10-05T19:40:49.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.047+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:36 6.18 6:1e85421f:gc::gc.1:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a39320 2023-10-05T19:40:49.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.051+0000 7f0ba8737700 1 -- 172.21.15.12:0/324485355 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(36 gc.1 [call out=15b] v0'0 uv21 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0b98052070 con 0x55d562a39320 2023-10-05T19:40:49.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.051+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:37 6.18 6:1e85421f:gc::gc.1:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa3960 con 0x55d562a39320 2023-10-05T19:40:49.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.051+0000 7f0ba8737700 1 -- 172.21.15.12:0/324485355 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(37 gc.1 [call out=24b] v0'0 uv21 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7f0b98052070 con 0x55d562a39320 2023-10-05T19:40:49.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.051+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:38 6.18 6:1e85421f:gc::gc.1:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a39320 2023-10-05T19:40:49.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.051+0000 7f0ba8737700 1 -- 172.21.15.12:0/324485355 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(38 gc.1 [call out=15b] v0'0 uv21 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0b98052070 con 0x55d562a39320 2023-10-05T19:40:49.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.051+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:39 6.b 6:d242335b:gc::gc.2:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a3cb00 2023-10-05T19:40:49.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.051+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(39 gc.2 [call out=15b] v0'0 uv21 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:49.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.051+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 6.b 6:d242335b:gc::gc.2:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa3960 con 0x55d562a3cb00 2023-10-05T19:40:49.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.051+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(40 gc.2 [call out=24b] v0'0 uv21 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:49.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.051+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 6.b 6:d242335b:gc::gc.2:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a3cb00 2023-10-05T19:40:49.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.055+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(41 gc.2 [call out=15b] v0'0 uv21 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:49.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.055+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 6.4 6:2166d8e1:gc::gc.3:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a35ba0 2023-10-05T19:40:49.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.055+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(42 gc.3 [call out=15b] v0'0 uv185 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a35ba0 2023-10-05T19:40:49.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.055+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 6.4 6:2166d8e1:gc::gc.3:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa3960 con 0x55d562a35ba0 2023-10-05T19:40:49.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.055+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(43 gc.3 [call out=24b] v0'0 uv185 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a35ba0 2023-10-05T19:40:49.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.055+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:44 6.4 6:2166d8e1:gc::gc.3:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a35ba0 2023-10-05T19:40:49.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.055+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(44 gc.3 [call out=15b] v0'0 uv185 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a35ba0 2023-10-05T19:40:49.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.055+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 6.3 6:c60da066:gc::gc.4:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a40900 2023-10-05T19:40:49.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.055+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(45 gc.4 [call out=15b] v0'0 uv51 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a40900 2023-10-05T19:40:49.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.055+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:46 6.3 6:c60da066:gc::gc.4:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa3960 con 0x55d562a40900 2023-10-05T19:40:49.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.055+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(46 gc.4 [call out=24b] v0'0 uv51 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a40900 2023-10-05T19:40:49.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.055+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:47 6.3 6:c60da066:gc::gc.4:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a40900 2023-10-05T19:40:49.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.059+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(47 gc.4 [call out=15b] v0'0 uv51 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a40900 2023-10-05T19:40:49.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.059+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:48 6.9 6:954be8d2:gc::gc.5:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a3cb00 2023-10-05T19:40:49.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.059+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(48 gc.5 [call out=15b] v0'0 uv39 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:49.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.059+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:49 6.9 6:954be8d2:gc::gc.5:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa3960 con 0x55d562a3cb00 2023-10-05T19:40:49.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.059+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(49 gc.5 [call out=24b] v0'0 uv39 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:49.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.059+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:50 6.9 6:954be8d2:gc::gc.5:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a3cb00 2023-10-05T19:40:49.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.059+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(50 gc.5 [call out=15b] v0'0 uv39 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:49.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.059+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:51 6.e 6:748722ff:gc::gc.6:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a35ba0 2023-10-05T19:40:49.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.059+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(51 gc.6 [call out=15b] v0'0 uv23 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a35ba0 2023-10-05T19:40:49.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.059+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:52 6.e 6:748722ff:gc::gc.6:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa3960 con 0x55d562a35ba0 2023-10-05T19:40:49.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.059+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(52 gc.6 [call out=24b] v0'0 uv23 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a35ba0 2023-10-05T19:40:49.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.059+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 6.e 6:748722ff:gc::gc.6:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a35ba0 2023-10-05T19:40:49.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.063+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(53 gc.6 [call out=15b] v0'0 uv23 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a35ba0 2023-10-05T19:40:49.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.063+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 6.1d 6:beaa4be4:gc::gc.7:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a39320 2023-10-05T19:40:49.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.063+0000 7f0ba8737700 1 -- 172.21.15.12:0/324485355 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(54 gc.7 [call out=15b] v0'0 uv36 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0b98052070 con 0x55d562a39320 2023-10-05T19:40:49.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.063+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:55 6.1d 6:beaa4be4:gc::gc.7:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa3960 con 0x55d562a39320 2023-10-05T19:40:49.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.063+0000 7f0ba8737700 1 -- 172.21.15.12:0/324485355 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(55 gc.7 [call out=24b] v0'0 uv36 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7f0b98052070 con 0x55d562a39320 2023-10-05T19:40:49.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.063+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:56 6.1d 6:beaa4be4:gc::gc.7:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a39320 2023-10-05T19:40:49.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.063+0000 7f0ba8737700 1 -- 172.21.15.12:0/324485355 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(56 gc.7 [call out=15b] v0'0 uv36 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0b98052070 con 0x55d562a39320 2023-10-05T19:40:49.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.063+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:57 6.12 6:4f78e9a2:gc::gc.8:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a40900 2023-10-05T19:40:49.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.067+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(57 gc.8 [call out=15b] v0'0 uv21 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a40900 2023-10-05T19:40:49.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.067+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:58 6.12 6:4f78e9a2:gc::gc.8:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa3960 con 0x55d562a40900 2023-10-05T19:40:49.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.067+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(58 gc.8 [call out=24b] v0'0 uv21 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a40900 2023-10-05T19:40:49.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.067+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:59 6.12 6:4f78e9a2:gc::gc.8:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a40900 2023-10-05T19:40:49.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.067+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(59 gc.8 [call out=15b] v0'0 uv21 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a40900 2023-10-05T19:40:49.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.067+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 6.1c 6:3e426232:gc::gc.9:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a3cb00 2023-10-05T19:40:49.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.067+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(60 gc.9 [call out=15b] v0'0 uv27 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:49.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.067+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 6.1c 6:3e426232:gc::gc.9:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa3960 con 0x55d562a3cb00 2023-10-05T19:40:49.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.067+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(61 gc.9 [call out=24b] v0'0 uv27 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:49.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.067+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:62 6.1c 6:3e426232:gc::gc.9:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a3cb00 2023-10-05T19:40:49.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.071+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(62 gc.9 [call out=15b] v0'0 uv27 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:49.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.071+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:63 6.3 6:c31f41d7:gc::gc.10:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a40900 2023-10-05T19:40:49.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.071+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(63 gc.10 [call out=15b] v0'0 uv53 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a40900 2023-10-05T19:40:49.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.071+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:64 6.3 6:c31f41d7:gc::gc.10:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa3960 con 0x55d562a40900 2023-10-05T19:40:49.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.071+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(64 gc.10 [call out=24b] v0'0 uv53 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a40900 2023-10-05T19:40:49.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.071+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:65 6.3 6:c31f41d7:gc::gc.10:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a40900 2023-10-05T19:40:49.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.071+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(65 gc.10 [call out=15b] v0'0 uv53 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a40900 2023-10-05T19:40:49.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.071+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:66 6.13 6:c8333aab:gc::gc.11:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a39320 2023-10-05T19:40:49.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.071+0000 7f0ba8737700 1 -- 172.21.15.12:0/324485355 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(66 gc.11 [call out=15b] v0'0 uv24 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0b98052070 con 0x55d562a39320 2023-10-05T19:40:49.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.071+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:67 6.13 6:c8333aab:gc::gc.11:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa3960 con 0x55d562a39320 2023-10-05T19:40:49.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.075+0000 7f0ba8737700 1 -- 172.21.15.12:0/324485355 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(67 gc.11 [call out=24b] v0'0 uv24 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f0b98052070 con 0x55d562a39320 2023-10-05T19:40:49.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.075+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 6.13 6:c8333aab:gc::gc.11:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a39320 2023-10-05T19:40:49.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.075+0000 7f0ba8737700 1 -- 172.21.15.12:0/324485355 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 gc.11 [call out=15b] v0'0 uv24 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0b98052070 con 0x55d562a39320 2023-10-05T19:40:49.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.075+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:69 6.15 6:ab569f81:gc::gc.12:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a40900 2023-10-05T19:40:49.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.075+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(69 gc.12 [call out=15b] v0'0 uv21 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a40900 2023-10-05T19:40:49.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.075+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:70 6.15 6:ab569f81:gc::gc.12:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa3960 con 0x55d562a40900 2023-10-05T19:40:49.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.079+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(70 gc.12 [call out=24b] v0'0 uv21 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a40900 2023-10-05T19:40:49.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.079+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 6.15 6:ab569f81:gc::gc.12:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a40900 2023-10-05T19:40:49.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.079+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(71 gc.12 [call out=15b] v0'0 uv21 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a40900 2023-10-05T19:40:49.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.079+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:72 6.0 6:0667bb13:gc::gc.13:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a39320 2023-10-05T19:40:49.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.079+0000 7f0ba8737700 1 -- 172.21.15.12:0/324485355 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(72 gc.13 [call out=15b] v0'0 uv45 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0b98052070 con 0x55d562a39320 2023-10-05T19:40:49.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.079+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 6.0 6:0667bb13:gc::gc.13:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa3960 con 0x55d562a39320 2023-10-05T19:40:49.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.079+0000 7f0ba8737700 1 -- 172.21.15.12:0/324485355 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(73 gc.13 [call out=24b] v0'0 uv45 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f0b98052070 con 0x55d562a39320 2023-10-05T19:40:49.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.079+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:74 6.0 6:0667bb13:gc::gc.13:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a39320 2023-10-05T19:40:49.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.079+0000 7f0ba8737700 1 -- 172.21.15.12:0/324485355 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(74 gc.13 [call out=15b] v0'0 uv45 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0b98052070 con 0x55d562a39320 2023-10-05T19:40:49.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.079+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:75 6.4 6:227f63b5:gc::gc.14:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a35ba0 2023-10-05T19:40:49.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.083+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(75 gc.14 [call out=15b] v0'0 uv187 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a35ba0 2023-10-05T19:40:49.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.083+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa3960 con 0x55d562a35ba0 2023-10-05T19:40:49.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.083+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(76 gc.14 [call out=24b] v0'0 uv187 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a35ba0 2023-10-05T19:40:49.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.083+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:77 6.4 6:227f63b5:gc::gc.14:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a35ba0 2023-10-05T19:40:49.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.083+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(77 gc.14 [call out=15b] v0'0 uv187 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a35ba0 2023-10-05T19:40:49.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.083+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:78 6.7 6:e07a6548:gc::gc.15:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a3cb00 2023-10-05T19:40:49.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.083+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(78 gc.15 [call out=15b] v0'0 uv49 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:49.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.083+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 6.7 6:e07a6548:gc::gc.15:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa3960 con 0x55d562a3cb00 2023-10-05T19:40:49.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.083+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(79 gc.15 [call out=24b] v0'0 uv49 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:49.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.083+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:80 6.7 6:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a3cb00 2023-10-05T19:40:49.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.087+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(80 gc.15 [call out=15b] v0'0 uv49 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:49.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.087+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:81 6.3 6:c623b173:gc::gc.16:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a40900 2023-10-05T19:40:49.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.087+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(81 gc.16 [call out=15b] v0'0 uv55 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a40900 2023-10-05T19:40:49.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.087+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:82 6.3 6:c623b173:gc::gc.16:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa3960 con 0x55d562a40900 2023-10-05T19:40:49.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.087+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(82 gc.16 [call out=24b] v0'0 uv55 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a40900 2023-10-05T19:40:49.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.087+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:83 6.3 6:c623b173:gc::gc.16:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a40900 2023-10-05T19:40:49.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.087+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(83 gc.16 [call out=15b] v0'0 uv55 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a40900 2023-10-05T19:40:49.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.087+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:84 6.16 6:6849fd06:gc::gc.17:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a3cb00 2023-10-05T19:40:49.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.087+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(84 gc.17 [call out=15b] v0'0 uv21 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:49.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.087+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:85 6.16 6:6849fd06:gc::gc.17:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa3960 con 0x55d562a3cb00 2023-10-05T19:40:49.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.091+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(85 gc.17 [call out=24b] v0'0 uv21 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:49.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.091+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:86 6.16 6:6849fd06:gc::gc.17:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a3cb00 2023-10-05T19:40:49.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.091+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 27 ==== osd_op_reply(86 gc.17 [call out=15b] v0'0 uv21 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:49.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.091+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 6.d 6:b019ea83:gc::gc.18:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a35ba0 2023-10-05T19:40:49.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.091+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(87 gc.18 [call out=15b] v0'0 uv41 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a35ba0 2023-10-05T19:40:49.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.091+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:88 6.d 6:b019ea83:gc::gc.18:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa3960 con 0x55d562a35ba0 2023-10-05T19:40:49.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.091+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(88 gc.18 [call out=24b] v0'0 uv41 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a35ba0 2023-10-05T19:40:49.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.091+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:89 6.d 6:b019ea83:gc::gc.18:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a35ba0 2023-10-05T19:40:49.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.091+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(89 gc.18 [call out=15b] v0'0 uv41 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a35ba0 2023-10-05T19:40:49.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.091+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:90 6.4 6:24c8e1ae:gc::gc.19:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a35ba0 2023-10-05T19:40:49.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.091+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(90 gc.19 [call out=15b] v0'0 uv183 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a35ba0 2023-10-05T19:40:49.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.091+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:91 6.4 6:24c8e1ae:gc::gc.19:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa3960 con 0x55d562a35ba0 2023-10-05T19:40:49.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.091+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(91 gc.19 [call out=24b] v0'0 uv183 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a35ba0 2023-10-05T19:40:49.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.095+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 6.4 6:24c8e1ae:gc::gc.19:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a35ba0 2023-10-05T19:40:49.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.095+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(92 gc.19 [call out=15b] v0'0 uv183 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a35ba0 2023-10-05T19:40:49.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.095+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:93 6.7 6:e622f76a:gc::gc.20:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a3cb00 2023-10-05T19:40:49.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.095+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 28 ==== osd_op_reply(93 gc.20 [call out=15b] v0'0 uv45 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:49.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.095+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:94 6.7 6:e622f76a:gc::gc.20:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa3960 con 0x55d562a3cb00 2023-10-05T19:40:49.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.095+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 29 ==== osd_op_reply(94 gc.20 [call out=24b] v0'0 uv45 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:49.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.095+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:95 6.7 6:e622f76a:gc::gc.20:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a3cb00 2023-10-05T19:40:49.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.095+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 30 ==== osd_op_reply(95 gc.20 [call out=15b] v0'0 uv45 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:49.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.095+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:96 6.a 6:5127e642:gc::gc.21:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a40900 2023-10-05T19:40:49.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.095+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.2 v2:172.21.15.12:6800/29934 23 ==== osd_op_reply(96 gc.21 [call out=15b] v0'0 uv31 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a40900 2023-10-05T19:40:49.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.095+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:97 6.a 6:5127e642:gc::gc.21:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa3960 con 0x55d562a40900 2023-10-05T19:40:49.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.095+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.2 v2:172.21.15.12:6800/29934 24 ==== osd_op_reply(97 gc.21 [call out=24b] v0'0 uv31 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a40900 2023-10-05T19:40:49.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.095+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:98 6.a 6:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a40900 2023-10-05T19:40:49.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.099+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.2 v2:172.21.15.12:6800/29934 25 ==== osd_op_reply(98 gc.21 [call out=15b] v0'0 uv31 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a40900 2023-10-05T19:40:49.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.099+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:99 6.11 6:8eb99c53:gc::gc.22:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a3cb00 2023-10-05T19:40:49.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.099+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 31 ==== osd_op_reply(99 gc.22 [call out=15b] v0'0 uv25 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:49.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.099+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:100 6.11 6:8eb99c53:gc::gc.22:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa3960 con 0x55d562a3cb00 2023-10-05T19:40:49.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.099+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 32 ==== osd_op_reply(100 gc.22 [call out=24b] v0'0 uv25 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:49.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.099+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:101 6.11 6:8eb99c53:gc::gc.22:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a3cb00 2023-10-05T19:40:49.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.099+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 33 ==== osd_op_reply(101 gc.22 [call out=15b] v0'0 uv25 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:49.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.099+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:102 6.8 6:17d7e8ed:gc::gc.23:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a3cb00 2023-10-05T19:40:49.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.099+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 34 ==== osd_op_reply(102 gc.23 [call out=15b] v0'0 uv30 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:49.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.099+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:103 6.8 6:17d7e8ed:gc::gc.23:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa3960 con 0x55d562a3cb00 2023-10-05T19:40:49.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.099+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 35 ==== osd_op_reply(103 gc.23 [call out=24b] v0'0 uv30 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:49.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.099+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:104 6.8 6:17d7e8ed:gc::gc.23:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a3cb00 2023-10-05T19:40:49.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.099+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 36 ==== osd_op_reply(104 gc.23 [call out=15b] v0'0 uv30 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:49.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.099+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:105 6.1b 6:db9d130d:gc::gc.24:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a3cb00 2023-10-05T19:40:49.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.103+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 37 ==== osd_op_reply(105 gc.24 [call out=15b] v0'0 uv18 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:49.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.103+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:106 6.1b 6:db9d130d:gc::gc.24:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa3960 con 0x55d562a3cb00 2023-10-05T19:40:49.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.103+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 38 ==== osd_op_reply(106 gc.24 [call out=24b] v0'0 uv18 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:49.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.103+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:107 6.1b 6:db9d130d:gc::gc.24:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a3cb00 2023-10-05T19:40:49.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.103+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 39 ==== osd_op_reply(107 gc.24 [call out=15b] v0'0 uv18 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:49.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.103+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:108 6.9 6:921ecdb1:gc::gc.25:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a3cb00 2023-10-05T19:40:49.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.103+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 40 ==== osd_op_reply(108 gc.25 [call out=15b] v0'0 uv37 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:49.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.103+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:109 6.9 6:921ecdb1:gc::gc.25:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa3960 con 0x55d562a3cb00 2023-10-05T19:40:49.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.103+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 41 ==== osd_op_reply(109 gc.25 [call out=24b] v0'0 uv37 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:49.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.103+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:110 6.9 6:921ecdb1:gc::gc.25:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a3cb00 2023-10-05T19:40:49.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.103+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 42 ==== osd_op_reply(110 gc.25 [call out=15b] v0'0 uv37 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:49.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.103+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:111 6.1f 6:fde4dd55:gc::gc.26:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a39320 2023-10-05T19:40:49.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.103+0000 7f0ba8737700 1 -- 172.21.15.12:0/324485355 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(111 gc.26 [call out=15b] v0'0 uv27 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0b98052070 con 0x55d562a39320 2023-10-05T19:40:49.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.103+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:112 6.1f 6:fde4dd55:gc::gc.26:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa3960 con 0x55d562a39320 2023-10-05T19:40:49.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.107+0000 7f0ba8737700 1 -- 172.21.15.12:0/324485355 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(112 gc.26 [call out=24b] v0'0 uv27 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f0b98052070 con 0x55d562a39320 2023-10-05T19:40:49.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.107+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:113 6.1f 6:fde4dd55:gc::gc.26:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a39320 2023-10-05T19:40:49.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.107+0000 7f0ba8737700 1 -- 172.21.15.12:0/324485355 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(113 gc.26 [call out=15b] v0'0 uv27 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0b98052070 con 0x55d562a39320 2023-10-05T19:40:49.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.107+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:114 6.1a 6:5909e671:gc::gc.27:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a3cb00 2023-10-05T19:40:49.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.107+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 43 ==== osd_op_reply(114 gc.27 [call out=15b] v0'0 uv21 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:49.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.107+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:115 6.1a 6:5909e671:gc::gc.27:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa3960 con 0x55d562a3cb00 2023-10-05T19:40:49.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.107+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 44 ==== osd_op_reply(115 gc.27 [call out=24b] v0'0 uv21 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:49.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.107+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:116 6.1a 6:5909e671:gc::gc.27:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a3cb00 2023-10-05T19:40:49.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.111+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 45 ==== osd_op_reply(116 gc.27 [call out=15b] v0'0 uv21 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:49.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.111+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:117 6.1d 6:b88e3271:gc::gc.28:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a39320 2023-10-05T19:40:49.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.111+0000 7f0ba8737700 1 -- 172.21.15.12:0/324485355 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(117 gc.28 [call out=15b] v0'0 uv34 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0b98052070 con 0x55d562a39320 2023-10-05T19:40:49.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.111+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:118 6.1d 6:b88e3271:gc::gc.28:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa3960 con 0x55d562a39320 2023-10-05T19:40:49.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.111+0000 7f0ba8737700 1 -- 172.21.15.12:0/324485355 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(118 gc.28 [call out=24b] v0'0 uv34 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f0b98052070 con 0x55d562a39320 2023-10-05T19:40:49.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.111+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:119 6.1d 6:b88e3271:gc::gc.28:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a39320 2023-10-05T19:40:49.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.111+0000 7f0ba8737700 1 -- 172.21.15.12:0/324485355 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(119 gc.28 [call out=15b] v0'0 uv34 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0b98052070 con 0x55d562a39320 2023-10-05T19:40:49.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.111+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:120 6.7 6:e47c9322:gc::gc.29:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a3cb00 2023-10-05T19:40:49.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.115+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 46 ==== osd_op_reply(120 gc.29 [call out=15b] v0'0 uv47 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:49.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.115+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:121 6.7 6:e47c9322:gc::gc.29:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa3960 con 0x55d562a3cb00 2023-10-05T19:40:49.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.115+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 47 ==== osd_op_reply(121 gc.29 [call out=24b] v0'0 uv47 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:49.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.115+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:122 6.7 6:e47c9322:gc::gc.29:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a3cb00 2023-10-05T19:40:49.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.115+0000 7f0ba9739700 1 -- 172.21.15.12:0/324485355 <== osd.3 v2:172.21.15.12:6816/29951 48 ==== osd_op_reply(122 gc.29 [call out=15b] v0'0 uv47 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0ba0014040 con 0x55d562a3cb00 2023-10-05T19:40:49.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.115+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:123 6.0 6:013d6c33:gc::gc.30:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a39320 2023-10-05T19:40:49.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.115+0000 7f0ba8737700 1 -- 172.21.15.12:0/324485355 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(123 gc.30 [call out=15b] v0'0 uv43 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0b98052070 con 0x55d562a39320 2023-10-05T19:40:49.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.115+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:124 6.0 6:013d6c33:gc::gc.30:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa3960 con 0x55d562a39320 2023-10-05T19:40:49.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.119+0000 7f0ba8737700 1 -- 172.21.15.12:0/324485355 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(124 gc.30 [call out=24b] v0'0 uv43 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f0b98052070 con 0x55d562a39320 2023-10-05T19:40:49.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.119+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:125 6.0 6:013d6c33:gc::gc.30:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a39320 2023-10-05T19:40:49.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.119+0000 7f0ba8737700 1 -- 172.21.15.12:0/324485355 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(125 gc.30 [call out=15b] v0'0 uv43 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0b98052070 con 0x55d562a39320 2023-10-05T19:40:49.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.119+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:126 6.f 6:f24cb4a8:gc::gc.31:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a40900 2023-10-05T19:40:49.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.119+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.2 v2:172.21.15.12:6800/29934 26 ==== osd_op_reply(126 gc.31 [call out=15b] v0'0 uv27 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a40900 2023-10-05T19:40:49.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.119+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:127 6.f 6:f24cb4a8:gc::gc.31:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa3960 con 0x55d562a40900 2023-10-05T19:40:49.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.119+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.2 v2:172.21.15.12:6800/29934 27 ==== osd_op_reply(127 gc.31 [call out=24b] v0'0 uv27 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a40900 2023-10-05T19:40:49.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.119+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:128 6.f 6:f24cb4a8:gc::gc.31:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562aa4970 con 0x55d562a40900 2023-10-05T19:40:49.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.119+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 <== osd.2 v2:172.21.15.12:6800/29934 28 ==== osd_op_reply(128 gc.31 [call out=15b] v0'0 uv27 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a40900 2023-10-05T19:40:49.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.123+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:102 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94374971121568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a62650 con 0x55d562a07c30 2023-10-05T19:40:49.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.123+0000 7f0ba8f38700 1 -- 172.21.15.12:0/1699685380 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(102 notify.0 [watch unwatch cookie 94374971121568] v30'197 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a07c30 2023-10-05T19:40:49.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.123+0000 7f0bab403e40 20 remove_watcher() i=0 2023-10-05T19:40:49.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.123+0000 7f0bab403e40 2 removed watcher, disabling cache 2023-10-05T19:40:49.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.123+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94374971123904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a11fa0 con 0x55d5629cef00 2023-10-05T19:40:49.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.127+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(103 notify.1 [watch unwatch cookie 94374971123904] v30'197 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0ba0014040 con 0x55d5629cef00 2023-10-05T19:40:49.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.127+0000 7f0bab403e40 20 remove_watcher() i=1 2023-10-05T19:40:49.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.127+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:104 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94374971128032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a11fa0 con 0x55d562a0c5b0 2023-10-05T19:40:49.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.131+0000 7f0ba8737700 1 -- 172.21.15.12:0/1699685380 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(104 notify.2 [watch unwatch cookie 94374971128032] v30'197 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0b98052070 con 0x55d562a0c5b0 2023-10-05T19:40:49.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.131+0000 7f0bab403e40 20 remove_watcher() i=2 2023-10-05T19:40:49.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.131+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:105 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94374971133328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a11fa0 con 0x55d562a07c30 2023-10-05T19:40:49.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.135+0000 7f0ba8f38700 1 -- 172.21.15.12:0/1699685380 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(105 notify.3 [watch unwatch cookie 94374971133328] v30'197 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0b9c012a10 con 0x55d562a07c30 2023-10-05T19:40:49.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.135+0000 7f0bab403e40 20 remove_watcher() i=3 2023-10-05T19:40:49.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.135+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:106 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94374971137056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a14d90 con 0x55d5629cef00 2023-10-05T19:40:49.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.135+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(106 notify.4 [watch unwatch cookie 94374971137056] v30'197 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0ba0014040 con 0x55d5629cef00 2023-10-05T19:40:49.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.135+0000 7f0bab403e40 20 remove_watcher() i=4 2023-10-05T19:40:49.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.135+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:107 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94374971139776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a11fa0 con 0x7f0b9c007030 2023-10-05T19:40:49.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.139+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(107 notify.5 [watch unwatch cookie 94374971139776] v30'197 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0ba0014040 con 0x7f0b9c007030 2023-10-05T19:40:49.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.139+0000 7f0bab403e40 20 remove_watcher() i=5 2023-10-05T19:40:49.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.139+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:108 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94374971134432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a15c20 con 0x7f0b9c007030 2023-10-05T19:40:49.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.143+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(108 notify.6 [watch unwatch cookie 94374971134432] v30'197 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0ba0014040 con 0x7f0b9c007030 2023-10-05T19:40:49.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.143+0000 7f0bab403e40 20 remove_watcher() i=6 2023-10-05T19:40:49.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.143+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:109 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94374971143456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a15280 con 0x7f0b9c007030 2023-10-05T19:40:49.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.147+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(109 notify.7 [watch unwatch cookie 94374971143456] v30'197 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0ba0014040 con 0x7f0b9c007030 2023-10-05T19:40:49.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.147+0000 7f0bab403e40 20 remove_watcher() i=7 2023-10-05T19:40:49.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.147+0000 7f0b7a7e4700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:49.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.147+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:110 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94374971146032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d562a635e0 con 0x55d5629cef00 2023-10-05T19:40:49.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.151+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(110 data_loggenerations_metadata [watch unwatch cookie 94374971146032] v30'193 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f0ba0014040 con 0x55d5629cef00 2023-10-05T19:40:49.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.151+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f0b9c007030 msgr2=0x7f0b9c046400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:49.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.151+0000 7f0bab403e40 1 --2- 172.21.15.12:0/1699685380 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f0b9c007030 0x7f0b9c046400 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:49.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.151+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55d5629cef00 msgr2=0x55d562a05b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:49.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.151+0000 7f0bab403e40 1 --2- 172.21.15.12:0/1699685380 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55d5629cef00 0x55d562a05b10 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:49.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.151+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55d562a07c30 msgr2=0x55d562a0a010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:49.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.151+0000 7f0bab403e40 1 --2- 172.21.15.12:0/1699685380 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55d562a07c30 0x55d562a0a010 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:49.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.151+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55d562a0c5b0 msgr2=0x55d562a0e990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:49.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.151+0000 7f0bab403e40 1 --2- 172.21.15.12:0/1699685380 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55d562a0c5b0 0x55d562a0e990 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:49.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.151+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f0b94030c40 msgr2=0x7f0b940330f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:49.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.151+0000 7f0bab403e40 1 --2- 172.21.15.12:0/1699685380 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f0b94030c40 0x7f0b940330f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0b980044f0 tx=0x7f0b98006690 comp rx=0 tx=0).stop 2023-10-05T19:40:49.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.151+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d5629423d0 msgr2=0x55d5629fa080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:49.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.151+0000 7f0ba9739700 1 -- 172.21.15.12:0/1699685380 reap_dead start 2023-10-05T19:40:49.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.151+0000 7f0bab403e40 1 --2- 172.21.15.12:0/1699685380 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d5629423d0 0x55d5629fa080 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c01f040 tx=0x7f0b9c01c000 comp rx=0 tx=0).stop 2023-10-05T19:40:49.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.151+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 shutdown_connections 2023-10-05T19:40:49.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.151+0000 7f0bab403e40 1 --2- 172.21.15.12:0/1699685380 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d5629423d0 0x55d5629fa080 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:49.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.151+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 >> 172.21.15.12:0/1699685380 conn(0x55d5629c8720 msgr2=0x55d5629ed990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:49.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.151+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 shutdown_connections 2023-10-05T19:40:49.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.151+0000 7f0bab403e40 1 -- 172.21.15.12:0/1699685380 wait complete. 2023-10-05T19:40:49.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.151+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55d562a39320 msgr2=0x55d562a3b700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:49.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.151+0000 7f0bab403e40 1 --2- 172.21.15.12:0/324485355 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55d562a39320 0x55d562a3b700 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:49.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.151+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55d562a35ba0 msgr2=0x55d562a37fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:49.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.151+0000 7f0bab403e40 1 --2- 172.21.15.12:0/324485355 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55d562a35ba0 0x55d562a37fb0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:49.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.151+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55d562a40900 msgr2=0x55d562a42ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:49.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.151+0000 7f0bab403e40 1 --2- 172.21.15.12:0/324485355 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55d562a40900 0x55d562a42ce0 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:49.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.151+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55d562a3cb00 msgr2=0x55d562a3eee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:49.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.151+0000 7f0bab403e40 1 --2- 172.21.15.12:0/324485355 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55d562a3cb00 0x55d562a3eee0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:49.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.151+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f0af0030bf0 msgr2=0x7f0af00330a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:49.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.151+0000 7f0bab403e40 1 --2- 172.21.15.12:0/324485355 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f0af0030bf0 0x7f0af00330a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x55d562a16ca0 tx=0x7f0ba00335d0 comp rx=0 tx=0).stop 2023-10-05T19:40:49.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.151+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d562a25500 msgr2=0x55d562a2af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:49.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.151+0000 7f0bab403e40 1 --2- 172.21.15.12:0/324485355 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d562a25500 0x55d562a2af80 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f0b98001330 tx=0x7f0b98038070 comp rx=0 tx=0).stop 2023-10-05T19:40:49.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.151+0000 7f0ba8f38700 1 -- 172.21.15.12:0/324485355 reap_dead start 2023-10-05T19:40:49.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.151+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 shutdown_connections 2023-10-05T19:40:49.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.151+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 >> 172.21.15.12:0/324485355 conn(0x55d562a208c0 msgr2=0x55d562a1e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:49.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.151+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 shutdown_connections 2023-10-05T19:40:49.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.151+0000 7f0bab403e40 1 -- 172.21.15.12:0/324485355 wait complete. 2023-10-05T19:40:49.168 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2023-10-05T19:40:49.169 DEBUG:tasks.util.rgw: json result: [] 2023-10-05T19:40:49.169 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'foo'] 2023-10-05T19:40:49.169 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'foo'] 2023-10-05T19:40:49.169 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid foo 2023-10-05T19:40:49.210 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:49.210 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:49.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.203+0000 7f747b921e40 1 Processor -- start 2023-10-05T19:40:49.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.203+0000 7f747b921e40 1 -- start start 2023-10-05T19:40:49.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.203+0000 7f747b921e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55a11f7b17e0 0x55a11f7b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:49.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.203+0000 7f747b921e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55a11f6fdbd0 con 0x55a11f7b1410 2023-10-05T19:40:49.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.203+0000 7f747b921e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x55a11f6baee0 con 0x55a11f7b17e0 2023-10-05T19:40:49.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.203+0000 7f7478c55700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55a11f7b17e0 0x55a11f7b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:49.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.203+0000 7f7478c55700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55a11f7b17e0 0x55a11f7b06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58436/0 (socket says 172.21.15.12:58436) 2023-10-05T19:40:49.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.203+0000 7f7479456700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x55a11f7b1410 0x55a11f7affc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:60222/0 (socket says 172.21.15.12:60222) 2023-10-05T19:40:49.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.203+0000 7f7478c55700 1 -- 172.21.15.12:0/3466153532 learned_addr learned my addr 172.21.15.12:0/3466153532 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:49.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.203+0000 7f7478c55700 1 -- 172.21.15.12:0/3466153532 >> v1:172.21.15.8:6789/0 conn(0x55a11f7b1410 legacy=0x55a11f7affc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:49.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.203+0000 7f7478c55700 1 -- 172.21.15.12:0/3466153532 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a11f7b0cd0 con 0x55a11f7b17e0 2023-10-05T19:40:49.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.203+0000 7f7478c55700 1 --2- 172.21.15.12:0/3466153532 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a11f7b17e0 0x55a11f7b06d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f746400a5c0 tx=0x7f7464005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee48472cdd546090 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:49.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.207+0000 7f7473fff700 1 -- 172.21.15.12:0/3466153532 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f746400e510 con 0x55a11f7b17e0 2023-10-05T19:40:49.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.207+0000 7f7473fff700 1 -- 172.21.15.12:0/3466153532 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f746400e670 con 0x55a11f7b17e0 2023-10-05T19:40:49.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.207+0000 7f7473fff700 1 -- 172.21.15.12:0/3466153532 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f746400e920 con 0x55a11f7b17e0 2023-10-05T19:40:49.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.207+0000 7f747b921e40 1 -- 172.21.15.12:0/3466153532 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a11f7b17e0 msgr2=0x55a11f7b06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:49.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.207+0000 7f747b921e40 1 --2- 172.21.15.12:0/3466153532 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a11f7b17e0 0x55a11f7b06d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f746400a5c0 tx=0x7f7464005e00 comp rx=0 tx=0).stop 2023-10-05T19:40:49.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.207+0000 7f747b921e40 1 -- 172.21.15.12:0/3466153532 shutdown_connections 2023-10-05T19:40:49.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.207+0000 7f747b921e40 1 --2- 172.21.15.12:0/3466153532 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a11f7b17e0 0x55a11f7b06d0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:49.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.207+0000 7f747b921e40 1 -- 172.21.15.12:0/3466153532 >> 172.21.15.12:0/3466153532 conn(0x55a11f7ab720 msgr2=0x55a11f7adb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:49.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.207+0000 7f747b921e40 1 -- 172.21.15.12:0/3466153532 shutdown_connections 2023-10-05T19:40:49.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.207+0000 7f747b921e40 1 -- 172.21.15.12:0/3466153532 wait complete. 2023-10-05T19:40:49.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.207+0000 7f747b921e40 1 Processor -- start 2023-10-05T19:40:49.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.207+0000 7f747b921e40 1 -- start start 2023-10-05T19:40:49.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.207+0000 7f747b921e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a11f7b1410 0x55a11f7d2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:49.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.207+0000 7f747b921e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a11f67ee30 con 0x55a11f7b1410 2023-10-05T19:40:49.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.207+0000 7f7479456700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a11f7b1410 0x55a11f7d2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:49.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.207+0000 7f7479456700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a11f7b1410 0x55a11f7d2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58440/0 (socket says 172.21.15.12:58440) 2023-10-05T19:40:49.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.207+0000 7f7479456700 1 -- 172.21.15.12:0/1649824056 learned_addr learned my addr 172.21.15.12:0/1649824056 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:49.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.207+0000 7f7479456700 1 -- 172.21.15.12:0/1649824056 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f746400a180 con 0x55a11f7b1410 2023-10-05T19:40:49.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.207+0000 7f7479456700 1 --2- 172.21.15.12:0/1649824056 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a11f7b1410 0x55a11f7d2120 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f746c001cb0 tx=0x7f746c001860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50c46113d673abe server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:49.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.207+0000 7f7472ffd700 1 -- 172.21.15.12:0/1649824056 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f746c014070 con 0x55a11f7b1410 2023-10-05T19:40:49.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.207+0000 7f7472ffd700 1 -- 172.21.15.12:0/1649824056 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f746c00bd40 con 0x55a11f7b1410 2023-10-05T19:40:49.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.211+0000 7f7472ffd700 1 -- 172.21.15.12:0/1649824056 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f746c00f450 con 0x55a11f7b1410 2023-10-05T19:40:49.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.211+0000 7f747b921e40 1 -- 172.21.15.12:0/1649824056 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a11f7b1410 msgr2=0x55a11f7d2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:49.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.211+0000 7f747b921e40 1 --2- 172.21.15.12:0/1649824056 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a11f7b1410 0x55a11f7d2120 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f746c001cb0 tx=0x7f746c001860 comp rx=0 tx=0).stop 2023-10-05T19:40:49.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.211+0000 7f747b921e40 1 -- 172.21.15.12:0/1649824056 shutdown_connections 2023-10-05T19:40:49.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.211+0000 7f747b921e40 1 --2- 172.21.15.12:0/1649824056 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a11f7b1410 0x55a11f7d2120 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:49.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.211+0000 7f747b921e40 1 -- 172.21.15.12:0/1649824056 >> 172.21.15.12:0/1649824056 conn(0x55a11f7ab720 msgr2=0x55a11f7cdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:49.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.211+0000 7f747b921e40 1 -- 172.21.15.12:0/1649824056 shutdown_connections 2023-10-05T19:40:49.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.211+0000 7f747b921e40 1 -- 172.21.15.12:0/1649824056 wait complete. 2023-10-05T19:40:49.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.211+0000 7f747b921e40 1 Processor -- start 2023-10-05T19:40:49.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.211+0000 7f747b921e40 1 -- start start 2023-10-05T19:40:49.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.211+0000 7f747b921e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a11f7b17e0 0x55a11f7d9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:49.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.211+0000 7f747b921e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a11f651ed0 con 0x55a11f7b17e0 2023-10-05T19:40:49.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.211+0000 7f7479456700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a11f7b17e0 0x55a11f7d9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:49.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.211+0000 7f7479456700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a11f7b17e0 0x55a11f7d9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58446/0 (socket says 172.21.15.12:58446) 2023-10-05T19:40:49.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.211+0000 7f7479456700 1 -- 172.21.15.12:0/1858356351 learned_addr learned my addr 172.21.15.12:0/1858356351 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:49.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.215+0000 7f7479456700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a11f6fdbd0 con 0x55a11f7b17e0 2023-10-05T19:40:49.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.215+0000 7f7479456700 1 --2- 172.21.15.12:0/1858356351 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a11f7b17e0 0x55a11f7d9060 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x55a11f7ce800 tx=0x7f746c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:49.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.215+0000 7f74717fa700 1 -- 172.21.15.12:0/1858356351 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f746c00beb0 con 0x55a11f7b17e0 2023-10-05T19:40:49.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.215+0000 7f74717fa700 1 -- 172.21.15.12:0/1858356351 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f746c00f450 con 0x55a11f7b17e0 2023-10-05T19:40:49.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.215+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a11f7d95a0 con 0x55a11f7b17e0 2023-10-05T19:40:49.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.215+0000 7f74717fa700 1 -- 172.21.15.12:0/1858356351 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f746c01cb90 con 0x55a11f7b17e0 2023-10-05T19:40:49.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.215+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a11f7d99e0 con 0x55a11f7b17e0 2023-10-05T19:40:49.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.215+0000 7f747b921e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.215+0000 7f74717fa700 1 -- 172.21.15.12:0/1858356351 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f746c023020 con 0x55a11f7b17e0 2023-10-05T19:40:49.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.219+0000 7f74717fa700 1 --2- 172.21.15.12:0/1858356351 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f745c030c50 0x7f745c033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:49.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.219+0000 7f74717fa700 1 -- 172.21.15.12:0/1858356351 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7f746c008030 con 0x55a11f7b17e0 2023-10-05T19:40:49.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.219+0000 7f7478c55700 1 --2- 172.21.15.12:0/1858356351 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f745c030c50 0x7f745c033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:49.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.219+0000 7f747b921e40 1 --2- 172.21.15.12:0/1858356351 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55a11f7e2b10 0x55a11f7e4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:49.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.219+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7e55a0 con 0x55a11f7e2b10 2023-10-05T19:40:49.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.219+0000 7f7479c57700 1 --2- 172.21.15.12:0/1858356351 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55a11f7e2b10 0x55a11f7e4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:49.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.219+0000 7f7478c55700 1 --2- 172.21.15.12:0/1858356351 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f745c030c50 0x7f745c033100 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7464005e40 tx=0x7f7464013000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:49.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.219+0000 7f7479c57700 1 --2- 172.21.15.12:0/1858356351 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55a11f7e2b10 0x55a11f7e4ef0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:49.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.219+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f747400a500 con 0x55a11f7e2b10 2023-10-05T19:40:49.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.219+0000 7f747b921e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:49.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.219+0000 7f747b921e40 20 realm 2023-10-05T19:40:49.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.219+0000 7f747b921e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.219+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7e2180 con 0x55a11f7e2b10 2023-10-05T19:40:49.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.219+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f747400a500 con 0x55a11f7e2b10 2023-10-05T19:40:49.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.219+0000 7f747b921e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:49.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.219+0000 7f747b921e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:49.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.219+0000 7f747b921e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.219+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7e6080 con 0x55a11f7e2b10 2023-10-05T19:40:49.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.219+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f747400a500 con 0x55a11f7e2b10 2023-10-05T19:40:49.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.219+0000 7f747b921e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:49.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.219+0000 7f747b921e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.219+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7e5c90 con 0x55a11f7e2b10 2023-10-05T19:40:49.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.223+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f747400a500 con 0x55a11f7e2b10 2023-10-05T19:40:49.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.223+0000 7f747b921e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:49.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.223+0000 7f747b921e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.223+0000 7f747b921e40 1 --2- 172.21.15.12:0/1858356351 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55a11f7e7420 0x55a11f7e9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:49.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.223+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7e9d40 con 0x55a11f7e7420 2023-10-05T19:40:49.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.223+0000 7f7479456700 1 --2- 172.21.15.12:0/1858356351 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55a11f7e7420 0x55a11f7e9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:49.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.223+0000 7f7479456700 1 --2- 172.21.15.12:0/1858356351 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55a11f7e7420 0x55a11f7e9800 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:49.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.223+0000 7f7479456700 1 -- 172.21.15.12:0/1858356351 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f746c016070 con 0x55a11f7e7420 2023-10-05T19:40:49.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.223+0000 7f747b921e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:49.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.223+0000 7f747b921e40 20 searching for the correct realm 2023-10-05T19:40:49.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.223+0000 7f747b921e40 1 --2- 172.21.15.12:0/1858356351 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55a11f7ebbc0 0x55a11f7edfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:49.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.223+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ee680 con 0x55a11f7ebbc0 2023-10-05T19:40:49.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.223+0000 7f7478c55700 1 --2- 172.21.15.12:0/1858356351 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55a11f7ebbc0 0x55a11f7edfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:49.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.223+0000 7f7478c55700 1 --2- 172.21.15.12:0/1858356351 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55a11f7ebbc0 0x55a11f7edfd0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:49.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.227+0000 7f7478c55700 1 -- 172.21.15.12:0/1858356351 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f7464017050 con 0x55a11f7ebbc0 2023-10-05T19:40:49.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.227+0000 7f7478c55700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ee680 con 0x55a11f7e7420 2023-10-05T19:40:49.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.227+0000 7f7479456700 1 -- 172.21.15.12:0/1858356351 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f746c016070 con 0x55a11f7e7420 2023-10-05T19:40:49.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.227+0000 7f7479456700 1 --2- 172.21.15.12:0/1858356351 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f746c01a070 0x7f746c042ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:49.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.227+0000 7f7479456700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ee680 con 0x7f746c01a070 2023-10-05T19:40:49.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.227+0000 7f7479c57700 1 --2- 172.21.15.12:0/1858356351 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f746c01a070 0x7f746c042ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:49.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.227+0000 7f7479c57700 1 --2- 172.21.15.12:0/1858356351 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f746c01a070 0x7f746c042ca0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:49.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.227+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7474014040 con 0x7f746c01a070 2023-10-05T19:40:49.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.227+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ee680 con 0x55a11f7e2b10 2023-10-05T19:40:49.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.231+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7474014040 con 0x55a11f7e2b10 2023-10-05T19:40:49.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.231+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ee680 con 0x55a11f7e2b10 2023-10-05T19:40:49.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.231+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7474014040 con 0x55a11f7e2b10 2023-10-05T19:40:49.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.231+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ee680 con 0x55a11f7e2b10 2023-10-05T19:40:49.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.231+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(11 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7474014040 con 0x55a11f7e2b10 2023-10-05T19:40:49.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.231+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ee680 con 0x7f746c01a070 2023-10-05T19:40:49.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.231+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7474014040 con 0x7f746c01a070 2023-10-05T19:40:49.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.231+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ee680 con 0x55a11f7e7420 2023-10-05T19:40:49.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.231+0000 7f7479456700 1 -- 172.21.15.12:0/1858356351 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f746c016070 con 0x55a11f7e7420 2023-10-05T19:40:49.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.231+0000 7f7479456700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ee680 con 0x55a11f7e7420 2023-10-05T19:40:49.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.235+0000 7f7479456700 1 -- 172.21.15.12:0/1858356351 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f746c016070 con 0x55a11f7e7420 2023-10-05T19:40:49.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.235+0000 7f7479456700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ee680 con 0x55a11f7e2b10 2023-10-05T19:40:49.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.235+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7474014040 con 0x55a11f7e2b10 2023-10-05T19:40:49.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.235+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ee680 con 0x7f746c01a070 2023-10-05T19:40:49.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.235+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7474014040 con 0x7f746c01a070 2023-10-05T19:40:49.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.235+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ee680 con 0x55a11f7e2b10 2023-10-05T19:40:49.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.235+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7474014040 con 0x55a11f7e2b10 2023-10-05T19:40:49.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.235+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ee680 con 0x55a11f7e7420 2023-10-05T19:40:49.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.235+0000 7f7479456700 1 -- 172.21.15.12:0/1858356351 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(18 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f746c016070 con 0x55a11f7e7420 2023-10-05T19:40:49.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.235+0000 7f7479456700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ee680 con 0x55a11f7ebbc0 2023-10-05T19:40:49.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.235+0000 7f7478c55700 1 -- 172.21.15.12:0/1858356351 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7464017050 con 0x55a11f7ebbc0 2023-10-05T19:40:49.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.239+0000 7f7478c55700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ee680 con 0x55a11f7ebbc0 2023-10-05T19:40:49.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.239+0000 7f7478c55700 1 -- 172.21.15.12:0/1858356351 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7464017050 con 0x55a11f7ebbc0 2023-10-05T19:40:49.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.239+0000 7f7478c55700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ee680 con 0x7f746c01a070 2023-10-05T19:40:49.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.239+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7474014040 con 0x7f746c01a070 2023-10-05T19:40:49.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.239+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ee680 con 0x55a11f7ebbc0 2023-10-05T19:40:49.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.239+0000 7f7478c55700 1 -- 172.21.15.12:0/1858356351 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7464017050 con 0x55a11f7ebbc0 2023-10-05T19:40:49.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.239+0000 7f7478c55700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ee680 con 0x55a11f7ebbc0 2023-10-05T19:40:49.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.239+0000 7f7478c55700 1 -- 172.21.15.12:0/1858356351 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7464017050 con 0x55a11f7ebbc0 2023-10-05T19:40:49.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.239+0000 7f7478c55700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ee680 con 0x55a11f7e2b10 2023-10-05T19:40:49.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.239+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7474014040 con 0x55a11f7e2b10 2023-10-05T19:40:49.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.239+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ee680 con 0x55a11f7e2b10 2023-10-05T19:40:49.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.243+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7474014040 con 0x55a11f7e2b10 2023-10-05T19:40:49.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.243+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ee680 con 0x7f746c01a070 2023-10-05T19:40:49.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.243+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7474014040 con 0x7f746c01a070 2023-10-05T19:40:49.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.243+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ee680 con 0x55a11f7ebbc0 2023-10-05T19:40:49.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.243+0000 7f7478c55700 1 -- 172.21.15.12:0/1858356351 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7464017050 con 0x55a11f7ebbc0 2023-10-05T19:40:49.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.243+0000 7f7478c55700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ee680 con 0x55a11f7e7420 2023-10-05T19:40:49.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.243+0000 7f7479456700 1 -- 172.21.15.12:0/1858356351 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f746c016070 con 0x55a11f7e7420 2023-10-05T19:40:49.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.243+0000 7f7479456700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ee680 con 0x55a11f7e2b10 2023-10-05T19:40:49.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.247+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7474014040 con 0x55a11f7e2b10 2023-10-05T19:40:49.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.247+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ee680 con 0x7f746c01a070 2023-10-05T19:40:49.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.247+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7474014040 con 0x7f746c01a070 2023-10-05T19:40:49.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.247+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ee680 con 0x55a11f7ebbc0 2023-10-05T19:40:49.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.247+0000 7f7478c55700 1 -- 172.21.15.12:0/1858356351 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7464017050 con 0x55a11f7ebbc0 2023-10-05T19:40:49.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.247+0000 7f7478c55700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ee680 con 0x7f746c01a070 2023-10-05T19:40:49.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.247+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7474014040 con 0x7f746c01a070 2023-10-05T19:40:49.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.247+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ee680 con 0x55a11f7e7420 2023-10-05T19:40:49.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.247+0000 7f7479456700 1 -- 172.21.15.12:0/1858356351 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f746c016070 con 0x55a11f7e7420 2023-10-05T19:40:49.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.247+0000 7f7479456700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ee680 con 0x55a11f7ebbc0 2023-10-05T19:40:49.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.251+0000 7f7478c55700 1 -- 172.21.15.12:0/1858356351 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7464017050 con 0x55a11f7ebbc0 2023-10-05T19:40:49.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.251+0000 7f7478c55700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ee680 con 0x7f746c01a070 2023-10-05T19:40:49.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.251+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7474014040 con 0x7f746c01a070 2023-10-05T19:40:49.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.251+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ee680 con 0x55a11f7e2b10 2023-10-05T19:40:49.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.251+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f7474014040 con 0x55a11f7e2b10 2023-10-05T19:40:49.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.251+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ee680 con 0x55a11f7e7420 2023-10-05T19:40:49.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.251+0000 7f7479456700 1 -- 172.21.15.12:0/1858356351 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(37 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f746c016070 con 0x55a11f7e7420 2023-10-05T19:40:49.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.251+0000 7f747b921e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:49.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.251+0000 7f747b921e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:49.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.251+0000 7f747b921e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:49.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.251+0000 7f747b921e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:49.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.251+0000 7f747b921e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.251+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ea940 con 0x55a11f7e2b10 2023-10-05T19:40:49.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.255+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7474014040 con 0x55a11f7e2b10 2023-10-05T19:40:49.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.255+0000 7f747b921e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:49.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.255+0000 7f747b921e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.255+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ea570 con 0x55a11f7e2b10 2023-10-05T19:40:49.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.255+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f7474014040 con 0x55a11f7e2b10 2023-10-05T19:40:49.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.255+0000 7f747b921e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:49.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.255+0000 7f747b921e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.255+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7eeaf0 con 0x55a11f7ebbc0 2023-10-05T19:40:49.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.255+0000 7f7478c55700 1 -- 172.21.15.12:0/1858356351 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f7464017050 con 0x55a11f7ebbc0 2023-10-05T19:40:49.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.255+0000 7f747b921e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:49.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.255+0000 7f747b921e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.255+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7eeea0 con 0x55a11f7ebbc0 2023-10-05T19:40:49.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.255+0000 7f7478c55700 1 -- 172.21.15.12:0/1858356351 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f7464017050 con 0x55a11f7ebbc0 2023-10-05T19:40:49.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.255+0000 7f747b921e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:49.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.255+0000 7f747b921e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.255+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ef250 con 0x55a11f7e2b10 2023-10-05T19:40:49.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.255+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7474014040 con 0x55a11f7e2b10 2023-10-05T19:40:49.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.255+0000 7f747b921e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:49.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.255+0000 7f747b921e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.255+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ef630 con 0x55a11f7e2b10 2023-10-05T19:40:49.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.259+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f7474014040 con 0x55a11f7e2b10 2023-10-05T19:40:49.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.259+0000 7f747b921e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:49.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.259+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ef600 con 0x55a11f7ebbc0 2023-10-05T19:40:49.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.259+0000 7f7478c55700 1 -- 172.21.15.12:0/1858356351 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(44 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f7464017050 con 0x55a11f7ebbc0 2023-10-05T19:40:49.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.259+0000 7f7478c55700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ef600 con 0x55a11f7e7420 2023-10-05T19:40:49.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.263+0000 7f7479456700 1 -- 172.21.15.12:0/1858356351 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(45 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f746c016070 con 0x55a11f7e7420 2023-10-05T19:40:49.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.263+0000 7f7479456700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ef600 con 0x7f746c01a070 2023-10-05T19:40:49.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.263+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(46 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7474014040 con 0x7f746c01a070 2023-10-05T19:40:49.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.263+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ef250 con 0x55a11f7e2b10 2023-10-05T19:40:49.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.263+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(47 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7474014040 con 0x55a11f7e2b10 2023-10-05T19:40:49.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.263+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ef250 con 0x55a11f7e2b10 2023-10-05T19:40:49.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.263+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(48 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7474014040 con 0x55a11f7e2b10 2023-10-05T19:40:49.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.263+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ef250 con 0x55a11f7e2b10 2023-10-05T19:40:49.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.263+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(49 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7474014040 con 0x55a11f7e2b10 2023-10-05T19:40:49.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.263+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ef250 con 0x7f746c01a070 2023-10-05T19:40:49.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.267+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(50 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7474014040 con 0x7f746c01a070 2023-10-05T19:40:49.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.267+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ef250 con 0x55a11f7e7420 2023-10-05T19:40:49.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.267+0000 7f7479456700 1 -- 172.21.15.12:0/1858356351 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(51 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f746c016070 con 0x55a11f7e7420 2023-10-05T19:40:49.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.267+0000 7f7479456700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ef250 con 0x55a11f7e7420 2023-10-05T19:40:49.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.267+0000 7f7479456700 1 -- 172.21.15.12:0/1858356351 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(52 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f746c016070 con 0x55a11f7e7420 2023-10-05T19:40:49.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.267+0000 7f7479456700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ef250 con 0x55a11f7e2b10 2023-10-05T19:40:49.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.267+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(53 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7474014040 con 0x55a11f7e2b10 2023-10-05T19:40:49.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.267+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ef250 con 0x7f746c01a070 2023-10-05T19:40:49.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.271+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(54 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7474014040 con 0x7f746c01a070 2023-10-05T19:40:49.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.271+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ef250 con 0x55a11f7e2b10 2023-10-05T19:40:49.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.271+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(55 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7474014040 con 0x55a11f7e2b10 2023-10-05T19:40:49.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.271+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ef250 con 0x55a11f7e7420 2023-10-05T19:40:49.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.271+0000 7f7479456700 1 -- 172.21.15.12:0/1858356351 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(56 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f746c016070 con 0x55a11f7e7420 2023-10-05T19:40:49.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.271+0000 7f7479456700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ef250 con 0x55a11f7ebbc0 2023-10-05T19:40:49.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.271+0000 7f7478c55700 1 -- 172.21.15.12:0/1858356351 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(57 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7464017050 con 0x55a11f7ebbc0 2023-10-05T19:40:49.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.271+0000 7f7478c55700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ef250 con 0x55a11f7ebbc0 2023-10-05T19:40:49.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.275+0000 7f7478c55700 1 -- 172.21.15.12:0/1858356351 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(58 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7464017050 con 0x55a11f7ebbc0 2023-10-05T19:40:49.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.275+0000 7f7478c55700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ef250 con 0x7f746c01a070 2023-10-05T19:40:49.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.275+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(59 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7474014040 con 0x7f746c01a070 2023-10-05T19:40:49.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.275+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ef250 con 0x55a11f7ebbc0 2023-10-05T19:40:49.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.275+0000 7f7478c55700 1 -- 172.21.15.12:0/1858356351 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(60 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7464017050 con 0x55a11f7ebbc0 2023-10-05T19:40:49.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.275+0000 7f7478c55700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ef250 con 0x55a11f7ebbc0 2023-10-05T19:40:49.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.275+0000 7f7478c55700 1 -- 172.21.15.12:0/1858356351 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(61 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7464017050 con 0x55a11f7ebbc0 2023-10-05T19:40:49.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.275+0000 7f7478c55700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ef250 con 0x55a11f7e2b10 2023-10-05T19:40:49.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.275+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(62 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7474014040 con 0x55a11f7e2b10 2023-10-05T19:40:49.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.275+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ef250 con 0x55a11f7e2b10 2023-10-05T19:40:49.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.279+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(63 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7474014040 con 0x55a11f7e2b10 2023-10-05T19:40:49.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.279+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ef250 con 0x7f746c01a070 2023-10-05T19:40:49.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.279+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(64 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7474014040 con 0x7f746c01a070 2023-10-05T19:40:49.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.279+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ef250 con 0x55a11f7ebbc0 2023-10-05T19:40:49.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.279+0000 7f7478c55700 1 -- 172.21.15.12:0/1858356351 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(65 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7464017050 con 0x55a11f7ebbc0 2023-10-05T19:40:49.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.279+0000 7f7478c55700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ef250 con 0x55a11f7e7420 2023-10-05T19:40:49.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.279+0000 7f7479456700 1 -- 172.21.15.12:0/1858356351 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(66 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f746c016070 con 0x55a11f7e7420 2023-10-05T19:40:49.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.279+0000 7f7479456700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ef250 con 0x55a11f7e2b10 2023-10-05T19:40:49.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.283+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(67 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7474014040 con 0x55a11f7e2b10 2023-10-05T19:40:49.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.283+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ef250 con 0x7f746c01a070 2023-10-05T19:40:49.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.283+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(68 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7474014040 con 0x7f746c01a070 2023-10-05T19:40:49.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.283+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ef250 con 0x55a11f7ebbc0 2023-10-05T19:40:49.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.283+0000 7f7478c55700 1 -- 172.21.15.12:0/1858356351 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(69 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7464017050 con 0x55a11f7ebbc0 2023-10-05T19:40:49.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.283+0000 7f7478c55700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ef250 con 0x7f746c01a070 2023-10-05T19:40:49.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.283+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(70 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7474014040 con 0x7f746c01a070 2023-10-05T19:40:49.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.283+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ef250 con 0x55a11f7e7420 2023-10-05T19:40:49.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.283+0000 7f7479456700 1 -- 172.21.15.12:0/1858356351 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(71 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f746c016070 con 0x55a11f7e7420 2023-10-05T19:40:49.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.283+0000 7f7479456700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ef250 con 0x55a11f7ebbc0 2023-10-05T19:40:49.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.287+0000 7f7478c55700 1 -- 172.21.15.12:0/1858356351 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(72 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7464017050 con 0x55a11f7ebbc0 2023-10-05T19:40:49.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.287+0000 7f7478c55700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ef250 con 0x7f746c01a070 2023-10-05T19:40:49.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.287+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(73 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7474014040 con 0x7f746c01a070 2023-10-05T19:40:49.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.287+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ef250 con 0x55a11f7e2b10 2023-10-05T19:40:49.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.287+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(74 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f7474014040 con 0x55a11f7e2b10 2023-10-05T19:40:49.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.287+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ef250 con 0x55a11f7e7420 2023-10-05T19:40:49.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.287+0000 7f7479456700 1 -- 172.21.15.12:0/1858356351 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(75 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f746c016070 con 0x55a11f7e7420 2023-10-05T19:40:49.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.287+0000 7f747b921e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:49.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.287+0000 7f747b921e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:49.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.287+0000 7f747b921e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:49.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.287+0000 7f747b921e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:49.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.287+0000 7f747b921e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.287+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7ef600 con 0x55a11f7e2b10 2023-10-05T19:40:49.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.291+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f7474014040 con 0x55a11f7e2b10 2023-10-05T19:40:49.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.291+0000 7f747b921e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:49.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.291+0000 7f747b921e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.291+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7efa60 con 0x55a11f7e7420 2023-10-05T19:40:49.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.291+0000 7f7479456700 1 -- 172.21.15.12:0/1858356351 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f746c016070 con 0x55a11f7e7420 2023-10-05T19:40:49.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.291+0000 7f747b921e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:49.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.291+0000 7f747b921e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.291+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7f0350 con 0x55a11f7e2b10 2023-10-05T19:40:49.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.291+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f7474014040 con 0x55a11f7e2b10 2023-10-05T19:40:49.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.291+0000 7f747b921e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:49.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.291+0000 7f747b921e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.291+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7f0350 con 0x55a11f7ebbc0 2023-10-05T19:40:49.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.291+0000 7f7478c55700 1 -- 172.21.15.12:0/1858356351 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f7464017050 con 0x55a11f7ebbc0 2023-10-05T19:40:49.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.291+0000 7f747b921e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:49.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.291+0000 7f747b921e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.291+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7f0ac0 con 0x7f746c01a070 2023-10-05T19:40:49.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.291+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f7474014040 con 0x7f746c01a070 2023-10-05T19:40:49.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.291+0000 7f747b921e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:49.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.291+0000 7f747b921e40 20 zone default found 2023-10-05T19:40:49.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.291+0000 7f747b921e40 4 Realm: () 2023-10-05T19:40:49.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.291+0000 7f747b921e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:49.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.291+0000 7f747b921e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:49.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.291+0000 7f747b921e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:49.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.291+0000 7f747b921e40 20 zonegroup default 2023-10-05T19:40:49.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.291+0000 7f747b921e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.295+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7f0ac0 con 0x7f746c01a070 2023-10-05T19:40:49.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.295+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f7474014040 con 0x7f746c01a070 2023-10-05T19:40:49.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.295+0000 7f747b921e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:49.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.295+0000 7f747b921e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.295+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7f2440 con 0x55a11f7ebbc0 2023-10-05T19:40:49.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.295+0000 7f7478c55700 1 -- 172.21.15.12:0/1858356351 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f7464017050 con 0x55a11f7ebbc0 2023-10-05T19:40:49.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.295+0000 7f747b921e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:49.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.295+0000 7f747b921e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.295+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7f27f0 con 0x55a11f7ebbc0 2023-10-05T19:40:49.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.295+0000 7f7478c55700 1 -- 172.21.15.12:0/1858356351 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f7464017050 con 0x55a11f7ebbc0 2023-10-05T19:40:49.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.295+0000 7f747b921e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:49.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.295+0000 7f747b921e40 20 started sync module instance, tier type = 2023-10-05T19:40:49.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.295+0000 7f747b921e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:49.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.295+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7f1b00 con 0x55a11f7e7420 2023-10-05T19:40:49.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.299+0000 7f7479456700 1 -- 172.21.15.12:0/1858356351 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(84 notify.0 [create] v30'198 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f746c016070 con 0x55a11f7e7420 2023-10-05T19:40:49.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.299+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94150506511968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7f2ba0 con 0x55a11f7e7420 2023-10-05T19:40:49.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.299+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7f3440 con 0x55a11f7e2b10 2023-10-05T19:40:49.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.303+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(86 notify.1 [create] v30'198 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7474014040 con 0x55a11f7e2b10 2023-10-05T19:40:49.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.303+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94150506524784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7f3a30 con 0x55a11f7e2b10 2023-10-05T19:40:49.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.303+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7f45c0 con 0x55a11f7ebbc0 2023-10-05T19:40:49.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.303+0000 7f7479456700 1 -- 172.21.15.12:0/1858356351 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(85 notify.0 [watch watch cookie 94150506511968] v30'199 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f746c016070 con 0x55a11f7e7420 2023-10-05T19:40:49.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.307+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(87 notify.1 [watch watch cookie 94150506524784] v30'199 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7474014040 con 0x55a11f7e2b10 2023-10-05T19:40:49.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.307+0000 7f7478c55700 1 -- 172.21.15.12:0/1858356351 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(88 notify.2 [create] v30'198 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7464017050 con 0x55a11f7ebbc0 2023-10-05T19:40:49.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.307+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94150506520320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7f34c0 con 0x55a11f7ebbc0 2023-10-05T19:40:49.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.307+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7f4eb0 con 0x55a11f7e7420 2023-10-05T19:40:49.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.307+0000 7f7478c55700 1 -- 172.21.15.12:0/1858356351 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(89 notify.2 [watch watch cookie 94150506520320] v30'199 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7464017050 con 0x55a11f7ebbc0 2023-10-05T19:40:49.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.311+0000 7f7479456700 1 -- 172.21.15.12:0/1858356351 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(90 notify.3 [create] v30'198 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f746c016070 con 0x55a11f7e7420 2023-10-05T19:40:49.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.311+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94150506528912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7f4ae0 con 0x55a11f7e7420 2023-10-05T19:40:49.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.311+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7f5e20 con 0x55a11f7e2b10 2023-10-05T19:40:49.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.311+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(92 notify.4 [create] v30'198 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7474014040 con 0x55a11f7e2b10 2023-10-05T19:40:49.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.311+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94150506535152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7f6230 con 0x55a11f7e2b10 2023-10-05T19:40:49.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.311+0000 7f7479456700 1 -- 172.21.15.12:0/1858356351 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(91 notify.3 [watch watch cookie 94150506528912] v30'199 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f746c016070 con 0x55a11f7e7420 2023-10-05T19:40:49.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.311+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7f5260 con 0x7f746c01a070 2023-10-05T19:40:49.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.315+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(93 notify.4 [watch watch cookie 94150506535152] v30'199 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7474014040 con 0x55a11f7e2b10 2023-10-05T19:40:49.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.315+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(94 notify.5 [create] v30'198 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7474014040 con 0x7f746c01a070 2023-10-05T19:40:49.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.315+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94150506536256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7f6190 con 0x7f746c01a070 2023-10-05T19:40:49.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.315+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7f7190 con 0x7f746c01a070 2023-10-05T19:40:49.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.319+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(95 notify.5 [watch watch cookie 94150506536256] v30'199 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7474014040 con 0x7f746c01a070 2023-10-05T19:40:49.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.319+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(96 notify.6 [create] v30'198 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7474014040 con 0x7f746c01a070 2023-10-05T19:40:49.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.319+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94150506538928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7f70f0 con 0x7f746c01a070 2023-10-05T19:40:49.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.319+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7f7b40 con 0x7f746c01a070 2023-10-05T19:40:49.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.323+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(98 notify.7 [create] v30'198 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7474014040 con 0x7f746c01a070 2023-10-05T19:40:49.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.323+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94150506542608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7f7f50 con 0x7f746c01a070 2023-10-05T19:40:49.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.323+0000 7f747b921e40 20 add_watcher() i=0 2023-10-05T19:40:49.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.323+0000 7f747b921e40 20 add_watcher() i=1 2023-10-05T19:40:49.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.323+0000 7f747b921e40 20 add_watcher() i=2 2023-10-05T19:40:49.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.323+0000 7f747b921e40 20 add_watcher() i=3 2023-10-05T19:40:49.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.323+0000 7f747b921e40 20 add_watcher() i=4 2023-10-05T19:40:49.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.323+0000 7f747b921e40 20 add_watcher() i=5 2023-10-05T19:40:49.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.323+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(97 notify.6 [watch watch cookie 94150506538928] v30'199 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7474014040 con 0x7f746c01a070 2023-10-05T19:40:49.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.323+0000 7f747b921e40 20 add_watcher() i=6 2023-10-05T19:40:49.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.323+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(99 notify.7 [watch watch cookie 94150506542608] v30'199 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7474014040 con 0x7f746c01a070 2023-10-05T19:40:49.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.323+0000 7f747b921e40 20 add_watcher() i=7 2023-10-05T19:40:49.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.323+0000 7f747b921e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:49.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.323+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7f6c00 con 0x55a11f7e2b10 2023-10-05T19:40:49.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.323+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f7474014040 con 0x55a11f7e2b10 2023-10-05T19:40:49.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.323+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94150506543712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7f8410 con 0x55a11f7e2b10 2023-10-05T19:40:49.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.327+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94150506543712] v30'194 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f7474014040 con 0x55a11f7e2b10 2023-10-05T19:40:49.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.327+0000 7f744a7e4700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:49.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.327+0000 7f747b921e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:49.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.327+0000 7f747b921e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:49.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.327+0000 7f747b921e40 1 Processor -- start 2023-10-05T19:40:49.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.327+0000 7f747b921e40 1 -- start start 2023-10-05T19:40:49.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.327+0000 7f747b921e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a11f8055f0 0x55a11f8059c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:49.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.327+0000 7f747b921e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a11f805f00 con 0x55a11f8055f0 2023-10-05T19:40:49.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.327+0000 7f7478c55700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a11f8055f0 0x55a11f8059c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:49.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.327+0000 7f7478c55700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a11f8055f0 0x55a11f8059c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58450/0 (socket says 172.21.15.12:58450) 2023-10-05T19:40:49.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.327+0000 7f7478c55700 1 -- 172.21.15.12:0/3376949493 learned_addr learned my addr 172.21.15.12:0/3376949493 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:49.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.327+0000 7f7478c55700 1 -- 172.21.15.12:0/3376949493 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a11f806040 con 0x55a11f8055f0 2023-10-05T19:40:49.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.331+0000 7f7478c55700 1 --2- 172.21.15.12:0/3376949493 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a11f8055f0 0x55a11f8059c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x55a11f7b0ca0 tx=0x7f7464010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f26da59a2f7f018 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:49.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.331+0000 7f7449fe3700 1 -- 172.21.15.12:0/3376949493 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7464010c50 con 0x55a11f8055f0 2023-10-05T19:40:49.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.331+0000 7f7449fe3700 1 -- 172.21.15.12:0/3376949493 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7464010db0 con 0x55a11f8055f0 2023-10-05T19:40:49.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.331+0000 7f7449fe3700 1 -- 172.21.15.12:0/3376949493 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f746401c430 con 0x55a11f8055f0 2023-10-05T19:40:49.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.331+0000 7f747b921e40 1 -- 172.21.15.12:0/3376949493 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a11f8055f0 msgr2=0x55a11f8059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:49.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.331+0000 7f747b921e40 1 --2- 172.21.15.12:0/3376949493 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a11f8055f0 0x55a11f8059c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x55a11f7b0ca0 tx=0x7f7464010000 comp rx=0 tx=0).stop 2023-10-05T19:40:49.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.331+0000 7f747b921e40 1 -- 172.21.15.12:0/3376949493 shutdown_connections 2023-10-05T19:40:49.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.331+0000 7f747b921e40 1 --2- 172.21.15.12:0/3376949493 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a11f8055f0 0x55a11f8059c0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:49.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.331+0000 7f747b921e40 1 -- 172.21.15.12:0/3376949493 >> 172.21.15.12:0/3376949493 conn(0x55a11f8009b0 msgr2=0x55a11f7feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:49.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.331+0000 7f747b921e40 1 -- 172.21.15.12:0/3376949493 shutdown_connections 2023-10-05T19:40:49.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.331+0000 7f747b921e40 1 -- 172.21.15.12:0/3376949493 wait complete. 2023-10-05T19:40:49.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.331+0000 7f747b921e40 1 Processor -- start 2023-10-05T19:40:49.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.331+0000 7f747b921e40 1 -- start start 2023-10-05T19:40:49.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.331+0000 7f747b921e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a11f8055f0 0x55a11f80b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:49.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.331+0000 7f747b921e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a11f80c110 con 0x55a11f8055f0 2023-10-05T19:40:49.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.331+0000 7f7478c55700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a11f8055f0 0x55a11f80b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:49.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.331+0000 7f7478c55700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a11f8055f0 0x55a11f80b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58456/0 (socket says 172.21.15.12:58456) 2023-10-05T19:40:49.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.331+0000 7f7478c55700 1 -- 172.21.15.12:0/3066183892 learned_addr learned my addr 172.21.15.12:0/3066183892 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:49.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.331+0000 7f7478c55700 1 -- 172.21.15.12:0/3066183892 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a11f7b0cd0 con 0x55a11f8055f0 2023-10-05T19:40:49.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.335+0000 7f7478c55700 1 --2- 172.21.15.12:0/3066183892 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a11f8055f0 0x55a11f80b110 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x55a11f7ff670 tx=0x7f7464033000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:49.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.335+0000 7f73bbfff700 1 -- 172.21.15.12:0/3066183892 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7464010c50 con 0x55a11f8055f0 2023-10-05T19:40:49.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.335+0000 7f73bbfff700 1 -- 172.21.15.12:0/3066183892 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7464041410 con 0x55a11f8055f0 2023-10-05T19:40:49.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.335+0000 7f747b921e40 1 -- 172.21.15.12:0/3066183892 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a11f80b710 con 0x55a11f8055f0 2023-10-05T19:40:49.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.335+0000 7f73bbfff700 1 -- 172.21.15.12:0/3066183892 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74640415c0 con 0x55a11f8055f0 2023-10-05T19:40:49.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.335+0000 7f747b921e40 1 -- 172.21.15.12:0/3066183892 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a11f80bb50 con 0x55a11f8055f0 2023-10-05T19:40:49.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.335+0000 7f73bbfff700 1 -- 172.21.15.12:0/3066183892 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f7464036040 con 0x55a11f8055f0 2023-10-05T19:40:49.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.335+0000 7f73bbfff700 1 --2- 172.21.15.12:0/3066183892 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f73c0030ca0 0x7f73c0033150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:49.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.335+0000 7f73bbfff700 1 -- 172.21.15.12:0/3066183892 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7f7464055bd0 con 0x55a11f8055f0 2023-10-05T19:40:49.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.335+0000 7f7479c57700 1 --2- 172.21.15.12:0/3066183892 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f73c0030ca0 0x7f73c0033150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:49.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.335+0000 7f747b921e40 1 --2- 172.21.15.12:0/3066183892 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55a11f815d60 0x55a11f818170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:49.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.335+0000 7f747b921e40 1 -- 172.21.15.12:0/3066183892 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f8187c0 con 0x55a11f815d60 2023-10-05T19:40:49.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.335+0000 7f7479456700 1 --2- 172.21.15.12:0/3066183892 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55a11f815d60 0x55a11f818170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:49.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.335+0000 7f7479c57700 1 --2- 172.21.15.12:0/3066183892 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f73c0030ca0 0x7f73c0033150 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x55a11f7f7ef0 tx=0x7f7474032880 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:49.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.335+0000 7f7479456700 1 --2- 172.21.15.12:0/3066183892 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55a11f815d60 0x55a11f818170 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:49.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.339+0000 7f7479456700 1 -- 172.21.15.12:0/3066183892 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v30'32 uv30 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f746c016070 con 0x55a11f815d60 2023-10-05T19:40:49.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.339+0000 7f747b921e40 1 --2- 172.21.15.12:0/3066183892 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55a11f819490 0x55a11f81b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:49.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.339+0000 7f747b921e40 1 -- 172.21.15.12:0/3066183892 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f81bf70 con 0x55a11f819490 2023-10-05T19:40:49.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.339+0000 7f7478c55700 1 --2- 172.21.15.12:0/3066183892 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55a11f819490 0x55a11f81b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:49.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.339+0000 7f7478c55700 1 --2- 172.21.15.12:0/3066183892 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55a11f819490 0x55a11f81b870 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:49.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.343+0000 7f7478c55700 1 -- 172.21.15.12:0/3066183892 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v30'23 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f7464017050 con 0x55a11f819490 2023-10-05T19:40:49.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.343+0000 7f747b921e40 1 --2- 172.21.15.12:0/3066183892 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55a11f81cc40 0x55a11f81f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:49.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.343+0000 7f747b921e40 1 -- 172.21.15.12:0/3066183892 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f81f780 con 0x55a11f81cc40 2023-10-05T19:40:49.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.343+0000 7f7479c57700 1 --2- 172.21.15.12:0/3066183892 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55a11f81cc40 0x55a11f81f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:49.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.343+0000 7f7479c57700 1 --2- 172.21.15.12:0/3066183892 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55a11f81cc40 0x55a11f81f050 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:49.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.347+0000 7f7479c57700 1 -- 172.21.15.12:0/3066183892 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v30'23 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f7474014040 con 0x55a11f81cc40 2023-10-05T19:40:49.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.347+0000 7f747b921e40 1 -- 172.21.15.12:0/3066183892 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f81fdd0 con 0x55a11f815d60 2023-10-05T19:40:49.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.351+0000 7f7479456700 1 -- 172.21.15.12:0/3066183892 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v30'195 uv185 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f746c016070 con 0x55a11f815d60 2023-10-05T19:40:49.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.351+0000 7f747b921e40 1 --2- 172.21.15.12:0/3066183892 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55a11f820a70 0x55a11f822e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:49.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.351+0000 7f747b921e40 1 -- 172.21.15.12:0/3066183892 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f8234c0 con 0x55a11f820a70 2023-10-05T19:40:49.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.351+0000 7f7479456700 1 --2- 172.21.15.12:0/3066183892 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55a11f820a70 0x55a11f822e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:49.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.351+0000 7f7479456700 1 --2- 172.21.15.12:0/3066183892 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55a11f820a70 0x55a11f822e50 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:49.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.355+0000 7f7479456700 1 -- 172.21.15.12:0/3066183892 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v30'59 uv51 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f746c016070 con 0x55a11f820a70 2023-10-05T19:40:49.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.355+0000 7f747b921e40 1 -- 172.21.15.12:0/3066183892 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f823ae0 con 0x55a11f81cc40 2023-10-05T19:40:49.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.355+0000 7f7479c57700 1 -- 172.21.15.12:0/3066183892 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v30'42 uv39 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f7474014040 con 0x55a11f81cc40 2023-10-05T19:40:49.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.355+0000 7f747b921e40 1 -- 172.21.15.12:0/3066183892 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f824100 con 0x55a11f815d60 2023-10-05T19:40:49.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.359+0000 7f7479456700 1 -- 172.21.15.12:0/3066183892 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v30'25 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f746c016070 con 0x55a11f815d60 2023-10-05T19:40:49.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.359+0000 7f747b921e40 1 -- 172.21.15.12:0/3066183892 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f824100 con 0x55a11f819490 2023-10-05T19:40:49.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.363+0000 7f7478c55700 1 -- 172.21.15.12:0/3066183892 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v30'39 uv36 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f7464017050 con 0x55a11f819490 2023-10-05T19:40:49.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.363+0000 7f747b921e40 1 -- 172.21.15.12:0/3066183892 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f824720 con 0x55a11f820a70 2023-10-05T19:40:49.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.367+0000 7f7479456700 1 -- 172.21.15.12:0/3066183892 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v30'23 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f746c016070 con 0x55a11f820a70 2023-10-05T19:40:49.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.367+0000 7f747b921e40 1 -- 172.21.15.12:0/3066183892 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f824720 con 0x55a11f81cc40 2023-10-05T19:40:49.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.367+0000 7f7479c57700 1 -- 172.21.15.12:0/3066183892 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v30'29 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f7474014040 con 0x55a11f81cc40 2023-10-05T19:40:49.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.367+0000 7f747b921e40 1 -- 172.21.15.12:0/3066183892 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f824d40 con 0x55a11f820a70 2023-10-05T19:40:49.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.371+0000 7f7479456700 1 -- 172.21.15.12:0/3066183892 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v30'60 uv53 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f746c016070 con 0x55a11f820a70 2023-10-05T19:40:49.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.371+0000 7f747b921e40 1 -- 172.21.15.12:0/3066183892 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f824d40 con 0x55a11f819490 2023-10-05T19:40:49.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.375+0000 7f7478c55700 1 -- 172.21.15.12:0/3066183892 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v30'26 uv24 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f7464017050 con 0x55a11f819490 2023-10-05T19:40:49.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.375+0000 7f747b921e40 1 -- 172.21.15.12:0/3066183892 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f825360 con 0x55a11f820a70 2023-10-05T19:40:49.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.375+0000 7f7479456700 1 -- 172.21.15.12:0/3066183892 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v30'23 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f746c016070 con 0x55a11f820a70 2023-10-05T19:40:49.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.375+0000 7f747b921e40 1 -- 172.21.15.12:0/3066183892 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f825360 con 0x55a11f819490 2023-10-05T19:40:49.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.379+0000 7f7478c55700 1 -- 172.21.15.12:0/3066183892 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v30'48 uv45 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f7464017050 con 0x55a11f819490 2023-10-05T19:40:49.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.379+0000 7f747b921e40 1 -- 172.21.15.12:0/3066183892 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f825850 con 0x55a11f815d60 2023-10-05T19:40:49.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.383+0000 7f7479456700 1 -- 172.21.15.12:0/3066183892 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v30'196 uv187 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f746c016070 con 0x55a11f815d60 2023-10-05T19:40:49.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.383+0000 7f747b921e40 1 -- 172.21.15.12:0/3066183892 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f825850 con 0x55a11f81cc40 2023-10-05T19:40:49.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.383+0000 7f7479c57700 1 -- 172.21.15.12:0/3066183892 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v30'53 uv49 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f7474014040 con 0x55a11f81cc40 2023-10-05T19:40:49.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.383+0000 7f747b921e40 1 -- 172.21.15.12:0/3066183892 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f811650 con 0x55a11f820a70 2023-10-05T19:40:49.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.387+0000 7f7479456700 1 -- 172.21.15.12:0/3066183892 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v30'61 uv55 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f746c016070 con 0x55a11f820a70 2023-10-05T19:40:49.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.387+0000 7f747b921e40 1 -- 172.21.15.12:0/3066183892 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f811650 con 0x55a11f81cc40 2023-10-05T19:40:49.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.391+0000 7f7479c57700 1 -- 172.21.15.12:0/3066183892 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v30'23 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f7474014040 con 0x55a11f81cc40 2023-10-05T19:40:49.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.391+0000 7f747b921e40 1 -- 172.21.15.12:0/3066183892 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f811650 con 0x55a11f815d60 2023-10-05T19:40:49.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.395+0000 7f7479456700 1 -- 172.21.15.12:0/3066183892 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v30'43 uv41 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f746c016070 con 0x55a11f815d60 2023-10-05T19:40:49.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.395+0000 7f747b921e40 1 -- 172.21.15.12:0/3066183892 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f811650 con 0x55a11f815d60 2023-10-05T19:40:49.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.395+0000 7f7479456700 1 -- 172.21.15.12:0/3066183892 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v30'197 uv183 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f746c016070 con 0x55a11f815d60 2023-10-05T19:40:49.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.395+0000 7f747b921e40 1 -- 172.21.15.12:0/3066183892 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f811650 con 0x55a11f81cc40 2023-10-05T19:40:49.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.399+0000 7f7479c57700 1 -- 172.21.15.12:0/3066183892 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v30'54 uv45 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f7474014040 con 0x55a11f81cc40 2023-10-05T19:40:49.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.399+0000 7f747b921e40 1 -- 172.21.15.12:0/3066183892 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f811650 con 0x55a11f820a70 2023-10-05T19:40:49.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.399+0000 7f7479456700 1 -- 172.21.15.12:0/3066183892 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v30'33 uv31 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f746c016070 con 0x55a11f820a70 2023-10-05T19:40:49.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.399+0000 7f747b921e40 1 -- 172.21.15.12:0/3066183892 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f811650 con 0x55a11f81cc40 2023-10-05T19:40:49.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.403+0000 7f7479c57700 1 -- 172.21.15.12:0/3066183892 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v30'27 uv25 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f7474014040 con 0x55a11f81cc40 2023-10-05T19:40:49.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.403+0000 7f747b921e40 1 -- 172.21.15.12:0/3066183892 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f811650 con 0x55a11f81cc40 2023-10-05T19:40:49.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.407+0000 7f7479c57700 1 -- 172.21.15.12:0/3066183892 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v30'32 uv30 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f7474014040 con 0x55a11f81cc40 2023-10-05T19:40:49.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.407+0000 7f747b921e40 1 -- 172.21.15.12:0/3066183892 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f811650 con 0x55a11f81cc40 2023-10-05T19:40:49.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.407+0000 7f7479c57700 1 -- 172.21.15.12:0/3066183892 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v30'20 uv18 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f7474014040 con 0x55a11f81cc40 2023-10-05T19:40:49.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.407+0000 7f747b921e40 1 -- 172.21.15.12:0/3066183892 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f811650 con 0x55a11f81cc40 2023-10-05T19:40:49.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.411+0000 7f7479c57700 1 -- 172.21.15.12:0/3066183892 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v30'43 uv37 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f7474014040 con 0x55a11f81cc40 2023-10-05T19:40:49.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.411+0000 7f747b921e40 1 -- 172.21.15.12:0/3066183892 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f811650 con 0x55a11f819490 2023-10-05T19:40:49.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.415+0000 7f7478c55700 1 -- 172.21.15.12:0/3066183892 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v30'29 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f7464017050 con 0x55a11f819490 2023-10-05T19:40:49.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.415+0000 7f747b921e40 1 -- 172.21.15.12:0/3066183892 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f811650 con 0x55a11f81cc40 2023-10-05T19:40:49.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.415+0000 7f7479c57700 1 -- 172.21.15.12:0/3066183892 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v30'23 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f7474014040 con 0x55a11f81cc40 2023-10-05T19:40:49.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.415+0000 7f747b921e40 1 -- 172.21.15.12:0/3066183892 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f811650 con 0x55a11f819490 2023-10-05T19:40:49.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.419+0000 7f7478c55700 1 -- 172.21.15.12:0/3066183892 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v30'40 uv34 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f7464017050 con 0x55a11f819490 2023-10-05T19:40:49.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.419+0000 7f747b921e40 1 -- 172.21.15.12:0/3066183892 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f811650 con 0x55a11f81cc40 2023-10-05T19:40:49.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.423+0000 7f7479c57700 1 -- 172.21.15.12:0/3066183892 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v30'55 uv47 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f7474014040 con 0x55a11f81cc40 2023-10-05T19:40:49.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.423+0000 7f747b921e40 1 -- 172.21.15.12:0/3066183892 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f811650 con 0x55a11f819490 2023-10-05T19:40:49.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.423+0000 7f7478c55700 1 -- 172.21.15.12:0/3066183892 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v30'49 uv43 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f7464017050 con 0x55a11f819490 2023-10-05T19:40:49.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.423+0000 7f747b921e40 1 -- 172.21.15.12:0/3066183892 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f811650 con 0x55a11f820a70 2023-10-05T19:40:49.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.427+0000 7f7479456700 1 -- 172.21.15.12:0/3066183892 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v30'29 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f746c016070 con 0x55a11f820a70 2023-10-05T19:40:49.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.431+0000 7f73b97fa700 20 reqs_thread_entry: start 2023-10-05T19:40:49.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.431+0000 7f73b2ffd700 20 reqs_thread_entry: start 2023-10-05T19:40:49.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.431+0000 7f747b921e40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:49.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.431+0000 7f747b921e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:49.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.431+0000 7f73b0ff9700 20 reqs_thread_entry: start 2023-10-05T19:40:49.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.431+0000 7f747b921e40 20 get_system_obj_state: rctx=0x7fff70f41d18 obj=default.rgw.meta:users.uid:foo state=0x55a11f68a0b0 s->prefetch_data=0 2023-10-05T19:40:49.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.431+0000 7f747b921e40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2023-10-05T19:40:49.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.431+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f884640 con 0x55a11f7e2b10 2023-10-05T19:40:49.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.431+0000 7f73b17fa700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:41:19 2023 2023-10-05T19:40:49.440 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:49.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.431+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(102 foo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv13 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7f7474014040 con 0x55a11f7e2b10 2023-10-05T19:40:49.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.431+0000 7f747b921e40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x16 2023-10-05T19:40:49.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.431+0000 7f747b921e40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:49.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.431+0000 7f747b921e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:49.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.431+0000 7f747b921e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:49.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.431+0000 7f747b921e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:49.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.431+0000 7f747b921e40 10 cache get: name=default.rgw.meta+users.uid+foo : type miss (requested=0x13, cached=0x16) 2023-10-05T19:40:49.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.431+0000 7f747b921e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.431+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f8870d0 con 0x55a11f7e2b10 2023-10-05T19:40:49.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.435+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(103 foo [call,call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv13 ondisk = 0) v8 ==== 273+0+480 (crc 0 0 0) 0x7f7474014040 con 0x55a11f7e2b10 2023-10-05T19:40:49.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.435+0000 7f747b921e40 20 rados_obj.operate() r=0 bl.length=357 2023-10-05T19:40:49.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.435+0000 7f747b921e40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2023-10-05T19:40:49.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.435+0000 7f747b921e40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:49.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.435+0000 7f747b921e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:49.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.435+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:104 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f8870d0 con 0x55a11f7e7420 2023-10-05T19:40:49.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.435+0000 7f7479456700 1 -- 172.21.15.12:0/1858356351 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(104 foo.buckets [call out=183b] v0'0 uv26 ondisk = 0) v8 ==== 155+0+183 (crc 0 0 0) 0x7f746c016070 con 0x55a11f7e7420 2023-10-05T19:40:49.444 INFO:teuthology.orchestra.run.smithi012.stderr:could not remove user: unable to remove user, must specify purge data to remove user with buckets 2023-10-05T19:40:49.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.435+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:105 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94150506511968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f842bf0 con 0x55a11f7e7420 2023-10-05T19:40:49.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.439+0000 7f7479456700 1 -- 172.21.15.12:0/1858356351 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(105 notify.0 [watch unwatch cookie 94150506511968] v30'200 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f746c016070 con 0x55a11f7e7420 2023-10-05T19:40:49.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.439+0000 7f747b921e40 20 remove_watcher() i=0 2023-10-05T19:40:49.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.439+0000 7f747b921e40 2 removed watcher, disabling cache 2023-10-05T19:40:49.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.439+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:106 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94150506524784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7c8a00 con 0x55a11f7e2b10 2023-10-05T19:40:49.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.443+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(106 notify.1 [watch unwatch cookie 94150506524784] v30'200 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7474014040 con 0x55a11f7e2b10 2023-10-05T19:40:49.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.443+0000 7f747b921e40 20 remove_watcher() i=1 2023-10-05T19:40:49.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.443+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:107 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94150506520320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7c8a00 con 0x55a11f7ebbc0 2023-10-05T19:40:49.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.443+0000 7f7478c55700 1 -- 172.21.15.12:0/1858356351 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(107 notify.2 [watch unwatch cookie 94150506520320] v30'200 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7464017050 con 0x55a11f7ebbc0 2023-10-05T19:40:49.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.443+0000 7f747b921e40 20 remove_watcher() i=2 2023-10-05T19:40:49.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.443+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:108 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94150506528912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7c8a00 con 0x55a11f7e7420 2023-10-05T19:40:49.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.447+0000 7f7479456700 1 -- 172.21.15.12:0/1858356351 <== osd.2 v2:172.21.15.12:6800/29934 23 ==== osd_op_reply(108 notify.3 [watch unwatch cookie 94150506528912] v30'200 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f746c016070 con 0x55a11f7e7420 2023-10-05T19:40:49.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.447+0000 7f747b921e40 20 remove_watcher() i=3 2023-10-05T19:40:49.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.447+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:109 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94150506535152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7c8a00 con 0x55a11f7e2b10 2023-10-05T19:40:49.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.451+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== osd_op_reply(109 notify.4 [watch unwatch cookie 94150506535152] v30'200 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7474014040 con 0x55a11f7e2b10 2023-10-05T19:40:49.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.451+0000 7f747b921e40 20 remove_watcher() i=4 2023-10-05T19:40:49.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.451+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:110 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94150506536256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7c8a00 con 0x7f746c01a070 2023-10-05T19:40:49.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.451+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(110 notify.5 [watch unwatch cookie 94150506536256] v30'200 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7474014040 con 0x7f746c01a070 2023-10-05T19:40:49.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.455+0000 7f747b921e40 20 remove_watcher() i=5 2023-10-05T19:40:49.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.455+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:111 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94150506538928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7f5de0 con 0x7f746c01a070 2023-10-05T19:40:49.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.455+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(111 notify.6 [watch unwatch cookie 94150506538928] v30'200 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7474014040 con 0x7f746c01a070 2023-10-05T19:40:49.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.455+0000 7f747b921e40 20 remove_watcher() i=6 2023-10-05T19:40:49.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.455+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:112 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94150506542608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7c8a00 con 0x7f746c01a070 2023-10-05T19:40:49.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.459+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(112 notify.7 [watch unwatch cookie 94150506542608] v30'200 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7474014040 con 0x7f746c01a070 2023-10-05T19:40:49.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.459+0000 7f747b921e40 20 remove_watcher() i=7 2023-10-05T19:40:49.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.463+0000 7f744a7e4700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:49.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.463+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:113 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94150506543712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a11f7c8a00 con 0x55a11f7e2b10 2023-10-05T19:40:49.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.463+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 <== osd.1 v2:172.21.15.12:6808/29933 39 ==== osd_op_reply(113 data_loggenerations_metadata [watch unwatch cookie 94150506543712] v30'198 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f7474014040 con 0x55a11f7e2b10 2023-10-05T19:40:49.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.463+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f746c01a070 msgr2=0x7f746c042ca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:49.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.463+0000 7f747b921e40 1 --2- 172.21.15.12:0/1858356351 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f746c01a070 0x7f746c042ca0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:49.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.463+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55a11f7e2b10 msgr2=0x55a11f7e4ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:49.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.463+0000 7f747b921e40 1 --2- 172.21.15.12:0/1858356351 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55a11f7e2b10 0x55a11f7e4ef0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:49.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.463+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55a11f7e7420 msgr2=0x55a11f7e9800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:49.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.463+0000 7f747b921e40 1 --2- 172.21.15.12:0/1858356351 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55a11f7e7420 0x55a11f7e9800 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:49.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.463+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55a11f7ebbc0 msgr2=0x55a11f7edfd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:49.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.463+0000 7f747b921e40 1 --2- 172.21.15.12:0/1858356351 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55a11f7ebbc0 0x55a11f7edfd0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:49.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.467+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f745c030c50 msgr2=0x7f745c033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:49.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.467+0000 7f747b921e40 1 --2- 172.21.15.12:0/1858356351 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f745c030c50 0x7f745c033100 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7464005e40 tx=0x7f7464013000 comp rx=0 tx=0).stop 2023-10-05T19:40:49.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.467+0000 7f7479c57700 1 -- 172.21.15.12:0/1858356351 reap_dead start 2023-10-05T19:40:49.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.467+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a11f7b17e0 msgr2=0x55a11f7d9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:49.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.467+0000 7f747b921e40 1 --2- 172.21.15.12:0/1858356351 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a11f7b17e0 0x55a11f7d9060 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x55a11f7ce800 tx=0x7f746c016000 comp rx=0 tx=0).stop 2023-10-05T19:40:49.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.467+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 shutdown_connections 2023-10-05T19:40:49.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.467+0000 7f747b921e40 1 --2- 172.21.15.12:0/1858356351 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a11f7b17e0 0x55a11f7d9060 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:49.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.467+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 >> 172.21.15.12:0/1858356351 conn(0x55a11f7ab720 msgr2=0x55a11f7cc760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:49.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.467+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 shutdown_connections 2023-10-05T19:40:49.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.467+0000 7f747b921e40 1 -- 172.21.15.12:0/1858356351 wait complete. 2023-10-05T19:40:49.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.467+0000 7f747b921e40 1 -- 172.21.15.12:0/3066183892 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55a11f819490 msgr2=0x55a11f81b870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:49.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.467+0000 7f747b921e40 1 --2- 172.21.15.12:0/3066183892 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55a11f819490 0x55a11f81b870 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:49.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.467+0000 7f747b921e40 1 -- 172.21.15.12:0/3066183892 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55a11f815d60 msgr2=0x55a11f818170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:49.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.467+0000 7f747b921e40 1 --2- 172.21.15.12:0/3066183892 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55a11f815d60 0x55a11f818170 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:49.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.467+0000 7f747b921e40 1 -- 172.21.15.12:0/3066183892 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55a11f820a70 msgr2=0x55a11f822e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:49.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.467+0000 7f747b921e40 1 --2- 172.21.15.12:0/3066183892 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55a11f820a70 0x55a11f822e50 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:49.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.467+0000 7f747b921e40 1 -- 172.21.15.12:0/3066183892 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55a11f81cc40 msgr2=0x55a11f81f050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:49.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.467+0000 7f747b921e40 1 --2- 172.21.15.12:0/3066183892 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55a11f81cc40 0x55a11f81f050 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:49.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.467+0000 7f747b921e40 1 -- 172.21.15.12:0/3066183892 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f73c0030ca0 msgr2=0x7f73c0033150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:49.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.467+0000 7f747b921e40 1 --2- 172.21.15.12:0/3066183892 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f73c0030ca0 0x7f73c0033150 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x55a11f7f7ef0 tx=0x7f7474032880 comp rx=0 tx=0).stop 2023-10-05T19:40:49.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.467+0000 7f747b921e40 1 -- 172.21.15.12:0/3066183892 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a11f8055f0 msgr2=0x55a11f80b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:49.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.467+0000 7f747b921e40 1 --2- 172.21.15.12:0/3066183892 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a11f8055f0 0x55a11f80b110 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x55a11f7ff670 tx=0x7f7464033000 comp rx=0 tx=0).stop 2023-10-05T19:40:49.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.467+0000 7f7479456700 1 -- 172.21.15.12:0/3066183892 reap_dead start 2023-10-05T19:40:49.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.467+0000 7f747b921e40 1 -- 172.21.15.12:0/3066183892 shutdown_connections 2023-10-05T19:40:49.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.467+0000 7f747b921e40 1 -- 172.21.15.12:0/3066183892 >> 172.21.15.12:0/3066183892 conn(0x55a11f8009b0 msgr2=0x55a11f7fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:49.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.467+0000 7f747b921e40 1 -- 172.21.15.12:0/3066183892 shutdown_connections 2023-10-05T19:40:49.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.467+0000 7f747b921e40 1 -- 172.21.15.12:0/3066183892 wait complete. 2023-10-05T19:40:49.481 DEBUG:teuthology.orchestra.run:got remote process result: 17 2023-10-05T19:40:49.500 INFO:tasks.radosgw_admin:RL: delete_bucket myfoo foo bytes_out=0 bytes_in=217 failed=True 2023-10-05T19:40:49.510 INFO:tasks.radosgw_admin:RL: delete_obj myfoo foo bytes_out=0 bytes_in=0 failed=False 2023-10-05T19:40:49.596 INFO:tasks.radosgw_admin:RL: delete_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2023-10-05T19:40:49.617 INFO:tasks.radosgw_admin:RL: create_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2023-10-05T19:40:49.663 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=5 bytes_in=0 failed=False 2023-10-05T19:40:49.674 INFO:tasks.radosgw_admin:RL: put_acls myfoo foo bytes_out=0 bytes_in=0 failed=False 2023-10-05T19:40:49.674 INFO:tasks.util.rgw:rgwadmin: client.0 : ['policy', '--bucket', 'myfoo', '--object', 'bb3aec0fdcdbc2974890f805c585d432'] 2023-10-05T19:40:49.674 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'xml', '-n', 'client.0', '--cluster', 'ceph', 'policy', '--bucket', 'myfoo', '--object', 'bb3aec0fdcdbc2974890f805c585d432'] 2023-10-05T19:40:49.674 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format xml -n client.0 --cluster ceph policy --bucket myfoo --object bb3aec0fdcdbc2974890f805c585d432 2023-10-05T19:40:49.713 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:49.713 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:49.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.707+0000 7f4f72641e40 1 Processor -- start 2023-10-05T19:40:49.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.707+0000 7f4f72641e40 1 -- start start 2023-10-05T19:40:49.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.707+0000 7f4f72641e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x563759500b50 0x563759592000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:49.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.707+0000 7f4f72641e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5637594d9bd0 con 0x563759592540 2023-10-05T19:40:49.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.707+0000 7f4f72641e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x563759496ee0 con 0x563759500b50 2023-10-05T19:40:49.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.707+0000 7f4f70176700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x563759500b50 0x563759592000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:49.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.707+0000 7f4f6f975700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x563759592540 0x563759594920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:60234/0 (socket says 172.21.15.12:60234) 2023-10-05T19:40:49.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.707+0000 7f4f6f975700 1 -- 172.21.15.12:0/1652309360 learned_addr learned my addr 172.21.15.12:0/1652309360 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:49.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.707+0000 7f4f6f174700 1 -- 172.21.15.12:0/1652309360 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3434843853 0 0) 0x5637594d9bd0 con 0x563759592540 2023-10-05T19:40:49.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.707+0000 7f4f6f174700 1 -- 172.21.15.12:0/1652309360 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f58001e10 con 0x563759592540 2023-10-05T19:40:49.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.707+0000 7f4f70176700 1 -- 172.21.15.12:0/1652309360 >> v1:172.21.15.8:6789/0 conn(0x563759592540 legacy=0x563759594920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:49.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.707+0000 7f4f70176700 1 -- 172.21.15.12:0/1652309360 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563759500f20 con 0x563759500b50 2023-10-05T19:40:49.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.707+0000 7f4f70176700 1 --2- 172.21.15.12:0/1652309360 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563759500b50 0x563759592000 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4f6400a5d0 tx=0x7f4f64005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e818823421b9ff3d server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:49.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.707+0000 7f4f6f174700 1 -- 172.21.15.12:0/1652309360 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f64007610 con 0x563759500b50 2023-10-05T19:40:49.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.707+0000 7f4f6f174700 1 -- 172.21.15.12:0/1652309360 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f64007770 con 0x563759500b50 2023-10-05T19:40:49.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.707+0000 7f4f6f174700 1 -- 172.21.15.12:0/1652309360 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f64007a40 con 0x563759500b50 2023-10-05T19:40:49.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.707+0000 7f4f72641e40 1 -- 172.21.15.12:0/1652309360 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563759500b50 msgr2=0x563759592000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:49.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.707+0000 7f4f72641e40 1 --2- 172.21.15.12:0/1652309360 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563759500b50 0x563759592000 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4f6400a5d0 tx=0x7f4f64005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:49.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.707+0000 7f4f72641e40 1 -- 172.21.15.12:0/1652309360 shutdown_connections 2023-10-05T19:40:49.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.707+0000 7f4f72641e40 1 --2- 172.21.15.12:0/1652309360 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563759500b50 0x563759592000 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:49.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.707+0000 7f4f72641e40 1 -- 172.21.15.12:0/1652309360 >> 172.21.15.12:0/1652309360 conn(0x563759587750 msgr2=0x563759589b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:49.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.707+0000 7f4f72641e40 1 -- 172.21.15.12:0/1652309360 shutdown_connections 2023-10-05T19:40:49.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.707+0000 7f4f72641e40 1 -- 172.21.15.12:0/1652309360 wait complete. 2023-10-05T19:40:49.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.711+0000 7f4f72641e40 1 Processor -- start 2023-10-05T19:40:49.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.711+0000 7f4f72641e40 1 -- start start 2023-10-05T19:40:49.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.711+0000 7f4f72641e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563759500b50 0x5637595adf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:49.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.711+0000 7f4f72641e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56375945ae30 con 0x563759500b50 2023-10-05T19:40:49.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.711+0000 7f4f70176700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563759500b50 0x5637595adf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:49.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.711+0000 7f4f70176700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563759500b50 0x5637595adf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58486/0 (socket says 172.21.15.12:58486) 2023-10-05T19:40:49.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.711+0000 7f4f70176700 1 -- 172.21.15.12:0/2613551542 learned_addr learned my addr 172.21.15.12:0/2613551542 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:49.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.711+0000 7f4f70176700 1 -- 172.21.15.12:0/2613551542 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f64005cf0 con 0x563759500b50 2023-10-05T19:40:49.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.711+0000 7f4f70176700 1 --2- 172.21.15.12:0/2613551542 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563759500b50 0x5637595adf40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4f64000c00 tx=0x7f4f64014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c357e3fc7130627 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:49.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.711+0000 7f4f6e172700 1 -- 172.21.15.12:0/2613551542 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f640078e0 con 0x563759500b50 2023-10-05T19:40:49.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.711+0000 7f4f6e172700 1 -- 172.21.15.12:0/2613551542 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f64007a40 con 0x563759500b50 2023-10-05T19:40:49.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.711+0000 7f4f6e172700 1 -- 172.21.15.12:0/2613551542 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f64007bd0 con 0x563759500b50 2023-10-05T19:40:49.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.711+0000 7f4f72641e40 1 -- 172.21.15.12:0/2613551542 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563759500b50 msgr2=0x5637595adf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:49.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.711+0000 7f4f72641e40 1 --2- 172.21.15.12:0/2613551542 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563759500b50 0x5637595adf40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4f64000c00 tx=0x7f4f64014000 comp rx=0 tx=0).stop 2023-10-05T19:40:49.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.711+0000 7f4f72641e40 1 -- 172.21.15.12:0/2613551542 shutdown_connections 2023-10-05T19:40:49.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.711+0000 7f4f72641e40 1 --2- 172.21.15.12:0/2613551542 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563759500b50 0x5637595adf40 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:49.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.711+0000 7f4f72641e40 1 -- 172.21.15.12:0/2613551542 >> 172.21.15.12:0/2613551542 conn(0x563759587750 msgr2=0x5637595a9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:49.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.711+0000 7f4f72641e40 1 -- 172.21.15.12:0/2613551542 shutdown_connections 2023-10-05T19:40:49.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.711+0000 7f4f72641e40 1 -- 172.21.15.12:0/2613551542 wait complete. 2023-10-05T19:40:49.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.711+0000 7f4f72641e40 1 Processor -- start 2023-10-05T19:40:49.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.711+0000 7f4f72641e40 1 -- start start 2023-10-05T19:40:49.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.715+0000 7f4f72641e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563759500b50 0x5637595b4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:49.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.715+0000 7f4f72641e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56375942ded0 con 0x563759500b50 2023-10-05T19:40:49.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.715+0000 7f4f70176700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563759500b50 0x5637595b4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:49.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.715+0000 7f4f70176700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563759500b50 0x5637595b4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58502/0 (socket says 172.21.15.12:58502) 2023-10-05T19:40:49.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.715+0000 7f4f70176700 1 -- 172.21.15.12:0/832711277 learned_addr learned my addr 172.21.15.12:0/832711277 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:49.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.715+0000 7f4f70176700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563759500f20 con 0x563759500b50 2023-10-05T19:40:49.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.715+0000 7f4f70176700 1 --2- 172.21.15.12:0/832711277 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563759500b50 0x5637595b4c70 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4f6401f040 tx=0x7f4f6401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:49.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.715+0000 7f4f6c96f700 1 -- 172.21.15.12:0/832711277 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f640078e0 con 0x563759500b50 2023-10-05T19:40:49.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.715+0000 7f4f6c96f700 1 -- 172.21.15.12:0/832711277 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f64026650 con 0x563759500b50 2023-10-05T19:40:49.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.715+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5637595b51b0 con 0x563759500b50 2023-10-05T19:40:49.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.715+0000 7f4f6c96f700 1 -- 172.21.15.12:0/832711277 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f64026820 con 0x563759500b50 2023-10-05T19:40:49.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.715+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5637595b55f0 con 0x563759500b50 2023-10-05T19:40:49.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.715+0000 7f4f72641e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.715+0000 7f4f6c96f700 1 -- 172.21.15.12:0/832711277 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f4f64026980 con 0x563759500b50 2023-10-05T19:40:49.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.719+0000 7f4f6c96f700 1 --2- 172.21.15.12:0/832711277 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f4f5c030c90 0x7f4f5c033140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:49.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.719+0000 7f4f6c96f700 1 -- 172.21.15.12:0/832711277 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7f4f64046a90 con 0x563759500b50 2023-10-05T19:40:49.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.719+0000 7f4f6f975700 1 --2- 172.21.15.12:0/832711277 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f4f5c030c90 0x7f4f5c033140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:49.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.719+0000 7f4f72641e40 1 --2- 172.21.15.12:0/832711277 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x563759592540 0x5637595c0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:49.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.719+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595c0da0 con 0x563759592540 2023-10-05T19:40:49.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.719+0000 7f4f70977700 1 --2- 172.21.15.12:0/832711277 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x563759592540 0x5637595c0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:49.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.719+0000 7f4f6f975700 1 --2- 172.21.15.12:0/832711277 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f4f5c030c90 0x7f4f5c033140 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4f60004170 tx=0x7f4f60001360 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:49.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.719+0000 7f4f70977700 1 --2- 172.21.15.12:0/832711277 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x563759592540 0x5637595c0730 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:49.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.719+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4f6800a5a0 con 0x563759592540 2023-10-05T19:40:49.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.719+0000 7f4f72641e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:49.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.719+0000 7f4f72641e40 20 realm 2023-10-05T19:40:49.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.719+0000 7f4f72641e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.719+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595bdd90 con 0x563759592540 2023-10-05T19:40:49.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.719+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4f6800a5a0 con 0x563759592540 2023-10-05T19:40:49.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.719+0000 7f4f72641e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:49.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.719+0000 7f4f72641e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:49.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.719+0000 7f4f72641e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.719+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595c1880 con 0x563759592540 2023-10-05T19:40:49.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.719+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4f6800a5a0 con 0x563759592540 2023-10-05T19:40:49.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.719+0000 7f4f72641e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:49.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.719+0000 7f4f72641e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.719+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595c1490 con 0x563759592540 2023-10-05T19:40:49.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.719+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f4f6800a5a0 con 0x563759592540 2023-10-05T19:40:49.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.719+0000 7f4f72641e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:49.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.719+0000 7f4f72641e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.723+0000 7f4f72641e40 1 --2- 172.21.15.12:0/832711277 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5637595c2c20 0x5637595c5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:49.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.723+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595c5540 con 0x5637595c2c20 2023-10-05T19:40:49.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.723+0000 7f4f70176700 1 --2- 172.21.15.12:0/832711277 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5637595c2c20 0x5637595c5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:49.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.723+0000 7f4f70176700 1 --2- 172.21.15.12:0/832711277 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5637595c2c20 0x5637595c5000 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:49.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.723+0000 7f4f70176700 1 -- 172.21.15.12:0/832711277 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f4f640129c0 con 0x5637595c2c20 2023-10-05T19:40:49.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.723+0000 7f4f72641e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:49.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.723+0000 7f4f72641e40 20 searching for the correct realm 2023-10-05T19:40:49.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.723+0000 7f4f72641e40 1 --2- 172.21.15.12:0/832711277 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5637595c73c0 0x5637595c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:49.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.723+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595c9e80 con 0x5637595c73c0 2023-10-05T19:40:49.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.723+0000 7f4f6f975700 1 --2- 172.21.15.12:0/832711277 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5637595c73c0 0x5637595c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:49.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.723+0000 7f4f6f975700 1 --2- 172.21.15.12:0/832711277 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5637595c73c0 0x5637595c97d0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:49.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.727+0000 7f4f6f975700 1 -- 172.21.15.12:0/832711277 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f4f60006070 con 0x5637595c73c0 2023-10-05T19:40:49.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.727+0000 7f4f6f975700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595c9e80 con 0x5637595c2c20 2023-10-05T19:40:49.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.727+0000 7f4f70176700 1 -- 172.21.15.12:0/832711277 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f640129c0 con 0x5637595c2c20 2023-10-05T19:40:49.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.727+0000 7f4f70176700 1 --2- 172.21.15.12:0/832711277 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f4f64007030 0x7f4f640115e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:49.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.727+0000 7f4f70176700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595c9e80 con 0x7f4f64007030 2023-10-05T19:40:49.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.727+0000 7f4f70977700 1 --2- 172.21.15.12:0/832711277 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f4f64007030 0x7f4f640115e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:49.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.727+0000 7f4f70977700 1 --2- 172.21.15.12:0/832711277 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f4f64007030 0x7f4f640115e0 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:49.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.727+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f6800a5a0 con 0x7f4f64007030 2023-10-05T19:40:49.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.727+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595c9e80 con 0x563759592540 2023-10-05T19:40:49.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.731+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f6800a5a0 con 0x563759592540 2023-10-05T19:40:49.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.731+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595c9e80 con 0x563759592540 2023-10-05T19:40:49.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.731+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f6800a5a0 con 0x563759592540 2023-10-05T19:40:49.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.731+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595c9e80 con 0x563759592540 2023-10-05T19:40:49.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.731+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(11 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f6800a5a0 con 0x563759592540 2023-10-05T19:40:49.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.731+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595c9e80 con 0x7f4f64007030 2023-10-05T19:40:49.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.731+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f6800a5a0 con 0x7f4f64007030 2023-10-05T19:40:49.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.731+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595c9e80 con 0x5637595c2c20 2023-10-05T19:40:49.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.731+0000 7f4f70176700 1 -- 172.21.15.12:0/832711277 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f640129c0 con 0x5637595c2c20 2023-10-05T19:40:49.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.731+0000 7f4f70176700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595c9e80 con 0x5637595c2c20 2023-10-05T19:40:49.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.735+0000 7f4f70176700 1 -- 172.21.15.12:0/832711277 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f640129c0 con 0x5637595c2c20 2023-10-05T19:40:49.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.735+0000 7f4f70176700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595c9e80 con 0x563759592540 2023-10-05T19:40:49.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.735+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f6800a5a0 con 0x563759592540 2023-10-05T19:40:49.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.735+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595c9e80 con 0x7f4f64007030 2023-10-05T19:40:49.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.735+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f6800a5a0 con 0x7f4f64007030 2023-10-05T19:40:49.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.735+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595c9e80 con 0x563759592540 2023-10-05T19:40:49.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.735+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f6800a5a0 con 0x563759592540 2023-10-05T19:40:49.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.735+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595c9e80 con 0x5637595c2c20 2023-10-05T19:40:49.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.735+0000 7f4f70176700 1 -- 172.21.15.12:0/832711277 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(18 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f640129c0 con 0x5637595c2c20 2023-10-05T19:40:49.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.735+0000 7f4f70176700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595c9e80 con 0x5637595c73c0 2023-10-05T19:40:49.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.739+0000 7f4f6f975700 1 -- 172.21.15.12:0/832711277 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f60006070 con 0x5637595c73c0 2023-10-05T19:40:49.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.739+0000 7f4f6f975700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595c9e80 con 0x5637595c73c0 2023-10-05T19:40:49.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.739+0000 7f4f6f975700 1 -- 172.21.15.12:0/832711277 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f60006070 con 0x5637595c73c0 2023-10-05T19:40:49.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.739+0000 7f4f6f975700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595c9e80 con 0x7f4f64007030 2023-10-05T19:40:49.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.739+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f6800a5a0 con 0x7f4f64007030 2023-10-05T19:40:49.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.739+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595c9e80 con 0x5637595c73c0 2023-10-05T19:40:49.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.739+0000 7f4f6f975700 1 -- 172.21.15.12:0/832711277 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f60006070 con 0x5637595c73c0 2023-10-05T19:40:49.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.739+0000 7f4f6f975700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595c9e80 con 0x5637595c73c0 2023-10-05T19:40:49.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.739+0000 7f4f6f975700 1 -- 172.21.15.12:0/832711277 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f60006070 con 0x5637595c73c0 2023-10-05T19:40:49.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.739+0000 7f4f6f975700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595c9e80 con 0x563759592540 2023-10-05T19:40:49.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.743+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f6800a5a0 con 0x563759592540 2023-10-05T19:40:49.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.743+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595c9e80 con 0x563759592540 2023-10-05T19:40:49.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.743+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f6800a5a0 con 0x563759592540 2023-10-05T19:40:49.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.743+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595c9e80 con 0x7f4f64007030 2023-10-05T19:40:49.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.743+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f6800a5a0 con 0x7f4f64007030 2023-10-05T19:40:49.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.743+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595c9e80 con 0x5637595c73c0 2023-10-05T19:40:49.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.743+0000 7f4f6f975700 1 -- 172.21.15.12:0/832711277 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f60006070 con 0x5637595c73c0 2023-10-05T19:40:49.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.743+0000 7f4f6f975700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595c9e80 con 0x5637595c2c20 2023-10-05T19:40:49.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.743+0000 7f4f70176700 1 -- 172.21.15.12:0/832711277 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f640129c0 con 0x5637595c2c20 2023-10-05T19:40:49.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.743+0000 7f4f70176700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595c9e80 con 0x563759592540 2023-10-05T19:40:49.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.747+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f6800a5a0 con 0x563759592540 2023-10-05T19:40:49.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.747+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595c9e80 con 0x7f4f64007030 2023-10-05T19:40:49.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.747+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f6800a5a0 con 0x7f4f64007030 2023-10-05T19:40:49.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.747+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595c9e80 con 0x5637595c73c0 2023-10-05T19:40:49.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.747+0000 7f4f6f975700 1 -- 172.21.15.12:0/832711277 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f60006070 con 0x5637595c73c0 2023-10-05T19:40:49.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.747+0000 7f4f6f975700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595c9e80 con 0x7f4f64007030 2023-10-05T19:40:49.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.747+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f6800a5a0 con 0x7f4f64007030 2023-10-05T19:40:49.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.747+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595c9e80 con 0x5637595c2c20 2023-10-05T19:40:49.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.747+0000 7f4f70176700 1 -- 172.21.15.12:0/832711277 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f640129c0 con 0x5637595c2c20 2023-10-05T19:40:49.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.747+0000 7f4f70176700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595c9e80 con 0x5637595c73c0 2023-10-05T19:40:49.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.751+0000 7f4f6f975700 1 -- 172.21.15.12:0/832711277 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f60006070 con 0x5637595c73c0 2023-10-05T19:40:49.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.751+0000 7f4f6f975700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595c9e80 con 0x7f4f64007030 2023-10-05T19:40:49.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.751+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f6800a5a0 con 0x7f4f64007030 2023-10-05T19:40:49.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.751+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595c9e80 con 0x563759592540 2023-10-05T19:40:49.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.751+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f4f6800a5a0 con 0x563759592540 2023-10-05T19:40:49.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.751+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595c9e80 con 0x5637595c2c20 2023-10-05T19:40:49.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.751+0000 7f4f70176700 1 -- 172.21.15.12:0/832711277 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(37 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f4f640129c0 con 0x5637595c2c20 2023-10-05T19:40:49.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.751+0000 7f4f72641e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:49.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.751+0000 7f4f72641e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:49.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.751+0000 7f4f72641e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:49.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.751+0000 7f4f72641e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:49.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.751+0000 7f4f72641e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.751+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595c6110 con 0x563759592540 2023-10-05T19:40:49.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.755+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4f6800a5a0 con 0x563759592540 2023-10-05T19:40:49.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.755+0000 7f4f72641e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:49.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.755+0000 7f4f72641e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.755+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595c5d40 con 0x563759592540 2023-10-05T19:40:49.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.755+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f4f6800a5a0 con 0x563759592540 2023-10-05T19:40:49.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.755+0000 7f4f72641e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:49.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.755+0000 7f4f72641e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.755+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595ca2f0 con 0x5637595c73c0 2023-10-05T19:40:49.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.755+0000 7f4f6f975700 1 -- 172.21.15.12:0/832711277 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f4f60006070 con 0x5637595c73c0 2023-10-05T19:40:49.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.755+0000 7f4f72641e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:49.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.755+0000 7f4f72641e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.755+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595ca6a0 con 0x5637595c73c0 2023-10-05T19:40:49.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.755+0000 7f4f6f975700 1 -- 172.21.15.12:0/832711277 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f4f60006070 con 0x5637595c73c0 2023-10-05T19:40:49.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.755+0000 7f4f72641e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:49.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.755+0000 7f4f72641e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.755+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595caa50 con 0x563759592540 2023-10-05T19:40:49.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.755+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4f6800a5a0 con 0x563759592540 2023-10-05T19:40:49.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.755+0000 7f4f72641e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:49.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.755+0000 7f4f72641e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.755+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595c65d0 con 0x563759592540 2023-10-05T19:40:49.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.759+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f4f6800a5a0 con 0x563759592540 2023-10-05T19:40:49.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.759+0000 7f4f72641e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:49.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.759+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595c65a0 con 0x5637595c73c0 2023-10-05T19:40:49.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.759+0000 7f4f6f975700 1 -- 172.21.15.12:0/832711277 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(44 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f4f60006070 con 0x5637595c73c0 2023-10-05T19:40:49.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.759+0000 7f4f6f975700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595c65a0 con 0x5637595c2c20 2023-10-05T19:40:49.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.759+0000 7f4f70176700 1 -- 172.21.15.12:0/832711277 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(45 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f640129c0 con 0x5637595c2c20 2023-10-05T19:40:49.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.759+0000 7f4f70176700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595c65a0 con 0x7f4f64007030 2023-10-05T19:40:49.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.763+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(46 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f6800a5a0 con 0x7f4f64007030 2023-10-05T19:40:49.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.763+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595caa50 con 0x563759592540 2023-10-05T19:40:49.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.763+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(47 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f6800a5a0 con 0x563759592540 2023-10-05T19:40:49.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.763+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595caa50 con 0x563759592540 2023-10-05T19:40:49.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.763+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(48 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f6800a5a0 con 0x563759592540 2023-10-05T19:40:49.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.763+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595caa50 con 0x563759592540 2023-10-05T19:40:49.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.763+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(49 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f6800a5a0 con 0x563759592540 2023-10-05T19:40:49.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.763+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595caa50 con 0x7f4f64007030 2023-10-05T19:40:49.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.767+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(50 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f6800a5a0 con 0x7f4f64007030 2023-10-05T19:40:49.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.767+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595caa50 con 0x5637595c2c20 2023-10-05T19:40:49.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.767+0000 7f4f70176700 1 -- 172.21.15.12:0/832711277 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(51 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f640129c0 con 0x5637595c2c20 2023-10-05T19:40:49.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.767+0000 7f4f70176700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595caa50 con 0x5637595c2c20 2023-10-05T19:40:49.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.767+0000 7f4f70176700 1 -- 172.21.15.12:0/832711277 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(52 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f640129c0 con 0x5637595c2c20 2023-10-05T19:40:49.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.767+0000 7f4f70176700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595caa50 con 0x563759592540 2023-10-05T19:40:49.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.767+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(53 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f6800a5a0 con 0x563759592540 2023-10-05T19:40:49.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.767+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595caa50 con 0x7f4f64007030 2023-10-05T19:40:49.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.767+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(54 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f6800a5a0 con 0x7f4f64007030 2023-10-05T19:40:49.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.767+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595caa50 con 0x563759592540 2023-10-05T19:40:49.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.771+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(55 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f6800a5a0 con 0x563759592540 2023-10-05T19:40:49.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.771+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595caa50 con 0x5637595c2c20 2023-10-05T19:40:49.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.771+0000 7f4f70176700 1 -- 172.21.15.12:0/832711277 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(56 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f640129c0 con 0x5637595c2c20 2023-10-05T19:40:49.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.771+0000 7f4f70176700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595caa50 con 0x5637595c73c0 2023-10-05T19:40:49.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.771+0000 7f4f6f975700 1 -- 172.21.15.12:0/832711277 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(57 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f60006070 con 0x5637595c73c0 2023-10-05T19:40:49.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.771+0000 7f4f6f975700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595caa50 con 0x5637595c73c0 2023-10-05T19:40:49.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.771+0000 7f4f6f975700 1 -- 172.21.15.12:0/832711277 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(58 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f60006070 con 0x5637595c73c0 2023-10-05T19:40:49.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.771+0000 7f4f6f975700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595caa50 con 0x7f4f64007030 2023-10-05T19:40:49.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.771+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(59 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f6800a5a0 con 0x7f4f64007030 2023-10-05T19:40:49.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.771+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595caa50 con 0x5637595c73c0 2023-10-05T19:40:49.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.771+0000 7f4f6f975700 1 -- 172.21.15.12:0/832711277 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(60 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f60006070 con 0x5637595c73c0 2023-10-05T19:40:49.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.771+0000 7f4f6f975700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595caa50 con 0x5637595c73c0 2023-10-05T19:40:49.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.775+0000 7f4f6f975700 1 -- 172.21.15.12:0/832711277 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(61 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f60006070 con 0x5637595c73c0 2023-10-05T19:40:49.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.775+0000 7f4f6f975700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595caa50 con 0x563759592540 2023-10-05T19:40:49.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.775+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(62 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f6800a5a0 con 0x563759592540 2023-10-05T19:40:49.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.775+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595caa50 con 0x563759592540 2023-10-05T19:40:49.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.775+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(63 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f6800a5a0 con 0x563759592540 2023-10-05T19:40:49.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.775+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595caa50 con 0x7f4f64007030 2023-10-05T19:40:49.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.775+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(64 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f6800a5a0 con 0x7f4f64007030 2023-10-05T19:40:49.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.775+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595caa50 con 0x5637595c73c0 2023-10-05T19:40:49.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.775+0000 7f4f6f975700 1 -- 172.21.15.12:0/832711277 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(65 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f60006070 con 0x5637595c73c0 2023-10-05T19:40:49.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.775+0000 7f4f6f975700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595caa50 con 0x5637595c2c20 2023-10-05T19:40:49.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.779+0000 7f4f70176700 1 -- 172.21.15.12:0/832711277 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(66 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f640129c0 con 0x5637595c2c20 2023-10-05T19:40:49.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.779+0000 7f4f70176700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595caa50 con 0x563759592540 2023-10-05T19:40:49.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.779+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(67 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f6800a5a0 con 0x563759592540 2023-10-05T19:40:49.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.779+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595caa50 con 0x7f4f64007030 2023-10-05T19:40:49.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.779+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(68 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f6800a5a0 con 0x7f4f64007030 2023-10-05T19:40:49.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.779+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595caa50 con 0x5637595c73c0 2023-10-05T19:40:49.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.779+0000 7f4f6f975700 1 -- 172.21.15.12:0/832711277 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(69 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f60006070 con 0x5637595c73c0 2023-10-05T19:40:49.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.779+0000 7f4f6f975700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595caa50 con 0x7f4f64007030 2023-10-05T19:40:49.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.783+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(70 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f6800a5a0 con 0x7f4f64007030 2023-10-05T19:40:49.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.783+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595caa50 con 0x5637595c2c20 2023-10-05T19:40:49.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.783+0000 7f4f70176700 1 -- 172.21.15.12:0/832711277 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(71 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f640129c0 con 0x5637595c2c20 2023-10-05T19:40:49.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.783+0000 7f4f70176700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595caa50 con 0x5637595c73c0 2023-10-05T19:40:49.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.783+0000 7f4f6f975700 1 -- 172.21.15.12:0/832711277 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(72 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f60006070 con 0x5637595c73c0 2023-10-05T19:40:49.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.783+0000 7f4f6f975700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595caa50 con 0x7f4f64007030 2023-10-05T19:40:49.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.783+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(73 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4f6800a5a0 con 0x7f4f64007030 2023-10-05T19:40:49.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.783+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595caa50 con 0x563759592540 2023-10-05T19:40:49.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.783+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(74 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f4f6800a5a0 con 0x563759592540 2023-10-05T19:40:49.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.783+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595caa50 con 0x5637595c2c20 2023-10-05T19:40:49.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.787+0000 7f4f70176700 1 -- 172.21.15.12:0/832711277 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(75 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f4f640129c0 con 0x5637595c2c20 2023-10-05T19:40:49.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.787+0000 7f4f72641e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:49.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.787+0000 7f4f72641e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:49.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.787+0000 7f4f72641e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:49.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.787+0000 7f4f72641e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:49.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.787+0000 7f4f72641e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.787+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595c65a0 con 0x563759592540 2023-10-05T19:40:49.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.787+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f4f6800a5a0 con 0x563759592540 2023-10-05T19:40:49.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.787+0000 7f4f72641e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:49.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.787+0000 7f4f72641e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.787+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595cb2f0 con 0x5637595c2c20 2023-10-05T19:40:49.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.787+0000 7f4f70176700 1 -- 172.21.15.12:0/832711277 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f4f640129c0 con 0x5637595c2c20 2023-10-05T19:40:49.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.787+0000 7f4f72641e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:49.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.787+0000 7f4f72641e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.787+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595cbbe0 con 0x563759592540 2023-10-05T19:40:49.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.787+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f4f6800a5a0 con 0x563759592540 2023-10-05T19:40:49.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.787+0000 7f4f72641e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:49.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.787+0000 7f4f72641e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.787+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595cbbe0 con 0x5637595c73c0 2023-10-05T19:40:49.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.791+0000 7f4f6f975700 1 -- 172.21.15.12:0/832711277 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f4f60006070 con 0x5637595c73c0 2023-10-05T19:40:49.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.791+0000 7f4f72641e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:49.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.791+0000 7f4f72641e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.791+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595cc350 con 0x7f4f64007030 2023-10-05T19:40:49.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.791+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f4f6800a5a0 con 0x7f4f64007030 2023-10-05T19:40:49.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.791+0000 7f4f72641e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:49.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.791+0000 7f4f72641e40 20 zone default found 2023-10-05T19:40:49.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.791+0000 7f4f72641e40 4 Realm: () 2023-10-05T19:40:49.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.791+0000 7f4f72641e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:49.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.791+0000 7f4f72641e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:49.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.791+0000 7f4f72641e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:49.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.791+0000 7f4f72641e40 20 zonegroup default 2023-10-05T19:40:49.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.791+0000 7f4f72641e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.791+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595cc350 con 0x7f4f64007030 2023-10-05T19:40:49.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.791+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f4f6800a5a0 con 0x7f4f64007030 2023-10-05T19:40:49.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.791+0000 7f4f72641e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:49.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.791+0000 7f4f72641e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.791+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595cdcd0 con 0x5637595c73c0 2023-10-05T19:40:49.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.791+0000 7f4f6f975700 1 -- 172.21.15.12:0/832711277 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f4f60006070 con 0x5637595c73c0 2023-10-05T19:40:49.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.791+0000 7f4f72641e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:49.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.791+0000 7f4f72641e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.791+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595ce080 con 0x5637595c73c0 2023-10-05T19:40:49.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.791+0000 7f4f6f975700 1 -- 172.21.15.12:0/832711277 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f4f60006070 con 0x5637595c73c0 2023-10-05T19:40:49.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.791+0000 7f4f72641e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:49.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.791+0000 7f4f72641e40 20 started sync module instance, tier type = 2023-10-05T19:40:49.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.791+0000 7f4f72641e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:49.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.791+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595cd390 con 0x5637595c2c20 2023-10-05T19:40:49.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.795+0000 7f4f70176700 1 -- 172.21.15.12:0/832711277 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(84 notify.0 [create] v30'201 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f640129c0 con 0x5637595c2c20 2023-10-05T19:40:49.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.795+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94795722438384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595ce430 con 0x5637595c2c20 2023-10-05T19:40:49.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.795+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595cecd0 con 0x563759592540 2023-10-05T19:40:49.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.799+0000 7f4f70176700 1 -- 172.21.15.12:0/832711277 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(85 notify.0 [watch watch cookie 94795722438384] v30'202 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f640129c0 con 0x5637595c2c20 2023-10-05T19:40:49.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.799+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(86 notify.1 [create] v30'201 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f6800a5a0 con 0x563759592540 2023-10-05T19:40:49.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.799+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94795722450912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595cf120 con 0x563759592540 2023-10-05T19:40:49.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.799+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595cf9c0 con 0x5637595c73c0 2023-10-05T19:40:49.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.803+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(87 notify.1 [watch watch cookie 94795722450912] v30'202 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f6800a5a0 con 0x563759592540 2023-10-05T19:40:49.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.803+0000 7f4f6f975700 1 -- 172.21.15.12:0/832711277 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(88 notify.2 [create] v30'201 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f60006070 con 0x5637595c73c0 2023-10-05T19:40:49.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.803+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94795722452304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595cfd70 con 0x5637595c73c0 2023-10-05T19:40:49.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.803+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595d0610 con 0x5637595c2c20 2023-10-05T19:40:49.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.803+0000 7f4f6f975700 1 -- 172.21.15.12:0/832711277 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(89 notify.2 [watch watch cookie 94795722452304] v30'202 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f60006070 con 0x5637595c73c0 2023-10-05T19:40:49.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.803+0000 7f4f70176700 1 -- 172.21.15.12:0/832711277 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(90 notify.3 [create] v30'201 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f640129c0 con 0x5637595c2c20 2023-10-05T19:40:49.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.803+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94795722457376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595cf300 con 0x5637595c2c20 2023-10-05T19:40:49.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.807+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595d11e0 con 0x563759592540 2023-10-05T19:40:49.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.807+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(92 notify.4 [create] v30'201 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f6800a5a0 con 0x563759592540 2023-10-05T19:40:49.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.807+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94795722460400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595d1140 con 0x563759592540 2023-10-05T19:40:49.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.807+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595d2170 con 0x7f4f64007030 2023-10-05T19:40:49.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.807+0000 7f4f70176700 1 -- 172.21.15.12:0/832711277 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(91 notify.3 [watch watch cookie 94795722457376] v30'202 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f640129c0 con 0x5637595c2c20 2023-10-05T19:40:49.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.811+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(93 notify.4 [watch watch cookie 94795722460400] v30'202 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f6800a5a0 con 0x563759592540 2023-10-05T19:40:49.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.811+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(94 notify.5 [create] v30'201 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f6800a5a0 con 0x7f4f64007030 2023-10-05T19:40:49.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.811+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94795722458480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595d1140 con 0x7f4f64007030 2023-10-05T19:40:49.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.811+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595d16e0 con 0x7f4f64007030 2023-10-05T19:40:49.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.815+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(95 notify.5 [watch watch cookie 94795722458480] v30'202 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f6800a5a0 con 0x7f4f64007030 2023-10-05T19:40:49.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.815+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(96 notify.6 [create] v30'201 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f6800a5a0 con 0x7f4f64007030 2023-10-05T19:40:49.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.815+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94795722464320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595d1140 con 0x7f4f64007030 2023-10-05T19:40:49.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.815+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595d2f50 con 0x7f4f64007030 2023-10-05T19:40:49.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.815+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(97 notify.6 [watch watch cookie 94795722464320] v30'202 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f6800a5a0 con 0x7f4f64007030 2023-10-05T19:40:49.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.815+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(98 notify.7 [create] v30'201 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f6800a5a0 con 0x7f4f64007030 2023-10-05T19:40:49.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.815+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94795722466672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595d2eb0 con 0x7f4f64007030 2023-10-05T19:40:49.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.815+0000 7f4f72641e40 20 add_watcher() i=0 2023-10-05T19:40:49.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.815+0000 7f4f72641e40 20 add_watcher() i=1 2023-10-05T19:40:49.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.815+0000 7f4f72641e40 20 add_watcher() i=2 2023-10-05T19:40:49.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.815+0000 7f4f72641e40 20 add_watcher() i=3 2023-10-05T19:40:49.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.815+0000 7f4f72641e40 20 add_watcher() i=4 2023-10-05T19:40:49.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.815+0000 7f4f72641e40 20 add_watcher() i=5 2023-10-05T19:40:49.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.815+0000 7f4f72641e40 20 add_watcher() i=6 2023-10-05T19:40:49.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.819+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(99 notify.7 [watch watch cookie 94795722466672] v30'202 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f6800a5a0 con 0x7f4f64007030 2023-10-05T19:40:49.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.819+0000 7f4f72641e40 20 add_watcher() i=7 2023-10-05T19:40:49.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.819+0000 7f4f72641e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:49.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.819+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595d6160 con 0x563759592540 2023-10-05T19:40:49.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.819+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f4f6800a5a0 con 0x563759592540 2023-10-05T19:40:49.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.819+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94795722468144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595d3470 con 0x563759592540 2023-10-05T19:40:49.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.823+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94795722468144] v30'199 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f4f6800a5a0 con 0x563759592540 2023-10-05T19:40:49.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.823+0000 7f4f41fe3700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:49.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.823+0000 7f4f72641e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:49.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.823+0000 7f4f72641e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:49.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.823+0000 7f4f72641e40 1 Processor -- start 2023-10-05T19:40:49.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.823+0000 7f4f72641e40 1 -- start start 2023-10-05T19:40:49.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.823+0000 7f4f72641e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5637595e0980 0x5637595e0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:49.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.823+0000 7f4f72641e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637595e1290 con 0x5637595e0980 2023-10-05T19:40:49.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.823+0000 7f4f6f975700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5637595e0980 0x5637595e0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:49.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.823+0000 7f4f6f975700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5637595e0980 0x5637595e0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58516/0 (socket says 172.21.15.12:58516) 2023-10-05T19:40:49.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.823+0000 7f4f6f975700 1 -- 172.21.15.12:0/164244640 learned_addr learned my addr 172.21.15.12:0/164244640 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:49.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.823+0000 7f4f6f975700 1 -- 172.21.15.12:0/164244640 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637595e13d0 con 0x5637595e0980 2023-10-05T19:40:49.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.827+0000 7f4f6f975700 1 --2- 172.21.15.12:0/164244640 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5637595e0980 0x5637595e0d50 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4f60001ed0 tx=0x7f4f6001e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=821ae999e15363de server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:49.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.827+0000 7f4f417e2700 1 -- 172.21.15.12:0/164244640 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f60030040 con 0x5637595e0980 2023-10-05T19:40:49.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.827+0000 7f4f417e2700 1 -- 172.21.15.12:0/164244640 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f6000edf0 con 0x5637595e0980 2023-10-05T19:40:49.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.827+0000 7f4f417e2700 1 -- 172.21.15.12:0/164244640 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f60021040 con 0x5637595e0980 2023-10-05T19:40:49.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.827+0000 7f4f72641e40 1 -- 172.21.15.12:0/164244640 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5637595e0980 msgr2=0x5637595e0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:49.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.827+0000 7f4f72641e40 1 --2- 172.21.15.12:0/164244640 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5637595e0980 0x5637595e0d50 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4f60001ed0 tx=0x7f4f6001e040 comp rx=0 tx=0).stop 2023-10-05T19:40:49.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.827+0000 7f4f72641e40 1 -- 172.21.15.12:0/164244640 shutdown_connections 2023-10-05T19:40:49.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.827+0000 7f4f72641e40 1 --2- 172.21.15.12:0/164244640 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5637595e0980 0x5637595e0d50 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:49.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.827+0000 7f4f72641e40 1 -- 172.21.15.12:0/164244640 >> 172.21.15.12:0/164244640 conn(0x5637595dbd40 msgr2=0x5637595d9e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:49.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.827+0000 7f4f72641e40 1 -- 172.21.15.12:0/164244640 shutdown_connections 2023-10-05T19:40:49.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.827+0000 7f4f72641e40 1 -- 172.21.15.12:0/164244640 wait complete. 2023-10-05T19:40:49.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.827+0000 7f4f72641e40 1 Processor -- start 2023-10-05T19:40:49.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.827+0000 7f4f72641e40 1 -- start start 2023-10-05T19:40:49.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.827+0000 7f4f72641e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5637595e0980 0x5637595e6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:49.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.827+0000 7f4f72641e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637595e7520 con 0x5637595e0980 2023-10-05T19:40:49.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.827+0000 7f4f6f975700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5637595e0980 0x5637595e6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:49.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.827+0000 7f4f6f975700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5637595e0980 0x5637595e6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58520/0 (socket says 172.21.15.12:58520) 2023-10-05T19:40:49.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.827+0000 7f4f6f975700 1 -- 172.21.15.12:0/3467749295 learned_addr learned my addr 172.21.15.12:0/3467749295 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:49.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.827+0000 7f4f6f975700 1 -- 172.21.15.12:0/3467749295 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f60033040 con 0x5637595e0980 2023-10-05T19:40:49.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.831+0000 7f4f6f975700 1 --2- 172.21.15.12:0/3467749295 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5637595e0980 0x5637595e6520 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f4f6003d040 tx=0x7f4f6003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:49.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.831+0000 7f4ebb7fe700 1 -- 172.21.15.12:0/3467749295 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f60021070 con 0x5637595e0980 2023-10-05T19:40:49.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.831+0000 7f4ebb7fe700 1 -- 172.21.15.12:0/3467749295 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f6002b780 con 0x5637595e0980 2023-10-05T19:40:49.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.831+0000 7f4f72641e40 1 -- 172.21.15.12:0/3467749295 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5637595e6b20 con 0x5637595e0980 2023-10-05T19:40:49.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.831+0000 7f4ebb7fe700 1 -- 172.21.15.12:0/3467749295 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f60030040 con 0x5637595e0980 2023-10-05T19:40:49.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.831+0000 7f4f72641e40 1 -- 172.21.15.12:0/3467749295 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5637595e6f60 con 0x5637595e0980 2023-10-05T19:40:49.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.831+0000 7f4ebb7fe700 1 -- 172.21.15.12:0/3467749295 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f4f6002b910 con 0x5637595e0980 2023-10-05T19:40:49.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.831+0000 7f4ebb7fe700 1 --2- 172.21.15.12:0/3467749295 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f4eb0030c00 0x7f4eb00330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:49.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.831+0000 7f4ebb7fe700 1 -- 172.21.15.12:0/3467749295 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7f4f6004fbe0 con 0x5637595e0980 2023-10-05T19:40:49.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.831+0000 7f4f70977700 1 --2- 172.21.15.12:0/3467749295 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f4eb0030c00 0x7f4eb00330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:49.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.831+0000 7f4f72641e40 5 note: GC not initialized 2023-10-05T19:40:49.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.831+0000 7f4f70977700 1 --2- 172.21.15.12:0/3467749295 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f4eb0030c00 0x7f4eb00330b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x5637595d61d0 tx=0x7f4f6800aed0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:49.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.835+0000 7f4eb8ff9700 20 reqs_thread_entry: start 2023-10-05T19:40:49.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.835+0000 7f4eae7fc700 20 reqs_thread_entry: start 2023-10-05T19:40:49.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.835+0000 7f4f72641e40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:49.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.835+0000 7f4f72641e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:49.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.835+0000 7f4eacff9700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:41:20 2023 2023-10-05T19:40:49.843 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:49.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.835+0000 7f4f72641e40 20 get_system_obj_state: rctx=0x7ffd66c83790 obj=default.rgw.meta:root:myfoo state=0x5637594660b0 s->prefetch_data=0 2023-10-05T19:40:49.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.835+0000 7f4f72641e40 10 cache get: name=default.rgw.meta+root+myfoo : miss 2023-10-05T19:40:49.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.835+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:102 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56375964f6d0 con 0x5637595c2c20 2023-10-05T19:40:49.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.835+0000 7f4e93fff700 20 reqs_thread_entry: start 2023-10-05T19:40:49.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.835+0000 7f4f70176700 1 -- 172.21.15.12:0/832711277 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(102 myfoo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv33 ondisk = 0) v8 ==== 233+0+139 (crc 0 0 0) 0x7f4f640129c0 con 0x5637595c2c20 2023-10-05T19:40:49.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.835+0000 7f4f72641e40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x16 2023-10-05T19:40:49.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.835+0000 7f4f72641e40 10 adding default.rgw.meta+root+myfoo to cache LRU end 2023-10-05T19:40:49.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.835+0000 7f4f72641e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:49.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.835+0000 7f4f72641e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:49.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.835+0000 7f4f72641e40 10 cache get: name=default.rgw.meta+root+myfoo : type miss (requested=0x11, cached=0x16) 2023-10-05T19:40:49.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.835+0000 7f4f72641e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.835+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:103 8.9 8:91b71841:root::myfoo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56375964f5c0 con 0x5637595c2c20 2023-10-05T19:40:49.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.835+0000 7f4f70176700 1 -- 172.21.15.12:0/832711277 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(103 myfoo [call,call out=48b,read 0~165 out=165b] v0'0 uv33 ondisk = 0) v8 ==== 233+0+213 (crc 0 0 0) 0x7f4f640129c0 con 0x5637595c2c20 2023-10-05T19:40:49.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.835+0000 7f4f72641e40 20 rados_obj.operate() r=0 bl.length=165 2023-10-05T19:40:49.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.835+0000 7f4f72641e40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2023-10-05T19:40:49.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.835+0000 7f4f72641e40 10 moving default.rgw.meta+root+myfoo to cache LRU end 2023-10-05T19:40:49.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.835+0000 7f4f72641e40 20 get_system_obj_state: rctx=0x7ffd66c83790 obj=default.rgw.meta:root:.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5 state=0x56375943c2b0 s->prefetch_data=0 2023-10-05T19:40:49.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.835+0000 7f4f72641e40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5 : miss 2023-10-05T19:40:49.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.835+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:104 8.10 8:0a6cdd29:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563759651150 con 0x5637595c73c0 2023-10-05T19:40:49.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.839+0000 7f4f6f975700 1 -- 172.21.15.12:0/832711277 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(104 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5 [call out=48b,getxattrs out=218b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 290+0+282 (crc 0 0 0) 0x7f4f60054070 con 0x5637595c73c0 2023-10-05T19:40:49.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.839+0000 7f4f72641e40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5 info.flags=0x16 2023-10-05T19:40:49.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.839+0000 7f4f72641e40 10 adding default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5 to cache LRU end 2023-10-05T19:40:49.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.839+0000 7f4f72641e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:49.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.839+0000 7f4f72641e40 10 updating xattr: name=user.rgw.acl bl.length()=123 2023-10-05T19:40:49.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.839+0000 7f4f72641e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:49.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.839+0000 7f4f72641e40 20 Read xattr: user.rgw.acl 2023-10-05T19:40:49.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.839+0000 7f4f72641e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:49.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.839+0000 7f4f72641e40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5 : type miss (requested=0x13, cached=0x16) 2023-10-05T19:40:49.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.839+0000 7f4f72641e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.839+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:105 8.10 8:0a6cdd29:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563759651cd0 con 0x5637595c73c0 2023-10-05T19:40:49.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.839+0000 7f4f6f975700 1 -- 172.21.15.12:0/832711277 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(105 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5 [call,call out=48b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 332+0+618 (crc 0 0 0) 0x7f4f60054070 con 0x5637595c73c0 2023-10-05T19:40:49.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.839+0000 7f4f72641e40 20 rados_obj.operate() r=0 bl.length=352 2023-10-05T19:40:49.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.839+0000 7f4f72641e40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5 info.flags=0x13 2023-10-05T19:40:49.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.839+0000 7f4f72641e40 10 moving default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5 to cache LRU end 2023-10-05T19:40:49.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.839+0000 7f4f72641e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:49.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.839+0000 7f4f72641e40 10 updating xattr: name=user.rgw.acl bl.length()=123 2023-10-05T19:40:49.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.839+0000 7f4f72641e40 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5 2023-10-05T19:40:49.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.839+0000 7f4f72641e40 20 get_obj_state: rctx=0x7ffd66c839d0 obj=myfoo:bb3aec0fdcdbc2974890f805c585d432 state=0x5637596547a8 s->prefetch_data=0 2023-10-05T19:40:49.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.839+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:106 3.26 3:6571379c:::93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5_bb3aec0fdcdbc2974890f805c585d432:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563759655110 con 0x5637595c2c20 2023-10-05T19:40:49.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.839+0000 7f4f70176700 1 -- 172.21.15.12:0/832711277 <== osd.2 v2:172.21.15.12:6800/29934 23 ==== osd_op_reply(106 93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5_bb3aec0fdcdbc2974890f805c585d432 [getxattrs out=849b,stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 262+0+865 (crc 0 0 0) 0x7f4f640129c0 con 0x5637595c2c20 2023-10-05T19:40:49.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.839+0000 7f4f72641e40 10 manifest: total_size = 5 2023-10-05T19:40:49.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.839+0000 7f4f72641e40 20 get_obj_state: setting s->obj_tag to _FnrJ8kFk8pG9NXwSgO0mbv_h1v_wY2l 2023-10-05T19:40:49.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.839+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:107 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94795722438384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595a4820 con 0x5637595c2c20 2023-10-05T19:40:49.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.843+0000 7f4f70176700 1 -- 172.21.15.12:0/832711277 <== osd.2 v2:172.21.15.12:6800/29934 24 ==== osd_op_reply(107 notify.0 [watch unwatch cookie 94795722438384] v30'203 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f640129c0 con 0x5637595c2c20 2023-10-05T19:40:49.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.843+0000 7f4f72641e40 20 remove_watcher() i=0 2023-10-05T19:40:49.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.843+0000 7f4f72641e40 2 removed watcher, disabling cache 2023-10-05T19:40:49.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.843+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:108 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94795722450912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595a4820 con 0x563759592540 2023-10-05T19:40:49.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.847+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(108 notify.1 [watch unwatch cookie 94795722450912] v30'203 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f6800a5a0 con 0x563759592540 2023-10-05T19:40:49.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.847+0000 7f4f72641e40 20 remove_watcher() i=1 2023-10-05T19:40:49.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.847+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:109 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94795722452304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595a4820 con 0x5637595c73c0 2023-10-05T19:40:49.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.847+0000 7f4f6f975700 1 -- 172.21.15.12:0/832711277 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(109 notify.2 [watch unwatch cookie 94795722452304] v30'203 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f60054070 con 0x5637595c73c0 2023-10-05T19:40:49.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.847+0000 7f4f72641e40 20 remove_watcher() i=2 2023-10-05T19:40:49.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.847+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:110 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94795722457376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595a4820 con 0x5637595c2c20 2023-10-05T19:40:49.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.851+0000 7f4f70176700 1 -- 172.21.15.12:0/832711277 <== osd.2 v2:172.21.15.12:6800/29934 25 ==== osd_op_reply(110 notify.3 [watch unwatch cookie 94795722457376] v30'203 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f640129c0 con 0x5637595c2c20 2023-10-05T19:40:49.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.851+0000 7f4f72641e40 20 remove_watcher() i=3 2023-10-05T19:40:49.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.851+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:111 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94795722460400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595a4820 con 0x563759592540 2023-10-05T19:40:49.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.855+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(111 notify.4 [watch unwatch cookie 94795722460400] v30'203 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f6800a5a0 con 0x563759592540 2023-10-05T19:40:49.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.855+0000 7f4f72641e40 20 remove_watcher() i=4 2023-10-05T19:40:49.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.855+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:112 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94795722458480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595a4820 con 0x7f4f64007030 2023-10-05T19:40:49.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.855+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(112 notify.5 [watch unwatch cookie 94795722458480] v30'203 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f6800a5a0 con 0x7f4f64007030 2023-10-05T19:40:49.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.855+0000 7f4f72641e40 20 remove_watcher() i=5 2023-10-05T19:40:49.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.855+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:113 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94795722464320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595a4820 con 0x7f4f64007030 2023-10-05T19:40:49.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.859+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(113 notify.6 [watch unwatch cookie 94795722464320] v30'203 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f6800a5a0 con 0x7f4f64007030 2023-10-05T19:40:49.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.859+0000 7f4f72641e40 20 remove_watcher() i=6 2023-10-05T19:40:49.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.859+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:114 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94795722466672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5637595a4820 con 0x7f4f64007030 2023-10-05T19:40:49.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.863+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(114 notify.7 [watch unwatch cookie 94795722466672] v30'203 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4f6800a5a0 con 0x7f4f64007030 2023-10-05T19:40:49.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.863+0000 7f4f72641e40 20 remove_watcher() i=7 2023-10-05T19:40:49.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.863+0000 7f4f41fe3700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:49.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.863+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:115 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94795722468144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56375960f9c0 con 0x563759592540 2023-10-05T19:40:49.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.867+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(115 data_loggenerations_metadata [watch unwatch cookie 94795722468144] v30'200 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f4f6800a5a0 con 0x563759592540 2023-10-05T19:40:49.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.867+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f4f64007030 msgr2=0x7f4f640115e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:49.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.867+0000 7f4f72641e40 1 --2- 172.21.15.12:0/832711277 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f4f64007030 0x7f4f640115e0 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:49.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.867+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x563759592540 msgr2=0x5637595c0730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:49.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.867+0000 7f4f72641e40 1 --2- 172.21.15.12:0/832711277 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x563759592540 0x5637595c0730 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:49.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.867+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5637595c2c20 msgr2=0x5637595c5000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:49.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.867+0000 7f4f72641e40 1 --2- 172.21.15.12:0/832711277 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5637595c2c20 0x5637595c5000 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:49.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.867+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5637595c73c0 msgr2=0x5637595c97d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:49.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.867+0000 7f4f72641e40 1 --2- 172.21.15.12:0/832711277 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5637595c73c0 0x5637595c97d0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:49.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.867+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f4f5c030c90 msgr2=0x7f4f5c033140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:49.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.867+0000 7f4f72641e40 1 --2- 172.21.15.12:0/832711277 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f4f5c030c90 0x7f4f5c033140 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4f60004170 tx=0x7f4f60001360 comp rx=0 tx=0).stop 2023-10-05T19:40:49.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.867+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563759500b50 msgr2=0x5637595b4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:49.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.867+0000 7f4f70977700 1 -- 172.21.15.12:0/832711277 reap_dead start 2023-10-05T19:40:49.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.867+0000 7f4f72641e40 1 --2- 172.21.15.12:0/832711277 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563759500b50 0x5637595b4c70 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4f6401f040 tx=0x7f4f6401c000 comp rx=0 tx=0).stop 2023-10-05T19:40:49.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.867+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 shutdown_connections 2023-10-05T19:40:49.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.867+0000 7f4f72641e40 1 --2- 172.21.15.12:0/832711277 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563759500b50 0x5637595b4c70 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:49.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.867+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 >> 172.21.15.12:0/832711277 conn(0x563759587750 msgr2=0x5637595a8580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:49.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.867+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 shutdown_connections 2023-10-05T19:40:49.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.867+0000 7f4f72641e40 1 -- 172.21.15.12:0/832711277 wait complete. 2023-10-05T19:40:49.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.867+0000 7f4f72641e40 1 -- 172.21.15.12:0/3467749295 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f4eb0030c00 msgr2=0x7f4eb00330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:49.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.867+0000 7f4f72641e40 1 --2- 172.21.15.12:0/3467749295 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f4eb0030c00 0x7f4eb00330b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x5637595d61d0 tx=0x7f4f6800aed0 comp rx=0 tx=0).stop 2023-10-05T19:40:49.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.867+0000 7f4f72641e40 1 -- 172.21.15.12:0/3467749295 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5637595e0980 msgr2=0x5637595e6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:49.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.867+0000 7f4f72641e40 1 --2- 172.21.15.12:0/3467749295 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5637595e0980 0x5637595e6520 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f4f6003d040 tx=0x7f4f6003f000 comp rx=0 tx=0).stop 2023-10-05T19:40:49.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.867+0000 7f4f72641e40 1 -- 172.21.15.12:0/3467749295 shutdown_connections 2023-10-05T19:40:49.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.867+0000 7f4f72641e40 1 --2- 172.21.15.12:0/3467749295 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f4eb0030c00 0x7f4eb00330b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:49.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.867+0000 7f4f72641e40 1 --2- 172.21.15.12:0/3467749295 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5637595e0980 0x5637595e6520 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:49.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.867+0000 7f4f72641e40 1 -- 172.21.15.12:0/3467749295 >> 172.21.15.12:0/3467749295 conn(0x5637595dbd40 msgr2=0x5637595d9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:49.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.867+0000 7f4f72641e40 1 -- 172.21.15.12:0/3467749295 shutdown_connections 2023-10-05T19:40:49.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.867+0000 7f4f72641e40 1 -- 172.21.15.12:0/3467749295 wait complete. 2023-10-05T19:40:49.882 INFO:teuthology.orchestra.run.smithi012.stdout:fooFoofooFooFULL_CONTROL 2023-10-05T19:40:49.882 DEBUG:tasks.util.rgw: raw result: fooFoofooFooFULL_CONTROL 2023-10-05T19:40:49.886 INFO:tasks.radosgw_admin:RL: get_acls myfoo foo bytes_out=0 bytes_in=417 failed=False 2023-10-05T19:40:49.895 INFO:tasks.radosgw_admin:RL: put_acls myfoo foo bytes_out=0 bytes_in=0 failed=False 2023-10-05T19:40:49.895 INFO:tasks.util.rgw:rgwadmin: client.0 : ['policy', '--bucket', 'myfoo', '--object', 'bb3aec0fdcdbc2974890f805c585d432'] 2023-10-05T19:40:49.895 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'xml', '-n', 'client.0', '--cluster', 'ceph', 'policy', '--bucket', 'myfoo', '--object', 'bb3aec0fdcdbc2974890f805c585d432'] 2023-10-05T19:40:49.895 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format xml -n client.0 --cluster ceph policy --bucket myfoo --object bb3aec0fdcdbc2974890f805c585d432 2023-10-05T19:40:49.932 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:49.932 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:49.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.923+0000 7f80c7e14e40 1 Processor -- start 2023-10-05T19:40:49.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.923+0000 7f80c7e14e40 1 -- start start 2023-10-05T19:40:49.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.927+0000 7f80c7e14e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55de91b8ed10 0x55de91b8f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:49.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.927+0000 7f80c7e14e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55de91adcbd0 con 0x55de91b8f620 2023-10-05T19:40:49.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.927+0000 7f80c7e14e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x55de91a99ee0 con 0x55de91b8ed10 2023-10-05T19:40:49.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.927+0000 7f80c5949700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55de91b8ed10 0x55de91b8f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:49.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.927+0000 7f80c5949700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55de91b8ed10 0x55de91b8f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58534/0 (socket says 172.21.15.12:58534) 2023-10-05T19:40:49.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.927+0000 7f80c5949700 1 -- 172.21.15.12:0/3165385928 learned_addr learned my addr 172.21.15.12:0/3165385928 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:49.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.927+0000 7f80c5148700 1 --1- 172.21.15.12:0/3165385928 >> v1:172.21.15.8:6789/0 conn(0x55de91b8f620 0x55de91b900c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:60246/0 (socket says 172.21.15.12:60246) 2023-10-05T19:40:49.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.927+0000 7f80c5949700 1 -- 172.21.15.12:0/3165385928 >> v1:172.21.15.8:6789/0 conn(0x55de91b8f620 legacy=0x55de91b900c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:49.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.927+0000 7f80c5949700 1 -- 172.21.15.12:0/3165385928 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de91b907d0 con 0x55de91b8ed10 2023-10-05T19:40:49.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.927+0000 7f80c5949700 1 --2- 172.21.15.12:0/3165385928 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de91b8ed10 0x55de91b8f0e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f80b800a760 tx=0x7f80b8005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16c36f6b96829093 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:49.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.927+0000 7f80c4947700 1 -- 172.21.15.12:0/3165385928 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f80b800f070 con 0x55de91b8ed10 2023-10-05T19:40:49.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.927+0000 7f80c4947700 1 -- 172.21.15.12:0/3165385928 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80b800a2e0 con 0x55de91b8ed10 2023-10-05T19:40:49.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.927+0000 7f80c4947700 1 -- 172.21.15.12:0/3165385928 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f80b80085b0 con 0x55de91b8ed10 2023-10-05T19:40:49.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.927+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3165385928 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de91b8ed10 msgr2=0x55de91b8f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:49.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.927+0000 7f80c7e14e40 1 --2- 172.21.15.12:0/3165385928 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de91b8ed10 0x55de91b8f0e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f80b800a760 tx=0x7f80b8005b10 comp rx=0 tx=0).stop 2023-10-05T19:40:49.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.927+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3165385928 shutdown_connections 2023-10-05T19:40:49.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.927+0000 7f80c7e14e40 1 --2- 172.21.15.12:0/3165385928 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de91b8ed10 0x55de91b8f0e0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:49.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.927+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3165385928 >> 172.21.15.12:0/3165385928 conn(0x55de91b8a730 msgr2=0x55de91b8cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:49.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.927+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3165385928 shutdown_connections 2023-10-05T19:40:49.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.927+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3165385928 wait complete. 2023-10-05T19:40:49.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.931+0000 7f80c7e14e40 1 Processor -- start 2023-10-05T19:40:49.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.931+0000 7f80c7e14e40 1 -- start start 2023-10-05T19:40:49.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.931+0000 7f80c7e14e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de91b8ed10 0x55de91bb1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:49.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.931+0000 7f80c7e14e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de91a5de30 con 0x55de91b8ed10 2023-10-05T19:40:49.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.931+0000 7f80c5949700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de91b8ed10 0x55de91bb1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:49.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.931+0000 7f80c5949700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de91b8ed10 0x55de91bb1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58544/0 (socket says 172.21.15.12:58544) 2023-10-05T19:40:49.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.931+0000 7f80c5949700 1 -- 172.21.15.12:0/3747733194 learned_addr learned my addr 172.21.15.12:0/3747733194 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:49.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.931+0000 7f80c5949700 1 -- 172.21.15.12:0/3747733194 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b8005db0 con 0x55de91b8ed10 2023-10-05T19:40:49.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.931+0000 7f80c5949700 1 --2- 172.21.15.12:0/3747733194 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de91b8ed10 0x55de91bb1150 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f80b800a6b0 tx=0x7f80b8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6714fcce09702f87 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:49.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.931+0000 7f80bf7fe700 1 -- 172.21.15.12:0/3747733194 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f80b8007510 con 0x55de91b8ed10 2023-10-05T19:40:49.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.931+0000 7f80bf7fe700 1 -- 172.21.15.12:0/3747733194 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80b8007670 con 0x55de91b8ed10 2023-10-05T19:40:49.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.931+0000 7f80bf7fe700 1 -- 172.21.15.12:0/3747733194 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f80b8007960 con 0x55de91b8ed10 2023-10-05T19:40:49.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.931+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3747733194 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de91b8ed10 msgr2=0x55de91bb1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:49.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.935+0000 7f80c7e14e40 1 --2- 172.21.15.12:0/3747733194 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de91b8ed10 0x55de91bb1150 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f80b800a6b0 tx=0x7f80b8018000 comp rx=0 tx=0).stop 2023-10-05T19:40:49.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.935+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3747733194 shutdown_connections 2023-10-05T19:40:49.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.935+0000 7f80c7e14e40 1 --2- 172.21.15.12:0/3747733194 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de91b8ed10 0x55de91bb1150 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:49.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.935+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3747733194 >> 172.21.15.12:0/3747733194 conn(0x55de91b8a730 msgr2=0x55de91ba8900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:49.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.935+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3747733194 shutdown_connections 2023-10-05T19:40:49.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.935+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3747733194 wait complete. 2023-10-05T19:40:49.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.935+0000 7f80c7e14e40 1 Processor -- start 2023-10-05T19:40:49.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.935+0000 7f80c7e14e40 1 -- start start 2023-10-05T19:40:49.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.935+0000 7f80c7e14e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de91b8ed10 0x55de91bb7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:49.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.935+0000 7f80c7e14e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de91a30ed0 con 0x55de91b8ed10 2023-10-05T19:40:49.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.935+0000 7f80c5949700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de91b8ed10 0x55de91bb7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:49.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.935+0000 7f80c5949700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de91b8ed10 0x55de91bb7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58548/0 (socket says 172.21.15.12:58548) 2023-10-05T19:40:49.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.935+0000 7f80c5949700 1 -- 172.21.15.12:0/3708625962 learned_addr learned my addr 172.21.15.12:0/3708625962 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:49.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.935+0000 7f80c5949700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de91b907d0 con 0x55de91b8ed10 2023-10-05T19:40:49.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.935+0000 7f80c5949700 1 --2- 172.21.15.12:0/3708625962 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de91b8ed10 0x55de91bb7e50 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f80b8005810 tx=0x7f80b801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:49.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.935+0000 7f80bdffb700 1 -- 172.21.15.12:0/3708625962 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f80b8007e40 con 0x55de91b8ed10 2023-10-05T19:40:49.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.935+0000 7f80bdffb700 1 -- 172.21.15.12:0/3708625962 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80b801c640 con 0x55de91b8ed10 2023-10-05T19:40:49.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.935+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55de91bb8390 con 0x55de91b8ed10 2023-10-05T19:40:49.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.935+0000 7f80bdffb700 1 -- 172.21.15.12:0/3708625962 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f80b801c8f0 con 0x55de91b8ed10 2023-10-05T19:40:49.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.935+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55de91bb8830 con 0x55de91b8ed10 2023-10-05T19:40:49.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.939+0000 7f80c7e14e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.939+0000 7f80bdffb700 1 -- 172.21.15.12:0/3708625962 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f80b801a040 con 0x55de91b8ed10 2023-10-05T19:40:49.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.939+0000 7f80bdffb700 1 --2- 172.21.15.12:0/3708625962 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f80a8030bf0 0x7f80a80330a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:49.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.939+0000 7f80bdffb700 1 -- 172.21.15.12:0/3708625962 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7f80b80450b0 con 0x55de91b8ed10 2023-10-05T19:40:49.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.939+0000 7f80c5148700 1 --2- 172.21.15.12:0/3708625962 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f80a8030bf0 0x7f80a80330a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:49.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.939+0000 7f80c7e14e40 1 --2- 172.21.15.12:0/3708625962 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55de91b8f620 0x55de91bc38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:49.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.939+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bc3fa0 con 0x55de91b8f620 2023-10-05T19:40:49.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.939+0000 7f80c614a700 1 --2- 172.21.15.12:0/3708625962 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55de91b8f620 0x55de91bc38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:49.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.939+0000 7f80c5148700 1 --2- 172.21.15.12:0/3708625962 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f80a8030bf0 0x7f80a80330a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f80b00016b0 tx=0x7f80b00067b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:49.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.939+0000 7f80c614a700 1 --2- 172.21.15.12:0/3708625962 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55de91b8f620 0x55de91bc38f0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:49.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.939+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f80c000a500 con 0x55de91b8f620 2023-10-05T19:40:49.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.939+0000 7f80c7e14e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:49.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.939+0000 7f80c7e14e40 20 realm 2023-10-05T19:40:49.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.939+0000 7f80c7e14e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.939+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bc0f50 con 0x55de91b8f620 2023-10-05T19:40:49.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.943+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f80c000a500 con 0x55de91b8f620 2023-10-05T19:40:49.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.943+0000 7f80c7e14e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:49.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.943+0000 7f80c7e14e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:49.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.943+0000 7f80c7e14e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.943+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bc4a80 con 0x55de91b8f620 2023-10-05T19:40:49.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.943+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f80c000a500 con 0x55de91b8f620 2023-10-05T19:40:49.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.943+0000 7f80c7e14e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:49.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.943+0000 7f80c7e14e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.943+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bc4690 con 0x55de91b8f620 2023-10-05T19:40:49.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.943+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f80c000a500 con 0x55de91b8f620 2023-10-05T19:40:49.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.943+0000 7f80c7e14e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:49.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.943+0000 7f80c7e14e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.943+0000 7f80c7e14e40 1 --2- 172.21.15.12:0/3708625962 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55de91bc5e20 0x55de91bc8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:49.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.943+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bc8740 con 0x55de91bc5e20 2023-10-05T19:40:49.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.943+0000 7f80c5949700 1 --2- 172.21.15.12:0/3708625962 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55de91bc5e20 0x55de91bc8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:49.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.943+0000 7f80c5949700 1 --2- 172.21.15.12:0/3708625962 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55de91bc5e20 0x55de91bc8200 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:49.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.943+0000 7f80c5949700 1 -- 172.21.15.12:0/3708625962 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f80b8023040 con 0x55de91bc5e20 2023-10-05T19:40:49.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.943+0000 7f80c7e14e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:49.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.943+0000 7f80c7e14e40 20 searching for the correct realm 2023-10-05T19:40:49.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.943+0000 7f80c7e14e40 1 --2- 172.21.15.12:0/3708625962 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55de91bca5c0 0x55de91bcc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:49.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.943+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcd080 con 0x55de91bca5c0 2023-10-05T19:40:49.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.943+0000 7f80c5148700 1 --2- 172.21.15.12:0/3708625962 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55de91bca5c0 0x55de91bcc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:49.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.943+0000 7f80c5148700 1 --2- 172.21.15.12:0/3708625962 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55de91bca5c0 0x55de91bcc9d0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:49.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.947+0000 7f80c5148700 1 -- 172.21.15.12:0/3708625962 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f80b0003450 con 0x55de91bca5c0 2023-10-05T19:40:49.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.947+0000 7f80c5148700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcd080 con 0x55de91bc5e20 2023-10-05T19:40:49.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.947+0000 7f80c5949700 1 -- 172.21.15.12:0/3708625962 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80b8023040 con 0x55de91bc5e20 2023-10-05T19:40:49.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.947+0000 7f80c5949700 1 --2- 172.21.15.12:0/3708625962 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f80b8007070 0x7f80b80438d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:49.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.947+0000 7f80c5949700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcd080 con 0x7f80b8007070 2023-10-05T19:40:49.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.947+0000 7f80c614a700 1 --2- 172.21.15.12:0/3708625962 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f80b8007070 0x7f80b80438d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:49.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.947+0000 7f80c614a700 1 --2- 172.21.15.12:0/3708625962 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f80b8007070 0x7f80b80438d0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:49.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.951+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80c0014040 con 0x7f80b8007070 2023-10-05T19:40:49.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.951+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcd080 con 0x55de91b8f620 2023-10-05T19:40:49.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.951+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80c0014040 con 0x55de91b8f620 2023-10-05T19:40:49.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.951+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcd080 con 0x55de91b8f620 2023-10-05T19:40:49.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.951+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80c0014040 con 0x55de91b8f620 2023-10-05T19:40:49.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.951+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcd080 con 0x55de91b8f620 2023-10-05T19:40:49.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.951+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(11 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80c0014040 con 0x55de91b8f620 2023-10-05T19:40:49.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.951+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcd080 con 0x7f80b8007070 2023-10-05T19:40:49.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.951+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80c0014040 con 0x7f80b8007070 2023-10-05T19:40:49.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.951+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcd080 con 0x55de91bc5e20 2023-10-05T19:40:49.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.955+0000 7f80c5949700 1 -- 172.21.15.12:0/3708625962 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80b8023040 con 0x55de91bc5e20 2023-10-05T19:40:49.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.955+0000 7f80c5949700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcd080 con 0x55de91bc5e20 2023-10-05T19:40:49.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.955+0000 7f80c5949700 1 -- 172.21.15.12:0/3708625962 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80b8023040 con 0x55de91bc5e20 2023-10-05T19:40:49.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.955+0000 7f80c5949700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcd080 con 0x55de91b8f620 2023-10-05T19:40:49.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.955+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80c0014040 con 0x55de91b8f620 2023-10-05T19:40:49.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.955+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcd080 con 0x7f80b8007070 2023-10-05T19:40:49.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.955+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80c0014040 con 0x7f80b8007070 2023-10-05T19:40:49.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.955+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcd080 con 0x55de91b8f620 2023-10-05T19:40:49.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.955+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80c0014040 con 0x55de91b8f620 2023-10-05T19:40:49.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.955+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcd080 con 0x55de91bc5e20 2023-10-05T19:40:49.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.959+0000 7f80c5949700 1 -- 172.21.15.12:0/3708625962 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(18 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80b8023040 con 0x55de91bc5e20 2023-10-05T19:40:49.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.959+0000 7f80c5949700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f80b8040040 con 0x55de91bca5c0 2023-10-05T19:40:49.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.959+0000 7f80c5148700 1 -- 172.21.15.12:0/3708625962 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80b0003450 con 0x55de91bca5c0 2023-10-05T19:40:49.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.959+0000 7f80c5148700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f80b8040040 con 0x55de91bca5c0 2023-10-05T19:40:49.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.959+0000 7f80c5148700 1 -- 172.21.15.12:0/3708625962 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80b0003450 con 0x55de91bca5c0 2023-10-05T19:40:49.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.959+0000 7f80c5148700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f80b8040040 con 0x7f80b8007070 2023-10-05T19:40:49.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.959+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80c0014040 con 0x7f80b8007070 2023-10-05T19:40:49.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.959+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f80b8040040 con 0x55de91bca5c0 2023-10-05T19:40:49.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.959+0000 7f80c5148700 1 -- 172.21.15.12:0/3708625962 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80b0003450 con 0x55de91bca5c0 2023-10-05T19:40:49.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.959+0000 7f80c5148700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f80b8040040 con 0x55de91bca5c0 2023-10-05T19:40:49.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.963+0000 7f80c5148700 1 -- 172.21.15.12:0/3708625962 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80b0003450 con 0x55de91bca5c0 2023-10-05T19:40:49.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.963+0000 7f80c5148700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f80b8040040 con 0x55de91b8f620 2023-10-05T19:40:49.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.963+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80c0014040 con 0x55de91b8f620 2023-10-05T19:40:49.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.963+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f80b8040040 con 0x55de91b8f620 2023-10-05T19:40:49.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.963+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80c0014040 con 0x55de91b8f620 2023-10-05T19:40:49.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.963+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f80b8040040 con 0x7f80b8007070 2023-10-05T19:40:49.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.963+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80c0014040 con 0x7f80b8007070 2023-10-05T19:40:49.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.963+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f80b8040040 con 0x55de91bca5c0 2023-10-05T19:40:49.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.967+0000 7f80c5148700 1 -- 172.21.15.12:0/3708625962 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80b001d040 con 0x55de91bca5c0 2023-10-05T19:40:49.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.967+0000 7f80c5148700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f80b8040040 con 0x55de91bc5e20 2023-10-05T19:40:49.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.967+0000 7f80c5949700 1 -- 172.21.15.12:0/3708625962 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80b8023040 con 0x55de91bc5e20 2023-10-05T19:40:49.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.967+0000 7f80c5949700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f80b8040040 con 0x55de91b8f620 2023-10-05T19:40:49.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.967+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80c0014040 con 0x55de91b8f620 2023-10-05T19:40:49.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.967+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f80b8040040 con 0x7f80b8007070 2023-10-05T19:40:49.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.967+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80c0014040 con 0x7f80b8007070 2023-10-05T19:40:49.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.967+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f80b8040040 con 0x55de91bca5c0 2023-10-05T19:40:49.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.967+0000 7f80c5148700 1 -- 172.21.15.12:0/3708625962 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80b001d040 con 0x55de91bca5c0 2023-10-05T19:40:49.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.967+0000 7f80c5148700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f80b8040040 con 0x7f80b8007070 2023-10-05T19:40:49.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.971+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80c0014040 con 0x7f80b8007070 2023-10-05T19:40:49.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.971+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f80b8040040 con 0x55de91bc5e20 2023-10-05T19:40:49.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.971+0000 7f80c5949700 1 -- 172.21.15.12:0/3708625962 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80b8023040 con 0x55de91bc5e20 2023-10-05T19:40:49.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.971+0000 7f80c5949700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f80b8040040 con 0x55de91bca5c0 2023-10-05T19:40:49.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.971+0000 7f80c5148700 1 -- 172.21.15.12:0/3708625962 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80b001d040 con 0x55de91bca5c0 2023-10-05T19:40:49.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.971+0000 7f80c5148700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f80b8040040 con 0x7f80b8007070 2023-10-05T19:40:49.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.971+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80c0014040 con 0x7f80b8007070 2023-10-05T19:40:49.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.971+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f80b8040040 con 0x55de91b8f620 2023-10-05T19:40:49.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.975+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f80c0014040 con 0x55de91b8f620 2023-10-05T19:40:49.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.975+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f80b8040040 con 0x55de91bc5e20 2023-10-05T19:40:49.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.975+0000 7f80c5949700 1 -- 172.21.15.12:0/3708625962 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(37 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f80b8023040 con 0x55de91bc5e20 2023-10-05T19:40:49.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.975+0000 7f80c7e14e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:49.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.975+0000 7f80c7e14e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:49.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.975+0000 7f80c7e14e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:49.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.975+0000 7f80c7e14e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:49.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.975+0000 7f80c7e14e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.975+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bc9340 con 0x55de91b8f620 2023-10-05T19:40:49.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.975+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f80c0014040 con 0x55de91b8f620 2023-10-05T19:40:49.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.975+0000 7f80c7e14e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:49.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.975+0000 7f80c7e14e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.975+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bc8f70 con 0x55de91b8f620 2023-10-05T19:40:49.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.975+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f80c0014040 con 0x55de91b8f620 2023-10-05T19:40:49.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.975+0000 7f80c7e14e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:49.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.975+0000 7f80c7e14e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.975+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcd4f0 con 0x55de91bca5c0 2023-10-05T19:40:49.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.975+0000 7f80c5148700 1 -- 172.21.15.12:0/3708625962 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f80b001d040 con 0x55de91bca5c0 2023-10-05T19:40:49.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.975+0000 7f80c7e14e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:49.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.975+0000 7f80c7e14e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.975+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcd8a0 con 0x55de91bca5c0 2023-10-05T19:40:49.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.979+0000 7f80c5148700 1 -- 172.21.15.12:0/3708625962 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f80b001d040 con 0x55de91bca5c0 2023-10-05T19:40:49.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.979+0000 7f80c7e14e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:49.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.979+0000 7f80c7e14e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.979+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcdc50 con 0x55de91b8f620 2023-10-05T19:40:49.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.979+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f80c0014040 con 0x55de91b8f620 2023-10-05T19:40:49.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.979+0000 7f80c7e14e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:49.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.979+0000 7f80c7e14e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:49.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.979+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bc9830 con 0x55de91b8f620 2023-10-05T19:40:49.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.979+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f80c0014040 con 0x55de91b8f620 2023-10-05T19:40:49.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.979+0000 7f80c7e14e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:49.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.979+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bc9800 con 0x55de91bca5c0 2023-10-05T19:40:49.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.983+0000 7f80c5148700 1 -- 172.21.15.12:0/3708625962 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(44 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f80b001d040 con 0x55de91bca5c0 2023-10-05T19:40:49.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.983+0000 7f80c5148700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bc9800 con 0x55de91bc5e20 2023-10-05T19:40:49.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.983+0000 7f80c5949700 1 -- 172.21.15.12:0/3708625962 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(45 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80b8023040 con 0x55de91bc5e20 2023-10-05T19:40:49.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.983+0000 7f80c5949700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bc9800 con 0x7f80b8007070 2023-10-05T19:40:49.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.983+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(46 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80c0014040 con 0x7f80b8007070 2023-10-05T19:40:49.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.983+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcdc50 con 0x55de91b8f620 2023-10-05T19:40:49.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.983+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(47 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80c0014040 con 0x55de91b8f620 2023-10-05T19:40:49.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.983+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcdc50 con 0x55de91b8f620 2023-10-05T19:40:49.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.987+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(48 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80c0014040 con 0x55de91b8f620 2023-10-05T19:40:49.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.987+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcdc50 con 0x55de91b8f620 2023-10-05T19:40:49.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.987+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(49 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80c0014040 con 0x55de91b8f620 2023-10-05T19:40:49.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.987+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcdc50 con 0x7f80b8007070 2023-10-05T19:40:49.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.987+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(50 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80c0014040 con 0x7f80b8007070 2023-10-05T19:40:49.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.987+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcdc50 con 0x55de91bc5e20 2023-10-05T19:40:49.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.987+0000 7f80c5949700 1 -- 172.21.15.12:0/3708625962 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(51 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80b8023040 con 0x55de91bc5e20 2023-10-05T19:40:49.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.987+0000 7f80c5949700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcdc50 con 0x55de91bc5e20 2023-10-05T19:40:49.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.987+0000 7f80c5949700 1 -- 172.21.15.12:0/3708625962 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(52 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80b8023040 con 0x55de91bc5e20 2023-10-05T19:40:49.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.991+0000 7f80c5949700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcdc50 con 0x55de91b8f620 2023-10-05T19:40:49.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.991+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(53 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80c0014040 con 0x55de91b8f620 2023-10-05T19:40:49.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.991+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcdc50 con 0x7f80b8007070 2023-10-05T19:40:49.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.991+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(54 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80c0014040 con 0x7f80b8007070 2023-10-05T19:40:49.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.991+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcdc50 con 0x55de91b8f620 2023-10-05T19:40:49.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.991+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(55 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80c0014040 con 0x55de91b8f620 2023-10-05T19:40:49.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.991+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcdc50 con 0x55de91bc5e20 2023-10-05T19:40:50.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.991+0000 7f80c5949700 1 -- 172.21.15.12:0/3708625962 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(56 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80b8023040 con 0x55de91bc5e20 2023-10-05T19:40:50.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.991+0000 7f80c5949700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcdc50 con 0x55de91bca5c0 2023-10-05T19:40:50.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.995+0000 7f80c5148700 1 -- 172.21.15.12:0/3708625962 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(57 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80b001d040 con 0x55de91bca5c0 2023-10-05T19:40:50.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.995+0000 7f80c5148700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcdc50 con 0x55de91bca5c0 2023-10-05T19:40:50.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.995+0000 7f80c5148700 1 -- 172.21.15.12:0/3708625962 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(58 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80b001d040 con 0x55de91bca5c0 2023-10-05T19:40:50.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.995+0000 7f80c5148700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcdc50 con 0x7f80b8007070 2023-10-05T19:40:50.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.995+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(59 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80c0014040 con 0x7f80b8007070 2023-10-05T19:40:50.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.995+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcdc50 con 0x55de91bca5c0 2023-10-05T19:40:50.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.995+0000 7f80c5148700 1 -- 172.21.15.12:0/3708625962 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(60 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80b001d040 con 0x55de91bca5c0 2023-10-05T19:40:50.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.995+0000 7f80c5148700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcdc50 con 0x55de91bca5c0 2023-10-05T19:40:50.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.999+0000 7f80c5148700 1 -- 172.21.15.12:0/3708625962 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(61 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80b001d040 con 0x55de91bca5c0 2023-10-05T19:40:50.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.999+0000 7f80c5148700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcdc50 con 0x55de91b8f620 2023-10-05T19:40:50.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.999+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(62 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80c0014040 con 0x55de91b8f620 2023-10-05T19:40:50.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.999+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcdc50 con 0x55de91b8f620 2023-10-05T19:40:50.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.999+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(63 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80c0014040 con 0x55de91b8f620 2023-10-05T19:40:50.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.999+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcdc50 con 0x7f80b8007070 2023-10-05T19:40:50.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.999+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(64 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80c0014040 con 0x7f80b8007070 2023-10-05T19:40:50.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:49.999+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcdc50 con 0x55de91bca5c0 2023-10-05T19:40:50.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.003+0000 7f80c5148700 1 -- 172.21.15.12:0/3708625962 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(65 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80b001d040 con 0x55de91bca5c0 2023-10-05T19:40:50.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.003+0000 7f80c5148700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcdc50 con 0x55de91bc5e20 2023-10-05T19:40:50.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.003+0000 7f80c5949700 1 -- 172.21.15.12:0/3708625962 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(66 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80b8023040 con 0x55de91bc5e20 2023-10-05T19:40:50.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.003+0000 7f80c5949700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcdc50 con 0x55de91b8f620 2023-10-05T19:40:50.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.003+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(67 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80c0014040 con 0x55de91b8f620 2023-10-05T19:40:50.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.003+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcdc50 con 0x7f80b8007070 2023-10-05T19:40:50.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.003+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(68 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80c0014040 con 0x7f80b8007070 2023-10-05T19:40:50.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.003+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcdc50 con 0x55de91bca5c0 2023-10-05T19:40:50.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.003+0000 7f80c5148700 1 -- 172.21.15.12:0/3708625962 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(69 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80b001d040 con 0x55de91bca5c0 2023-10-05T19:40:50.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.003+0000 7f80c5148700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcdc50 con 0x7f80b8007070 2023-10-05T19:40:50.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.007+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(70 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80c0014040 con 0x7f80b8007070 2023-10-05T19:40:50.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.007+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcdc50 con 0x55de91bc5e20 2023-10-05T19:40:50.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.007+0000 7f80c5949700 1 -- 172.21.15.12:0/3708625962 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(71 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80b8023040 con 0x55de91bc5e20 2023-10-05T19:40:50.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.007+0000 7f80c5949700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcdc50 con 0x55de91bca5c0 2023-10-05T19:40:50.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.007+0000 7f80c5148700 1 -- 172.21.15.12:0/3708625962 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(72 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80b001d040 con 0x55de91bca5c0 2023-10-05T19:40:50.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.007+0000 7f80c5148700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcdc50 con 0x7f80b8007070 2023-10-05T19:40:50.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.007+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(73 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f80c0014040 con 0x7f80b8007070 2023-10-05T19:40:50.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.007+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcdc50 con 0x55de91b8f620 2023-10-05T19:40:50.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.007+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(74 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f80c0014040 con 0x55de91b8f620 2023-10-05T19:40:50.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.007+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcdc50 con 0x55de91bc5e20 2023-10-05T19:40:50.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.011+0000 7f80c5949700 1 -- 172.21.15.12:0/3708625962 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(75 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f80b8023040 con 0x55de91bc5e20 2023-10-05T19:40:50.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.011+0000 7f80c7e14e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:50.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.011+0000 7f80c7e14e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:50.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.011+0000 7f80c7e14e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:50.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.011+0000 7f80c7e14e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:50.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.011+0000 7f80c7e14e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.011+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bc9800 con 0x55de91b8f620 2023-10-05T19:40:50.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.011+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f80c0014040 con 0x55de91b8f620 2023-10-05T19:40:50.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.011+0000 7f80c7e14e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:50.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.011+0000 7f80c7e14e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.011+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bce590 con 0x55de91bc5e20 2023-10-05T19:40:50.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.011+0000 7f80c5949700 1 -- 172.21.15.12:0/3708625962 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f80b8023040 con 0x55de91bc5e20 2023-10-05T19:40:50.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.011+0000 7f80c7e14e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:50.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.011+0000 7f80c7e14e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.011+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcee30 con 0x55de91b8f620 2023-10-05T19:40:50.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.011+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f80c0014040 con 0x55de91b8f620 2023-10-05T19:40:50.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.011+0000 7f80c7e14e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:50.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.011+0000 7f80c7e14e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.011+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcee30 con 0x55de91bca5c0 2023-10-05T19:40:50.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.015+0000 7f80c5148700 1 -- 172.21.15.12:0/3708625962 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f80b001d040 con 0x55de91bca5c0 2023-10-05T19:40:50.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.015+0000 7f80c7e14e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:50.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.015+0000 7f80c7e14e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.015+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcf6d0 con 0x7f80b8007070 2023-10-05T19:40:50.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.015+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f80c0014040 con 0x7f80b8007070 2023-10-05T19:40:50.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.015+0000 7f80c7e14e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:50.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.015+0000 7f80c7e14e40 20 zone default found 2023-10-05T19:40:50.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.015+0000 7f80c7e14e40 4 Realm: () 2023-10-05T19:40:50.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.015+0000 7f80c7e14e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:50.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.015+0000 7f80c7e14e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:50.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.015+0000 7f80c7e14e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:50.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.015+0000 7f80c7e14e40 20 zonegroup default 2023-10-05T19:40:50.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.015+0000 7f80c7e14e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.015+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bcf6d0 con 0x7f80b8007070 2023-10-05T19:40:50.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.015+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f80c0014040 con 0x7f80b8007070 2023-10-05T19:40:50.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.015+0000 7f80c7e14e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:50.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.015+0000 7f80c7e14e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.015+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bd1050 con 0x55de91bca5c0 2023-10-05T19:40:50.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.015+0000 7f80c5148700 1 -- 172.21.15.12:0/3708625962 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f80b001d040 con 0x55de91bca5c0 2023-10-05T19:40:50.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.015+0000 7f80c7e14e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:50.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.015+0000 7f80c7e14e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.015+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bd1400 con 0x55de91bca5c0 2023-10-05T19:40:50.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.015+0000 7f80c5148700 1 -- 172.21.15.12:0/3708625962 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f80b001d040 con 0x55de91bca5c0 2023-10-05T19:40:50.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.015+0000 7f80c7e14e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:50.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.015+0000 7f80c7e14e40 20 started sync module instance, tier type = 2023-10-05T19:40:50.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.015+0000 7f80c7e14e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:50.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.015+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bd0710 con 0x55de91bc5e20 2023-10-05T19:40:50.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.019+0000 7f80c5949700 1 -- 172.21.15.12:0/3708625962 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(84 notify.0 [create] v30'204 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f80b8023040 con 0x55de91bc5e20 2023-10-05T19:40:50.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.019+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94414416183840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bd17b0 con 0x55de91bc5e20 2023-10-05T19:40:50.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.019+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bd2050 con 0x55de91b8f620 2023-10-05T19:40:50.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.023+0000 7f80c5949700 1 -- 172.21.15.12:0/3708625962 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(85 notify.0 [watch watch cookie 94414416183840] v30'205 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f80b8023040 con 0x55de91bc5e20 2023-10-05T19:40:50.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.023+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(86 notify.1 [create] v30'204 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f80c0014040 con 0x55de91b8f620 2023-10-05T19:40:50.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.023+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94414416186048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bd1ca0 con 0x55de91b8f620 2023-10-05T19:40:50.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.023+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bd2720 con 0x55de91bca5c0 2023-10-05T19:40:50.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.023+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(87 notify.1 [watch watch cookie 94414416186048] v30'205 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f80c0014040 con 0x55de91b8f620 2023-10-05T19:40:50.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.027+0000 7f80c5148700 1 -- 172.21.15.12:0/3708625962 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(88 notify.2 [create] v30'204 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f80b001d040 con 0x55de91bca5c0 2023-10-05T19:40:50.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.027+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94414416189360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bd2350 con 0x55de91bca5c0 2023-10-05T19:40:50.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.027+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bd3500 con 0x55de91bc5e20 2023-10-05T19:40:50.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.027+0000 7f80c5148700 1 -- 172.21.15.12:0/3708625962 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(89 notify.2 [watch watch cookie 94414416189360] v30'205 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f80b001d040 con 0x55de91bca5c0 2023-10-05T19:40:50.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.027+0000 7f80c5949700 1 -- 172.21.15.12:0/3708625962 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(90 notify.3 [create] v30'204 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f80b8023040 con 0x55de91bc5e20 2023-10-05T19:40:50.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.027+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94414416194624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bd3580 con 0x55de91bc5e20 2023-10-05T19:40:50.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.027+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bd3ff0 con 0x55de91b8f620 2023-10-05T19:40:50.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.031+0000 7f80c5949700 1 -- 172.21.15.12:0/3708625962 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(91 notify.3 [watch watch cookie 94414416194624] v30'205 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f80b8023040 con 0x55de91bc5e20 2023-10-05T19:40:50.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.031+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(92 notify.4 [create] v30'204 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f80c0014040 con 0x55de91b8f620 2023-10-05T19:40:50.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.031+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94414416196288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bd3ff0 con 0x55de91b8f620 2023-10-05T19:40:50.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.031+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bd4a00 con 0x7f80b8007070 2023-10-05T19:40:50.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.035+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(93 notify.4 [watch watch cookie 94414416196288] v30'205 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f80c0014040 con 0x55de91b8f620 2023-10-05T19:40:50.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.035+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(94 notify.5 [create] v30'204 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f80c0014040 con 0x7f80b8007070 2023-10-05T19:40:50.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.035+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94414416198960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bd4a00 con 0x7f80b8007070 2023-10-05T19:40:50.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.035+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bd5410 con 0x7f80b8007070 2023-10-05T19:40:50.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.035+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(96 notify.6 [create] v30'204 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f80c0014040 con 0x7f80b8007070 2023-10-05T19:40:50.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.035+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94414416203552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bd5860 con 0x7f80b8007070 2023-10-05T19:40:50.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.035+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bd6310 con 0x7f80b8007070 2023-10-05T19:40:50.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.035+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(95 notify.5 [watch watch cookie 94414416198960] v30'205 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f80c0014040 con 0x7f80b8007070 2023-10-05T19:40:50.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.039+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(97 notify.6 [watch watch cookie 94414416203552] v30'205 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f80c0014040 con 0x7f80b8007070 2023-10-05T19:40:50.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.039+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(98 notify.7 [create] v30'204 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f80c0014040 con 0x7f80b8007070 2023-10-05T19:40:50.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.039+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94414416204656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bd5de0 con 0x7f80b8007070 2023-10-05T19:40:50.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.039+0000 7f80c7e14e40 20 add_watcher() i=0 2023-10-05T19:40:50.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.039+0000 7f80c7e14e40 20 add_watcher() i=1 2023-10-05T19:40:50.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.039+0000 7f80c7e14e40 20 add_watcher() i=2 2023-10-05T19:40:50.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.039+0000 7f80c7e14e40 20 add_watcher() i=3 2023-10-05T19:40:50.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.039+0000 7f80c7e14e40 20 add_watcher() i=4 2023-10-05T19:40:50.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.039+0000 7f80c7e14e40 20 add_watcher() i=5 2023-10-05T19:40:50.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.039+0000 7f80c7e14e40 20 add_watcher() i=6 2023-10-05T19:40:50.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.039+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(99 notify.7 [watch watch cookie 94414416204656] v30'205 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f80c0014040 con 0x7f80b8007070 2023-10-05T19:40:50.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.039+0000 7f80c7e14e40 20 add_watcher() i=7 2023-10-05T19:40:50.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.039+0000 7f80c7e14e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:50.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.039+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bd9300 con 0x55de91b8f620 2023-10-05T19:40:50.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.043+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f80c0014040 con 0x55de91b8f620 2023-10-05T19:40:50.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.043+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94414416207328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bd6720 con 0x55de91b8f620 2023-10-05T19:40:50.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.043+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94414416207328] v30'201 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f80c0014040 con 0x55de91b8f620 2023-10-05T19:40:50.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.043+0000 7f8096fe5700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:50.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.043+0000 7f80c7e14e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:50.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.043+0000 7f80c7e14e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:50.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.043+0000 7f80c7e14e40 1 Processor -- start 2023-10-05T19:40:50.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.043+0000 7f80c7e14e40 1 -- start start 2023-10-05T19:40:50.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.043+0000 7f80c7e14e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de91be3750 0x55de91be3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:50.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.043+0000 7f80c7e14e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de91be4060 con 0x55de91be3750 2023-10-05T19:40:50.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.043+0000 7f80c5148700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de91be3750 0x55de91be3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:50.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.043+0000 7f80c5148700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de91be3750 0x55de91be3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58556/0 (socket says 172.21.15.12:58556) 2023-10-05T19:40:50.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.043+0000 7f80c5148700 1 -- 172.21.15.12:0/3449323703 learned_addr learned my addr 172.21.15.12:0/3449323703 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:50.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.047+0000 7f80c5148700 1 -- 172.21.15.12:0/3449323703 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de91be41a0 con 0x55de91be3750 2023-10-05T19:40:50.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.047+0000 7f80c5148700 1 --2- 172.21.15.12:0/3449323703 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de91be3750 0x55de91be3b20 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f80b8043550 tx=0x7f80b0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ef0f7cee07d69e server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:50.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.047+0000 7f80967e4700 1 -- 172.21.15.12:0/3449323703 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f80b001a040 con 0x55de91be3750 2023-10-05T19:40:50.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.047+0000 7f80967e4700 1 -- 172.21.15.12:0/3449323703 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80b002d040 con 0x55de91be3750 2023-10-05T19:40:50.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.047+0000 7f80967e4700 1 -- 172.21.15.12:0/3449323703 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f80b0003e20 con 0x55de91be3750 2023-10-05T19:40:50.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.047+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3449323703 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de91be3750 msgr2=0x55de91be3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.047+0000 7f80c7e14e40 1 --2- 172.21.15.12:0/3449323703 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de91be3750 0x55de91be3b20 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f80b8043550 tx=0x7f80b0010000 comp rx=0 tx=0).stop 2023-10-05T19:40:50.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.047+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3449323703 shutdown_connections 2023-10-05T19:40:50.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.047+0000 7f80c7e14e40 1 --2- 172.21.15.12:0/3449323703 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de91be3750 0x55de91be3b20 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:50.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.047+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3449323703 >> 172.21.15.12:0/3449323703 conn(0x55de91bdeb10 msgr2=0x55de91bdcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:50.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.047+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3449323703 shutdown_connections 2023-10-05T19:40:50.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.047+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3449323703 wait complete. 2023-10-05T19:40:50.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.047+0000 7f80c7e14e40 1 Processor -- start 2023-10-05T19:40:50.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.047+0000 7f80c7e14e40 1 -- start start 2023-10-05T19:40:50.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.047+0000 7f80c7e14e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de91be3750 0x55de91be92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:50.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.047+0000 7f80c7e14e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de91bea2b0 con 0x55de91be3750 2023-10-05T19:40:50.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.047+0000 7f80c5148700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de91be3750 0x55de91be92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:50.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.047+0000 7f80c5148700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de91be3750 0x55de91be92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58560/0 (socket says 172.21.15.12:58560) 2023-10-05T19:40:50.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.047+0000 7f80c5148700 1 -- 172.21.15.12:0/1344754847 learned_addr learned my addr 172.21.15.12:0/1344754847 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:50.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.051+0000 7f80c5148700 1 -- 172.21.15.12:0/1344754847 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de91adcbd0 con 0x55de91be3750 2023-10-05T19:40:50.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.051+0000 7f80c5148700 1 --2- 172.21.15.12:0/1344754847 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de91be3750 0x55de91be92b0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f80b0006470 tx=0x7f80b0038000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:50.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.051+0000 7f8094fe1700 1 -- 172.21.15.12:0/1344754847 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f80b001a040 con 0x55de91be3750 2023-10-05T19:40:50.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.051+0000 7f8094fe1700 1 -- 172.21.15.12:0/1344754847 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80b001b040 con 0x55de91be3750 2023-10-05T19:40:50.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.051+0000 7f80c7e14e40 1 -- 172.21.15.12:0/1344754847 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55de91be98b0 con 0x55de91be3750 2023-10-05T19:40:50.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.051+0000 7f8094fe1700 1 -- 172.21.15.12:0/1344754847 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f80b0025e80 con 0x55de91be3750 2023-10-05T19:40:50.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.051+0000 7f80c7e14e40 1 -- 172.21.15.12:0/1344754847 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55de91be9c90 con 0x55de91be3750 2023-10-05T19:40:50.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.051+0000 7f8094fe1700 1 -- 172.21.15.12:0/1344754847 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f80b0026440 con 0x55de91be3750 2023-10-05T19:40:50.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.051+0000 7f8094fe1700 1 --2- 172.21.15.12:0/1344754847 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f800c030c50 0x7f800c033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:50.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.051+0000 7f8094fe1700 1 -- 172.21.15.12:0/1344754847 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7f80b0056bd0 con 0x55de91be3750 2023-10-05T19:40:50.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.051+0000 7f80c7e14e40 5 note: GC not initialized 2023-10-05T19:40:50.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.051+0000 7f80c614a700 1 --2- 172.21.15.12:0/1344754847 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f800c030c50 0x7f800c033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:50.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.051+0000 7f80c614a700 1 --2- 172.21.15.12:0/1344754847 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f800c030c50 0x7f800c033100 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x55de91bd4720 tx=0x7f80c000ab80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:50.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.055+0000 7f8005ffb700 20 reqs_thread_entry: start 2023-10-05T19:40:50.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.055+0000 7f7fff7fe700 20 reqs_thread_entry: start 2023-10-05T19:40:50.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.055+0000 7f80c7e14e40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:50.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.055+0000 7f80c7e14e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:50.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.055+0000 7f7ffdffb700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:41:20 2023 2023-10-05T19:40:50.063 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:50.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.055+0000 7f7ffd7fa700 20 reqs_thread_entry: start 2023-10-05T19:40:50.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.055+0000 7f80c7e14e40 20 get_system_obj_state: rctx=0x7ffcc5817cb0 obj=default.rgw.meta:root:myfoo state=0x55de91a690b0 s->prefetch_data=0 2023-10-05T19:40:50.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.055+0000 7f80c7e14e40 10 cache get: name=default.rgw.meta+root+myfoo : miss 2023-10-05T19:40:50.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.055+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:102 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91c522a0 con 0x55de91bc5e20 2023-10-05T19:40:50.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.055+0000 7f80c5949700 1 -- 172.21.15.12:0/3708625962 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(102 myfoo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv33 ondisk = 0) v8 ==== 233+0+139 (crc 0 0 0) 0x7f80b8023040 con 0x55de91bc5e20 2023-10-05T19:40:50.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.055+0000 7f80c7e14e40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x16 2023-10-05T19:40:50.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.055+0000 7f80c7e14e40 10 adding default.rgw.meta+root+myfoo to cache LRU end 2023-10-05T19:40:50.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.055+0000 7f80c7e14e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:50.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.055+0000 7f80c7e14e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:50.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.055+0000 7f80c7e14e40 10 cache get: name=default.rgw.meta+root+myfoo : type miss (requested=0x11, cached=0x16) 2023-10-05T19:40:50.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.055+0000 7f80c7e14e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.055+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:103 8.9 8:91b71841:root::myfoo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91c52190 con 0x55de91bc5e20 2023-10-05T19:40:50.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.055+0000 7f80c5949700 1 -- 172.21.15.12:0/3708625962 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(103 myfoo [call,call out=48b,read 0~165 out=165b] v0'0 uv33 ondisk = 0) v8 ==== 233+0+213 (crc 0 0 0) 0x7f80b8023040 con 0x55de91bc5e20 2023-10-05T19:40:50.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.055+0000 7f80c7e14e40 20 rados_obj.operate() r=0 bl.length=165 2023-10-05T19:40:50.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.055+0000 7f80c7e14e40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2023-10-05T19:40:50.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.055+0000 7f80c7e14e40 10 moving default.rgw.meta+root+myfoo to cache LRU end 2023-10-05T19:40:50.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.055+0000 7f80c7e14e40 20 get_system_obj_state: rctx=0x7ffcc5817cb0 obj=default.rgw.meta:root:.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5 state=0x55de91a3f2b0 s->prefetch_data=0 2023-10-05T19:40:50.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.055+0000 7f80c7e14e40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5 : miss 2023-10-05T19:40:50.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.055+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:104 8.10 8:0a6cdd29:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91c53d20 con 0x55de91bca5c0 2023-10-05T19:40:50.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.059+0000 7f80c5148700 1 -- 172.21.15.12:0/3708625962 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(104 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5 [call out=48b,getxattrs out=218b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 290+0+282 (crc 0 0 0) 0x7f80b001d040 con 0x55de91bca5c0 2023-10-05T19:40:50.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.059+0000 7f80c7e14e40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5 info.flags=0x16 2023-10-05T19:40:50.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.059+0000 7f80c7e14e40 10 adding default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5 to cache LRU end 2023-10-05T19:40:50.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.059+0000 7f80c7e14e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:50.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.059+0000 7f80c7e14e40 10 updating xattr: name=user.rgw.acl bl.length()=123 2023-10-05T19:40:50.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.059+0000 7f80c7e14e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:50.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.059+0000 7f80c7e14e40 20 Read xattr: user.rgw.acl 2023-10-05T19:40:50.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.059+0000 7f80c7e14e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:50.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.059+0000 7f80c7e14e40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5 : type miss (requested=0x13, cached=0x16) 2023-10-05T19:40:50.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.059+0000 7f80c7e14e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.059+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:105 8.10 8:0a6cdd29:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91c53d20 con 0x55de91bca5c0 2023-10-05T19:40:50.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.059+0000 7f80c5148700 1 -- 172.21.15.12:0/3708625962 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(105 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5 [call,call out=48b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 332+0+618 (crc 0 0 0) 0x7f80b001d040 con 0x55de91bca5c0 2023-10-05T19:40:50.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.059+0000 7f80c7e14e40 20 rados_obj.operate() r=0 bl.length=352 2023-10-05T19:40:50.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.059+0000 7f80c7e14e40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5 info.flags=0x13 2023-10-05T19:40:50.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.059+0000 7f80c7e14e40 10 moving default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5 to cache LRU end 2023-10-05T19:40:50.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.059+0000 7f80c7e14e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:50.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.059+0000 7f80c7e14e40 10 updating xattr: name=user.rgw.acl bl.length()=123 2023-10-05T19:40:50.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.059+0000 7f80c7e14e40 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5 2023-10-05T19:40:50.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.059+0000 7f80c7e14e40 20 get_obj_state: rctx=0x7ffcc5817ef0 obj=myfoo:bb3aec0fdcdbc2974890f805c585d432 state=0x55de91c57378 s->prefetch_data=0 2023-10-05T19:40:50.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.059+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:106 3.26 3:6571379c:::93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5_bb3aec0fdcdbc2974890f805c585d432:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91c57ce0 con 0x55de91bc5e20 2023-10-05T19:40:50.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.059+0000 7f80c5949700 1 -- 172.21.15.12:0/3708625962 <== osd.2 v2:172.21.15.12:6800/29934 23 ==== osd_op_reply(106 93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5_bb3aec0fdcdbc2974890f805c585d432 [getxattrs out=911b,stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 262+0+927 (crc 0 0 0) 0x7f80b8023040 con 0x55de91bc5e20 2023-10-05T19:40:50.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.059+0000 7f80c7e14e40 10 manifest: total_size = 5 2023-10-05T19:40:50.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.059+0000 7f80c7e14e40 20 get_obj_state: setting s->obj_tag to _7PC3k5E6JWvxHQbDyNW5w0aEodn5W5E 2023-10-05T19:40:50.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.059+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:107 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94414416183840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f80b8052000 con 0x55de91bc5e20 2023-10-05T19:40:50.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.063+0000 7f80c5949700 1 -- 172.21.15.12:0/3708625962 <== osd.2 v2:172.21.15.12:6800/29934 24 ==== osd_op_reply(107 notify.0 [watch unwatch cookie 94414416183840] v30'206 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f80b8023040 con 0x55de91bc5e20 2023-10-05T19:40:50.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.063+0000 7f80c7e14e40 20 remove_watcher() i=0 2023-10-05T19:40:50.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.063+0000 7f80c7e14e40 2 removed watcher, disabling cache 2023-10-05T19:40:50.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.063+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:108 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94414416186048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bd0220 con 0x55de91b8f620 2023-10-05T19:40:50.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.067+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(108 notify.1 [watch unwatch cookie 94414416186048] v30'206 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f80c0014040 con 0x55de91b8f620 2023-10-05T19:40:50.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.067+0000 7f80c7e14e40 20 remove_watcher() i=1 2023-10-05T19:40:50.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.067+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:109 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94414416189360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bd0220 con 0x55de91bca5c0 2023-10-05T19:40:50.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.067+0000 7f80c5148700 1 -- 172.21.15.12:0/3708625962 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(109 notify.2 [watch unwatch cookie 94414416189360] v30'206 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f80b001d040 con 0x55de91bca5c0 2023-10-05T19:40:50.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.067+0000 7f80c7e14e40 20 remove_watcher() i=2 2023-10-05T19:40:50.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.067+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:110 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94414416194624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bd17b0 con 0x55de91bc5e20 2023-10-05T19:40:50.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.071+0000 7f80c5949700 1 -- 172.21.15.12:0/3708625962 <== osd.2 v2:172.21.15.12:6800/29934 25 ==== osd_op_reply(110 notify.3 [watch unwatch cookie 94414416194624] v30'206 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f80b8023040 con 0x55de91bc5e20 2023-10-05T19:40:50.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.071+0000 7f80c7e14e40 20 remove_watcher() i=3 2023-10-05T19:40:50.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.071+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:111 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94414416196288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bd0220 con 0x55de91b8f620 2023-10-05T19:40:50.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.075+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(111 notify.4 [watch unwatch cookie 94414416196288] v30'206 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f80c0014040 con 0x55de91b8f620 2023-10-05T19:40:50.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.075+0000 7f80c7e14e40 20 remove_watcher() i=4 2023-10-05T19:40:50.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.075+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:112 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94414416198960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bd2c40 con 0x7f80b8007070 2023-10-05T19:40:50.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.075+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(112 notify.5 [watch unwatch cookie 94414416198960] v30'206 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f80c0014040 con 0x7f80b8007070 2023-10-05T19:40:50.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.075+0000 7f80c7e14e40 20 remove_watcher() i=5 2023-10-05T19:40:50.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.075+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:113 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94414416203552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bd17b0 con 0x7f80b8007070 2023-10-05T19:40:50.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.079+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(113 notify.6 [watch unwatch cookie 94414416203552] v30'206 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f80c0014040 con 0x7f80b8007070 2023-10-05T19:40:50.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.079+0000 7f80c7e14e40 20 remove_watcher() i=6 2023-10-05T19:40:50.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.079+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:114 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94414416204656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bd32c0 con 0x7f80b8007070 2023-10-05T19:40:50.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.083+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(114 notify.7 [watch unwatch cookie 94414416204656] v30'206 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f80c0014040 con 0x7f80b8007070 2023-10-05T19:40:50.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.083+0000 7f80c7e14e40 20 remove_watcher() i=7 2023-10-05T19:40:50.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.083+0000 7f8096fe5700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:50.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.083+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:115 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94414416207328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55de91bd3d30 con 0x55de91b8f620 2023-10-05T19:40:50.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.087+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(115 data_loggenerations_metadata [watch unwatch cookie 94414416207328] v30'202 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f80c0014040 con 0x55de91b8f620 2023-10-05T19:40:50.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.087+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f80b8007070 msgr2=0x7f80b80438d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.087+0000 7f80c7e14e40 1 --2- 172.21.15.12:0/3708625962 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f80b8007070 0x7f80b80438d0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:50.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.087+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55de91b8f620 msgr2=0x55de91bc38f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.087+0000 7f80c7e14e40 1 --2- 172.21.15.12:0/3708625962 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55de91b8f620 0x55de91bc38f0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:50.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.087+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55de91bc5e20 msgr2=0x55de91bc8200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.087+0000 7f80c7e14e40 1 --2- 172.21.15.12:0/3708625962 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55de91bc5e20 0x55de91bc8200 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:50.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.087+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55de91bca5c0 msgr2=0x55de91bcc9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.087+0000 7f80c7e14e40 1 --2- 172.21.15.12:0/3708625962 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55de91bca5c0 0x55de91bcc9d0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:50.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.087+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f80a8030bf0 msgr2=0x7f80a80330a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.087+0000 7f80c7e14e40 1 --2- 172.21.15.12:0/3708625962 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f80a8030bf0 0x7f80a80330a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f80b00016b0 tx=0x7f80b00067b0 comp rx=0 tx=0).stop 2023-10-05T19:40:50.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.087+0000 7f80c614a700 1 -- 172.21.15.12:0/3708625962 reap_dead start 2023-10-05T19:40:50.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.087+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de91b8ed10 msgr2=0x55de91bb7e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.087+0000 7f80c7e14e40 1 --2- 172.21.15.12:0/3708625962 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de91b8ed10 0x55de91bb7e50 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f80b8005810 tx=0x7f80b801b000 comp rx=0 tx=0).stop 2023-10-05T19:40:50.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.087+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 shutdown_connections 2023-10-05T19:40:50.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.087+0000 7f80c7e14e40 1 --2- 172.21.15.12:0/3708625962 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de91b8ed10 0x55de91bb7e50 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:50.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.087+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 >> 172.21.15.12:0/3708625962 conn(0x55de91b8a730 msgr2=0x55de91ba7380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:50.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.087+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 shutdown_connections 2023-10-05T19:40:50.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.087+0000 7f80c7e14e40 1 -- 172.21.15.12:0/3708625962 wait complete. 2023-10-05T19:40:50.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.087+0000 7f80c7e14e40 1 -- 172.21.15.12:0/1344754847 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f800c030c50 msgr2=0x7f800c033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.087+0000 7f80c7e14e40 1 --2- 172.21.15.12:0/1344754847 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f800c030c50 0x7f800c033100 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x55de91bd4720 tx=0x7f80c000ab80 comp rx=0 tx=0).stop 2023-10-05T19:40:50.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.087+0000 7f80c7e14e40 1 -- 172.21.15.12:0/1344754847 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de91be3750 msgr2=0x55de91be92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.087+0000 7f80c7e14e40 1 --2- 172.21.15.12:0/1344754847 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de91be3750 0x55de91be92b0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f80b0006470 tx=0x7f80b0038000 comp rx=0 tx=0).stop 2023-10-05T19:40:50.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.087+0000 7f80c7e14e40 1 -- 172.21.15.12:0/1344754847 shutdown_connections 2023-10-05T19:40:50.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.087+0000 7f80c7e14e40 1 --2- 172.21.15.12:0/1344754847 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f800c030c50 0x7f800c033100 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:50.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.087+0000 7f80c7e14e40 1 --2- 172.21.15.12:0/1344754847 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55de91be3750 0x55de91be92b0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:50.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.087+0000 7f80c7e14e40 1 -- 172.21.15.12:0/1344754847 >> 172.21.15.12:0/1344754847 conn(0x55de91bdeb10 msgr2=0x55de91bdcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:50.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.087+0000 7f80c7e14e40 1 -- 172.21.15.12:0/1344754847 shutdown_connections 2023-10-05T19:40:50.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.087+0000 7f80c7e14e40 1 -- 172.21.15.12:0/1344754847 wait complete. 2023-10-05T19:40:50.102 INFO:teuthology.orchestra.run.smithi012.stdout:fooFoohttp://acs.amazonaws.com/groups/global/AllUsersREADfooFooFULL_CONTROL 2023-10-05T19:40:50.102 DEBUG:tasks.util.rgw: raw result: fooFoohttp://acs.amazonaws.com/groups/global/AllUsersREADfooFooFULL_CONTROL 2023-10-05T19:40:50.104 INFO:tasks.radosgw_admin:RL: get_acls myfoo foo bytes_out=0 bytes_in=609 failed=False 2023-10-05T19:40:50.111 INFO:tasks.radosgw_admin:RL: create_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2023-10-05T19:40:50.178 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=5 bytes_in=0 failed=False 2023-10-05T19:40:50.179 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=4 bytes_in=0 failed=False 2023-10-05T19:40:50.179 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=3 bytes_in=0 failed=False 2023-10-05T19:40:50.179 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=6 bytes_in=0 failed=False 2023-10-05T19:40:50.179 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'rm', '--bucket', 'myfoo', '--purge-objects'] 2023-10-05T19:40:50.179 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'rm', '--bucket', 'myfoo', '--purge-objects'] 2023-10-05T19:40:50.180 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket rm --bucket myfoo --purge-objects 2023-10-05T19:40:50.220 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:50.220 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:50.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.215+0000 7f505c998e40 1 Processor -- start 2023-10-05T19:40:50.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.215+0000 7f505c998e40 1 -- start start 2023-10-05T19:40:50.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.215+0000 7f505c998e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x560ed4f35420 0x560ed4f37800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:50.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.215+0000 7f505c998e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x560ed4e7dbd0 con 0x560ed4f30e40 2023-10-05T19:40:50.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.215+0000 7f505c998e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x560ed4e3aee0 con 0x560ed4f35420 2023-10-05T19:40:50.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.215+0000 7f5051ccc700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x560ed4f35420 0x560ed4f37800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:50.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.215+0000 7f5051ccc700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x560ed4f35420 0x560ed4f37800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58570/0 (socket says 172.21.15.12:58570) 2023-10-05T19:40:50.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.215+0000 7f5051ccc700 1 -- 172.21.15.12:0/4104908486 learned_addr learned my addr 172.21.15.12:0/4104908486 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:50.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.215+0000 7f505a4cd700 1 --1- 172.21.15.12:0/4104908486 >> v1:172.21.15.8:6789/0 conn(0x560ed4f30e40 0x560ed4f31210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:60248/0 (socket says 172.21.15.12:60248) 2023-10-05T19:40:50.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.215+0000 7f5051ccc700 1 -- 172.21.15.12:0/4104908486 >> v1:172.21.15.8:6789/0 conn(0x560ed4f30e40 legacy=0x560ed4f31210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.215+0000 7f5051ccc700 1 -- 172.21.15.12:0/4104908486 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ed4f31a50 con 0x560ed4f35420 2023-10-05T19:40:50.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.215+0000 7f5051ccc700 1 --2- 172.21.15.12:0/4104908486 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560ed4f35420 0x560ed4f37800 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f504400a650 tx=0x7f5044005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=563b5739cacb43d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:50.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.215+0000 7f5059ccc700 1 -- 172.21.15.12:0/4104908486 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5044008760 con 0x560ed4f35420 2023-10-05T19:40:50.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.215+0000 7f5059ccc700 1 -- 172.21.15.12:0/4104908486 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50440088c0 con 0x560ed4f35420 2023-10-05T19:40:50.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.215+0000 7f5059ccc700 1 -- 172.21.15.12:0/4104908486 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5044008bb0 con 0x560ed4f35420 2023-10-05T19:40:50.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.215+0000 7f505c998e40 1 -- 172.21.15.12:0/4104908486 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560ed4f35420 msgr2=0x560ed4f37800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.215+0000 7f505c998e40 1 --2- 172.21.15.12:0/4104908486 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560ed4f35420 0x560ed4f37800 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f504400a650 tx=0x7f5044005b10 comp rx=0 tx=0).stop 2023-10-05T19:40:50.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.215+0000 7f505c998e40 1 -- 172.21.15.12:0/4104908486 shutdown_connections 2023-10-05T19:40:50.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.215+0000 7f505c998e40 1 --2- 172.21.15.12:0/4104908486 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560ed4f35420 0x560ed4f37800 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:50.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.215+0000 7f505c998e40 1 -- 172.21.15.12:0/4104908486 >> 172.21.15.12:0/4104908486 conn(0x560ed4f2c860 msgr2=0x560ed4f2ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:50.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.215+0000 7f505c998e40 1 -- 172.21.15.12:0/4104908486 shutdown_connections 2023-10-05T19:40:50.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.215+0000 7f505c998e40 1 -- 172.21.15.12:0/4104908486 wait complete. 2023-10-05T19:40:50.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.219+0000 7f505c998e40 1 Processor -- start 2023-10-05T19:40:50.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.219+0000 7f505c998e40 1 -- start start 2023-10-05T19:40:50.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.219+0000 7f505c998e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560ed4f30e40 0x560ed4ea68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:50.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.219+0000 7f505c998e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ed4dfee30 con 0x560ed4f30e40 2023-10-05T19:40:50.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.219+0000 7f505a4cd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560ed4f30e40 0x560ed4ea68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:50.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.219+0000 7f505a4cd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560ed4f30e40 0x560ed4ea68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58582/0 (socket says 172.21.15.12:58582) 2023-10-05T19:40:50.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.219+0000 7f505a4cd700 1 -- 172.21.15.12:0/2548245870 learned_addr learned my addr 172.21.15.12:0/2548245870 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:50.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.219+0000 7f505a4cd700 1 -- 172.21.15.12:0/2548245870 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5044005db0 con 0x560ed4f30e40 2023-10-05T19:40:50.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.219+0000 7f505a4cd700 1 --2- 172.21.15.12:0/2548245870 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560ed4f30e40 0x560ed4ea68f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f504c001cb0 tx=0x7f504c001860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3dc2857864b2510 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:50.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.219+0000 7f5058cca700 1 -- 172.21.15.12:0/2548245870 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f504c014070 con 0x560ed4f30e40 2023-10-05T19:40:50.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.219+0000 7f5058cca700 1 -- 172.21.15.12:0/2548245870 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f504c00bd40 con 0x560ed4f30e40 2023-10-05T19:40:50.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.219+0000 7f5058cca700 1 -- 172.21.15.12:0/2548245870 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f504c00f450 con 0x560ed4f30e40 2023-10-05T19:40:50.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.223+0000 7f505c998e40 1 -- 172.21.15.12:0/2548245870 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560ed4f30e40 msgr2=0x560ed4ea68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.223+0000 7f505c998e40 1 --2- 172.21.15.12:0/2548245870 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560ed4f30e40 0x560ed4ea68f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f504c001cb0 tx=0x7f504c001860 comp rx=0 tx=0).stop 2023-10-05T19:40:50.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.223+0000 7f505c998e40 1 -- 172.21.15.12:0/2548245870 shutdown_connections 2023-10-05T19:40:50.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.223+0000 7f505c998e40 1 --2- 172.21.15.12:0/2548245870 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560ed4f30e40 0x560ed4ea68f0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:50.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.223+0000 7f505c998e40 1 -- 172.21.15.12:0/2548245870 >> 172.21.15.12:0/2548245870 conn(0x560ed4f2c860 msgr2=0x560ed4f4bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:50.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.223+0000 7f505c998e40 1 -- 172.21.15.12:0/2548245870 shutdown_connections 2023-10-05T19:40:50.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.223+0000 7f505c998e40 1 -- 172.21.15.12:0/2548245870 wait complete. 2023-10-05T19:40:50.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.223+0000 7f505c998e40 1 Processor -- start 2023-10-05T19:40:50.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.223+0000 7f505c998e40 1 -- start start 2023-10-05T19:40:50.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.223+0000 7f505c998e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560ed4f30e40 0x560ed4f58ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:50.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.223+0000 7f505c998e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ed4dd1ed0 con 0x560ed4f30e40 2023-10-05T19:40:50.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.223+0000 7f505a4cd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560ed4f30e40 0x560ed4f58ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:50.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.223+0000 7f505a4cd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560ed4f30e40 0x560ed4f58ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58590/0 (socket says 172.21.15.12:58590) 2023-10-05T19:40:50.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.223+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 learned_addr learned my addr 172.21.15.12:0/1789209205 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:50.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.223+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ed4e7dbd0 con 0x560ed4f30e40 2023-10-05T19:40:50.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.223+0000 7f505a4cd700 1 --2- 172.21.15.12:0/1789209205 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560ed4f30e40 0x560ed4f58ee0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f504c000c00 tx=0x7f504c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:50.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.223+0000 7f5052ffd700 1 -- 172.21.15.12:0/1789209205 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f504c00beb0 con 0x560ed4f30e40 2023-10-05T19:40:50.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.223+0000 7f5052ffd700 1 -- 172.21.15.12:0/1789209205 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f504c00f450 con 0x560ed4f30e40 2023-10-05T19:40:50.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.223+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560ed4f59420 con 0x560ed4f30e40 2023-10-05T19:40:50.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.223+0000 7f5052ffd700 1 -- 172.21.15.12:0/1789209205 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f504c01cc80 con 0x560ed4f30e40 2023-10-05T19:40:50.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.223+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560ed4f59860 con 0x560ed4f30e40 2023-10-05T19:40:50.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.227+0000 7f5052ffd700 1 -- 172.21.15.12:0/1789209205 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f504c023020 con 0x560ed4f30e40 2023-10-05T19:40:50.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.227+0000 7f505c998e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.227+0000 7f5052ffd700 1 --2- 172.21.15.12:0/1789209205 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f503c030c50 0x7f503c033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:50.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.227+0000 7f5052ffd700 1 -- 172.21.15.12:0/1789209205 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7f504c008030 con 0x560ed4f30e40 2023-10-05T19:40:50.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.227+0000 7f5051ccc700 1 --2- 172.21.15.12:0/1789209205 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f503c030c50 0x7f503c033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:50.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.227+0000 7f505c998e40 1 --2- 172.21.15.12:0/1789209205 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x560ed4f35420 0x560ed4f649d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:50.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.227+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f65080 con 0x560ed4f35420 2023-10-05T19:40:50.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.227+0000 7f5051ccc700 1 --2- 172.21.15.12:0/1789209205 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f503c030c50 0x7f503c033100 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f50440030a0 tx=0x7f5044015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:50.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.227+0000 7f505acce700 1 --2- 172.21.15.12:0/1789209205 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x560ed4f35420 0x560ed4f649d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:50.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.227+0000 7f505acce700 1 --2- 172.21.15.12:0/1789209205 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x560ed4f35420 0x560ed4f649d0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:50.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.227+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.227+0000 7f505c998e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:50.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.227+0000 7f505c998e40 20 realm 2023-10-05T19:40:50.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.227+0000 7f505c998e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.227+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f61ff0 con 0x560ed4f35420 2023-10-05T19:40:50.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.227+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.231+0000 7f505c998e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:50.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.231+0000 7f505c998e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:50.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.231+0000 7f505c998e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.231+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f65b60 con 0x560ed4f35420 2023-10-05T19:40:50.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.231+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.231+0000 7f505c998e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:50.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.231+0000 7f505c998e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.231+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f65740 con 0x560ed4f35420 2023-10-05T19:40:50.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.231+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.231+0000 7f505c998e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:50.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.231+0000 7f505c998e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.231+0000 7f505c998e40 1 --2- 172.21.15.12:0/1789209205 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x560ed4f66c20 0x560ed4f69000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:50.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.231+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f696a0 con 0x560ed4f66c20 2023-10-05T19:40:50.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.231+0000 7f505a4cd700 1 --2- 172.21.15.12:0/1789209205 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x560ed4f66c20 0x560ed4f69000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:50.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.231+0000 7f505a4cd700 1 --2- 172.21.15.12:0/1789209205 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x560ed4f66c20 0x560ed4f69000 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:50.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.231+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f504c016070 con 0x560ed4f66c20 2023-10-05T19:40:50.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.231+0000 7f505c998e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:50.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.231+0000 7f505c998e40 20 searching for the correct realm 2023-10-05T19:40:50.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.231+0000 7f505c998e40 1 --2- 172.21.15.12:0/1789209205 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x560ed4f6b5e0 0x560ed4f6d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:50.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.231+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6e0a0 con 0x560ed4f6b5e0 2023-10-05T19:40:50.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.231+0000 7f5051ccc700 1 --2- 172.21.15.12:0/1789209205 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x560ed4f6b5e0 0x560ed4f6d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:50.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.231+0000 7f5051ccc700 1 --2- 172.21.15.12:0/1789209205 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x560ed4f6b5e0 0x560ed4f6d9f0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:50.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.235+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.235+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6e0a0 con 0x560ed4f66c20 2023-10-05T19:40:50.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.235+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f504c016070 con 0x560ed4f66c20 2023-10-05T19:40:50.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.235+0000 7f505a4cd700 1 --2- 172.21.15.12:0/1789209205 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f504c01a070 0x7f504c042220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:50.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.235+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6e0a0 con 0x7f504c01a070 2023-10-05T19:40:50.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.235+0000 7f505acce700 1 --2- 172.21.15.12:0/1789209205 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f504c01a070 0x7f504c042220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:50.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.235+0000 7f505acce700 1 --2- 172.21.15.12:0/1789209205 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f504c01a070 0x7f504c042220 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:50.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.235+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f505400a5a0 con 0x7f504c01a070 2023-10-05T19:40:50.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.235+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6e0a0 con 0x560ed4f35420 2023-10-05T19:40:50.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.235+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.235+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6e0a0 con 0x560ed4f35420 2023-10-05T19:40:50.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.235+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.235+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6e0a0 con 0x560ed4f35420 2023-10-05T19:40:50.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.239+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(11 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.239+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6e0a0 con 0x7f504c01a070 2023-10-05T19:40:50.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.239+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f505400a5a0 con 0x7f504c01a070 2023-10-05T19:40:50.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.239+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6e0a0 con 0x560ed4f66c20 2023-10-05T19:40:50.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.239+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f504c016070 con 0x560ed4f66c20 2023-10-05T19:40:50.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.239+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6e0a0 con 0x560ed4f66c20 2023-10-05T19:40:50.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.239+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f504c016070 con 0x560ed4f66c20 2023-10-05T19:40:50.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.239+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6e0a0 con 0x560ed4f35420 2023-10-05T19:40:50.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.239+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.239+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6e0a0 con 0x7f504c01a070 2023-10-05T19:40:50.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.239+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f505400a5a0 con 0x7f504c01a070 2023-10-05T19:40:50.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.239+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6e0a0 con 0x560ed4f35420 2023-10-05T19:40:50.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.243+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.243+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6e0a0 con 0x560ed4f66c20 2023-10-05T19:40:50.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.243+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(18 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f66c20 2023-10-05T19:40:50.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.243+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6e0a0 con 0x560ed4f6b5e0 2023-10-05T19:40:50.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.243+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.243+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6e0a0 con 0x560ed4f6b5e0 2023-10-05T19:40:50.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.243+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.243+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6e0a0 con 0x7f504c01a070 2023-10-05T19:40:50.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.243+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f505400a5a0 con 0x7f504c01a070 2023-10-05T19:40:50.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.243+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6e0a0 con 0x560ed4f6b5e0 2023-10-05T19:40:50.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.247+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.247+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6e0a0 con 0x560ed4f6b5e0 2023-10-05T19:40:50.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.247+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.247+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6e0a0 con 0x560ed4f35420 2023-10-05T19:40:50.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.247+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.247+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6e0a0 con 0x560ed4f35420 2023-10-05T19:40:50.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.247+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.247+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6e0a0 con 0x7f504c01a070 2023-10-05T19:40:50.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.247+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f505400a5a0 con 0x7f504c01a070 2023-10-05T19:40:50.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.247+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6e0a0 con 0x560ed4f6b5e0 2023-10-05T19:40:50.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.247+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.247+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6e0a0 con 0x560ed4f66c20 2023-10-05T19:40:50.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.251+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f66c20 2023-10-05T19:40:50.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.251+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6e0a0 con 0x560ed4f35420 2023-10-05T19:40:50.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.251+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.251+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6e0a0 con 0x7f504c01a070 2023-10-05T19:40:50.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.251+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f505400a5a0 con 0x7f504c01a070 2023-10-05T19:40:50.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.251+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6e0a0 con 0x560ed4f6b5e0 2023-10-05T19:40:50.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.251+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.251+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6e0a0 con 0x7f504c01a070 2023-10-05T19:40:50.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.251+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f505400a5a0 con 0x7f504c01a070 2023-10-05T19:40:50.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.251+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6e0a0 con 0x560ed4f66c20 2023-10-05T19:40:50.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.255+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f66c20 2023-10-05T19:40:50.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.255+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6e0a0 con 0x560ed4f6b5e0 2023-10-05T19:40:50.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.255+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.255+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6e0a0 con 0x7f504c01a070 2023-10-05T19:40:50.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.255+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f505400a5a0 con 0x7f504c01a070 2023-10-05T19:40:50.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.255+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6e0a0 con 0x560ed4f35420 2023-10-05T19:40:50.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.255+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.255+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6e0a0 con 0x560ed4f66c20 2023-10-05T19:40:50.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.255+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(37 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f66c20 2023-10-05T19:40:50.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.255+0000 7f505c998e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:50.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.255+0000 7f505c998e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:50.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.255+0000 7f505c998e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:50.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.255+0000 7f505c998e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:50.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.259+0000 7f505c998e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.259+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6a360 con 0x560ed4f35420 2023-10-05T19:40:50.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.259+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.259+0000 7f505c998e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:50.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.259+0000 7f505c998e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.259+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f69f90 con 0x560ed4f35420 2023-10-05T19:40:50.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.259+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.259+0000 7f505c998e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:50.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.259+0000 7f505c998e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.259+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6e510 con 0x560ed4f6b5e0 2023-10-05T19:40:50.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.259+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.259+0000 7f505c998e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:50.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.259+0000 7f505c998e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.259+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6e8c0 con 0x560ed4f6b5e0 2023-10-05T19:40:50.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.259+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.259+0000 7f505c998e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:50.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.259+0000 7f505c998e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.259+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6ec70 con 0x560ed4f35420 2023-10-05T19:40:50.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.259+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.259+0000 7f505c998e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:50.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.259+0000 7f505c998e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.259+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6f080 con 0x560ed4f35420 2023-10-05T19:40:50.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.263+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.263+0000 7f505c998e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:50.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.263+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6f050 con 0x560ed4f6b5e0 2023-10-05T19:40:50.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.263+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(44 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.263+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6f050 con 0x560ed4f66c20 2023-10-05T19:40:50.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.263+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(45 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f66c20 2023-10-05T19:40:50.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.263+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f504c046040 con 0x7f504c01a070 2023-10-05T19:40:50.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.267+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(46 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f505400a5a0 con 0x7f504c01a070 2023-10-05T19:40:50.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.267+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6ec70 con 0x560ed4f35420 2023-10-05T19:40:50.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.267+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(47 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.267+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6ec70 con 0x560ed4f35420 2023-10-05T19:40:50.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.267+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(48 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.267+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6ec70 con 0x560ed4f35420 2023-10-05T19:40:50.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.267+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(49 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.267+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6ec70 con 0x7f504c01a070 2023-10-05T19:40:50.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.267+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(50 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f505400a5a0 con 0x7f504c01a070 2023-10-05T19:40:50.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.267+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6ec70 con 0x560ed4f66c20 2023-10-05T19:40:50.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.271+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(51 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f66c20 2023-10-05T19:40:50.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.271+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6ec70 con 0x560ed4f66c20 2023-10-05T19:40:50.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.271+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(52 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f66c20 2023-10-05T19:40:50.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.271+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6ec70 con 0x560ed4f35420 2023-10-05T19:40:50.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.271+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(53 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.271+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6ec70 con 0x7f504c01a070 2023-10-05T19:40:50.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.271+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(54 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f505400a5a0 con 0x7f504c01a070 2023-10-05T19:40:50.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.271+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6ec70 con 0x560ed4f35420 2023-10-05T19:40:50.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.271+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(55 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.271+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6ec70 con 0x560ed4f66c20 2023-10-05T19:40:50.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.271+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(56 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f66c20 2023-10-05T19:40:50.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.271+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6ec70 con 0x560ed4f6b5e0 2023-10-05T19:40:50.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.275+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(57 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.275+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6ec70 con 0x560ed4f6b5e0 2023-10-05T19:40:50.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.275+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(58 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.275+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6ec70 con 0x7f504c01a070 2023-10-05T19:40:50.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.275+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(59 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f505400a5a0 con 0x7f504c01a070 2023-10-05T19:40:50.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.275+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6ec70 con 0x560ed4f6b5e0 2023-10-05T19:40:50.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.275+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(60 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.275+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6ec70 con 0x560ed4f6b5e0 2023-10-05T19:40:50.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.275+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(61 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.275+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6ec70 con 0x560ed4f35420 2023-10-05T19:40:50.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.279+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(62 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.279+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6ec70 con 0x560ed4f35420 2023-10-05T19:40:50.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.279+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(63 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.279+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6ec70 con 0x7f504c01a070 2023-10-05T19:40:50.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.279+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(64 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f505400a5a0 con 0x7f504c01a070 2023-10-05T19:40:50.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.279+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6ec70 con 0x560ed4f6b5e0 2023-10-05T19:40:50.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.279+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(65 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.279+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6ec70 con 0x560ed4f66c20 2023-10-05T19:40:50.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.279+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(66 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f66c20 2023-10-05T19:40:50.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.279+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6ec70 con 0x560ed4f35420 2023-10-05T19:40:50.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.283+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(67 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.283+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6ec70 con 0x7f504c01a070 2023-10-05T19:40:50.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.283+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(68 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f505400a5a0 con 0x7f504c01a070 2023-10-05T19:40:50.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.283+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6ec70 con 0x560ed4f6b5e0 2023-10-05T19:40:50.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.283+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(69 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.283+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6ec70 con 0x7f504c01a070 2023-10-05T19:40:50.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.283+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(70 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f505400a5a0 con 0x7f504c01a070 2023-10-05T19:40:50.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.283+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6ec70 con 0x560ed4f66c20 2023-10-05T19:40:50.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.283+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(71 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f66c20 2023-10-05T19:40:50.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.283+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6ec70 con 0x560ed4f6b5e0 2023-10-05T19:40:50.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.287+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(72 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.287+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6ec70 con 0x7f504c01a070 2023-10-05T19:40:50.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.287+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(73 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f505400a5a0 con 0x7f504c01a070 2023-10-05T19:40:50.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.287+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6ec70 con 0x560ed4f35420 2023-10-05T19:40:50.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.287+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(74 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.287+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6ec70 con 0x560ed4f66c20 2023-10-05T19:40:50.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.287+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(75 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f66c20 2023-10-05T19:40:50.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.287+0000 7f505c998e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:50.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.287+0000 7f505c998e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:50.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.287+0000 7f505c998e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:50.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.287+0000 7f505c998e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:50.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.287+0000 7f505c998e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.287+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6f4b0 con 0x560ed4f35420 2023-10-05T19:40:50.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.287+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.287+0000 7f505c998e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:50.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.287+0000 7f505c998e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.287+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f6f860 con 0x560ed4f66c20 2023-10-05T19:40:50.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.291+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f66c20 2023-10-05T19:40:50.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.291+0000 7f505c998e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:50.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.291+0000 7f505c998e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.291+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f70100 con 0x560ed4f35420 2023-10-05T19:40:50.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.291+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.291+0000 7f505c998e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:50.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.291+0000 7f505c998e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.291+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f70100 con 0x560ed4f6b5e0 2023-10-05T19:40:50.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.291+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.291+0000 7f505c998e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:50.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.291+0000 7f505c998e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.291+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f709a0 con 0x7f504c01a070 2023-10-05T19:40:50.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.291+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f505400a5a0 con 0x7f504c01a070 2023-10-05T19:40:50.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.291+0000 7f505c998e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:50.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.291+0000 7f505c998e40 20 zone default found 2023-10-05T19:40:50.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.291+0000 7f505c998e40 4 Realm: () 2023-10-05T19:40:50.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.291+0000 7f505c998e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:50.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.291+0000 7f505c998e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:50.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.291+0000 7f505c998e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:50.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.291+0000 7f505c998e40 20 zonegroup default 2023-10-05T19:40:50.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.291+0000 7f505c998e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.291+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f709a0 con 0x7f504c01a070 2023-10-05T19:40:50.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.291+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f505400a5a0 con 0x7f504c01a070 2023-10-05T19:40:50.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.291+0000 7f505c998e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:50.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.291+0000 7f505c998e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.291+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f72320 con 0x560ed4f6b5e0 2023-10-05T19:40:50.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.295+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.295+0000 7f505c998e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:50.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.295+0000 7f505c998e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.295+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f726d0 con 0x560ed4f6b5e0 2023-10-05T19:40:50.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.295+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.295+0000 7f505c998e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:50.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.295+0000 7f505c998e40 20 started sync module instance, tier type = 2023-10-05T19:40:50.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.295+0000 7f505c998e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:50.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.295+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f719e0 con 0x560ed4f66c20 2023-10-05T19:40:50.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.295+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(84 notify.0 [create] v30'207 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f66c20 2023-10-05T19:40:50.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.295+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94621702493424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f72a80 con 0x560ed4f66c20 2023-10-05T19:40:50.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.299+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f72a80 con 0x560ed4f35420 2023-10-05T19:40:50.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.299+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(85 notify.0 [watch watch cookie 94621702493424] v30'208 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f66c20 2023-10-05T19:40:50.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.299+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(86 notify.1 [create] v30'207 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.299+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94621702498944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f71eb0 con 0x560ed4f35420 2023-10-05T19:40:50.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.299+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f73ad0 con 0x560ed4f6b5e0 2023-10-05T19:40:50.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.303+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(87 notify.1 [watch watch cookie 94621702498944] v30'208 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.303+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(88 notify.2 [create] v30'207 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.303+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94621702501856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f73ed0 con 0x560ed4f6b5e0 2023-10-05T19:40:50.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.303+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f74770 con 0x560ed4f66c20 2023-10-05T19:40:50.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.307+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(90 notify.3 [create] v30'207 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f66c20 2023-10-05T19:40:50.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.307+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94621702505376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f74da0 con 0x560ed4f66c20 2023-10-05T19:40:50.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.307+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f75960 con 0x560ed4f35420 2023-10-05T19:40:50.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.307+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(89 notify.2 [watch watch cookie 94621702501856] v30'208 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.307+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(91 notify.3 [watch watch cookie 94621702505376] v30'208 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f66c20 2023-10-05T19:40:50.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.307+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(92 notify.4 [create] v30'207 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.307+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94621702500048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f75430 con 0x560ed4f35420 2023-10-05T19:40:50.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.307+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f76250 con 0x7f504c01a070 2023-10-05T19:40:50.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.311+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(93 notify.4 [watch watch cookie 94621702500048] v30'208 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.311+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(94 notify.5 [create] v30'207 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f505400a5a0 con 0x7f504c01a070 2023-10-05T19:40:50.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.311+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94621702506544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f75fb0 con 0x7f504c01a070 2023-10-05T19:40:50.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.311+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f76850 con 0x7f504c01a070 2023-10-05T19:40:50.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.315+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(96 notify.6 [create] v30'207 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f505400a5a0 con 0x7f504c01a070 2023-10-05T19:40:50.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.315+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94621702512832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f76b50 con 0x7f504c01a070 2023-10-05T19:40:50.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.315+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f776b0 con 0x7f504c01a070 2023-10-05T19:40:50.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.315+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(95 notify.5 [watch watch cookie 94621702506544] v30'208 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f505400a5a0 con 0x7f504c01a070 2023-10-05T19:40:50.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.315+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(97 notify.6 [watch watch cookie 94621702512832] v30'208 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f505400a5a0 con 0x7f504c01a070 2023-10-05T19:40:50.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.315+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(98 notify.7 [create] v30'207 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f505400a5a0 con 0x7f504c01a070 2023-10-05T19:40:50.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.315+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94621702511296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f77180 con 0x7f504c01a070 2023-10-05T19:40:50.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.315+0000 7f505c998e40 20 add_watcher() i=0 2023-10-05T19:40:50.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.315+0000 7f505c998e40 20 add_watcher() i=1 2023-10-05T19:40:50.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.315+0000 7f505c998e40 20 add_watcher() i=2 2023-10-05T19:40:50.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.315+0000 7f505c998e40 20 add_watcher() i=3 2023-10-05T19:40:50.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.315+0000 7f505c998e40 20 add_watcher() i=4 2023-10-05T19:40:50.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.315+0000 7f505c998e40 20 add_watcher() i=5 2023-10-05T19:40:50.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.315+0000 7f505c998e40 20 add_watcher() i=6 2023-10-05T19:40:50.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.319+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(99 notify.7 [watch watch cookie 94621702511296] v30'208 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f505400a5a0 con 0x7f504c01a070 2023-10-05T19:40:50.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.319+0000 7f505c998e40 20 add_watcher() i=7 2023-10-05T19:40:50.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.319+0000 7f505c998e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:50.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.319+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f7a6a0 con 0x560ed4f35420 2023-10-05T19:40:50.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.319+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.319+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94621702517120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f77ac0 con 0x560ed4f35420 2023-10-05T19:40:50.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.323+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94621702517120] v30'203 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.323+0000 7f502bfdf700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:50.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.323+0000 7f505c998e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:50.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.323+0000 7f505c998e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:50.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.323+0000 7f505c998e40 1 Processor -- start 2023-10-05T19:40:50.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.323+0000 7f505c998e40 1 -- start start 2023-10-05T19:40:50.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.323+0000 7f505c998e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560ed4f84b40 0x560ed4f84f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:50.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.323+0000 7f505c998e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ed4f85450 con 0x560ed4f84b40 2023-10-05T19:40:50.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.323+0000 7f5051ccc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560ed4f84b40 0x560ed4f84f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:50.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.323+0000 7f5051ccc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560ed4f84b40 0x560ed4f84f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58604/0 (socket says 172.21.15.12:58604) 2023-10-05T19:40:50.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.323+0000 7f5051ccc700 1 -- 172.21.15.12:0/1642977086 learned_addr learned my addr 172.21.15.12:0/1642977086 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:50.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.323+0000 7f5051ccc700 1 -- 172.21.15.12:0/1642977086 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ed4f85590 con 0x560ed4f84b40 2023-10-05T19:40:50.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.323+0000 7f5051ccc700 1 --2- 172.21.15.12:0/1642977086 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560ed4f84b40 0x560ed4f84f10 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5044002000 tx=0x7f5044012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14eed82045a37205 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:50.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.323+0000 7f502b7de700 1 -- 172.21.15.12:0/1642977086 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5044030930 con 0x560ed4f84b40 2023-10-05T19:40:50.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.323+0000 7f502b7de700 1 -- 172.21.15.12:0/1642977086 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5044030a90 con 0x560ed4f84b40 2023-10-05T19:40:50.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.323+0000 7f502b7de700 1 -- 172.21.15.12:0/1642977086 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5044030c40 con 0x560ed4f84b40 2023-10-05T19:40:50.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.327+0000 7f505c998e40 1 -- 172.21.15.12:0/1642977086 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560ed4f84b40 msgr2=0x560ed4f84f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.327+0000 7f505c998e40 1 --2- 172.21.15.12:0/1642977086 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560ed4f84b40 0x560ed4f84f10 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5044002000 tx=0x7f5044012000 comp rx=0 tx=0).stop 2023-10-05T19:40:50.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.327+0000 7f505c998e40 1 -- 172.21.15.12:0/1642977086 shutdown_connections 2023-10-05T19:40:50.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.327+0000 7f505c998e40 1 --2- 172.21.15.12:0/1642977086 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560ed4f84b40 0x560ed4f84f10 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:50.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.327+0000 7f505c998e40 1 -- 172.21.15.12:0/1642977086 >> 172.21.15.12:0/1642977086 conn(0x560ed4f7ff00 msgr2=0x560ed4f7e050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:50.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.327+0000 7f505c998e40 1 -- 172.21.15.12:0/1642977086 shutdown_connections 2023-10-05T19:40:50.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.327+0000 7f505c998e40 1 -- 172.21.15.12:0/1642977086 wait complete. 2023-10-05T19:40:50.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.327+0000 7f505c998e40 1 Processor -- start 2023-10-05T19:40:50.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.327+0000 7f505c998e40 1 -- start start 2023-10-05T19:40:50.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.327+0000 7f505c998e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560ed4f84b40 0x560ed4f8a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:50.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.327+0000 7f505c998e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ed4f8b5e0 con 0x560ed4f84b40 2023-10-05T19:40:50.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.327+0000 7f5051ccc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560ed4f84b40 0x560ed4f8a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:50.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.327+0000 7f5051ccc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560ed4f84b40 0x560ed4f8a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58618/0 (socket says 172.21.15.12:58618) 2023-10-05T19:40:50.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.327+0000 7f5051ccc700 1 -- 172.21.15.12:0/2961973426 learned_addr learned my addr 172.21.15.12:0/2961973426 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:50.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.327+0000 7f5051ccc700 1 -- 172.21.15.12:0/2961973426 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ed4f31a50 con 0x560ed4f84b40 2023-10-05T19:40:50.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.327+0000 7f5051ccc700 1 --2- 172.21.15.12:0/2961973426 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560ed4f84b40 0x560ed4f8a5e0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5044008fd0 tx=0x7f5044033000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:50.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.327+0000 7f5029fdb700 1 -- 172.21.15.12:0/2961973426 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5044027400 con 0x560ed4f84b40 2023-10-05T19:40:50.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.327+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560ed4f8abe0 con 0x560ed4f84b40 2023-10-05T19:40:50.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.327+0000 7f5029fdb700 1 -- 172.21.15.12:0/2961973426 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5044027580 con 0x560ed4f84b40 2023-10-05T19:40:50.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.327+0000 7f5029fdb700 1 -- 172.21.15.12:0/2961973426 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5044027730 con 0x560ed4f84b40 2023-10-05T19:40:50.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.327+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560ed4f8b020 con 0x560ed4f84b40 2023-10-05T19:40:50.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.331+0000 7f5029fdb700 1 -- 172.21.15.12:0/2961973426 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f5044036040 con 0x560ed4f84b40 2023-10-05T19:40:50.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.331+0000 7f5029fdb700 1 --2- 172.21.15.12:0/2961973426 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f4fa0030c50 0x7f4fa0033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:50.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.331+0000 7f5029fdb700 1 -- 172.21.15.12:0/2961973426 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7f5044055c10 con 0x560ed4f84b40 2023-10-05T19:40:50.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.331+0000 7f505acce700 1 --2- 172.21.15.12:0/2961973426 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f4fa0030c50 0x7f4fa0033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:50.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.331+0000 7f505c998e40 1 --2- 172.21.15.12:0/2961973426 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x560ed4f951d0 0x560ed4f975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:50.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.331+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f97c80 con 0x560ed4f951d0 2023-10-05T19:40:50.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.331+0000 7f505a4cd700 1 --2- 172.21.15.12:0/2961973426 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x560ed4f951d0 0x560ed4f975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:50.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.331+0000 7f505acce700 1 --2- 172.21.15.12:0/2961973426 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f4fa0030c50 0x7f4fa0033100 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x560ed4f76870 tx=0x7f505400af20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:50.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.331+0000 7f505a4cd700 1 --2- 172.21.15.12:0/2961973426 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x560ed4f951d0 0x560ed4f975e0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:50.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.335+0000 7f505a4cd700 1 -- 172.21.15.12:0/2961973426 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v30'33 uv30 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f951d0 2023-10-05T19:40:50.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.335+0000 7f505c998e40 1 --2- 172.21.15.12:0/2961973426 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x560ed4f98950 0x560ed4f9ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:50.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.335+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f9b460 con 0x560ed4f98950 2023-10-05T19:40:50.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.335+0000 7f5051ccc700 1 --2- 172.21.15.12:0/2961973426 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x560ed4f98950 0x560ed4f9ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:50.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.335+0000 7f5051ccc700 1 --2- 172.21.15.12:0/2961973426 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x560ed4f98950 0x560ed4f9ad30 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:50.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.339+0000 7f5051ccc700 1 -- 172.21.15.12:0/2961973426 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v30'24 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f98950 2023-10-05T19:40:50.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.339+0000 7f505c998e40 1 --2- 172.21.15.12:0/2961973426 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x560ed4f9c130 0x560ed4f9e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:50.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.339+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f9ec70 con 0x560ed4f9c130 2023-10-05T19:40:50.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.339+0000 7f505acce700 1 --2- 172.21.15.12:0/2961973426 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x560ed4f9c130 0x560ed4f9e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:50.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.339+0000 7f505acce700 1 --2- 172.21.15.12:0/2961973426 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x560ed4f9c130 0x560ed4f9e540 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:50.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.343+0000 7f505acce700 1 -- 172.21.15.12:0/2961973426 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v30'24 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f9c130 2023-10-05T19:40:50.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.343+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f9f2c0 con 0x560ed4f951d0 2023-10-05T19:40:50.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.343+0000 7f505a4cd700 1 -- 172.21.15.12:0/2961973426 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v30'204 uv185 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f951d0 2023-10-05T19:40:50.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.343+0000 7f505c998e40 1 --2- 172.21.15.12:0/2961973426 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x560ed4f9ff60 0x560ed4fa2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:50.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.343+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4fa29b0 con 0x560ed4f9ff60 2023-10-05T19:40:50.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.343+0000 7f505a4cd700 1 --2- 172.21.15.12:0/2961973426 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x560ed4f9ff60 0x560ed4fa2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:50.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.343+0000 7f505a4cd700 1 --2- 172.21.15.12:0/2961973426 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x560ed4f9ff60 0x560ed4fa2340 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:50.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.347+0000 7f505a4cd700 1 -- 172.21.15.12:0/2961973426 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v30'62 uv51 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f9ff60 2023-10-05T19:40:50.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.347+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4fa2fd0 con 0x560ed4f9c130 2023-10-05T19:40:50.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.347+0000 7f505acce700 1 -- 172.21.15.12:0/2961973426 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v30'44 uv39 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f9c130 2023-10-05T19:40:50.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.347+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4fa35f0 con 0x560ed4f951d0 2023-10-05T19:40:50.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.355+0000 7f505a4cd700 1 -- 172.21.15.12:0/2961973426 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v30'26 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f951d0 2023-10-05T19:40:50.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.355+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4fa35f0 con 0x560ed4f98950 2023-10-05T19:40:50.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.355+0000 7f5051ccc700 1 -- 172.21.15.12:0/2961973426 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v30'41 uv36 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f98950 2023-10-05T19:40:50.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.355+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4fa3c10 con 0x560ed4f9ff60 2023-10-05T19:40:50.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.359+0000 7f505a4cd700 1 -- 172.21.15.12:0/2961973426 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v30'24 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f9ff60 2023-10-05T19:40:50.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.359+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4fa3c10 con 0x560ed4f9c130 2023-10-05T19:40:50.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.359+0000 7f505acce700 1 -- 172.21.15.12:0/2961973426 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v30'30 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f9c130 2023-10-05T19:40:50.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.359+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4fa4100 con 0x560ed4f9ff60 2023-10-05T19:40:50.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.363+0000 7f505a4cd700 1 -- 172.21.15.12:0/2961973426 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v30'63 uv53 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f9ff60 2023-10-05T19:40:50.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.363+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4fa4100 con 0x560ed4f98950 2023-10-05T19:40:50.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.367+0000 7f5051ccc700 1 -- 172.21.15.12:0/2961973426 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v30'27 uv24 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f98950 2023-10-05T19:40:50.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.367+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4fa4720 con 0x560ed4f9ff60 2023-10-05T19:40:50.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.367+0000 7f505a4cd700 1 -- 172.21.15.12:0/2961973426 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v30'35 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f9ff60 2023-10-05T19:40:50.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.367+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4fa4c10 con 0x560ed4f98950 2023-10-05T19:40:50.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.371+0000 7f5051ccc700 1 -- 172.21.15.12:0/2961973426 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v30'50 uv45 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f98950 2023-10-05T19:40:50.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.371+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4fa4c10 con 0x560ed4f951d0 2023-10-05T19:40:50.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.375+0000 7f505a4cd700 1 -- 172.21.15.12:0/2961973426 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v30'205 uv187 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f951d0 2023-10-05T19:40:50.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.375+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4fa4c10 con 0x560ed4f9c130 2023-10-05T19:40:50.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.375+0000 7f505acce700 1 -- 172.21.15.12:0/2961973426 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v30'59 uv49 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f9c130 2023-10-05T19:40:50.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.375+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4fa4c10 con 0x560ed4f9ff60 2023-10-05T19:40:50.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.379+0000 7f505a4cd700 1 -- 172.21.15.12:0/2961973426 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v30'64 uv55 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f9ff60 2023-10-05T19:40:50.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.379+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4fa4c10 con 0x560ed4f9c130 2023-10-05T19:40:50.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.383+0000 7f505acce700 1 -- 172.21.15.12:0/2961973426 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v30'24 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f9c130 2023-10-05T19:40:50.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.383+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4fa4c10 con 0x560ed4f951d0 2023-10-05T19:40:50.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.383+0000 7f505a4cd700 1 -- 172.21.15.12:0/2961973426 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v30'47 uv41 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f951d0 2023-10-05T19:40:50.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.383+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4fa4c10 con 0x560ed4f951d0 2023-10-05T19:40:50.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.387+0000 7f505a4cd700 1 -- 172.21.15.12:0/2961973426 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v30'206 uv183 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f951d0 2023-10-05T19:40:50.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.387+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4fa4c10 con 0x560ed4f9c130 2023-10-05T19:40:50.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.391+0000 7f505acce700 1 -- 172.21.15.12:0/2961973426 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v30'60 uv45 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f9c130 2023-10-05T19:40:50.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.391+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4fa4c10 con 0x560ed4f9ff60 2023-10-05T19:40:50.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.391+0000 7f505a4cd700 1 -- 172.21.15.12:0/2961973426 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v30'34 uv31 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f9ff60 2023-10-05T19:40:50.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.391+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4fa4c10 con 0x560ed4f9c130 2023-10-05T19:40:50.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.395+0000 7f505acce700 1 -- 172.21.15.12:0/2961973426 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v30'28 uv25 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f9c130 2023-10-05T19:40:50.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.395+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f90b20 con 0x560ed4f9c130 2023-10-05T19:40:50.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.395+0000 7f505acce700 1 -- 172.21.15.12:0/2961973426 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v30'33 uv30 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f9c130 2023-10-05T19:40:50.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.395+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f90b20 con 0x560ed4f9c130 2023-10-05T19:40:50.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.399+0000 7f505acce700 1 -- 172.21.15.12:0/2961973426 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v30'21 uv18 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f9c130 2023-10-05T19:40:50.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.399+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f90b20 con 0x560ed4f9c130 2023-10-05T19:40:50.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.399+0000 7f505acce700 1 -- 172.21.15.12:0/2961973426 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v30'45 uv37 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f9c130 2023-10-05T19:40:50.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.399+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f90b20 con 0x560ed4f98950 2023-10-05T19:40:50.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.403+0000 7f5051ccc700 1 -- 172.21.15.12:0/2961973426 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v30'30 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f98950 2023-10-05T19:40:50.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.403+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f90b20 con 0x560ed4f9c130 2023-10-05T19:40:50.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.403+0000 7f505acce700 1 -- 172.21.15.12:0/2961973426 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v30'24 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f9c130 2023-10-05T19:40:50.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.403+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f908b0 con 0x560ed4f98950 2023-10-05T19:40:50.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.407+0000 7f5051ccc700 1 -- 172.21.15.12:0/2961973426 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v30'42 uv34 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f98950 2023-10-05T19:40:50.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.407+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f908b0 con 0x560ed4f9c130 2023-10-05T19:40:50.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.407+0000 7f505acce700 1 -- 172.21.15.12:0/2961973426 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v30'61 uv47 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f9c130 2023-10-05T19:40:50.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.407+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f908b0 con 0x560ed4f98950 2023-10-05T19:40:50.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.411+0000 7f5051ccc700 1 -- 172.21.15.12:0/2961973426 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v30'51 uv43 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f98950 2023-10-05T19:40:50.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.411+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed4f90b20 con 0x560ed4f9ff60 2023-10-05T19:40:50.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.415+0000 7f505a4cd700 1 -- 172.21.15.12:0/2961973426 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v30'30 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f9ff60 2023-10-05T19:40:50.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.415+0000 7f4f9affd700 20 reqs_thread_entry: start 2023-10-05T19:40:50.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.415+0000 7f505c998e40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:50.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.415+0000 7f4f98ff9700 20 reqs_thread_entry: start 2023-10-05T19:40:50.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.415+0000 7f505c998e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:50.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.415+0000 7f4f8affd700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:41:20 2023 2023-10-05T19:40:50.425 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:50.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.415+0000 7f4f8a7fc700 20 reqs_thread_entry: start 2023-10-05T19:40:50.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.415+0000 7f505c998e40 20 get_system_obj_state: rctx=0x7ffec71ae450 obj=default.rgw.meta:root:myfoo state=0x560ed4e0a0b0 s->prefetch_data=0 2023-10-05T19:40:50.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.415+0000 7f505c998e40 10 cache get: name=default.rgw.meta+root+myfoo : miss 2023-10-05T19:40:50.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.415+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:102 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5004030 con 0x560ed4f66c20 2023-10-05T19:40:50.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.419+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(102 myfoo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv33 ondisk = 0) v8 ==== 233+0+139 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f66c20 2023-10-05T19:40:50.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.419+0000 7f505c998e40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x16 2023-10-05T19:40:50.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.419+0000 7f505c998e40 10 adding default.rgw.meta+root+myfoo to cache LRU end 2023-10-05T19:40:50.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.419+0000 7f505c998e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:50.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.419+0000 7f505c998e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:50.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.419+0000 7f505c998e40 10 cache get: name=default.rgw.meta+root+myfoo : type miss (requested=0x11, cached=0x16) 2023-10-05T19:40:50.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.419+0000 7f505c998e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.419+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:103 8.9 8:91b71841:root::myfoo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5003f40 con 0x560ed4f66c20 2023-10-05T19:40:50.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.419+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(103 myfoo [call,call out=48b,read 0~165 out=165b] v0'0 uv33 ondisk = 0) v8 ==== 233+0+213 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f66c20 2023-10-05T19:40:50.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.419+0000 7f505c998e40 20 rados_obj.operate() r=0 bl.length=165 2023-10-05T19:40:50.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.419+0000 7f505c998e40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2023-10-05T19:40:50.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.419+0000 7f505c998e40 10 moving default.rgw.meta+root+myfoo to cache LRU end 2023-10-05T19:40:50.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.419+0000 7f505c998e40 20 get_system_obj_state: rctx=0x7ffec71ae450 obj=default.rgw.meta:root:.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5 state=0x560ed4de02b0 s->prefetch_data=0 2023-10-05T19:40:50.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.419+0000 7f505c998e40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5 : miss 2023-10-05T19:40:50.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.419+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:104 8.10 8:0a6cdd29:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5005ab0 con 0x560ed4f6b5e0 2023-10-05T19:40:50.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.419+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(104 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5 [call out=48b,getxattrs out=218b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 290+0+282 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.419+0000 7f505c998e40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5 info.flags=0x16 2023-10-05T19:40:50.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.419+0000 7f505c998e40 10 adding default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5 to cache LRU end 2023-10-05T19:40:50.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.419+0000 7f505c998e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:50.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.419+0000 7f505c998e40 10 updating xattr: name=user.rgw.acl bl.length()=123 2023-10-05T19:40:50.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.419+0000 7f505c998e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:50.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.419+0000 7f505c998e40 20 Read xattr: user.rgw.acl 2023-10-05T19:40:50.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.419+0000 7f505c998e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:50.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.419+0000 7f505c998e40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5 : type miss (requested=0x13, cached=0x16) 2023-10-05T19:40:50.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.419+0000 7f505c998e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.419+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:105 8.10 8:0a6cdd29:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5006640 con 0x560ed4f6b5e0 2023-10-05T19:40:50.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.419+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(105 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5 [call,call out=48b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 332+0+618 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.419+0000 7f505c998e40 20 rados_obj.operate() r=0 bl.length=352 2023-10-05T19:40:50.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.419+0000 7f505c998e40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5 info.flags=0x13 2023-10-05T19:40:50.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.419+0000 7f505c998e40 10 moving default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5 to cache LRU end 2023-10-05T19:40:50.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.419+0000 7f505c998e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:50.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.419+0000 7f505c998e40 10 updating xattr: name=user.rgw.acl bl.length()=123 2023-10-05T19:40:50.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.419+0000 7f505c998e40 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5 2023-10-05T19:40:50.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.419+0000 7f505c998e40 10 int RGWRados::cls_bucket_list_unordered(const DoutPrefixProvider*, RGWBucketInfo&, int, const rgw_obj_index_key&, const string&, uint32_t, bool, std::vector&, bool*, rgw_obj_index_key*, optional_yield, RGWBucketListNameFilter) :myfoo[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5]) start_after="", prefix="", shard_id=-1", num_entries=1100, list_versions=1, force_check_filter is unset 2023-10-05T19:40:50.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.419+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:106 4.16 4:6b26b3e1:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed50839a0 con 0x560ed4f6b5e0 2023-10-05T19:40:50.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.419+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(106 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.419+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:107 4.26 4:66fd5889:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed50839a0 con 0x7f504c01a070 2023-10-05T19:40:50.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.423+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(107 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f505400a5a0 con 0x7f504c01a070 2023-10-05T19:40:50.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.423+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:108 4.17 4:ea8eaa27:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed50839a0 con 0x7f504c01a070 2023-10-05T19:40:50.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.423+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(108 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.2 [call out=425b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+425 (crc 0 0 0) 0x7f505400a5a0 con 0x7f504c01a070 2023-10-05T19:40:50.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.423+0000 7f505c998e40 10 cls_bucket_list_unordered: got 9c8454ddf7aa50116496bac348d7550d 2023-10-05T19:40:50.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.423+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:109 4.1d 4:bb7bd124:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed50839a0 con 0x560ed4f6b5e0 2023-10-05T19:40:50.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.423+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 27 ==== osd_op_reply(109 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.3 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.423+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:110 4.1b 4:d8781366:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed50839a0 con 0x560ed4f66c20 2023-10-05T19:40:50.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.423+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 23 ==== osd_op_reply(110 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f66c20 2023-10-05T19:40:50.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.423+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:111 4.22 4:45f456fe:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed50839a0 con 0x560ed4f35420 2023-10-05T19:40:50.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.423+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(111 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.5 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.423+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:112 4.10 4:09e5769e:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed50839a0 con 0x560ed4f6b5e0 2023-10-05T19:40:50.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.427+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 28 ==== osd_op_reply(112 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.6 [call out=426b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+426 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.427+0000 7f505c998e40 10 cls_bucket_list_unordered: got b1b9a972ccd8c962a473909b97007eb4 2023-10-05T19:40:50.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.427+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:113 4.12 4:48215a58:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed50839a0 con 0x560ed4f35420 2023-10-05T19:40:50.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.427+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(113 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.7 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.427+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:114 4.18 4:1a9cb3ef:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed50839a0 con 0x560ed4f66c20 2023-10-05T19:40:50.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.427+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 24 ==== osd_op_reply(114 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.8 [call out=426b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+426 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f66c20 2023-10-05T19:40:50.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.427+0000 7f505c998e40 10 cls_bucket_list_unordered: got c785e1ed2950e3e36b1e2ca01f299a54 2023-10-05T19:40:50.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.427+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:115 4.3d 4:be3e4782:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed50839a0 con 0x560ed4f6b5e0 2023-10-05T19:40:50.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.427+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 29 ==== osd_op_reply(115 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.9 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.427+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:116 4.31 4:8f38cb88:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed50839a0 con 0x560ed4f35420 2023-10-05T19:40:50.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.427+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(116 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.10 [call out=697b] v0'0 uv11 ondisk = 0) v8 ==== 195+0+697 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.427+0000 7f505c998e40 10 cls_bucket_list_unordered: got 24d27c169c2c881eb09a065116f2aa5c 2023-10-05T19:40:50.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.427+0000 7f505c998e40 10 cls_bucket_list_unordered: got bb3aec0fdcdbc2974890f805c585d432 2023-10-05T19:40:50.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.427+0000 7f505c998e40 20 get_obj_state: rctx=0x7ffec71ae080 obj=myfoo:9c8454ddf7aa50116496bac348d7550d state=0x560ed5008f28 s->prefetch_data=0 2023-10-05T19:40:50.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.427+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:117 3.2 3:41e6dc41:::93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5_9c8454ddf7aa50116496bac348d7550d:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5008210 con 0x560ed4f35420 2023-10-05T19:40:50.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.431+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== osd_op_reply(117 93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5_9c8454ddf7aa50116496bac348d7550d [getxattrs out=877b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 262+0+893 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.431+0000 7f505c998e40 10 manifest: total_size = 6 2023-10-05T19:40:50.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.431+0000 7f505c998e40 20 get_obj_state: setting s->obj_tag to 93ac49f4-ff49-4f99-811b-6453b28decd5.4219.8265908493818609014 2023-10-05T19:40:50.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.431+0000 7f505c998e40 20 get_obj_state: rctx=0x7ffec71ae080 obj=myfoo:9c8454ddf7aa50116496bac348d7550d state=0x560ed5008f28 s->prefetch_data=0 2023-10-05T19:40:50.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.431+0000 7f505c998e40 20 prepare_atomic_modification: state is not atomic. state=0x560ed5008f28 2023-10-05T19:40:50.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.431+0000 7f505c998e40 20 bucket index object: default.rgw.buckets.index:.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.2 2023-10-05T19:40:50.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.431+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:118 4.17 4:ea8eaa27:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.2:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5008870 con 0x7f504c01a070 2023-10-05T19:40:50.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.431+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(118 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.2 [stat,call,call] v30'6 uv6 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x7f505400a5a0 con 0x7f504c01a070 2023-10-05T19:40:50.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.431+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:119 3.2 3:41e6dc41:::93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5_9c8454ddf7aa50116496bac348d7550d:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed500b8b0 con 0x560ed4f35420 2023-10-05T19:40:50.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.435+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 39 ==== osd_op_reply(119 93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5_9c8454ddf7aa50116496bac348d7550d [call] v30'2 uv2 ondisk = 0) v8 ==== 220+0+0 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.435+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:120 4.17 4:ea8eaa27:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.2:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed500ddc0 con 0x7f504c01a070 2023-10-05T19:40:50.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.435+0000 7f505c998e40 10 cache get: name=default.rgw.log++bucket.sync-source-hints.myfoo : miss 2023-10-05T19:40:50.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.435+0000 7f505c998e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.435+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:121 6.18 6:1df25bda:::bucket.sync-source-hints.myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed50108d0 con 0x7f504c01a070 2023-10-05T19:40:50.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.435+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(121 bucket.sync-source-hints.myfoo [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 216+0+0 (crc 0 0 0) 0x7f505400a5a0 con 0x7f504c01a070 2023-10-05T19:40:50.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.435+0000 7f505c998e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:50.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.435+0000 7f505c998e40 10 cache put: name=default.rgw.log++bucket.sync-source-hints.myfoo info.flags=0x0 2023-10-05T19:40:50.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.435+0000 7f505c998e40 10 adding default.rgw.log++bucket.sync-source-hints.myfoo to cache LRU end 2023-10-05T19:40:50.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.435+0000 7f505c998e40 10 cache get: name=default.rgw.log++bucket.sync-target-hints.myfoo : miss 2023-10-05T19:40:50.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.435+0000 7f505c998e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.435+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:122 6.4 6:26e4c4bc:::bucket.sync-target-hints.myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed50108d0 con 0x560ed4f35420 2023-10-05T19:40:50.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.435+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 40 ==== osd_op_reply(122 bucket.sync-target-hints.myfoo [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 216+0+0 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.435+0000 7f505c998e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:50.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.435+0000 7f505c998e40 10 cache put: name=default.rgw.log++bucket.sync-target-hints.myfoo info.flags=0x0 2023-10-05T19:40:50.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.435+0000 7f505c998e40 10 adding default.rgw.log++bucket.sync-target-hints.myfoo to cache LRU end 2023-10-05T19:40:50.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.435+0000 7f505c998e40 10 chain_cache_entry: cache_locator= 2023-10-05T19:40:50.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.435+0000 7f505c998e40 20 chain_cache_entry: couldn't find cache locator 2023-10-05T19:40:50.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.435+0000 7f505c998e40 20 couldn't put bucket_sync_policy cache entry, might have raced with data changes 2023-10-05T19:40:50.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.435+0000 7f505c998e40 20 get_obj_state: rctx=0x7ffec71ae080 obj=myfoo:b1b9a972ccd8c962a473909b97007eb4 state=0x560ed5008f28 s->prefetch_data=0 2023-10-05T19:40:50.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.435+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:123 3.38 3:1ca8789b:::93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5_b1b9a972ccd8c962a473909b97007eb4:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed500e720 con 0x7f504c01a070 2023-10-05T19:40:50.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.439+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(123 93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5_b1b9a972ccd8c962a473909b97007eb4 [getxattrs out=879b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 262+0+895 (crc 0 0 0) 0x7f505400a5a0 con 0x7f504c01a070 2023-10-05T19:40:50.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.439+0000 7f505c998e40 10 manifest: total_size = 3 2023-10-05T19:40:50.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.439+0000 7f505c998e40 20 get_obj_state: setting s->obj_tag to 93ac49f4-ff49-4f99-811b-6453b28decd5.4219.17188618769800132403 2023-10-05T19:40:50.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.439+0000 7f505c998e40 20 get_obj_state: rctx=0x7ffec71ae080 obj=myfoo:b1b9a972ccd8c962a473909b97007eb4 state=0x560ed5008f28 s->prefetch_data=0 2023-10-05T19:40:50.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.439+0000 7f505c998e40 20 prepare_atomic_modification: state is not atomic. state=0x560ed5008f28 2023-10-05T19:40:50.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.439+0000 7f505c998e40 20 bucket index object: default.rgw.buckets.index:.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.6 2023-10-05T19:40:50.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.439+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:124 4.10 4:09e5769e:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.6:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5010520 con 0x560ed4f6b5e0 2023-10-05T19:40:50.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.439+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.0 v2:172.21.15.8:6800/27563 30 ==== osd_op_reply(120 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.2 [stat,call,call] v30'7 uv7 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x7f505400a5a0 con 0x7f504c01a070 2023-10-05T19:40:50.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.439+0000 7f5058cca700 20 handle_completion(): completion ok for obj=9c8454ddf7aa50116496bac348d7550d 2023-10-05T19:40:50.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.439+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 30 ==== osd_op_reply(124 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.6 [stat,call,call] v30'6 uv6 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.439+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:125 3.38 3:1ca8789b:::93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5_b1b9a972ccd8c962a473909b97007eb4:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5010520 con 0x7f504c01a070 2023-10-05T19:40:50.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.443+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.0 v2:172.21.15.8:6800/27563 31 ==== osd_op_reply(125 93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5_b1b9a972ccd8c962a473909b97007eb4 [call] v30'2 uv2 ondisk = 0) v8 ==== 220+0+0 (crc 0 0 0) 0x7f505400a5a0 con 0x7f504c01a070 2023-10-05T19:40:50.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.443+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:126 4.10 4:09e5769e:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.6:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5010520 con 0x560ed4f6b5e0 2023-10-05T19:40:50.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.443+0000 7f505c998e40 10 cache get: name=default.rgw.log++bucket.sync-source-hints.myfoo : hit (negative entry) 2023-10-05T19:40:50.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.443+0000 7f505c998e40 10 cache get: name=default.rgw.log++bucket.sync-target-hints.myfoo : hit (negative entry) 2023-10-05T19:40:50.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.443+0000 7f505c998e40 10 chain_cache_entry: cache_locator= 2023-10-05T19:40:50.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.443+0000 7f505c998e40 20 chain_cache_entry: couldn't find cache locator 2023-10-05T19:40:50.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.443+0000 7f505c998e40 20 couldn't put bucket_sync_policy cache entry, might have raced with data changes 2023-10-05T19:40:50.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.443+0000 7f505c998e40 20 get_obj_state: rctx=0x7ffec71ae080 obj=myfoo:c785e1ed2950e3e36b1e2ca01f299a54 state=0x560ed5008f28 s->prefetch_data=0 2023-10-05T19:40:50.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.443+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:127 3.33 3:cca9eba0:::93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5_c785e1ed2950e3e36b1e2ca01f299a54:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5010520 con 0x7f504c01a070 2023-10-05T19:40:50.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.443+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.0 v2:172.21.15.8:6800/27563 32 ==== osd_op_reply(127 93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5_c785e1ed2950e3e36b1e2ca01f299a54 [getxattrs out=879b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 262+0+895 (crc 0 0 0) 0x7f505400a5a0 con 0x7f504c01a070 2023-10-05T19:40:50.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.443+0000 7f505c998e40 10 manifest: total_size = 4 2023-10-05T19:40:50.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.443+0000 7f505c998e40 20 get_obj_state: setting s->obj_tag to 93ac49f4-ff49-4f99-811b-6453b28decd5.4219.14478364633210238579 2023-10-05T19:40:50.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.443+0000 7f505c998e40 20 get_obj_state: rctx=0x7ffec71ae080 obj=myfoo:c785e1ed2950e3e36b1e2ca01f299a54 state=0x560ed5008f28 s->prefetch_data=0 2023-10-05T19:40:50.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.443+0000 7f505c998e40 20 prepare_atomic_modification: state is not atomic. state=0x560ed5008f28 2023-10-05T19:40:50.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.443+0000 7f505c998e40 20 bucket index object: default.rgw.buckets.index:.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.8 2023-10-05T19:40:50.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.443+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:128 4.18 4:1a9cb3ef:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.8:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5010520 con 0x560ed4f66c20 2023-10-05T19:40:50.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.447+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 31 ==== osd_op_reply(126 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.6 [stat,call,call] v30'7 uv7 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.447+0000 7f5053fff700 20 handle_completion(): completion ok for obj=b1b9a972ccd8c962a473909b97007eb4 2023-10-05T19:40:50.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.447+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 25 ==== osd_op_reply(128 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.8 [stat,call,call] v30'4 uv4 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f66c20 2023-10-05T19:40:50.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.447+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:129 3.33 3:cca9eba0:::93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5_c785e1ed2950e3e36b1e2ca01f299a54:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5010520 con 0x7f504c01a070 2023-10-05T19:40:50.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.451+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.0 v2:172.21.15.8:6800/27563 33 ==== osd_op_reply(129 93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5_c785e1ed2950e3e36b1e2ca01f299a54 [call] v30'2 uv2 ondisk = 0) v8 ==== 220+0+0 (crc 0 0 0) 0x7f505400a5a0 con 0x7f504c01a070 2023-10-05T19:40:50.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.451+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:130 4.18 4:1a9cb3ef:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.8:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5010520 con 0x560ed4f66c20 2023-10-05T19:40:50.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.451+0000 7f505c998e40 10 cache get: name=default.rgw.log++bucket.sync-source-hints.myfoo : hit (negative entry) 2023-10-05T19:40:50.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.451+0000 7f505c998e40 10 cache get: name=default.rgw.log++bucket.sync-target-hints.myfoo : hit (negative entry) 2023-10-05T19:40:50.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.451+0000 7f505c998e40 10 chain_cache_entry: cache_locator= 2023-10-05T19:40:50.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.451+0000 7f505c998e40 20 chain_cache_entry: couldn't find cache locator 2023-10-05T19:40:50.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.451+0000 7f505c998e40 20 couldn't put bucket_sync_policy cache entry, might have raced with data changes 2023-10-05T19:40:50.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.451+0000 7f505c998e40 20 get_obj_state: rctx=0x7ffec71ae080 obj=myfoo:24d27c169c2c881eb09a065116f2aa5c state=0x560ed5008f28 s->prefetch_data=0 2023-10-05T19:40:50.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.451+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:131 3.c 3:31173a33:::93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5_24d27c169c2c881eb09a065116f2aa5c:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5010520 con 0x560ed4f6b5e0 2023-10-05T19:40:50.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.451+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 32 ==== osd_op_reply(131 93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5_24d27c169c2c881eb09a065116f2aa5c [getxattrs out=877b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 262+0+893 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.451+0000 7f505c998e40 10 manifest: total_size = 5 2023-10-05T19:40:50.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.451+0000 7f505c998e40 20 get_obj_state: setting s->obj_tag to 93ac49f4-ff49-4f99-811b-6453b28decd5.4219.7911635305410494314 2023-10-05T19:40:50.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.451+0000 7f505c998e40 20 get_obj_state: rctx=0x7ffec71ae080 obj=myfoo:24d27c169c2c881eb09a065116f2aa5c state=0x560ed5008f28 s->prefetch_data=0 2023-10-05T19:40:50.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.451+0000 7f505c998e40 20 prepare_atomic_modification: state is not atomic. state=0x560ed5008f28 2023-10-05T19:40:50.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.451+0000 7f505c998e40 20 bucket index object: default.rgw.buckets.index:.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.10 2023-10-05T19:40:50.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.451+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:132 4.31 4:8f38cb88:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.10:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5010520 con 0x560ed4f35420 2023-10-05T19:40:50.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.455+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 26 ==== osd_op_reply(130 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.8 [stat,call,call] v30'5 uv5 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f66c20 2023-10-05T19:40:50.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.455+0000 7f5058cca700 20 handle_completion(): completion ok for obj=c785e1ed2950e3e36b1e2ca01f299a54 2023-10-05T19:40:50.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.455+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 41 ==== osd_op_reply(132 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.10 [stat,call,call] v30'12 uv12 ondisk = 0) v8 ==== 279+0+0 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.455+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:133 3.c 3:31173a33:::93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5_24d27c169c2c881eb09a065116f2aa5c:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5010520 con 0x560ed4f6b5e0 2023-10-05T19:40:50.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.459+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 33 ==== osd_op_reply(133 93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5_24d27c169c2c881eb09a065116f2aa5c [call] v30'2 uv2 ondisk = 0) v8 ==== 220+0+0 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.459+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:134 4.31 4:8f38cb88:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.10:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5010520 con 0x560ed4f35420 2023-10-05T19:40:50.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.459+0000 7f505c998e40 10 cache get: name=default.rgw.log++bucket.sync-source-hints.myfoo : hit (negative entry) 2023-10-05T19:40:50.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.459+0000 7f505c998e40 10 cache get: name=default.rgw.log++bucket.sync-target-hints.myfoo : hit (negative entry) 2023-10-05T19:40:50.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.459+0000 7f505c998e40 10 chain_cache_entry: cache_locator= 2023-10-05T19:40:50.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.459+0000 7f505c998e40 20 chain_cache_entry: couldn't find cache locator 2023-10-05T19:40:50.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.459+0000 7f505c998e40 20 couldn't put bucket_sync_policy cache entry, might have raced with data changes 2023-10-05T19:40:50.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.459+0000 7f505c998e40 20 get_obj_state: rctx=0x7ffec71ae080 obj=myfoo:bb3aec0fdcdbc2974890f805c585d432 state=0x560ed5008f28 s->prefetch_data=0 2023-10-05T19:40:50.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.459+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:135 3.26 3:6571379c:::93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5_bb3aec0fdcdbc2974890f805c585d432:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5010520 con 0x560ed4f66c20 2023-10-05T19:40:50.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.459+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 27 ==== osd_op_reply(135 93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5_bb3aec0fdcdbc2974890f805c585d432 [getxattrs out=911b,stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 262+0+927 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f66c20 2023-10-05T19:40:50.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.459+0000 7f505c998e40 10 manifest: total_size = 5 2023-10-05T19:40:50.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.459+0000 7f505c998e40 20 get_obj_state: setting s->obj_tag to _7PC3k5E6JWvxHQbDyNW5w0aEodn5W5E 2023-10-05T19:40:50.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.459+0000 7f505c998e40 20 get_obj_state: rctx=0x7ffec71ae080 obj=myfoo:bb3aec0fdcdbc2974890f805c585d432 state=0x560ed5008f28 s->prefetch_data=0 2023-10-05T19:40:50.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.459+0000 7f505c998e40 20 prepare_atomic_modification: state is not atomic. state=0x560ed5008f28 2023-10-05T19:40:50.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.459+0000 7f505c998e40 20 bucket index object: default.rgw.buckets.index:.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.10 2023-10-05T19:40:50.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.459+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:136 4.31 4:8f38cb88:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.10:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5010f90 con 0x560ed4f35420 2023-10-05T19:40:50.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.459+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 42 ==== osd_op_reply(134 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.10 [stat,call,call] v30'13 uv13 ondisk = 0) v8 ==== 279+0+0 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.459+0000 7f5053fff700 20 handle_completion(): completion ok for obj=24d27c169c2c881eb09a065116f2aa5c 2023-10-05T19:40:50.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.463+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 43 ==== osd_op_reply(136 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.10 [stat,call,call] v30'14 uv14 ondisk = 0) v8 ==== 279+0+0 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.463+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:137 3.26 3:6571379c:::93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5_bb3aec0fdcdbc2974890f805c585d432:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5010f90 con 0x560ed4f66c20 2023-10-05T19:40:50.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.467+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 28 ==== osd_op_reply(137 93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5_bb3aec0fdcdbc2974890f805c585d432 [call] v30'4 uv4 ondisk = 0) v8 ==== 220+0+0 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f66c20 2023-10-05T19:40:50.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.467+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:138 4.31 4:8f38cb88:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.10:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5011960 con 0x560ed4f35420 2023-10-05T19:40:50.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.467+0000 7f505c998e40 10 cache get: name=default.rgw.log++bucket.sync-source-hints.myfoo : hit (negative entry) 2023-10-05T19:40:50.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.467+0000 7f505c998e40 10 cache get: name=default.rgw.log++bucket.sync-target-hints.myfoo : hit (negative entry) 2023-10-05T19:40:50.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.467+0000 7f505c998e40 10 chain_cache_entry: cache_locator= 2023-10-05T19:40:50.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.467+0000 7f505c998e40 20 chain_cache_entry: couldn't find cache locator 2023-10-05T19:40:50.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.467+0000 7f505c998e40 20 couldn't put bucket_sync_policy cache entry, might have raced with data changes 2023-10-05T19:40:50.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.467+0000 7f505c998e40 20 int RGWRados::Bucket::List::list_objects_ordered(const DoutPrefixProvider*, int64_t, std::vector*, std::map, bool>*, bool*, optional_yield): starting attempt 1 2023-10-05T19:40:50.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.467+0000 7f505c998e40 10 int RGWRados::cls_bucket_list_ordered(const DoutPrefixProvider*, RGWBucketInfo&, int, const rgw_obj_index_key&, const string&, const string&, uint32_t, bool, uint16_t, RGWRados::ent_map_t&, bool*, bool*, rgw_obj_index_key*, optional_yield, RGWBucketListNameFilter): :myfoo[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5]) start_after="_multipart_", prefix="_multipart_, delimiter="", shard_id=-1", num_entries=1001, list_versions=0, expansion_factor=1, force_check_filter is unset 2023-10-05T19:40:50.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.467+0000 7f505c998e40 10 int RGWRados::cls_bucket_list_ordered(const DoutPrefixProvider*, RGWBucketInfo&, int, const rgw_obj_index_key&, const string&, const string&, uint32_t, bool, uint16_t, RGWRados::ent_map_t&, bool*, bool*, rgw_obj_index_key*, optional_yield, RGWBucketListNameFilter): request from each of 11 shard(s) for 112 entries to get 1001 total entries 2023-10-05T19:40:50.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.467+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:139 4.16 4:6b26b3e1:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.0:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f504c063000 con 0x560ed4f6b5e0 2023-10-05T19:40:50.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.467+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:140 4.26 4:66fd5889:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.1:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5011a80 con 0x7f504c01a070 2023-10-05T19:40:50.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.467+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:141 4.17 4:ea8eaa27:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.2:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed50142a0 con 0x7f504c01a070 2023-10-05T19:40:50.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.467+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:142 4.1d 4:bb7bd124:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.3:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5014d50 con 0x560ed4f6b5e0 2023-10-05T19:40:50.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.467+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:143 4.1b 4:d8781366:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.4:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed50142a0 con 0x560ed4f66c20 2023-10-05T19:40:50.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.467+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:144 4.22 4:45f456fe:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.5:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed501ff30 con 0x560ed4f35420 2023-10-05T19:40:50.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.467+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:145 4.10 4:09e5769e:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.6:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5014d50 con 0x560ed4f6b5e0 2023-10-05T19:40:50.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.467+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:146 4.12 4:48215a58:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.7:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed50142a0 con 0x560ed4f35420 2023-10-05T19:40:50.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.467+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:147 4.18 4:1a9cb3ef:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.8:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5028080 con 0x560ed4f66c20 2023-10-05T19:40:50.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.467+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:148 4.3d 4:be3e4782:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.9:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed50142a0 con 0x560ed4f6b5e0 2023-10-05T19:40:50.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.467+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:149 4.31 4:8f38cb88:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.10:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5014d50 con 0x560ed4f35420 2023-10-05T19:40:50.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.467+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 34 ==== osd_op_reply(139 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.467+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.0 v2:172.21.15.8:6800/27563 34 ==== osd_op_reply(141 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.2 [call out=124b] v0'0 uv7 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x7f505400a5a0 con 0x7f504c01a070 2023-10-05T19:40:50.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.467+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 35 ==== osd_op_reply(142 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.3 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.467+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 36 ==== osd_op_reply(145 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.6 [call out=124b] v0'0 uv7 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.467+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 37 ==== osd_op_reply(148 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.9 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.467+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 44 ==== osd_op_reply(144 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.5 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.467+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 29 ==== osd_op_reply(143 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f66c20 2023-10-05T19:40:50.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.467+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 45 ==== osd_op_reply(146 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.7 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.467+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.0 v2:172.21.15.8:6800/27563 35 ==== osd_op_reply(140 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f505400a5a0 con 0x7f504c01a070 2023-10-05T19:40:50.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.467+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 30 ==== osd_op_reply(147 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.8 [call out=124b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f66c20 2023-10-05T19:40:50.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.467+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 46 ==== osd_op_reply(138 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.10 [stat,call,call] v30'15 uv15 ondisk = 0) v8 ==== 279+0+0 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.467+0000 7f5058cca700 20 handle_completion(): completion ok for obj=bb3aec0fdcdbc2974890f805c585d432 2023-10-05T19:40:50.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.467+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 47 ==== osd_op_reply(149 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.10 [call out=124b] v0'0 uv15 ondisk = 0) v8 ==== 195+0+124 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f35420 2023-10-05T19:40:50.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.467+0000 7f505c998e40 20 int RGWRados::cls_bucket_list_ordered(const DoutPrefixProvider*, RGWBucketInfo&, int, const rgw_obj_index_key&, const string&, const string&, uint32_t, bool, uint16_t, RGWRados::ent_map_t&, bool*, bool*, rgw_obj_index_key*, optional_yield, RGWBucketListNameFilter): returning, count=0, is_truncated=0 2023-10-05T19:40:50.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.467+0000 7f505c998e40 20 int RGWRados::cls_bucket_list_ordered(const DoutPrefixProvider*, RGWBucketInfo&, int, const rgw_obj_index_key&, const string&, const string&, uint32_t, bool, uint16_t, RGWRados::ent_map_t&, bool*, bool*, rgw_obj_index_key*, optional_yield, RGWBucketListNameFilter): returning, last_entry NOT SET 2023-10-05T19:40:50.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.467+0000 7f505c998e40 10 int RGWRados::Bucket::List::list_objects_ordered(const DoutPrefixProvider*, int64_t, std::vector*, std::map, bool>*, bool*, optional_yield): end of outer loop, truncated=0, count=0, attempt=1 2023-10-05T19:40:50.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.467+0000 7f505c998e40 10 int RGWRados::Bucket::List::list_objects_ordered(const DoutPrefixProvider*, int64_t, std::vector*, std::map, bool>*, bool*, optional_yield): exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2023-10-05T19:40:50.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.467+0000 7f505c998e40 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":myfoo[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5])"; uploads.size()=0; is_truncated=0 2023-10-05T19:40:50.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.467+0000 7f505c998e40 10 lifecycle: RGWRados::convert_old_bucket_info(): bucket=:myfoo[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5]) 2023-10-05T19:40:50.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.471+0000 7f505c998e40 20 lifecycle: get_system_obj_state: rctx=0x7ffec71adb18 obj=default.rgw.meta:root:myfoo state=0x560ed4de02b0 s->prefetch_data=0 2023-10-05T19:40:50.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.471+0000 7f505c998e40 10 lifecycle: cache get: name=default.rgw.meta+root+myfoo : type miss (requested=0x16, cached=0x13) 2023-10-05T19:40:50.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.471+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:150 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5028000 con 0x560ed4f66c20 2023-10-05T19:40:50.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.471+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 31 ==== osd_op_reply(150 myfoo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv33 ondisk = 0) v8 ==== 233+0+139 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f66c20 2023-10-05T19:40:50.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.471+0000 7f505c998e40 10 lifecycle: cache put: name=default.rgw.meta+root+myfoo info.flags=0x16 2023-10-05T19:40:50.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.471+0000 7f505c998e40 10 lifecycle: moving default.rgw.meta+root+myfoo to cache LRU end 2023-10-05T19:40:50.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.471+0000 7f505c998e40 10 lifecycle: updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:50.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.471+0000 7f505c998e40 20 lifecycle: get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:50.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.471+0000 7f505c998e40 20 lifecycle: Read xattr: user.rgw.idtag 2023-10-05T19:40:50.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.471+0000 7f505c998e40 10 lifecycle: cache get: name=default.rgw.meta+root+myfoo : hit (requested=0x13, cached=0x17) 2023-10-05T19:40:50.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.471+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:151 8.10 8:0a6cdd29:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~352 in=352b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5022640 con 0x560ed4f6b5e0 2023-10-05T19:40:50.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.471+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 38 ==== osd_op_reply(151 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5 [delete,create,call,call,writefull 0~352,setxattr (123)] v30'2 uv2 ondisk = 0) v8 ==== 416+0+0 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.471+0000 7f505c998e40 10 lifecycle: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5 info.flags=0x17 2023-10-05T19:40:50.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.471+0000 7f505c998e40 10 lifecycle: moving default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5 to cache LRU end 2023-10-05T19:40:50.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.471+0000 7f505c998e40 10 lifecycle: updating xattr: name=user.rgw.acl bl.length()=123 2023-10-05T19:40:50.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.471+0000 7f505c998e40 10 lifecycle: distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5, ofs0, ns] 2023-10-05T19:40:50.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.471+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:152 7.e 7:7759931f:::notify.3:head [notify cookie 94621703102800 in=737b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5022640 con 0x560ed4f66c20 2023-10-05T19:40:50.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.471+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 32 ==== watch-notify(notify (1) cookie 94621702505376 notify 128849018904 ret 0) v3 ==== 767+0+0 (crc 0 0 0) 0x560ed4f59860 con 0x560ed4f66c20 2023-10-05T19:40:50.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.471+0000 7f5058cca700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018904 cookie 94621702505376 notifier 4639 bl.length()=725 2023-10-05T19:40:50.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.471+0000 7f5058cca700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5 info.flags=0x17 2023-10-05T19:40:50.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.471+0000 7f5058cca700 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5 to cache LRU end 2023-10-05T19:40:50.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.471+0000 7f5058cca700 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2023-10-05T19:40:50.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.471+0000 7f5058cca700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:153 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f5040003d90 con 0x560ed4f66c20 2023-10-05T19:40:50.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.471+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 33 ==== osd_op_reply(152 notify.3 [notify cookie 94621703102800 out=8b] v0'0 uv207 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f66c20 2023-10-05T19:40:50.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.475+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 34 ==== osd_op_reply(153 notify.3 [notify-ack] v0'0 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f66c20 2023-10-05T19:40:50.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.475+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 35 ==== watch-notify(notify_complete (2) cookie 94621703102800 notify 128849018904 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x560ed4f59420 con 0x560ed4f66c20 2023-10-05T19:40:50.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.475+0000 7f505c998e40 20 lifecycle: handle_bi_update(): bucket=:myfoo[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5]): orig_sources= new_sources= 2023-10-05T19:40:50.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.475+0000 7f505c998e40 20 lifecycle: handle_bi_update(): bucket=:myfoo[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5]): potential sources added=[] removed=[] 2023-10-05T19:40:50.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.475+0000 7f505c998e40 20 lifecycle: handle_bi_update(): bucket=:myfoo[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5]): orig_dests= new_dests= 2023-10-05T19:40:50.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.475+0000 7f505c998e40 20 lifecycle: handle_bi_update(): bucket=:myfoo[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5]): potential dests added=[] removed=[] 2023-10-05T19:40:50.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.475+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:33 6.7 6:e31d3d5f:lc::lc.28:head [call lock.lock in=65b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed501f9e0 con 0x560ed4f9c130 2023-10-05T19:40:50.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.475+0000 7f505acce700 1 -- 172.21.15.12:0/2961973426 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(33 lc.28 [call] v30'62 uv62 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f505400a5a0 con 0x560ed4f9c130 2023-10-05T19:40:50.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.475+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 6.7 6:e31d3d5f:lc::lc.28:head [call rgw.lc_rm_entry in=92b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed501f9e0 con 0x560ed4f9c130 2023-10-05T19:40:50.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.479+0000 7f505acce700 1 -- 172.21.15.12:0/2961973426 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(34 lc.28 [call] v30'63 uv63 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f5054062040 con 0x560ed4f9c130 2023-10-05T19:40:50.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.479+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:35 6.7 6:e31d3d5f:lc::lc.28:head [call lock.unlock in=49b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed501f9e0 con 0x560ed4f9c130 2023-10-05T19:40:50.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.483+0000 7f505acce700 1 -- 172.21.15.12:0/2961973426 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(35 lc.28 [call] v30'64 uv64 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f5054062040 con 0x560ed4f9c130 2023-10-05T19:40:50.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.483+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:154 4.16 4:6b26b3e1:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed501f9e0 con 0x560ed4f6b5e0 2023-10-05T19:40:50.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.483+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:155 4.26 4:66fd5889:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed502cc40 con 0x7f504c01a070 2023-10-05T19:40:50.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.483+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:156 4.17 4:ea8eaa27:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed502d080 con 0x7f504c01a070 2023-10-05T19:40:50.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.483+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:157 4.1d 4:bb7bd124:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed501f9e0 con 0x560ed4f6b5e0 2023-10-05T19:40:50.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.483+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:158 4.1b 4:d8781366:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5011470 con 0x560ed4f66c20 2023-10-05T19:40:50.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.483+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:159 4.22 4:45f456fe:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed501f9e0 con 0x560ed4f35420 2023-10-05T19:40:50.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.483+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:160 4.10 4:09e5769e:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed500c230 con 0x560ed4f6b5e0 2023-10-05T19:40:50.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.483+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:161 4.12 4:48215a58:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5011470 con 0x560ed4f35420 2023-10-05T19:40:50.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.483+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:162 4.18 4:1a9cb3ef:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed502a210 con 0x560ed4f66c20 2023-10-05T19:40:50.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.483+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:163 4.3d 4:be3e4782:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed502a210 con 0x560ed4f6b5e0 2023-10-05T19:40:50.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.483+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:164 4.31 4:8f38cb88:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed502cc40 con 0x560ed4f35420 2023-10-05T19:40:50.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.483+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 39 ==== osd_op_reply(154 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.483+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 40 ==== osd_op_reply(157 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.3 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.483+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 41 ==== osd_op_reply(160 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.6 [call out=124b] v0'0 uv7 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.483+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 36 ==== osd_op_reply(162 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.8 [call out=124b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f66c20 2023-10-05T19:40:50.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.483+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 37 ==== osd_op_reply(158 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f66c20 2023-10-05T19:40:50.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.483+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 42 ==== osd_op_reply(163 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.9 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.483+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.0 v2:172.21.15.8:6800/27563 36 ==== osd_op_reply(155 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f5054062040 con 0x7f504c01a070 2023-10-05T19:40:50.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.483+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.0 v2:172.21.15.8:6800/27563 37 ==== osd_op_reply(156 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.2 [call out=124b] v0'0 uv7 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x7f5054062040 con 0x7f504c01a070 2023-10-05T19:40:50.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.483+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 48 ==== osd_op_reply(161 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.7 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f5054062040 con 0x560ed4f35420 2023-10-05T19:40:50.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.483+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 49 ==== osd_op_reply(159 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.5 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f5054062040 con 0x560ed4f35420 2023-10-05T19:40:50.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.483+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 50 ==== osd_op_reply(164 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.10 [call out=124b] v0'0 uv15 ondisk = 0) v8 ==== 195+0+124 (crc 0 0 0) 0x7f5054062040 con 0x560ed4f35420 2023-10-05T19:40:50.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.483+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:165 8.9 8:96d98480:users.uid::foo.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed502d080 con 0x560ed4f66c20 2023-10-05T19:40:50.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.487+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 38 ==== osd_op_reply(165 foo.buckets [call] v30'36 uv36 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f66c20 2023-10-05T19:40:50.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.487+0000 7f505c998e40 20 get_system_obj_state: rctx=0x7ffec71ad598 obj=default.rgw.meta:root:myfoo state=0x560ed4de02b0 s->prefetch_data=0 2023-10-05T19:40:50.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.487+0000 7f505c998e40 10 cache get: name=default.rgw.meta+root+myfoo : hit (requested=0x16, cached=0x17) 2023-10-05T19:40:50.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.487+0000 7f505c998e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:50.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.487+0000 7f505c998e40 10 cache get: name=default.rgw.meta+root+myfoo : hit (requested=0x11, cached=0x17) 2023-10-05T19:40:50.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.487+0000 7f505c998e40 10 removing default.rgw.meta+root+myfoo from cache 2023-10-05T19:40:50.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.487+0000 7f505c998e40 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2023-10-05T19:40:50.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.487+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:166 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94621703102800 in=161b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed502d080 con 0x7f504c01a070 2023-10-05T19:40:50.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.487+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.0 v2:172.21.15.8:6800/27563 38 ==== watch-notify(notify (1) cookie 94621702512832 notify 128849018904 ret 0) v3 ==== 191+0+0 (crc 0 0 0) 0x7f505402f040 con 0x7f504c01a070 2023-10-05T19:40:50.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.487+0000 7f5058cca700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018904 cookie 94621702512832 notifier 4639 bl.length()=149 2023-10-05T19:40:50.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.487+0000 7f5058cca700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:167 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f5040003d90 con 0x7f504c01a070 2023-10-05T19:40:50.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.487+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.0 v2:172.21.15.8:6800/27563 39 ==== osd_op_reply(166 notify.6 [notify cookie 94621703102800 out=8b] v0'0 uv207 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f5054062040 con 0x7f504c01a070 2023-10-05T19:40:50.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.487+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.0 v2:172.21.15.8:6800/27563 40 ==== osd_op_reply(167 notify.6 [notify-ack] v0'0 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5054062040 con 0x7f504c01a070 2023-10-05T19:40:50.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.487+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.0 v2:172.21.15.8:6800/27563 41 ==== watch-notify(notify_complete (2) cookie 94621703102800 notify 128849018904 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f505402b870 con 0x7f504c01a070 2023-10-05T19:40:50.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.491+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:168 8.9 8:91b71841:root::myfoo:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5022640 con 0x560ed4f66c20 2023-10-05T19:40:50.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.491+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 39 ==== osd_op_reply(168 myfoo [call,call,delete] v30'37 uv37 ondisk = 0) v8 ==== 233+0+0 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f66c20 2023-10-05T19:40:50.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.491+0000 7f505c998e40 10 removing default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5 from cache 2023-10-05T19:40:50.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.491+0000 7f505c998e40 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5, ofs0, ns] 2023-10-05T19:40:50.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.491+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:169 7.e 7:7759931f:::notify.3:head [notify cookie 94621703102800 in=218b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5022640 con 0x560ed4f66c20 2023-10-05T19:40:50.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.491+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 40 ==== watch-notify(notify (1) cookie 94621702505376 notify 128849018905 ret 0) v3 ==== 248+0+0 (crc 0 0 0) 0x560ed4f59420 con 0x560ed4f66c20 2023-10-05T19:40:50.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.491+0000 7f5053fff700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018905 cookie 94621702505376 notifier 4639 bl.length()=206 2023-10-05T19:40:50.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.491+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 41 ==== osd_op_reply(169 notify.3 [notify cookie 94621703102800 out=8b] v0'0 uv207 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f66c20 2023-10-05T19:40:50.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.491+0000 7f5053fff700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:170 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f4fa8003340 con 0x560ed4f66c20 2023-10-05T19:40:50.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.495+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 42 ==== osd_op_reply(170 notify.3 [notify-ack] v0'0 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f66c20 2023-10-05T19:40:50.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.495+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 43 ==== watch-notify(notify_complete (2) cookie 94621703102800 notify 128849018905 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x560ed4e7dbd0 con 0x560ed4f66c20 2023-10-05T19:40:50.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.495+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:171 8.10 8:0a6cdd29:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5:head [call version.check_conds in=122b,call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5022640 con 0x560ed4f6b5e0 2023-10-05T19:40:50.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.495+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 43 ==== osd_op_reply(171 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5 [call,call,delete] v30'3 uv3 ondisk = 0) v8 ==== 290+0+0 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.495+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:172 4.16 4:6b26b3e1:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5022640 con 0x560ed4f6b5e0 2023-10-05T19:40:50.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.495+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:173 4.26 4:66fd5889:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5002960 con 0x7f504c01a070 2023-10-05T19:40:50.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.495+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:174 4.17 4:ea8eaa27:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5003200 con 0x7f504c01a070 2023-10-05T19:40:50.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.495+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:175 4.1d 4:bb7bd124:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5004aa0 con 0x560ed4f6b5e0 2023-10-05T19:40:50.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.495+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:176 4.1b 4:d8781366:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.4:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5022640 con 0x560ed4f66c20 2023-10-05T19:40:50.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.495+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:177 4.22 4:45f456fe:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5002960 con 0x560ed4f35420 2023-10-05T19:40:50.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.495+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:178 4.10 4:09e5769e:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5004aa0 con 0x560ed4f6b5e0 2023-10-05T19:40:50.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.495+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:179 4.12 4:48215a58:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5003200 con 0x560ed4f35420 2023-10-05T19:40:50.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.495+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:180 4.18 4:1a9cb3ef:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5022640 con 0x560ed4f66c20 2023-10-05T19:40:50.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.495+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:181 4.3d 4:be3e4782:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5004aa0 con 0x560ed4f6b5e0 2023-10-05T19:40:50.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.499+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:182 4.31 4:8f38cb88:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.10:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5002960 con 0x560ed4f35420 2023-10-05T19:40:50.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.499+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.0 v2:172.21.15.8:6800/27563 42 ==== osd_op_reply(173 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.1 [delete] v30'2 uv2 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f5054062040 con 0x7f504c01a070 2023-10-05T19:40:50.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.499+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.0 v2:172.21.15.8:6800/27563 43 ==== osd_op_reply(174 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.2 [delete] v30'8 uv8 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f5054062040 con 0x7f504c01a070 2023-10-05T19:40:50.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.499+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 44 ==== osd_op_reply(180 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.8 [delete] v30'6 uv6 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f66c20 2023-10-05T19:40:50.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.499+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 45 ==== osd_op_reply(176 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.4 [delete] v30'2 uv2 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f66c20 2023-10-05T19:40:50.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.499+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 44 ==== osd_op_reply(172 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.0 [delete] v30'2 uv2 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.499+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 45 ==== osd_op_reply(175 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.3 [delete] v30'2 uv2 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.503+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 51 ==== osd_op_reply(177 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.5 [delete] v30'2 uv2 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f5054062040 con 0x560ed4f35420 2023-10-05T19:40:50.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.503+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 46 ==== osd_op_reply(178 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.6 [delete] v30'8 uv8 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.503+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 47 ==== osd_op_reply(181 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.9 [delete] v30'4 uv4 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.503+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 52 ==== osd_op_reply(182 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.10 [delete] v30'16 uv16 ondisk = 0) v8 ==== 195+0+0 (crc 0 0 0) 0x7f5054062040 con 0x560ed4f35420 2023-10-05T19:40:50.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.503+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 53 ==== osd_op_reply(179 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5.7 [delete] v30'2 uv2 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f5054062040 con 0x560ed4f35420 2023-10-05T19:40:50.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.503+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:183 6.d 6:b3423d31:::pubsub..bucket.myfoo%2f93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5022640 con 0x560ed4f35420 2023-10-05T19:40:50.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.503+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 54 ==== osd_op_reply(183 pubsub..bucket.myfoo/93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5 [getxattrs,stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 250+0+0 (crc 0 0 0) 0x7f5054062040 con 0x560ed4f35420 2023-10-05T19:40:50.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.503+0000 7f505c998e40 10 removing default.rgw.log++pubsub..bucket.myfoo/93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5 from cache 2023-10-05T19:40:50.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.503+0000 7f505c998e40 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.log:pubsub..bucket.myfoo/93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5, ofs0, ns] 2023-10-05T19:40:50.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.503+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:184 7.15 7:a93a5511:::notify.2:head [notify cookie 94621703102800 in=215b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5022640 con 0x560ed4f6b5e0 2023-10-05T19:40:50.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.503+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 48 ==== watch-notify(notify (1) cookie 94621702501856 notify 128849018889 ret 0) v3 ==== 245+0+0 (crc 0 0 0) 0x560ed4f8b020 con 0x560ed4f6b5e0 2023-10-05T19:40:50.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.503+0000 7f5058cca700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018889 cookie 94621702501856 notifier 4639 bl.length()=203 2023-10-05T19:40:50.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.503+0000 7f5058cca700 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:185 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f5040003d90 con 0x560ed4f6b5e0 2023-10-05T19:40:50.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.503+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 49 ==== osd_op_reply(184 notify.2 [notify cookie 94621703102800 out=8b] v0'0 uv207 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.507+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 50 ==== osd_op_reply(185 notify.2 [notify-ack] v0'0 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.507+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 51 ==== watch-notify(notify_complete (2) cookie 94621703102800 notify 128849018889 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x560ed4f8abe0 con 0x560ed4f6b5e0 2023-10-05T19:40:50.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.507+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:186 6.d 6:b3423d31:::pubsub..bucket.myfoo%2f93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5022640 con 0x560ed4f35420 2023-10-05T19:40:50.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.507+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 55 ==== osd_op_reply(186 pubsub..bucket.myfoo/93ac49f4-ff49-4f99-811b-6453b28decd5.4221.5 [delete] v30'48 uv46 ondisk = -2 ((2) No such file or directory)) v8 ==== 208+0+0 (crc 0 0 0) 0x7f5054062040 con 0x560ed4f35420 2023-10-05T19:40:50.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.507+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:187 8.9 8:96d98480:users.uid::foo.buckets:head [call user.remove_bucket in=58b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5022640 con 0x560ed4f66c20 2023-10-05T19:40:50.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.511+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 46 ==== osd_op_reply(187 foo.buckets [call] v30'38 uv38 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f66c20 2023-10-05T19:40:50.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.511+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:188 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94621702493424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5022640 con 0x560ed4f66c20 2023-10-05T19:40:50.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.511+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 47 ==== osd_op_reply(188 notify.0 [watch unwatch cookie 94621702493424] v30'209 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f66c20 2023-10-05T19:40:50.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.511+0000 7f505c998e40 20 remove_watcher() i=0 2023-10-05T19:40:50.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.511+0000 7f505c998e40 2 removed watcher, disabling cache 2023-10-05T19:40:50.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.511+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:189 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94621702498944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5022640 con 0x560ed4f35420 2023-10-05T19:40:50.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.515+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 56 ==== osd_op_reply(189 notify.1 [watch unwatch cookie 94621702498944] v30'209 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5054062040 con 0x560ed4f35420 2023-10-05T19:40:50.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.515+0000 7f505c998e40 20 remove_watcher() i=1 2023-10-05T19:40:50.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.515+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:190 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94621702501856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5022640 con 0x560ed4f6b5e0 2023-10-05T19:40:50.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.519+0000 7f5051ccc700 1 -- 172.21.15.12:0/1789209205 <== osd.3 v2:172.21.15.12:6816/29951 52 ==== osd_op_reply(190 notify.2 [watch unwatch cookie 94621702501856] v30'209 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5044012450 con 0x560ed4f6b5e0 2023-10-05T19:40:50.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.519+0000 7f505c998e40 20 remove_watcher() i=2 2023-10-05T19:40:50.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.519+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:191 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94621702505376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5022640 con 0x560ed4f66c20 2023-10-05T19:40:50.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.519+0000 7f505a4cd700 1 -- 172.21.15.12:0/1789209205 <== osd.2 v2:172.21.15.12:6800/29934 48 ==== osd_op_reply(191 notify.3 [watch unwatch cookie 94621702505376] v30'209 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f504c039040 con 0x560ed4f66c20 2023-10-05T19:40:50.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.519+0000 7f505c998e40 20 remove_watcher() i=3 2023-10-05T19:40:50.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.519+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:192 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94621702500048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5022640 con 0x560ed4f35420 2023-10-05T19:40:50.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.523+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 57 ==== osd_op_reply(192 notify.4 [watch unwatch cookie 94621702500048] v30'209 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5054062040 con 0x560ed4f35420 2023-10-05T19:40:50.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.523+0000 7f505c998e40 20 remove_watcher() i=4 2023-10-05T19:40:50.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.523+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:193 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94621702506544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5022640 con 0x7f504c01a070 2023-10-05T19:40:50.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.527+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.0 v2:172.21.15.8:6800/27563 44 ==== osd_op_reply(193 notify.5 [watch unwatch cookie 94621702506544] v30'209 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5054062040 con 0x7f504c01a070 2023-10-05T19:40:50.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.527+0000 7f505c998e40 20 remove_watcher() i=5 2023-10-05T19:40:50.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.527+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:194 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94621702512832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5022640 con 0x7f504c01a070 2023-10-05T19:40:50.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.527+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.0 v2:172.21.15.8:6800/27563 45 ==== osd_op_reply(194 notify.6 [watch unwatch cookie 94621702512832] v30'209 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5054062040 con 0x7f504c01a070 2023-10-05T19:40:50.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.527+0000 7f505c998e40 20 remove_watcher() i=6 2023-10-05T19:40:50.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.527+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:195 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94621702511296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5022640 con 0x7f504c01a070 2023-10-05T19:40:50.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.531+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.0 v2:172.21.15.8:6800/27563 46 ==== osd_op_reply(195 notify.7 [watch unwatch cookie 94621702511296] v30'209 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5054062040 con 0x7f504c01a070 2023-10-05T19:40:50.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.531+0000 7f505c998e40 20 remove_watcher() i=7 2023-10-05T19:40:50.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.531+0000 7f502bfdf700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:50.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.535+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:196 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94621702517120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560ed5022640 con 0x560ed4f35420 2023-10-05T19:40:50.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.535+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 <== osd.1 v2:172.21.15.12:6808/29933 58 ==== osd_op_reply(196 data_loggenerations_metadata [watch unwatch cookie 94621702517120] v30'207 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f5054062040 con 0x560ed4f35420 2023-10-05T19:40:50.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.535+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f504c01a070 msgr2=0x7f504c042220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.535+0000 7f505c998e40 1 --2- 172.21.15.12:0/1789209205 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f504c01a070 0x7f504c042220 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:50.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.535+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x560ed4f35420 msgr2=0x560ed4f649d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.535+0000 7f505c998e40 1 --2- 172.21.15.12:0/1789209205 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x560ed4f35420 0x560ed4f649d0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:50.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.535+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x560ed4f66c20 msgr2=0x560ed4f69000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.535+0000 7f505c998e40 1 --2- 172.21.15.12:0/1789209205 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x560ed4f66c20 0x560ed4f69000 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:50.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.535+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x560ed4f6b5e0 msgr2=0x560ed4f6d9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.535+0000 7f505c998e40 1 --2- 172.21.15.12:0/1789209205 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x560ed4f6b5e0 0x560ed4f6d9f0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:50.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.535+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f503c030c50 msgr2=0x7f503c033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.535+0000 7f505c998e40 1 --2- 172.21.15.12:0/1789209205 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f503c030c50 0x7f503c033100 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f50440030a0 tx=0x7f5044015000 comp rx=0 tx=0).stop 2023-10-05T19:40:50.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.535+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560ed4f30e40 msgr2=0x560ed4f58ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.535+0000 7f505acce700 1 -- 172.21.15.12:0/1789209205 reap_dead start 2023-10-05T19:40:50.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.535+0000 7f505c998e40 1 --2- 172.21.15.12:0/1789209205 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560ed4f30e40 0x560ed4f58ee0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f504c000c00 tx=0x7f504c016000 comp rx=0 tx=0).stop 2023-10-05T19:40:50.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.535+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 shutdown_connections 2023-10-05T19:40:50.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.535+0000 7f505c998e40 1 --2- 172.21.15.12:0/1789209205 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560ed4f30e40 0x560ed4f58ee0 secure :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f504c000c00 tx=0x7f504c016000 comp rx=0 tx=0).stop 2023-10-05T19:40:50.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.535+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 >> 172.21.15.12:0/1789209205 conn(0x560ed4f2c860 msgr2=0x560ed4f4a550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:50.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.535+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 shutdown_connections 2023-10-05T19:40:50.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.535+0000 7f505c998e40 1 -- 172.21.15.12:0/1789209205 wait complete. 2023-10-05T19:40:50.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.539+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x560ed4f98950 msgr2=0x560ed4f9ad30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.539+0000 7f505c998e40 1 --2- 172.21.15.12:0/2961973426 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x560ed4f98950 0x560ed4f9ad30 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:50.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.539+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x560ed4f951d0 msgr2=0x560ed4f975e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.539+0000 7f505c998e40 1 --2- 172.21.15.12:0/2961973426 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x560ed4f951d0 0x560ed4f975e0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:50.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.539+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x560ed4f9ff60 msgr2=0x560ed4fa2340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.539+0000 7f505c998e40 1 --2- 172.21.15.12:0/2961973426 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x560ed4f9ff60 0x560ed4fa2340 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:50.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.539+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x560ed4f9c130 msgr2=0x560ed4f9e540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.539+0000 7f505c998e40 1 --2- 172.21.15.12:0/2961973426 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x560ed4f9c130 0x560ed4f9e540 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:50.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.539+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f4fa0030c50 msgr2=0x7f4fa0033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.539+0000 7f505c998e40 1 --2- 172.21.15.12:0/2961973426 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f4fa0030c50 0x7f4fa0033100 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x560ed4f76870 tx=0x7f505400af20 comp rx=0 tx=0).stop 2023-10-05T19:40:50.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.539+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560ed4f84b40 msgr2=0x560ed4f8a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.539+0000 7f505c998e40 1 --2- 172.21.15.12:0/2961973426 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560ed4f84b40 0x560ed4f8a5e0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5044008fd0 tx=0x7f5044033000 comp rx=0 tx=0).stop 2023-10-05T19:40:50.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.539+0000 7f505a4cd700 1 -- 172.21.15.12:0/2961973426 reap_dead start 2023-10-05T19:40:50.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.539+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 shutdown_connections 2023-10-05T19:40:50.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.539+0000 7f505c998e40 1 --2- 172.21.15.12:0/2961973426 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560ed4f84b40 0x560ed4f8a5e0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:50.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.539+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 >> 172.21.15.12:0/2961973426 conn(0x560ed4f7ff00 msgr2=0x560ed4f7dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:50.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.539+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 shutdown_connections 2023-10-05T19:40:50.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.539+0000 7f505c998e40 1 -- 172.21.15.12:0/2961973426 wait complete. 2023-10-05T19:40:50.551 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'add', '--uid', 'foo', '--caps', 'user=read'] 2023-10-05T19:40:50.551 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'add', '--uid', 'foo', '--caps', 'user=read'] 2023-10-05T19:40:50.551 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps add --uid foo --caps user=read 2023-10-05T19:40:50.593 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:50.593 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:50.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.587+0000 7f3cd0a04e40 1 Processor -- start 2023-10-05T19:40:50.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.587+0000 7f3cd0a04e40 1 -- start start 2023-10-05T19:40:50.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.587+0000 7f3cd0a04e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55fb2b85c420 0x55fb2b85afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:50.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.587+0000 7f3cd0a04e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55fb2b7a8bd0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.587+0000 7f3cd0a04e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x55fb2b765ee0 con 0x55fb2b85c420 2023-10-05T19:40:50.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.587+0000 7f3cce539700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55fb2b85c420 0x55fb2b85afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:50.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.587+0000 7f3ccdd38700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x55fb2b85c7f0 0x55fb2b85b510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:60264/0 (socket says 172.21.15.12:60264) 2023-10-05T19:40:50.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.587+0000 7f3ccdd38700 1 -- 172.21.15.12:0/2868443363 learned_addr learned my addr 172.21.15.12:0/2868443363 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:50.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.587+0000 7f3cce539700 1 --2- 172.21.15.12:0/2868443363 >> v2:172.21.15.8:3300/0 conn(0x55fb2b85c420 0x55fb2b85afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58620/0 (socket says 172.21.15.12:58620) 2023-10-05T19:40:50.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.587+0000 7f3cce539700 1 -- 172.21.15.12:0/2868443363 >> v1:172.21.15.8:6789/0 conn(0x55fb2b85c7f0 legacy=0x55fb2b85b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.587+0000 7f3cce539700 1 -- 172.21.15.12:0/2868443363 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb2b85bce0 con 0x55fb2b85c420 2023-10-05T19:40:50.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.587+0000 7f3cce539700 1 --2- 172.21.15.12:0/2868443363 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fb2b85c420 0x55fb2b85afd0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f3cc000a5d0 tx=0x7f3cc0005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb58a9a186717a8b server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:50.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.587+0000 7f3ccd537700 1 -- 172.21.15.12:0/2868443363 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3cc00086a0 con 0x55fb2b85c420 2023-10-05T19:40:50.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.587+0000 7f3ccd537700 1 -- 172.21.15.12:0/2868443363 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc0008800 con 0x55fb2b85c420 2023-10-05T19:40:50.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.587+0000 7f3ccd537700 1 -- 172.21.15.12:0/2868443363 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3cc0008ad0 con 0x55fb2b85c420 2023-10-05T19:40:50.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.587+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/2868443363 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fb2b85c420 msgr2=0x55fb2b85afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.591+0000 7f3cd0a04e40 1 --2- 172.21.15.12:0/2868443363 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fb2b85c420 0x55fb2b85afd0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f3cc000a5d0 tx=0x7f3cc0005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:50.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.591+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/2868443363 shutdown_connections 2023-10-05T19:40:50.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.591+0000 7f3cd0a04e40 1 --2- 172.21.15.12:0/2868443363 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fb2b85c420 0x55fb2b85afd0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:50.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.591+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/2868443363 >> 172.21.15.12:0/2868443363 conn(0x55fb2b856730 msgr2=0x55fb2b858b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:50.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.591+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/2868443363 shutdown_connections 2023-10-05T19:40:50.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.591+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/2868443363 wait complete. 2023-10-05T19:40:50.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.591+0000 7f3cd0a04e40 1 Processor -- start 2023-10-05T19:40:50.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.591+0000 7f3cd0a04e40 1 -- start start 2023-10-05T19:40:50.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.591+0000 7f3cd0a04e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fb2b85c420 0x55fb2b87d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:50.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.591+0000 7f3cd0a04e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb2b729e30 con 0x55fb2b85c420 2023-10-05T19:40:50.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.591+0000 7f3cce539700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fb2b85c420 0x55fb2b87d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:50.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.591+0000 7f3cce539700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fb2b85c420 0x55fb2b87d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58630/0 (socket says 172.21.15.12:58630) 2023-10-05T19:40:50.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.591+0000 7f3cce539700 1 -- 172.21.15.12:0/2118726725 learned_addr learned my addr 172.21.15.12:0/2118726725 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:50.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.591+0000 7f3cce539700 1 -- 172.21.15.12:0/2118726725 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc0005cf0 con 0x55fb2b85c420 2023-10-05T19:40:50.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.591+0000 7f3cce539700 1 --2- 172.21.15.12:0/2118726725 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fb2b85c420 0x55fb2b87d160 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0000c00 tx=0x7f3cc0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7f41a5f36119a0d server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:50.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.591+0000 7f3cc7fff700 1 -- 172.21.15.12:0/2118726725 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3cc0008970 con 0x55fb2b85c420 2023-10-05T19:40:50.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.591+0000 7f3cc7fff700 1 -- 172.21.15.12:0/2118726725 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc0008af0 con 0x55fb2b85c420 2023-10-05T19:40:50.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.591+0000 7f3cc7fff700 1 -- 172.21.15.12:0/2118726725 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3cc0008de0 con 0x55fb2b85c420 2023-10-05T19:40:50.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.591+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/2118726725 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fb2b85c420 msgr2=0x55fb2b87d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.591+0000 7f3cd0a04e40 1 --2- 172.21.15.12:0/2118726725 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fb2b85c420 0x55fb2b87d160 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0000c00 tx=0x7f3cc0015000 comp rx=0 tx=0).stop 2023-10-05T19:40:50.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.591+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/2118726725 shutdown_connections 2023-10-05T19:40:50.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.591+0000 7f3cd0a04e40 1 --2- 172.21.15.12:0/2118726725 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fb2b85c420 0x55fb2b87d160 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:50.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.591+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/2118726725 >> 172.21.15.12:0/2118726725 conn(0x55fb2b856730 msgr2=0x55fb2b878d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:50.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.595+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/2118726725 shutdown_connections 2023-10-05T19:40:50.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.595+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/2118726725 wait complete. 2023-10-05T19:40:50.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.595+0000 7f3cd0a04e40 1 Processor -- start 2023-10-05T19:40:50.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.595+0000 7f3cd0a04e40 1 -- start start 2023-10-05T19:40:50.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.595+0000 7f3cd0a04e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fb2b85c420 0x55fb2b883d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:50.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.595+0000 7f3cd0a04e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb2b6fced0 con 0x55fb2b85c420 2023-10-05T19:40:50.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.595+0000 7f3cce539700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fb2b85c420 0x55fb2b883d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:50.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.595+0000 7f3cce539700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fb2b85c420 0x55fb2b883d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58646/0 (socket says 172.21.15.12:58646) 2023-10-05T19:40:50.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.595+0000 7f3cce539700 1 -- 172.21.15.12:0/757573675 learned_addr learned my addr 172.21.15.12:0/757573675 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:50.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.595+0000 7f3cce539700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb2b85bce0 con 0x55fb2b85c420 2023-10-05T19:40:50.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.595+0000 7f3cce539700 1 --2- 172.21.15.12:0/757573675 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fb2b85c420 0x55fb2b883d70 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0000c00 tx=0x7f3cc0003050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:50.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.595+0000 7f3cc67fc700 1 -- 172.21.15.12:0/757573675 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3cc0012b70 con 0x55fb2b85c420 2023-10-05T19:40:50.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.595+0000 7f3cc67fc700 1 -- 172.21.15.12:0/757573675 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc002a070 con 0x55fb2b85c420 2023-10-05T19:40:50.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.595+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55fb2b8842b0 con 0x55fb2b85c420 2023-10-05T19:40:50.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.595+0000 7f3cc67fc700 1 -- 172.21.15.12:0/757573675 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3cc0012e50 con 0x55fb2b85c420 2023-10-05T19:40:50.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.595+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb2b884750 con 0x55fb2b85c420 2023-10-05T19:40:50.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.599+0000 7f3cd0a04e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.599+0000 7f3cc67fc700 1 -- 172.21.15.12:0/757573675 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f3cc0019070 con 0x55fb2b85c420 2023-10-05T19:40:50.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.599+0000 7f3cc67fc700 1 --2- 172.21.15.12:0/757573675 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3cb0030b10 0x7f3cb0032fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:50.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.599+0000 7f3cc67fc700 1 -- 172.21.15.12:0/757573675 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7f3cc002c070 con 0x55fb2b85c420 2023-10-05T19:40:50.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.599+0000 7f3ccdd38700 1 --2- 172.21.15.12:0/757573675 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3cb0030b10 0x7f3cb0032fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:50.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.599+0000 7f3cd0a04e40 1 --2- 172.21.15.12:0/757573675 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55fb2b85c7f0 0x55fb2b88f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:50.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.599+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b88ff80 con 0x55fb2b85c7f0 2023-10-05T19:40:50.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.599+0000 7f3cced3a700 1 --2- 172.21.15.12:0/757573675 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55fb2b85c7f0 0x55fb2b88f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:50.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.599+0000 7f3ccdd38700 1 --2- 172.21.15.12:0/757573675 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3cb0030b10 0x7f3cb0032fc0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8001e90 tx=0x7f3cb80013b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:50.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.599+0000 7f3cced3a700 1 --2- 172.21.15.12:0/757573675 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55fb2b85c7f0 0x55fb2b88f960 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:50.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.599+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3cc800a5a0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.599+0000 7f3cd0a04e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:50.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.599+0000 7f3cd0a04e40 20 realm 2023-10-05T19:40:50.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.599+0000 7f3cd0a04e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.599+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b88cfc0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.603+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3cc800a5a0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.603+0000 7f3cd0a04e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:50.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.603+0000 7f3cd0a04e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:50.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.603+0000 7f3cd0a04e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.603+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b890a60 con 0x55fb2b85c7f0 2023-10-05T19:40:50.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.603+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3cc800a5a0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.603+0000 7f3cd0a04e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:50.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.603+0000 7f3cd0a04e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.603+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b890670 con 0x55fb2b85c7f0 2023-10-05T19:40:50.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.603+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f3cc800a5a0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.603+0000 7f3cd0a04e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:50.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.603+0000 7f3cd0a04e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.603+0000 7f3cd0a04e40 1 --2- 172.21.15.12:0/757573675 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55fb2b891e00 0x55fb2b8941e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:50.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.603+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b894720 con 0x55fb2b891e00 2023-10-05T19:40:50.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.603+0000 7f3cce539700 1 --2- 172.21.15.12:0/757573675 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55fb2b891e00 0x55fb2b8941e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:50.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.603+0000 7f3cce539700 1 --2- 172.21.15.12:0/757573675 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55fb2b891e00 0x55fb2b8941e0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:50.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.603+0000 7f3cce539700 1 -- 172.21.15.12:0/757573675 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f3cc0022020 con 0x55fb2b891e00 2023-10-05T19:40:50.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.603+0000 7f3cd0a04e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:50.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.603+0000 7f3cd0a04e40 20 searching for the correct realm 2023-10-05T19:40:50.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.607+0000 7f3cd0a04e40 1 --2- 172.21.15.12:0/757573675 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55fb2b896510 0x55fb2b898920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:50.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.607+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b898fd0 con 0x55fb2b896510 2023-10-05T19:40:50.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.607+0000 7f3ccdd38700 1 --2- 172.21.15.12:0/757573675 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55fb2b896510 0x55fb2b898920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:50.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.607+0000 7f3ccdd38700 1 --2- 172.21.15.12:0/757573675 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55fb2b896510 0x55fb2b898920 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:50.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.607+0000 7f3ccdd38700 1 -- 172.21.15.12:0/757573675 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f3cb8006070 con 0x55fb2b896510 2023-10-05T19:40:50.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.607+0000 7f3ccdd38700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b898fd0 con 0x55fb2b891e00 2023-10-05T19:40:50.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.607+0000 7f3cce539700 1 -- 172.21.15.12:0/757573675 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cc0022020 con 0x55fb2b891e00 2023-10-05T19:40:50.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.607+0000 7f3cce539700 1 --2- 172.21.15.12:0/757573675 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f3cc0007030 0x7f3cc0031480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:50.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.607+0000 7f3cce539700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b898fd0 con 0x7f3cc0007030 2023-10-05T19:40:50.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.611+0000 7f3cced3a700 1 --2- 172.21.15.12:0/757573675 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f3cc0007030 0x7f3cc0031480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:50.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.611+0000 7f3cced3a700 1 --2- 172.21.15.12:0/757573675 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f3cc0007030 0x7f3cc0031480 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:50.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.611+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cc800a5a0 con 0x7f3cc0007030 2023-10-05T19:40:50.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.611+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b898fd0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.611+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cc800a5a0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.611+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b898fd0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.611+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cc800a5a0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.611+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b898fd0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.611+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(11 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cc800a5a0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.611+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b898fd0 con 0x7f3cc0007030 2023-10-05T19:40:50.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.615+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cc800a5a0 con 0x7f3cc0007030 2023-10-05T19:40:50.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.615+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b898fd0 con 0x55fb2b891e00 2023-10-05T19:40:50.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.615+0000 7f3cce539700 1 -- 172.21.15.12:0/757573675 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cc0022020 con 0x55fb2b891e00 2023-10-05T19:40:50.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.615+0000 7f3cce539700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b898fd0 con 0x55fb2b891e00 2023-10-05T19:40:50.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.615+0000 7f3cce539700 1 -- 172.21.15.12:0/757573675 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cc0022020 con 0x55fb2b891e00 2023-10-05T19:40:50.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.615+0000 7f3cce539700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b898fd0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.615+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cc800a5a0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.615+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b898fd0 con 0x7f3cc0007030 2023-10-05T19:40:50.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.615+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cc800a5a0 con 0x7f3cc0007030 2023-10-05T19:40:50.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.615+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b898fd0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.615+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cc800a5a0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.615+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b898fd0 con 0x55fb2b891e00 2023-10-05T19:40:50.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.619+0000 7f3cce539700 1 -- 172.21.15.12:0/757573675 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(18 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cc0022020 con 0x55fb2b891e00 2023-10-05T19:40:50.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.619+0000 7f3cce539700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b898fd0 con 0x55fb2b896510 2023-10-05T19:40:50.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.619+0000 7f3ccdd38700 1 -- 172.21.15.12:0/757573675 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cb8006070 con 0x55fb2b896510 2023-10-05T19:40:50.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.619+0000 7f3ccdd38700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b898fd0 con 0x55fb2b896510 2023-10-05T19:40:50.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.619+0000 7f3ccdd38700 1 -- 172.21.15.12:0/757573675 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cb8006070 con 0x55fb2b896510 2023-10-05T19:40:50.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.619+0000 7f3ccdd38700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b898fd0 con 0x7f3cc0007030 2023-10-05T19:40:50.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.619+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cc800a5a0 con 0x7f3cc0007030 2023-10-05T19:40:50.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.619+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b898fd0 con 0x55fb2b896510 2023-10-05T19:40:50.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.623+0000 7f3ccdd38700 1 -- 172.21.15.12:0/757573675 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cb8006070 con 0x55fb2b896510 2023-10-05T19:40:50.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.623+0000 7f3ccdd38700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b898fd0 con 0x55fb2b896510 2023-10-05T19:40:50.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.623+0000 7f3ccdd38700 1 -- 172.21.15.12:0/757573675 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cb8006070 con 0x55fb2b896510 2023-10-05T19:40:50.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.623+0000 7f3ccdd38700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b898fd0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.623+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cc800a5a0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.623+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b898fd0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.623+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cc800a5a0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.623+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b898fd0 con 0x7f3cc0007030 2023-10-05T19:40:50.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.623+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cc800a5a0 con 0x7f3cc0007030 2023-10-05T19:40:50.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.623+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b898fd0 con 0x55fb2b896510 2023-10-05T19:40:50.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.627+0000 7f3ccdd38700 1 -- 172.21.15.12:0/757573675 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cb8006070 con 0x55fb2b896510 2023-10-05T19:40:50.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.627+0000 7f3ccdd38700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b898fd0 con 0x55fb2b891e00 2023-10-05T19:40:50.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.627+0000 7f3cce539700 1 -- 172.21.15.12:0/757573675 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cc0022020 con 0x55fb2b891e00 2023-10-05T19:40:50.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.627+0000 7f3cce539700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b898fd0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.627+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cc800a5a0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.627+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b898fd0 con 0x7f3cc0007030 2023-10-05T19:40:50.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.627+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cc800a5a0 con 0x7f3cc0007030 2023-10-05T19:40:50.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.627+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b898fd0 con 0x55fb2b896510 2023-10-05T19:40:50.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.631+0000 7f3ccdd38700 1 -- 172.21.15.12:0/757573675 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cb8006070 con 0x55fb2b896510 2023-10-05T19:40:50.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.631+0000 7f3ccdd38700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b898fd0 con 0x7f3cc0007030 2023-10-05T19:40:50.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.631+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cc800a5a0 con 0x7f3cc0007030 2023-10-05T19:40:50.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.631+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b898fd0 con 0x55fb2b891e00 2023-10-05T19:40:50.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.631+0000 7f3cce539700 1 -- 172.21.15.12:0/757573675 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cc0022020 con 0x55fb2b891e00 2023-10-05T19:40:50.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.631+0000 7f3cce539700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b898fd0 con 0x55fb2b896510 2023-10-05T19:40:50.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.631+0000 7f3ccdd38700 1 -- 172.21.15.12:0/757573675 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cb8006070 con 0x55fb2b896510 2023-10-05T19:40:50.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.631+0000 7f3ccdd38700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b898fd0 con 0x7f3cc0007030 2023-10-05T19:40:50.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.635+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cc800a5a0 con 0x7f3cc0007030 2023-10-05T19:40:50.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.635+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b898fd0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.635+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f3cc800a5a0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.635+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b898fd0 con 0x55fb2b891e00 2023-10-05T19:40:50.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.635+0000 7f3cce539700 1 -- 172.21.15.12:0/757573675 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(37 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f3cc0022020 con 0x55fb2b891e00 2023-10-05T19:40:50.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.635+0000 7f3cd0a04e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:50.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.635+0000 7f3cd0a04e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:50.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.635+0000 7f3cd0a04e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:50.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.635+0000 7f3cd0a04e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:50.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.635+0000 7f3cd0a04e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.635+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b8952c0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.635+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3cc800a5a0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.635+0000 7f3cd0a04e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:50.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.635+0000 7f3cd0a04e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.635+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b894ef0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.635+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f3cc800a5a0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.635+0000 7f3cd0a04e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:50.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.635+0000 7f3cd0a04e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.635+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b899440 con 0x55fb2b896510 2023-10-05T19:40:50.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.639+0000 7f3ccdd38700 1 -- 172.21.15.12:0/757573675 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f3cb8006070 con 0x55fb2b896510 2023-10-05T19:40:50.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.639+0000 7f3cd0a04e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:50.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.639+0000 7f3cd0a04e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.639+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b8997f0 con 0x55fb2b896510 2023-10-05T19:40:50.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.639+0000 7f3ccdd38700 1 -- 172.21.15.12:0/757573675 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f3cb8006070 con 0x55fb2b896510 2023-10-05T19:40:50.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.639+0000 7f3cd0a04e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:50.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.639+0000 7f3cd0a04e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.639+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b899ba0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.639+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3cc800a5a0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.639+0000 7f3cd0a04e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:50.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.639+0000 7f3cd0a04e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.639+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b8957b0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.639+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f3cc800a5a0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.639+0000 7f3cd0a04e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:50.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.639+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b895780 con 0x55fb2b896510 2023-10-05T19:40:50.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.643+0000 7f3ccdd38700 1 -- 172.21.15.12:0/757573675 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(44 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f3cb8006070 con 0x55fb2b896510 2023-10-05T19:40:50.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.643+0000 7f3ccdd38700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b895780 con 0x55fb2b891e00 2023-10-05T19:40:50.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.643+0000 7f3cce539700 1 -- 172.21.15.12:0/757573675 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(45 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cc0022020 con 0x55fb2b891e00 2023-10-05T19:40:50.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.643+0000 7f3cce539700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b895780 con 0x7f3cc0007030 2023-10-05T19:40:50.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.643+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(46 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cc800a5a0 con 0x7f3cc0007030 2023-10-05T19:40:50.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.643+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b899ba0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.647+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(47 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cc800a5a0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.647+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b899ba0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.647+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(48 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cc800a5a0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.647+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b899ba0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.647+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(49 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cc800a5a0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.647+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b899ba0 con 0x7f3cc0007030 2023-10-05T19:40:50.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.647+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(50 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cc800a5a0 con 0x7f3cc0007030 2023-10-05T19:40:50.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.647+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b899ba0 con 0x55fb2b891e00 2023-10-05T19:40:50.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.651+0000 7f3cce539700 1 -- 172.21.15.12:0/757573675 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(51 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cc0022020 con 0x55fb2b891e00 2023-10-05T19:40:50.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.651+0000 7f3cce539700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b899ba0 con 0x55fb2b891e00 2023-10-05T19:40:50.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.651+0000 7f3cce539700 1 -- 172.21.15.12:0/757573675 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(52 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cc0022020 con 0x55fb2b891e00 2023-10-05T19:40:50.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.651+0000 7f3cce539700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b899ba0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.651+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(53 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cc800a5a0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.651+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b899ba0 con 0x7f3cc0007030 2023-10-05T19:40:50.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.651+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(54 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cc800a5a0 con 0x7f3cc0007030 2023-10-05T19:40:50.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.651+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b899ba0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.655+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(55 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cc800a5a0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.655+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b899ba0 con 0x55fb2b891e00 2023-10-05T19:40:50.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.655+0000 7f3cce539700 1 -- 172.21.15.12:0/757573675 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(56 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cc0022020 con 0x55fb2b891e00 2023-10-05T19:40:50.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.655+0000 7f3cce539700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b899ba0 con 0x55fb2b896510 2023-10-05T19:40:50.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.655+0000 7f3ccdd38700 1 -- 172.21.15.12:0/757573675 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(57 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cb8006070 con 0x55fb2b896510 2023-10-05T19:40:50.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.655+0000 7f3ccdd38700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b899ba0 con 0x55fb2b896510 2023-10-05T19:40:50.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.655+0000 7f3ccdd38700 1 -- 172.21.15.12:0/757573675 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(58 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cb8006070 con 0x55fb2b896510 2023-10-05T19:40:50.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.655+0000 7f3ccdd38700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b899ba0 con 0x7f3cc0007030 2023-10-05T19:40:50.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.655+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(59 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cc800a5a0 con 0x7f3cc0007030 2023-10-05T19:40:50.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.655+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b899ba0 con 0x55fb2b896510 2023-10-05T19:40:50.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.655+0000 7f3ccdd38700 1 -- 172.21.15.12:0/757573675 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(60 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cb8006070 con 0x55fb2b896510 2023-10-05T19:40:50.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.655+0000 7f3ccdd38700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b899ba0 con 0x55fb2b896510 2023-10-05T19:40:50.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.659+0000 7f3ccdd38700 1 -- 172.21.15.12:0/757573675 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(61 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cb8006070 con 0x55fb2b896510 2023-10-05T19:40:50.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.659+0000 7f3ccdd38700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b899ba0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.659+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(62 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cc800a5a0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.659+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b899ba0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.659+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(63 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cc800a5a0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.659+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b899ba0 con 0x7f3cc0007030 2023-10-05T19:40:50.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.659+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(64 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cc800a5a0 con 0x7f3cc0007030 2023-10-05T19:40:50.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.659+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b899ba0 con 0x55fb2b896510 2023-10-05T19:40:50.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.659+0000 7f3ccdd38700 1 -- 172.21.15.12:0/757573675 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(65 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cb8006070 con 0x55fb2b896510 2023-10-05T19:40:50.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.659+0000 7f3ccdd38700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b899ba0 con 0x55fb2b891e00 2023-10-05T19:40:50.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.659+0000 7f3cce539700 1 -- 172.21.15.12:0/757573675 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(66 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cc0022020 con 0x55fb2b891e00 2023-10-05T19:40:50.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.659+0000 7f3cce539700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b899ba0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.663+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(67 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cc800a5a0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.663+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b899ba0 con 0x7f3cc0007030 2023-10-05T19:40:50.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.663+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(68 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cc800a5a0 con 0x7f3cc0007030 2023-10-05T19:40:50.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.663+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b899ba0 con 0x55fb2b896510 2023-10-05T19:40:50.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.663+0000 7f3ccdd38700 1 -- 172.21.15.12:0/757573675 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(69 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cb8006070 con 0x55fb2b896510 2023-10-05T19:40:50.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.663+0000 7f3ccdd38700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b899ba0 con 0x7f3cc0007030 2023-10-05T19:40:50.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.663+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(70 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cc800a5a0 con 0x7f3cc0007030 2023-10-05T19:40:50.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.663+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b899ba0 con 0x55fb2b891e00 2023-10-05T19:40:50.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.663+0000 7f3cce539700 1 -- 172.21.15.12:0/757573675 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(71 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cc0022020 con 0x55fb2b891e00 2023-10-05T19:40:50.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.663+0000 7f3cce539700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b899ba0 con 0x55fb2b896510 2023-10-05T19:40:50.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.667+0000 7f3ccdd38700 1 -- 172.21.15.12:0/757573675 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(72 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cb8006070 con 0x55fb2b896510 2023-10-05T19:40:50.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.667+0000 7f3ccdd38700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b899ba0 con 0x7f3cc0007030 2023-10-05T19:40:50.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.667+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(73 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3cc800a5a0 con 0x7f3cc0007030 2023-10-05T19:40:50.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.667+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b899ba0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.667+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(74 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f3cc800a5a0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.667+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b899ba0 con 0x55fb2b891e00 2023-10-05T19:40:50.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.667+0000 7f3cce539700 1 -- 172.21.15.12:0/757573675 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(75 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f3cc0022020 con 0x55fb2b891e00 2023-10-05T19:40:50.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.667+0000 7f3cd0a04e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:50.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.667+0000 7f3cd0a04e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:50.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.667+0000 7f3cd0a04e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:50.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.667+0000 7f3cd0a04e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:50.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.667+0000 7f3cd0a04e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.667+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b895780 con 0x55fb2b85c7f0 2023-10-05T19:40:50.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.667+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f3cc800a5a0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.667+0000 7f3cd0a04e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:50.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.667+0000 7f3cd0a04e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.667+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b89a4e0 con 0x55fb2b891e00 2023-10-05T19:40:50.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.671+0000 7f3cce539700 1 -- 172.21.15.12:0/757573675 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f3cc0022020 con 0x55fb2b891e00 2023-10-05T19:40:50.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.671+0000 7f3cd0a04e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:50.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.671+0000 7f3cd0a04e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.671+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b89ad80 con 0x55fb2b85c7f0 2023-10-05T19:40:50.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.671+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f3cc800a5a0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.671+0000 7f3cd0a04e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:50.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.671+0000 7f3cd0a04e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.671+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b89ad80 con 0x55fb2b896510 2023-10-05T19:40:50.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.671+0000 7f3ccdd38700 1 -- 172.21.15.12:0/757573675 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f3cb8006070 con 0x55fb2b896510 2023-10-05T19:40:50.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.671+0000 7f3cd0a04e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:50.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.671+0000 7f3cd0a04e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.671+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b89b620 con 0x7f3cc0007030 2023-10-05T19:40:50.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.671+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f3cc800a5a0 con 0x7f3cc0007030 2023-10-05T19:40:50.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.671+0000 7f3cd0a04e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:50.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.671+0000 7f3cd0a04e40 20 zone default found 2023-10-05T19:40:50.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.671+0000 7f3cd0a04e40 4 Realm: () 2023-10-05T19:40:50.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.671+0000 7f3cd0a04e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:50.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.671+0000 7f3cd0a04e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:50.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.671+0000 7f3cd0a04e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:50.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.671+0000 7f3cd0a04e40 20 zonegroup default 2023-10-05T19:40:50.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.671+0000 7f3cd0a04e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.671+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b89b620 con 0x7f3cc0007030 2023-10-05T19:40:50.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.671+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f3cc800a5a0 con 0x7f3cc0007030 2023-10-05T19:40:50.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.671+0000 7f3cd0a04e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:50.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.671+0000 7f3cd0a04e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.671+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b89cfa0 con 0x55fb2b896510 2023-10-05T19:40:50.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.675+0000 7f3ccdd38700 1 -- 172.21.15.12:0/757573675 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f3cb8006070 con 0x55fb2b896510 2023-10-05T19:40:50.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.675+0000 7f3cd0a04e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:50.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.675+0000 7f3cd0a04e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.675+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b89d350 con 0x55fb2b896510 2023-10-05T19:40:50.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.675+0000 7f3ccdd38700 1 -- 172.21.15.12:0/757573675 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f3cb8006070 con 0x55fb2b896510 2023-10-05T19:40:50.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.675+0000 7f3cd0a04e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:50.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.675+0000 7f3cd0a04e40 20 started sync module instance, tier type = 2023-10-05T19:40:50.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.675+0000 7f3cd0a04e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:50.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.675+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b89c660 con 0x55fb2b891e00 2023-10-05T19:40:50.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.679+0000 7f3cce539700 1 -- 172.21.15.12:0/757573675 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(84 notify.0 [create] v30'210 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3cc0022020 con 0x55fb2b891e00 2023-10-05T19:40:50.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.679+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94537255600496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b89d700 con 0x55fb2b891e00 2023-10-05T19:40:50.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.679+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b89dfa0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.679+0000 7f3cce539700 1 -- 172.21.15.12:0/757573675 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(85 notify.0 [watch watch cookie 94537255600496] v30'211 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3cc0022020 con 0x55fb2b891e00 2023-10-05T19:40:50.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.679+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(86 notify.1 [create] v30'210 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3cc800a5a0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.679+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94537255606016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b89cb80 con 0x55fb2b85c7f0 2023-10-05T19:40:50.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.679+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b89e740 con 0x55fb2b896510 2023-10-05T19:40:50.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.683+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(87 notify.1 [watch watch cookie 94537255606016] v30'211 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3cc800a5a0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.683+0000 7f3ccdd38700 1 -- 172.21.15.12:0/757573675 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(88 notify.2 [create] v30'210 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3cb8006070 con 0x55fb2b896510 2023-10-05T19:40:50.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.683+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94537255607120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b89eb40 con 0x55fb2b896510 2023-10-05T19:40:50.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.683+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b89f550 con 0x55fb2b891e00 2023-10-05T19:40:50.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.687+0000 7f3ccdd38700 1 -- 172.21.15.12:0/757573675 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(89 notify.2 [watch watch cookie 94537255607120] v30'211 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3cb8006070 con 0x55fb2b896510 2023-10-05T19:40:50.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.687+0000 7f3cce539700 1 -- 172.21.15.12:0/757573675 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(90 notify.3 [create] v30'210 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3cc0022020 con 0x55fb2b891e00 2023-10-05T19:40:50.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.687+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94537255612512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b89e330 con 0x55fb2b891e00 2023-10-05T19:40:50.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.687+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b8a0090 con 0x55fb2b85c7f0 2023-10-05T19:40:50.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.691+0000 7f3cce539700 1 -- 172.21.15.12:0/757573675 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(91 notify.3 [watch watch cookie 94537255612512] v30'211 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3cc0022020 con 0x55fb2b891e00 2023-10-05T19:40:50.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.691+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(92 notify.4 [create] v30'210 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3cc800a5a0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.691+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94537255613616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b8a0090 con 0x55fb2b85c7f0 2023-10-05T19:40:50.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.691+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b8a0af0 con 0x7f3cc0007030 2023-10-05T19:40:50.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.695+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(94 notify.5 [create] v30'210 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3cc800a5a0 con 0x7f3cc0007030 2023-10-05T19:40:50.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.695+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94537255618048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b8a0a50 con 0x7f3cc0007030 2023-10-05T19:40:50.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.695+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b8a1a40 con 0x7f3cc0007030 2023-10-05T19:40:50.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.695+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(93 notify.4 [watch watch cookie 94537255613616] v30'211 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3cc800a5a0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.695+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(95 notify.5 [watch watch cookie 94537255618048] v30'211 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3cc800a5a0 con 0x7f3cc0007030 2023-10-05T19:40:50.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.695+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(96 notify.6 [create] v30'210 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3cc800a5a0 con 0x7f3cc0007030 2023-10-05T19:40:50.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.695+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94537255615392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b8a1550 con 0x7f3cc0007030 2023-10-05T19:40:50.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.695+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b8a2330 con 0x7f3cc0007030 2023-10-05T19:40:50.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.699+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(97 notify.6 [watch watch cookie 94537255615392] v30'211 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3cc800a5a0 con 0x7f3cc0007030 2023-10-05T19:40:50.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.699+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(98 notify.7 [create] v30'210 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3cc800a5a0 con 0x7f3cc0007030 2023-10-05T19:40:50.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.699+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94537255619712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b8a20d0 con 0x7f3cc0007030 2023-10-05T19:40:50.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.699+0000 7f3cd0a04e40 20 add_watcher() i=0 2023-10-05T19:40:50.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.699+0000 7f3cd0a04e40 20 add_watcher() i=1 2023-10-05T19:40:50.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.699+0000 7f3cd0a04e40 20 add_watcher() i=2 2023-10-05T19:40:50.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.699+0000 7f3cd0a04e40 20 add_watcher() i=3 2023-10-05T19:40:50.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.699+0000 7f3cd0a04e40 20 add_watcher() i=4 2023-10-05T19:40:50.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.699+0000 7f3cd0a04e40 20 add_watcher() i=5 2023-10-05T19:40:50.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.699+0000 7f3cd0a04e40 20 add_watcher() i=6 2023-10-05T19:40:50.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.703+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(99 notify.7 [watch watch cookie 94537255619712] v30'211 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3cc800a5a0 con 0x7f3cc0007030 2023-10-05T19:40:50.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.703+0000 7f3cd0a04e40 20 add_watcher() i=7 2023-10-05T19:40:50.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.703+0000 7f3cd0a04e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:50.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.703+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b8a5100 con 0x55fb2b85c7f0 2023-10-05T19:40:50.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.703+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f3cc800a5a0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.703+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94537255623648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b8a2630 con 0x55fb2b85c7f0 2023-10-05T19:40:50.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.703+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94537255623648] v30'208 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f3cc800a5a0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.703+0000 7f3c9ffdf700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:50.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.707+0000 7f3cd0a04e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:50.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.707+0000 7f3cd0a04e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:50.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.707+0000 7f3cd0a04e40 1 Processor -- start 2023-10-05T19:40:50.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.707+0000 7f3cd0a04e40 1 -- start start 2023-10-05T19:40:50.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.707+0000 7f3cd0a04e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fb2b8af890 0x55fb2b8afc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:50.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.707+0000 7f3cd0a04e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb2b8b01a0 con 0x55fb2b8af890 2023-10-05T19:40:50.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.707+0000 7f3ccdd38700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fb2b8af890 0x55fb2b8afc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:50.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.707+0000 7f3ccdd38700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fb2b8af890 0x55fb2b8afc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58660/0 (socket says 172.21.15.12:58660) 2023-10-05T19:40:50.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.707+0000 7f3ccdd38700 1 -- 172.21.15.12:0/1016171757 learned_addr learned my addr 172.21.15.12:0/1016171757 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:50.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.707+0000 7f3ccdd38700 1 -- 172.21.15.12:0/1016171757 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb2b8b02e0 con 0x55fb2b8af890 2023-10-05T19:40:50.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.707+0000 7f3ccdd38700 1 --2- 172.21.15.12:0/1016171757 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fb2b8af890 0x55fb2b8afc60 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x55fb2b88d3f0 tx=0x7f3cb801e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4583ecbf9a49bbc2 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:50.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.707+0000 7f3c9f7de700 1 -- 172.21.15.12:0/1016171757 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3cb8030040 con 0x55fb2b8af890 2023-10-05T19:40:50.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.707+0000 7f3c9f7de700 1 -- 172.21.15.12:0/1016171757 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cb803a560 con 0x55fb2b8af890 2023-10-05T19:40:50.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.707+0000 7f3c9f7de700 1 -- 172.21.15.12:0/1016171757 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3cb8021040 con 0x55fb2b8af890 2023-10-05T19:40:50.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.707+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/1016171757 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fb2b8af890 msgr2=0x55fb2b8afc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.707+0000 7f3cd0a04e40 1 --2- 172.21.15.12:0/1016171757 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fb2b8af890 0x55fb2b8afc60 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x55fb2b88d3f0 tx=0x7f3cb801e040 comp rx=0 tx=0).stop 2023-10-05T19:40:50.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.711+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/1016171757 shutdown_connections 2023-10-05T19:40:50.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.711+0000 7f3cd0a04e40 1 --2- 172.21.15.12:0/1016171757 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fb2b8af890 0x55fb2b8afc60 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:50.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.711+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/1016171757 >> 172.21.15.12:0/1016171757 conn(0x55fb2b8aac50 msgr2=0x55fb2b8a8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:50.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.711+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/1016171757 shutdown_connections 2023-10-05T19:40:50.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.711+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/1016171757 wait complete. 2023-10-05T19:40:50.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.711+0000 7f3cd0a04e40 1 Processor -- start 2023-10-05T19:40:50.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.711+0000 7f3cd0a04e40 1 -- start start 2023-10-05T19:40:50.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.711+0000 7f3cd0a04e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fb2b8af890 0x55fb2b8b5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:50.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.711+0000 7f3cd0a04e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb2b8b6420 con 0x55fb2b8af890 2023-10-05T19:40:50.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.711+0000 7f3ccdd38700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fb2b8af890 0x55fb2b8b5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:50.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.711+0000 7f3ccdd38700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fb2b8af890 0x55fb2b8b5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58668/0 (socket says 172.21.15.12:58668) 2023-10-05T19:40:50.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.711+0000 7f3ccdd38700 1 -- 172.21.15.12:0/4119421240 learned_addr learned my addr 172.21.15.12:0/4119421240 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:50.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.711+0000 7f3ccdd38700 1 -- 172.21.15.12:0/4119421240 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb8033040 con 0x55fb2b8af890 2023-10-05T19:40:50.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.711+0000 7f3ccdd38700 1 --2- 172.21.15.12:0/4119421240 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fb2b8af890 0x55fb2b8b5420 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x55fb2b894ce0 tx=0x7f3cb803c030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:50.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.711+0000 7f3c9dfdb700 1 -- 172.21.15.12:0/4119421240 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3cb8021030 con 0x55fb2b8af890 2023-10-05T19:40:50.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.711+0000 7f3c9dfdb700 1 -- 172.21.15.12:0/4119421240 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cb802b730 con 0x55fb2b8af890 2023-10-05T19:40:50.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.711+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/4119421240 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55fb2b8b5a20 con 0x55fb2b8af890 2023-10-05T19:40:50.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.711+0000 7f3c9dfdb700 1 -- 172.21.15.12:0/4119421240 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3cb8030040 con 0x55fb2b8af890 2023-10-05T19:40:50.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.711+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/4119421240 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb2b8b5e00 con 0x55fb2b8af890 2023-10-05T19:40:50.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.715+0000 7f3c9dfdb700 1 -- 172.21.15.12:0/4119421240 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f3cb802b8c0 con 0x55fb2b8af890 2023-10-05T19:40:50.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.715+0000 7f3c9dfdb700 1 --2- 172.21.15.12:0/4119421240 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3c14030c50 0x7f3c14033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:50.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.715+0000 7f3c9dfdb700 1 -- 172.21.15.12:0/4119421240 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7f3cb8058bd0 con 0x55fb2b8af890 2023-10-05T19:40:50.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.715+0000 7f3cced3a700 1 --2- 172.21.15.12:0/4119421240 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3c14030c50 0x7f3c14033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:50.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.715+0000 7f3cd0a04e40 5 note: GC not initialized 2023-10-05T19:40:50.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.715+0000 7f3cced3a700 1 --2- 172.21.15.12:0/4119421240 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3c14030c50 0x7f3c14033100 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x55fb2b8a2590 tx=0x7f3cc800af30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:50.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.715+0000 7f3c0cff9700 20 reqs_thread_entry: start 2023-10-05T19:40:50.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.715+0000 7f3c0effd700 20 reqs_thread_entry: start 2023-10-05T19:40:50.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.715+0000 7f3cd0a04e40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:50.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.715+0000 7f3cd0a04e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:50.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.715+0000 7f3c06ffd700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:41:20 2023 2023-10-05T19:40:50.725 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:50.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.715+0000 7f3c067fc700 20 reqs_thread_entry: start 2023-10-05T19:40:50.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.715+0000 7f3cd0a04e40 20 get_system_obj_state: rctx=0x7ffea1bf11b8 obj=default.rgw.meta:users.uid:foo state=0x55fb2b7350b0 s->prefetch_data=0 2023-10-05T19:40:50.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.715+0000 7f3cd0a04e40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2023-10-05T19:40:50.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.719+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b91dcb0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.719+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(102 foo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv13 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7f3cc800a5a0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.719+0000 7f3cd0a04e40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x16 2023-10-05T19:40:50.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.719+0000 7f3cd0a04e40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:50.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.719+0000 7f3cd0a04e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:50.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.719+0000 7f3cd0a04e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:50.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.719+0000 7f3cd0a04e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:50.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.719+0000 7f3cd0a04e40 10 cache get: name=default.rgw.meta+users.uid+foo : type miss (requested=0x13, cached=0x16) 2023-10-05T19:40:50.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.719+0000 7f3cd0a04e40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.719+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b920740 con 0x55fb2b85c7f0 2023-10-05T19:40:50.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.719+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(103 foo [call,call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv13 ondisk = 0) v8 ==== 273+0+480 (crc 0 0 0) 0x7f3cc800a5a0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.719+0000 7f3cd0a04e40 20 rados_obj.operate() r=0 bl.length=357 2023-10-05T19:40:50.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.719+0000 7f3cd0a04e40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2023-10-05T19:40:50.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.719+0000 7f3cd0a04e40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:50.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.719+0000 7f3cd0a04e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:50.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.719+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:104 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.set in=58b,writefull 0~369 in=369b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b9219c0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.723+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(104 foo [delete,create,call,writefull 0~369] v30'14 uv14 ondisk = 0) v8 ==== 273+0+0 (crc 0 0 0) 0x7f3cc800a5a0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.723+0000 7f3cd0a04e40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2023-10-05T19:40:50.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.723+0000 7f3cd0a04e40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:50.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.723+0000 7f3cd0a04e40 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2023-10-05T19:40:50.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.723+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:105 7.d 7:b4812045:::notify.4:head [notify cookie 94537256146128 in=557b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b920740 con 0x55fb2b85c7f0 2023-10-05T19:40:50.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.723+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== watch-notify(notify (1) cookie 94537255613616 notify 128849018896 ret 0) v3 ==== 587+0+0 (crc 0 0 0) 0x7f3cc802f040 con 0x55fb2b85c7f0 2023-10-05T19:40:50.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.723+0000 7f3cc7fff700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018896 cookie 94537255613616 notifier 4645 bl.length()=545 2023-10-05T19:40:50.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.723+0000 7f3cc7fff700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2023-10-05T19:40:50.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.723+0000 7f3cc7fff700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:50.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.723+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.1 v2:172.21.15.12:6808/29933 39 ==== osd_op_reply(105 notify.4 [notify cookie 94537256146128 out=8b] v0'0 uv210 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f3cc800a5a0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.723+0000 7f3cc7fff700 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:106 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f3cb4003d90 con 0x55fb2b85c7f0 2023-10-05T19:40:50.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.723+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.1 v2:172.21.15.12:6808/29933 40 ==== watch-notify(notify_complete (2) cookie 94537256146128 notify 128849018896 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f3cc8039ea0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.723+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.1 v2:172.21.15.12:6808/29933 41 ==== osd_op_reply(106 notify.4 [notify-ack] v0'0 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3cc800a5a0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.732 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:40:50.732 INFO:teuthology.orchestra.run.smithi012.stdout: "user_id": "foo", 2023-10-05T19:40:50.732 INFO:teuthology.orchestra.run.smithi012.stdout: "display_name": "Foo", 2023-10-05T19:40:50.732 INFO:teuthology.orchestra.run.smithi012.stdout: "email": "foo@foo.com", 2023-10-05T19:40:50.732 INFO:teuthology.orchestra.run.smithi012.stdout: "suspended": 0, 2023-10-05T19:40:50.732 INFO:teuthology.orchestra.run.smithi012.stdout: "max_buckets": 4, 2023-10-05T19:40:50.732 INFO:teuthology.orchestra.run.smithi012.stdout: "subusers": [], 2023-10-05T19:40:50.733 INFO:teuthology.orchestra.run.smithi012.stdout: "keys": [ 2023-10-05T19:40:50.733 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:50.733 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:40:50.733 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2023-10-05T19:40:50.733 INFO:teuthology.orchestra.run.smithi012.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2023-10-05T19:40:50.733 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:50.733 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:50.733 INFO:teuthology.orchestra.run.smithi012.stdout: "swift_keys": [], 2023-10-05T19:40:50.733 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": [ 2023-10-05T19:40:50.733 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:50.733 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "user", 2023-10-05T19:40:50.733 INFO:teuthology.orchestra.run.smithi012.stdout: "perm": "read" 2023-10-05T19:40:50.734 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:50.734 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:50.734 INFO:teuthology.orchestra.run.smithi012.stdout: "op_mask": "read, write, delete", 2023-10-05T19:40:50.734 INFO:teuthology.orchestra.run.smithi012.stdout: "default_placement": "", 2023-10-05T19:40:50.734 INFO:teuthology.orchestra.run.smithi012.stdout: "default_storage_class": "", 2023-10-05T19:40:50.734 INFO:teuthology.orchestra.run.smithi012.stdout: "placement_tags": [], 2023-10-05T19:40:50.734 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket_quota": { 2023-10-05T19:40:50.734 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:50.734 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:50.734 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:50.734 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:50.735 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:50.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:50.735 INFO:teuthology.orchestra.run.smithi012.stdout: "user_quota": { 2023-10-05T19:40:50.735 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:50.735 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:50.735 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:50.735 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:50.735 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:50.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:50.735 INFO:teuthology.orchestra.run.smithi012.stdout: "temp_url_keys": [], 2023-10-05T19:40:50.735 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "rgw", 2023-10-05T19:40:50.735 INFO:teuthology.orchestra.run.smithi012.stdout: "mfa_ids": [] 2023-10-05T19:40:50.736 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:40:50.736 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:40:50.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.723+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:107 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94537255600496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b873a40 con 0x55fb2b891e00 2023-10-05T19:40:50.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.727+0000 7f3cce539700 1 -- 172.21.15.12:0/757573675 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(107 notify.0 [watch unwatch cookie 94537255600496] v30'212 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3cc0022020 con 0x55fb2b891e00 2023-10-05T19:40:50.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.727+0000 7f3cd0a04e40 20 remove_watcher() i=0 2023-10-05T19:40:50.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.727+0000 7f3cd0a04e40 2 removed watcher, disabling cache 2023-10-05T19:40:50.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.727+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:108 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94537255606016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b89c170 con 0x55fb2b85c7f0 2023-10-05T19:40:50.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.731+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.1 v2:172.21.15.12:6808/29933 42 ==== osd_op_reply(108 notify.1 [watch unwatch cookie 94537255606016] v30'212 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3cc800a5a0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.731+0000 7f3cd0a04e40 20 remove_watcher() i=1 2023-10-05T19:40:50.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.731+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:109 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94537255607120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b89c170 con 0x55fb2b896510 2023-10-05T19:40:50.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.731+0000 7f3ccdd38700 1 -- 172.21.15.12:0/757573675 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(109 notify.2 [watch unwatch cookie 94537255607120] v30'212 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3cb8006070 con 0x55fb2b896510 2023-10-05T19:40:50.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.731+0000 7f3cd0a04e40 20 remove_watcher() i=2 2023-10-05T19:40:50.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.731+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:110 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94537255612512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b89c170 con 0x55fb2b891e00 2023-10-05T19:40:50.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.735+0000 7f3cce539700 1 -- 172.21.15.12:0/757573675 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(110 notify.3 [watch unwatch cookie 94537255612512] v30'212 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3cc0022020 con 0x55fb2b891e00 2023-10-05T19:40:50.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.735+0000 7f3cd0a04e40 20 remove_watcher() i=3 2023-10-05T19:40:50.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.735+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:111 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94537255613616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b89c170 con 0x55fb2b85c7f0 2023-10-05T19:40:50.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.739+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.1 v2:172.21.15.12:6808/29933 43 ==== osd_op_reply(111 notify.4 [watch unwatch cookie 94537255613616] v30'212 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3cc800a5a0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.739+0000 7f3cd0a04e40 20 remove_watcher() i=4 2023-10-05T19:40:50.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.739+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:112 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94537255618048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b89f550 con 0x7f3cc0007030 2023-10-05T19:40:50.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.739+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(112 notify.5 [watch unwatch cookie 94537255618048] v30'212 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3cc800a5a0 con 0x7f3cc0007030 2023-10-05T19:40:50.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.743+0000 7f3cd0a04e40 20 remove_watcher() i=5 2023-10-05T19:40:50.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.743+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:113 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94537255615392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b89f550 con 0x7f3cc0007030 2023-10-05T19:40:50.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.743+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(113 notify.6 [watch unwatch cookie 94537255615392] v30'212 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3cc800a5a0 con 0x7f3cc0007030 2023-10-05T19:40:50.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.743+0000 7f3cd0a04e40 20 remove_watcher() i=6 2023-10-05T19:40:50.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.743+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:114 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94537255619712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b89f550 con 0x7f3cc0007030 2023-10-05T19:40:50.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.747+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(114 notify.7 [watch unwatch cookie 94537255619712] v30'212 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3cc800a5a0 con 0x7f3cc0007030 2023-10-05T19:40:50.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.747+0000 7f3cd0a04e40 20 remove_watcher() i=7 2023-10-05T19:40:50.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.751+0000 7f3c9ffdf700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:50.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.751+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:115 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94537255623648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2b89f550 con 0x55fb2b85c7f0 2023-10-05T19:40:50.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.751+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 <== osd.1 v2:172.21.15.12:6808/29933 44 ==== osd_op_reply(115 data_loggenerations_metadata [watch unwatch cookie 94537255623648] v30'209 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f3cc800a5a0 con 0x55fb2b85c7f0 2023-10-05T19:40:50.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.751+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f3cc0007030 msgr2=0x7f3cc0031480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.751+0000 7f3cd0a04e40 1 --2- 172.21.15.12:0/757573675 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f3cc0007030 0x7f3cc0031480 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:50.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.751+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55fb2b85c7f0 msgr2=0x55fb2b88f960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.751+0000 7f3cd0a04e40 1 --2- 172.21.15.12:0/757573675 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55fb2b85c7f0 0x55fb2b88f960 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:50.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.751+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55fb2b891e00 msgr2=0x55fb2b8941e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.751+0000 7f3cd0a04e40 1 --2- 172.21.15.12:0/757573675 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55fb2b891e00 0x55fb2b8941e0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:50.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.751+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55fb2b896510 msgr2=0x55fb2b898920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.751+0000 7f3cd0a04e40 1 --2- 172.21.15.12:0/757573675 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55fb2b896510 0x55fb2b898920 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:50.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.755+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3cb0030b10 msgr2=0x7f3cb0032fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.755+0000 7f3cd0a04e40 1 --2- 172.21.15.12:0/757573675 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3cb0030b10 0x7f3cb0032fc0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8001e90 tx=0x7f3cb80013b0 comp rx=0 tx=0).stop 2023-10-05T19:40:50.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.755+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fb2b85c420 msgr2=0x55fb2b883d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.755+0000 7f3cd0a04e40 1 --2- 172.21.15.12:0/757573675 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fb2b85c420 0x55fb2b883d70 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0000c00 tx=0x7f3cc0003050 comp rx=0 tx=0).stop 2023-10-05T19:40:50.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.755+0000 7f3cced3a700 1 -- 172.21.15.12:0/757573675 reap_dead start 2023-10-05T19:40:50.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.755+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 shutdown_connections 2023-10-05T19:40:50.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.755+0000 7f3cd0a04e40 1 --2- 172.21.15.12:0/757573675 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fb2b85c420 0x55fb2b883d70 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:50.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.755+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 >> 172.21.15.12:0/757573675 conn(0x55fb2b856730 msgr2=0x55fb2b8777a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:50.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.755+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 shutdown_connections 2023-10-05T19:40:50.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.755+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/757573675 wait complete. 2023-10-05T19:40:50.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.755+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/4119421240 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3c14030c50 msgr2=0x7f3c14033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.755+0000 7f3cd0a04e40 1 --2- 172.21.15.12:0/4119421240 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3c14030c50 0x7f3c14033100 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x55fb2b8a2590 tx=0x7f3cc800af30 comp rx=0 tx=0).stop 2023-10-05T19:40:50.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.755+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/4119421240 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fb2b8af890 msgr2=0x55fb2b8b5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.755+0000 7f3cd0a04e40 1 --2- 172.21.15.12:0/4119421240 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fb2b8af890 0x55fb2b8b5420 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x55fb2b894ce0 tx=0x7f3cb803c030 comp rx=0 tx=0).stop 2023-10-05T19:40:50.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.755+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/4119421240 shutdown_connections 2023-10-05T19:40:50.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.755+0000 7f3cd0a04e40 1 --2- 172.21.15.12:0/4119421240 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3c14030c50 0x7f3c14033100 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:50.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.755+0000 7f3cd0a04e40 1 --2- 172.21.15.12:0/4119421240 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fb2b8af890 0x55fb2b8b5420 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:50.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.755+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/4119421240 >> 172.21.15.12:0/4119421240 conn(0x55fb2b8aac50 msgr2=0x55fb2b8a8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:50.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.755+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/4119421240 shutdown_connections 2023-10-05T19:40:50.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.755+0000 7f3cd0a04e40 1 -- 172.21.15.12:0/4119421240 wait complete. 2023-10-05T19:40:50.768 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [{'type': 'user', 'perm': 'read'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:40:50.768 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'rm', '--uid', 'foo', '--caps', 'user=read'] 2023-10-05T19:40:50.768 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'rm', '--uid', 'foo', '--caps', 'user=read'] 2023-10-05T19:40:50.768 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps rm --uid foo --caps user=read 2023-10-05T19:40:50.812 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:50.812 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:50.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.803+0000 7f388dbfae40 1 Processor -- start 2023-10-05T19:40:50.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.807+0000 7f388dbfae40 1 -- start start 2023-10-05T19:40:50.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.807+0000 7f388dbfae40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x560f1f4cd420 0x560f1f4cbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:50.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.807+0000 7f388dbfae40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x560f1f419bd0 con 0x560f1f4cd7f0 2023-10-05T19:40:50.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.807+0000 7f388dbfae40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x560f1f3d6ee0 con 0x560f1f4cd420 2023-10-05T19:40:50.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.807+0000 7f388b72f700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x560f1f4cd420 0x560f1f4cbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:50.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.807+0000 7f388af2e700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x560f1f4cd7f0 0x560f1f4cc510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:60266/0 (socket says 172.21.15.12:60266) 2023-10-05T19:40:50.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.807+0000 7f388af2e700 1 -- 172.21.15.12:0/1192131430 learned_addr learned my addr 172.21.15.12:0/1192131430 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:50.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.807+0000 7f388b72f700 1 -- 172.21.15.12:0/1192131430 >> v1:172.21.15.8:6789/0 conn(0x560f1f4cd7f0 legacy=0x560f1f4cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.807+0000 7f388b72f700 1 -- 172.21.15.12:0/1192131430 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f1f4ccce0 con 0x560f1f4cd420 2023-10-05T19:40:50.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.807+0000 7f388b72f700 1 --2- 172.21.15.12:0/1192131430 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f1f4cd420 0x560f1f4cbfd0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f387c00a520 tx=0x7f387c005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1241ef5784894a2c server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:50.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.807+0000 7f388a72d700 1 -- 172.21.15.12:0/1192131430 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f387c00f590 con 0x560f1f4cd420 2023-10-05T19:40:50.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.807+0000 7f388a72d700 1 -- 172.21.15.12:0/1192131430 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f387c00f6f0 con 0x560f1f4cd420 2023-10-05T19:40:50.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.807+0000 7f388a72d700 1 -- 172.21.15.12:0/1192131430 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f387c00f880 con 0x560f1f4cd420 2023-10-05T19:40:50.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.807+0000 7f388dbfae40 1 -- 172.21.15.12:0/1192131430 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f1f4cd420 msgr2=0x560f1f4cbfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.807+0000 7f388dbfae40 1 --2- 172.21.15.12:0/1192131430 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f1f4cd420 0x560f1f4cbfd0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f387c00a520 tx=0x7f387c005d40 comp rx=0 tx=0).stop 2023-10-05T19:40:50.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.807+0000 7f388dbfae40 1 -- 172.21.15.12:0/1192131430 shutdown_connections 2023-10-05T19:40:50.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.807+0000 7f388dbfae40 1 --2- 172.21.15.12:0/1192131430 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f1f4cd420 0x560f1f4cbfd0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:50.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.807+0000 7f388dbfae40 1 -- 172.21.15.12:0/1192131430 >> 172.21.15.12:0/1192131430 conn(0x560f1f4c7730 msgr2=0x560f1f4c9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:50.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.807+0000 7f388dbfae40 1 -- 172.21.15.12:0/1192131430 shutdown_connections 2023-10-05T19:40:50.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.807+0000 7f388dbfae40 1 -- 172.21.15.12:0/1192131430 wait complete. 2023-10-05T19:40:50.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.807+0000 7f388dbfae40 1 Processor -- start 2023-10-05T19:40:50.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.811+0000 7f388dbfae40 1 -- start start 2023-10-05T19:40:50.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.811+0000 7f388dbfae40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f1f4cd420 0x560f1f4ee120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:50.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.811+0000 7f388dbfae40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f1f39ae30 con 0x560f1f4cd420 2023-10-05T19:40:50.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.811+0000 7f388b72f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f1f4cd420 0x560f1f4ee120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:50.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.811+0000 7f388b72f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f1f4cd420 0x560f1f4ee120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58694/0 (socket says 172.21.15.12:58694) 2023-10-05T19:40:50.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.811+0000 7f388b72f700 1 -- 172.21.15.12:0/1520251947 learned_addr learned my addr 172.21.15.12:0/1520251947 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:50.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.811+0000 7f388b72f700 1 -- 172.21.15.12:0/1520251947 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f387c00a0e0 con 0x560f1f4cd420 2023-10-05T19:40:50.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.811+0000 7f388b72f700 1 --2- 172.21.15.12:0/1520251947 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f1f4cd420 0x560f1f4ee120 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f387c00a470 tx=0x7f387c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3030f0d0f9969b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:50.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.811+0000 7f388972b700 1 -- 172.21.15.12:0/1520251947 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f387c00f860 con 0x560f1f4cd420 2023-10-05T19:40:50.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.811+0000 7f388972b700 1 -- 172.21.15.12:0/1520251947 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f387c00f9e0 con 0x560f1f4cd420 2023-10-05T19:40:50.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.811+0000 7f388972b700 1 -- 172.21.15.12:0/1520251947 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f387c00fb90 con 0x560f1f4cd420 2023-10-05T19:40:50.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.811+0000 7f388dbfae40 1 -- 172.21.15.12:0/1520251947 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f1f4cd420 msgr2=0x560f1f4ee120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.811+0000 7f388dbfae40 1 --2- 172.21.15.12:0/1520251947 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f1f4cd420 0x560f1f4ee120 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f387c00a470 tx=0x7f387c014000 comp rx=0 tx=0).stop 2023-10-05T19:40:50.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.811+0000 7f388dbfae40 1 -- 172.21.15.12:0/1520251947 shutdown_connections 2023-10-05T19:40:50.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.811+0000 7f388dbfae40 1 --2- 172.21.15.12:0/1520251947 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f1f4cd420 0x560f1f4ee120 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:50.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.811+0000 7f388dbfae40 1 -- 172.21.15.12:0/1520251947 >> 172.21.15.12:0/1520251947 conn(0x560f1f4c7730 msgr2=0x560f1f4e9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:50.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.811+0000 7f388dbfae40 1 -- 172.21.15.12:0/1520251947 shutdown_connections 2023-10-05T19:40:50.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.811+0000 7f388dbfae40 1 -- 172.21.15.12:0/1520251947 wait complete. 2023-10-05T19:40:50.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.811+0000 7f388dbfae40 1 Processor -- start 2023-10-05T19:40:50.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.811+0000 7f388dbfae40 1 -- start start 2023-10-05T19:40:50.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.811+0000 7f388dbfae40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f1f4cd420 0x560f1f4f4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:50.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.811+0000 7f388dbfae40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f1f36ded0 con 0x560f1f4cd420 2023-10-05T19:40:50.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.815+0000 7f388b72f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f1f4cd420 0x560f1f4f4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:50.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.815+0000 7f388b72f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f1f4cd420 0x560f1f4f4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58700/0 (socket says 172.21.15.12:58700) 2023-10-05T19:40:50.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.815+0000 7f388b72f700 1 -- 172.21.15.12:0/1544794375 learned_addr learned my addr 172.21.15.12:0/1544794375 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:50.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.815+0000 7f388b72f700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f1f4ccce0 con 0x560f1f4cd420 2023-10-05T19:40:50.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.815+0000 7f388b72f700 1 --2- 172.21.15.12:0/1544794375 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f1f4cd420 0x560f1f4f4d30 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f387c00a470 tx=0x7f387c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:50.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.815+0000 7f38837fe700 1 -- 172.21.15.12:0/1544794375 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f387c01ce00 con 0x560f1f4cd420 2023-10-05T19:40:50.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.815+0000 7f38837fe700 1 -- 172.21.15.12:0/1544794375 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f387c010630 con 0x560f1f4cd420 2023-10-05T19:40:50.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.815+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560f1f4f5270 con 0x560f1f4cd420 2023-10-05T19:40:50.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.815+0000 7f38837fe700 1 -- 172.21.15.12:0/1544794375 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f387c010820 con 0x560f1f4cd420 2023-10-05T19:40:50.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.815+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560f1f4f56b0 con 0x560f1f4cd420 2023-10-05T19:40:50.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.815+0000 7f38837fe700 1 -- 172.21.15.12:0/1544794375 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f387c010a90 con 0x560f1f4cd420 2023-10-05T19:40:50.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.815+0000 7f38837fe700 1 --2- 172.21.15.12:0/1544794375 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f386c030c00 0x7f386c0330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:50.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.815+0000 7f38837fe700 1 -- 172.21.15.12:0/1544794375 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7f387c02b070 con 0x560f1f4cd420 2023-10-05T19:40:50.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.815+0000 7f388af2e700 1 --2- 172.21.15.12:0/1544794375 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f386c030c00 0x7f386c0330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:50.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.815+0000 7f388dbfae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.819+0000 7f388dbfae40 1 --2- 172.21.15.12:0/1544794375 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x560f1f4cd7f0 0x560f1f500750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:50.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.819+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f500e80 con 0x560f1f4cd7f0 2023-10-05T19:40:50.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.819+0000 7f388bf30700 1 --2- 172.21.15.12:0/1544794375 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x560f1f4cd7f0 0x560f1f500750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:50.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.819+0000 7f388af2e700 1 --2- 172.21.15.12:0/1544794375 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f386c030c00 0x7f386c0330b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3874001e90 tx=0x7f38740013b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:50.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.819+0000 7f388bf30700 1 --2- 172.21.15.12:0/1544794375 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x560f1f4cd7f0 0x560f1f500750 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:50.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.819+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f388400a500 con 0x560f1f4cd7f0 2023-10-05T19:40:50.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.819+0000 7f388dbfae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:50.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.819+0000 7f388dbfae40 20 realm 2023-10-05T19:40:50.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.819+0000 7f388dbfae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.819+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f4fdd80 con 0x560f1f4cd7f0 2023-10-05T19:40:50.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.819+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f388400a500 con 0x560f1f4cd7f0 2023-10-05T19:40:50.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.819+0000 7f388dbfae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:50.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.819+0000 7f388dbfae40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:50.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.819+0000 7f388dbfae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.819+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f5019b0 con 0x560f1f4cd7f0 2023-10-05T19:40:50.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.819+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f388400a500 con 0x560f1f4cd7f0 2023-10-05T19:40:50.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.819+0000 7f388dbfae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:50.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.819+0000 7f388dbfae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.819+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f5015c0 con 0x560f1f4cd7f0 2023-10-05T19:40:50.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.823+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f388400a500 con 0x560f1f4cd7f0 2023-10-05T19:40:50.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.823+0000 7f388dbfae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:50.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.823+0000 7f388dbfae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.823+0000 7f388dbfae40 1 --2- 172.21.15.12:0/1544794375 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x560f1f502d50 0x560f1f505130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:50.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.823+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f505670 con 0x560f1f502d50 2023-10-05T19:40:50.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.823+0000 7f388b72f700 1 --2- 172.21.15.12:0/1544794375 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x560f1f502d50 0x560f1f505130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:50.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.823+0000 7f388b72f700 1 --2- 172.21.15.12:0/1544794375 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x560f1f502d50 0x560f1f505130 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:50.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.823+0000 7f388b72f700 1 -- 172.21.15.12:0/1544794375 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f387c03ad70 con 0x560f1f502d50 2023-10-05T19:40:50.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.823+0000 7f388dbfae40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:50.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.823+0000 7f388dbfae40 20 searching for the correct realm 2023-10-05T19:40:50.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.823+0000 7f388dbfae40 1 --2- 172.21.15.12:0/1544794375 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x560f1f5074f0 0x560f1f509900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:50.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.823+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f509fb0 con 0x560f1f5074f0 2023-10-05T19:40:50.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.823+0000 7f388af2e700 1 --2- 172.21.15.12:0/1544794375 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x560f1f5074f0 0x560f1f509900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:50.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.823+0000 7f388af2e700 1 --2- 172.21.15.12:0/1544794375 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x560f1f5074f0 0x560f1f509900 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:50.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.827+0000 7f388af2e700 1 -- 172.21.15.12:0/1544794375 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f3874006070 con 0x560f1f5074f0 2023-10-05T19:40:50.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.827+0000 7f388af2e700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f509fb0 con 0x560f1f502d50 2023-10-05T19:40:50.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.827+0000 7f388b72f700 1 -- 172.21.15.12:0/1544794375 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f387c03ad70 con 0x560f1f502d50 2023-10-05T19:40:50.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.827+0000 7f388b72f700 1 --2- 172.21.15.12:0/1544794375 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f387c044040 0x7f387c039540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:50.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.827+0000 7f388b72f700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f509fb0 con 0x7f387c044040 2023-10-05T19:40:50.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.827+0000 7f388bf30700 1 --2- 172.21.15.12:0/1544794375 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f387c044040 0x7f387c039540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:50.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.827+0000 7f388bf30700 1 --2- 172.21.15.12:0/1544794375 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f387c044040 0x7f387c039540 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:50.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.831+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3884014040 con 0x7f387c044040 2023-10-05T19:40:50.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.831+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f509fb0 con 0x560f1f4cd7f0 2023-10-05T19:40:50.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.831+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3884014040 con 0x560f1f4cd7f0 2023-10-05T19:40:50.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.831+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f509fb0 con 0x560f1f4cd7f0 2023-10-05T19:40:50.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.831+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3884014040 con 0x560f1f4cd7f0 2023-10-05T19:40:50.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.831+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f509fb0 con 0x560f1f4cd7f0 2023-10-05T19:40:50.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.831+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(11 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3884014040 con 0x560f1f4cd7f0 2023-10-05T19:40:50.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.831+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f509fb0 con 0x7f387c044040 2023-10-05T19:40:50.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.831+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3884014040 con 0x7f387c044040 2023-10-05T19:40:50.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.831+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f509fb0 con 0x560f1f502d50 2023-10-05T19:40:50.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.835+0000 7f388b72f700 1 -- 172.21.15.12:0/1544794375 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f387c03ad70 con 0x560f1f502d50 2023-10-05T19:40:50.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.835+0000 7f388b72f700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f509fb0 con 0x560f1f502d50 2023-10-05T19:40:50.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.835+0000 7f388b72f700 1 -- 172.21.15.12:0/1544794375 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f387c03ad70 con 0x560f1f502d50 2023-10-05T19:40:50.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.835+0000 7f388b72f700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f509fb0 con 0x560f1f4cd7f0 2023-10-05T19:40:50.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.835+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3884014040 con 0x560f1f4cd7f0 2023-10-05T19:40:50.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.835+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f509fb0 con 0x7f387c044040 2023-10-05T19:40:50.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.835+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3884014040 con 0x7f387c044040 2023-10-05T19:40:50.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.835+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f509fb0 con 0x560f1f4cd7f0 2023-10-05T19:40:50.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.835+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3884014040 con 0x560f1f4cd7f0 2023-10-05T19:40:50.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.835+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f509fb0 con 0x560f1f502d50 2023-10-05T19:40:50.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.835+0000 7f388b72f700 1 -- 172.21.15.12:0/1544794375 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(18 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f387c03ad70 con 0x560f1f502d50 2023-10-05T19:40:50.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.835+0000 7f388b72f700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f509fb0 con 0x560f1f5074f0 2023-10-05T19:40:50.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.839+0000 7f388af2e700 1 -- 172.21.15.12:0/1544794375 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3874006070 con 0x560f1f5074f0 2023-10-05T19:40:50.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.839+0000 7f388af2e700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f509fb0 con 0x560f1f5074f0 2023-10-05T19:40:50.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.839+0000 7f388af2e700 1 -- 172.21.15.12:0/1544794375 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3874006070 con 0x560f1f5074f0 2023-10-05T19:40:50.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.839+0000 7f388af2e700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f509fb0 con 0x7f387c044040 2023-10-05T19:40:50.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.839+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3884014040 con 0x7f387c044040 2023-10-05T19:40:50.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.839+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f509fb0 con 0x560f1f5074f0 2023-10-05T19:40:50.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.839+0000 7f388af2e700 1 -- 172.21.15.12:0/1544794375 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3874006070 con 0x560f1f5074f0 2023-10-05T19:40:50.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.839+0000 7f388af2e700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f509fb0 con 0x560f1f5074f0 2023-10-05T19:40:50.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.839+0000 7f388af2e700 1 -- 172.21.15.12:0/1544794375 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3874006070 con 0x560f1f5074f0 2023-10-05T19:40:50.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.839+0000 7f388af2e700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f509fb0 con 0x560f1f4cd7f0 2023-10-05T19:40:50.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.843+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3884014040 con 0x560f1f4cd7f0 2023-10-05T19:40:50.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.843+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f509fb0 con 0x560f1f4cd7f0 2023-10-05T19:40:50.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.843+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3884014040 con 0x560f1f4cd7f0 2023-10-05T19:40:50.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.843+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f509fb0 con 0x7f387c044040 2023-10-05T19:40:50.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.843+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3884014040 con 0x7f387c044040 2023-10-05T19:40:50.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.843+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f509fb0 con 0x560f1f5074f0 2023-10-05T19:40:50.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.843+0000 7f388af2e700 1 -- 172.21.15.12:0/1544794375 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3874006070 con 0x560f1f5074f0 2023-10-05T19:40:50.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.843+0000 7f388af2e700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f509fb0 con 0x560f1f502d50 2023-10-05T19:40:50.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.843+0000 7f388b72f700 1 -- 172.21.15.12:0/1544794375 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f387c03ad70 con 0x560f1f502d50 2023-10-05T19:40:50.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.843+0000 7f388b72f700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f509fb0 con 0x560f1f4cd7f0 2023-10-05T19:40:50.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.847+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3884014040 con 0x560f1f4cd7f0 2023-10-05T19:40:50.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.847+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f509fb0 con 0x7f387c044040 2023-10-05T19:40:50.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.847+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3884014040 con 0x7f387c044040 2023-10-05T19:40:50.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.847+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f509fb0 con 0x560f1f5074f0 2023-10-05T19:40:50.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.847+0000 7f388af2e700 1 -- 172.21.15.12:0/1544794375 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3874006070 con 0x560f1f5074f0 2023-10-05T19:40:50.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.847+0000 7f388af2e700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f509fb0 con 0x7f387c044040 2023-10-05T19:40:50.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.847+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3884014040 con 0x7f387c044040 2023-10-05T19:40:50.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.847+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f509fb0 con 0x560f1f502d50 2023-10-05T19:40:50.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.851+0000 7f388b72f700 1 -- 172.21.15.12:0/1544794375 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f387c03ad70 con 0x560f1f502d50 2023-10-05T19:40:50.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.851+0000 7f388b72f700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f509fb0 con 0x560f1f5074f0 2023-10-05T19:40:50.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.851+0000 7f388af2e700 1 -- 172.21.15.12:0/1544794375 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3874006070 con 0x560f1f5074f0 2023-10-05T19:40:50.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.851+0000 7f388af2e700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f509fb0 con 0x7f387c044040 2023-10-05T19:40:50.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.851+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3884014040 con 0x7f387c044040 2023-10-05T19:40:50.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.851+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f509fb0 con 0x560f1f4cd7f0 2023-10-05T19:40:50.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.851+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f3884014040 con 0x560f1f4cd7f0 2023-10-05T19:40:50.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.851+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f509fb0 con 0x560f1f502d50 2023-10-05T19:40:50.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.851+0000 7f388b72f700 1 -- 172.21.15.12:0/1544794375 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(37 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f387c03ad70 con 0x560f1f502d50 2023-10-05T19:40:50.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.855+0000 7f388dbfae40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:50.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.855+0000 7f388dbfae40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:50.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.855+0000 7f388dbfae40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:50.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.855+0000 7f388dbfae40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:50.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.855+0000 7f388dbfae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.855+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f506270 con 0x560f1f4cd7f0 2023-10-05T19:40:50.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.855+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3884014040 con 0x560f1f4cd7f0 2023-10-05T19:40:50.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.855+0000 7f388dbfae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:50.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.855+0000 7f388dbfae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.855+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f505ea0 con 0x560f1f4cd7f0 2023-10-05T19:40:50.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.855+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f3884014040 con 0x560f1f4cd7f0 2023-10-05T19:40:50.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.855+0000 7f388dbfae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:50.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.855+0000 7f388dbfae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.855+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f50a420 con 0x560f1f5074f0 2023-10-05T19:40:50.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.855+0000 7f388af2e700 1 -- 172.21.15.12:0/1544794375 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f3874006070 con 0x560f1f5074f0 2023-10-05T19:40:50.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.855+0000 7f388dbfae40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:50.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.855+0000 7f388dbfae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.855+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f50a7d0 con 0x560f1f5074f0 2023-10-05T19:40:50.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.855+0000 7f388af2e700 1 -- 172.21.15.12:0/1544794375 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f3874006070 con 0x560f1f5074f0 2023-10-05T19:40:50.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.855+0000 7f388dbfae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:50.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.855+0000 7f388dbfae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.855+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f50ab80 con 0x560f1f4cd7f0 2023-10-05T19:40:50.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.855+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3884014040 con 0x560f1f4cd7f0 2023-10-05T19:40:50.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.855+0000 7f388dbfae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:50.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.855+0000 7f388dbfae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.855+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f506790 con 0x560f1f4cd7f0 2023-10-05T19:40:50.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.859+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f3884014040 con 0x560f1f4cd7f0 2023-10-05T19:40:50.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.859+0000 7f388dbfae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:50.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.859+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f506790 con 0x560f1f5074f0 2023-10-05T19:40:50.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.859+0000 7f388af2e700 1 -- 172.21.15.12:0/1544794375 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(44 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f3874006070 con 0x560f1f5074f0 2023-10-05T19:40:50.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.859+0000 7f388af2e700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f506790 con 0x560f1f502d50 2023-10-05T19:40:50.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.859+0000 7f388b72f700 1 -- 172.21.15.12:0/1544794375 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(45 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f387c03ad70 con 0x560f1f502d50 2023-10-05T19:40:50.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.859+0000 7f388b72f700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f506790 con 0x7f387c044040 2023-10-05T19:40:50.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.863+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(46 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3884014040 con 0x7f387c044040 2023-10-05T19:40:50.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.863+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f50ab80 con 0x560f1f4cd7f0 2023-10-05T19:40:50.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.863+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(47 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3884014040 con 0x560f1f4cd7f0 2023-10-05T19:40:50.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.863+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f50ab80 con 0x560f1f4cd7f0 2023-10-05T19:40:50.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.863+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(48 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3884014040 con 0x560f1f4cd7f0 2023-10-05T19:40:50.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.863+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f50ab80 con 0x560f1f4cd7f0 2023-10-05T19:40:50.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.863+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(49 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3884014040 con 0x560f1f4cd7f0 2023-10-05T19:40:50.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.863+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f50ab80 con 0x7f387c044040 2023-10-05T19:40:50.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.867+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(50 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3884014040 con 0x7f387c044040 2023-10-05T19:40:50.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.867+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f50ab80 con 0x560f1f502d50 2023-10-05T19:40:50.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.867+0000 7f388b72f700 1 -- 172.21.15.12:0/1544794375 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(51 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f387c03ad70 con 0x560f1f502d50 2023-10-05T19:40:50.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.867+0000 7f388b72f700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f50ab80 con 0x560f1f502d50 2023-10-05T19:40:50.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.867+0000 7f388b72f700 1 -- 172.21.15.12:0/1544794375 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(52 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f387c03ad70 con 0x560f1f502d50 2023-10-05T19:40:50.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.867+0000 7f388b72f700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f50ab80 con 0x560f1f4cd7f0 2023-10-05T19:40:50.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.867+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(53 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3884014040 con 0x560f1f4cd7f0 2023-10-05T19:40:50.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.867+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f50ab80 con 0x7f387c044040 2023-10-05T19:40:50.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.871+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(54 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3884014040 con 0x7f387c044040 2023-10-05T19:40:50.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.871+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f50ab80 con 0x560f1f4cd7f0 2023-10-05T19:40:50.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.871+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(55 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3884014040 con 0x560f1f4cd7f0 2023-10-05T19:40:50.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.871+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f50ab80 con 0x560f1f502d50 2023-10-05T19:40:50.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.871+0000 7f388b72f700 1 -- 172.21.15.12:0/1544794375 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(56 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f387c03ad70 con 0x560f1f502d50 2023-10-05T19:40:50.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.871+0000 7f388b72f700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f50ab80 con 0x560f1f5074f0 2023-10-05T19:40:50.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.871+0000 7f388af2e700 1 -- 172.21.15.12:0/1544794375 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(57 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3874006070 con 0x560f1f5074f0 2023-10-05T19:40:50.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.871+0000 7f388af2e700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f50ab80 con 0x560f1f5074f0 2023-10-05T19:40:50.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.871+0000 7f388af2e700 1 -- 172.21.15.12:0/1544794375 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(58 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3874006070 con 0x560f1f5074f0 2023-10-05T19:40:50.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.871+0000 7f388af2e700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f50ab80 con 0x7f387c044040 2023-10-05T19:40:50.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.875+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(59 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f388402e040 con 0x7f387c044040 2023-10-05T19:40:50.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.875+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f50ab80 con 0x560f1f5074f0 2023-10-05T19:40:50.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.875+0000 7f388af2e700 1 -- 172.21.15.12:0/1544794375 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(60 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3874006070 con 0x560f1f5074f0 2023-10-05T19:40:50.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.875+0000 7f388af2e700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f50ab80 con 0x560f1f5074f0 2023-10-05T19:40:50.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.875+0000 7f388af2e700 1 -- 172.21.15.12:0/1544794375 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(61 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3874006070 con 0x560f1f5074f0 2023-10-05T19:40:50.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.875+0000 7f388af2e700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f50ab80 con 0x560f1f4cd7f0 2023-10-05T19:40:50.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.875+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(62 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f388402e040 con 0x560f1f4cd7f0 2023-10-05T19:40:50.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.875+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f50ab80 con 0x560f1f4cd7f0 2023-10-05T19:40:50.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.875+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(63 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f388402e040 con 0x560f1f4cd7f0 2023-10-05T19:40:50.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.875+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f50ab80 con 0x7f387c044040 2023-10-05T19:40:50.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.879+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(64 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f388402e040 con 0x7f387c044040 2023-10-05T19:40:50.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.879+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f50ab80 con 0x560f1f5074f0 2023-10-05T19:40:50.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.879+0000 7f388af2e700 1 -- 172.21.15.12:0/1544794375 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(65 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3874006070 con 0x560f1f5074f0 2023-10-05T19:40:50.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.879+0000 7f388af2e700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f50ab80 con 0x560f1f502d50 2023-10-05T19:40:50.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.879+0000 7f388b72f700 1 -- 172.21.15.12:0/1544794375 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(66 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f387c03ad70 con 0x560f1f502d50 2023-10-05T19:40:50.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.879+0000 7f388b72f700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f50ab80 con 0x560f1f4cd7f0 2023-10-05T19:40:50.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.879+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(67 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f388402e040 con 0x560f1f4cd7f0 2023-10-05T19:40:50.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.879+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f50ab80 con 0x7f387c044040 2023-10-05T19:40:50.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.883+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(68 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f388402e040 con 0x7f387c044040 2023-10-05T19:40:50.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.883+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f50ab80 con 0x560f1f5074f0 2023-10-05T19:40:50.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.883+0000 7f388af2e700 1 -- 172.21.15.12:0/1544794375 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(69 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3874006070 con 0x560f1f5074f0 2023-10-05T19:40:50.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.883+0000 7f388af2e700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f50ab80 con 0x7f387c044040 2023-10-05T19:40:50.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.883+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(70 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f388402e040 con 0x7f387c044040 2023-10-05T19:40:50.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.883+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f50ab80 con 0x560f1f502d50 2023-10-05T19:40:50.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.883+0000 7f388b72f700 1 -- 172.21.15.12:0/1544794375 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(71 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f387c03ad70 con 0x560f1f502d50 2023-10-05T19:40:50.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.883+0000 7f388b72f700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f50ab80 con 0x560f1f5074f0 2023-10-05T19:40:50.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.887+0000 7f388af2e700 1 -- 172.21.15.12:0/1544794375 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(72 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3874006070 con 0x560f1f5074f0 2023-10-05T19:40:50.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.887+0000 7f388af2e700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f50ab80 con 0x7f387c044040 2023-10-05T19:40:50.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.887+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(73 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f388402e040 con 0x7f387c044040 2023-10-05T19:40:50.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.887+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f50ab80 con 0x560f1f4cd7f0 2023-10-05T19:40:50.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.887+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(74 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f388402e040 con 0x560f1f4cd7f0 2023-10-05T19:40:50.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.887+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f50ab80 con 0x560f1f502d50 2023-10-05T19:40:50.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.887+0000 7f388b72f700 1 -- 172.21.15.12:0/1544794375 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(75 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f387c03ad70 con 0x560f1f502d50 2023-10-05T19:40:50.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.887+0000 7f388dbfae40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:50.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.887+0000 7f388dbfae40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:50.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.887+0000 7f388dbfae40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:50.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.887+0000 7f388dbfae40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:50.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.887+0000 7f388dbfae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.887+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f506790 con 0x560f1f4cd7f0 2023-10-05T19:40:50.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.891+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f388402e040 con 0x560f1f4cd7f0 2023-10-05T19:40:50.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.891+0000 7f388dbfae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:50.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.891+0000 7f388dbfae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.891+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f50b520 con 0x560f1f502d50 2023-10-05T19:40:50.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.891+0000 7f388b72f700 1 -- 172.21.15.12:0/1544794375 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f387c03ad70 con 0x560f1f502d50 2023-10-05T19:40:50.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.891+0000 7f388dbfae40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:50.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.891+0000 7f388dbfae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.891+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f50be10 con 0x560f1f4cd7f0 2023-10-05T19:40:50.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.891+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f388402e040 con 0x560f1f4cd7f0 2023-10-05T19:40:50.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.891+0000 7f388dbfae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:50.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.891+0000 7f388dbfae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.891+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f50be10 con 0x560f1f5074f0 2023-10-05T19:40:50.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.891+0000 7f388af2e700 1 -- 172.21.15.12:0/1544794375 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f3874006070 con 0x560f1f5074f0 2023-10-05T19:40:50.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.891+0000 7f388dbfae40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:50.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.891+0000 7f388dbfae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.891+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f50c580 con 0x7f387c044040 2023-10-05T19:40:50.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.891+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f388402e040 con 0x7f387c044040 2023-10-05T19:40:50.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.891+0000 7f388dbfae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:50.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.891+0000 7f388dbfae40 20 zone default found 2023-10-05T19:40:50.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.891+0000 7f388dbfae40 4 Realm: () 2023-10-05T19:40:50.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.891+0000 7f388dbfae40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:50.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.891+0000 7f388dbfae40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:50.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.891+0000 7f388dbfae40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:50.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.891+0000 7f388dbfae40 20 zonegroup default 2023-10-05T19:40:50.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.891+0000 7f388dbfae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.891+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f50c580 con 0x7f387c044040 2023-10-05T19:40:50.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.895+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f388402e040 con 0x7f387c044040 2023-10-05T19:40:50.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.895+0000 7f388dbfae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:50.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.895+0000 7f388dbfae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.895+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f50df00 con 0x560f1f5074f0 2023-10-05T19:40:50.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.895+0000 7f388af2e700 1 -- 172.21.15.12:0/1544794375 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f3874006070 con 0x560f1f5074f0 2023-10-05T19:40:50.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.895+0000 7f388dbfae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:50.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.895+0000 7f388dbfae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.895+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f50e2b0 con 0x560f1f5074f0 2023-10-05T19:40:50.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.895+0000 7f388af2e700 1 -- 172.21.15.12:0/1544794375 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f3874006070 con 0x560f1f5074f0 2023-10-05T19:40:50.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.895+0000 7f388dbfae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:50.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.895+0000 7f388dbfae40 20 started sync module instance, tier type = 2023-10-05T19:40:50.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.895+0000 7f388dbfae40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:50.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.895+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f50d5c0 con 0x560f1f502d50 2023-10-05T19:40:50.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.899+0000 7f388b72f700 1 -- 172.21.15.12:0/1544794375 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(84 notify.0 [create] v30'213 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f387c03ad70 con 0x560f1f502d50 2023-10-05T19:40:50.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.899+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94622949882144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f50e660 con 0x560f1f502d50 2023-10-05T19:40:50.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.899+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f50ef00 con 0x560f1f4cd7f0 2023-10-05T19:40:50.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.903+0000 7f388b72f700 1 -- 172.21.15.12:0/1544794375 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(85 notify.0 [watch watch cookie 94622949882144] v30'214 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f387c03ad70 con 0x560f1f502d50 2023-10-05T19:40:50.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.903+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(86 notify.1 [create] v30'213 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f388402e040 con 0x560f1f4cd7f0 2023-10-05T19:40:50.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.903+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94622949894672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f50da90 con 0x560f1f4cd7f0 2023-10-05T19:40:50.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.903+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f50fa10 con 0x560f1f5074f0 2023-10-05T19:40:50.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.907+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(87 notify.1 [watch watch cookie 94622949894672] v30'214 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f388402e040 con 0x560f1f4cd7f0 2023-10-05T19:40:50.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.907+0000 7f388af2e700 1 -- 172.21.15.12:0/1544794375 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(88 notify.2 [create] v30'213 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3874006070 con 0x560f1f5074f0 2023-10-05T19:40:50.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.907+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94622949897504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f50fe10 con 0x560f1f5074f0 2023-10-05T19:40:50.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.907+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f5106f0 con 0x560f1f502d50 2023-10-05T19:40:50.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.907+0000 7f388b72f700 1 -- 172.21.15.12:0/1544794375 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(90 notify.3 [create] v30'213 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f387c03ad70 con 0x560f1f502d50 2023-10-05T19:40:50.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.907+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94622949900736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f510fe0 con 0x560f1f502d50 2023-10-05T19:40:50.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.907+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f5119f0 con 0x560f1f4cd7f0 2023-10-05T19:40:50.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.911+0000 7f388af2e700 1 -- 172.21.15.12:0/1544794375 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(89 notify.2 [watch watch cookie 94622949897504] v30'214 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3874006070 con 0x560f1f5074f0 2023-10-05T19:40:50.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.911+0000 7f388b72f700 1 -- 172.21.15.12:0/1544794375 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(91 notify.3 [watch watch cookie 94622949900736] v30'214 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f387c061040 con 0x560f1f502d50 2023-10-05T19:40:50.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.911+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(92 notify.4 [create] v30'213 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f388402e040 con 0x560f1f4cd7f0 2023-10-05T19:40:50.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.911+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94622949903088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f50ef70 con 0x560f1f4cd7f0 2023-10-05T19:40:50.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.911+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f511ee0 con 0x7f387c044040 2023-10-05T19:40:50.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.915+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(93 notify.4 [watch watch cookie 94622949903088] v30'214 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f388402e040 con 0x560f1f4cd7f0 2023-10-05T19:40:50.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.915+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(94 notify.5 [create] v30'213 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f388402e040 con 0x7f387c044040 2023-10-05T19:40:50.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.915+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94622949905664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f511e40 con 0x7f387c044040 2023-10-05T19:40:50.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.915+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f5126e0 con 0x7f387c044040 2023-10-05T19:40:50.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.919+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(95 notify.5 [watch watch cookie 94622949905664] v30'214 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f388402e040 con 0x7f387c044040 2023-10-05T19:40:50.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.919+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(96 notify.6 [create] v30'213 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f388402e040 con 0x7f387c044040 2023-10-05T19:40:50.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.919+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94622949907328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f5124c0 con 0x7f387c044040 2023-10-05T19:40:50.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.919+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f513020 con 0x7f387c044040 2023-10-05T19:40:50.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.919+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(97 notify.6 [watch watch cookie 94622949907328] v30'214 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f388402e040 con 0x7f387c044040 2023-10-05T19:40:50.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.919+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(98 notify.7 [create] v30'213 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f388402e040 con 0x7f387c044040 2023-10-05T19:40:50.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.919+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94622949908432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f512460 con 0x7f387c044040 2023-10-05T19:40:50.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.919+0000 7f388dbfae40 20 add_watcher() i=0 2023-10-05T19:40:50.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.919+0000 7f388dbfae40 20 add_watcher() i=1 2023-10-05T19:40:50.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.919+0000 7f388dbfae40 20 add_watcher() i=2 2023-10-05T19:40:50.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.919+0000 7f388dbfae40 20 add_watcher() i=3 2023-10-05T19:40:50.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.919+0000 7f388dbfae40 20 add_watcher() i=4 2023-10-05T19:40:50.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.919+0000 7f388dbfae40 20 add_watcher() i=5 2023-10-05T19:40:50.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.919+0000 7f388dbfae40 20 add_watcher() i=6 2023-10-05T19:40:50.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.923+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(99 notify.7 [watch watch cookie 94622949908432] v30'214 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f388402e040 con 0x7f387c044040 2023-10-05T19:40:50.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.923+0000 7f388dbfae40 20 add_watcher() i=7 2023-10-05T19:40:50.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.923+0000 7f388dbfae40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:50.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.923+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f516380 con 0x560f1f4cd7f0 2023-10-05T19:40:50.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.923+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f388402e040 con 0x560f1f4cd7f0 2023-10-05T19:40:50.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.923+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94622949911568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f513870 con 0x560f1f4cd7f0 2023-10-05T19:40:50.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.927+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94622949911568] v30'210 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f388402e040 con 0x560f1f4cd7f0 2023-10-05T19:40:50.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.927+0000 7f385cfe1700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:50.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.927+0000 7f388dbfae40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:50.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.927+0000 7f388dbfae40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:50.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.927+0000 7f388dbfae40 1 Processor -- start 2023-10-05T19:40:50.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.927+0000 7f388dbfae40 1 -- start start 2023-10-05T19:40:50.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.927+0000 7f388dbfae40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f1f520aa0 0x560f1f520e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:50.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.927+0000 7f388dbfae40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f1f5213b0 con 0x560f1f520aa0 2023-10-05T19:40:50.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.927+0000 7f388af2e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f1f520aa0 0x560f1f520e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:50.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.927+0000 7f388af2e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f1f520aa0 0x560f1f520e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58704/0 (socket says 172.21.15.12:58704) 2023-10-05T19:40:50.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.927+0000 7f388af2e700 1 -- 172.21.15.12:0/2600465682 learned_addr learned my addr 172.21.15.12:0/2600465682 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:50.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.931+0000 7f388af2e700 1 -- 172.21.15.12:0/2600465682 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f1f5214f0 con 0x560f1f520aa0 2023-10-05T19:40:50.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.931+0000 7f388af2e700 1 --2- 172.21.15.12:0/2600465682 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f1f520aa0 0x560f1f520e70 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f387c01adc0 tx=0x7f387401e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c05fc3869cd994b server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:50.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.931+0000 7f37d7fff700 1 -- 172.21.15.12:0/2600465682 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3874030040 con 0x560f1f520aa0 2023-10-05T19:40:50.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.931+0000 7f37d7fff700 1 -- 172.21.15.12:0/2600465682 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f387403a560 con 0x560f1f520aa0 2023-10-05T19:40:50.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.931+0000 7f37d7fff700 1 -- 172.21.15.12:0/2600465682 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3874021040 con 0x560f1f520aa0 2023-10-05T19:40:50.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.931+0000 7f388dbfae40 1 -- 172.21.15.12:0/2600465682 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f1f520aa0 msgr2=0x560f1f520e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.931+0000 7f388dbfae40 1 --2- 172.21.15.12:0/2600465682 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f1f520aa0 0x560f1f520e70 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f387c01adc0 tx=0x7f387401e040 comp rx=0 tx=0).stop 2023-10-05T19:40:50.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.931+0000 7f388dbfae40 1 -- 172.21.15.12:0/2600465682 shutdown_connections 2023-10-05T19:40:50.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.931+0000 7f388dbfae40 1 --2- 172.21.15.12:0/2600465682 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f1f520aa0 0x560f1f520e70 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:50.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.931+0000 7f388dbfae40 1 -- 172.21.15.12:0/2600465682 >> 172.21.15.12:0/2600465682 conn(0x560f1f51be60 msgr2=0x560f1f519fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:50.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.931+0000 7f388dbfae40 1 -- 172.21.15.12:0/2600465682 shutdown_connections 2023-10-05T19:40:50.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.931+0000 7f388dbfae40 1 -- 172.21.15.12:0/2600465682 wait complete. 2023-10-05T19:40:50.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.931+0000 7f388dbfae40 1 Processor -- start 2023-10-05T19:40:50.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.931+0000 7f388dbfae40 1 -- start start 2023-10-05T19:40:50.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.931+0000 7f388dbfae40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f1f520aa0 0x560f1f5265e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:50.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.931+0000 7f388dbfae40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f1f5275e0 con 0x560f1f520aa0 2023-10-05T19:40:50.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.931+0000 7f388af2e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f1f520aa0 0x560f1f5265e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:50.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.931+0000 7f388af2e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f1f520aa0 0x560f1f5265e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58716/0 (socket says 172.21.15.12:58716) 2023-10-05T19:40:50.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.931+0000 7f388af2e700 1 -- 172.21.15.12:0/1135975092 learned_addr learned my addr 172.21.15.12:0/1135975092 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:50.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.935+0000 7f388af2e700 1 -- 172.21.15.12:0/1135975092 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3874033040 con 0x560f1f520aa0 2023-10-05T19:40:50.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.935+0000 7f388af2e700 1 --2- 172.21.15.12:0/1135975092 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f1f520aa0 0x560f1f5265e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x560f1f51ab20 tx=0x7f387403c030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:50.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.935+0000 7f37d67fc700 1 -- 172.21.15.12:0/1135975092 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3874030070 con 0x560f1f520aa0 2023-10-05T19:40:50.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.935+0000 7f37d67fc700 1 -- 172.21.15.12:0/1135975092 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f387403a260 con 0x560f1f520aa0 2023-10-05T19:40:50.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.935+0000 7f388dbfae40 1 -- 172.21.15.12:0/1135975092 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560f1f526be0 con 0x560f1f520aa0 2023-10-05T19:40:50.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.935+0000 7f37d67fc700 1 -- 172.21.15.12:0/1135975092 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3874021040 con 0x560f1f520aa0 2023-10-05T19:40:50.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.935+0000 7f388dbfae40 1 -- 172.21.15.12:0/1135975092 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560f1f526fc0 con 0x560f1f520aa0 2023-10-05T19:40:50.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.935+0000 7f37d67fc700 1 -- 172.21.15.12:0/1135975092 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f387402b5a0 con 0x560f1f520aa0 2023-10-05T19:40:50.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.935+0000 7f37d67fc700 1 --2- 172.21.15.12:0/1135975092 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f37cc030c00 0x7f37cc0330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:50.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.935+0000 7f37d67fc700 1 -- 172.21.15.12:0/1135975092 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7f3874050c30 con 0x560f1f520aa0 2023-10-05T19:40:50.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.935+0000 7f388bf30700 1 --2- 172.21.15.12:0/1135975092 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f37cc030c00 0x7f37cc0330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:50.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.935+0000 7f388dbfae40 5 note: GC not initialized 2023-10-05T19:40:50.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.939+0000 7f388bf30700 1 --2- 172.21.15.12:0/1135975092 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f37cc030c00 0x7f37cc0330b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x560f1f512780 tx=0x7f388400ab20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:50.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.939+0000 7f37cb7fe700 20 reqs_thread_entry: start 2023-10-05T19:40:50.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.939+0000 7f37c97fa700 20 reqs_thread_entry: start 2023-10-05T19:40:50.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.939+0000 7f388dbfae40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:50.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.939+0000 7f388dbfae40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:50.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.939+0000 7f37b77fe700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:41:21 2023 2023-10-05T19:40:50.948 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:50.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.939+0000 7f37b6ffd700 20 reqs_thread_entry: start 2023-10-05T19:40:50.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.939+0000 7f388dbfae40 20 get_system_obj_state: rctx=0x7ffc0bf26b58 obj=default.rgw.meta:users.uid:foo state=0x560f1f3a60b0 s->prefetch_data=0 2023-10-05T19:40:50.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.939+0000 7f388dbfae40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2023-10-05T19:40:50.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.939+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f58ee30 con 0x560f1f4cd7f0 2023-10-05T19:40:50.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.943+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(102 foo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7f388402e040 con 0x560f1f4cd7f0 2023-10-05T19:40:50.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.943+0000 7f388dbfae40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x16 2023-10-05T19:40:50.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.943+0000 7f388dbfae40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:50.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.943+0000 7f388dbfae40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:50.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.943+0000 7f388dbfae40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:50.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.943+0000 7f388dbfae40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:50.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.943+0000 7f388dbfae40 10 cache get: name=default.rgw.meta+users.uid+foo : type miss (requested=0x13, cached=0x16) 2023-10-05T19:40:50.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.943+0000 7f388dbfae40 20 rados->read ofs=0 len=0 2023-10-05T19:40:50.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.943+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f5918c0 con 0x560f1f4cd7f0 2023-10-05T19:40:50.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.943+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(103 foo [call,call out=48b,read 0~369 out=369b,getxattrs out=75b] v0'0 uv14 ondisk = 0) v8 ==== 273+0+492 (crc 0 0 0) 0x7f388402e040 con 0x560f1f4cd7f0 2023-10-05T19:40:50.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.943+0000 7f388dbfae40 20 rados_obj.operate() r=0 bl.length=369 2023-10-05T19:40:50.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.943+0000 7f388dbfae40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2023-10-05T19:40:50.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.943+0000 7f388dbfae40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:50.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.943+0000 7f388dbfae40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:50.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.943+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:104 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.set in=58b,writefull 0~357 in=357b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f592b40 con 0x560f1f4cd7f0 2023-10-05T19:40:50.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.947+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(104 foo [delete,create,call,writefull 0~357] v30'15 uv15 ondisk = 0) v8 ==== 273+0+0 (crc 0 0 0) 0x7f388402e040 con 0x560f1f4cd7f0 2023-10-05T19:40:50.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.947+0000 7f388dbfae40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2023-10-05T19:40:50.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.947+0000 7f388dbfae40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:50.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.947+0000 7f388dbfae40 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2023-10-05T19:40:50.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.947+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:105 7.d 7:b4812045:::notify.4:head [notify cookie 94622950434384 in=545b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f5918c0 con 0x560f1f4cd7f0 2023-10-05T19:40:50.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.947+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== watch-notify(notify (1) cookie 94622949903088 notify 128849018897 ret 0) v3 ==== 575+0+0 (crc 0 0 0) 0x7f3884035560 con 0x560f1f4cd7f0 2023-10-05T19:40:50.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.947+0000 7f388972b700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018897 cookie 94622949903088 notifier 4651 bl.length()=533 2023-10-05T19:40:50.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.947+0000 7f388972b700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2023-10-05T19:40:50.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.947+0000 7f388972b700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:50.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.947+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.1 v2:172.21.15.12:6808/29933 39 ==== osd_op_reply(105 notify.4 [notify cookie 94622950434384 out=8b] v0'0 uv213 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f388402e040 con 0x560f1f4cd7f0 2023-10-05T19:40:50.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.947+0000 7f388972b700 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:106 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f3870003d90 con 0x560f1f4cd7f0 2023-10-05T19:40:50.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.947+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.1 v2:172.21.15.12:6808/29933 40 ==== osd_op_reply(106 notify.4 [notify-ack] v0'0 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f388402e040 con 0x560f1f4cd7f0 2023-10-05T19:40:50.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.947+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.1 v2:172.21.15.12:6808/29933 41 ==== watch-notify(notify_complete (2) cookie 94622950434384 notify 128849018897 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f3884035ab0 con 0x560f1f4cd7f0 2023-10-05T19:40:50.956 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:40:50.956 INFO:teuthology.orchestra.run.smithi012.stdout: "user_id": "foo", 2023-10-05T19:40:50.956 INFO:teuthology.orchestra.run.smithi012.stdout: "display_name": "Foo", 2023-10-05T19:40:50.956 INFO:teuthology.orchestra.run.smithi012.stdout: "email": "foo@foo.com", 2023-10-05T19:40:50.956 INFO:teuthology.orchestra.run.smithi012.stdout: "suspended": 0, 2023-10-05T19:40:50.956 INFO:teuthology.orchestra.run.smithi012.stdout: "max_buckets": 4, 2023-10-05T19:40:50.956 INFO:teuthology.orchestra.run.smithi012.stdout: "subusers": [], 2023-10-05T19:40:50.956 INFO:teuthology.orchestra.run.smithi012.stdout: "keys": [ 2023-10-05T19:40:50.957 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:40:50.957 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:40:50.957 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2023-10-05T19:40:50.957 INFO:teuthology.orchestra.run.smithi012.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2023-10-05T19:40:50.957 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:40:50.957 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:40:50.957 INFO:teuthology.orchestra.run.smithi012.stdout: "swift_keys": [], 2023-10-05T19:40:50.957 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": [], 2023-10-05T19:40:50.957 INFO:teuthology.orchestra.run.smithi012.stdout: "op_mask": "read, write, delete", 2023-10-05T19:40:50.957 INFO:teuthology.orchestra.run.smithi012.stdout: "default_placement": "", 2023-10-05T19:40:50.957 INFO:teuthology.orchestra.run.smithi012.stdout: "default_storage_class": "", 2023-10-05T19:40:50.957 INFO:teuthology.orchestra.run.smithi012.stdout: "placement_tags": [], 2023-10-05T19:40:50.958 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket_quota": { 2023-10-05T19:40:50.958 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:50.958 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:50.958 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:50.958 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:50.958 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:50.958 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:50.958 INFO:teuthology.orchestra.run.smithi012.stdout: "user_quota": { 2023-10-05T19:40:50.958 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:40:50.958 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:40:50.958 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:40:50.959 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:40:50.959 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:40:50.959 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:40:50.959 INFO:teuthology.orchestra.run.smithi012.stdout: "temp_url_keys": [], 2023-10-05T19:40:50.959 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "rgw", 2023-10-05T19:40:50.959 INFO:teuthology.orchestra.run.smithi012.stdout: "mfa_ids": [] 2023-10-05T19:40:50.959 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:40:50.959 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:40:50.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.947+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:107 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94622949882144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f4e4a00 con 0x560f1f502d50 2023-10-05T19:40:50.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.951+0000 7f388b72f700 1 -- 172.21.15.12:0/1544794375 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(107 notify.0 [watch unwatch cookie 94622949882144] v30'215 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f387c061040 con 0x560f1f502d50 2023-10-05T19:40:50.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.951+0000 7f388dbfae40 20 remove_watcher() i=0 2023-10-05T19:40:50.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.951+0000 7f388dbfae40 2 removed watcher, disabling cache 2023-10-05T19:40:50.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.951+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:108 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94622949894672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f4e4a00 con 0x560f1f4cd7f0 2023-10-05T19:40:50.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.955+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.1 v2:172.21.15.12:6808/29933 42 ==== osd_op_reply(108 notify.1 [watch unwatch cookie 94622949894672] v30'215 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f388402e040 con 0x560f1f4cd7f0 2023-10-05T19:40:50.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.955+0000 7f388dbfae40 20 remove_watcher() i=1 2023-10-05T19:40:50.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.955+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:109 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94622949897504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f592b20 con 0x560f1f5074f0 2023-10-05T19:40:50.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.955+0000 7f388af2e700 1 -- 172.21.15.12:0/1544794375 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(109 notify.2 [watch unwatch cookie 94622949897504] v30'215 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3874006070 con 0x560f1f5074f0 2023-10-05T19:40:50.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.955+0000 7f388dbfae40 20 remove_watcher() i=2 2023-10-05T19:40:50.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.955+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:110 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94622949900736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f592b20 con 0x560f1f502d50 2023-10-05T19:40:50.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.959+0000 7f388b72f700 1 -- 172.21.15.12:0/1544794375 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(110 notify.3 [watch unwatch cookie 94622949900736] v30'215 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f387c061040 con 0x560f1f502d50 2023-10-05T19:40:50.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.959+0000 7f388dbfae40 20 remove_watcher() i=3 2023-10-05T19:40:50.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.959+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:111 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94622949903088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f592b20 con 0x560f1f4cd7f0 2023-10-05T19:40:50.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.963+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.1 v2:172.21.15.12:6808/29933 43 ==== osd_op_reply(111 notify.4 [watch unwatch cookie 94622949903088] v30'215 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f388402e040 con 0x560f1f4cd7f0 2023-10-05T19:40:50.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.963+0000 7f388dbfae40 20 remove_watcher() i=4 2023-10-05T19:40:50.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.963+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:112 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94622949905664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f592b20 con 0x7f387c044040 2023-10-05T19:40:50.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.967+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(112 notify.5 [watch unwatch cookie 94622949905664] v30'215 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f388402e040 con 0x7f387c044040 2023-10-05T19:40:50.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.967+0000 7f388dbfae40 20 remove_watcher() i=5 2023-10-05T19:40:50.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.967+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:113 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94622949907328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f592b20 con 0x7f387c044040 2023-10-05T19:40:50.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.967+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(113 notify.6 [watch unwatch cookie 94622949907328] v30'215 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f388402e040 con 0x7f387c044040 2023-10-05T19:40:50.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.967+0000 7f388dbfae40 20 remove_watcher() i=6 2023-10-05T19:40:50.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.967+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:114 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94622949908432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f592b20 con 0x7f387c044040 2023-10-05T19:40:50.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.971+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(114 notify.7 [watch unwatch cookie 94622949908432] v30'215 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f388402e040 con 0x7f387c044040 2023-10-05T19:40:50.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.971+0000 7f388dbfae40 20 remove_watcher() i=7 2023-10-05T19:40:50.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.971+0000 7f385cfe1700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:50.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.971+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:115 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94622949911568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560f1f592b20 con 0x560f1f4cd7f0 2023-10-05T19:40:50.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.975+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 <== osd.1 v2:172.21.15.12:6808/29933 44 ==== osd_op_reply(115 data_loggenerations_metadata [watch unwatch cookie 94622949911568] v30'211 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f388402e040 con 0x560f1f4cd7f0 2023-10-05T19:40:50.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.975+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f387c044040 msgr2=0x7f387c039540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.975+0000 7f388dbfae40 1 --2- 172.21.15.12:0/1544794375 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f387c044040 0x7f387c039540 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:50.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.975+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x560f1f4cd7f0 msgr2=0x560f1f500750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.975+0000 7f388dbfae40 1 --2- 172.21.15.12:0/1544794375 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x560f1f4cd7f0 0x560f1f500750 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:50.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.975+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x560f1f502d50 msgr2=0x560f1f505130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.975+0000 7f388dbfae40 1 --2- 172.21.15.12:0/1544794375 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x560f1f502d50 0x560f1f505130 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:50.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.975+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x560f1f5074f0 msgr2=0x560f1f509900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.975+0000 7f388dbfae40 1 --2- 172.21.15.12:0/1544794375 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x560f1f5074f0 0x560f1f509900 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:50.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.975+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f386c030c00 msgr2=0x7f386c0330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.975+0000 7f388dbfae40 1 --2- 172.21.15.12:0/1544794375 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f386c030c00 0x7f386c0330b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3874001e90 tx=0x7f38740013b0 comp rx=0 tx=0).stop 2023-10-05T19:40:50.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.975+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f1f4cd420 msgr2=0x560f1f4f4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.975+0000 7f388bf30700 1 -- 172.21.15.12:0/1544794375 reap_dead start 2023-10-05T19:40:50.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.975+0000 7f388dbfae40 1 --2- 172.21.15.12:0/1544794375 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f1f4cd420 0x560f1f4f4d30 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f387c00a470 tx=0x7f387c01d000 comp rx=0 tx=0).stop 2023-10-05T19:40:50.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.975+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 shutdown_connections 2023-10-05T19:40:50.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.975+0000 7f388dbfae40 1 --2- 172.21.15.12:0/1544794375 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f1f4cd420 0x560f1f4f4d30 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:50.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.975+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 >> 172.21.15.12:0/1544794375 conn(0x560f1f4c7730 msgr2=0x560f1f4e8760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:50.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.975+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 shutdown_connections 2023-10-05T19:40:50.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.975+0000 7f388dbfae40 1 -- 172.21.15.12:0/1544794375 wait complete. 2023-10-05T19:40:50.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.975+0000 7f388dbfae40 1 -- 172.21.15.12:0/1135975092 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f37cc030c00 msgr2=0x7f37cc0330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.975+0000 7f388dbfae40 1 --2- 172.21.15.12:0/1135975092 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f37cc030c00 0x7f37cc0330b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x560f1f512780 tx=0x7f388400ab20 comp rx=0 tx=0).stop 2023-10-05T19:40:50.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.975+0000 7f388dbfae40 1 -- 172.21.15.12:0/1135975092 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f1f520aa0 msgr2=0x560f1f5265e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:50.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.975+0000 7f388dbfae40 1 --2- 172.21.15.12:0/1135975092 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f1f520aa0 0x560f1f5265e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x560f1f51ab20 tx=0x7f387403c030 comp rx=0 tx=0).stop 2023-10-05T19:40:50.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.975+0000 7f388dbfae40 1 -- 172.21.15.12:0/1135975092 shutdown_connections 2023-10-05T19:40:50.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.975+0000 7f388dbfae40 1 --2- 172.21.15.12:0/1135975092 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f37cc030c00 0x7f37cc0330b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:50.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.975+0000 7f388dbfae40 1 --2- 172.21.15.12:0/1135975092 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560f1f520aa0 0x560f1f5265e0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:50.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.975+0000 7f388dbfae40 1 -- 172.21.15.12:0/1135975092 >> 172.21.15.12:0/1135975092 conn(0x560f1f51be60 msgr2=0x560f1f519f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:50.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.979+0000 7f388dbfae40 1 -- 172.21.15.12:0/1135975092 shutdown_connections 2023-10-05T19:40:50.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:50.979+0000 7f388dbfae40 1 -- 172.21.15.12:0/1135975092 wait complete. 2023-10-05T19:40:50.991 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:40:51.013 INFO:tasks.radosgw_admin:RL: create_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2023-10-05T19:40:51.013 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'foo'] 2023-10-05T19:40:51.013 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'foo'] 2023-10-05T19:40:51.013 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid foo 2023-10-05T19:40:51.074 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:40:51.074 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:40:51.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.067+0000 7fba2054ce40 1 Processor -- start 2023-10-05T19:40:51.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.067+0000 7fba2054ce40 1 -- start start 2023-10-05T19:40:51.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.067+0000 7fba2054ce40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x5557d3a87410 0x5557d3a85fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:51.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.067+0000 7fba2054ce40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5557d39d3bd0 con 0x5557d3a877e0 2023-10-05T19:40:51.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.067+0000 7fba2054ce40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x5557d3990ee0 con 0x5557d3a87410 2023-10-05T19:40:51.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.067+0000 7fba1e081700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x5557d3a87410 0x5557d3a85fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:51.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.067+0000 7fba1d880700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x5557d3a877e0 0x5557d3a86500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:60268/0 (socket says 172.21.15.12:60268) 2023-10-05T19:40:51.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.067+0000 7fba1d880700 1 -- 172.21.15.12:0/2538321890 learned_addr learned my addr 172.21.15.12:0/2538321890 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:40:51.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.067+0000 7fba1d07f700 1 -- 172.21.15.12:0/2538321890 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3551020755 0 0) 0x5557d39d3bd0 con 0x5557d3a877e0 2023-10-05T19:40:51.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.067+0000 7fba1d07f700 1 -- 172.21.15.12:0/2538321890 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba0c001e10 con 0x5557d3a877e0 2023-10-05T19:40:51.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.067+0000 7fba1e081700 1 -- 172.21.15.12:0/2538321890 >> v1:172.21.15.8:6789/0 conn(0x5557d3a877e0 legacy=0x5557d3a86500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:51.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.067+0000 7fba1e081700 1 -- 172.21.15.12:0/2538321890 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557d3a86cd0 con 0x5557d3a87410 2023-10-05T19:40:51.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.071+0000 7fba1e081700 1 --2- 172.21.15.12:0/2538321890 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5557d3a87410 0x5557d3a85fc0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fba1000a5d0 tx=0x7fba10005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7fbb85bde1cd672 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:51.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.071+0000 7fba1d07f700 1 -- 172.21.15.12:0/2538321890 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba10007610 con 0x5557d3a87410 2023-10-05T19:40:51.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.071+0000 7fba1d07f700 1 -- 172.21.15.12:0/2538321890 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba10007770 con 0x5557d3a87410 2023-10-05T19:40:51.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.071+0000 7fba1d07f700 1 -- 172.21.15.12:0/2538321890 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba10007a40 con 0x5557d3a87410 2023-10-05T19:40:51.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.071+0000 7fba2054ce40 1 -- 172.21.15.12:0/2538321890 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5557d3a87410 msgr2=0x5557d3a85fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:51.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.071+0000 7fba2054ce40 1 --2- 172.21.15.12:0/2538321890 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5557d3a87410 0x5557d3a85fc0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fba1000a5d0 tx=0x7fba10005a50 comp rx=0 tx=0).stop 2023-10-05T19:40:51.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.071+0000 7fba2054ce40 1 -- 172.21.15.12:0/2538321890 shutdown_connections 2023-10-05T19:40:51.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.071+0000 7fba2054ce40 1 --2- 172.21.15.12:0/2538321890 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5557d3a87410 0x5557d3a85fc0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:51.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.071+0000 7fba2054ce40 1 -- 172.21.15.12:0/2538321890 >> 172.21.15.12:0/2538321890 conn(0x5557d3a81720 msgr2=0x5557d3a83b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:51.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.071+0000 7fba2054ce40 1 -- 172.21.15.12:0/2538321890 shutdown_connections 2023-10-05T19:40:51.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.071+0000 7fba2054ce40 1 -- 172.21.15.12:0/2538321890 wait complete. 2023-10-05T19:40:51.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.071+0000 7fba2054ce40 1 Processor -- start 2023-10-05T19:40:51.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.071+0000 7fba2054ce40 1 -- start start 2023-10-05T19:40:51.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.071+0000 7fba2054ce40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5557d3a87410 0x5557d3aa7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:51.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.071+0000 7fba2054ce40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557d3954e30 con 0x5557d3a87410 2023-10-05T19:40:51.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.071+0000 7fba1e081700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5557d3a87410 0x5557d3aa7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:51.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.071+0000 7fba1e081700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5557d3a87410 0x5557d3aa7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58736/0 (socket says 172.21.15.12:58736) 2023-10-05T19:40:51.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.071+0000 7fba1e081700 1 -- 172.21.15.12:0/4283207162 learned_addr learned my addr 172.21.15.12:0/4283207162 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:51.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.071+0000 7fba1e081700 1 -- 172.21.15.12:0/4283207162 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba10005cf0 con 0x5557d3a87410 2023-10-05T19:40:51.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.075+0000 7fba1e081700 1 --2- 172.21.15.12:0/4283207162 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5557d3a87410 0x5557d3aa7f10 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fba1000a520 tx=0x7fba10014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae6cdcf4208f96 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:51.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.075+0000 7fba17fff700 1 -- 172.21.15.12:0/4283207162 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba10007960 con 0x5557d3a87410 2023-10-05T19:40:51.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.075+0000 7fba17fff700 1 -- 172.21.15.12:0/4283207162 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba10007ae0 con 0x5557d3a87410 2023-10-05T19:40:51.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.075+0000 7fba17fff700 1 -- 172.21.15.12:0/4283207162 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba10007c90 con 0x5557d3a87410 2023-10-05T19:40:51.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.075+0000 7fba2054ce40 1 -- 172.21.15.12:0/4283207162 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5557d3a87410 msgr2=0x5557d3aa7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:51.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.075+0000 7fba2054ce40 1 --2- 172.21.15.12:0/4283207162 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5557d3a87410 0x5557d3aa7f10 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fba1000a520 tx=0x7fba10014000 comp rx=0 tx=0).stop 2023-10-05T19:40:51.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.075+0000 7fba2054ce40 1 -- 172.21.15.12:0/4283207162 shutdown_connections 2023-10-05T19:40:51.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.075+0000 7fba2054ce40 1 --2- 172.21.15.12:0/4283207162 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5557d3a87410 0x5557d3aa7f10 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:51.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.075+0000 7fba2054ce40 1 -- 172.21.15.12:0/4283207162 >> 172.21.15.12:0/4283207162 conn(0x5557d3a81720 msgr2=0x5557d3aa3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:51.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.075+0000 7fba2054ce40 1 -- 172.21.15.12:0/4283207162 shutdown_connections 2023-10-05T19:40:51.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.075+0000 7fba2054ce40 1 -- 172.21.15.12:0/4283207162 wait complete. 2023-10-05T19:40:51.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.075+0000 7fba2054ce40 1 Processor -- start 2023-10-05T19:40:51.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.075+0000 7fba2054ce40 1 -- start start 2023-10-05T19:40:51.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.075+0000 7fba2054ce40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5557d3a87410 0x5557d3aaeb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:51.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.075+0000 7fba2054ce40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557d3927ed0 con 0x5557d3a87410 2023-10-05T19:40:51.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.075+0000 7fba1e081700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5557d3a87410 0x5557d3aaeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:51.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.075+0000 7fba1e081700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5557d3a87410 0x5557d3aaeb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58752/0 (socket says 172.21.15.12:58752) 2023-10-05T19:40:51.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.075+0000 7fba1e081700 1 -- 172.21.15.12:0/2919343689 learned_addr learned my addr 172.21.15.12:0/2919343689 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:51.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.075+0000 7fba1e081700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557d3a86cd0 con 0x5557d3a87410 2023-10-05T19:40:51.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.075+0000 7fba1e081700 1 --2- 172.21.15.12:0/2919343689 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5557d3a87410 0x5557d3aaeb70 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fba1001f040 tx=0x7fba1001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:51.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.079+0000 7fba167fc700 1 -- 172.21.15.12:0/2919343689 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba10007e40 con 0x5557d3a87410 2023-10-05T19:40:51.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.079+0000 7fba167fc700 1 -- 172.21.15.12:0/2919343689 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba100078e0 con 0x5557d3a87410 2023-10-05T19:40:51.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.079+0000 7fba167fc700 1 -- 172.21.15.12:0/2919343689 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba10012610 con 0x5557d3a87410 2023-10-05T19:40:51.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.079+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5557d3aaf0b0 con 0x5557d3a87410 2023-10-05T19:40:51.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.079+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5557d3aaf4f0 con 0x5557d3a87410 2023-10-05T19:40:51.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.079+0000 7fba167fc700 1 -- 172.21.15.12:0/2919343689 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fba1002b070 con 0x5557d3a87410 2023-10-05T19:40:51.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.079+0000 7fba2054ce40 20 rados->read ofs=0 len=0 2023-10-05T19:40:51.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.079+0000 7fba167fc700 1 --2- 172.21.15.12:0/2919343689 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fba00030c50 0x7fba00033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:51.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.079+0000 7fba167fc700 1 -- 172.21.15.12:0/2919343689 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7fba10045a80 con 0x5557d3a87410 2023-10-05T19:40:51.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.079+0000 7fba1d880700 1 --2- 172.21.15.12:0/2919343689 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fba00030c50 0x7fba00033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:51.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.079+0000 7fba2054ce40 1 --2- 172.21.15.12:0/2919343689 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5557d3a877e0 0x5557d3aba5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:51.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.079+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3abac80 con 0x5557d3a877e0 2023-10-05T19:40:51.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.079+0000 7fba1e882700 1 --2- 172.21.15.12:0/2919343689 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5557d3a877e0 0x5557d3aba5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:51.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.083+0000 7fba1d880700 1 --2- 172.21.15.12:0/2919343689 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fba00030c50 0x7fba00033100 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fba08004170 tx=0x7fba08001400 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:51.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.083+0000 7fba1e882700 1 --2- 172.21.15.12:0/2919343689 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5557d3a877e0 0x5557d3aba5d0 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:51.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.083+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3a877e0 2023-10-05T19:40:51.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.083+0000 7fba2054ce40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:51.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.083+0000 7fba2054ce40 20 realm 2023-10-05T19:40:51.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.083+0000 7fba2054ce40 20 rados->read ofs=0 len=0 2023-10-05T19:40:51.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.083+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ab7c30 con 0x5557d3a877e0 2023-10-05T19:40:51.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.083+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3a877e0 2023-10-05T19:40:51.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.083+0000 7fba2054ce40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:51.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.083+0000 7fba2054ce40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:40:51.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.083+0000 7fba2054ce40 20 rados->read ofs=0 len=0 2023-10-05T19:40:51.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.083+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3abb760 con 0x5557d3a877e0 2023-10-05T19:40:51.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.083+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3a877e0 2023-10-05T19:40:51.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.083+0000 7fba2054ce40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:51.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.083+0000 7fba2054ce40 20 rados->read ofs=0 len=0 2023-10-05T19:40:51.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.083+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3abb370 con 0x5557d3a877e0 2023-10-05T19:40:51.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.083+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3a877e0 2023-10-05T19:40:51.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.083+0000 7fba2054ce40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:51.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.083+0000 7fba2054ce40 20 rados->read ofs=0 len=0 2023-10-05T19:40:51.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.083+0000 7fba2054ce40 1 --2- 172.21.15.12:0/2919343689 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5557d3abcb00 0x5557d3abeee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:51.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.083+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3abf420 con 0x5557d3abcb00 2023-10-05T19:40:51.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.087+0000 7fba1e081700 1 --2- 172.21.15.12:0/2919343689 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5557d3abcb00 0x5557d3abeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:51.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.087+0000 7fba1e081700 1 --2- 172.21.15.12:0/2919343689 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5557d3abcb00 0x5557d3abeee0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:51.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.087+0000 7fba1e081700 1 -- 172.21.15.12:0/2919343689 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fba1002a030 con 0x5557d3abcb00 2023-10-05T19:40:51.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.087+0000 7fba2054ce40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:51.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.087+0000 7fba2054ce40 20 searching for the correct realm 2023-10-05T19:40:51.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.087+0000 7fba2054ce40 1 --2- 172.21.15.12:0/2919343689 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5557d3ac1210 0x5557d3ac3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:51.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.087+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac3cd0 con 0x5557d3ac1210 2023-10-05T19:40:51.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.087+0000 7fba1d880700 1 --2- 172.21.15.12:0/2919343689 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5557d3ac1210 0x5557d3ac3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:51.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.087+0000 7fba1d880700 1 --2- 172.21.15.12:0/2919343689 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5557d3ac1210 0x5557d3ac3620 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:51.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.091+0000 7fba1d880700 1 -- 172.21.15.12:0/2919343689 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fba08006070 con 0x5557d3ac1210 2023-10-05T19:40:51.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.091+0000 7fba1d880700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac3cd0 con 0x5557d3abcb00 2023-10-05T19:40:51.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.091+0000 7fba1e081700 1 -- 172.21.15.12:0/2919343689 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1002a030 con 0x5557d3abcb00 2023-10-05T19:40:51.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.091+0000 7fba1e081700 1 --2- 172.21.15.12:0/2919343689 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fba10007030 0x7fba1001b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:51.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.091+0000 7fba1e081700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac3cd0 con 0x7fba10007030 2023-10-05T19:40:51.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.091+0000 7fba1e882700 1 --2- 172.21.15.12:0/2919343689 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fba10007030 0x7fba1001b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:51.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.091+0000 7fba1e882700 1 --2- 172.21.15.12:0/2919343689 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fba10007030 0x7fba1001b970 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:51.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.095+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1800a5a0 con 0x7fba10007030 2023-10-05T19:40:51.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.095+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac3cd0 con 0x5557d3a877e0 2023-10-05T19:40:51.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.095+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3a877e0 2023-10-05T19:40:51.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.095+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac3cd0 con 0x5557d3a877e0 2023-10-05T19:40:51.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.095+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3a877e0 2023-10-05T19:40:51.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.095+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac3cd0 con 0x5557d3a877e0 2023-10-05T19:40:51.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.095+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(11 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3a877e0 2023-10-05T19:40:51.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.095+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac3cd0 con 0x7fba10007030 2023-10-05T19:40:51.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.095+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1800a5a0 con 0x7fba10007030 2023-10-05T19:40:51.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.095+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac3cd0 con 0x5557d3abcb00 2023-10-05T19:40:51.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.099+0000 7fba1e081700 1 -- 172.21.15.12:0/2919343689 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1002a030 con 0x5557d3abcb00 2023-10-05T19:40:51.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.099+0000 7fba1e081700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac3cd0 con 0x5557d3abcb00 2023-10-05T19:40:51.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.099+0000 7fba1e081700 1 -- 172.21.15.12:0/2919343689 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1002a030 con 0x5557d3abcb00 2023-10-05T19:40:51.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.099+0000 7fba1e081700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac3cd0 con 0x5557d3a877e0 2023-10-05T19:40:51.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.099+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3a877e0 2023-10-05T19:40:51.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.099+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac3cd0 con 0x7fba10007030 2023-10-05T19:40:51.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.099+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1800a5a0 con 0x7fba10007030 2023-10-05T19:40:51.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.099+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac3cd0 con 0x5557d3a877e0 2023-10-05T19:40:51.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.099+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3a877e0 2023-10-05T19:40:51.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.099+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac3cd0 con 0x5557d3abcb00 2023-10-05T19:40:51.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.099+0000 7fba1e081700 1 -- 172.21.15.12:0/2919343689 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(18 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1002a030 con 0x5557d3abcb00 2023-10-05T19:40:51.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.099+0000 7fba1e081700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac3cd0 con 0x5557d3ac1210 2023-10-05T19:40:51.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.103+0000 7fba1d880700 1 -- 172.21.15.12:0/2919343689 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba08006070 con 0x5557d3ac1210 2023-10-05T19:40:51.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.103+0000 7fba1d880700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac3cd0 con 0x5557d3ac1210 2023-10-05T19:40:51.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.103+0000 7fba1d880700 1 -- 172.21.15.12:0/2919343689 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba08006070 con 0x5557d3ac1210 2023-10-05T19:40:51.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.103+0000 7fba1d880700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac3cd0 con 0x7fba10007030 2023-10-05T19:40:51.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.103+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1800a5a0 con 0x7fba10007030 2023-10-05T19:40:51.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.103+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac3cd0 con 0x5557d3ac1210 2023-10-05T19:40:51.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.103+0000 7fba1d880700 1 -- 172.21.15.12:0/2919343689 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba08006070 con 0x5557d3ac1210 2023-10-05T19:40:51.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.103+0000 7fba1d880700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac3cd0 con 0x5557d3ac1210 2023-10-05T19:40:51.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.103+0000 7fba1d880700 1 -- 172.21.15.12:0/2919343689 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba08006070 con 0x5557d3ac1210 2023-10-05T19:40:51.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.103+0000 7fba1d880700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac3cd0 con 0x5557d3a877e0 2023-10-05T19:40:51.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.107+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3a877e0 2023-10-05T19:40:51.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.107+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac3cd0 con 0x5557d3a877e0 2023-10-05T19:40:51.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.107+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3a877e0 2023-10-05T19:40:51.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.107+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac3cd0 con 0x7fba10007030 2023-10-05T19:40:51.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.107+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1800a5a0 con 0x7fba10007030 2023-10-05T19:40:51.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.107+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac3cd0 con 0x5557d3ac1210 2023-10-05T19:40:51.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.107+0000 7fba1d880700 1 -- 172.21.15.12:0/2919343689 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba08006070 con 0x5557d3ac1210 2023-10-05T19:40:51.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.107+0000 7fba1d880700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac3cd0 con 0x5557d3abcb00 2023-10-05T19:40:51.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.107+0000 7fba1e081700 1 -- 172.21.15.12:0/2919343689 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1002a030 con 0x5557d3abcb00 2023-10-05T19:40:51.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.107+0000 7fba1e081700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac3cd0 con 0x5557d3a877e0 2023-10-05T19:40:51.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.111+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3a877e0 2023-10-05T19:40:51.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.111+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac3cd0 con 0x7fba10007030 2023-10-05T19:40:51.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.111+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1800a5a0 con 0x7fba10007030 2023-10-05T19:40:51.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.111+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac3cd0 con 0x5557d3ac1210 2023-10-05T19:40:51.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.111+0000 7fba1d880700 1 -- 172.21.15.12:0/2919343689 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba08006070 con 0x5557d3ac1210 2023-10-05T19:40:51.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.111+0000 7fba1d880700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac3cd0 con 0x7fba10007030 2023-10-05T19:40:51.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.111+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1800a5a0 con 0x7fba10007030 2023-10-05T19:40:51.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.111+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac3cd0 con 0x5557d3abcb00 2023-10-05T19:40:51.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.115+0000 7fba1e081700 1 -- 172.21.15.12:0/2919343689 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1002a030 con 0x5557d3abcb00 2023-10-05T19:40:51.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.115+0000 7fba1e081700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac3cd0 con 0x5557d3ac1210 2023-10-05T19:40:51.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.115+0000 7fba1d880700 1 -- 172.21.15.12:0/2919343689 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba08006070 con 0x5557d3ac1210 2023-10-05T19:40:51.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.115+0000 7fba1d880700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac3cd0 con 0x7fba10007030 2023-10-05T19:40:51.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.115+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1800a5a0 con 0x7fba10007030 2023-10-05T19:40:51.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.115+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac3cd0 con 0x5557d3a877e0 2023-10-05T19:40:51.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.115+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3a877e0 2023-10-05T19:40:51.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.115+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac3cd0 con 0x5557d3abcb00 2023-10-05T19:40:51.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.115+0000 7fba1e081700 1 -- 172.21.15.12:0/2919343689 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(37 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fba1002a030 con 0x5557d3abcb00 2023-10-05T19:40:51.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.115+0000 7fba2054ce40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:51.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.115+0000 7fba2054ce40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:51.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.115+0000 7fba2054ce40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:51.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.115+0000 7fba2054ce40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:51.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.115+0000 7fba2054ce40 20 rados->read ofs=0 len=0 2023-10-05T19:40:51.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.115+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3abffc0 con 0x5557d3a877e0 2023-10-05T19:40:51.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.119+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3a877e0 2023-10-05T19:40:51.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.119+0000 7fba2054ce40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:51.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.119+0000 7fba2054ce40 20 rados->read ofs=0 len=0 2023-10-05T19:40:51.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.119+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3abfbf0 con 0x5557d3a877e0 2023-10-05T19:40:51.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.119+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3a877e0 2023-10-05T19:40:51.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.119+0000 7fba2054ce40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:51.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.119+0000 7fba2054ce40 20 rados->read ofs=0 len=0 2023-10-05T19:40:51.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.119+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac4140 con 0x5557d3ac1210 2023-10-05T19:40:51.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.119+0000 7fba1d880700 1 -- 172.21.15.12:0/2919343689 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fba08006070 con 0x5557d3ac1210 2023-10-05T19:40:51.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.119+0000 7fba2054ce40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:51.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.119+0000 7fba2054ce40 20 rados->read ofs=0 len=0 2023-10-05T19:40:51.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.119+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac44f0 con 0x5557d3ac1210 2023-10-05T19:40:51.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.119+0000 7fba1d880700 1 -- 172.21.15.12:0/2919343689 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7fba08006070 con 0x5557d3ac1210 2023-10-05T19:40:51.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.119+0000 7fba2054ce40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:51.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.119+0000 7fba2054ce40 20 rados->read ofs=0 len=0 2023-10-05T19:40:51.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.119+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac48a0 con 0x5557d3a877e0 2023-10-05T19:40:51.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.119+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3a877e0 2023-10-05T19:40:51.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.119+0000 7fba2054ce40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:51.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.119+0000 7fba2054ce40 20 rados->read ofs=0 len=0 2023-10-05T19:40:51.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.119+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac0450 con 0x5557d3a877e0 2023-10-05T19:40:51.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.123+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3a877e0 2023-10-05T19:40:51.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.123+0000 7fba2054ce40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:51.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.123+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac0420 con 0x5557d3ac1210 2023-10-05T19:40:51.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.123+0000 7fba1d880700 1 -- 172.21.15.12:0/2919343689 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(44 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fba08006070 con 0x5557d3ac1210 2023-10-05T19:40:51.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.123+0000 7fba1d880700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac0420 con 0x5557d3abcb00 2023-10-05T19:40:51.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.123+0000 7fba1e081700 1 -- 172.21.15.12:0/2919343689 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(45 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1002a030 con 0x5557d3abcb00 2023-10-05T19:40:51.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.123+0000 7fba1e081700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac0420 con 0x7fba10007030 2023-10-05T19:40:51.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.127+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(46 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1800a5a0 con 0x7fba10007030 2023-10-05T19:40:51.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.127+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac48a0 con 0x5557d3a877e0 2023-10-05T19:40:51.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.127+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(47 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3a877e0 2023-10-05T19:40:51.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.127+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac48a0 con 0x5557d3a877e0 2023-10-05T19:40:51.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.127+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(48 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3a877e0 2023-10-05T19:40:51.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.127+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac48a0 con 0x5557d3a877e0 2023-10-05T19:40:51.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.127+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(49 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3a877e0 2023-10-05T19:40:51.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.127+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac48a0 con 0x7fba10007030 2023-10-05T19:40:51.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.131+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(50 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1800a5a0 con 0x7fba10007030 2023-10-05T19:40:51.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.131+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac48a0 con 0x5557d3abcb00 2023-10-05T19:40:51.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.131+0000 7fba1e081700 1 -- 172.21.15.12:0/2919343689 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(51 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1002a030 con 0x5557d3abcb00 2023-10-05T19:40:51.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.131+0000 7fba1e081700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac48a0 con 0x5557d3abcb00 2023-10-05T19:40:51.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.131+0000 7fba1e081700 1 -- 172.21.15.12:0/2919343689 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(52 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1002a030 con 0x5557d3abcb00 2023-10-05T19:40:51.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.131+0000 7fba1e081700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac48a0 con 0x5557d3a877e0 2023-10-05T19:40:51.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.131+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(53 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3a877e0 2023-10-05T19:40:51.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.131+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac48a0 con 0x7fba10007030 2023-10-05T19:40:51.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.135+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(54 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1800a5a0 con 0x7fba10007030 2023-10-05T19:40:51.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.135+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac48a0 con 0x5557d3a877e0 2023-10-05T19:40:51.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.135+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(55 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3a877e0 2023-10-05T19:40:51.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.135+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac48a0 con 0x5557d3abcb00 2023-10-05T19:40:51.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.135+0000 7fba1e081700 1 -- 172.21.15.12:0/2919343689 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(56 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1002a030 con 0x5557d3abcb00 2023-10-05T19:40:51.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.135+0000 7fba1e081700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac48a0 con 0x5557d3ac1210 2023-10-05T19:40:51.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.135+0000 7fba1d880700 1 -- 172.21.15.12:0/2919343689 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(57 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba08006070 con 0x5557d3ac1210 2023-10-05T19:40:51.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.135+0000 7fba1d880700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac48a0 con 0x5557d3ac1210 2023-10-05T19:40:51.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.135+0000 7fba1d880700 1 -- 172.21.15.12:0/2919343689 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(58 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba08006070 con 0x5557d3ac1210 2023-10-05T19:40:51.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.135+0000 7fba1d880700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac48a0 con 0x7fba10007030 2023-10-05T19:40:51.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.139+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(59 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1800a5a0 con 0x7fba10007030 2023-10-05T19:40:51.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.139+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac48a0 con 0x5557d3ac1210 2023-10-05T19:40:51.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.139+0000 7fba1d880700 1 -- 172.21.15.12:0/2919343689 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(60 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba08006070 con 0x5557d3ac1210 2023-10-05T19:40:51.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.139+0000 7fba1d880700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac48a0 con 0x5557d3ac1210 2023-10-05T19:40:51.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.139+0000 7fba1d880700 1 -- 172.21.15.12:0/2919343689 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(61 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba08006070 con 0x5557d3ac1210 2023-10-05T19:40:51.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.139+0000 7fba1d880700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac48a0 con 0x5557d3a877e0 2023-10-05T19:40:51.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.139+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(62 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3a877e0 2023-10-05T19:40:51.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.139+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac48a0 con 0x5557d3a877e0 2023-10-05T19:40:51.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.139+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(63 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3a877e0 2023-10-05T19:40:51.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.139+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac48a0 con 0x7fba10007030 2023-10-05T19:40:51.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.143+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(64 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1800a5a0 con 0x7fba10007030 2023-10-05T19:40:51.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.143+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac48a0 con 0x5557d3ac1210 2023-10-05T19:40:51.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.143+0000 7fba1d880700 1 -- 172.21.15.12:0/2919343689 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(65 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba08006070 con 0x5557d3ac1210 2023-10-05T19:40:51.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.143+0000 7fba1d880700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac48a0 con 0x5557d3abcb00 2023-10-05T19:40:51.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.143+0000 7fba1e081700 1 -- 172.21.15.12:0/2919343689 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(66 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1002a030 con 0x5557d3abcb00 2023-10-05T19:40:51.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.143+0000 7fba1e081700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac48a0 con 0x5557d3a877e0 2023-10-05T19:40:51.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.143+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(67 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3a877e0 2023-10-05T19:40:51.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.143+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac48a0 con 0x7fba10007030 2023-10-05T19:40:51.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.143+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(68 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1800a5a0 con 0x7fba10007030 2023-10-05T19:40:51.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.147+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac48a0 con 0x5557d3ac1210 2023-10-05T19:40:51.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.147+0000 7fba1d880700 1 -- 172.21.15.12:0/2919343689 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(69 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba08006070 con 0x5557d3ac1210 2023-10-05T19:40:51.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.147+0000 7fba1d880700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac48a0 con 0x7fba10007030 2023-10-05T19:40:51.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.147+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(70 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1800a5a0 con 0x7fba10007030 2023-10-05T19:40:51.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.147+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac48a0 con 0x5557d3abcb00 2023-10-05T19:40:51.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.147+0000 7fba1e081700 1 -- 172.21.15.12:0/2919343689 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(71 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1002a030 con 0x5557d3abcb00 2023-10-05T19:40:51.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.147+0000 7fba1e081700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac48a0 con 0x5557d3ac1210 2023-10-05T19:40:51.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.147+0000 7fba1d880700 1 -- 172.21.15.12:0/2919343689 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(72 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba08006070 con 0x5557d3ac1210 2023-10-05T19:40:51.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.147+0000 7fba1d880700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac48a0 con 0x7fba10007030 2023-10-05T19:40:51.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.151+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(73 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1800a5a0 con 0x7fba10007030 2023-10-05T19:40:51.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.151+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac48a0 con 0x5557d3a877e0 2023-10-05T19:40:51.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.151+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(74 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3a877e0 2023-10-05T19:40:51.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.151+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac48a0 con 0x5557d3abcb00 2023-10-05T19:40:51.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.151+0000 7fba1e081700 1 -- 172.21.15.12:0/2919343689 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(75 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fba1002a030 con 0x5557d3abcb00 2023-10-05T19:40:51.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.151+0000 7fba2054ce40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:40:51.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.151+0000 7fba2054ce40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:40:51.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.151+0000 7fba2054ce40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:40:51.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.151+0000 7fba2054ce40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:40:51.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.151+0000 7fba2054ce40 20 rados->read ofs=0 len=0 2023-10-05T19:40:51.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.151+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac0420 con 0x5557d3a877e0 2023-10-05T19:40:51.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.151+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3a877e0 2023-10-05T19:40:51.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.151+0000 7fba2054ce40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:51.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.151+0000 7fba2054ce40 20 rados->read ofs=0 len=0 2023-10-05T19:40:51.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.151+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac5190 con 0x5557d3abcb00 2023-10-05T19:40:51.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.155+0000 7fba1e081700 1 -- 172.21.15.12:0/2919343689 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fba1002a030 con 0x5557d3abcb00 2023-10-05T19:40:51.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.155+0000 7fba2054ce40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:40:51.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.155+0000 7fba2054ce40 20 rados->read ofs=0 len=0 2023-10-05T19:40:51.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.155+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac5a80 con 0x5557d3a877e0 2023-10-05T19:40:51.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.155+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3a877e0 2023-10-05T19:40:51.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.155+0000 7fba2054ce40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:40:51.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.155+0000 7fba2054ce40 20 rados->read ofs=0 len=0 2023-10-05T19:40:51.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.155+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac5a80 con 0x5557d3ac1210 2023-10-05T19:40:51.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.155+0000 7fba1d880700 1 -- 172.21.15.12:0/2919343689 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fba08006070 con 0x5557d3ac1210 2023-10-05T19:40:51.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.155+0000 7fba2054ce40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:40:51.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.155+0000 7fba2054ce40 20 rados->read ofs=0 len=0 2023-10-05T19:40:51.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.155+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac61f0 con 0x7fba10007030 2023-10-05T19:40:51.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.155+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7fba1800a5a0 con 0x7fba10007030 2023-10-05T19:40:51.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.155+0000 7fba2054ce40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:51.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.155+0000 7fba2054ce40 20 zone default found 2023-10-05T19:40:51.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.155+0000 7fba2054ce40 4 Realm: () 2023-10-05T19:40:51.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.155+0000 7fba2054ce40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:40:51.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.155+0000 7fba2054ce40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:40:51.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.155+0000 7fba2054ce40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:40:51.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.155+0000 7fba2054ce40 20 zonegroup default 2023-10-05T19:40:51.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.155+0000 7fba2054ce40 20 rados->read ofs=0 len=0 2023-10-05T19:40:51.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.155+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac61f0 con 0x7fba10007030 2023-10-05T19:40:51.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.155+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fba1800a5a0 con 0x7fba10007030 2023-10-05T19:40:51.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.155+0000 7fba2054ce40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:51.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.155+0000 7fba2054ce40 20 rados->read ofs=0 len=0 2023-10-05T19:40:51.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.155+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac7b70 con 0x5557d3ac1210 2023-10-05T19:40:51.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.159+0000 7fba1d880700 1 -- 172.21.15.12:0/2919343689 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fba08006070 con 0x5557d3ac1210 2023-10-05T19:40:51.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.159+0000 7fba2054ce40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:51.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.159+0000 7fba2054ce40 20 rados->read ofs=0 len=0 2023-10-05T19:40:51.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.159+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac7f20 con 0x5557d3ac1210 2023-10-05T19:40:51.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.159+0000 7fba1d880700 1 -- 172.21.15.12:0/2919343689 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fba08006070 con 0x5557d3ac1210 2023-10-05T19:40:51.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.159+0000 7fba2054ce40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:40:51.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.159+0000 7fba2054ce40 20 started sync module instance, tier type = 2023-10-05T19:40:51.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.159+0000 7fba2054ce40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:40:51.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.159+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac7230 con 0x5557d3abcb00 2023-10-05T19:40:51.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.163+0000 7fba1e081700 1 -- 172.21.15.12:0/2919343689 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(84 notify.0 [create] v30'216 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba1002a030 con 0x5557d3abcb00 2023-10-05T19:40:51.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.163+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93835701802384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac82d0 con 0x5557d3abcb00 2023-10-05T19:40:51.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.163+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac8b70 con 0x5557d3a877e0 2023-10-05T19:40:51.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.163+0000 7fba1e081700 1 -- 172.21.15.12:0/2919343689 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(85 notify.0 [watch watch cookie 93835701802384] v30'217 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba1002a030 con 0x5557d3abcb00 2023-10-05T19:40:51.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.163+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(86 notify.1 [create] v30'216 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3a877e0 2023-10-05T19:40:51.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.163+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93835701814912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac8fc0 con 0x5557d3a877e0 2023-10-05T19:40:51.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.163+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac9860 con 0x5557d3ac1210 2023-10-05T19:40:51.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.167+0000 7fba1d880700 1 -- 172.21.15.12:0/2919343689 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(88 notify.2 [create] v30'216 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba08006070 con 0x5557d3ac1210 2023-10-05T19:40:51.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.167+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93835701817568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3aca100 con 0x5557d3ac1210 2023-10-05T19:40:51.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.167+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3acab10 con 0x5557d3abcb00 2023-10-05T19:40:51.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.167+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(87 notify.1 [watch watch cookie 93835701814912] v30'217 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3a877e0 2023-10-05T19:40:51.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.171+0000 7fba1e081700 1 -- 172.21.15.12:0/2919343689 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(90 notify.3 [create] v30'216 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba1002a030 con 0x5557d3abcb00 2023-10-05T19:40:51.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.171+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 93835701816016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3aca740 con 0x5557d3abcb00 2023-10-05T19:40:51.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.171+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3acbab0 con 0x5557d3a877e0 2023-10-05T19:40:51.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.171+0000 7fba1d880700 1 -- 172.21.15.12:0/2919343689 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(89 notify.2 [watch watch cookie 93835701817568] v30'217 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba08006070 con 0x5557d3ac1210 2023-10-05T19:40:51.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.175+0000 7fba1e081700 1 -- 172.21.15.12:0/2919343689 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(91 notify.3 [watch watch cookie 93835701816016] v30'217 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba1002a030 con 0x5557d3abcb00 2023-10-05T19:40:51.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.175+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(92 notify.4 [create] v30'216 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3a877e0 2023-10-05T19:40:51.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.175+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 93835701825216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3aca740 con 0x5557d3a877e0 2023-10-05T19:40:51.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.175+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3acb6f0 con 0x7fba10007030 2023-10-05T19:40:51.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.175+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(93 notify.4 [watch watch cookie 93835701825216] v30'217 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3a877e0 2023-10-05T19:40:51.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.175+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(94 notify.5 [create] v30'216 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba1800a5a0 con 0x7fba10007030 2023-10-05T19:40:51.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.175+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 93835701820432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3acc280 con 0x7fba10007030 2023-10-05T19:40:51.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.175+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3accc90 con 0x7fba10007030 2023-10-05T19:40:51.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.179+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(95 notify.5 [watch watch cookie 93835701820432] v30'217 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba1800a5a0 con 0x7fba10007030 2023-10-05T19:40:51.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.179+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(96 notify.6 [create] v30'216 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba1800a5a0 con 0x7fba10007030 2023-10-05T19:40:51.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.179+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93835701829568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3accc90 con 0x7fba10007030 2023-10-05T19:40:51.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.179+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3acd6a0 con 0x7fba10007030 2023-10-05T19:40:51.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.183+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(97 notify.6 [watch watch cookie 93835701829568] v30'217 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba1800a5a0 con 0x7fba10007030 2023-10-05T19:40:51.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.183+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(98 notify.7 [create] v30'216 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba1800a5a0 con 0x7fba10007030 2023-10-05T19:40:51.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.183+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93835701832144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3acd6a0 con 0x7fba10007030 2023-10-05T19:40:51.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.183+0000 7fba2054ce40 20 add_watcher() i=0 2023-10-05T19:40:51.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.183+0000 7fba2054ce40 20 add_watcher() i=1 2023-10-05T19:40:51.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.183+0000 7fba2054ce40 20 add_watcher() i=2 2023-10-05T19:40:51.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.183+0000 7fba2054ce40 20 add_watcher() i=3 2023-10-05T19:40:51.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.183+0000 7fba2054ce40 20 add_watcher() i=4 2023-10-05T19:40:51.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.183+0000 7fba2054ce40 20 add_watcher() i=5 2023-10-05T19:40:51.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.183+0000 7fba2054ce40 20 add_watcher() i=6 2023-10-05T19:40:51.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.183+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(99 notify.7 [watch watch cookie 93835701832144] v30'217 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba1800a5a0 con 0x7fba10007030 2023-10-05T19:40:51.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.183+0000 7fba2054ce40 20 add_watcher() i=7 2023-10-05T19:40:51.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.183+0000 7fba2054ce40 2 all 8 watchers are set, enabling cache 2023-10-05T19:40:51.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.183+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ad0880 con 0x5557d3a877e0 2023-10-05T19:40:51.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.187+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3a877e0 2023-10-05T19:40:51.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.187+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93835701834528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3acde00 con 0x5557d3a877e0 2023-10-05T19:40:51.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.187+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 93835701834528] v30'212 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3a877e0 2023-10-05T19:40:51.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.187+0000 7fb9effdf700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:51.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.187+0000 7fba2054ce40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:40:51.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.187+0000 7fba2054ce40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:40:51.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.187+0000 7fba2054ce40 1 Processor -- start 2023-10-05T19:40:51.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.187+0000 7fba2054ce40 1 -- start start 2023-10-05T19:40:51.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.191+0000 7fba2054ce40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5557d3adb0b0 0x5557d3adb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:51.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.191+0000 7fba2054ce40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557d3adb9c0 con 0x5557d3adb0b0 2023-10-05T19:40:51.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.191+0000 7fba1d880700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5557d3adb0b0 0x5557d3adb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:51.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.191+0000 7fba1d880700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5557d3adb0b0 0x5557d3adb480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58764/0 (socket says 172.21.15.12:58764) 2023-10-05T19:40:51.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.191+0000 7fba1d880700 1 -- 172.21.15.12:0/1412223935 learned_addr learned my addr 172.21.15.12:0/1412223935 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:51.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.191+0000 7fba1d880700 1 -- 172.21.15.12:0/1412223935 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557d3adbb00 con 0x5557d3adb0b0 2023-10-05T19:40:51.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.191+0000 7fba1d880700 1 --2- 172.21.15.12:0/1412223935 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5557d3adb0b0 0x5557d3adb480 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x5557d3abc5a0 tx=0x7fba0801e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20989be783e52c27 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:51.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.191+0000 7fb9ef7de700 1 -- 172.21.15.12:0/1412223935 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba08030040 con 0x5557d3adb0b0 2023-10-05T19:40:51.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.191+0000 7fb9ef7de700 1 -- 172.21.15.12:0/1412223935 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba0800edf0 con 0x5557d3adb0b0 2023-10-05T19:40:51.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.191+0000 7fb9ef7de700 1 -- 172.21.15.12:0/1412223935 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba08021040 con 0x5557d3adb0b0 2023-10-05T19:40:51.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.191+0000 7fba2054ce40 1 -- 172.21.15.12:0/1412223935 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5557d3adb0b0 msgr2=0x5557d3adb480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:51.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.191+0000 7fba2054ce40 1 --2- 172.21.15.12:0/1412223935 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5557d3adb0b0 0x5557d3adb480 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x5557d3abc5a0 tx=0x7fba0801e040 comp rx=0 tx=0).stop 2023-10-05T19:40:51.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.191+0000 7fba2054ce40 1 -- 172.21.15.12:0/1412223935 shutdown_connections 2023-10-05T19:40:51.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.191+0000 7fba2054ce40 1 --2- 172.21.15.12:0/1412223935 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5557d3adb0b0 0x5557d3adb480 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:51.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.191+0000 7fba2054ce40 1 -- 172.21.15.12:0/1412223935 >> 172.21.15.12:0/1412223935 conn(0x5557d3ad6470 msgr2=0x5557d3ad45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:51.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.191+0000 7fba2054ce40 1 -- 172.21.15.12:0/1412223935 shutdown_connections 2023-10-05T19:40:51.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.191+0000 7fba2054ce40 1 -- 172.21.15.12:0/1412223935 wait complete. 2023-10-05T19:40:51.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.195+0000 7fba2054ce40 1 Processor -- start 2023-10-05T19:40:51.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.195+0000 7fba2054ce40 1 -- start start 2023-10-05T19:40:51.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.195+0000 7fba2054ce40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5557d3adb0b0 0x5557d3ae0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:51.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.195+0000 7fba2054ce40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557d3ae1cb0 con 0x5557d3adb0b0 2023-10-05T19:40:51.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.195+0000 7fba1d880700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5557d3adb0b0 0x5557d3ae0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:51.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.195+0000 7fba1d880700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5557d3adb0b0 0x5557d3ae0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:58766/0 (socket says 172.21.15.12:58766) 2023-10-05T19:40:51.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.195+0000 7fba1d880700 1 -- 172.21.15.12:0/274825098 learned_addr learned my addr 172.21.15.12:0/274825098 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:40:51.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.195+0000 7fba1d880700 1 -- 172.21.15.12:0/274825098 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba08033040 con 0x5557d3adb0b0 2023-10-05T19:40:51.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.195+0000 7fba1d880700 1 --2- 172.21.15.12:0/274825098 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5557d3adb0b0 0x5557d3ae0cb0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fba08001880 tx=0x7fba0803c030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:51.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.195+0000 7fb9edfdb700 1 -- 172.21.15.12:0/274825098 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba08021070 con 0x5557d3adb0b0 2023-10-05T19:40:51.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.195+0000 7fb9edfdb700 1 -- 172.21.15.12:0/274825098 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba0802b730 con 0x5557d3adb0b0 2023-10-05T19:40:51.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.195+0000 7fba2054ce40 1 -- 172.21.15.12:0/274825098 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5557d3ae12b0 con 0x5557d3adb0b0 2023-10-05T19:40:51.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.195+0000 7fb9edfdb700 1 -- 172.21.15.12:0/274825098 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba08030040 con 0x5557d3adb0b0 2023-10-05T19:40:51.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.195+0000 7fba2054ce40 1 -- 172.21.15.12:0/274825098 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5557d3ae16f0 con 0x5557d3adb0b0 2023-10-05T19:40:51.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.199+0000 7fb9edfdb700 1 -- 172.21.15.12:0/274825098 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fba0802b8a0 con 0x5557d3adb0b0 2023-10-05T19:40:51.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.199+0000 7fb9edfdb700 1 --2- 172.21.15.12:0/274825098 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fb964030c50 0x7fb964033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:51.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.199+0000 7fb9edfdb700 1 -- 172.21.15.12:0/274825098 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7fba0804fbe0 con 0x5557d3adb0b0 2023-10-05T19:40:51.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.199+0000 7fba1e882700 1 --2- 172.21.15.12:0/274825098 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fb964030c50 0x7fb964033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:51.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.199+0000 7fba2054ce40 1 --2- 172.21.15.12:0/274825098 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5557d3aeb870 0x5557d3aedc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:51.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.199+0000 7fba2054ce40 1 -- 172.21.15.12:0/274825098 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3aee2d0 con 0x5557d3aeb870 2023-10-05T19:40:51.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.199+0000 7fba1e081700 1 --2- 172.21.15.12:0/274825098 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5557d3aeb870 0x5557d3aedc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:51.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.199+0000 7fba1e882700 1 --2- 172.21.15.12:0/274825098 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fb964030c50 0x7fb964033100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x5557d3acdd60 tx=0x7fba1800aed0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:51.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.199+0000 7fba1e081700 1 --2- 172.21.15.12:0/274825098 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5557d3aeb870 0x5557d3aedc80 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:51.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.203+0000 7fba1e081700 1 -- 172.21.15.12:0/274825098 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v30'34 uv30 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fba1002a030 con 0x5557d3aeb870 2023-10-05T19:40:51.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.203+0000 7fba2054ce40 1 --2- 172.21.15.12:0/274825098 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x5557d3aeefe0 0x5557d3af13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:51.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.203+0000 7fba2054ce40 1 -- 172.21.15.12:0/274825098 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3af1a30 con 0x5557d3aeefe0 2023-10-05T19:40:51.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.203+0000 7fba1d880700 1 --2- 172.21.15.12:0/274825098 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x5557d3aeefe0 0x5557d3af13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:51.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.203+0000 7fba1d880700 1 --2- 172.21.15.12:0/274825098 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x5557d3aeefe0 0x5557d3af13f0 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:51.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.207+0000 7fba1d880700 1 -- 172.21.15.12:0/274825098 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v30'25 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fba08006070 con 0x5557d3aeefe0 2023-10-05T19:40:51.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.207+0000 7fba2054ce40 1 --2- 172.21.15.12:0/274825098 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5557d3af2700 0x5557d3af4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:51.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.207+0000 7fba2054ce40 1 -- 172.21.15.12:0/274825098 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3af5210 con 0x5557d3af2700 2023-10-05T19:40:51.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.207+0000 7fba1e882700 1 --2- 172.21.15.12:0/274825098 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5557d3af2700 0x5557d3af4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:51.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.207+0000 7fba1e882700 1 --2- 172.21.15.12:0/274825098 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5557d3af2700 0x5557d3af4ae0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:51.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.207+0000 7fba1e882700 1 -- 172.21.15.12:0/274825098 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v30'25 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3af2700 2023-10-05T19:40:51.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.207+0000 7fba2054ce40 1 -- 172.21.15.12:0/274825098 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3af5860 con 0x5557d3aeb870 2023-10-05T19:40:51.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.211+0000 7fba1e081700 1 -- 172.21.15.12:0/274825098 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v30'213 uv185 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fba1002a030 con 0x5557d3aeb870 2023-10-05T19:40:51.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.211+0000 7fba2054ce40 1 --2- 172.21.15.12:0/274825098 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5557d3af6500 0x5557d3af88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:40:51.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.211+0000 7fba2054ce40 1 -- 172.21.15.12:0/274825098 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3af8f90 con 0x5557d3af6500 2023-10-05T19:40:51.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.211+0000 7fba1e081700 1 --2- 172.21.15.12:0/274825098 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5557d3af6500 0x5557d3af88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:40:51.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.211+0000 7fba1e081700 1 --2- 172.21.15.12:0/274825098 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5557d3af6500 0x5557d3af88e0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:40:51.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.215+0000 7fba1e081700 1 -- 172.21.15.12:0/274825098 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v30'65 uv51 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fba1002a030 con 0x5557d3af6500 2023-10-05T19:40:51.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.215+0000 7fba2054ce40 1 -- 172.21.15.12:0/274825098 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3af9480 con 0x5557d3af2700 2023-10-05T19:40:51.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.219+0000 7fba1e882700 1 -- 172.21.15.12:0/274825098 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v30'46 uv39 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3af2700 2023-10-05T19:40:51.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.219+0000 7fba2054ce40 1 -- 172.21.15.12:0/274825098 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ae71f0 con 0x5557d3aeb870 2023-10-05T19:40:51.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.219+0000 7fba1e081700 1 -- 172.21.15.12:0/274825098 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v30'27 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fba1002a030 con 0x5557d3aeb870 2023-10-05T19:40:51.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.219+0000 7fba2054ce40 1 -- 172.21.15.12:0/274825098 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ae71f0 con 0x5557d3aeefe0 2023-10-05T19:40:51.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.223+0000 7fba1d880700 1 -- 172.21.15.12:0/274825098 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v30'43 uv36 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fba08006070 con 0x5557d3aeefe0 2023-10-05T19:40:51.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.223+0000 7fba2054ce40 1 -- 172.21.15.12:0/274825098 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3af9fa0 con 0x5557d3af6500 2023-10-05T19:40:51.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.227+0000 7fba1e081700 1 -- 172.21.15.12:0/274825098 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v30'25 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fba1002a030 con 0x5557d3af6500 2023-10-05T19:40:51.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.227+0000 7fba2054ce40 1 -- 172.21.15.12:0/274825098 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3af9fa0 con 0x5557d3af2700 2023-10-05T19:40:51.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.227+0000 7fba1e882700 1 -- 172.21.15.12:0/274825098 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v30'31 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3af2700 2023-10-05T19:40:51.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.227+0000 7fba2054ce40 1 -- 172.21.15.12:0/274825098 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3afa5c0 con 0x5557d3af6500 2023-10-05T19:40:51.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.231+0000 7fba1e081700 1 -- 172.21.15.12:0/274825098 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v30'66 uv53 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fba1002a030 con 0x5557d3af6500 2023-10-05T19:40:51.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.231+0000 7fba2054ce40 1 -- 172.21.15.12:0/274825098 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3afa5c0 con 0x5557d3aeefe0 2023-10-05T19:40:51.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.235+0000 7fba1d880700 1 -- 172.21.15.12:0/274825098 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v30'29 uv24 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fba08006070 con 0x5557d3aeefe0 2023-10-05T19:40:51.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.235+0000 7fba2054ce40 1 -- 172.21.15.12:0/274825098 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3afa5c0 con 0x5557d3af6500 2023-10-05T19:40:51.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.235+0000 7fba1e081700 1 -- 172.21.15.12:0/274825098 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v30'36 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fba1002a030 con 0x5557d3af6500 2023-10-05T19:40:51.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.235+0000 7fba2054ce40 1 -- 172.21.15.12:0/274825098 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3afa5c0 con 0x5557d3aeefe0 2023-10-05T19:40:51.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.239+0000 7fba1d880700 1 -- 172.21.15.12:0/274825098 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v30'52 uv45 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fba08006070 con 0x5557d3aeefe0 2023-10-05T19:40:51.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.239+0000 7fba2054ce40 1 -- 172.21.15.12:0/274825098 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3afa5c0 con 0x5557d3aeb870 2023-10-05T19:40:51.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.243+0000 7fba1e081700 1 -- 172.21.15.12:0/274825098 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v30'214 uv187 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fba1002a030 con 0x5557d3aeb870 2023-10-05T19:40:51.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.243+0000 7fba2054ce40 1 -- 172.21.15.12:0/274825098 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3afa5c0 con 0x5557d3af2700 2023-10-05T19:40:51.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.247+0000 7fba1e882700 1 -- 172.21.15.12:0/274825098 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v30'65 uv49 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3af2700 2023-10-05T19:40:51.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.247+0000 7fba2054ce40 1 -- 172.21.15.12:0/274825098 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3afa820 con 0x5557d3af6500 2023-10-05T19:40:51.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.247+0000 7fba1e081700 1 -- 172.21.15.12:0/274825098 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v30'67 uv55 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fba1002a030 con 0x5557d3af6500 2023-10-05T19:40:51.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.247+0000 7fba2054ce40 1 -- 172.21.15.12:0/274825098 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3afa820 con 0x5557d3af2700 2023-10-05T19:40:51.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.251+0000 7fba1e882700 1 -- 172.21.15.12:0/274825098 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v30'25 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3af2700 2023-10-05T19:40:51.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.251+0000 7fba2054ce40 1 -- 172.21.15.12:0/274825098 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3afa790 con 0x5557d3aeb870 2023-10-05T19:40:51.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.255+0000 7fba1e081700 1 -- 172.21.15.12:0/274825098 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v30'49 uv41 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fba1002a030 con 0x5557d3aeb870 2023-10-05T19:40:51.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.255+0000 7fba2054ce40 1 -- 172.21.15.12:0/274825098 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3afa700 con 0x5557d3aeb870 2023-10-05T19:40:51.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.255+0000 7fba1e081700 1 -- 172.21.15.12:0/274825098 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v30'215 uv183 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fba1002a030 con 0x5557d3aeb870 2023-10-05T19:40:51.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.255+0000 7fba2054ce40 1 -- 172.21.15.12:0/274825098 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3afa700 con 0x5557d3af2700 2023-10-05T19:40:51.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.259+0000 7fba1e882700 1 -- 172.21.15.12:0/274825098 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v30'66 uv45 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3af2700 2023-10-05T19:40:51.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.259+0000 7fba2054ce40 1 -- 172.21.15.12:0/274825098 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3afa700 con 0x5557d3af6500 2023-10-05T19:40:51.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.263+0000 7fba1e081700 1 -- 172.21.15.12:0/274825098 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v30'35 uv31 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fba1002a030 con 0x5557d3af6500 2023-10-05T19:40:51.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.263+0000 7fba2054ce40 1 -- 172.21.15.12:0/274825098 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3afa700 con 0x5557d3af2700 2023-10-05T19:40:51.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.263+0000 7fba1e882700 1 -- 172.21.15.12:0/274825098 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v30'29 uv25 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3af2700 2023-10-05T19:40:51.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.263+0000 7fba2054ce40 1 -- 172.21.15.12:0/274825098 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3afa7f0 con 0x5557d3af2700 2023-10-05T19:40:51.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.267+0000 7fba1e882700 1 -- 172.21.15.12:0/274825098 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v30'34 uv30 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3af2700 2023-10-05T19:40:51.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.267+0000 7fba2054ce40 1 -- 172.21.15.12:0/274825098 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3afa7f0 con 0x5557d3af2700 2023-10-05T19:40:51.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.271+0000 7fba1e882700 1 -- 172.21.15.12:0/274825098 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v30'22 uv18 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3af2700 2023-10-05T19:40:51.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.271+0000 7fba2054ce40 1 -- 172.21.15.12:0/274825098 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3afa790 con 0x5557d3af2700 2023-10-05T19:40:51.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.271+0000 7fba1e882700 1 -- 172.21.15.12:0/274825098 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v30'47 uv37 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3af2700 2023-10-05T19:40:51.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.271+0000 7fba2054ce40 1 -- 172.21.15.12:0/274825098 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3afa7c0 con 0x5557d3aeefe0 2023-10-05T19:40:51.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.275+0000 7fba1d880700 1 -- 172.21.15.12:0/274825098 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v30'31 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fba08006070 con 0x5557d3aeefe0 2023-10-05T19:40:51.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.275+0000 7fba2054ce40 1 -- 172.21.15.12:0/274825098 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3afad70 con 0x5557d3af2700 2023-10-05T19:40:51.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.279+0000 7fba1e882700 1 -- 172.21.15.12:0/274825098 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v30'25 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3af2700 2023-10-05T19:40:51.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.279+0000 7fba2054ce40 1 -- 172.21.15.12:0/274825098 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3afad70 con 0x5557d3aeefe0 2023-10-05T19:40:51.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.279+0000 7fba1d880700 1 -- 172.21.15.12:0/274825098 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v30'44 uv34 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fba08006070 con 0x5557d3aeefe0 2023-10-05T19:40:51.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.279+0000 7fba2054ce40 1 -- 172.21.15.12:0/274825098 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3afad70 con 0x5557d3af2700 2023-10-05T19:40:51.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.283+0000 7fba1e882700 1 -- 172.21.15.12:0/274825098 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v30'67 uv47 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3af2700 2023-10-05T19:40:51.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.283+0000 7fba2054ce40 1 -- 172.21.15.12:0/274825098 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3afa7c0 con 0x5557d3aeefe0 2023-10-05T19:40:51.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.287+0000 7fba1d880700 1 -- 172.21.15.12:0/274825098 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v30'53 uv43 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fba08006070 con 0x5557d3aeefe0 2023-10-05T19:40:51.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.287+0000 7fba2054ce40 1 -- 172.21.15.12:0/274825098 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3afa7c0 con 0x5557d3af6500 2023-10-05T19:40:51.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.291+0000 7fba1e081700 1 -- 172.21.15.12:0/274825098 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v30'31 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fba1002a030 con 0x5557d3af6500 2023-10-05T19:40:51.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.291+0000 7fb95effd700 20 reqs_thread_entry: start 2023-10-05T19:40:51.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.291+0000 7fba2054ce40 20 init_complete bucket index max shards: 11 2023-10-05T19:40:51.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.291+0000 7fb95cff9700 20 reqs_thread_entry: start 2023-10-05T19:40:51.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.295+0000 7fba2054ce40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:40:51.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.295+0000 7fb94effd700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:41:21 2023 2023-10-05T19:40:51.302 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:40:51.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.295+0000 7fb94e7fc700 20 reqs_thread_entry: start 2023-10-05T19:40:51.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.295+0000 7fba2054ce40 20 get_system_obj_state: rctx=0x7ffd5b29c478 obj=default.rgw.meta:users.uid:foo state=0x5557d39600b0 s->prefetch_data=0 2023-10-05T19:40:51.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.295+0000 7fba2054ce40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2023-10-05T19:40:51.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.295+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3b592b0 con 0x5557d3a877e0 2023-10-05T19:40:51.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.295+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(102 foo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv15 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3a877e0 2023-10-05T19:40:51.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.295+0000 7fba2054ce40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x16 2023-10-05T19:40:51.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.295+0000 7fba2054ce40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:51.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.295+0000 7fba2054ce40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:51.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.295+0000 7fba2054ce40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:40:51.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.295+0000 7fba2054ce40 20 Read xattr: user.rgw.idtag 2023-10-05T19:40:51.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.295+0000 7fba2054ce40 10 cache get: name=default.rgw.meta+users.uid+foo : type miss (requested=0x13, cached=0x16) 2023-10-05T19:40:51.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.295+0000 7fba2054ce40 20 rados->read ofs=0 len=0 2023-10-05T19:40:51.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.295+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3b5bd40 con 0x5557d3a877e0 2023-10-05T19:40:51.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.295+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(103 foo [call,call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv15 ondisk = 0) v8 ==== 273+0+480 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3a877e0 2023-10-05T19:40:51.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.295+0000 7fba2054ce40 20 rados_obj.operate() r=0 bl.length=357 2023-10-05T19:40:51.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.295+0000 7fba2054ce40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2023-10-05T19:40:51.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.295+0000 7fba2054ce40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:40:51.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.295+0000 7fba2054ce40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:40:51.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.295+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:104 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3b5bd40 con 0x5557d3abcb00 2023-10-05T19:40:51.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.295+0000 7fba1e081700 1 -- 172.21.15.12:0/2919343689 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(104 foo.buckets [call out=183b] v0'0 uv40 ondisk = 0) v8 ==== 155+0+183 (crc 0 0 0) 0x7fba1002a030 con 0x5557d3abcb00 2023-10-05T19:40:51.305 INFO:teuthology.orchestra.run.smithi012.stderr:could not remove user: unable to remove user, must specify purge data to remove user with buckets 2023-10-05T19:40:51.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.299+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:105 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93835701802384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3b17860 con 0x5557d3abcb00 2023-10-05T19:40:51.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.299+0000 7fba1e081700 1 -- 172.21.15.12:0/2919343689 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(105 notify.0 [watch unwatch cookie 93835701802384] v30'218 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba1002a030 con 0x5557d3abcb00 2023-10-05T19:40:51.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.299+0000 7fba2054ce40 20 remove_watcher() i=0 2023-10-05T19:40:51.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.299+0000 7fba2054ce40 2 removed watcher, disabling cache 2023-10-05T19:40:51.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.299+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:106 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93835701814912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac5590 con 0x5557d3a877e0 2023-10-05T19:40:51.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.303+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(106 notify.1 [watch unwatch cookie 93835701814912] v30'218 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3a877e0 2023-10-05T19:40:51.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.303+0000 7fba2054ce40 20 remove_watcher() i=1 2023-10-05T19:40:51.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.303+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:107 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93835701817568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac8680 con 0x5557d3ac1210 2023-10-05T19:40:51.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.307+0000 7fba1d880700 1 -- 172.21.15.12:0/2919343689 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(107 notify.2 [watch unwatch cookie 93835701817568] v30'218 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba08006070 con 0x5557d3ac1210 2023-10-05T19:40:51.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.307+0000 7fba2054ce40 20 remove_watcher() i=2 2023-10-05T19:40:51.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.307+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:108 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93835701816016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac8680 con 0x5557d3abcb00 2023-10-05T19:40:51.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.311+0000 7fba1e081700 1 -- 172.21.15.12:0/2919343689 <== osd.2 v2:172.21.15.12:6800/29934 23 ==== osd_op_reply(108 notify.3 [watch unwatch cookie 93835701816016] v30'218 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba1002a030 con 0x5557d3abcb00 2023-10-05T19:40:51.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.311+0000 7fba2054ce40 20 remove_watcher() i=3 2023-10-05T19:40:51.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.311+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:109 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93835701825216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac90e0 con 0x5557d3a877e0 2023-10-05T19:40:51.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.311+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== osd_op_reply(109 notify.4 [watch unwatch cookie 93835701825216] v30'218 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3a877e0 2023-10-05T19:40:51.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.311+0000 7fba2054ce40 20 remove_watcher() i=4 2023-10-05T19:40:51.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.311+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:110 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93835701820432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac90e0 con 0x7fba10007030 2023-10-05T19:40:51.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.315+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(110 notify.5 [watch unwatch cookie 93835701820432] v30'218 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba1800a5a0 con 0x7fba10007030 2023-10-05T19:40:51.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.315+0000 7fba2054ce40 20 remove_watcher() i=5 2023-10-05T19:40:51.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.315+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:111 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93835701829568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac9c10 con 0x7fba10007030 2023-10-05T19:40:51.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.319+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(111 notify.6 [watch unwatch cookie 93835701829568] v30'218 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba1800a5a0 con 0x7fba10007030 2023-10-05T19:40:51.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.319+0000 7fba2054ce40 20 remove_watcher() i=6 2023-10-05T19:40:51.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.319+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:112 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93835701832144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac9c10 con 0x7fba10007030 2023-10-05T19:40:51.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.323+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(112 notify.7 [watch unwatch cookie 93835701832144] v30'218 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba1800a5a0 con 0x7fba10007030 2023-10-05T19:40:51.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.323+0000 7fba2054ce40 20 remove_watcher() i=7 2023-10-05T19:40:51.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.323+0000 7fb9effdf700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:40:51.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.323+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:113 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93835701834528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5557d3ac9c10 con 0x5557d3a877e0 2023-10-05T19:40:51.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.327+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 <== osd.1 v2:172.21.15.12:6808/29933 39 ==== osd_op_reply(113 data_loggenerations_metadata [watch unwatch cookie 93835701834528] v30'216 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fba1800a5a0 con 0x5557d3a877e0 2023-10-05T19:40:51.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.327+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fba10007030 msgr2=0x7fba1001b970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:51.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.327+0000 7fba2054ce40 1 --2- 172.21.15.12:0/2919343689 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fba10007030 0x7fba1001b970 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:51.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.327+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5557d3a877e0 msgr2=0x5557d3aba5d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:51.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.327+0000 7fba2054ce40 1 --2- 172.21.15.12:0/2919343689 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5557d3a877e0 0x5557d3aba5d0 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:51.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.327+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5557d3abcb00 msgr2=0x5557d3abeee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:51.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.327+0000 7fba2054ce40 1 --2- 172.21.15.12:0/2919343689 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5557d3abcb00 0x5557d3abeee0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:51.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.327+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5557d3ac1210 msgr2=0x5557d3ac3620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:51.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.327+0000 7fba2054ce40 1 --2- 172.21.15.12:0/2919343689 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5557d3ac1210 0x5557d3ac3620 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:51.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.327+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fba00030c50 msgr2=0x7fba00033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:51.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.327+0000 7fba2054ce40 1 --2- 172.21.15.12:0/2919343689 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fba00030c50 0x7fba00033100 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fba08004170 tx=0x7fba08001400 comp rx=0 tx=0).stop 2023-10-05T19:40:51.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.327+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5557d3a87410 msgr2=0x5557d3aaeb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:51.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.327+0000 7fba1e882700 1 -- 172.21.15.12:0/2919343689 reap_dead start 2023-10-05T19:40:51.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.327+0000 7fba2054ce40 1 --2- 172.21.15.12:0/2919343689 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5557d3a87410 0x5557d3aaeb70 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fba1001f040 tx=0x7fba1001c000 comp rx=0 tx=0).stop 2023-10-05T19:40:51.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.327+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 shutdown_connections 2023-10-05T19:40:51.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.327+0000 7fba2054ce40 1 --2- 172.21.15.12:0/2919343689 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5557d3a87410 0x5557d3aaeb70 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:51.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.327+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 >> 172.21.15.12:0/2919343689 conn(0x5557d3a81720 msgr2=0x5557d3aa2550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:51.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.327+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 shutdown_connections 2023-10-05T19:40:51.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.327+0000 7fba2054ce40 1 -- 172.21.15.12:0/2919343689 wait complete. 2023-10-05T19:40:51.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.331+0000 7fba2054ce40 1 -- 172.21.15.12:0/274825098 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x5557d3aeefe0 msgr2=0x5557d3af13f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:51.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.331+0000 7fba2054ce40 1 --2- 172.21.15.12:0/274825098 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x5557d3aeefe0 0x5557d3af13f0 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:51.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.331+0000 7fba2054ce40 1 -- 172.21.15.12:0/274825098 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5557d3aeb870 msgr2=0x5557d3aedc80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:51.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.331+0000 7fba2054ce40 1 --2- 172.21.15.12:0/274825098 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5557d3aeb870 0x5557d3aedc80 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:51.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.331+0000 7fba2054ce40 1 -- 172.21.15.12:0/274825098 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5557d3af6500 msgr2=0x5557d3af88e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:51.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.331+0000 7fba2054ce40 1 --2- 172.21.15.12:0/274825098 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5557d3af6500 0x5557d3af88e0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:51.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.331+0000 7fba2054ce40 1 -- 172.21.15.12:0/274825098 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5557d3af2700 msgr2=0x5557d3af4ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:51.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.331+0000 7fba2054ce40 1 --2- 172.21.15.12:0/274825098 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5557d3af2700 0x5557d3af4ae0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:51.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.331+0000 7fba2054ce40 1 -- 172.21.15.12:0/274825098 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fb964030c50 msgr2=0x7fb964033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:51.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.331+0000 7fba2054ce40 1 --2- 172.21.15.12:0/274825098 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fb964030c50 0x7fb964033100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x5557d3acdd60 tx=0x7fba1800aed0 comp rx=0 tx=0).stop 2023-10-05T19:40:51.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.331+0000 7fba2054ce40 1 -- 172.21.15.12:0/274825098 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5557d3adb0b0 msgr2=0x5557d3ae0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:40:51.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.331+0000 7fba1e081700 1 -- 172.21.15.12:0/274825098 reap_dead start 2023-10-05T19:40:51.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.331+0000 7fba2054ce40 1 --2- 172.21.15.12:0/274825098 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5557d3adb0b0 0x5557d3ae0cb0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fba08001880 tx=0x7fba0803c030 comp rx=0 tx=0).stop 2023-10-05T19:40:51.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.331+0000 7fba2054ce40 1 -- 172.21.15.12:0/274825098 shutdown_connections 2023-10-05T19:40:51.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.331+0000 7fba2054ce40 1 --2- 172.21.15.12:0/274825098 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5557d3adb0b0 0x5557d3ae0cb0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:40:51.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.331+0000 7fba2054ce40 1 -- 172.21.15.12:0/274825098 >> 172.21.15.12:0/274825098 conn(0x5557d3ad6470 msgr2=0x5557d3ad4510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:40:51.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.331+0000 7fba2054ce40 1 -- 172.21.15.12:0/274825098 shutdown_connections 2023-10-05T19:40:51.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:40:51.331+0000 7fba2054ce40 1 -- 172.21.15.12:0/274825098 wait complete. 2023-10-05T19:40:51.345 DEBUG:teuthology.orchestra.run:got remote process result: 17 2023-10-05T19:40:51.352 INFO:tasks.radosgw_admin:RL: create_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2023-10-05T19:40:51.394 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=6 bytes_in=0 failed=False 2023-10-05T19:41:26.395 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--categories', 'delete_obj'] 2023-10-05T19:41:26.395 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--categories', 'delete_obj'] 2023-10-05T19:41:26.396 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --categories delete_obj 2023-10-05T19:41:26.437 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:41:26.437 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:41:26.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.432+0000 7fee61dc5e40 1 Processor -- start 2023-10-05T19:41:26.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.432+0000 7fee61dc5e40 1 -- start start 2023-10-05T19:41:26.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.432+0000 7fee61dc5e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55bd945f0e30 0x55bd945f1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:26.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.432+0000 7fee61dc5e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55bd9453dbd0 con 0x55bd945f1740 2023-10-05T19:41:26.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.432+0000 7fee61dc5e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x55bd944faee0 con 0x55bd945f0e30 2023-10-05T19:41:26.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.432+0000 7fee5f8fa700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55bd945f0e30 0x55bd945f1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:26.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.432+0000 7fee5f0f9700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x55bd945f1740 0x55bd945f21e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:57632/0 (socket says 172.21.15.12:57632) 2023-10-05T19:41:26.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.432+0000 7fee5f0f9700 1 -- 172.21.15.12:0/1139477141 learned_addr learned my addr 172.21.15.12:0/1139477141 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:41:26.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.432+0000 7fee5e8f8700 1 -- 172.21.15.12:0/1139477141 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3037344952 0 0) 0x55bd9453dbd0 con 0x55bd945f1740 2023-10-05T19:41:26.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.432+0000 7fee5e8f8700 1 -- 172.21.15.12:0/1139477141 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee4c001e10 con 0x55bd945f1740 2023-10-05T19:41:26.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.432+0000 7fee5f8fa700 1 -- 172.21.15.12:0/1139477141 >> v1:172.21.15.8:6789/0 conn(0x55bd945f1740 legacy=0x55bd945f21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:26.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.436+0000 7fee5f8fa700 1 -- 172.21.15.12:0/1139477141 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd945f28f0 con 0x55bd945f0e30 2023-10-05T19:41:26.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.436+0000 7fee5f8fa700 1 --2- 172.21.15.12:0/1139477141 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd945f0e30 0x55bd945f1200 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fee5800a5d0 tx=0x7fee58005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a24fac9bb092be server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:26.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.436+0000 7fee5e8f8700 1 -- 172.21.15.12:0/1139477141 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fee58007610 con 0x55bd945f0e30 2023-10-05T19:41:26.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.436+0000 7fee5e8f8700 1 -- 172.21.15.12:0/1139477141 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee58007770 con 0x55bd945f0e30 2023-10-05T19:41:26.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.436+0000 7fee5e8f8700 1 -- 172.21.15.12:0/1139477141 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fee58007a40 con 0x55bd945f0e30 2023-10-05T19:41:26.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.436+0000 7fee61dc5e40 1 -- 172.21.15.12:0/1139477141 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd945f0e30 msgr2=0x55bd945f1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:26.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.436+0000 7fee61dc5e40 1 --2- 172.21.15.12:0/1139477141 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd945f0e30 0x55bd945f1200 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fee5800a5d0 tx=0x7fee58005a50 comp rx=0 tx=0).stop 2023-10-05T19:41:26.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.436+0000 7fee61dc5e40 1 -- 172.21.15.12:0/1139477141 shutdown_connections 2023-10-05T19:41:26.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.436+0000 7fee61dc5e40 1 --2- 172.21.15.12:0/1139477141 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd945f0e30 0x55bd945f1200 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:26.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.436+0000 7fee61dc5e40 1 -- 172.21.15.12:0/1139477141 >> 172.21.15.12:0/1139477141 conn(0x55bd945ec850 msgr2=0x55bd945eec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:26.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.436+0000 7fee61dc5e40 1 -- 172.21.15.12:0/1139477141 shutdown_connections 2023-10-05T19:41:26.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.436+0000 7fee61dc5e40 1 -- 172.21.15.12:0/1139477141 wait complete. 2023-10-05T19:41:26.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.436+0000 7fee61dc5e40 1 Processor -- start 2023-10-05T19:41:26.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.436+0000 7fee61dc5e40 1 -- start start 2023-10-05T19:41:26.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.436+0000 7fee61dc5e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd945f0e30 0x55bd945668e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:26.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.436+0000 7fee61dc5e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd944bee30 con 0x55bd945f0e30 2023-10-05T19:41:26.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.436+0000 7fee5f8fa700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd945f0e30 0x55bd945668e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:26.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.436+0000 7fee5f8fa700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd945f0e30 0x55bd945668e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:50358/0 (socket says 172.21.15.12:50358) 2023-10-05T19:41:26.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.436+0000 7fee5f8fa700 1 -- 172.21.15.12:0/3228902694 learned_addr learned my addr 172.21.15.12:0/3228902694 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:26.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.440+0000 7fee5f8fa700 1 -- 172.21.15.12:0/3228902694 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee58005cf0 con 0x55bd945f0e30 2023-10-05T19:41:26.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.440+0000 7fee5f8fa700 1 --2- 172.21.15.12:0/3228902694 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd945f0e30 0x55bd945668e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fee58000c00 tx=0x7fee58014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59814465055ed69a server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:26.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.440+0000 7fee5d8f6700 1 -- 172.21.15.12:0/3228902694 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fee580078e0 con 0x55bd945f0e30 2023-10-05T19:41:26.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.440+0000 7fee5d8f6700 1 -- 172.21.15.12:0/3228902694 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee58007a40 con 0x55bd945f0e30 2023-10-05T19:41:26.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.440+0000 7fee5d8f6700 1 -- 172.21.15.12:0/3228902694 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fee58007bd0 con 0x55bd945f0e30 2023-10-05T19:41:26.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.440+0000 7fee61dc5e40 1 -- 172.21.15.12:0/3228902694 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd945f0e30 msgr2=0x55bd945668e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:26.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.440+0000 7fee61dc5e40 1 --2- 172.21.15.12:0/3228902694 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd945f0e30 0x55bd945668e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fee58000c00 tx=0x7fee58014000 comp rx=0 tx=0).stop 2023-10-05T19:41:26.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.440+0000 7fee61dc5e40 1 -- 172.21.15.12:0/3228902694 shutdown_connections 2023-10-05T19:41:26.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.440+0000 7fee61dc5e40 1 --2- 172.21.15.12:0/3228902694 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd945f0e30 0x55bd945668e0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:26.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.440+0000 7fee61dc5e40 1 -- 172.21.15.12:0/3228902694 >> 172.21.15.12:0/3228902694 conn(0x55bd945ec850 msgr2=0x55bd9460b900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:26.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.440+0000 7fee61dc5e40 1 -- 172.21.15.12:0/3228902694 shutdown_connections 2023-10-05T19:41:26.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.440+0000 7fee61dc5e40 1 -- 172.21.15.12:0/3228902694 wait complete. 2023-10-05T19:41:26.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.440+0000 7fee61dc5e40 1 Processor -- start 2023-10-05T19:41:26.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.440+0000 7fee61dc5e40 1 -- start start 2023-10-05T19:41:26.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.440+0000 7fee61dc5e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd945f0e30 0x55bd9461d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:26.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.440+0000 7fee61dc5e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd94491ed0 con 0x55bd945f0e30 2023-10-05T19:41:26.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.440+0000 7fee5f8fa700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd945f0e30 0x55bd9461d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:26.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.440+0000 7fee5f8fa700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd945f0e30 0x55bd9461d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:50362/0 (socket says 172.21.15.12:50362) 2023-10-05T19:41:26.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.440+0000 7fee5f8fa700 1 -- 172.21.15.12:0/888756500 learned_addr learned my addr 172.21.15.12:0/888756500 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:26.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.440+0000 7fee5f8fa700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd945f28f0 con 0x55bd945f0e30 2023-10-05T19:41:26.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.444+0000 7fee5f8fa700 1 --2- 172.21.15.12:0/888756500 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd945f0e30 0x55bd9461d000 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fee5801f040 tx=0x7fee5801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:26.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.444+0000 7fee57fff700 1 -- 172.21.15.12:0/888756500 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fee58007bb0 con 0x55bd945f0e30 2023-10-05T19:41:26.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.444+0000 7fee57fff700 1 -- 172.21.15.12:0/888756500 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee58029070 con 0x55bd945f0e30 2023-10-05T19:41:26.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.444+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bd9461d540 con 0x55bd945f0e30 2023-10-05T19:41:26.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.444+0000 7fee57fff700 1 -- 172.21.15.12:0/888756500 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fee58012610 con 0x55bd945f0e30 2023-10-05T19:41:26.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.444+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bd9461d980 con 0x55bd945f0e30 2023-10-05T19:41:26.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.444+0000 7fee61dc5e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:26.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.444+0000 7fee57fff700 1 -- 172.21.15.12:0/888756500 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fee58012770 con 0x55bd945f0e30 2023-10-05T19:41:26.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.444+0000 7fee57fff700 1 --2- 172.21.15.12:0/888756500 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fee40030ca0 0x7fee40033150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:26.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.444+0000 7fee57fff700 1 -- 172.21.15.12:0/888756500 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7fee5802b070 con 0x55bd945f0e30 2023-10-05T19:41:26.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.444+0000 7fee5f0f9700 1 --2- 172.21.15.12:0/888756500 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fee40030ca0 0x7fee40033150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:26.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.444+0000 7fee61dc5e40 1 --2- 172.21.15.12:0/888756500 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55bd945f1740 0x55bd94628a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:26.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.444+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94629100 con 0x55bd945f1740 2023-10-05T19:41:26.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.444+0000 7fee600fb700 1 --2- 172.21.15.12:0/888756500 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55bd945f1740 0x55bd94628a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:26.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.448+0000 7fee5f0f9700 1 --2- 172.21.15.12:0/888756500 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fee40030ca0 0x7fee40033150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fee480044f0 tx=0x7fee48006690 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:26.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.448+0000 7fee600fb700 1 --2- 172.21.15.12:0/888756500 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55bd945f1740 0x55bd94628a90 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:26.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.448+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fee5000a500 con 0x55bd945f1740 2023-10-05T19:41:26.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.448+0000 7fee61dc5e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:26.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.448+0000 7fee61dc5e40 20 realm 2023-10-05T19:41:26.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.448+0000 7fee61dc5e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:26.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.448+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd946260f0 con 0x55bd945f1740 2023-10-05T19:41:26.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.448+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fee5000a500 con 0x55bd945f1740 2023-10-05T19:41:26.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.448+0000 7fee61dc5e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:26.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.448+0000 7fee61dc5e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:41:26.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.448+0000 7fee61dc5e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:26.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.448+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94629be0 con 0x55bd945f1740 2023-10-05T19:41:26.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.448+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fee5000a500 con 0x55bd945f1740 2023-10-05T19:41:26.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.448+0000 7fee61dc5e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:26.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.448+0000 7fee61dc5e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:26.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.448+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd946297c0 con 0x55bd945f1740 2023-10-05T19:41:26.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.448+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fee5000a500 con 0x55bd945f1740 2023-10-05T19:41:26.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.448+0000 7fee61dc5e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:26.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.448+0000 7fee61dc5e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:26.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.448+0000 7fee61dc5e40 1 --2- 172.21.15.12:0/888756500 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55bd9462ab70 0x55bd9462cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:26.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.448+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd9462d6c0 con 0x55bd9462ab70 2023-10-05T19:41:26.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.448+0000 7fee5f8fa700 1 --2- 172.21.15.12:0/888756500 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55bd9462ab70 0x55bd9462cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:26.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.452+0000 7fee5f8fa700 1 --2- 172.21.15.12:0/888756500 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55bd9462ab70 0x55bd9462cf50 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:26.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.452+0000 7fee5f8fa700 1 -- 172.21.15.12:0/888756500 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fee58041040 con 0x55bd9462ab70 2023-10-05T19:41:26.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.452+0000 7fee61dc5e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:41:26.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.452+0000 7fee61dc5e40 20 searching for the correct realm 2023-10-05T19:41:26.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.452+0000 7fee61dc5e40 1 --2- 172.21.15.12:0/888756500 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55bd9462f4f0 0x55bd946318d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:26.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.452+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94631e10 con 0x55bd9462f4f0 2023-10-05T19:41:26.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.452+0000 7fee5f0f9700 1 --2- 172.21.15.12:0/888756500 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55bd9462f4f0 0x55bd946318d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:26.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.452+0000 7fee5f0f9700 1 --2- 172.21.15.12:0/888756500 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55bd9462f4f0 0x55bd946318d0 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:26.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.452+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fee48004520 con 0x55bd9462f4f0 2023-10-05T19:41:26.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.452+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94631e10 con 0x55bd9462ab70 2023-10-05T19:41:26.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.456+0000 7fee5f8fa700 1 -- 172.21.15.12:0/888756500 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee58041040 con 0x55bd9462ab70 2023-10-05T19:41:26.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.456+0000 7fee5f8fa700 1 --2- 172.21.15.12:0/888756500 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fee58007030 0x7fee58046050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:26.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.456+0000 7fee5f8fa700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94631e10 con 0x7fee58007030 2023-10-05T19:41:26.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.456+0000 7fee600fb700 1 --2- 172.21.15.12:0/888756500 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fee58007030 0x7fee58046050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:26.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.456+0000 7fee600fb700 1 --2- 172.21.15.12:0/888756500 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fee58007030 0x7fee58046050 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:26.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.456+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee50014040 con 0x7fee58007030 2023-10-05T19:41:26.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.456+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94631e10 con 0x55bd945f1740 2023-10-05T19:41:26.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.456+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee50014040 con 0x55bd945f1740 2023-10-05T19:41:26.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.456+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94631e10 con 0x55bd945f1740 2023-10-05T19:41:26.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.456+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee50014040 con 0x55bd945f1740 2023-10-05T19:41:26.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.456+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94631e10 con 0x55bd945f1740 2023-10-05T19:41:26.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.460+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(11 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee50014040 con 0x55bd945f1740 2023-10-05T19:41:26.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.460+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94631e10 con 0x7fee58007030 2023-10-05T19:41:26.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.460+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee50014040 con 0x7fee58007030 2023-10-05T19:41:26.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.460+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94631e10 con 0x55bd9462ab70 2023-10-05T19:41:26.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.460+0000 7fee5f8fa700 1 -- 172.21.15.12:0/888756500 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee58041040 con 0x55bd9462ab70 2023-10-05T19:41:26.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.460+0000 7fee5f8fa700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94631e10 con 0x55bd9462ab70 2023-10-05T19:41:26.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.460+0000 7fee5f8fa700 1 -- 172.21.15.12:0/888756500 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee58041040 con 0x55bd9462ab70 2023-10-05T19:41:26.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.460+0000 7fee5f8fa700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94631e10 con 0x55bd945f1740 2023-10-05T19:41:26.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.464+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee50014040 con 0x55bd945f1740 2023-10-05T19:41:26.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.464+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94631e10 con 0x7fee58007030 2023-10-05T19:41:26.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.464+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee50014040 con 0x7fee58007030 2023-10-05T19:41:26.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.464+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94631e10 con 0x55bd945f1740 2023-10-05T19:41:26.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.464+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee50014040 con 0x55bd945f1740 2023-10-05T19:41:26.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.464+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94631e10 con 0x55bd9462ab70 2023-10-05T19:41:26.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.464+0000 7fee5f8fa700 1 -- 172.21.15.12:0/888756500 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(18 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee58041040 con 0x55bd9462ab70 2023-10-05T19:41:26.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.464+0000 7fee5f8fa700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94631e10 con 0x55bd9462f4f0 2023-10-05T19:41:26.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.464+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee48004520 con 0x55bd9462f4f0 2023-10-05T19:41:26.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.464+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94631e10 con 0x55bd9462f4f0 2023-10-05T19:41:26.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.468+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee48004520 con 0x55bd9462f4f0 2023-10-05T19:41:26.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.468+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94631e10 con 0x7fee58007030 2023-10-05T19:41:26.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.468+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee50014040 con 0x7fee58007030 2023-10-05T19:41:26.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.468+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94631e10 con 0x55bd9462f4f0 2023-10-05T19:41:26.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.468+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee4801d040 con 0x55bd9462f4f0 2023-10-05T19:41:26.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.468+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94631e10 con 0x55bd9462f4f0 2023-10-05T19:41:26.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.468+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee4801d040 con 0x55bd9462f4f0 2023-10-05T19:41:26.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.468+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94631e10 con 0x55bd945f1740 2023-10-05T19:41:26.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.468+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee50014040 con 0x55bd945f1740 2023-10-05T19:41:26.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.468+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94631e10 con 0x55bd945f1740 2023-10-05T19:41:26.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.472+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee50014040 con 0x55bd945f1740 2023-10-05T19:41:26.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.472+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94631e10 con 0x7fee58007030 2023-10-05T19:41:26.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.472+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee50014040 con 0x7fee58007030 2023-10-05T19:41:26.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.472+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94631e10 con 0x55bd9462f4f0 2023-10-05T19:41:26.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.472+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee4801d040 con 0x55bd9462f4f0 2023-10-05T19:41:26.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.472+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94631e10 con 0x55bd9462ab70 2023-10-05T19:41:26.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.472+0000 7fee5f8fa700 1 -- 172.21.15.12:0/888756500 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee58041040 con 0x55bd9462ab70 2023-10-05T19:41:26.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.472+0000 7fee5f8fa700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94631e10 con 0x55bd945f1740 2023-10-05T19:41:26.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.476+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee50014040 con 0x55bd945f1740 2023-10-05T19:41:26.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.476+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94631e10 con 0x7fee58007030 2023-10-05T19:41:26.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.476+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee50014040 con 0x7fee58007030 2023-10-05T19:41:26.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.476+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94631e10 con 0x55bd9462f4f0 2023-10-05T19:41:26.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.476+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee4801d040 con 0x55bd9462f4f0 2023-10-05T19:41:26.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.476+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94631e10 con 0x7fee58007030 2023-10-05T19:41:26.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.476+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee50014040 con 0x7fee58007030 2023-10-05T19:41:26.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.476+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94631e10 con 0x55bd9462ab70 2023-10-05T19:41:26.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.480+0000 7fee5f8fa700 1 -- 172.21.15.12:0/888756500 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee58041040 con 0x55bd9462ab70 2023-10-05T19:41:26.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.480+0000 7fee5f8fa700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94631e10 con 0x55bd9462f4f0 2023-10-05T19:41:26.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.480+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee4801d040 con 0x55bd9462f4f0 2023-10-05T19:41:26.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.480+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94631e10 con 0x7fee58007030 2023-10-05T19:41:26.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.480+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee50014040 con 0x7fee58007030 2023-10-05T19:41:26.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.480+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94631e10 con 0x55bd945f1740 2023-10-05T19:41:26.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.480+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fee50014040 con 0x55bd945f1740 2023-10-05T19:41:26.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.480+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94631e10 con 0x55bd9462ab70 2023-10-05T19:41:26.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.484+0000 7fee5f8fa700 1 -- 172.21.15.12:0/888756500 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(37 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fee58041040 con 0x55bd9462ab70 2023-10-05T19:41:26.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.484+0000 7fee61dc5e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:41:26.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.484+0000 7fee61dc5e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:41:26.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.484+0000 7fee61dc5e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:41:26.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.484+0000 7fee61dc5e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:41:26.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.484+0000 7fee61dc5e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:26.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.484+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd9462e270 con 0x55bd945f1740 2023-10-05T19:41:26.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.484+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fee50014040 con 0x55bd945f1740 2023-10-05T19:41:26.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.484+0000 7fee61dc5e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:26.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.484+0000 7fee61dc5e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:26.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.484+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd9462ebc0 con 0x55bd945f1740 2023-10-05T19:41:26.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.484+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fee50014040 con 0x55bd945f1740 2023-10-05T19:41:26.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.484+0000 7fee61dc5e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:26.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.484+0000 7fee61dc5e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:26.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.484+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd9462de30 con 0x55bd9462f4f0 2023-10-05T19:41:26.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.484+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fee4801d040 con 0x55bd9462f4f0 2023-10-05T19:41:26.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.484+0000 7fee61dc5e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:41:26.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.484+0000 7fee61dc5e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:26.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.484+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94632740 con 0x55bd9462f4f0 2023-10-05T19:41:26.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.484+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7fee4801d040 con 0x55bd9462f4f0 2023-10-05T19:41:26.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.484+0000 7fee61dc5e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:26.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.484+0000 7fee61dc5e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:26.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.484+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94632af0 con 0x55bd945f1740 2023-10-05T19:41:26.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.488+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fee50014040 con 0x55bd945f1740 2023-10-05T19:41:26.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.488+0000 7fee61dc5e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:26.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.488+0000 7fee61dc5e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:26.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.488+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94632ea0 con 0x55bd945f1740 2023-10-05T19:41:26.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.488+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fee50014040 con 0x55bd945f1740 2023-10-05T19:41:26.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.488+0000 7fee61dc5e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:26.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.488+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94633eb0 con 0x55bd9462f4f0 2023-10-05T19:41:26.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.488+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(44 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fee4801d040 con 0x55bd9462f4f0 2023-10-05T19:41:26.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.488+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94633eb0 con 0x55bd9462ab70 2023-10-05T19:41:26.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.492+0000 7fee5f8fa700 1 -- 172.21.15.12:0/888756500 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(45 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee58041040 con 0x55bd9462ab70 2023-10-05T19:41:26.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.492+0000 7fee5f8fa700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94633eb0 con 0x7fee58007030 2023-10-05T19:41:26.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.492+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(46 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee50014040 con 0x7fee58007030 2023-10-05T19:41:26.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.492+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94632af0 con 0x55bd945f1740 2023-10-05T19:41:26.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.492+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(47 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee50014040 con 0x55bd945f1740 2023-10-05T19:41:26.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.492+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94632af0 con 0x55bd945f1740 2023-10-05T19:41:26.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.492+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(48 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee50014040 con 0x55bd945f1740 2023-10-05T19:41:26.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.492+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94632af0 con 0x55bd945f1740 2023-10-05T19:41:26.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.492+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(49 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee50014040 con 0x55bd945f1740 2023-10-05T19:41:26.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.492+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94632af0 con 0x7fee58007030 2023-10-05T19:41:26.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.496+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(50 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee50014040 con 0x7fee58007030 2023-10-05T19:41:26.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.496+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94632af0 con 0x55bd9462ab70 2023-10-05T19:41:26.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.496+0000 7fee5f8fa700 1 -- 172.21.15.12:0/888756500 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(51 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee58041040 con 0x55bd9462ab70 2023-10-05T19:41:26.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.496+0000 7fee5f8fa700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94632af0 con 0x55bd9462ab70 2023-10-05T19:41:26.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.496+0000 7fee5f8fa700 1 -- 172.21.15.12:0/888756500 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(52 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee58041040 con 0x55bd9462ab70 2023-10-05T19:41:26.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.496+0000 7fee5f8fa700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94632af0 con 0x55bd945f1740 2023-10-05T19:41:26.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.496+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(53 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee50014040 con 0x55bd945f1740 2023-10-05T19:41:26.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.496+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94632af0 con 0x7fee58007030 2023-10-05T19:41:26.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.500+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(54 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee50014040 con 0x7fee58007030 2023-10-05T19:41:26.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.500+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94632af0 con 0x55bd945f1740 2023-10-05T19:41:26.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.500+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(55 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee50014040 con 0x55bd945f1740 2023-10-05T19:41:26.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.500+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94632af0 con 0x55bd9462ab70 2023-10-05T19:41:26.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.500+0000 7fee5f8fa700 1 -- 172.21.15.12:0/888756500 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(56 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee58041040 con 0x55bd9462ab70 2023-10-05T19:41:26.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.500+0000 7fee5f8fa700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94632af0 con 0x55bd9462f4f0 2023-10-05T19:41:26.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.500+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(57 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee4801d040 con 0x55bd9462f4f0 2023-10-05T19:41:26.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.500+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94632af0 con 0x55bd9462f4f0 2023-10-05T19:41:26.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.500+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(58 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee4801d040 con 0x55bd9462f4f0 2023-10-05T19:41:26.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.500+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94632af0 con 0x7fee58007030 2023-10-05T19:41:26.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.504+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(59 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee50014040 con 0x7fee58007030 2023-10-05T19:41:26.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.504+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94632af0 con 0x55bd9462f4f0 2023-10-05T19:41:26.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.504+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(60 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee4801d040 con 0x55bd9462f4f0 2023-10-05T19:41:26.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.504+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94632af0 con 0x55bd9462f4f0 2023-10-05T19:41:26.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.504+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(61 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee4801d040 con 0x55bd9462f4f0 2023-10-05T19:41:26.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.504+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94632af0 con 0x55bd945f1740 2023-10-05T19:41:26.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.504+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(62 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee50014040 con 0x55bd945f1740 2023-10-05T19:41:26.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.504+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94632af0 con 0x55bd945f1740 2023-10-05T19:41:26.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.508+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(63 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee50014040 con 0x55bd945f1740 2023-10-05T19:41:26.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.508+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94632af0 con 0x7fee58007030 2023-10-05T19:41:26.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.508+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(64 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee50014040 con 0x7fee58007030 2023-10-05T19:41:26.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.508+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94632af0 con 0x55bd9462f4f0 2023-10-05T19:41:26.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.508+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(65 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee4801d040 con 0x55bd9462f4f0 2023-10-05T19:41:26.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.508+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94632af0 con 0x55bd9462ab70 2023-10-05T19:41:26.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.508+0000 7fee5f8fa700 1 -- 172.21.15.12:0/888756500 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(66 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee58041040 con 0x55bd9462ab70 2023-10-05T19:41:26.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.508+0000 7fee5f8fa700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94632af0 con 0x55bd945f1740 2023-10-05T19:41:26.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.512+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(67 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee50014040 con 0x55bd945f1740 2023-10-05T19:41:26.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.512+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94632af0 con 0x7fee58007030 2023-10-05T19:41:26.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.512+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(68 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee50014040 con 0x7fee58007030 2023-10-05T19:41:26.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.512+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94632af0 con 0x55bd9462f4f0 2023-10-05T19:41:26.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.512+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(69 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee4801d040 con 0x55bd9462f4f0 2023-10-05T19:41:26.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.512+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94632af0 con 0x7fee58007030 2023-10-05T19:41:26.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.512+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(70 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee50014040 con 0x7fee58007030 2023-10-05T19:41:26.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.512+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94632af0 con 0x55bd9462ab70 2023-10-05T19:41:26.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.516+0000 7fee5f8fa700 1 -- 172.21.15.12:0/888756500 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(71 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee58041040 con 0x55bd9462ab70 2023-10-05T19:41:26.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.516+0000 7fee5f8fa700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94632af0 con 0x55bd9462f4f0 2023-10-05T19:41:26.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.516+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(72 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee4801d040 con 0x55bd9462f4f0 2023-10-05T19:41:26.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.516+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94632af0 con 0x7fee58007030 2023-10-05T19:41:26.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.516+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(73 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fee50014040 con 0x7fee58007030 2023-10-05T19:41:26.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.516+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94632af0 con 0x55bd945f1740 2023-10-05T19:41:26.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.516+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(74 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fee50014040 con 0x55bd945f1740 2023-10-05T19:41:26.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.516+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94632af0 con 0x55bd9462ab70 2023-10-05T19:41:26.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.520+0000 7fee5f8fa700 1 -- 172.21.15.12:0/888756500 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(75 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fee58041040 con 0x55bd9462ab70 2023-10-05T19:41:26.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.520+0000 7fee61dc5e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:41:26.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.520+0000 7fee61dc5e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:41:26.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.520+0000 7fee61dc5e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:41:26.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.520+0000 7fee61dc5e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:41:26.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.520+0000 7fee61dc5e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:26.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.520+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94632ea0 con 0x55bd945f1740 2023-10-05T19:41:26.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.520+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fee50014040 con 0x55bd945f1740 2023-10-05T19:41:26.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.520+0000 7fee61dc5e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:26.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.520+0000 7fee61dc5e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:26.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.520+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94633250 con 0x55bd9462ab70 2023-10-05T19:41:26.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.520+0000 7fee5f8fa700 1 -- 172.21.15.12:0/888756500 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fee58041040 con 0x55bd9462ab70 2023-10-05T19:41:26.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.520+0000 7fee61dc5e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:41:26.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.520+0000 7fee61dc5e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:26.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.520+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94633af0 con 0x55bd945f1740 2023-10-05T19:41:26.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.520+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fee50014040 con 0x55bd945f1740 2023-10-05T19:41:26.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.520+0000 7fee61dc5e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:26.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.520+0000 7fee61dc5e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:26.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.520+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94633af0 con 0x55bd9462f4f0 2023-10-05T19:41:26.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.524+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fee4801d040 con 0x55bd9462f4f0 2023-10-05T19:41:26.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.524+0000 7fee61dc5e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:41:26.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.524+0000 7fee61dc5e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:26.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.524+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd946336b0 con 0x7fee58007030 2023-10-05T19:41:26.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.524+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7fee50014040 con 0x7fee58007030 2023-10-05T19:41:26.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.524+0000 7fee61dc5e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:26.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.524+0000 7fee61dc5e40 20 zone default found 2023-10-05T19:41:26.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.524+0000 7fee61dc5e40 4 Realm: () 2023-10-05T19:41:26.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.524+0000 7fee61dc5e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:41:26.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.524+0000 7fee61dc5e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:41:26.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.524+0000 7fee61dc5e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:41:26.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.524+0000 7fee61dc5e40 20 zonegroup default 2023-10-05T19:41:26.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.524+0000 7fee61dc5e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:26.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.524+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd946336b0 con 0x7fee58007030 2023-10-05T19:41:26.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.524+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fee50014040 con 0x7fee58007030 2023-10-05T19:41:26.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.524+0000 7fee61dc5e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:26.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.524+0000 7fee61dc5e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:26.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.524+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd946336b0 con 0x55bd9462f4f0 2023-10-05T19:41:26.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.524+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fee4801d040 con 0x55bd9462f4f0 2023-10-05T19:41:26.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.524+0000 7fee61dc5e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:26.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.524+0000 7fee61dc5e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:26.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.524+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94635d10 con 0x55bd9462f4f0 2023-10-05T19:41:26.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.528+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fee4801d040 con 0x55bd9462f4f0 2023-10-05T19:41:26.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.528+0000 7fee61dc5e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:26.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.528+0000 7fee61dc5e40 20 started sync module instance, tier type = 2023-10-05T19:41:26.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.528+0000 7fee61dc5e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:41:26.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.528+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd946353d0 con 0x55bd9462ab70 2023-10-05T19:41:26.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.528+0000 7fee5f8fa700 1 -- 172.21.15.12:0/888756500 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(84 notify.0 [create] v30'219 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fee58041040 con 0x55bd9462ab70 2023-10-05T19:41:26.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.528+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94272726718416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94635000 con 0x55bd9462ab70 2023-10-05T19:41:26.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.528+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94636aa0 con 0x55bd945f1740 2023-10-05T19:41:26.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.532+0000 7fee5f8fa700 1 -- 172.21.15.12:0/888756500 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(85 notify.0 [watch watch cookie 94272726718416] v30'220 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fee58041040 con 0x55bd9462ab70 2023-10-05T19:41:26.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.532+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(86 notify.1 [create] v30'219 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fee50014040 con 0x55bd945f1740 2023-10-05T19:41:26.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.532+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94272726720992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94636ea0 con 0x55bd945f1740 2023-10-05T19:41:26.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.532+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94637740 con 0x55bd9462f4f0 2023-10-05T19:41:26.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.536+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(87 notify.1 [watch watch cookie 94272726720992] v30'220 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fee50014040 con 0x55bd945f1740 2023-10-05T19:41:26.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.536+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(88 notify.2 [create] v30'219 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fee4801d040 con 0x55bd9462f4f0 2023-10-05T19:41:26.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.536+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94272726722448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94637af0 con 0x55bd9462f4f0 2023-10-05T19:41:26.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.536+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94638390 con 0x55bd9462ab70 2023-10-05T19:41:26.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.540+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(89 notify.2 [watch watch cookie 94272726722448] v30'220 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fee4801d040 con 0x55bd9462f4f0 2023-10-05T19:41:26.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.540+0000 7fee5f8fa700 1 -- 172.21.15.12:0/888756500 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(90 notify.3 [create] v30'219 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fee58041040 con 0x55bd9462ab70 2023-10-05T19:41:26.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.540+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94272726724608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd946383d0 con 0x55bd9462ab70 2023-10-05T19:41:26.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.540+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94638f90 con 0x55bd945f1740 2023-10-05T19:41:26.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.544+0000 7fee5f8fa700 1 -- 172.21.15.12:0/888756500 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(91 notify.3 [watch watch cookie 94272726724608] v30'220 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fee58041040 con 0x55bd9462ab70 2023-10-05T19:41:26.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.544+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(92 notify.4 [create] v30'219 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fee50014040 con 0x55bd945f1740 2023-10-05T19:41:26.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.544+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94272726731360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94638390 con 0x55bd945f1740 2023-10-05T19:41:26.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.544+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd946399f0 con 0x7fee58007030 2023-10-05T19:41:26.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.544+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(93 notify.4 [watch watch cookie 94272726731360] v30'220 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fee50014040 con 0x55bd945f1740 2023-10-05T19:41:26.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.544+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(94 notify.5 [create] v30'219 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fee50014040 con 0x7fee58007030 2023-10-05T19:41:26.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.544+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94272726728624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd946399f0 con 0x7fee58007030 2023-10-05T19:41:26.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.544+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd9463a290 con 0x7fee58007030 2023-10-05T19:41:26.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.548+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(96 notify.6 [create] v30'219 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fee50014040 con 0x7fee58007030 2023-10-05T19:41:26.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.548+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(95 notify.5 [watch watch cookie 94272726728624] v30'220 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fee50014040 con 0x7fee58007030 2023-10-05T19:41:26.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.548+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94272726735984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd9463a630 con 0x7fee58007030 2023-10-05T19:41:26.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.548+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd9463acc0 con 0x7fee58007030 2023-10-05T19:41:26.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.552+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(97 notify.6 [watch watch cookie 94272726735984] v30'220 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fee50014040 con 0x7fee58007030 2023-10-05T19:41:26.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.552+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(98 notify.7 [create] v30'219 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fee50014040 con 0x7fee58007030 2023-10-05T19:41:26.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.552+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94272726734080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd9463acc0 con 0x7fee58007030 2023-10-05T19:41:26.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.552+0000 7fee61dc5e40 20 add_watcher() i=0 2023-10-05T19:41:26.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.552+0000 7fee61dc5e40 20 add_watcher() i=1 2023-10-05T19:41:26.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.552+0000 7fee61dc5e40 20 add_watcher() i=2 2023-10-05T19:41:26.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.552+0000 7fee61dc5e40 20 add_watcher() i=3 2023-10-05T19:41:26.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.552+0000 7fee61dc5e40 20 add_watcher() i=4 2023-10-05T19:41:26.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.552+0000 7fee61dc5e40 20 add_watcher() i=5 2023-10-05T19:41:26.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.552+0000 7fee61dc5e40 20 add_watcher() i=6 2023-10-05T19:41:26.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.556+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(99 notify.7 [watch watch cookie 94272726734080] v30'220 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fee50014040 con 0x7fee58007030 2023-10-05T19:41:26.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.556+0000 7fee61dc5e40 20 add_watcher() i=7 2023-10-05T19:41:26.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.556+0000 7fee61dc5e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:41:26.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.556+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd9463e1e0 con 0x55bd945f1740 2023-10-05T19:41:26.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.556+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fee50014040 con 0x55bd945f1740 2023-10-05T19:41:26.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.556+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94272726740160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd9463b600 con 0x55bd945f1740 2023-10-05T19:41:26.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.560+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94272726740160] v30'217 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fee50014040 con 0x55bd945f1740 2023-10-05T19:41:26.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.560+0000 7fee317e2700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:41:26.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.560+0000 7fee61dc5e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:41:26.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.560+0000 7fee61dc5e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:41:26.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.560+0000 7fee61dc5e40 1 Processor -- start 2023-10-05T19:41:26.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.560+0000 7fee61dc5e40 1 -- start start 2023-10-05T19:41:26.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.560+0000 7fee61dc5e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd946484b0 0x55bd94648880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:26.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.560+0000 7fee61dc5e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd94648dc0 con 0x55bd946484b0 2023-10-05T19:41:26.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.560+0000 7fee5f0f9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd946484b0 0x55bd94648880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:26.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.560+0000 7fee5f0f9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd946484b0 0x55bd94648880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:50372/0 (socket says 172.21.15.12:50372) 2023-10-05T19:41:26.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.560+0000 7fee5f0f9700 1 -- 172.21.15.12:0/1578362151 learned_addr learned my addr 172.21.15.12:0/1578362151 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:26.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.560+0000 7fee5f0f9700 1 -- 172.21.15.12:0/1578362151 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd94648f00 con 0x55bd946484b0 2023-10-05T19:41:26.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.560+0000 7fee5f0f9700 1 --2- 172.21.15.12:0/1578362151 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd946484b0 0x55bd94648880 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x55bd94626520 tx=0x7fee48010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49ea45423f2a1d7e server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:26.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.560+0000 7fee30fe1700 1 -- 172.21.15.12:0/1578362151 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fee4802a040 con 0x55bd946484b0 2023-10-05T19:41:26.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.560+0000 7fee30fe1700 1 -- 172.21.15.12:0/1578362151 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee4801b040 con 0x55bd946484b0 2023-10-05T19:41:26.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.560+0000 7fee30fe1700 1 -- 172.21.15.12:0/1578362151 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fee4801a040 con 0x55bd946484b0 2023-10-05T19:41:26.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.564+0000 7fee61dc5e40 1 -- 172.21.15.12:0/1578362151 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd946484b0 msgr2=0x55bd94648880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:26.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.564+0000 7fee61dc5e40 1 --2- 172.21.15.12:0/1578362151 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd946484b0 0x55bd94648880 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x55bd94626520 tx=0x7fee48010000 comp rx=0 tx=0).stop 2023-10-05T19:41:26.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.564+0000 7fee61dc5e40 1 -- 172.21.15.12:0/1578362151 shutdown_connections 2023-10-05T19:41:26.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.564+0000 7fee61dc5e40 1 --2- 172.21.15.12:0/1578362151 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd946484b0 0x55bd94648880 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:26.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.564+0000 7fee61dc5e40 1 -- 172.21.15.12:0/1578362151 >> 172.21.15.12:0/1578362151 conn(0x55bd94643870 msgr2=0x55bd94641a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:26.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.564+0000 7fee61dc5e40 1 -- 172.21.15.12:0/1578362151 shutdown_connections 2023-10-05T19:41:26.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.564+0000 7fee61dc5e40 1 -- 172.21.15.12:0/1578362151 wait complete. 2023-10-05T19:41:26.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.564+0000 7fee61dc5e40 1 Processor -- start 2023-10-05T19:41:26.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.564+0000 7fee61dc5e40 1 -- start start 2023-10-05T19:41:26.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.564+0000 7fee61dc5e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd946484b0 0x55bd9464dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:26.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.564+0000 7fee61dc5e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9464efc0 con 0x55bd946484b0 2023-10-05T19:41:26.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.564+0000 7fee5f0f9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd946484b0 0x55bd9464dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:26.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.564+0000 7fee5f0f9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd946484b0 0x55bd9464dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:50382/0 (socket says 172.21.15.12:50382) 2023-10-05T19:41:26.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.564+0000 7fee5f0f9700 1 -- 172.21.15.12:0/243028048 learned_addr learned my addr 172.21.15.12:0/243028048 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:26.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.564+0000 7fee5f0f9700 1 -- 172.21.15.12:0/243028048 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee4c001e10 con 0x55bd946484b0 2023-10-05T19:41:26.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.564+0000 7fee5f0f9700 1 --2- 172.21.15.12:0/243028048 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd946484b0 0x55bd9464dfc0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fee4803f000 tx=0x7fee48038070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:26.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.568+0000 7fedaaffd700 1 -- 172.21.15.12:0/243028048 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fee4802a030 con 0x55bd946484b0 2023-10-05T19:41:26.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.568+0000 7fedaaffd700 1 -- 172.21.15.12:0/243028048 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee48033e40 con 0x55bd946484b0 2023-10-05T19:41:26.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.568+0000 7fee61dc5e40 1 -- 172.21.15.12:0/243028048 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bd9464e5c0 con 0x55bd946484b0 2023-10-05T19:41:26.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.568+0000 7fedaaffd700 1 -- 172.21.15.12:0/243028048 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fee4801a070 con 0x55bd946484b0 2023-10-05T19:41:26.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.568+0000 7fee61dc5e40 1 -- 172.21.15.12:0/243028048 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bd9464e9a0 con 0x55bd946484b0 2023-10-05T19:41:26.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.568+0000 7fedaaffd700 1 -- 172.21.15.12:0/243028048 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fee48026cd0 con 0x55bd946484b0 2023-10-05T19:41:26.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.568+0000 7fedaaffd700 1 --2- 172.21.15.12:0/243028048 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7feda0030c00 0x7feda00330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:26.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.568+0000 7fedaaffd700 1 -- 172.21.15.12:0/243028048 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7fee48056bd0 con 0x55bd946484b0 2023-10-05T19:41:26.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.568+0000 7fee61dc5e40 5 note: GC not initialized 2023-10-05T19:41:26.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.568+0000 7fee600fb700 1 --2- 172.21.15.12:0/243028048 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7feda0030c00 0x7feda00330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:26.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.568+0000 7fee600fb700 1 --2- 172.21.15.12:0/243028048 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7feda0030c00 0x7feda00330b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x55bd9463a3a0 tx=0x7fee5000ab80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:26.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.572+0000 7fed9ffff700 20 reqs_thread_entry: start 2023-10-05T19:41:26.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.572+0000 7fee61dc5e40 20 init_complete bucket index max shards: 11 2023-10-05T19:41:26.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.572+0000 7fed9dffb700 20 reqs_thread_entry: start 2023-10-05T19:41:26.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.572+0000 7fee61dc5e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:41:26.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.572+0000 7fed8bfff700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:41:56 2023 2023-10-05T19:41:26.578 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:41:26.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.572+0000 7fed8b7fe700 20 reqs_thread_entry: start 2023-10-05T19:41:26.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.572+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 6.e 6:74cd889a:usage::usage.0:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd946b7180 con 0x55bd945f1740 2023-10-05T19:41:26.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.572+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(102 usage.0 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7fee50014040 con 0x55bd945f1740 2023-10-05T19:41:26.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.572+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:103 6.3 6:c0e78a64:usage::usage.1:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd946b7180 con 0x55bd9462ab70 2023-10-05T19:41:26.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.572+0000 7fee5f8fa700 1 -- 172.21.15.12:0/888756500 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(103 usage.1 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7fee58041040 con 0x55bd9462ab70 2023-10-05T19:41:26.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.572+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:104 6.d 6:b564efa4:usage::usage.2:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd946b7180 con 0x55bd945f1740 2023-10-05T19:41:26.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.576+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(104 usage.2 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7fee50014040 con 0x55bd945f1740 2023-10-05T19:41:26.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.576+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:105 6.2 6:42d64401:usage::usage.3:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd946b7180 con 0x55bd9462f4f0 2023-10-05T19:41:26.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.576+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(105 usage.3 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7fee48052070 con 0x55bd9462f4f0 2023-10-05T19:41:26.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.576+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:106 6.7 6:e09b6f01:usage::usage.4:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd946b7180 con 0x55bd9462f4f0 2023-10-05T19:41:26.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.576+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(106 usage.4 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7fee48052070 con 0x55bd9462f4f0 2023-10-05T19:41:26.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.576+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:107 6.6 6:672810f1:usage::usage.5:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd946b7180 con 0x55bd9462f4f0 2023-10-05T19:41:26.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.576+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(107 usage.5 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7fee4805b040 con 0x55bd9462f4f0 2023-10-05T19:41:26.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.576+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:108 6.4 6:25a86931:usage::usage.6:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd946b7180 con 0x55bd945f1740 2023-10-05T19:41:26.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.576+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(108 usage.6 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7fee50014040 con 0x55bd945f1740 2023-10-05T19:41:26.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.576+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:109 6.17 6:ebbe7322:usage::usage.7:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd946b7180 con 0x55bd945f1740 2023-10-05T19:41:26.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.576+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== osd_op_reply(109 usage.7 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7fee50014040 con 0x55bd945f1740 2023-10-05T19:41:26.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.576+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:110 6.6 6:607e1922:usage::usage.8:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd946b7180 con 0x55bd9462f4f0 2023-10-05T19:41:26.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.580+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 <== osd.3 v2:172.21.15.12:6816/29951 27 ==== osd_op_reply(110 usage.8 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7fee4805b040 con 0x55bd9462f4f0 2023-10-05T19:41:26.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.580+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:111 6.b 6:d7809252:usage::usage.9:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd946b7180 con 0x55bd9462f4f0 2023-10-05T19:41:26.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.580+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 <== osd.3 v2:172.21.15.12:6816/29951 28 ==== osd_op_reply(111 usage.9 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7fee4805b040 con 0x55bd9462f4f0 2023-10-05T19:41:26.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.580+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:112 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd946b7180 con 0x7fee58007030 2023-10-05T19:41:26.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.580+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(112 usage.10 [call out=1268b] v0'0 uv32 ondisk = 0) v8 ==== 152+0+1268 (crc 0 0 0) 0x7fee50014040 con 0x7fee58007030 2023-10-05T19:41:26.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.580+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:113 6.4 6:21f845ae:usage::usage.11:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd946b86d0 con 0x55bd945f1740 2023-10-05T19:41:26.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.580+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 39 ==== osd_op_reply(113 usage.11 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fee50014040 con 0x55bd945f1740 2023-10-05T19:41:26.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.580+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:114 6.f 6:f0386354:usage::usage.12:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd946b86d0 con 0x55bd9462ab70 2023-10-05T19:41:26.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.580+0000 7fee5f8fa700 1 -- 172.21.15.12:0/888756500 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(114 usage.12 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fee58041040 con 0x55bd9462ab70 2023-10-05T19:41:26.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.584+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:115 6.c 6:3735eec4:usage::usage.13:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd946b86d0 con 0x55bd9462f4f0 2023-10-05T19:41:26.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.584+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 <== osd.3 v2:172.21.15.12:6816/29951 29 ==== osd_op_reply(115 usage.13 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fee4805b040 con 0x55bd9462f4f0 2023-10-05T19:41:26.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.584+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:116 6.18 6:1f520d0a:usage::usage.14:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd946b86d0 con 0x7fee58007030 2023-10-05T19:41:26.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.584+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(116 usage.14 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fee50014040 con 0x7fee58007030 2023-10-05T19:41:26.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.584+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:117 6.0 6:029cee12:usage::usage.15:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd946b86d0 con 0x7fee58007030 2023-10-05T19:41:26.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.584+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(117 usage.15 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fee50014040 con 0x7fee58007030 2023-10-05T19:41:26.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.584+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:118 6.1e 6:7f3e030e:usage::usage.16:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd946b86d0 con 0x55bd9462f4f0 2023-10-05T19:41:26.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.584+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 <== osd.3 v2:172.21.15.12:6816/29951 30 ==== osd_op_reply(118 usage.16 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fee4805b040 con 0x55bd9462f4f0 2023-10-05T19:41:26.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.584+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:119 6.10 6:0fbdcb32:usage::usage.17:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd946b86d0 con 0x7fee58007030 2023-10-05T19:41:26.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.588+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(119 usage.17 [call out=163b] v0'0 uv5 ondisk = 0) v8 ==== 152+0+163 (crc 0 0 0) 0x7fee50014040 con 0x7fee58007030 2023-10-05T19:41:26.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.588+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:120 6.19 6:993b03e9:usage::usage.18:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd946b86d0 con 0x55bd945f1740 2023-10-05T19:41:26.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.588+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 40 ==== osd_op_reply(120 usage.18 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fee50014040 con 0x55bd945f1740 2023-10-05T19:41:26.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.588+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:121 6.1a 6:58fcd270:usage::usage.19:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd946b86d0 con 0x55bd9462f4f0 2023-10-05T19:41:26.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.588+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 <== osd.3 v2:172.21.15.12:6816/29951 31 ==== osd_op_reply(121 usage.19 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fee4805b040 con 0x55bd9462f4f0 2023-10-05T19:41:26.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.588+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:122 6.1f 6:fcd63d49:usage::usage.20:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd946b86d0 con 0x7fee58007030 2023-10-05T19:41:26.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.588+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(122 usage.20 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fee50014040 con 0x7fee58007030 2023-10-05T19:41:26.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.588+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:123 6.5 6:a2c1a412:usage::usage.21:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd946b86d0 con 0x55bd9462ab70 2023-10-05T19:41:26.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.588+0000 7fee5f8fa700 1 -- 172.21.15.12:0/888756500 <== osd.2 v2:172.21.15.12:6800/29934 23 ==== osd_op_reply(123 usage.21 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fee58041040 con 0x55bd9462ab70 2023-10-05T19:41:26.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.588+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:124 6.13 6:cdb71d68:usage::usage.22:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd946b86d0 con 0x7fee58007030 2023-10-05T19:41:26.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.592+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.0 v2:172.21.15.8:6800/27563 30 ==== osd_op_reply(124 usage.22 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fee50014040 con 0x7fee58007030 2023-10-05T19:41:26.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.592+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:125 6.17 6:ef3bca03:usage::usage.23:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd946b86d0 con 0x55bd945f1740 2023-10-05T19:41:26.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.592+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 41 ==== osd_op_reply(125 usage.23 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fee50014040 con 0x55bd945f1740 2023-10-05T19:41:26.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.592+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:126 6.e 6:7412960a:usage::usage.24:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd946b86d0 con 0x55bd945f1740 2023-10-05T19:41:26.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.592+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 42 ==== osd_op_reply(126 usage.24 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fee50014040 con 0x55bd945f1740 2023-10-05T19:41:26.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.592+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:127 6.7 6:e62d1024:usage::usage.25:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd946b86d0 con 0x55bd9462f4f0 2023-10-05T19:41:26.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.592+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 <== osd.3 v2:172.21.15.12:6816/29951 32 ==== osd_op_reply(127 usage.25 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fee4805b040 con 0x55bd9462f4f0 2023-10-05T19:41:26.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.592+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:128 6.d 6:b172cd59:usage::usage.26:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd946b86d0 con 0x55bd945f1740 2023-10-05T19:41:26.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.592+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 43 ==== osd_op_reply(128 usage.26 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fee50014040 con 0x55bd945f1740 2023-10-05T19:41:26.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.592+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:129 6.1 6:83912023:usage::usage.27:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd946b86d0 con 0x55bd945f1740 2023-10-05T19:41:26.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.592+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 44 ==== osd_op_reply(129 usage.27 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fee50014040 con 0x55bd945f1740 2023-10-05T19:41:26.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.592+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:130 6.1e 6:7c73bdb3:usage::usage.28:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd946b86d0 con 0x55bd9462f4f0 2023-10-05T19:41:26.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.596+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 <== osd.3 v2:172.21.15.12:6816/29951 33 ==== osd_op_reply(130 usage.28 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fee4805b040 con 0x55bd9462f4f0 2023-10-05T19:41:26.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.596+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:131 6.16 6:6e59fa35:usage::usage.29:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd946b86d0 con 0x55bd9462f4f0 2023-10-05T19:41:26.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.596+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 <== osd.3 v2:172.21.15.12:6816/29951 34 ==== osd_op_reply(131 usage.29 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fee4805b040 con 0x55bd9462f4f0 2023-10-05T19:41:26.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.596+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:132 6.1c 6:38fe6885:usage::usage.30:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd946b86d0 con 0x55bd9462f4f0 2023-10-05T19:41:26.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.596+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 <== osd.3 v2:172.21.15.12:6816/29951 35 ==== osd_op_reply(132 usage.30 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fee4805b040 con 0x55bd9462f4f0 2023-10-05T19:41:26.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.596+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:133 6.1d 6:bb28dff0:usage::usage.31:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd946b86d0 con 0x7fee58007030 2023-10-05T19:41:26.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.596+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.0 v2:172.21.15.8:6800/27563 31 ==== osd_op_reply(133 usage.31 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fee50014040 con 0x7fee58007030 2023-10-05T19:41:26.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.596+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:134 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94272726718416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94674ca0 con 0x55bd9462ab70 2023-10-05T19:41:26.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.600+0000 7fee5f8fa700 1 -- 172.21.15.12:0/888756500 <== osd.2 v2:172.21.15.12:6800/29934 24 ==== osd_op_reply(134 notify.0 [watch unwatch cookie 94272726718416] v30'221 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fee58041040 con 0x55bd9462ab70 2023-10-05T19:41:26.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.600+0000 7fee61dc5e40 20 remove_watcher() i=0 2023-10-05T19:41:26.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.600+0000 7fee61dc5e40 2 removed watcher, disabling cache 2023-10-05T19:41:26.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.600+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:135 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94272726720992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd946357d0 con 0x55bd945f1740 2023-10-05T19:41:26.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.604+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 45 ==== osd_op_reply(135 notify.1 [watch unwatch cookie 94272726720992] v30'221 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fee50014040 con 0x55bd945f1740 2023-10-05T19:41:26.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.604+0000 7fee61dc5e40 20 remove_watcher() i=1 2023-10-05T19:41:26.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.604+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:136 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94272726722448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd946357d0 con 0x55bd9462f4f0 2023-10-05T19:41:26.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.608+0000 7fee5f0f9700 1 -- 172.21.15.12:0/888756500 <== osd.3 v2:172.21.15.12:6816/29951 36 ==== osd_op_reply(136 notify.2 [watch unwatch cookie 94272726722448] v30'221 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fee4805b040 con 0x55bd9462f4f0 2023-10-05T19:41:26.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.608+0000 7fee61dc5e40 20 remove_watcher() i=2 2023-10-05T19:41:26.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.608+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:137 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94272726724608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd946357d0 con 0x55bd9462ab70 2023-10-05T19:41:26.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.608+0000 7fee5f8fa700 1 -- 172.21.15.12:0/888756500 <== osd.2 v2:172.21.15.12:6800/29934 25 ==== osd_op_reply(137 notify.3 [watch unwatch cookie 94272726724608] v30'221 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fee58041040 con 0x55bd9462ab70 2023-10-05T19:41:26.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.608+0000 7fee61dc5e40 20 remove_watcher() i=3 2023-10-05T19:41:26.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.608+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:138 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94272726731360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94637000 con 0x55bd945f1740 2023-10-05T19:41:26.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.612+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 46 ==== osd_op_reply(138 notify.4 [watch unwatch cookie 94272726731360] v30'221 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fee50014040 con 0x55bd945f1740 2023-10-05T19:41:26.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.612+0000 7fee61dc5e40 20 remove_watcher() i=4 2023-10-05T19:41:26.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.612+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:139 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94272726728624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd946357d0 con 0x7fee58007030 2023-10-05T19:41:26.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.616+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.0 v2:172.21.15.8:6800/27563 32 ==== osd_op_reply(139 notify.5 [watch unwatch cookie 94272726728624] v30'221 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fee50014040 con 0x7fee58007030 2023-10-05T19:41:26.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.616+0000 7fee61dc5e40 20 remove_watcher() i=5 2023-10-05T19:41:26.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.616+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:140 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94272726735984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94637fb0 con 0x7fee58007030 2023-10-05T19:41:26.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.620+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.0 v2:172.21.15.8:6800/27563 33 ==== osd_op_reply(140 notify.6 [watch unwatch cookie 94272726735984] v30'221 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fee50014040 con 0x7fee58007030 2023-10-05T19:41:26.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.620+0000 7fee61dc5e40 20 remove_watcher() i=6 2023-10-05T19:41:26.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.620+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:141 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94272726734080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd946357d0 con 0x7fee58007030 2023-10-05T19:41:26.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.620+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.0 v2:172.21.15.8:6800/27563 34 ==== osd_op_reply(141 notify.7 [watch unwatch cookie 94272726734080] v30'221 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fee50014040 con 0x7fee58007030 2023-10-05T19:41:26.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.620+0000 7fee61dc5e40 20 remove_watcher() i=7 2023-10-05T19:41:26.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.624+0000 7fee317e2700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:41:26.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.624+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:142 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94272726740160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd94677420 con 0x55bd945f1740 2023-10-05T19:41:26.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.628+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 <== osd.1 v2:172.21.15.12:6808/29933 47 ==== osd_op_reply(142 data_loggenerations_metadata [watch unwatch cookie 94272726740160] v30'218 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fee50014040 con 0x55bd945f1740 2023-10-05T19:41:26.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.628+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fee58007030 msgr2=0x7fee58046050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:26.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.628+0000 7fee61dc5e40 1 --2- 172.21.15.12:0/888756500 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fee58007030 0x7fee58046050 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:26.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.628+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55bd945f1740 msgr2=0x55bd94628a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:26.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.628+0000 7fee61dc5e40 1 --2- 172.21.15.12:0/888756500 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55bd945f1740 0x55bd94628a90 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:26.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.628+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55bd9462ab70 msgr2=0x55bd9462cf50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:26.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.628+0000 7fee61dc5e40 1 --2- 172.21.15.12:0/888756500 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55bd9462ab70 0x55bd9462cf50 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:26.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.628+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55bd9462f4f0 msgr2=0x55bd946318d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:26.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.628+0000 7fee61dc5e40 1 --2- 172.21.15.12:0/888756500 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55bd9462f4f0 0x55bd946318d0 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:26.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.628+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fee40030ca0 msgr2=0x7fee40033150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:26.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.628+0000 7fee61dc5e40 1 --2- 172.21.15.12:0/888756500 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fee40030ca0 0x7fee40033150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fee480044f0 tx=0x7fee48006690 comp rx=0 tx=0).stop 2023-10-05T19:41:26.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.628+0000 7fee600fb700 1 -- 172.21.15.12:0/888756500 reap_dead start 2023-10-05T19:41:26.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.628+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd945f0e30 msgr2=0x55bd9461d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:26.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.628+0000 7fee61dc5e40 1 --2- 172.21.15.12:0/888756500 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd945f0e30 0x55bd9461d000 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fee5801f040 tx=0x7fee5801c000 comp rx=0 tx=0).stop 2023-10-05T19:41:26.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.628+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 shutdown_connections 2023-10-05T19:41:26.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.628+0000 7fee61dc5e40 1 --2- 172.21.15.12:0/888756500 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd945f0e30 0x55bd9461d000 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:26.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.628+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 >> 172.21.15.12:0/888756500 conn(0x55bd945ec850 msgr2=0x55bd9460a380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:26.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.628+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 shutdown_connections 2023-10-05T19:41:26.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.628+0000 7fee61dc5e40 1 -- 172.21.15.12:0/888756500 wait complete. 2023-10-05T19:41:26.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.628+0000 7fee61dc5e40 1 -- 172.21.15.12:0/243028048 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7feda0030c00 msgr2=0x7feda00330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:26.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.628+0000 7fee61dc5e40 1 --2- 172.21.15.12:0/243028048 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7feda0030c00 0x7feda00330b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x55bd9463a3a0 tx=0x7fee5000ab80 comp rx=0 tx=0).stop 2023-10-05T19:41:26.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.628+0000 7fee61dc5e40 1 -- 172.21.15.12:0/243028048 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd946484b0 msgr2=0x55bd9464dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:26.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.628+0000 7fee61dc5e40 1 --2- 172.21.15.12:0/243028048 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd946484b0 0x55bd9464dfc0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fee4803f000 tx=0x7fee48038070 comp rx=0 tx=0).stop 2023-10-05T19:41:26.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.628+0000 7fee61dc5e40 1 -- 172.21.15.12:0/243028048 shutdown_connections 2023-10-05T19:41:26.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.628+0000 7fee61dc5e40 1 --2- 172.21.15.12:0/243028048 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7feda0030c00 0x7feda00330b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:26.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.628+0000 7fee61dc5e40 1 --2- 172.21.15.12:0/243028048 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd946484b0 0x55bd9464dfc0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:26.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.628+0000 7fee61dc5e40 1 -- 172.21.15.12:0/243028048 >> 172.21.15.12:0/243028048 conn(0x55bd94643870 msgr2=0x55bd94641970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:26.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.628+0000 7fee61dc5e40 1 -- 172.21.15.12:0/243028048 shutdown_connections 2023-10-05T19:41:26.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.632+0000 7fee61dc5e40 1 -- 172.21.15.12:0/243028048 wait complete. 2023-10-05T19:41:26.638 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:41:26.638 INFO:teuthology.orchestra.run.smithi012.stdout: "entries": [ 2023-10-05T19:41:26.638 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.638 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "anonymous", 2023-10-05T19:41:26.638 INFO:teuthology.orchestra.run.smithi012.stdout: "buckets": [ 2023-10-05T19:41:26.638 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.638 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "", 2023-10-05T19:41:26.638 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:00:00.000000Z", 2023-10-05T19:41:26.639 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 1696532400, 2023-10-05T19:41:26.639 INFO:teuthology.orchestra.run.smithi012.stdout: "owner": "anonymous", 2023-10-05T19:41:26.639 INFO:teuthology.orchestra.run.smithi012.stdout: "categories": [] 2023-10-05T19:41:26.639 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:26.639 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2023-10-05T19:41:26.639 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:26.639 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.639 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:41:26.639 INFO:teuthology.orchestra.run.smithi012.stdout: "buckets": [ 2023-10-05T19:41:26.639 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.639 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "", 2023-10-05T19:41:26.639 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:00:00.000000Z", 2023-10-05T19:41:26.640 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 1696532400, 2023-10-05T19:41:26.640 INFO:teuthology.orchestra.run.smithi012.stdout: "owner": "foo", 2023-10-05T19:41:26.640 INFO:teuthology.orchestra.run.smithi012.stdout: "categories": [] 2023-10-05T19:41:26.640 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:26.640 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.640 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo", 2023-10-05T19:41:26.640 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:00:00.000000Z", 2023-10-05T19:41:26.640 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 1696532400, 2023-10-05T19:41:26.640 INFO:teuthology.orchestra.run.smithi012.stdout: "owner": "foo", 2023-10-05T19:41:26.640 INFO:teuthology.orchestra.run.smithi012.stdout: "categories": [ 2023-10-05T19:41:26.640 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.640 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "delete_obj", 2023-10-05T19:41:26.641 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:26.641 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:26.641 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 4, 2023-10-05T19:41:26.641 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 4 2023-10-05T19:41:26.641 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:26.641 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2023-10-05T19:41:26.641 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:26.641 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.641 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo2", 2023-10-05T19:41:26.641 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:00:00.000000Z", 2023-10-05T19:41:26.641 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 1696532400, 2023-10-05T19:41:26.641 INFO:teuthology.orchestra.run.smithi012.stdout: "owner": "foo", 2023-10-05T19:41:26.642 INFO:teuthology.orchestra.run.smithi012.stdout: "categories": [] 2023-10-05T19:41:26.642 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:26.642 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.642 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo3", 2023-10-05T19:41:26.642 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:00:00.000000Z", 2023-10-05T19:41:26.642 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 1696532400, 2023-10-05T19:41:26.642 INFO:teuthology.orchestra.run.smithi012.stdout: "owner": "foo", 2023-10-05T19:41:26.642 INFO:teuthology.orchestra.run.smithi012.stdout: "categories": [] 2023-10-05T19:41:26.642 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:26.642 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.642 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo4", 2023-10-05T19:41:26.642 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:00:00.000000Z", 2023-10-05T19:41:26.643 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 1696532400, 2023-10-05T19:41:26.643 INFO:teuthology.orchestra.run.smithi012.stdout: "owner": "foo", 2023-10-05T19:41:26.643 INFO:teuthology.orchestra.run.smithi012.stdout: "categories": [] 2023-10-05T19:41:26.643 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:26.643 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2023-10-05T19:41:26.643 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:26.643 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:41:26.643 INFO:teuthology.orchestra.run.smithi012.stdout: "summary": [ 2023-10-05T19:41:26.643 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.643 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "anonymous", 2023-10-05T19:41:26.643 INFO:teuthology.orchestra.run.smithi012.stdout: "categories": [], 2023-10-05T19:41:26.644 INFO:teuthology.orchestra.run.smithi012.stdout: "total": { 2023-10-05T19:41:26.644 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:26.644 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:26.644 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 0, 2023-10-05T19:41:26.644 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 0 2023-10-05T19:41:26.644 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:26.644 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:26.644 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.644 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:41:26.644 INFO:teuthology.orchestra.run.smithi012.stdout: "categories": [ 2023-10-05T19:41:26.644 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.644 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "delete_obj", 2023-10-05T19:41:26.645 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:26.645 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:26.645 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 4, 2023-10-05T19:41:26.645 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 4 2023-10-05T19:41:26.645 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:26.645 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:41:26.645 INFO:teuthology.orchestra.run.smithi012.stdout: "total": { 2023-10-05T19:41:26.645 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:26.645 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:26.645 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 4, 2023-10-05T19:41:26.645 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 4 2023-10-05T19:41:26.645 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:26.646 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:26.646 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2023-10-05T19:41:26.646 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:41:26.647 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'anonymous', 'buckets': [{'bucket': '', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'anonymous', 'categories': []}]}, {'user': 'foo', 'buckets': [{'bucket': '', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}]}, {'bucket': 'myfoo2', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo3', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo4', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': []}]}], 'summary': [{'user': 'anonymous', 'categories': [], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 0, 'successful_ops': 0}}, {'user': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}}]} 2023-10-05T19:41:26.647 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show'] 2023-10-05T19:41:26.647 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show'] 2023-10-05T19:41:26.647 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show 2023-10-05T19:41:26.682 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:41:26.682 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:41:26.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.676+0000 7f13d69a3e40 1 Processor -- start 2023-10-05T19:41:26.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.676+0000 7f13d69a3e40 1 -- start start 2023-10-05T19:41:26.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.676+0000 7f13d69a3e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x555d8b4abff0 0x555d8b4ac3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:26.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.676+0000 7f13d69a3e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x555d8b3f6c00 con 0x555d8b4ac900 2023-10-05T19:41:26.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.676+0000 7f13d69a3e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x555d8b3b3ee0 con 0x555d8b4abff0 2023-10-05T19:41:26.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.680+0000 7f13d44d8700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x555d8b4abff0 0x555d8b4ac3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:26.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.680+0000 7f13d44d8700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x555d8b4abff0 0x555d8b4ac3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:50398/0 (socket says 172.21.15.12:50398) 2023-10-05T19:41:26.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.680+0000 7f13d44d8700 1 -- 172.21.15.12:0/2758982957 learned_addr learned my addr 172.21.15.12:0/2758982957 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:26.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.680+0000 7f13d3cd7700 1 --1- 172.21.15.12:0/2758982957 >> v1:172.21.15.8:6789/0 conn(0x555d8b4ac900 0x555d8b4ad3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:57646/0 (socket says 172.21.15.12:57646) 2023-10-05T19:41:26.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.680+0000 7f13d44d8700 1 -- 172.21.15.12:0/2758982957 >> v1:172.21.15.8:6789/0 conn(0x555d8b4ac900 legacy=0x555d8b4ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:26.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.680+0000 7f13d44d8700 1 -- 172.21.15.12:0/2758982957 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d8b4adab0 con 0x555d8b4abff0 2023-10-05T19:41:26.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.680+0000 7f13d44d8700 1 --2- 172.21.15.12:0/2758982957 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555d8b4abff0 0x555d8b4ac3c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f13cc00a5c0 tx=0x7f13cc005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba85214b0a87cd31 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:26.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.680+0000 7f13d34d6700 1 -- 172.21.15.12:0/2758982957 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f13cc00e510 con 0x555d8b4abff0 2023-10-05T19:41:26.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.680+0000 7f13d34d6700 1 -- 172.21.15.12:0/2758982957 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13cc00e670 con 0x555d8b4abff0 2023-10-05T19:41:26.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.680+0000 7f13d34d6700 1 -- 172.21.15.12:0/2758982957 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f13cc00e920 con 0x555d8b4abff0 2023-10-05T19:41:26.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.680+0000 7f13d69a3e40 1 -- 172.21.15.12:0/2758982957 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555d8b4abff0 msgr2=0x555d8b4ac3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:26.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.680+0000 7f13d69a3e40 1 --2- 172.21.15.12:0/2758982957 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555d8b4abff0 0x555d8b4ac3c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f13cc00a5c0 tx=0x7f13cc005e00 comp rx=0 tx=0).stop 2023-10-05T19:41:26.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.680+0000 7f13d69a3e40 1 -- 172.21.15.12:0/2758982957 shutdown_connections 2023-10-05T19:41:26.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.680+0000 7f13d69a3e40 1 --2- 172.21.15.12:0/2758982957 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555d8b4abff0 0x555d8b4ac3c0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:26.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.680+0000 7f13d69a3e40 1 -- 172.21.15.12:0/2758982957 >> 172.21.15.12:0/2758982957 conn(0x555d8b41f840 msgr2=0x555d8b4aa4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:26.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.680+0000 7f13d69a3e40 1 -- 172.21.15.12:0/2758982957 shutdown_connections 2023-10-05T19:41:26.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.680+0000 7f13d69a3e40 1 -- 172.21.15.12:0/2758982957 wait complete. 2023-10-05T19:41:26.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.684+0000 7f13d69a3e40 1 Processor -- start 2023-10-05T19:41:26.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.684+0000 7f13d69a3e40 1 -- start start 2023-10-05T19:41:26.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.684+0000 7f13d69a3e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555d8b4abff0 0x555d8b4cb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:26.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.684+0000 7f13d69a3e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d8b377e30 con 0x555d8b4abff0 2023-10-05T19:41:26.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.684+0000 7f13d44d8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555d8b4abff0 0x555d8b4cb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:26.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.684+0000 7f13d44d8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555d8b4abff0 0x555d8b4cb0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:50412/0 (socket says 172.21.15.12:50412) 2023-10-05T19:41:26.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.684+0000 7f13d44d8700 1 -- 172.21.15.12:0/1308690161 learned_addr learned my addr 172.21.15.12:0/1308690161 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:26.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.684+0000 7f13d44d8700 1 -- 172.21.15.12:0/1308690161 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13cc00a180 con 0x555d8b4abff0 2023-10-05T19:41:26.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.684+0000 7f13d44d8700 1 --2- 172.21.15.12:0/1308690161 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555d8b4abff0 0x555d8b4cb0c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f13cc00ad10 tx=0x7f13cc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab106807d7fce6f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:26.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.684+0000 7f13d24d4700 1 -- 172.21.15.12:0/1308690161 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f13cc0032f0 con 0x555d8b4abff0 2023-10-05T19:41:26.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.684+0000 7f13d24d4700 1 -- 172.21.15.12:0/1308690161 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13cc00e7e0 con 0x555d8b4abff0 2023-10-05T19:41:26.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.684+0000 7f13d24d4700 1 -- 172.21.15.12:0/1308690161 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f13cc00eab0 con 0x555d8b4abff0 2023-10-05T19:41:26.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.688+0000 7f13d69a3e40 1 -- 172.21.15.12:0/1308690161 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555d8b4abff0 msgr2=0x555d8b4cb0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:26.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.688+0000 7f13d69a3e40 1 --2- 172.21.15.12:0/1308690161 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555d8b4abff0 0x555d8b4cb0c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f13cc00ad10 tx=0x7f13cc013000 comp rx=0 tx=0).stop 2023-10-05T19:41:26.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.688+0000 7f13d69a3e40 1 -- 172.21.15.12:0/1308690161 shutdown_connections 2023-10-05T19:41:26.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.688+0000 7f13d69a3e40 1 --2- 172.21.15.12:0/1308690161 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555d8b4abff0 0x555d8b4cb0c0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:26.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.688+0000 7f13d69a3e40 1 -- 172.21.15.12:0/1308690161 >> 172.21.15.12:0/1308690161 conn(0x555d8b41f840 msgr2=0x555d8b4c6c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:26.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.688+0000 7f13d69a3e40 1 -- 172.21.15.12:0/1308690161 shutdown_connections 2023-10-05T19:41:26.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.688+0000 7f13d69a3e40 1 -- 172.21.15.12:0/1308690161 wait complete. 2023-10-05T19:41:26.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.688+0000 7f13d69a3e40 1 Processor -- start 2023-10-05T19:41:26.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.688+0000 7f13d69a3e40 1 -- start start 2023-10-05T19:41:26.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.688+0000 7f13d69a3e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555d8b4abff0 0x555d8b4d1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:26.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.688+0000 7f13d69a3e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d8b34aed0 con 0x555d8b4abff0 2023-10-05T19:41:26.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.688+0000 7f13d44d8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555d8b4abff0 0x555d8b4d1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:26.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.688+0000 7f13d44d8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555d8b4abff0 0x555d8b4d1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:50424/0 (socket says 172.21.15.12:50424) 2023-10-05T19:41:26.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.688+0000 7f13d44d8700 1 -- 172.21.15.12:0/953947386 learned_addr learned my addr 172.21.15.12:0/953947386 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:26.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.688+0000 7f13d44d8700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d8b4adab0 con 0x555d8b4abff0 2023-10-05T19:41:26.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.692+0000 7f13d44d8700 1 --2- 172.21.15.12:0/953947386 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555d8b4abff0 0x555d8b4d1d20 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f13cc00a5c0 tx=0x7f13cc01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:26.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.692+0000 7f13d0cd1700 1 -- 172.21.15.12:0/953947386 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f13cc005e70 con 0x555d8b4abff0 2023-10-05T19:41:26.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.692+0000 7f13d0cd1700 1 -- 172.21.15.12:0/953947386 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13cc0109f0 con 0x555d8b4abff0 2023-10-05T19:41:26.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.692+0000 7f13d0cd1700 1 -- 172.21.15.12:0/953947386 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f13cc010ce0 con 0x555d8b4abff0 2023-10-05T19:41:26.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.692+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555d8b4d2260 con 0x555d8b4abff0 2023-10-05T19:41:26.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.692+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555d8b4d2640 con 0x555d8b4abff0 2023-10-05T19:41:26.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.692+0000 7f13d0cd1700 1 -- 172.21.15.12:0/953947386 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f13cc037450 con 0x555d8b4abff0 2023-10-05T19:41:26.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.692+0000 7f13d0cd1700 1 --2- 172.21.15.12:0/953947386 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f13c0030ca0 0x7f13c0033150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:26.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.692+0000 7f13d0cd1700 1 -- 172.21.15.12:0/953947386 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7f13cc046370 con 0x555d8b4abff0 2023-10-05T19:41:26.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.692+0000 7f13d3cd7700 1 --2- 172.21.15.12:0/953947386 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f13c0030ca0 0x7f13c0033150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:26.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.692+0000 7f13d69a3e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:26.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.692+0000 7f13d69a3e40 1 --2- 172.21.15.12:0/953947386 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x555d8b4ac900 0x555d8b4dd790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:26.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.692+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4ddec0 con 0x555d8b4ac900 2023-10-05T19:41:26.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.696+0000 7f13d4cd9700 1 --2- 172.21.15.12:0/953947386 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x555d8b4ac900 0x555d8b4dd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:26.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.696+0000 7f13d3cd7700 1 --2- 172.21.15.12:0/953947386 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f13c0030ca0 0x7f13c0033150 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f13c800b300 tx=0x7f13c8001430 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:26.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.696+0000 7f13d4cd9700 1 --2- 172.21.15.12:0/953947386 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x555d8b4ac900 0x555d8b4dd790 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:26.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.696+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f13c400a500 con 0x555d8b4ac900 2023-10-05T19:41:26.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.696+0000 7f13d69a3e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:26.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.696+0000 7f13d69a3e40 20 realm 2023-10-05T19:41:26.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.696+0000 7f13d69a3e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:26.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.696+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4dad70 con 0x555d8b4ac900 2023-10-05T19:41:26.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.696+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f13c400a500 con 0x555d8b4ac900 2023-10-05T19:41:26.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.696+0000 7f13d69a3e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:26.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.696+0000 7f13d69a3e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:41:26.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.696+0000 7f13d69a3e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:26.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.696+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4de9a0 con 0x555d8b4ac900 2023-10-05T19:41:26.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.696+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f13c400a500 con 0x555d8b4ac900 2023-10-05T19:41:26.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.696+0000 7f13d69a3e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:26.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.696+0000 7f13d69a3e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:26.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.696+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4de580 con 0x555d8b4ac900 2023-10-05T19:41:26.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.696+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f13c400a500 con 0x555d8b4ac900 2023-10-05T19:41:26.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.696+0000 7f13d69a3e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:26.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.696+0000 7f13d69a3e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:26.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.696+0000 7f13d69a3e40 1 --2- 172.21.15.12:0/953947386 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x555d8b4df930 0x555d8b4e1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:26.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.696+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e2480 con 0x555d8b4df930 2023-10-05T19:41:26.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.700+0000 7f13d44d8700 1 --2- 172.21.15.12:0/953947386 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x555d8b4df930 0x555d8b4e1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:26.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.700+0000 7f13d44d8700 1 --2- 172.21.15.12:0/953947386 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x555d8b4df930 0x555d8b4e1d10 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:26.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.700+0000 7f13d44d8700 1 -- 172.21.15.12:0/953947386 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f13cc034070 con 0x555d8b4df930 2023-10-05T19:41:26.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.700+0000 7f13d69a3e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:41:26.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.700+0000 7f13d69a3e40 20 searching for the correct realm 2023-10-05T19:41:26.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.700+0000 7f13d69a3e40 1 --2- 172.21.15.12:0/953947386 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x555d8b4e42b0 0x555d8b4e6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:26.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.700+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e6bd0 con 0x555d8b4e42b0 2023-10-05T19:41:26.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.700+0000 7f13d3cd7700 1 --2- 172.21.15.12:0/953947386 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x555d8b4e42b0 0x555d8b4e6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:26.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.700+0000 7f13d3cd7700 1 --2- 172.21.15.12:0/953947386 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x555d8b4e42b0 0x555d8b4e6690 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:26.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.700+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f13c800f560 con 0x555d8b4e42b0 2023-10-05T19:41:26.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.700+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e6bd0 con 0x555d8b4df930 2023-10-05T19:41:26.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.700+0000 7f13d44d8700 1 -- 172.21.15.12:0/953947386 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13cc034070 con 0x555d8b4df930 2023-10-05T19:41:26.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.700+0000 7f13d44d8700 1 --2- 172.21.15.12:0/953947386 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f13cc010070 0x7f13cc037700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:26.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.700+0000 7f13d44d8700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e6bd0 con 0x7f13cc010070 2023-10-05T19:41:26.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.704+0000 7f13d4cd9700 1 --2- 172.21.15.12:0/953947386 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f13cc010070 0x7f13cc037700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:26.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.704+0000 7f13d4cd9700 1 --2- 172.21.15.12:0/953947386 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f13cc010070 0x7f13cc037700 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:26.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.704+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13c4014040 con 0x7f13cc010070 2023-10-05T19:41:26.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.704+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e6bd0 con 0x555d8b4ac900 2023-10-05T19:41:26.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.704+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13c4014040 con 0x555d8b4ac900 2023-10-05T19:41:26.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.704+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e6bd0 con 0x555d8b4ac900 2023-10-05T19:41:26.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.704+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13c4014040 con 0x555d8b4ac900 2023-10-05T19:41:26.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.704+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e6bd0 con 0x555d8b4ac900 2023-10-05T19:41:26.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.704+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(11 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13c4014040 con 0x555d8b4ac900 2023-10-05T19:41:26.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.704+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e6bd0 con 0x7f13cc010070 2023-10-05T19:41:26.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.704+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13c4014040 con 0x7f13cc010070 2023-10-05T19:41:26.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.704+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e6bd0 con 0x555d8b4df930 2023-10-05T19:41:26.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.708+0000 7f13d44d8700 1 -- 172.21.15.12:0/953947386 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13cc034070 con 0x555d8b4df930 2023-10-05T19:41:26.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.708+0000 7f13d44d8700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e6bd0 con 0x555d8b4df930 2023-10-05T19:41:26.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.708+0000 7f13d44d8700 1 -- 172.21.15.12:0/953947386 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13cc034070 con 0x555d8b4df930 2023-10-05T19:41:26.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.708+0000 7f13d44d8700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e6bd0 con 0x555d8b4ac900 2023-10-05T19:41:26.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.708+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13c4014040 con 0x555d8b4ac900 2023-10-05T19:41:26.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.708+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e6bd0 con 0x7f13cc010070 2023-10-05T19:41:26.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.708+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13c4014040 con 0x7f13cc010070 2023-10-05T19:41:26.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.708+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e6bd0 con 0x555d8b4ac900 2023-10-05T19:41:26.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.708+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13c4014040 con 0x555d8b4ac900 2023-10-05T19:41:26.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.708+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e6bd0 con 0x555d8b4df930 2023-10-05T19:41:26.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.708+0000 7f13d44d8700 1 -- 172.21.15.12:0/953947386 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(18 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13cc034070 con 0x555d8b4df930 2023-10-05T19:41:26.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.708+0000 7f13d44d8700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e6bd0 con 0x555d8b4e42b0 2023-10-05T19:41:26.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.708+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13c800f560 con 0x555d8b4e42b0 2023-10-05T19:41:26.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.708+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e6bd0 con 0x555d8b4e42b0 2023-10-05T19:41:26.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.712+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13c800f560 con 0x555d8b4e42b0 2023-10-05T19:41:26.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.712+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e6bd0 con 0x7f13cc010070 2023-10-05T19:41:26.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.712+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13c4014040 con 0x7f13cc010070 2023-10-05T19:41:26.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.712+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e6bd0 con 0x555d8b4e42b0 2023-10-05T19:41:26.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.712+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13c800f560 con 0x555d8b4e42b0 2023-10-05T19:41:26.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.712+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e6bd0 con 0x555d8b4e42b0 2023-10-05T19:41:26.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.712+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13c800f560 con 0x555d8b4e42b0 2023-10-05T19:41:26.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.712+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e6bd0 con 0x555d8b4ac900 2023-10-05T19:41:26.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.712+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13c4014040 con 0x555d8b4ac900 2023-10-05T19:41:26.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.712+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e6bd0 con 0x555d8b4ac900 2023-10-05T19:41:26.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.712+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13c4014040 con 0x555d8b4ac900 2023-10-05T19:41:26.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.712+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e6bd0 con 0x7f13cc010070 2023-10-05T19:41:26.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.716+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13c4014040 con 0x7f13cc010070 2023-10-05T19:41:26.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.716+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e6bd0 con 0x555d8b4e42b0 2023-10-05T19:41:26.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.716+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13c800f560 con 0x555d8b4e42b0 2023-10-05T19:41:26.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.716+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e6bd0 con 0x555d8b4df930 2023-10-05T19:41:26.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.716+0000 7f13d44d8700 1 -- 172.21.15.12:0/953947386 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13cc034070 con 0x555d8b4df930 2023-10-05T19:41:26.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.716+0000 7f13d44d8700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e6bd0 con 0x555d8b4ac900 2023-10-05T19:41:26.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.716+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13c4014040 con 0x555d8b4ac900 2023-10-05T19:41:26.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.716+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e6bd0 con 0x7f13cc010070 2023-10-05T19:41:26.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.716+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13c4014040 con 0x7f13cc010070 2023-10-05T19:41:26.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.716+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e6bd0 con 0x555d8b4e42b0 2023-10-05T19:41:26.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.720+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13c800f560 con 0x555d8b4e42b0 2023-10-05T19:41:26.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.720+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e6bd0 con 0x7f13cc010070 2023-10-05T19:41:26.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.720+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13c4014040 con 0x7f13cc010070 2023-10-05T19:41:26.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.720+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e6bd0 con 0x555d8b4df930 2023-10-05T19:41:26.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.720+0000 7f13d44d8700 1 -- 172.21.15.12:0/953947386 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13cc034070 con 0x555d8b4df930 2023-10-05T19:41:26.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.720+0000 7f13d44d8700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e6bd0 con 0x555d8b4e42b0 2023-10-05T19:41:26.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.720+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13c800f560 con 0x555d8b4e42b0 2023-10-05T19:41:26.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.720+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e6bd0 con 0x7f13cc010070 2023-10-05T19:41:26.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.720+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13c4014040 con 0x7f13cc010070 2023-10-05T19:41:26.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.720+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e6bd0 con 0x555d8b4ac900 2023-10-05T19:41:26.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.724+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f13c4014040 con 0x555d8b4ac900 2023-10-05T19:41:26.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.724+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e6bd0 con 0x555d8b4df930 2023-10-05T19:41:26.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.724+0000 7f13d44d8700 1 -- 172.21.15.12:0/953947386 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(37 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f13cc034070 con 0x555d8b4df930 2023-10-05T19:41:26.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.724+0000 7f13d69a3e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:41:26.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.724+0000 7f13d69a3e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:41:26.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.724+0000 7f13d69a3e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:41:26.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.724+0000 7f13d69a3e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:41:26.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.724+0000 7f13d69a3e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:26.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.724+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e3030 con 0x555d8b4ac900 2023-10-05T19:41:26.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.724+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f13c4014040 con 0x555d8b4ac900 2023-10-05T19:41:26.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.724+0000 7f13d69a3e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:26.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.724+0000 7f13d69a3e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:26.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.724+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e3980 con 0x555d8b4ac900 2023-10-05T19:41:26.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.724+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f13c4014040 con 0x555d8b4ac900 2023-10-05T19:41:26.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.724+0000 7f13d69a3e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:26.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.724+0000 7f13d69a3e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:26.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.724+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e2bf0 con 0x555d8b4e42b0 2023-10-05T19:41:26.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.724+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f13c800f560 con 0x555d8b4e42b0 2023-10-05T19:41:26.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.724+0000 7f13d69a3e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:41:26.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.724+0000 7f13d69a3e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:26.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.724+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e7500 con 0x555d8b4e42b0 2023-10-05T19:41:26.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.724+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f13c800f560 con 0x555d8b4e42b0 2023-10-05T19:41:26.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.724+0000 7f13d69a3e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:26.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.724+0000 7f13d69a3e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:26.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.724+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e78b0 con 0x555d8b4ac900 2023-10-05T19:41:26.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.728+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f13c4014040 con 0x555d8b4ac900 2023-10-05T19:41:26.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.728+0000 7f13d69a3e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:26.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.728+0000 7f13d69a3e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:26.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.728+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e7c60 con 0x555d8b4ac900 2023-10-05T19:41:26.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.728+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f13c4014040 con 0x555d8b4ac900 2023-10-05T19:41:26.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.728+0000 7f13d69a3e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:26.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.728+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e8c70 con 0x555d8b4e42b0 2023-10-05T19:41:26.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.728+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(44 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f13c8027080 con 0x555d8b4e42b0 2023-10-05T19:41:26.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.728+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e8c70 con 0x555d8b4df930 2023-10-05T19:41:26.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.732+0000 7f13d44d8700 1 -- 172.21.15.12:0/953947386 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(45 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13cc034070 con 0x555d8b4df930 2023-10-05T19:41:26.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.732+0000 7f13d44d8700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e8c70 con 0x7f13cc010070 2023-10-05T19:41:26.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.732+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(46 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13c4014040 con 0x7f13cc010070 2023-10-05T19:41:26.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.732+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e78b0 con 0x555d8b4ac900 2023-10-05T19:41:26.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.732+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(47 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13c4014040 con 0x555d8b4ac900 2023-10-05T19:41:26.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.732+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e78b0 con 0x555d8b4ac900 2023-10-05T19:41:26.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.732+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(48 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13c4014040 con 0x555d8b4ac900 2023-10-05T19:41:26.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.732+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e78b0 con 0x555d8b4ac900 2023-10-05T19:41:26.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.732+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(49 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13c4014040 con 0x555d8b4ac900 2023-10-05T19:41:26.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.732+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e78b0 con 0x7f13cc010070 2023-10-05T19:41:26.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.736+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(50 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13c4014040 con 0x7f13cc010070 2023-10-05T19:41:26.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.736+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e78b0 con 0x555d8b4df930 2023-10-05T19:41:26.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.736+0000 7f13d44d8700 1 -- 172.21.15.12:0/953947386 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(51 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13cc034070 con 0x555d8b4df930 2023-10-05T19:41:26.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.736+0000 7f13d44d8700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e78b0 con 0x555d8b4df930 2023-10-05T19:41:26.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.736+0000 7f13d44d8700 1 -- 172.21.15.12:0/953947386 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(52 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13cc034070 con 0x555d8b4df930 2023-10-05T19:41:26.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.736+0000 7f13d44d8700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e78b0 con 0x555d8b4ac900 2023-10-05T19:41:26.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.736+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(53 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13c4014040 con 0x555d8b4ac900 2023-10-05T19:41:26.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.736+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e78b0 con 0x7f13cc010070 2023-10-05T19:41:26.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.736+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(54 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13c4014040 con 0x7f13cc010070 2023-10-05T19:41:26.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.736+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e78b0 con 0x555d8b4ac900 2023-10-05T19:41:26.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.740+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(55 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13c4014040 con 0x555d8b4ac900 2023-10-05T19:41:26.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.740+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e78b0 con 0x555d8b4df930 2023-10-05T19:41:26.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.740+0000 7f13d44d8700 1 -- 172.21.15.12:0/953947386 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(56 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13cc034070 con 0x555d8b4df930 2023-10-05T19:41:26.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.740+0000 7f13d44d8700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e78b0 con 0x555d8b4e42b0 2023-10-05T19:41:26.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.740+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(57 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13c8027080 con 0x555d8b4e42b0 2023-10-05T19:41:26.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.740+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e78b0 con 0x555d8b4e42b0 2023-10-05T19:41:26.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.740+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(58 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13c8027080 con 0x555d8b4e42b0 2023-10-05T19:41:26.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.740+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e78b0 con 0x7f13cc010070 2023-10-05T19:41:26.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.744+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(59 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13c4014040 con 0x7f13cc010070 2023-10-05T19:41:26.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.744+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e78b0 con 0x555d8b4e42b0 2023-10-05T19:41:26.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.744+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(60 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13c8027080 con 0x555d8b4e42b0 2023-10-05T19:41:26.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.744+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e78b0 con 0x555d8b4e42b0 2023-10-05T19:41:26.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.744+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(61 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13c8027080 con 0x555d8b4e42b0 2023-10-05T19:41:26.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.744+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e78b0 con 0x555d8b4ac900 2023-10-05T19:41:26.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.744+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(62 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13c4014040 con 0x555d8b4ac900 2023-10-05T19:41:26.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.744+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e78b0 con 0x555d8b4ac900 2023-10-05T19:41:26.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.744+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(63 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13c4014040 con 0x555d8b4ac900 2023-10-05T19:41:26.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.744+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e78b0 con 0x7f13cc010070 2023-10-05T19:41:26.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.748+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(64 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13c4014040 con 0x7f13cc010070 2023-10-05T19:41:26.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.748+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e78b0 con 0x555d8b4e42b0 2023-10-05T19:41:26.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.748+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(65 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13c8027080 con 0x555d8b4e42b0 2023-10-05T19:41:26.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.748+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e78b0 con 0x555d8b4df930 2023-10-05T19:41:26.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.748+0000 7f13d44d8700 1 -- 172.21.15.12:0/953947386 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(66 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13cc034070 con 0x555d8b4df930 2023-10-05T19:41:26.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.748+0000 7f13d44d8700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e78b0 con 0x555d8b4ac900 2023-10-05T19:41:26.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.748+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(67 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13c4014040 con 0x555d8b4ac900 2023-10-05T19:41:26.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.748+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e78b0 con 0x7f13cc010070 2023-10-05T19:41:26.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.752+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(68 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13c4014040 con 0x7f13cc010070 2023-10-05T19:41:26.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.752+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e78b0 con 0x555d8b4e42b0 2023-10-05T19:41:26.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.752+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(69 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13c8027080 con 0x555d8b4e42b0 2023-10-05T19:41:26.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.752+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e78b0 con 0x7f13cc010070 2023-10-05T19:41:26.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.752+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(70 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13c4014040 con 0x7f13cc010070 2023-10-05T19:41:26.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.752+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e78b0 con 0x555d8b4df930 2023-10-05T19:41:26.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.752+0000 7f13d44d8700 1 -- 172.21.15.12:0/953947386 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(71 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13cc034070 con 0x555d8b4df930 2023-10-05T19:41:26.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.752+0000 7f13d44d8700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e78b0 con 0x555d8b4e42b0 2023-10-05T19:41:26.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.752+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(72 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13c8027080 con 0x555d8b4e42b0 2023-10-05T19:41:26.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.752+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e78b0 con 0x7f13cc010070 2023-10-05T19:41:26.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.756+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(73 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f13c4014040 con 0x7f13cc010070 2023-10-05T19:41:26.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.756+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e78b0 con 0x555d8b4ac900 2023-10-05T19:41:26.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.756+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(74 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f13c4014040 con 0x555d8b4ac900 2023-10-05T19:41:26.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.756+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e78b0 con 0x555d8b4df930 2023-10-05T19:41:26.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.756+0000 7f13d44d8700 1 -- 172.21.15.12:0/953947386 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(75 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f13cc034070 con 0x555d8b4df930 2023-10-05T19:41:26.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.756+0000 7f13d69a3e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:41:26.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.756+0000 7f13d69a3e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:41:26.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.756+0000 7f13d69a3e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:41:26.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.756+0000 7f13d69a3e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:41:26.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.756+0000 7f13d69a3e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:26.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.756+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e7c60 con 0x555d8b4ac900 2023-10-05T19:41:26.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.756+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f13c4014040 con 0x555d8b4ac900 2023-10-05T19:41:26.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.756+0000 7f13d69a3e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:26.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.756+0000 7f13d69a3e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:26.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.756+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e8010 con 0x555d8b4df930 2023-10-05T19:41:26.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.760+0000 7f13d44d8700 1 -- 172.21.15.12:0/953947386 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f13cc034070 con 0x555d8b4df930 2023-10-05T19:41:26.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.760+0000 7f13d69a3e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:41:26.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.760+0000 7f13d69a3e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:26.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.760+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e88b0 con 0x555d8b4ac900 2023-10-05T19:41:26.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.760+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f13c4014040 con 0x555d8b4ac900 2023-10-05T19:41:26.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.760+0000 7f13d69a3e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:26.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.760+0000 7f13d69a3e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:26.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.760+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e88b0 con 0x555d8b4e42b0 2023-10-05T19:41:26.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.760+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f13c8027080 con 0x555d8b4e42b0 2023-10-05T19:41:26.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.760+0000 7f13d69a3e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:41:26.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.760+0000 7f13d69a3e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:26.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.760+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e9150 con 0x7f13cc010070 2023-10-05T19:41:26.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.760+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f13c4014040 con 0x7f13cc010070 2023-10-05T19:41:26.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.760+0000 7f13d69a3e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:26.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.760+0000 7f13d69a3e40 20 zone default found 2023-10-05T19:41:26.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.760+0000 7f13d69a3e40 4 Realm: () 2023-10-05T19:41:26.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.760+0000 7f13d69a3e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:41:26.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.760+0000 7f13d69a3e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:41:26.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.760+0000 7f13d69a3e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:41:26.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.760+0000 7f13d69a3e40 20 zonegroup default 2023-10-05T19:41:26.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.760+0000 7f13d69a3e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:26.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.760+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e9150 con 0x7f13cc010070 2023-10-05T19:41:26.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.764+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f13c4014040 con 0x7f13cc010070 2023-10-05T19:41:26.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.764+0000 7f13d69a3e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:26.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.764+0000 7f13d69a3e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:26.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.764+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4eaad0 con 0x555d8b4e42b0 2023-10-05T19:41:26.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.764+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f13c8027080 con 0x555d8b4e42b0 2023-10-05T19:41:26.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.764+0000 7f13d69a3e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:26.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.764+0000 7f13d69a3e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:26.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.764+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4eae80 con 0x555d8b4e42b0 2023-10-05T19:41:26.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.764+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f13c8027080 con 0x555d8b4e42b0 2023-10-05T19:41:26.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.764+0000 7f13d69a3e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:26.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.764+0000 7f13d69a3e40 20 started sync module instance, tier type = 2023-10-05T19:41:26.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.764+0000 7f13d69a3e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:41:26.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.764+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4ea190 con 0x555d8b4df930 2023-10-05T19:41:26.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.768+0000 7f13d44d8700 1 -- 172.21.15.12:0/953947386 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(84 notify.0 [create] v30'222 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f13cc034070 con 0x555d8b4df930 2023-10-05T19:41:26.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.768+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93860257504416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4eb230 con 0x555d8b4df930 2023-10-05T19:41:26.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.768+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4ebad0 con 0x555d8b4ac900 2023-10-05T19:41:26.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.772+0000 7f13d44d8700 1 -- 172.21.15.12:0/953947386 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(85 notify.0 [watch watch cookie 93860257504416] v30'223 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f13cc034070 con 0x555d8b4df930 2023-10-05T19:41:26.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.772+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(86 notify.1 [create] v30'222 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f13c4014040 con 0x555d8b4ac900 2023-10-05T19:41:26.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.772+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93860257506704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4ebad0 con 0x555d8b4ac900 2023-10-05T19:41:26.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.772+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4ec5f0 con 0x555d8b4e42b0 2023-10-05T19:41:26.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.772+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(88 notify.2 [create] v30'222 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f13c8027080 con 0x555d8b4e42b0 2023-10-05T19:41:26.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.772+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93860257513728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4ecee0 con 0x555d8b4e42b0 2023-10-05T19:41:26.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.772+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4ed8f0 con 0x555d8b4df930 2023-10-05T19:41:26.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.776+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(87 notify.1 [watch watch cookie 93860257506704] v30'223 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f13c4014040 con 0x555d8b4ac900 2023-10-05T19:41:26.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.776+0000 7f13d44d8700 1 -- 172.21.15.12:0/953947386 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(90 notify.3 [create] v30'222 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f13cc034070 con 0x555d8b4df930 2023-10-05T19:41:26.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.776+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 93860257518592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4eb700 con 0x555d8b4df930 2023-10-05T19:41:26.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.776+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4ee500 con 0x555d8b4ac900 2023-10-05T19:41:26.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.776+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(89 notify.2 [watch watch cookie 93860257513728] v30'223 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f13c8027080 con 0x555d8b4e42b0 2023-10-05T19:41:26.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.780+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(92 notify.4 [create] v30'222 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f13c4014040 con 0x555d8b4ac900 2023-10-05T19:41:26.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.780+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 93860257516016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4ee4c0 con 0x555d8b4ac900 2023-10-05T19:41:26.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.780+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4eeed0 con 0x7f13cc010070 2023-10-05T19:41:26.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.780+0000 7f13d44d8700 1 -- 172.21.15.12:0/953947386 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(91 notify.3 [watch watch cookie 93860257518592] v30'223 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f13cc034070 con 0x555d8b4df930 2023-10-05T19:41:26.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.780+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(94 notify.5 [create] v30'222 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f13c4014040 con 0x7f13cc010070 2023-10-05T19:41:26.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.784+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 93860257519696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4eeed0 con 0x7f13cc010070 2023-10-05T19:41:26.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.784+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(93 notify.4 [watch watch cookie 93860257516016] v30'223 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f13c4014040 con 0x555d8b4ac900 2023-10-05T19:41:26.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.784+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4ef770 con 0x7f13cc010070 2023-10-05T19:41:26.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.784+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(96 notify.6 [create] v30'222 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f13c4014040 con 0x7f13cc010070 2023-10-05T19:41:26.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.784+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93860257525456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4ef810 con 0x7f13cc010070 2023-10-05T19:41:26.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.784+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4f0220 con 0x7f13cc010070 2023-10-05T19:41:26.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.784+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(95 notify.5 [watch watch cookie 93860257519696] v30'223 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f13c4014040 con 0x7f13cc010070 2023-10-05T19:41:26.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.788+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(97 notify.6 [watch watch cookie 93860257525456] v30'223 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f13c4014040 con 0x7f13cc010070 2023-10-05T19:41:26.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.788+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(98 notify.7 [create] v30'222 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f13c4014040 con 0x7f13cc010070 2023-10-05T19:41:26.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.788+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93860257526560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4ef770 con 0x7f13cc010070 2023-10-05T19:41:26.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.788+0000 7f13d69a3e40 20 add_watcher() i=0 2023-10-05T19:41:26.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.788+0000 7f13d69a3e40 20 add_watcher() i=1 2023-10-05T19:41:26.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.788+0000 7f13d69a3e40 20 add_watcher() i=2 2023-10-05T19:41:26.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.788+0000 7f13d69a3e40 20 add_watcher() i=3 2023-10-05T19:41:26.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.788+0000 7f13d69a3e40 20 add_watcher() i=4 2023-10-05T19:41:26.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.788+0000 7f13d69a3e40 20 add_watcher() i=5 2023-10-05T19:41:26.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.788+0000 7f13d69a3e40 20 add_watcher() i=6 2023-10-05T19:41:26.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.788+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(99 notify.7 [watch watch cookie 93860257526560] v30'223 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f13c4014040 con 0x7f13cc010070 2023-10-05T19:41:26.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.788+0000 7f13d69a3e40 20 add_watcher() i=7 2023-10-05T19:41:26.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.788+0000 7f13d69a3e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:41:26.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.788+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4eeb00 con 0x555d8b4ac900 2023-10-05T19:41:26.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.792+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f13c4014040 con 0x555d8b4ac900 2023-10-05T19:41:26.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.792+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93860257529136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4f0670 con 0x555d8b4ac900 2023-10-05T19:41:26.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.792+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 93860257529136] v30'219 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f13c4014040 con 0x555d8b4ac900 2023-10-05T19:41:26.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.792+0000 7f13a5fe3700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:41:26.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.792+0000 7f13d69a3e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:41:26.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.792+0000 7f13d69a3e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:41:26.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.792+0000 7f13d69a3e40 1 Processor -- start 2023-10-05T19:41:26.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.792+0000 7f13d69a3e40 1 -- start start 2023-10-05T19:41:26.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.796+0000 7f13d69a3e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555d8b4fd290 0x555d8b4fd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:26.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.796+0000 7f13d69a3e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d8b4fdba0 con 0x555d8b4fd290 2023-10-05T19:41:26.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.796+0000 7f13d3cd7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555d8b4fd290 0x555d8b4fd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:26.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.796+0000 7f13d3cd7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555d8b4fd290 0x555d8b4fd660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:50438/0 (socket says 172.21.15.12:50438) 2023-10-05T19:41:26.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.796+0000 7f13d3cd7700 1 -- 172.21.15.12:0/858105503 learned_addr learned my addr 172.21.15.12:0/858105503 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:26.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.796+0000 7f13d3cd7700 1 -- 172.21.15.12:0/858105503 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d8b4fdce0 con 0x555d8b4fd290 2023-10-05T19:41:26.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.796+0000 7f13d3cd7700 1 --2- 172.21.15.12:0/858105503 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555d8b4fd290 0x555d8b4fd660 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f13c800bf50 tx=0x7f13c8010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3479c520fb1397aa server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:26.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.796+0000 7f13a57e2700 1 -- 172.21.15.12:0/858105503 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f13c8040070 con 0x555d8b4fd290 2023-10-05T19:41:26.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.796+0000 7f13a57e2700 1 -- 172.21.15.12:0/858105503 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c8025040 con 0x555d8b4fd290 2023-10-05T19:41:26.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.796+0000 7f13a57e2700 1 -- 172.21.15.12:0/858105503 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f13c8024040 con 0x555d8b4fd290 2023-10-05T19:41:26.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.796+0000 7f13d69a3e40 1 -- 172.21.15.12:0/858105503 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555d8b4fd290 msgr2=0x555d8b4fd660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:26.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.796+0000 7f13d69a3e40 1 --2- 172.21.15.12:0/858105503 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555d8b4fd290 0x555d8b4fd660 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f13c800bf50 tx=0x7f13c8010000 comp rx=0 tx=0).stop 2023-10-05T19:41:26.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.796+0000 7f13d69a3e40 1 -- 172.21.15.12:0/858105503 shutdown_connections 2023-10-05T19:41:26.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.796+0000 7f13d69a3e40 1 --2- 172.21.15.12:0/858105503 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555d8b4fd290 0x555d8b4fd660 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:26.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.796+0000 7f13d69a3e40 1 -- 172.21.15.12:0/858105503 >> 172.21.15.12:0/858105503 conn(0x555d8b4f8650 msgr2=0x555d8b4f6800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:26.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.796+0000 7f13d69a3e40 1 -- 172.21.15.12:0/858105503 shutdown_connections 2023-10-05T19:41:26.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.796+0000 7f13d69a3e40 1 -- 172.21.15.12:0/858105503 wait complete. 2023-10-05T19:41:26.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.796+0000 7f13d69a3e40 1 Processor -- start 2023-10-05T19:41:26.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.800+0000 7f13d69a3e40 1 -- start start 2023-10-05T19:41:26.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.800+0000 7f13d69a3e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555d8b4fd290 0x555d8b502d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:26.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.800+0000 7f13d69a3e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d8b503d90 con 0x555d8b4fd290 2023-10-05T19:41:26.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.800+0000 7f13d3cd7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555d8b4fd290 0x555d8b502d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:26.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.800+0000 7f13d3cd7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555d8b4fd290 0x555d8b502d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:50444/0 (socket says 172.21.15.12:50444) 2023-10-05T19:41:26.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.800+0000 7f13d3cd7700 1 -- 172.21.15.12:0/677565029 learned_addr learned my addr 172.21.15.12:0/677565029 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:26.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.800+0000 7f13d3cd7700 1 -- 172.21.15.12:0/677565029 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c8006070 con 0x555d8b4fd290 2023-10-05T19:41:26.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.800+0000 7f13d3cd7700 1 --2- 172.21.15.12:0/677565029 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555d8b4fd290 0x555d8b502d90 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f13c800ba20 tx=0x7f13c803c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:26.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.800+0000 7f131f7fe700 1 -- 172.21.15.12:0/677565029 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f13c8024040 con 0x555d8b4fd290 2023-10-05T19:41:26.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.800+0000 7f131f7fe700 1 -- 172.21.15.12:0/677565029 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c80398c0 con 0x555d8b4fd290 2023-10-05T19:41:26.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.800+0000 7f13d69a3e40 1 -- 172.21.15.12:0/677565029 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555d8b503390 con 0x555d8b4fd290 2023-10-05T19:41:26.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.800+0000 7f131f7fe700 1 -- 172.21.15.12:0/677565029 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f13c8039a70 con 0x555d8b4fd290 2023-10-05T19:41:26.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.800+0000 7f13d69a3e40 1 -- 172.21.15.12:0/677565029 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555d8b503830 con 0x555d8b4fd290 2023-10-05T19:41:26.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.800+0000 7f131f7fe700 1 -- 172.21.15.12:0/677565029 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f13c8043020 con 0x555d8b4fd290 2023-10-05T19:41:26.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.804+0000 7f131f7fe700 1 --2- 172.21.15.12:0/677565029 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1314030c50 0x7f1314033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:26.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.804+0000 7f131f7fe700 1 -- 172.21.15.12:0/677565029 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7f13c8022040 con 0x555d8b4fd290 2023-10-05T19:41:26.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.804+0000 7f13d4cd9700 1 --2- 172.21.15.12:0/677565029 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1314030c50 0x7f1314033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:26.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.804+0000 7f13d69a3e40 5 note: GC not initialized 2023-10-05T19:41:26.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.804+0000 7f13d4cd9700 1 --2- 172.21.15.12:0/677565029 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1314030c50 0x7f1314033100 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x555d8b4eeb30 tx=0x7f13c400ab80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:26.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.804+0000 7f131cff9700 20 reqs_thread_entry: start 2023-10-05T19:41:26.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.804+0000 7f13d69a3e40 20 init_complete bucket index max shards: 11 2023-10-05T19:41:26.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.804+0000 7f13127fc700 20 reqs_thread_entry: start 2023-10-05T19:41:26.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.804+0000 7f13d69a3e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:41:26.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.804+0000 7f1310ff9700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:41:57 2023 2023-10-05T19:41:26.812 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:41:26.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.804+0000 7f12f7fff700 20 reqs_thread_entry: start 2023-10-05T19:41:26.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.804+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 6.e 6:74cd889a:usage::usage.0:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b56bf50 con 0x555d8b4ac900 2023-10-05T19:41:26.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.808+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(102 usage.0 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f13c4014040 con 0x555d8b4ac900 2023-10-05T19:41:26.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.808+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:103 6.3 6:c0e78a64:usage::usage.1:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b56bf50 con 0x555d8b4df930 2023-10-05T19:41:26.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.808+0000 7f13d44d8700 1 -- 172.21.15.12:0/953947386 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(103 usage.1 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f13cc034070 con 0x555d8b4df930 2023-10-05T19:41:26.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.808+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:104 6.d 6:b564efa4:usage::usage.2:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b56bf50 con 0x555d8b4ac900 2023-10-05T19:41:26.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.808+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(104 usage.2 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f13c4014040 con 0x555d8b4ac900 2023-10-05T19:41:26.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.808+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:105 6.2 6:42d64401:usage::usage.3:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b56bf50 con 0x555d8b4e42b0 2023-10-05T19:41:26.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.808+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(105 usage.3 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f13c8027080 con 0x555d8b4e42b0 2023-10-05T19:41:26.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.808+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:106 6.7 6:e09b6f01:usage::usage.4:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b56bf50 con 0x555d8b4e42b0 2023-10-05T19:41:26.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.808+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(106 usage.4 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f13c8027080 con 0x555d8b4e42b0 2023-10-05T19:41:26.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.808+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:107 6.6 6:672810f1:usage::usage.5:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b56bf50 con 0x555d8b4e42b0 2023-10-05T19:41:26.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.808+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(107 usage.5 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f13c8027080 con 0x555d8b4e42b0 2023-10-05T19:41:26.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.808+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:108 6.4 6:25a86931:usage::usage.6:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b56bf50 con 0x555d8b4ac900 2023-10-05T19:41:26.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.812+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(108 usage.6 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f13c4014040 con 0x555d8b4ac900 2023-10-05T19:41:26.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.812+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:109 6.17 6:ebbe7322:usage::usage.7:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b56bf50 con 0x555d8b4ac900 2023-10-05T19:41:26.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.812+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== osd_op_reply(109 usage.7 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f13c4014040 con 0x555d8b4ac900 2023-10-05T19:41:26.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.812+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:110 6.6 6:607e1922:usage::usage.8:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b56bf50 con 0x555d8b4e42b0 2023-10-05T19:41:26.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.812+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 <== osd.3 v2:172.21.15.12:6816/29951 27 ==== osd_op_reply(110 usage.8 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f13c8027080 con 0x555d8b4e42b0 2023-10-05T19:41:26.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.812+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:111 6.b 6:d7809252:usage::usage.9:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b56bf50 con 0x555d8b4e42b0 2023-10-05T19:41:26.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.812+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 <== osd.3 v2:172.21.15.12:6816/29951 28 ==== osd_op_reply(111 usage.9 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f13c8027080 con 0x555d8b4e42b0 2023-10-05T19:41:26.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.812+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:112 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b56bf50 con 0x7f13cc010070 2023-10-05T19:41:26.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.812+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(112 usage.10 [call out=1268b] v0'0 uv32 ondisk = 0) v8 ==== 152+0+1268 (crc 0 0 0) 0x7f13c4014040 con 0x7f13cc010070 2023-10-05T19:41:26.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.812+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:113 6.4 6:21f845ae:usage::usage.11:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b56a0a0 con 0x555d8b4ac900 2023-10-05T19:41:26.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.816+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 39 ==== osd_op_reply(113 usage.11 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f13c4014040 con 0x555d8b4ac900 2023-10-05T19:41:26.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.816+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:114 6.f 6:f0386354:usage::usage.12:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b569bb0 con 0x555d8b4df930 2023-10-05T19:41:26.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.816+0000 7f13d44d8700 1 -- 172.21.15.12:0/953947386 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(114 usage.12 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f13cc034070 con 0x555d8b4df930 2023-10-05T19:41:26.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.816+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:115 6.c 6:3735eec4:usage::usage.13:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b569bb0 con 0x555d8b4e42b0 2023-10-05T19:41:26.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.816+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 <== osd.3 v2:172.21.15.12:6816/29951 29 ==== osd_op_reply(115 usage.13 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f13c8027080 con 0x555d8b4e42b0 2023-10-05T19:41:26.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.816+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:116 6.18 6:1f520d0a:usage::usage.14:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b569bb0 con 0x7f13cc010070 2023-10-05T19:41:26.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.816+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(116 usage.14 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f13c4014040 con 0x7f13cc010070 2023-10-05T19:41:26.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.816+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:117 6.0 6:029cee12:usage::usage.15:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b569bb0 con 0x7f13cc010070 2023-10-05T19:41:26.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.816+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(117 usage.15 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f13c4014040 con 0x7f13cc010070 2023-10-05T19:41:26.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.816+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:118 6.1e 6:7f3e030e:usage::usage.16:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b569bb0 con 0x555d8b4e42b0 2023-10-05T19:41:26.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.816+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 <== osd.3 v2:172.21.15.12:6816/29951 30 ==== osd_op_reply(118 usage.16 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f13c8027080 con 0x555d8b4e42b0 2023-10-05T19:41:26.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.816+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:119 6.10 6:0fbdcb32:usage::usage.17:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b569bb0 con 0x7f13cc010070 2023-10-05T19:41:26.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.820+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(119 usage.17 [call out=163b] v0'0 uv5 ondisk = 0) v8 ==== 152+0+163 (crc 0 0 0) 0x7f13c4014040 con 0x7f13cc010070 2023-10-05T19:41:26.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.820+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:120 6.19 6:993b03e9:usage::usage.18:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b569bb0 con 0x555d8b4ac900 2023-10-05T19:41:26.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.820+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 40 ==== osd_op_reply(120 usage.18 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f13c4014040 con 0x555d8b4ac900 2023-10-05T19:41:26.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.820+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:121 6.1a 6:58fcd270:usage::usage.19:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b569bb0 con 0x555d8b4e42b0 2023-10-05T19:41:26.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.820+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 <== osd.3 v2:172.21.15.12:6816/29951 31 ==== osd_op_reply(121 usage.19 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f13c8027080 con 0x555d8b4e42b0 2023-10-05T19:41:26.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.820+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:122 6.1f 6:fcd63d49:usage::usage.20:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b569bb0 con 0x7f13cc010070 2023-10-05T19:41:26.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.820+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(122 usage.20 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f13c4014040 con 0x7f13cc010070 2023-10-05T19:41:26.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.820+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:123 6.5 6:a2c1a412:usage::usage.21:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b569bb0 con 0x555d8b4df930 2023-10-05T19:41:26.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.820+0000 7f13d44d8700 1 -- 172.21.15.12:0/953947386 <== osd.2 v2:172.21.15.12:6800/29934 23 ==== osd_op_reply(123 usage.21 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f13cc034070 con 0x555d8b4df930 2023-10-05T19:41:26.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.820+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:124 6.13 6:cdb71d68:usage::usage.22:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b569bb0 con 0x7f13cc010070 2023-10-05T19:41:26.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.824+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.0 v2:172.21.15.8:6800/27563 30 ==== osd_op_reply(124 usage.22 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f13c4014040 con 0x7f13cc010070 2023-10-05T19:41:26.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.824+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:125 6.17 6:ef3bca03:usage::usage.23:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b569bb0 con 0x555d8b4ac900 2023-10-05T19:41:26.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.824+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 41 ==== osd_op_reply(125 usage.23 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f13c4014040 con 0x555d8b4ac900 2023-10-05T19:41:26.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.824+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:126 6.e 6:7412960a:usage::usage.24:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b569bb0 con 0x555d8b4ac900 2023-10-05T19:41:26.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.824+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 42 ==== osd_op_reply(126 usage.24 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f13c4014040 con 0x555d8b4ac900 2023-10-05T19:41:26.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.824+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:127 6.7 6:e62d1024:usage::usage.25:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b569bb0 con 0x555d8b4e42b0 2023-10-05T19:41:26.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.824+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 <== osd.3 v2:172.21.15.12:6816/29951 32 ==== osd_op_reply(127 usage.25 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f13c8027080 con 0x555d8b4e42b0 2023-10-05T19:41:26.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.824+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:128 6.d 6:b172cd59:usage::usage.26:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b569bb0 con 0x555d8b4ac900 2023-10-05T19:41:26.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.824+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 43 ==== osd_op_reply(128 usage.26 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f13c4014040 con 0x555d8b4ac900 2023-10-05T19:41:26.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.824+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:129 6.1 6:83912023:usage::usage.27:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b569bb0 con 0x555d8b4ac900 2023-10-05T19:41:26.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.824+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 44 ==== osd_op_reply(129 usage.27 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f13c4014040 con 0x555d8b4ac900 2023-10-05T19:41:26.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.824+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:130 6.1e 6:7c73bdb3:usage::usage.28:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b569bb0 con 0x555d8b4e42b0 2023-10-05T19:41:26.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.828+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 <== osd.3 v2:172.21.15.12:6816/29951 33 ==== osd_op_reply(130 usage.28 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f13c8027080 con 0x555d8b4e42b0 2023-10-05T19:41:26.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.828+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:131 6.16 6:6e59fa35:usage::usage.29:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b569bb0 con 0x555d8b4e42b0 2023-10-05T19:41:26.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.828+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 <== osd.3 v2:172.21.15.12:6816/29951 34 ==== osd_op_reply(131 usage.29 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f13c8027080 con 0x555d8b4e42b0 2023-10-05T19:41:26.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.828+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:132 6.1c 6:38fe6885:usage::usage.30:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b569bb0 con 0x555d8b4e42b0 2023-10-05T19:41:26.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.828+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 <== osd.3 v2:172.21.15.12:6816/29951 35 ==== osd_op_reply(132 usage.30 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f13c8027080 con 0x555d8b4e42b0 2023-10-05T19:41:26.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.828+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:133 6.1d 6:bb28dff0:usage::usage.31:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b569bb0 con 0x7f13cc010070 2023-10-05T19:41:26.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.828+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.0 v2:172.21.15.8:6800/27563 31 ==== osd_op_reply(133 usage.31 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f13c4014040 con 0x7f13cc010070 2023-10-05T19:41:26.836 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:41:26.836 INFO:teuthology.orchestra.run.smithi012.stdout: "entries": [ 2023-10-05T19:41:26.836 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.836 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "anonymous", 2023-10-05T19:41:26.837 INFO:teuthology.orchestra.run.smithi012.stdout: "buckets": [ 2023-10-05T19:41:26.837 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.837 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "", 2023-10-05T19:41:26.837 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:00:00.000000Z", 2023-10-05T19:41:26.837 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 1696532400, 2023-10-05T19:41:26.837 INFO:teuthology.orchestra.run.smithi012.stdout: "owner": "anonymous", 2023-10-05T19:41:26.838 INFO:teuthology.orchestra.run.smithi012.stdout: "categories": [ 2023-10-05T19:41:26.838 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.838 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "list_buckets", 2023-10-05T19:41:26.838 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 231, 2023-10-05T19:41:26.838 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:26.838 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 1, 2023-10-05T19:41:26.838 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 1 2023-10-05T19:41:26.838 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:26.838 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2023-10-05T19:41:26.839 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:26.839 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2023-10-05T19:41:26.839 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:26.839 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.839 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:41:26.839 INFO:teuthology.orchestra.run.smithi012.stdout: "buckets": [ 2023-10-05T19:41:26.839 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.839 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "", 2023-10-05T19:41:26.839 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:00:00.000000Z", 2023-10-05T19:41:26.839 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 1696532400, 2023-10-05T19:41:26.839 INFO:teuthology.orchestra.run.smithi012.stdout: "owner": "foo", 2023-10-05T19:41:26.840 INFO:teuthology.orchestra.run.smithi012.stdout: "categories": [ 2023-10-05T19:41:26.840 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.840 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "list_buckets", 2023-10-05T19:41:26.840 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 322, 2023-10-05T19:41:26.840 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:26.840 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 1, 2023-10-05T19:41:26.840 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 1 2023-10-05T19:41:26.840 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:26.840 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2023-10-05T19:41:26.840 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:26.840 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.841 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo", 2023-10-05T19:41:26.841 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:00:00.000000Z", 2023-10-05T19:41:26.841 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 1696532400, 2023-10-05T19:41:26.841 INFO:teuthology.orchestra.run.smithi012.stdout: "owner": "foo", 2023-10-05T19:41:26.841 INFO:teuthology.orchestra.run.smithi012.stdout: "categories": [ 2023-10-05T19:41:26.841 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.841 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "create_bucket", 2023-10-05T19:41:26.841 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:26.841 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:26.841 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 5, 2023-10-05T19:41:26.841 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 5 2023-10-05T19:41:26.841 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:26.842 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.842 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "delete_bucket", 2023-10-05T19:41:26.842 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 217, 2023-10-05T19:41:26.842 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:26.842 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 2, 2023-10-05T19:41:26.842 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 1 2023-10-05T19:41:26.842 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:26.842 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.842 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "delete_obj", 2023-10-05T19:41:26.842 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:26.842 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:26.843 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 4, 2023-10-05T19:41:26.843 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 4 2023-10-05T19:41:26.843 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:26.843 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.843 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "get_acls", 2023-10-05T19:41:26.843 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 1026, 2023-10-05T19:41:26.843 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:26.843 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 2, 2023-10-05T19:41:26.843 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 2 2023-10-05T19:41:26.843 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:26.843 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.843 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "get_obj", 2023-10-05T19:41:26.844 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 4, 2023-10-05T19:41:26.844 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:26.844 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 1, 2023-10-05T19:41:26.844 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 1 2023-10-05T19:41:26.844 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:26.844 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.844 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "list_bucket", 2023-10-05T19:41:26.844 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 515, 2023-10-05T19:41:26.844 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:26.844 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 1, 2023-10-05T19:41:26.844 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 1 2023-10-05T19:41:26.844 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:26.845 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.845 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "put_acls", 2023-10-05T19:41:26.845 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:26.845 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:26.845 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 2, 2023-10-05T19:41:26.845 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 2 2023-10-05T19:41:26.845 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:26.845 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.845 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "put_obj", 2023-10-05T19:41:26.845 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:26.846 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 30000042, 2023-10-05T19:41:26.846 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 11, 2023-10-05T19:41:26.846 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 11 2023-10-05T19:41:26.846 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:26.846 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2023-10-05T19:41:26.846 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:26.846 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.846 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo2", 2023-10-05T19:41:26.846 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:00:00.000000Z", 2023-10-05T19:41:26.846 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 1696532400, 2023-10-05T19:41:26.847 INFO:teuthology.orchestra.run.smithi012.stdout: "owner": "foo", 2023-10-05T19:41:26.847 INFO:teuthology.orchestra.run.smithi012.stdout: "categories": [ 2023-10-05T19:41:26.847 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.847 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "create_bucket", 2023-10-05T19:41:26.847 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:26.847 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:26.847 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 1, 2023-10-05T19:41:26.847 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 1 2023-10-05T19:41:26.847 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:26.847 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.847 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "delete_bucket", 2023-10-05T19:41:26.847 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:26.848 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:26.848 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 1, 2023-10-05T19:41:26.848 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 1 2023-10-05T19:41:26.848 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:26.848 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2023-10-05T19:41:26.848 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:26.848 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.848 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo3", 2023-10-05T19:41:26.848 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:00:00.000000Z", 2023-10-05T19:41:26.848 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 1696532400, 2023-10-05T19:41:26.848 INFO:teuthology.orchestra.run.smithi012.stdout: "owner": "foo", 2023-10-05T19:41:26.848 INFO:teuthology.orchestra.run.smithi012.stdout: "categories": [ 2023-10-05T19:41:26.849 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.849 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "create_bucket", 2023-10-05T19:41:26.849 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:26.849 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:26.849 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 1, 2023-10-05T19:41:26.849 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 1 2023-10-05T19:41:26.849 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:26.849 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.849 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "delete_bucket", 2023-10-05T19:41:26.849 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:26.849 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:26.850 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 1, 2023-10-05T19:41:26.850 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 1 2023-10-05T19:41:26.850 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:26.850 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2023-10-05T19:41:26.850 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:26.850 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.850 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo4", 2023-10-05T19:41:26.850 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:00:00.000000Z", 2023-10-05T19:41:26.850 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 1696532400, 2023-10-05T19:41:26.850 INFO:teuthology.orchestra.run.smithi012.stdout: "owner": "foo", 2023-10-05T19:41:26.850 INFO:teuthology.orchestra.run.smithi012.stdout: "categories": [ 2023-10-05T19:41:26.851 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.851 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "create_bucket", 2023-10-05T19:41:26.851 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:26.851 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:26.851 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 1, 2023-10-05T19:41:26.851 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 1 2023-10-05T19:41:26.851 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:26.851 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.851 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "delete_bucket", 2023-10-05T19:41:26.851 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:26.851 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:26.851 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 1, 2023-10-05T19:41:26.852 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 1 2023-10-05T19:41:26.852 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:26.852 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2023-10-05T19:41:26.852 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:26.852 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2023-10-05T19:41:26.852 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:26.852 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:41:26.852 INFO:teuthology.orchestra.run.smithi012.stdout: "summary": [ 2023-10-05T19:41:26.852 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.852 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "anonymous", 2023-10-05T19:41:26.852 INFO:teuthology.orchestra.run.smithi012.stdout: "categories": [ 2023-10-05T19:41:26.853 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.853 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "list_buckets", 2023-10-05T19:41:26.853 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 231, 2023-10-05T19:41:26.853 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:26.853 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 1, 2023-10-05T19:41:26.853 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 1 2023-10-05T19:41:26.853 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:26.853 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:41:26.853 INFO:teuthology.orchestra.run.smithi012.stdout: "total": { 2023-10-05T19:41:26.853 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 231, 2023-10-05T19:41:26.853 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:26.853 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 1, 2023-10-05T19:41:26.854 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 1 2023-10-05T19:41:26.854 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:26.854 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:26.854 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.854 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:41:26.854 INFO:teuthology.orchestra.run.smithi012.stdout: "categories": [ 2023-10-05T19:41:26.854 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.854 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "create_bucket", 2023-10-05T19:41:26.854 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:26.854 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:26.854 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 8, 2023-10-05T19:41:26.855 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 8 2023-10-05T19:41:26.855 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:26.855 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.855 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "delete_bucket", 2023-10-05T19:41:26.855 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 217, 2023-10-05T19:41:26.855 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:26.855 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 5, 2023-10-05T19:41:26.855 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 4 2023-10-05T19:41:26.855 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:26.855 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.855 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "delete_obj", 2023-10-05T19:41:26.855 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:26.856 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:26.856 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 4, 2023-10-05T19:41:26.856 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 4 2023-10-05T19:41:26.856 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:26.856 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.856 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "get_acls", 2023-10-05T19:41:26.856 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 1026, 2023-10-05T19:41:26.856 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:26.856 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 2, 2023-10-05T19:41:26.856 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 2 2023-10-05T19:41:26.856 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:26.856 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.857 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "get_obj", 2023-10-05T19:41:26.857 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 4, 2023-10-05T19:41:26.857 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:26.857 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 1, 2023-10-05T19:41:26.857 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 1 2023-10-05T19:41:26.857 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:26.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.828+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:134 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93860257504416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b569fa0 con 0x555d8b4df930 2023-10-05T19:41:26.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.832+0000 7f13d44d8700 1 -- 172.21.15.12:0/953947386 <== osd.2 v2:172.21.15.12:6800/29934 24 ==== osd_op_reply(134 notify.0 [watch unwatch cookie 93860257504416] v30'224 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f13cc034070 con 0x555d8b4df930 2023-10-05T19:41:26.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.832+0000 7f13d69a3e40 20 remove_watcher() i=0 2023-10-05T19:41:26.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.832+0000 7f13d69a3e40 2 removed watcher, disabling cache 2023-10-05T19:41:26.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.832+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:135 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93860257506704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e9ca0 con 0x555d8b4ac900 2023-10-05T19:41:26.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.836+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 45 ==== osd_op_reply(135 notify.1 [watch unwatch cookie 93860257506704] v30'224 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f13c4014040 con 0x555d8b4ac900 2023-10-05T19:41:26.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.836+0000 7f13d69a3e40 20 remove_watcher() i=1 2023-10-05T19:41:26.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.836+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:136 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93860257513728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e9ca0 con 0x555d8b4e42b0 2023-10-05T19:41:26.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.840+0000 7f13d3cd7700 1 -- 172.21.15.12:0/953947386 <== osd.3 v2:172.21.15.12:6816/29951 36 ==== osd_op_reply(136 notify.2 [watch unwatch cookie 93860257513728] v30'224 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f13c8027080 con 0x555d8b4e42b0 2023-10-05T19:41:26.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.840+0000 7f13d69a3e40 20 remove_watcher() i=2 2023-10-05T19:41:26.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.840+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:137 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93860257518592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4ec100 con 0x555d8b4df930 2023-10-05T19:41:26.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.840+0000 7f13d44d8700 1 -- 172.21.15.12:0/953947386 <== osd.2 v2:172.21.15.12:6800/29934 25 ==== osd_op_reply(137 notify.3 [watch unwatch cookie 93860257518592] v30'224 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f13cc034070 con 0x555d8b4df930 2023-10-05T19:41:26.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.840+0000 7f13d69a3e40 20 remove_watcher() i=3 2023-10-05T19:41:26.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.840+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:138 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93860257516016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4e9ca0 con 0x555d8b4ac900 2023-10-05T19:41:26.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.844+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 46 ==== osd_op_reply(138 notify.4 [watch unwatch cookie 93860257516016] v30'224 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f13c4014040 con 0x555d8b4ac900 2023-10-05T19:41:26.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.844+0000 7f13d69a3e40 20 remove_watcher() i=4 2023-10-05T19:41:26.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.844+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:139 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93860257519696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4ed400 con 0x7f13cc010070 2023-10-05T19:41:26.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.848+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.0 v2:172.21.15.8:6800/27563 32 ==== osd_op_reply(139 notify.5 [watch unwatch cookie 93860257519696] v30'224 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f13c4014040 con 0x7f13cc010070 2023-10-05T19:41:26.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.848+0000 7f13d69a3e40 20 remove_watcher() i=5 2023-10-05T19:41:26.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.848+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:140 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93860257525456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4ec100 con 0x7f13cc010070 2023-10-05T19:41:26.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.852+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.0 v2:172.21.15.8:6800/27563 33 ==== osd_op_reply(140 notify.6 [watch unwatch cookie 93860257525456] v30'224 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f13c4014040 con 0x7f13cc010070 2023-10-05T19:41:26.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.852+0000 7f13d69a3e40 20 remove_watcher() i=6 2023-10-05T19:41:26.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.852+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:141 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93860257526560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4eeed0 con 0x7f13cc010070 2023-10-05T19:41:26.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.852+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.0 v2:172.21.15.8:6800/27563 34 ==== osd_op_reply(141 notify.7 [watch unwatch cookie 93860257526560] v30'224 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f13c4014040 con 0x7f13cc010070 2023-10-05T19:41:26.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.852+0000 7f13d69a3e40 20 remove_watcher() i=7 2023-10-05T19:41:26.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.856+0000 7f13a5fe3700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:41:26.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.856+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:142 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93860257529136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x555d8b4ec100 con 0x555d8b4ac900 2023-10-05T19:41:26.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.860+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 <== osd.1 v2:172.21.15.12:6808/29933 47 ==== osd_op_reply(142 data_loggenerations_metadata [watch unwatch cookie 93860257529136] v30'220 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f13c4014040 con 0x555d8b4ac900 2023-10-05T19:41:26.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.860+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f13cc010070 msgr2=0x7f13cc037700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:26.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.860+0000 7f13d69a3e40 1 --2- 172.21.15.12:0/953947386 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f13cc010070 0x7f13cc037700 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:26.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.860+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x555d8b4ac900 msgr2=0x555d8b4dd790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:26.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.860+0000 7f13d69a3e40 1 --2- 172.21.15.12:0/953947386 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x555d8b4ac900 0x555d8b4dd790 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:26.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.860+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x555d8b4df930 msgr2=0x555d8b4e1d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:26.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.860+0000 7f13d69a3e40 1 --2- 172.21.15.12:0/953947386 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x555d8b4df930 0x555d8b4e1d10 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:26.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.860+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x555d8b4e42b0 msgr2=0x555d8b4e6690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:26.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.860+0000 7f13d69a3e40 1 --2- 172.21.15.12:0/953947386 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x555d8b4e42b0 0x555d8b4e6690 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:26.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.860+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f13c0030ca0 msgr2=0x7f13c0033150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:26.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.860+0000 7f13d69a3e40 1 --2- 172.21.15.12:0/953947386 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f13c0030ca0 0x7f13c0033150 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f13c800b300 tx=0x7f13c8001430 comp rx=0 tx=0).stop 2023-10-05T19:41:26.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.860+0000 7f13d4cd9700 1 -- 172.21.15.12:0/953947386 reap_dead start 2023-10-05T19:41:26.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.860+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555d8b4abff0 msgr2=0x555d8b4d1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:26.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.860+0000 7f13d69a3e40 1 --2- 172.21.15.12:0/953947386 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555d8b4abff0 0x555d8b4d1d20 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f13cc00a5c0 tx=0x7f13cc01c000 comp rx=0 tx=0).stop 2023-10-05T19:41:26.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.860+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 shutdown_connections 2023-10-05T19:41:26.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.860+0000 7f13d69a3e40 1 --2- 172.21.15.12:0/953947386 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555d8b4abff0 0x555d8b4d1d20 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:26.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.860+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 >> 172.21.15.12:0/953947386 conn(0x555d8b41f840 msgr2=0x555d8b4c5700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:26.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.860+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 shutdown_connections 2023-10-05T19:41:26.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.860+0000 7f13d69a3e40 1 -- 172.21.15.12:0/953947386 wait complete. 2023-10-05T19:41:26.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.860+0000 7f13d69a3e40 1 -- 172.21.15.12:0/677565029 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1314030c50 msgr2=0x7f1314033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:26.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.860+0000 7f13d69a3e40 1 --2- 172.21.15.12:0/677565029 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1314030c50 0x7f1314033100 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x555d8b4eeb30 tx=0x7f13c400ab80 comp rx=0 tx=0).stop 2023-10-05T19:41:26.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.860+0000 7f13d69a3e40 1 -- 172.21.15.12:0/677565029 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555d8b4fd290 msgr2=0x555d8b502d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:26.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.860+0000 7f13d69a3e40 1 --2- 172.21.15.12:0/677565029 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555d8b4fd290 0x555d8b502d90 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f13c800ba20 tx=0x7f13c803c000 comp rx=0 tx=0).stop 2023-10-05T19:41:26.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.860+0000 7f13d69a3e40 1 -- 172.21.15.12:0/677565029 shutdown_connections 2023-10-05T19:41:26.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.860+0000 7f13d69a3e40 1 --2- 172.21.15.12:0/677565029 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f1314030c50 0x7f1314033100 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:26.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.860+0000 7f13d69a3e40 1 --2- 172.21.15.12:0/677565029 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x555d8b4fd290 0x555d8b502d90 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:26.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.860+0000 7f13d69a3e40 1 -- 172.21.15.12:0/677565029 >> 172.21.15.12:0/677565029 conn(0x555d8b4f8650 msgr2=0x555d8b4f6750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:26.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.860+0000 7f13d69a3e40 1 -- 172.21.15.12:0/677565029 shutdown_connections 2023-10-05T19:41:26.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.860+0000 7f13d69a3e40 1 -- 172.21.15.12:0/677565029 wait complete. 2023-10-05T19:41:26.869 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.869 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "list_bucket", 2023-10-05T19:41:26.870 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 515, 2023-10-05T19:41:26.870 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:26.870 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 1, 2023-10-05T19:41:26.870 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 1 2023-10-05T19:41:26.870 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:26.870 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.870 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "list_buckets", 2023-10-05T19:41:26.870 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 322, 2023-10-05T19:41:26.870 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:26.870 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 1, 2023-10-05T19:41:26.870 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 1 2023-10-05T19:41:26.871 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:26.871 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.871 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "put_acls", 2023-10-05T19:41:26.871 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:26.871 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:26.871 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 2, 2023-10-05T19:41:26.871 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 2 2023-10-05T19:41:26.871 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:26.872 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:26.872 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "put_obj", 2023-10-05T19:41:26.872 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:26.872 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 30000042, 2023-10-05T19:41:26.872 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 11, 2023-10-05T19:41:26.872 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 11 2023-10-05T19:41:26.872 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:26.872 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:41:26.873 INFO:teuthology.orchestra.run.smithi012.stdout: "total": { 2023-10-05T19:41:26.873 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 2084, 2023-10-05T19:41:26.873 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 30000042, 2023-10-05T19:41:26.873 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 35, 2023-10-05T19:41:26.873 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 34 2023-10-05T19:41:26.873 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:26.873 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:26.874 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2023-10-05T19:41:26.874 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:41:26.878 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'anonymous', 'buckets': [{'bucket': '', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'anonymous', 'categories': [{'category': 'list_buckets', 'bytes_sent': 231, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}, {'user': 'foo', 'buckets': [{'bucket': '', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': [{'category': 'list_buckets', 'bytes_sent': 322, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 5, 'successful_ops': 5}, {'category': 'delete_bucket', 'bytes_sent': 217, 'bytes_received': 0, 'ops': 2, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}, {'category': 'get_acls', 'bytes_sent': 1026, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'get_obj', 'bytes_sent': 4, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_bucket', 'bytes_sent': 515, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_acls', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}]}, {'bucket': 'myfoo2', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo3', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo4', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'anonymous', 'categories': [{'category': 'list_buckets', 'bytes_sent': 231, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 'total': {'bytes_sent': 231, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}}, {'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 8, 'successful_ops': 8}, {'category': 'delete_bucket', 'bytes_sent': 217, 'bytes_received': 0, 'ops': 5, 'successful_ops': 4}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}, {'category': 'get_acls', 'bytes_sent': 1026, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'get_obj', 'bytes_sent': 4, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_bucket', 'bytes_sent': 515, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_buckets', 'bytes_sent': 322, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_acls', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}], 'total': {'bytes_sent': 2084, 'bytes_received': 30000042, 'ops': 35, 'successful_ops': 34}}]} 2023-10-05T19:41:26.878 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo'] 2023-10-05T19:41:26.879 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo'] 2023-10-05T19:41:26.879 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo 2023-10-05T19:41:26.960 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:41:26.961 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:41:26.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.956+0000 7f5d81530e40 1 Processor -- start 2023-10-05T19:41:26.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.956+0000 7f5d81530e40 1 -- start start 2023-10-05T19:41:26.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.956+0000 7f5d81530e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55a4b0e65b40 0x55a4b0ef4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:26.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.956+0000 7f5d81530e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55a4b0e3ebd0 con 0x55a4b0ef5130 2023-10-05T19:41:26.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.956+0000 7f5d81530e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x55a4b0dfbee0 con 0x55a4b0e65b40 2023-10-05T19:41:26.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.956+0000 7f5d7f065700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55a4b0e65b40 0x55a4b0ef4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:26.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.956+0000 7f5d7e864700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x55a4b0ef5130 0x55a4b0ef7510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:57652/0 (socket says 172.21.15.12:57652) 2023-10-05T19:41:26.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.956+0000 7f5d7e864700 1 -- 172.21.15.12:0/3919062579 learned_addr learned my addr 172.21.15.12:0/3919062579 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:41:26.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.956+0000 7f5d7e063700 1 -- 172.21.15.12:0/3919062579 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2645815275 0 0) 0x55a4b0e3ebd0 con 0x55a4b0ef5130 2023-10-05T19:41:26.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.956+0000 7f5d7e063700 1 -- 172.21.15.12:0/3919062579 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d6c001e10 con 0x55a4b0ef5130 2023-10-05T19:41:26.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.956+0000 7f5d7f065700 1 -- 172.21.15.12:0/3919062579 >> v1:172.21.15.8:6789/0 conn(0x55a4b0ef5130 legacy=0x55a4b0ef7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:26.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.956+0000 7f5d7f065700 1 -- 172.21.15.12:0/3919062579 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4b0ef9c40 con 0x55a4b0e65b40 2023-10-05T19:41:26.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.960+0000 7f5d7f065700 1 --2- 172.21.15.12:0/3919062579 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a4b0e65b40 0x55a4b0ef4bf0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f5d7000a520 tx=0x7f5d70005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3265fec0c30d6e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:26.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.960+0000 7f5d7e063700 1 -- 172.21.15.12:0/3919062579 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d7000f590 con 0x55a4b0e65b40 2023-10-05T19:41:26.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.960+0000 7f5d7e063700 1 -- 172.21.15.12:0/3919062579 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d7000f6f0 con 0x55a4b0e65b40 2023-10-05T19:41:26.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.960+0000 7f5d7e063700 1 -- 172.21.15.12:0/3919062579 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d7000f880 con 0x55a4b0e65b40 2023-10-05T19:41:26.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.960+0000 7f5d81530e40 1 -- 172.21.15.12:0/3919062579 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a4b0e65b40 msgr2=0x55a4b0ef4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:26.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.960+0000 7f5d81530e40 1 --2- 172.21.15.12:0/3919062579 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a4b0e65b40 0x55a4b0ef4bf0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f5d7000a520 tx=0x7f5d70005d40 comp rx=0 tx=0).stop 2023-10-05T19:41:26.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.960+0000 7f5d81530e40 1 -- 172.21.15.12:0/3919062579 shutdown_connections 2023-10-05T19:41:26.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.960+0000 7f5d81530e40 1 --2- 172.21.15.12:0/3919062579 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a4b0e65b40 0x55a4b0ef4bf0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:26.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.960+0000 7f5d81530e40 1 -- 172.21.15.12:0/3919062579 >> 172.21.15.12:0/3919062579 conn(0x55a4b0eec740 msgr2=0x55a4b0eeeb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:26.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.960+0000 7f5d81530e40 1 -- 172.21.15.12:0/3919062579 shutdown_connections 2023-10-05T19:41:26.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.960+0000 7f5d81530e40 1 -- 172.21.15.12:0/3919062579 wait complete. 2023-10-05T19:41:26.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.960+0000 7f5d81530e40 1 Processor -- start 2023-10-05T19:41:26.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.960+0000 7f5d81530e40 1 -- start start 2023-10-05T19:41:26.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.960+0000 7f5d81530e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a4b0e65b40 0x55a4b0f10ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:26.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.960+0000 7f5d81530e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4b0dbfe30 con 0x55a4b0e65b40 2023-10-05T19:41:26.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.960+0000 7f5d7f065700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a4b0e65b40 0x55a4b0f10ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:26.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.960+0000 7f5d7f065700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a4b0e65b40 0x55a4b0f10ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:50462/0 (socket says 172.21.15.12:50462) 2023-10-05T19:41:26.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.960+0000 7f5d7f065700 1 -- 172.21.15.12:0/3474859844 learned_addr learned my addr 172.21.15.12:0/3474859844 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:26.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.960+0000 7f5d7f065700 1 -- 172.21.15.12:0/3474859844 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d7000a0e0 con 0x55a4b0e65b40 2023-10-05T19:41:26.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.960+0000 7f5d7f065700 1 --2- 172.21.15.12:0/3474859844 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a4b0e65b40 0x55a4b0f10ce0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f5d70005d80 tx=0x7f5d70014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a0f069c58bbb3c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:26.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.964+0000 7f5d7d061700 1 -- 172.21.15.12:0/3474859844 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d7000f860 con 0x55a4b0e65b40 2023-10-05T19:41:26.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.964+0000 7f5d7d061700 1 -- 172.21.15.12:0/3474859844 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d7000f9c0 con 0x55a4b0e65b40 2023-10-05T19:41:26.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.964+0000 7f5d7d061700 1 -- 172.21.15.12:0/3474859844 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d7000fb70 con 0x55a4b0e65b40 2023-10-05T19:41:26.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.964+0000 7f5d81530e40 1 -- 172.21.15.12:0/3474859844 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a4b0e65b40 msgr2=0x55a4b0f10ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:26.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.964+0000 7f5d81530e40 1 --2- 172.21.15.12:0/3474859844 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a4b0e65b40 0x55a4b0f10ce0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f5d70005d80 tx=0x7f5d70014000 comp rx=0 tx=0).stop 2023-10-05T19:41:26.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.964+0000 7f5d81530e40 1 -- 172.21.15.12:0/3474859844 shutdown_connections 2023-10-05T19:41:26.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.964+0000 7f5d81530e40 1 --2- 172.21.15.12:0/3474859844 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a4b0e65b40 0x55a4b0f10ce0 secure :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f5d70005d80 tx=0x7f5d70014000 comp rx=0 tx=0).stop 2023-10-05T19:41:26.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.964+0000 7f5d81530e40 1 -- 172.21.15.12:0/3474859844 >> 172.21.15.12:0/3474859844 conn(0x55a4b0eec740 msgr2=0x55a4b0f0c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:26.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.964+0000 7f5d81530e40 1 -- 172.21.15.12:0/3474859844 shutdown_connections 2023-10-05T19:41:26.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.964+0000 7f5d81530e40 1 -- 172.21.15.12:0/3474859844 wait complete. 2023-10-05T19:41:26.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.964+0000 7f5d81530e40 1 Processor -- start 2023-10-05T19:41:26.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.964+0000 7f5d81530e40 1 -- start start 2023-10-05T19:41:26.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.964+0000 7f5d81530e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a4b0ef5130 0x55a4b0f17ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:26.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.964+0000 7f5d81530e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4b0d92ed0 con 0x55a4b0ef5130 2023-10-05T19:41:26.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.964+0000 7f5d7f065700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a4b0ef5130 0x55a4b0f17ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:26.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.964+0000 7f5d7f065700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a4b0ef5130 0x55a4b0f17ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:50478/0 (socket says 172.21.15.12:50478) 2023-10-05T19:41:26.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.964+0000 7f5d7f065700 1 -- 172.21.15.12:0/2068800540 learned_addr learned my addr 172.21.15.12:0/2068800540 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:26.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.964+0000 7f5d7f065700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4b0ef9c40 con 0x55a4b0ef5130 2023-10-05T19:41:26.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.964+0000 7f5d7f065700 1 --2- 172.21.15.12:0/2068800540 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a4b0ef5130 0x55a4b0f17ab0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5d70005d80 tx=0x7f5d7001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:26.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.968+0000 7f5d777fe700 1 -- 172.21.15.12:0/2068800540 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d7002f430 con 0x55a4b0ef5130 2023-10-05T19:41:26.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.968+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a4b0f17ff0 con 0x55a4b0ef5130 2023-10-05T19:41:26.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.968+0000 7f5d777fe700 1 -- 172.21.15.12:0/2068800540 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d7002f590 con 0x55a4b0ef5130 2023-10-05T19:41:26.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.968+0000 7f5d777fe700 1 -- 172.21.15.12:0/2068800540 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d7002f740 con 0x55a4b0ef5130 2023-10-05T19:41:26.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.968+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a4b0f18490 con 0x55a4b0ef5130 2023-10-05T19:41:26.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.968+0000 7f5d81530e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:26.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.968+0000 7f5d777fe700 1 -- 172.21.15.12:0/2068800540 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f5d70010420 con 0x55a4b0ef5130 2023-10-05T19:41:26.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.968+0000 7f5d777fe700 1 --2- 172.21.15.12:0/2068800540 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5d60030ca0 0x7f5d60033150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:26.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.968+0000 7f5d777fe700 1 -- 172.21.15.12:0/2068800540 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7f5d7002b070 con 0x55a4b0ef5130 2023-10-05T19:41:26.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.968+0000 7f5d7e864700 1 --2- 172.21.15.12:0/2068800540 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5d60030ca0 0x7f5d60033150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:26.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.968+0000 7f5d81530e40 1 --2- 172.21.15.12:0/2068800540 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55a4b0f21590 0x55a4b0f23970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:26.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.968+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f23fe0 con 0x55a4b0f21590 2023-10-05T19:41:26.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.968+0000 7f5d7f866700 1 --2- 172.21.15.12:0/2068800540 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55a4b0f21590 0x55a4b0f23970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:26.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.972+0000 7f5d7e864700 1 --2- 172.21.15.12:0/2068800540 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5d60030ca0 0x7f5d60033150 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5d68004170 tx=0x7f5d68001360 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:26.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.972+0000 7f5d7f866700 1 --2- 172.21.15.12:0/2068800540 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55a4b0f21590 0x55a4b0f23970 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:26.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.972+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5d7800a500 con 0x55a4b0f21590 2023-10-05T19:41:26.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.972+0000 7f5d81530e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:26.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.972+0000 7f5d81530e40 20 realm 2023-10-05T19:41:26.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.972+0000 7f5d81530e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:26.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.972+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f20c00 con 0x55a4b0f21590 2023-10-05T19:41:26.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.972+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5d7800a500 con 0x55a4b0f21590 2023-10-05T19:41:26.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.972+0000 7f5d81530e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:26.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.972+0000 7f5d81530e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:41:26.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.972+0000 7f5d81530e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:26.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.972+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f24ac0 con 0x55a4b0f21590 2023-10-05T19:41:26.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.972+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5d7800a500 con 0x55a4b0f21590 2023-10-05T19:41:26.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.972+0000 7f5d81530e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:26.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.972+0000 7f5d81530e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:26.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.972+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f246a0 con 0x55a4b0f21590 2023-10-05T19:41:26.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.972+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f5d7800a500 con 0x55a4b0f21590 2023-10-05T19:41:26.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.972+0000 7f5d81530e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:26.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.972+0000 7f5d81530e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:26.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.972+0000 7f5d81530e40 1 --2- 172.21.15.12:0/2068800540 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55a4b0f25a50 0x55a4b0f27e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:26.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.972+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f285a0 con 0x55a4b0f25a50 2023-10-05T19:41:26.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.972+0000 7f5d7f065700 1 --2- 172.21.15.12:0/2068800540 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55a4b0f25a50 0x55a4b0f27e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:26.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.972+0000 7f5d7f065700 1 --2- 172.21.15.12:0/2068800540 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55a4b0f25a50 0x55a4b0f27e30 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:26.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.976+0000 7f5d7f065700 1 -- 172.21.15.12:0/2068800540 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f5d70038780 con 0x55a4b0f25a50 2023-10-05T19:41:26.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.976+0000 7f5d81530e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:41:26.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.976+0000 7f5d81530e40 20 searching for the correct realm 2023-10-05T19:41:26.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.976+0000 7f5d81530e40 1 --2- 172.21.15.12:0/2068800540 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55a4b0f2a3d0 0x55a4b0f2c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:26.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.976+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2ccf0 con 0x55a4b0f2a3d0 2023-10-05T19:41:26.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.976+0000 7f5d7e864700 1 --2- 172.21.15.12:0/2068800540 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55a4b0f2a3d0 0x55a4b0f2c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:26.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.976+0000 7f5d7e864700 1 --2- 172.21.15.12:0/2068800540 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55a4b0f2a3d0 0x55a4b0f2c7b0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:26.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.980+0000 7f5d7e864700 1 -- 172.21.15.12:0/2068800540 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f5d68006070 con 0x55a4b0f2a3d0 2023-10-05T19:41:26.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.980+0000 7f5d7e864700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2ccf0 con 0x55a4b0f25a50 2023-10-05T19:41:26.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.980+0000 7f5d7f065700 1 -- 172.21.15.12:0/2068800540 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d70038780 con 0x55a4b0f25a50 2023-10-05T19:41:26.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.980+0000 7f5d7f065700 1 --2- 172.21.15.12:0/2068800540 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55a4b0e65b40 0x7f5d700289f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:26.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.980+0000 7f5d7f065700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2ccf0 con 0x55a4b0e65b40 2023-10-05T19:41:26.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.980+0000 7f5d7f866700 1 --2- 172.21.15.12:0/2068800540 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55a4b0e65b40 0x7f5d700289f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:26.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.980+0000 7f5d7f866700 1 --2- 172.21.15.12:0/2068800540 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55a4b0e65b40 0x7f5d700289f0 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:26.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.980+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0e65b40 2023-10-05T19:41:26.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.980+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2ccf0 con 0x55a4b0f21590 2023-10-05T19:41:26.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.984+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0f21590 2023-10-05T19:41:26.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.984+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2ccf0 con 0x55a4b0f21590 2023-10-05T19:41:26.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.984+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0f21590 2023-10-05T19:41:26.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.984+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2ccf0 con 0x55a4b0f21590 2023-10-05T19:41:26.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.984+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(11 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0f21590 2023-10-05T19:41:26.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.984+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2ccf0 con 0x55a4b0e65b40 2023-10-05T19:41:26.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.984+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0e65b40 2023-10-05T19:41:26.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.984+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2ccf0 con 0x55a4b0f25a50 2023-10-05T19:41:26.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.984+0000 7f5d7f065700 1 -- 172.21.15.12:0/2068800540 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d70038780 con 0x55a4b0f25a50 2023-10-05T19:41:26.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.984+0000 7f5d7f065700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2ccf0 con 0x55a4b0f25a50 2023-10-05T19:41:26.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.984+0000 7f5d7f065700 1 -- 172.21.15.12:0/2068800540 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d70038780 con 0x55a4b0f25a50 2023-10-05T19:41:26.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.984+0000 7f5d7f065700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2ccf0 con 0x55a4b0f21590 2023-10-05T19:41:26.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.988+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0f21590 2023-10-05T19:41:26.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.988+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2ccf0 con 0x55a4b0e65b40 2023-10-05T19:41:26.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.988+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0e65b40 2023-10-05T19:41:26.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.988+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2ccf0 con 0x55a4b0f21590 2023-10-05T19:41:26.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.988+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0f21590 2023-10-05T19:41:26.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.988+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2ccf0 con 0x55a4b0f25a50 2023-10-05T19:41:26.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.988+0000 7f5d7f065700 1 -- 172.21.15.12:0/2068800540 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(18 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d7004d040 con 0x55a4b0f25a50 2023-10-05T19:41:26.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.988+0000 7f5d7f065700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2ccf0 con 0x55a4b0f2a3d0 2023-10-05T19:41:26.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.988+0000 7f5d7e864700 1 -- 172.21.15.12:0/2068800540 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d68006070 con 0x55a4b0f2a3d0 2023-10-05T19:41:26.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.992+0000 7f5d7e864700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2ccf0 con 0x55a4b0f2a3d0 2023-10-05T19:41:26.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.992+0000 7f5d7e864700 1 -- 172.21.15.12:0/2068800540 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d68006070 con 0x55a4b0f2a3d0 2023-10-05T19:41:26.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.992+0000 7f5d7e864700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2ccf0 con 0x55a4b0e65b40 2023-10-05T19:41:26.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.992+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0e65b40 2023-10-05T19:41:26.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.992+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2ccf0 con 0x55a4b0f2a3d0 2023-10-05T19:41:26.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.992+0000 7f5d7e864700 1 -- 172.21.15.12:0/2068800540 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d68006070 con 0x55a4b0f2a3d0 2023-10-05T19:41:26.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.992+0000 7f5d7e864700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2ccf0 con 0x55a4b0f2a3d0 2023-10-05T19:41:27.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.992+0000 7f5d7e864700 1 -- 172.21.15.12:0/2068800540 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d68006070 con 0x55a4b0f2a3d0 2023-10-05T19:41:27.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.992+0000 7f5d7e864700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2ccf0 con 0x55a4b0f21590 2023-10-05T19:41:27.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.996+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0f21590 2023-10-05T19:41:27.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.996+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2ccf0 con 0x55a4b0f21590 2023-10-05T19:41:27.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.996+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0f21590 2023-10-05T19:41:27.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.996+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2ccf0 con 0x55a4b0e65b40 2023-10-05T19:41:27.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.996+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0e65b40 2023-10-05T19:41:27.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.996+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2ccf0 con 0x55a4b0f2a3d0 2023-10-05T19:41:27.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.996+0000 7f5d7e864700 1 -- 172.21.15.12:0/2068800540 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d68006070 con 0x55a4b0f2a3d0 2023-10-05T19:41:27.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.996+0000 7f5d7e864700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2ccf0 con 0x55a4b0f25a50 2023-10-05T19:41:27.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:26.996+0000 7f5d7f065700 1 -- 172.21.15.12:0/2068800540 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d7004d040 con 0x55a4b0f25a50 2023-10-05T19:41:27.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.000+0000 7f5d7f065700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2ccf0 con 0x55a4b0f21590 2023-10-05T19:41:27.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.000+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0f21590 2023-10-05T19:41:27.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.000+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2ccf0 con 0x55a4b0e65b40 2023-10-05T19:41:27.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.000+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0e65b40 2023-10-05T19:41:27.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.000+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2ccf0 con 0x55a4b0f2a3d0 2023-10-05T19:41:27.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.000+0000 7f5d7e864700 1 -- 172.21.15.12:0/2068800540 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d68006070 con 0x55a4b0f2a3d0 2023-10-05T19:41:27.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.000+0000 7f5d7e864700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2ccf0 con 0x55a4b0e65b40 2023-10-05T19:41:27.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.000+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0e65b40 2023-10-05T19:41:27.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.000+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2ccf0 con 0x55a4b0f25a50 2023-10-05T19:41:27.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.004+0000 7f5d7f065700 1 -- 172.21.15.12:0/2068800540 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d7004d040 con 0x55a4b0f25a50 2023-10-05T19:41:27.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.004+0000 7f5d7f065700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2ccf0 con 0x55a4b0f2a3d0 2023-10-05T19:41:27.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.004+0000 7f5d7e864700 1 -- 172.21.15.12:0/2068800540 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d68006070 con 0x55a4b0f2a3d0 2023-10-05T19:41:27.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.004+0000 7f5d7e864700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2ccf0 con 0x55a4b0e65b40 2023-10-05T19:41:27.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.004+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0e65b40 2023-10-05T19:41:27.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.004+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2ccf0 con 0x55a4b0f21590 2023-10-05T19:41:27.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.004+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0f21590 2023-10-05T19:41:27.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.004+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2ccf0 con 0x55a4b0f25a50 2023-10-05T19:41:27.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.008+0000 7f5d7f065700 1 -- 172.21.15.12:0/2068800540 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(37 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f5d7004d040 con 0x55a4b0f25a50 2023-10-05T19:41:27.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.008+0000 7f5d81530e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:41:27.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.008+0000 7f5d81530e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:41:27.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.008+0000 7f5d81530e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:41:27.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.008+0000 7f5d81530e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:41:27.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.008+0000 7f5d81530e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.008+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f29120 con 0x55a4b0f21590 2023-10-05T19:41:27.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.008+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0f21590 2023-10-05T19:41:27.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.008+0000 7f5d81530e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.008+0000 7f5d81530e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.008+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f28cf0 con 0x55a4b0f21590 2023-10-05T19:41:27.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.008+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0f21590 2023-10-05T19:41:27.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.008+0000 7f5d81530e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:27.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.008+0000 7f5d81530e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.008+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f29a50 con 0x55a4b0f2a3d0 2023-10-05T19:41:27.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.008+0000 7f5d7e864700 1 -- 172.21.15.12:0/2068800540 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f5d68006070 con 0x55a4b0f2a3d0 2023-10-05T19:41:27.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.008+0000 7f5d81530e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:41:27.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.008+0000 7f5d81530e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.008+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2d620 con 0x55a4b0f2a3d0 2023-10-05T19:41:27.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.008+0000 7f5d7e864700 1 -- 172.21.15.12:0/2068800540 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f5d68006070 con 0x55a4b0f2a3d0 2023-10-05T19:41:27.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.008+0000 7f5d81530e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.012+0000 7f5d81530e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.012+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2d9d0 con 0x55a4b0f21590 2023-10-05T19:41:27.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.012+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0f21590 2023-10-05T19:41:27.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.012+0000 7f5d81530e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.012+0000 7f5d81530e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.012+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2dd80 con 0x55a4b0f21590 2023-10-05T19:41:27.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.012+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0f21590 2023-10-05T19:41:27.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.012+0000 7f5d81530e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:27.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.012+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2ed90 con 0x55a4b0f2a3d0 2023-10-05T19:41:27.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.016+0000 7f5d7e864700 1 -- 172.21.15.12:0/2068800540 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(44 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f5d68006070 con 0x55a4b0f2a3d0 2023-10-05T19:41:27.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.016+0000 7f5d7e864700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2ed90 con 0x55a4b0f25a50 2023-10-05T19:41:27.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.016+0000 7f5d7f065700 1 -- 172.21.15.12:0/2068800540 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(45 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d7004d040 con 0x55a4b0f25a50 2023-10-05T19:41:27.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.016+0000 7f5d7f065700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2ed90 con 0x55a4b0e65b40 2023-10-05T19:41:27.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.016+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(46 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0e65b40 2023-10-05T19:41:27.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.016+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2d9d0 con 0x55a4b0f21590 2023-10-05T19:41:27.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.016+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(47 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0f21590 2023-10-05T19:41:27.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.016+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2d9d0 con 0x55a4b0f21590 2023-10-05T19:41:27.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.020+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(48 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0f21590 2023-10-05T19:41:27.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.020+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2d9d0 con 0x55a4b0f21590 2023-10-05T19:41:27.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.020+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(49 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0f21590 2023-10-05T19:41:27.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.020+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2d9d0 con 0x55a4b0e65b40 2023-10-05T19:41:27.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.020+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(50 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0e65b40 2023-10-05T19:41:27.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.020+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2d9d0 con 0x55a4b0f25a50 2023-10-05T19:41:27.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.020+0000 7f5d7f065700 1 -- 172.21.15.12:0/2068800540 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(51 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d7004d040 con 0x55a4b0f25a50 2023-10-05T19:41:27.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.020+0000 7f5d7f065700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2d9d0 con 0x55a4b0f25a50 2023-10-05T19:41:27.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.020+0000 7f5d7f065700 1 -- 172.21.15.12:0/2068800540 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(52 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d7004d040 con 0x55a4b0f25a50 2023-10-05T19:41:27.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.020+0000 7f5d7f065700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2d9d0 con 0x55a4b0f21590 2023-10-05T19:41:27.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.024+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(53 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0f21590 2023-10-05T19:41:27.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.024+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2d9d0 con 0x55a4b0e65b40 2023-10-05T19:41:27.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.024+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(54 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0e65b40 2023-10-05T19:41:27.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.024+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2d9d0 con 0x55a4b0f21590 2023-10-05T19:41:27.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.024+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(55 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0f21590 2023-10-05T19:41:27.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.024+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2d9d0 con 0x55a4b0f25a50 2023-10-05T19:41:27.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.024+0000 7f5d7f065700 1 -- 172.21.15.12:0/2068800540 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(56 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d7004d040 con 0x55a4b0f25a50 2023-10-05T19:41:27.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.024+0000 7f5d7f065700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2d9d0 con 0x55a4b0f2a3d0 2023-10-05T19:41:27.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.024+0000 7f5d7e864700 1 -- 172.21.15.12:0/2068800540 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(57 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d68006070 con 0x55a4b0f2a3d0 2023-10-05T19:41:27.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.028+0000 7f5d7e864700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2d9d0 con 0x55a4b0f2a3d0 2023-10-05T19:41:27.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.028+0000 7f5d7e864700 1 -- 172.21.15.12:0/2068800540 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(58 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d68006070 con 0x55a4b0f2a3d0 2023-10-05T19:41:27.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.028+0000 7f5d7e864700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2d9d0 con 0x55a4b0e65b40 2023-10-05T19:41:27.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.028+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(59 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0e65b40 2023-10-05T19:41:27.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.028+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2d9d0 con 0x55a4b0f2a3d0 2023-10-05T19:41:27.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.028+0000 7f5d7e864700 1 -- 172.21.15.12:0/2068800540 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(60 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d68006070 con 0x55a4b0f2a3d0 2023-10-05T19:41:27.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.028+0000 7f5d7e864700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2d9d0 con 0x55a4b0f2a3d0 2023-10-05T19:41:27.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.028+0000 7f5d7e864700 1 -- 172.21.15.12:0/2068800540 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(61 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d68006070 con 0x55a4b0f2a3d0 2023-10-05T19:41:27.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.028+0000 7f5d7e864700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2d9d0 con 0x55a4b0f21590 2023-10-05T19:41:27.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.032+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(62 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0f21590 2023-10-05T19:41:27.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.032+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2d9d0 con 0x55a4b0f21590 2023-10-05T19:41:27.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.032+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(63 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0f21590 2023-10-05T19:41:27.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.032+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2d9d0 con 0x55a4b0e65b40 2023-10-05T19:41:27.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.032+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(64 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0e65b40 2023-10-05T19:41:27.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.032+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2d9d0 con 0x55a4b0f2a3d0 2023-10-05T19:41:27.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.032+0000 7f5d7e864700 1 -- 172.21.15.12:0/2068800540 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(65 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d68006070 con 0x55a4b0f2a3d0 2023-10-05T19:41:27.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.032+0000 7f5d7e864700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2d9d0 con 0x55a4b0f25a50 2023-10-05T19:41:27.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.036+0000 7f5d7f065700 1 -- 172.21.15.12:0/2068800540 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(66 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d7004d040 con 0x55a4b0f25a50 2023-10-05T19:41:27.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.036+0000 7f5d7f065700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2d9d0 con 0x55a4b0f21590 2023-10-05T19:41:27.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.036+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(67 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0f21590 2023-10-05T19:41:27.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.036+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2d9d0 con 0x55a4b0e65b40 2023-10-05T19:41:27.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.036+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(68 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0e65b40 2023-10-05T19:41:27.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.036+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2d9d0 con 0x55a4b0f2a3d0 2023-10-05T19:41:27.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.036+0000 7f5d7e864700 1 -- 172.21.15.12:0/2068800540 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(69 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d68006070 con 0x55a4b0f2a3d0 2023-10-05T19:41:27.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.036+0000 7f5d7e864700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2d9d0 con 0x55a4b0e65b40 2023-10-05T19:41:27.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.040+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(70 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0e65b40 2023-10-05T19:41:27.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.040+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2d9d0 con 0x55a4b0f25a50 2023-10-05T19:41:27.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.040+0000 7f5d7f065700 1 -- 172.21.15.12:0/2068800540 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(71 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d7004d040 con 0x55a4b0f25a50 2023-10-05T19:41:27.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.040+0000 7f5d7f065700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2d9d0 con 0x55a4b0f2a3d0 2023-10-05T19:41:27.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.040+0000 7f5d7e864700 1 -- 172.21.15.12:0/2068800540 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(72 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d68006070 con 0x55a4b0f2a3d0 2023-10-05T19:41:27.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.040+0000 7f5d7e864700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2d9d0 con 0x55a4b0e65b40 2023-10-05T19:41:27.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.040+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(73 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0e65b40 2023-10-05T19:41:27.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.040+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2d9d0 con 0x55a4b0f21590 2023-10-05T19:41:27.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.040+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(74 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0f21590 2023-10-05T19:41:27.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.040+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2d9d0 con 0x55a4b0f25a50 2023-10-05T19:41:27.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.044+0000 7f5d7f065700 1 -- 172.21.15.12:0/2068800540 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(75 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f5d7004d040 con 0x55a4b0f25a50 2023-10-05T19:41:27.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.044+0000 7f5d81530e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:41:27.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.044+0000 7f5d81530e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:41:27.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.044+0000 7f5d81530e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:41:27.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.044+0000 7f5d81530e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:41:27.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.044+0000 7f5d81530e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.044+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2dd80 con 0x55a4b0f21590 2023-10-05T19:41:27.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.044+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0f21590 2023-10-05T19:41:27.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.044+0000 7f5d81530e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:27.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.044+0000 7f5d81530e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.044+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2e130 con 0x55a4b0f25a50 2023-10-05T19:41:27.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.044+0000 7f5d7f065700 1 -- 172.21.15.12:0/2068800540 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f5d7004d040 con 0x55a4b0f25a50 2023-10-05T19:41:27.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.044+0000 7f5d81530e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:41:27.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.044+0000 7f5d81530e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.044+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2e9d0 con 0x55a4b0f21590 2023-10-05T19:41:27.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.044+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0f21590 2023-10-05T19:41:27.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.044+0000 7f5d81530e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:27.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.044+0000 7f5d81530e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.044+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2e9d0 con 0x55a4b0f2a3d0 2023-10-05T19:41:27.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.048+0000 7f5d7e864700 1 -- 172.21.15.12:0/2068800540 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f5d68006070 con 0x55a4b0f2a3d0 2023-10-05T19:41:27.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.048+0000 7f5d81530e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:41:27.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.048+0000 7f5d81530e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.048+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2e560 con 0x55a4b0e65b40 2023-10-05T19:41:27.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.048+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0e65b40 2023-10-05T19:41:27.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.048+0000 7f5d81530e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.048+0000 7f5d81530e40 20 zone default found 2023-10-05T19:41:27.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.048+0000 7f5d81530e40 4 Realm: () 2023-10-05T19:41:27.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.048+0000 7f5d81530e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:41:27.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.048+0000 7f5d81530e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:41:27.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.048+0000 7f5d81530e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:41:27.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.048+0000 7f5d81530e40 20 zonegroup default 2023-10-05T19:41:27.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.048+0000 7f5d81530e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.048+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2e560 con 0x55a4b0e65b40 2023-10-05T19:41:27.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.048+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0e65b40 2023-10-05T19:41:27.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.048+0000 7f5d81530e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.048+0000 7f5d81530e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.048+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2e560 con 0x55a4b0f2a3d0 2023-10-05T19:41:27.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.048+0000 7f5d7e864700 1 -- 172.21.15.12:0/2068800540 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f5d68006070 con 0x55a4b0f2a3d0 2023-10-05T19:41:27.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.048+0000 7f5d81530e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.048+0000 7f5d81530e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.048+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f30bf0 con 0x55a4b0f2a3d0 2023-10-05T19:41:27.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.052+0000 7f5d7e864700 1 -- 172.21.15.12:0/2068800540 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f5d68006070 con 0x55a4b0f2a3d0 2023-10-05T19:41:27.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.052+0000 7f5d81530e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.052+0000 7f5d81530e40 20 started sync module instance, tier type = 2023-10-05T19:41:27.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.052+0000 7f5d81530e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:41:27.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.052+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f302b0 con 0x55a4b0f25a50 2023-10-05T19:41:27.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.052+0000 7f5d7f065700 1 -- 172.21.15.12:0/2068800540 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(84 notify.0 [create] v30'225 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5d7004d040 con 0x55a4b0f25a50 2023-10-05T19:41:27.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.052+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94165831714480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f2fee0 con 0x55a4b0f25a50 2023-10-05T19:41:27.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.056+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f319c0 con 0x55a4b0f21590 2023-10-05T19:41:27.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.056+0000 7f5d7f065700 1 -- 172.21.15.12:0/2068800540 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(85 notify.0 [watch watch cookie 94165831714480] v30'226 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5d7004d040 con 0x55a4b0f25a50 2023-10-05T19:41:27.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.056+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(86 notify.1 [create] v30'225 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0f21590 2023-10-05T19:41:27.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.056+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94165831716768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f31dc0 con 0x55a4b0f21590 2023-10-05T19:41:27.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.056+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f32660 con 0x55a4b0f2a3d0 2023-10-05T19:41:27.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.060+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(87 notify.1 [watch watch cookie 94165831716768] v30'226 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0f21590 2023-10-05T19:41:27.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.060+0000 7f5d7e864700 1 -- 172.21.15.12:0/2068800540 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(88 notify.2 [create] v30'225 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5d68006070 con 0x55a4b0f2a3d0 2023-10-05T19:41:27.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.060+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94165831720384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f32210 con 0x55a4b0f2a3d0 2023-10-05T19:41:27.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.060+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f33440 con 0x55a4b0f25a50 2023-10-05T19:41:27.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.064+0000 7f5d7f065700 1 -- 172.21.15.12:0/2068800540 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(90 notify.3 [create] v30'225 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5d7004d040 con 0x55a4b0f25a50 2023-10-05T19:41:27.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.064+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94165831724880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f338d0 con 0x55a4b0f25a50 2023-10-05T19:41:27.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.064+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f34460 con 0x55a4b0f21590 2023-10-05T19:41:27.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.064+0000 7f5d7e864700 1 -- 172.21.15.12:0/2068800540 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(89 notify.2 [watch watch cookie 94165831720384] v30'226 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5d68006070 con 0x55a4b0f2a3d0 2023-10-05T19:41:27.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.064+0000 7f5d7f065700 1 -- 172.21.15.12:0/2068800540 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(91 notify.3 [watch watch cookie 94165831724880] v30'226 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5d7004d040 con 0x55a4b0f25a50 2023-10-05T19:41:27.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.068+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(92 notify.4 [create] v30'225 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0f21590 2023-10-05T19:41:27.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.068+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94165831723616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f33f30 con 0x55a4b0f21590 2023-10-05T19:41:27.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.068+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f34d50 con 0x55a4b0e65b40 2023-10-05T19:41:27.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.068+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(93 notify.4 [watch watch cookie 94165831723616] v30'226 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0f21590 2023-10-05T19:41:27.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.068+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(94 notify.5 [create] v30'225 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0e65b40 2023-10-05T19:41:27.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.068+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94165831725984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f34a80 con 0x55a4b0e65b40 2023-10-05T19:41:27.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.068+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f35320 con 0x55a4b0e65b40 2023-10-05T19:41:27.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.072+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(95 notify.5 [watch watch cookie 94165831725984] v30'226 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0e65b40 2023-10-05T19:41:27.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.072+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(96 notify.6 [create] v30'225 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0e65b40 2023-10-05T19:41:27.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.072+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94165831731136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f35100 con 0x55a4b0e65b40 2023-10-05T19:41:27.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.072+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f35c40 con 0x55a4b0e65b40 2023-10-05T19:41:27.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.076+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(97 notify.6 [watch watch cookie 94165831731136] v30'226 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0e65b40 2023-10-05T19:41:27.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.076+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(98 notify.7 [create] v30'225 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0e65b40 2023-10-05T19:41:27.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.076+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94165831735056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f35100 con 0x55a4b0e65b40 2023-10-05T19:41:27.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.076+0000 7f5d81530e40 20 add_watcher() i=0 2023-10-05T19:41:27.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.076+0000 7f5d81530e40 20 add_watcher() i=1 2023-10-05T19:41:27.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.076+0000 7f5d81530e40 20 add_watcher() i=2 2023-10-05T19:41:27.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.076+0000 7f5d81530e40 20 add_watcher() i=3 2023-10-05T19:41:27.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.076+0000 7f5d81530e40 20 add_watcher() i=4 2023-10-05T19:41:27.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.076+0000 7f5d81530e40 20 add_watcher() i=5 2023-10-05T19:41:27.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.076+0000 7f5d81530e40 20 add_watcher() i=6 2023-10-05T19:41:27.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.076+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(99 notify.7 [watch watch cookie 94165831735056] v30'226 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0e65b40 2023-10-05T19:41:27.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.076+0000 7f5d81530e40 20 add_watcher() i=7 2023-10-05T19:41:27.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.080+0000 7f5d81530e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:41:27.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.080+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f38fe0 con 0x55a4b0f21590 2023-10-05T19:41:27.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.080+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0f21590 2023-10-05T19:41:27.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.080+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94165831732704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f364a0 con 0x55a4b0f21590 2023-10-05T19:41:27.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.080+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94165831732704] v30'221 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0f21590 2023-10-05T19:41:27.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.080+0000 7f5d50fe1700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:41:27.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.080+0000 7f5d81530e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:41:27.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.080+0000 7f5d81530e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:41:27.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.084+0000 7f5d81530e40 1 Processor -- start 2023-10-05T19:41:27.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.084+0000 7f5d81530e40 1 -- start start 2023-10-05T19:41:27.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.084+0000 7f5d81530e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a4b0f43360 0x55a4b0f43730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:27.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.084+0000 7f5d81530e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4b0f43c70 con 0x55a4b0f43360 2023-10-05T19:41:27.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.084+0000 7f5d7e864700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a4b0f43360 0x55a4b0f43730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:27.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.084+0000 7f5d7e864700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a4b0f43360 0x55a4b0f43730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:50486/0 (socket says 172.21.15.12:50486) 2023-10-05T19:41:27.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.084+0000 7f5d7e864700 1 -- 172.21.15.12:0/632909539 learned_addr learned my addr 172.21.15.12:0/632909539 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:27.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.084+0000 7f5d7e864700 1 -- 172.21.15.12:0/632909539 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4b0f43db0 con 0x55a4b0f43360 2023-10-05T19:41:27.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.084+0000 7f5d7e864700 1 --2- 172.21.15.12:0/632909539 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a4b0f43360 0x55a4b0f43730 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x55a4b0f251c0 tx=0x7f5d6801e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4226e2d78553233f server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:27.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.084+0000 7f5ccbfff700 1 -- 172.21.15.12:0/632909539 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d68030040 con 0x55a4b0f43360 2023-10-05T19:41:27.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.084+0000 7f5ccbfff700 1 -- 172.21.15.12:0/632909539 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d6800ec00 con 0x55a4b0f43360 2023-10-05T19:41:27.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.084+0000 7f5ccbfff700 1 -- 172.21.15.12:0/632909539 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d68021040 con 0x55a4b0f43360 2023-10-05T19:41:27.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.084+0000 7f5d81530e40 1 -- 172.21.15.12:0/632909539 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a4b0f43360 msgr2=0x55a4b0f43730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.084+0000 7f5d81530e40 1 --2- 172.21.15.12:0/632909539 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a4b0f43360 0x55a4b0f43730 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x55a4b0f251c0 tx=0x7f5d6801e040 comp rx=0 tx=0).stop 2023-10-05T19:41:27.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.084+0000 7f5d81530e40 1 -- 172.21.15.12:0/632909539 shutdown_connections 2023-10-05T19:41:27.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.084+0000 7f5d81530e40 1 --2- 172.21.15.12:0/632909539 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a4b0f43360 0x55a4b0f43730 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:27.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.084+0000 7f5d81530e40 1 -- 172.21.15.12:0/632909539 >> 172.21.15.12:0/632909539 conn(0x55a4b0f3e720 msgr2=0x55a4b0f3c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:27.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.084+0000 7f5d81530e40 1 -- 172.21.15.12:0/632909539 shutdown_connections 2023-10-05T19:41:27.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.084+0000 7f5d81530e40 1 -- 172.21.15.12:0/632909539 wait complete. 2023-10-05T19:41:27.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.088+0000 7f5d81530e40 1 Processor -- start 2023-10-05T19:41:27.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.088+0000 7f5d81530e40 1 -- start start 2023-10-05T19:41:27.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.088+0000 7f5d81530e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a4b0f43360 0x55a4b0ef23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:27.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.088+0000 7f5d81530e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4b0ef1ca0 con 0x55a4b0f43360 2023-10-05T19:41:27.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.088+0000 7f5d7e864700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a4b0f43360 0x55a4b0ef23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:27.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.088+0000 7f5d7e864700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a4b0f43360 0x55a4b0ef23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:50490/0 (socket says 172.21.15.12:50490) 2023-10-05T19:41:27.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.088+0000 7f5d7e864700 1 -- 172.21.15.12:0/817760974 learned_addr learned my addr 172.21.15.12:0/817760974 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:27.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.088+0000 7f5d7e864700 1 -- 172.21.15.12:0/817760974 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d68033040 con 0x55a4b0f43360 2023-10-05T19:41:27.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.088+0000 7f5d7e864700 1 --2- 172.21.15.12:0/817760974 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a4b0f43360 0x55a4b0ef23a0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f5d68044000 tx=0x7f5d6803c030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:27.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.088+0000 7f5cca7fc700 1 -- 172.21.15.12:0/817760974 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d68021070 con 0x55a4b0f43360 2023-10-05T19:41:27.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.088+0000 7f5cca7fc700 1 -- 172.21.15.12:0/817760974 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d6802b7b0 con 0x55a4b0f43360 2023-10-05T19:41:27.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.088+0000 7f5d81530e40 1 -- 172.21.15.12:0/817760974 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a4b0ef29a0 con 0x55a4b0f43360 2023-10-05T19:41:27.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.088+0000 7f5cca7fc700 1 -- 172.21.15.12:0/817760974 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d68030040 con 0x55a4b0f43360 2023-10-05T19:41:27.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.088+0000 7f5d81530e40 1 -- 172.21.15.12:0/817760974 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a4b0ef0d00 con 0x55a4b0f43360 2023-10-05T19:41:27.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.088+0000 7f5cca7fc700 1 -- 172.21.15.12:0/817760974 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f5d6802b920 con 0x55a4b0f43360 2023-10-05T19:41:27.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.092+0000 7f5cca7fc700 1 --2- 172.21.15.12:0/817760974 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5cc0030c50 0x7f5cc0033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:27.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.092+0000 7f5cca7fc700 1 -- 172.21.15.12:0/817760974 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7f5d6804fbe0 con 0x55a4b0f43360 2023-10-05T19:41:27.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.092+0000 7f5d7f866700 1 --2- 172.21.15.12:0/817760974 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5cc0030c50 0x7f5cc0033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:27.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.092+0000 7f5d81530e40 5 note: GC not initialized 2023-10-05T19:41:27.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.092+0000 7f5d7f866700 1 --2- 172.21.15.12:0/817760974 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5cc0030c50 0x7f5cc0033100 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x55a4b0f35330 tx=0x7f5d7800ab60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:27.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.092+0000 7f5cbf7fe700 20 reqs_thread_entry: start 2023-10-05T19:41:27.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.092+0000 7f5cbd7fa700 20 reqs_thread_entry: start 2023-10-05T19:41:27.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.092+0000 7f5d81530e40 20 init_complete bucket index max shards: 11 2023-10-05T19:41:27.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.092+0000 7f5d81530e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:41:27.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.092+0000 7f5cab7fe700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:41:57 2023 2023-10-05T19:41:27.102 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:41:27.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.092+0000 7f5caaffd700 20 reqs_thread_entry: start 2023-10-05T19:41:27.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.092+0000 7f5d81530e40 20 get_system_obj_state: rctx=0x7fffe7514d68 obj=default.rgw.meta:users.uid:foo state=0x55a4b0dcb0b0 s->prefetch_data=0 2023-10-05T19:41:27.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.092+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0fb39d0 con 0x55a4b0f21590 2023-10-05T19:41:27.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.096+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(102 foo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv15 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0f21590 2023-10-05T19:41:27.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.096+0000 7f5d81530e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:41:27.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.096+0000 7f5d81530e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:41:27.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.096+0000 7f5d81530e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.096+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0fb39a0 con 0x55a4b0f21590 2023-10-05T19:41:27.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.096+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(103 foo [call,call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv15 ondisk = 0) v8 ==== 273+0+480 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0f21590 2023-10-05T19:41:27.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.096+0000 7f5d81530e40 20 rados_obj.operate() r=0 bl.length=357 2023-10-05T19:41:27.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.096+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:104 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0fb39a0 con 0x55a4b0e65b40 2023-10-05T19:41:27.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.096+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(104 usage.10 [call out=1268b] v0'0 uv32 ondisk = 0) v8 ==== 152+0+1268 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0e65b40 2023-10-05T19:41:27.105 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:41:27.105 INFO:teuthology.orchestra.run.smithi012.stdout: "entries": [ 2023-10-05T19:41:27.105 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.105 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:41:27.105 INFO:teuthology.orchestra.run.smithi012.stdout: "buckets": [ 2023-10-05T19:41:27.106 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.106 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "", 2023-10-05T19:41:27.106 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:00:00.000000Z", 2023-10-05T19:41:27.106 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 1696532400, 2023-10-05T19:41:27.106 INFO:teuthology.orchestra.run.smithi012.stdout: "owner": "foo", 2023-10-05T19:41:27.106 INFO:teuthology.orchestra.run.smithi012.stdout: "categories": [ 2023-10-05T19:41:27.107 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.107 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "list_buckets", 2023-10-05T19:41:27.107 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 322, 2023-10-05T19:41:27.107 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:27.107 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 1, 2023-10-05T19:41:27.107 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 1 2023-10-05T19:41:27.107 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:27.108 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2023-10-05T19:41:27.108 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:27.108 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.108 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo", 2023-10-05T19:41:27.108 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:00:00.000000Z", 2023-10-05T19:41:27.108 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 1696532400, 2023-10-05T19:41:27.108 INFO:teuthology.orchestra.run.smithi012.stdout: "owner": "foo", 2023-10-05T19:41:27.109 INFO:teuthology.orchestra.run.smithi012.stdout: "categories": [ 2023-10-05T19:41:27.109 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.109 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "create_bucket", 2023-10-05T19:41:27.109 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:27.109 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:27.109 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 5, 2023-10-05T19:41:27.109 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 5 2023-10-05T19:41:27.109 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:27.110 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.110 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "delete_bucket", 2023-10-05T19:41:27.110 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 217, 2023-10-05T19:41:27.110 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:27.110 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 2, 2023-10-05T19:41:27.110 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 1 2023-10-05T19:41:27.110 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:27.110 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.111 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "delete_obj", 2023-10-05T19:41:27.111 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:27.111 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:27.111 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 4, 2023-10-05T19:41:27.111 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 4 2023-10-05T19:41:27.111 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:27.112 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.112 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "get_acls", 2023-10-05T19:41:27.112 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 1026, 2023-10-05T19:41:27.112 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:27.112 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 2, 2023-10-05T19:41:27.112 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 2 2023-10-05T19:41:27.112 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:27.112 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.112 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "get_obj", 2023-10-05T19:41:27.113 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 4, 2023-10-05T19:41:27.113 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:27.113 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 1, 2023-10-05T19:41:27.113 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 1 2023-10-05T19:41:27.113 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:27.113 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.113 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "list_bucket", 2023-10-05T19:41:27.113 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 515, 2023-10-05T19:41:27.113 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:27.113 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 1, 2023-10-05T19:41:27.113 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 1 2023-10-05T19:41:27.113 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:27.114 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.114 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "put_acls", 2023-10-05T19:41:27.114 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:27.114 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:27.114 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 2, 2023-10-05T19:41:27.114 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 2 2023-10-05T19:41:27.114 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:27.114 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.114 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "put_obj", 2023-10-05T19:41:27.114 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:27.114 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 30000042, 2023-10-05T19:41:27.115 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 11, 2023-10-05T19:41:27.115 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 11 2023-10-05T19:41:27.115 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:27.115 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2023-10-05T19:41:27.115 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:27.115 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.115 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo2", 2023-10-05T19:41:27.115 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:00:00.000000Z", 2023-10-05T19:41:27.115 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 1696532400, 2023-10-05T19:41:27.115 INFO:teuthology.orchestra.run.smithi012.stdout: "owner": "foo", 2023-10-05T19:41:27.115 INFO:teuthology.orchestra.run.smithi012.stdout: "categories": [ 2023-10-05T19:41:27.115 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.116 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "create_bucket", 2023-10-05T19:41:27.116 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:27.116 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:27.116 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 1, 2023-10-05T19:41:27.116 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 1 2023-10-05T19:41:27.116 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:27.116 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.116 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "delete_bucket", 2023-10-05T19:41:27.116 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:27.116 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:27.116 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 1, 2023-10-05T19:41:27.116 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 1 2023-10-05T19:41:27.116 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:27.117 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2023-10-05T19:41:27.117 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:27.117 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.096+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:105 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94165831714480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f71dc0 con 0x55a4b0f25a50 2023-10-05T19:41:27.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.100+0000 7f5d7f065700 1 -- 172.21.15.12:0/2068800540 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(105 notify.0 [watch unwatch cookie 94165831714480] v30'227 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5d7004d040 con 0x55a4b0f25a50 2023-10-05T19:41:27.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.100+0000 7f5d81530e40 20 remove_watcher() i=0 2023-10-05T19:41:27.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.100+0000 7f5d81530e40 2 removed watcher, disabling cache 2023-10-05T19:41:27.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.100+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:106 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94165831716768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f075c0 con 0x55a4b0f21590 2023-10-05T19:41:27.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.104+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(106 notify.1 [watch unwatch cookie 94165831716768] v30'227 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0f21590 2023-10-05T19:41:27.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.104+0000 7f5d81530e40 20 remove_watcher() i=1 2023-10-05T19:41:27.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.104+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:107 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94165831720384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f30fa0 con 0x55a4b0f2a3d0 2023-10-05T19:41:27.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.104+0000 7f5d7e864700 1 -- 172.21.15.12:0/2068800540 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(107 notify.2 [watch unwatch cookie 94165831720384] v30'227 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5d68006070 con 0x55a4b0f2a3d0 2023-10-05T19:41:27.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.104+0000 7f5d81530e40 20 remove_watcher() i=2 2023-10-05T19:41:27.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.104+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:108 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94165831724880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f31dc0 con 0x55a4b0f25a50 2023-10-05T19:41:27.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.108+0000 7f5d7f065700 1 -- 172.21.15.12:0/2068800540 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(108 notify.3 [watch unwatch cookie 94165831724880] v30'227 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5d7004d040 con 0x55a4b0f25a50 2023-10-05T19:41:27.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.108+0000 7f5d81530e40 20 remove_watcher() i=3 2023-10-05T19:41:27.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.108+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:109 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94165831723616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f30fa0 con 0x55a4b0f21590 2023-10-05T19:41:27.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.112+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== osd_op_reply(109 notify.4 [watch unwatch cookie 94165831723616] v30'227 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0f21590 2023-10-05T19:41:27.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.112+0000 7f5d81530e40 20 remove_watcher() i=4 2023-10-05T19:41:27.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.112+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:110 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94165831725984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f32f50 con 0x55a4b0e65b40 2023-10-05T19:41:27.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.116+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(110 notify.5 [watch unwatch cookie 94165831725984] v30'227 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0e65b40 2023-10-05T19:41:27.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.116+0000 7f5d81530e40 20 remove_watcher() i=5 2023-10-05T19:41:27.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.116+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:111 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94165831731136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f33440 con 0x55a4b0e65b40 2023-10-05T19:41:27.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.116+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(111 notify.6 [watch unwatch cookie 94165831731136] v30'227 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0e65b40 2023-10-05T19:41:27.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.116+0000 7f5d81530e40 20 remove_watcher() i=6 2023-10-05T19:41:27.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.116+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:112 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94165831735056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f33440 con 0x55a4b0e65b40 2023-10-05T19:41:27.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.120+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(112 notify.7 [watch unwatch cookie 94165831735056] v30'227 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0e65b40 2023-10-05T19:41:27.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.120+0000 7f5d81530e40 20 remove_watcher() i=7 2023-10-05T19:41:27.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.124+0000 7f5d50fe1700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:41:27.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.124+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:113 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94165831732704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a4b0f33440 con 0x55a4b0f21590 2023-10-05T19:41:27.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.124+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 <== osd.1 v2:172.21.15.12:6808/29933 39 ==== osd_op_reply(113 data_loggenerations_metadata [watch unwatch cookie 94165831732704] v30'222 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f5d78014040 con 0x55a4b0f21590 2023-10-05T19:41:27.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.124+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55a4b0e65b40 msgr2=0x7f5d700289f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.124+0000 7f5d81530e40 1 --2- 172.21.15.12:0/2068800540 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55a4b0e65b40 0x7f5d700289f0 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:27.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.124+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55a4b0f21590 msgr2=0x55a4b0f23970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.124+0000 7f5d81530e40 1 --2- 172.21.15.12:0/2068800540 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55a4b0f21590 0x55a4b0f23970 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:27.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.124+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55a4b0f25a50 msgr2=0x55a4b0f27e30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.124+0000 7f5d81530e40 1 --2- 172.21.15.12:0/2068800540 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55a4b0f25a50 0x55a4b0f27e30 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:27.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.124+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55a4b0f2a3d0 msgr2=0x55a4b0f2c7b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.124+0000 7f5d81530e40 1 --2- 172.21.15.12:0/2068800540 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55a4b0f2a3d0 0x55a4b0f2c7b0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:27.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.128+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5d60030ca0 msgr2=0x7f5d60033150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.128+0000 7f5d81530e40 1 --2- 172.21.15.12:0/2068800540 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5d60030ca0 0x7f5d60033150 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5d68004170 tx=0x7f5d68001360 comp rx=0 tx=0).stop 2023-10-05T19:41:27.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.128+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a4b0ef5130 msgr2=0x55a4b0f17ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.128+0000 7f5d7f866700 1 -- 172.21.15.12:0/2068800540 reap_dead start 2023-10-05T19:41:27.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.128+0000 7f5d81530e40 1 --2- 172.21.15.12:0/2068800540 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a4b0ef5130 0x55a4b0f17ab0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5d70005d80 tx=0x7f5d7001d000 comp rx=0 tx=0).stop 2023-10-05T19:41:27.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.128+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 shutdown_connections 2023-10-05T19:41:27.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.128+0000 7f5d81530e40 1 --2- 172.21.15.12:0/2068800540 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a4b0ef5130 0x55a4b0f17ab0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:27.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.128+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 >> 172.21.15.12:0/2068800540 conn(0x55a4b0eec740 msgr2=0x55a4b0f0b320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:27.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.128+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 shutdown_connections 2023-10-05T19:41:27.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.128+0000 7f5d81530e40 1 -- 172.21.15.12:0/2068800540 wait complete. 2023-10-05T19:41:27.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.128+0000 7f5d81530e40 1 -- 172.21.15.12:0/817760974 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5cc0030c50 msgr2=0x7f5cc0033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.128+0000 7f5d81530e40 1 --2- 172.21.15.12:0/817760974 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5cc0030c50 0x7f5cc0033100 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x55a4b0f35330 tx=0x7f5d7800ab60 comp rx=0 tx=0).stop 2023-10-05T19:41:27.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.128+0000 7f5d81530e40 1 -- 172.21.15.12:0/817760974 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a4b0f43360 msgr2=0x55a4b0ef23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.128+0000 7f5d81530e40 1 --2- 172.21.15.12:0/817760974 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a4b0f43360 0x55a4b0ef23a0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f5d68044000 tx=0x7f5d6803c030 comp rx=0 tx=0).stop 2023-10-05T19:41:27.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.128+0000 7f5d81530e40 1 -- 172.21.15.12:0/817760974 shutdown_connections 2023-10-05T19:41:27.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.128+0000 7f5d81530e40 1 --2- 172.21.15.12:0/817760974 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5cc0030c50 0x7f5cc0033100 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:27.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.128+0000 7f5d81530e40 1 --2- 172.21.15.12:0/817760974 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a4b0f43360 0x55a4b0ef23a0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:27.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.128+0000 7f5d81530e40 1 -- 172.21.15.12:0/817760974 >> 172.21.15.12:0/817760974 conn(0x55a4b0f3e720 msgr2=0x55a4b0f3c820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:27.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.128+0000 7f5d81530e40 1 -- 172.21.15.12:0/817760974 shutdown_connections 2023-10-05T19:41:27.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.128+0000 7f5d81530e40 1 -- 172.21.15.12:0/817760974 wait complete. 2023-10-05T19:41:27.140 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo3", 2023-10-05T19:41:27.140 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:00:00.000000Z", 2023-10-05T19:41:27.141 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 1696532400, 2023-10-05T19:41:27.141 INFO:teuthology.orchestra.run.smithi012.stdout: "owner": "foo", 2023-10-05T19:41:27.141 INFO:teuthology.orchestra.run.smithi012.stdout: "categories": [ 2023-10-05T19:41:27.141 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.141 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "create_bucket", 2023-10-05T19:41:27.141 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:27.141 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:27.141 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 1, 2023-10-05T19:41:27.141 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 1 2023-10-05T19:41:27.142 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:27.142 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.142 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "delete_bucket", 2023-10-05T19:41:27.142 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:27.142 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:27.142 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 1, 2023-10-05T19:41:27.142 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 1 2023-10-05T19:41:27.142 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:27.142 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2023-10-05T19:41:27.142 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:27.142 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.143 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo4", 2023-10-05T19:41:27.143 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:00:00.000000Z", 2023-10-05T19:41:27.143 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 1696532400, 2023-10-05T19:41:27.143 INFO:teuthology.orchestra.run.smithi012.stdout: "owner": "foo", 2023-10-05T19:41:27.143 INFO:teuthology.orchestra.run.smithi012.stdout: "categories": [ 2023-10-05T19:41:27.143 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.143 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "create_bucket", 2023-10-05T19:41:27.143 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:27.143 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:27.143 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 1, 2023-10-05T19:41:27.143 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 1 2023-10-05T19:41:27.143 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:27.144 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.144 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "delete_bucket", 2023-10-05T19:41:27.144 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:27.144 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:27.144 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 1, 2023-10-05T19:41:27.144 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 1 2023-10-05T19:41:27.144 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:27.144 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2023-10-05T19:41:27.144 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:27.144 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2023-10-05T19:41:27.144 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:27.144 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:41:27.145 INFO:teuthology.orchestra.run.smithi012.stdout: "summary": [ 2023-10-05T19:41:27.145 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.145 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:41:27.145 INFO:teuthology.orchestra.run.smithi012.stdout: "categories": [ 2023-10-05T19:41:27.145 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.145 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "create_bucket", 2023-10-05T19:41:27.145 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:27.145 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:27.145 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 8, 2023-10-05T19:41:27.145 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 8 2023-10-05T19:41:27.145 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:27.146 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.146 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "delete_bucket", 2023-10-05T19:41:27.146 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 217, 2023-10-05T19:41:27.146 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:27.146 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 5, 2023-10-05T19:41:27.146 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 4 2023-10-05T19:41:27.146 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:27.146 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.146 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "delete_obj", 2023-10-05T19:41:27.146 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:27.147 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:27.147 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 4, 2023-10-05T19:41:27.147 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 4 2023-10-05T19:41:27.147 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:27.147 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.147 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "get_acls", 2023-10-05T19:41:27.147 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 1026, 2023-10-05T19:41:27.147 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:27.147 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 2, 2023-10-05T19:41:27.147 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 2 2023-10-05T19:41:27.147 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:27.147 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.148 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "get_obj", 2023-10-05T19:41:27.148 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 4, 2023-10-05T19:41:27.148 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:27.148 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 1, 2023-10-05T19:41:27.148 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 1 2023-10-05T19:41:27.148 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:27.148 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.148 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "list_bucket", 2023-10-05T19:41:27.148 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 515, 2023-10-05T19:41:27.148 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:27.148 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 1, 2023-10-05T19:41:27.148 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 1 2023-10-05T19:41:27.148 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:27.149 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.149 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "list_buckets", 2023-10-05T19:41:27.149 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 322, 2023-10-05T19:41:27.149 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:27.149 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 1, 2023-10-05T19:41:27.149 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 1 2023-10-05T19:41:27.149 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:27.149 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.149 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "put_acls", 2023-10-05T19:41:27.149 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:27.149 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:27.149 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 2, 2023-10-05T19:41:27.150 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 2 2023-10-05T19:41:27.150 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:27.150 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.150 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "put_obj", 2023-10-05T19:41:27.150 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:27.150 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 30000042, 2023-10-05T19:41:27.150 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 11, 2023-10-05T19:41:27.150 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 11 2023-10-05T19:41:27.150 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:27.150 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:41:27.150 INFO:teuthology.orchestra.run.smithi012.stdout: "total": { 2023-10-05T19:41:27.150 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 2084, 2023-10-05T19:41:27.151 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 30000042, 2023-10-05T19:41:27.151 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 35, 2023-10-05T19:41:27.151 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 34 2023-10-05T19:41:27.151 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:27.151 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:27.151 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2023-10-05T19:41:27.151 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:41:27.151 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': '', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': [{'category': 'list_buckets', 'bytes_sent': 322, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 5, 'successful_ops': 5}, {'category': 'delete_bucket', 'bytes_sent': 217, 'bytes_received': 0, 'ops': 2, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}, {'category': 'get_acls', 'bytes_sent': 1026, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'get_obj', 'bytes_sent': 4, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_bucket', 'bytes_sent': 515, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_acls', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}]}, {'bucket': 'myfoo2', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo3', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo4', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 8, 'successful_ops': 8}, {'category': 'delete_bucket', 'bytes_sent': 217, 'bytes_received': 0, 'ops': 5, 'successful_ops': 4}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}, {'category': 'get_acls', 'bytes_sent': 1026, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'get_obj', 'bytes_sent': 4, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_bucket', 'bytes_sent': 515, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_buckets', 'bytes_sent': 322, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_acls', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}], 'total': {'bytes_sent': 2084, 'bytes_received': 30000042, 'ops': 35, 'successful_ops': 34}}]} 2023-10-05T19:41:27.152 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo', '--categories', 'create_bucket'] 2023-10-05T19:41:27.152 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo', '--categories', 'create_bucket'] 2023-10-05T19:41:27.152 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo --categories create_bucket 2023-10-05T19:41:27.195 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:41:27.195 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:41:27.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.192+0000 7efc2448de40 1 Processor -- start 2023-10-05T19:41:27.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.192+0000 7efc2448de40 1 -- start start 2023-10-05T19:41:27.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.192+0000 7efc2448de40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x56264ba88320 0x56264ba8a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:27.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.192+0000 7efc2448de40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x56264b9d0bd0 con 0x56264b9f7b50 2023-10-05T19:41:27.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.192+0000 7efc2448de40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x56264b98dee0 con 0x56264ba88320 2023-10-05T19:41:27.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.192+0000 7efc217c1700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x56264ba88320 0x56264ba8a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:27.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.192+0000 7efc21fc2700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x56264b9f7b50 0x56264ba86c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:57664/0 (socket says 172.21.15.12:57664) 2023-10-05T19:41:27.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.192+0000 7efc21fc2700 1 -- 172.21.15.12:0/2901777322 learned_addr learned my addr 172.21.15.12:0/2901777322 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:41:27.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.192+0000 7efc217c1700 1 -- 172.21.15.12:0/2901777322 >> v1:172.21.15.8:6789/0 conn(0x56264b9f7b50 legacy=0x56264ba86c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.192+0000 7efc20fc0700 1 -- 172.21.15.12:0/2901777322 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3298507238 0 0) 0x56264b9d0bd0 con 0x56264b9f7b50 2023-10-05T19:41:27.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.192+0000 7efc217c1700 1 -- 172.21.15.12:0/2901777322 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56264ba8bc50 con 0x56264ba88320 2023-10-05T19:41:27.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.192+0000 7efc217c1700 1 --2- 172.21.15.12:0/2901777322 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56264ba88320 0x56264ba8a700 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00a520 tx=0x7efc1c005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e857a77e04298b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:27.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.192+0000 7efc20fc0700 1 -- 172.21.15.12:0/2901777322 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc1c00f590 con 0x56264ba88320 2023-10-05T19:41:27.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.192+0000 7efc20fc0700 1 -- 172.21.15.12:0/2901777322 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc1c00f6f0 con 0x56264ba88320 2023-10-05T19:41:27.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.192+0000 7efc20fc0700 1 -- 172.21.15.12:0/2901777322 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc1c00f880 con 0x56264ba88320 2023-10-05T19:41:27.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.192+0000 7efc2448de40 1 -- 172.21.15.12:0/2901777322 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56264ba88320 msgr2=0x56264ba8a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.192+0000 7efc2448de40 1 --2- 172.21.15.12:0/2901777322 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56264ba88320 0x56264ba8a700 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00a520 tx=0x7efc1c005d40 comp rx=0 tx=0).stop 2023-10-05T19:41:27.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.192+0000 7efc2448de40 1 -- 172.21.15.12:0/2901777322 shutdown_connections 2023-10-05T19:41:27.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.192+0000 7efc2448de40 1 --2- 172.21.15.12:0/2901777322 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56264ba88320 0x56264ba8a700 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:27.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.192+0000 7efc2448de40 1 -- 172.21.15.12:0/2901777322 >> 172.21.15.12:0/2901777322 conn(0x56264ba7e750 msgr2=0x56264ba80b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:27.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.192+0000 7efc2448de40 1 -- 172.21.15.12:0/2901777322 shutdown_connections 2023-10-05T19:41:27.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.192+0000 7efc2448de40 1 -- 172.21.15.12:0/2901777322 wait complete. 2023-10-05T19:41:27.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.196+0000 7efc2448de40 1 Processor -- start 2023-10-05T19:41:27.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.196+0000 7efc2448de40 1 -- start start 2023-10-05T19:41:27.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.196+0000 7efc2448de40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56264b9f7b50 0x56264ba84180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:27.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.196+0000 7efc2448de40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56264b951e30 con 0x56264b9f7b50 2023-10-05T19:41:27.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.196+0000 7efc21fc2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56264b9f7b50 0x56264ba84180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:27.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.196+0000 7efc21fc2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56264b9f7b50 0x56264ba84180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:50508/0 (socket says 172.21.15.12:50508) 2023-10-05T19:41:27.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.196+0000 7efc21fc2700 1 -- 172.21.15.12:0/2134536145 learned_addr learned my addr 172.21.15.12:0/2134536145 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:27.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.196+0000 7efc21fc2700 1 -- 172.21.15.12:0/2134536145 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc1c00a0e0 con 0x56264b9f7b50 2023-10-05T19:41:27.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.196+0000 7efc21fc2700 1 --2- 172.21.15.12:0/2134536145 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56264b9f7b50 0x56264ba84180 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7efc0c001dc0 tx=0x7efc0c001790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec04b7b0235fed1b server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:27.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.196+0000 7efc1b7fe700 1 -- 172.21.15.12:0/2134536145 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc0c00bce0 con 0x56264b9f7b50 2023-10-05T19:41:27.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.200+0000 7efc1b7fe700 1 -- 172.21.15.12:0/2134536145 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc0c00be40 con 0x56264b9f7b50 2023-10-05T19:41:27.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.200+0000 7efc1b7fe700 1 -- 172.21.15.12:0/2134536145 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc0c00e5d0 con 0x56264b9f7b50 2023-10-05T19:41:27.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.200+0000 7efc2448de40 1 -- 172.21.15.12:0/2134536145 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56264b9f7b50 msgr2=0x56264ba84180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.200+0000 7efc2448de40 1 --2- 172.21.15.12:0/2134536145 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56264b9f7b50 0x56264ba84180 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7efc0c001dc0 tx=0x7efc0c001790 comp rx=0 tx=0).stop 2023-10-05T19:41:27.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.200+0000 7efc2448de40 1 -- 172.21.15.12:0/2134536145 shutdown_connections 2023-10-05T19:41:27.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.200+0000 7efc2448de40 1 --2- 172.21.15.12:0/2134536145 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56264b9f7b50 0x56264ba84180 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:27.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.200+0000 7efc2448de40 1 -- 172.21.15.12:0/2134536145 >> 172.21.15.12:0/2134536145 conn(0x56264ba7e750 msgr2=0x56264ba9e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:27.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.200+0000 7efc2448de40 1 -- 172.21.15.12:0/2134536145 shutdown_connections 2023-10-05T19:41:27.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.200+0000 7efc2448de40 1 -- 172.21.15.12:0/2134536145 wait complete. 2023-10-05T19:41:27.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.200+0000 7efc2448de40 1 Processor -- start 2023-10-05T19:41:27.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.200+0000 7efc2448de40 1 -- start start 2023-10-05T19:41:27.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.200+0000 7efc2448de40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56264b9f7b50 0x56264baabd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:27.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.200+0000 7efc2448de40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56264b924ed0 con 0x56264b9f7b50 2023-10-05T19:41:27.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.200+0000 7efc21fc2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56264b9f7b50 0x56264baabd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:27.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.200+0000 7efc21fc2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56264b9f7b50 0x56264baabd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:50516/0 (socket says 172.21.15.12:50516) 2023-10-05T19:41:27.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.200+0000 7efc21fc2700 1 -- 172.21.15.12:0/2246347852 learned_addr learned my addr 172.21.15.12:0/2246347852 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:27.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.200+0000 7efc21fc2700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc0c001a70 con 0x56264b9f7b50 2023-10-05T19:41:27.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.204+0000 7efc21fc2700 1 --2- 172.21.15.12:0/2246347852 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56264b9f7b50 0x56264baabd20 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7efc0c0017d0 tx=0x7efc0c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:27.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.204+0000 7efc19ffb700 1 -- 172.21.15.12:0/2246347852 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc0c01f070 con 0x56264b9f7b50 2023-10-05T19:41:27.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.204+0000 7efc19ffb700 1 -- 172.21.15.12:0/2246347852 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc0c01a4d0 con 0x56264b9f7b50 2023-10-05T19:41:27.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.204+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56264baac260 con 0x56264b9f7b50 2023-10-05T19:41:27.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.204+0000 7efc19ffb700 1 -- 172.21.15.12:0/2246347852 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc0c01a7a0 con 0x56264b9f7b50 2023-10-05T19:41:27.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.204+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56264baac6a0 con 0x56264b9f7b50 2023-10-05T19:41:27.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.204+0000 7efc19ffb700 1 -- 172.21.15.12:0/2246347852 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7efc0c01a900 con 0x56264b9f7b50 2023-10-05T19:41:27.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.204+0000 7efc2448de40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.204+0000 7efc19ffb700 1 --2- 172.21.15.12:0/2246347852 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7efc08030c50 0x7efc08033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:27.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.204+0000 7efc19ffb700 1 -- 172.21.15.12:0/2246347852 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7efc0c045150 con 0x56264b9f7b50 2023-10-05T19:41:27.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.204+0000 7efc217c1700 1 --2- 172.21.15.12:0/2246347852 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7efc08030c50 0x7efc08033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:27.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.204+0000 7efc2448de40 1 --2- 172.21.15.12:0/2246347852 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x56264ba88320 0x56264bab77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:27.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.204+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bab7e20 con 0x56264ba88320 2023-10-05T19:41:27.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.208+0000 7efc227c3700 1 --2- 172.21.15.12:0/2246347852 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x56264ba88320 0x56264bab77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:27.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.208+0000 7efc217c1700 1 --2- 172.21.15.12:0/2246347852 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7efc08030c50 0x7efc08033100 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7efc1c005d80 tx=0x7efc1c014000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:27.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.208+0000 7efc227c3700 1 --2- 172.21.15.12:0/2246347852 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x56264ba88320 0x56264bab77b0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:27.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.208+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7efc1400a500 con 0x56264ba88320 2023-10-05T19:41:27.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.208+0000 7efc2448de40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.208+0000 7efc2448de40 20 realm 2023-10-05T19:41:27.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.208+0000 7efc2448de40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.208+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bab4e10 con 0x56264ba88320 2023-10-05T19:41:27.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.208+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7efc1400a500 con 0x56264ba88320 2023-10-05T19:41:27.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.208+0000 7efc2448de40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.208+0000 7efc2448de40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:41:27.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.208+0000 7efc2448de40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.208+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bab8900 con 0x56264ba88320 2023-10-05T19:41:27.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.208+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7efc1400a500 con 0x56264ba88320 2023-10-05T19:41:27.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.208+0000 7efc2448de40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.208+0000 7efc2448de40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.208+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bab84e0 con 0x56264ba88320 2023-10-05T19:41:27.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.208+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7efc1400a500 con 0x56264ba88320 2023-10-05T19:41:27.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.208+0000 7efc2448de40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:27.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.208+0000 7efc2448de40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.208+0000 7efc2448de40 1 --2- 172.21.15.12:0/2246347852 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x56264bab9890 0x56264babbc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:27.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.208+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264babc3e0 con 0x56264bab9890 2023-10-05T19:41:27.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.208+0000 7efc21fc2700 1 --2- 172.21.15.12:0/2246347852 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x56264bab9890 0x56264babbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:27.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.208+0000 7efc21fc2700 1 --2- 172.21.15.12:0/2246347852 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x56264bab9890 0x56264babbc70 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:27.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.212+0000 7efc21fc2700 1 -- 172.21.15.12:0/2246347852 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7efc0c014070 con 0x56264bab9890 2023-10-05T19:41:27.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.212+0000 7efc2448de40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:41:27.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.212+0000 7efc2448de40 20 searching for the correct realm 2023-10-05T19:41:27.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.212+0000 7efc2448de40 1 --2- 172.21.15.12:0/2246347852 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x56264babe210 0x56264bac05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:27.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.212+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac0b30 con 0x56264babe210 2023-10-05T19:41:27.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.212+0000 7efc217c1700 1 --2- 172.21.15.12:0/2246347852 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x56264babe210 0x56264bac05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:27.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.212+0000 7efc217c1700 1 --2- 172.21.15.12:0/2246347852 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x56264babe210 0x56264bac05f0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:27.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.216+0000 7efc217c1700 1 -- 172.21.15.12:0/2246347852 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7efc1c00fcc0 con 0x56264babe210 2023-10-05T19:41:27.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.216+0000 7efc217c1700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac0b30 con 0x56264bab9890 2023-10-05T19:41:27.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.216+0000 7efc21fc2700 1 -- 172.21.15.12:0/2246347852 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc0c014070 con 0x56264bab9890 2023-10-05T19:41:27.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.216+0000 7efc21fc2700 1 --2- 172.21.15.12:0/2246347852 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7efc0c016730 0x7efc0c042260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:27.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.216+0000 7efc21fc2700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac0b30 con 0x7efc0c016730 2023-10-05T19:41:27.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.216+0000 7efc227c3700 1 --2- 172.21.15.12:0/2246347852 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7efc0c016730 0x7efc0c042260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:27.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.216+0000 7efc227c3700 1 --2- 172.21.15.12:0/2246347852 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7efc0c016730 0x7efc0c042260 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:27.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.216+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc14014040 con 0x7efc0c016730 2023-10-05T19:41:27.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.216+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac0b30 con 0x56264ba88320 2023-10-05T19:41:27.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.216+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc14014040 con 0x56264ba88320 2023-10-05T19:41:27.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.216+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac0b30 con 0x56264ba88320 2023-10-05T19:41:27.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.220+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc14014040 con 0x56264ba88320 2023-10-05T19:41:27.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.220+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac0b30 con 0x56264ba88320 2023-10-05T19:41:27.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.220+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(11 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc14014040 con 0x56264ba88320 2023-10-05T19:41:27.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.220+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac0b30 con 0x7efc0c016730 2023-10-05T19:41:27.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.220+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc14014040 con 0x7efc0c016730 2023-10-05T19:41:27.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.220+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac0b30 con 0x56264bab9890 2023-10-05T19:41:27.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.220+0000 7efc21fc2700 1 -- 172.21.15.12:0/2246347852 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc0c014070 con 0x56264bab9890 2023-10-05T19:41:27.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.220+0000 7efc21fc2700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac0b30 con 0x56264bab9890 2023-10-05T19:41:27.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.220+0000 7efc21fc2700 1 -- 172.21.15.12:0/2246347852 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc0c014070 con 0x56264bab9890 2023-10-05T19:41:27.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.220+0000 7efc21fc2700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac0b30 con 0x56264ba88320 2023-10-05T19:41:27.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.224+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc14014040 con 0x56264ba88320 2023-10-05T19:41:27.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.224+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac0b30 con 0x7efc0c016730 2023-10-05T19:41:27.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.224+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc14014040 con 0x7efc0c016730 2023-10-05T19:41:27.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.224+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac0b30 con 0x56264ba88320 2023-10-05T19:41:27.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.224+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc14014040 con 0x56264ba88320 2023-10-05T19:41:27.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.224+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac0b30 con 0x56264bab9890 2023-10-05T19:41:27.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.224+0000 7efc21fc2700 1 -- 172.21.15.12:0/2246347852 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(18 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc0c014070 con 0x56264bab9890 2023-10-05T19:41:27.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.224+0000 7efc21fc2700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac0b30 con 0x56264babe210 2023-10-05T19:41:27.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.224+0000 7efc217c1700 1 -- 172.21.15.12:0/2246347852 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc1c00fcc0 con 0x56264babe210 2023-10-05T19:41:27.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.224+0000 7efc217c1700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac0b30 con 0x56264babe210 2023-10-05T19:41:27.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.224+0000 7efc217c1700 1 -- 172.21.15.12:0/2246347852 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc1c00fcc0 con 0x56264babe210 2023-10-05T19:41:27.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.224+0000 7efc217c1700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac0b30 con 0x7efc0c016730 2023-10-05T19:41:27.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.228+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc14014040 con 0x7efc0c016730 2023-10-05T19:41:27.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.228+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac0b30 con 0x56264babe210 2023-10-05T19:41:27.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.228+0000 7efc217c1700 1 -- 172.21.15.12:0/2246347852 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc1c00fcc0 con 0x56264babe210 2023-10-05T19:41:27.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.228+0000 7efc217c1700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac0b30 con 0x56264babe210 2023-10-05T19:41:27.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.228+0000 7efc217c1700 1 -- 172.21.15.12:0/2246347852 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc1c00fcc0 con 0x56264babe210 2023-10-05T19:41:27.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.228+0000 7efc217c1700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac0b30 con 0x56264ba88320 2023-10-05T19:41:27.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.228+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc14014040 con 0x56264ba88320 2023-10-05T19:41:27.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.228+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac0b30 con 0x56264ba88320 2023-10-05T19:41:27.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.228+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc14014040 con 0x56264ba88320 2023-10-05T19:41:27.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.228+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac0b30 con 0x7efc0c016730 2023-10-05T19:41:27.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.232+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc14014040 con 0x7efc0c016730 2023-10-05T19:41:27.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.232+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac0b30 con 0x56264babe210 2023-10-05T19:41:27.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.232+0000 7efc217c1700 1 -- 172.21.15.12:0/2246347852 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc1c00fcc0 con 0x56264babe210 2023-10-05T19:41:27.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.232+0000 7efc217c1700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac0b30 con 0x56264bab9890 2023-10-05T19:41:27.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.232+0000 7efc21fc2700 1 -- 172.21.15.12:0/2246347852 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc0c014070 con 0x56264bab9890 2023-10-05T19:41:27.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.232+0000 7efc21fc2700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac0b30 con 0x56264ba88320 2023-10-05T19:41:27.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.232+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc14014040 con 0x56264ba88320 2023-10-05T19:41:27.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.232+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac0b30 con 0x7efc0c016730 2023-10-05T19:41:27.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.232+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc14014040 con 0x7efc0c016730 2023-10-05T19:41:27.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.232+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac0b30 con 0x56264babe210 2023-10-05T19:41:27.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.236+0000 7efc217c1700 1 -- 172.21.15.12:0/2246347852 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc1c00fcc0 con 0x56264babe210 2023-10-05T19:41:27.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.236+0000 7efc217c1700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac0b30 con 0x7efc0c016730 2023-10-05T19:41:27.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.236+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc14014040 con 0x7efc0c016730 2023-10-05T19:41:27.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.236+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac0b30 con 0x56264bab9890 2023-10-05T19:41:27.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.236+0000 7efc21fc2700 1 -- 172.21.15.12:0/2246347852 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc0c014070 con 0x56264bab9890 2023-10-05T19:41:27.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.236+0000 7efc21fc2700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac0b30 con 0x56264babe210 2023-10-05T19:41:27.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.236+0000 7efc217c1700 1 -- 172.21.15.12:0/2246347852 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc1c00fcc0 con 0x56264babe210 2023-10-05T19:41:27.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.236+0000 7efc217c1700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac0b30 con 0x7efc0c016730 2023-10-05T19:41:27.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.236+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc14014040 con 0x7efc0c016730 2023-10-05T19:41:27.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.236+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac0b30 con 0x56264ba88320 2023-10-05T19:41:27.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.240+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7efc14014040 con 0x56264ba88320 2023-10-05T19:41:27.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.240+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac0b30 con 0x56264bab9890 2023-10-05T19:41:27.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.240+0000 7efc21fc2700 1 -- 172.21.15.12:0/2246347852 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(37 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7efc0c014070 con 0x56264bab9890 2023-10-05T19:41:27.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.240+0000 7efc2448de40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:41:27.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.240+0000 7efc2448de40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:41:27.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.240+0000 7efc2448de40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:41:27.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.240+0000 7efc2448de40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:41:27.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.240+0000 7efc2448de40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.240+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264babcf90 con 0x56264ba88320 2023-10-05T19:41:27.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.240+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7efc14014040 con 0x56264ba88320 2023-10-05T19:41:27.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.240+0000 7efc2448de40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.240+0000 7efc2448de40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.240+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264babd8e0 con 0x56264ba88320 2023-10-05T19:41:27.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.240+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7efc14014040 con 0x56264ba88320 2023-10-05T19:41:27.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.240+0000 7efc2448de40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:27.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.240+0000 7efc2448de40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.240+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264babcb50 con 0x56264babe210 2023-10-05T19:41:27.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.240+0000 7efc217c1700 1 -- 172.21.15.12:0/2246347852 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7efc1c00fcc0 con 0x56264babe210 2023-10-05T19:41:27.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.240+0000 7efc2448de40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:41:27.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.240+0000 7efc2448de40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.240+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac1460 con 0x56264babe210 2023-10-05T19:41:27.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.244+0000 7efc217c1700 1 -- 172.21.15.12:0/2246347852 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7efc1c00fcc0 con 0x56264babe210 2023-10-05T19:41:27.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.244+0000 7efc2448de40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.244+0000 7efc2448de40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.244+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac1810 con 0x56264ba88320 2023-10-05T19:41:27.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.244+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7efc14014040 con 0x56264ba88320 2023-10-05T19:41:27.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.244+0000 7efc2448de40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.244+0000 7efc2448de40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.244+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac1bc0 con 0x56264ba88320 2023-10-05T19:41:27.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.244+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7efc14014040 con 0x56264ba88320 2023-10-05T19:41:27.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.244+0000 7efc2448de40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:27.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.244+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac2bd0 con 0x56264babe210 2023-10-05T19:41:27.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.248+0000 7efc217c1700 1 -- 172.21.15.12:0/2246347852 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(44 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7efc1c00fcc0 con 0x56264babe210 2023-10-05T19:41:27.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.248+0000 7efc217c1700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac2bd0 con 0x56264bab9890 2023-10-05T19:41:27.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.248+0000 7efc21fc2700 1 -- 172.21.15.12:0/2246347852 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(45 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc0c014070 con 0x56264bab9890 2023-10-05T19:41:27.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.248+0000 7efc21fc2700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac2bd0 con 0x7efc0c016730 2023-10-05T19:41:27.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.248+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(46 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc14014040 con 0x7efc0c016730 2023-10-05T19:41:27.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.248+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac1810 con 0x56264ba88320 2023-10-05T19:41:27.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.248+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(47 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc14014040 con 0x56264ba88320 2023-10-05T19:41:27.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.248+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac1810 con 0x56264ba88320 2023-10-05T19:41:27.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.248+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(48 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc14014040 con 0x56264ba88320 2023-10-05T19:41:27.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.248+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac1810 con 0x56264ba88320 2023-10-05T19:41:27.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.252+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(49 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc14014040 con 0x56264ba88320 2023-10-05T19:41:27.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.252+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac1810 con 0x7efc0c016730 2023-10-05T19:41:27.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.252+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(50 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc14014040 con 0x7efc0c016730 2023-10-05T19:41:27.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.252+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac1810 con 0x56264bab9890 2023-10-05T19:41:27.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.252+0000 7efc21fc2700 1 -- 172.21.15.12:0/2246347852 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(51 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc0c014070 con 0x56264bab9890 2023-10-05T19:41:27.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.252+0000 7efc21fc2700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac1810 con 0x56264bab9890 2023-10-05T19:41:27.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.252+0000 7efc21fc2700 1 -- 172.21.15.12:0/2246347852 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(52 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc0c014070 con 0x56264bab9890 2023-10-05T19:41:27.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.252+0000 7efc21fc2700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac1810 con 0x56264ba88320 2023-10-05T19:41:27.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.252+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(53 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc14014040 con 0x56264ba88320 2023-10-05T19:41:27.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.252+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac1810 con 0x7efc0c016730 2023-10-05T19:41:27.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.256+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(54 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc14014040 con 0x7efc0c016730 2023-10-05T19:41:27.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.256+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac1810 con 0x56264ba88320 2023-10-05T19:41:27.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.256+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(55 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc14014040 con 0x56264ba88320 2023-10-05T19:41:27.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.256+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac1810 con 0x56264bab9890 2023-10-05T19:41:27.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.256+0000 7efc21fc2700 1 -- 172.21.15.12:0/2246347852 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(56 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc0c014070 con 0x56264bab9890 2023-10-05T19:41:27.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.256+0000 7efc21fc2700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac1810 con 0x56264babe210 2023-10-05T19:41:27.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.256+0000 7efc217c1700 1 -- 172.21.15.12:0/2246347852 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(57 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc1c00fcc0 con 0x56264babe210 2023-10-05T19:41:27.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.256+0000 7efc217c1700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac1810 con 0x56264babe210 2023-10-05T19:41:27.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.256+0000 7efc217c1700 1 -- 172.21.15.12:0/2246347852 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(58 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc1c00fcc0 con 0x56264babe210 2023-10-05T19:41:27.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.256+0000 7efc217c1700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac1810 con 0x7efc0c016730 2023-10-05T19:41:27.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.260+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(59 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc14014040 con 0x7efc0c016730 2023-10-05T19:41:27.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.260+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac1810 con 0x56264babe210 2023-10-05T19:41:27.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.260+0000 7efc217c1700 1 -- 172.21.15.12:0/2246347852 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(60 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc1c00fcc0 con 0x56264babe210 2023-10-05T19:41:27.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.260+0000 7efc217c1700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac1810 con 0x56264babe210 2023-10-05T19:41:27.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.260+0000 7efc217c1700 1 -- 172.21.15.12:0/2246347852 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(61 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc1c00fcc0 con 0x56264babe210 2023-10-05T19:41:27.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.260+0000 7efc217c1700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac1810 con 0x56264ba88320 2023-10-05T19:41:27.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.260+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(62 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc14014040 con 0x56264ba88320 2023-10-05T19:41:27.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.260+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac1810 con 0x56264ba88320 2023-10-05T19:41:27.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.264+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(63 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc14014040 con 0x56264ba88320 2023-10-05T19:41:27.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.264+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac1810 con 0x7efc0c016730 2023-10-05T19:41:27.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.264+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(64 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc14014040 con 0x7efc0c016730 2023-10-05T19:41:27.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.264+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac1810 con 0x56264babe210 2023-10-05T19:41:27.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.264+0000 7efc217c1700 1 -- 172.21.15.12:0/2246347852 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(65 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc1c00fcc0 con 0x56264babe210 2023-10-05T19:41:27.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.264+0000 7efc217c1700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac1810 con 0x56264bab9890 2023-10-05T19:41:27.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.264+0000 7efc21fc2700 1 -- 172.21.15.12:0/2246347852 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(66 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc0c014070 con 0x56264bab9890 2023-10-05T19:41:27.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.264+0000 7efc21fc2700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac1810 con 0x56264ba88320 2023-10-05T19:41:27.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.268+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(67 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc14014040 con 0x56264ba88320 2023-10-05T19:41:27.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.268+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac1810 con 0x7efc0c016730 2023-10-05T19:41:27.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.268+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(68 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc14014040 con 0x7efc0c016730 2023-10-05T19:41:27.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.268+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac1810 con 0x56264babe210 2023-10-05T19:41:27.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.268+0000 7efc217c1700 1 -- 172.21.15.12:0/2246347852 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(69 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc1c00fcc0 con 0x56264babe210 2023-10-05T19:41:27.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.268+0000 7efc217c1700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac1810 con 0x7efc0c016730 2023-10-05T19:41:27.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.268+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(70 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc14014040 con 0x7efc0c016730 2023-10-05T19:41:27.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.268+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac1810 con 0x56264bab9890 2023-10-05T19:41:27.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.272+0000 7efc21fc2700 1 -- 172.21.15.12:0/2246347852 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(71 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc0c014070 con 0x56264bab9890 2023-10-05T19:41:27.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.272+0000 7efc21fc2700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac1810 con 0x56264babe210 2023-10-05T19:41:27.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.272+0000 7efc217c1700 1 -- 172.21.15.12:0/2246347852 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(72 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc1c00fcc0 con 0x56264babe210 2023-10-05T19:41:27.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.272+0000 7efc217c1700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac1810 con 0x7efc0c016730 2023-10-05T19:41:27.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.272+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(73 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc14014040 con 0x7efc0c016730 2023-10-05T19:41:27.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.272+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac1810 con 0x56264ba88320 2023-10-05T19:41:27.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.272+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(74 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7efc14014040 con 0x56264ba88320 2023-10-05T19:41:27.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.272+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac1810 con 0x56264bab9890 2023-10-05T19:41:27.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.272+0000 7efc21fc2700 1 -- 172.21.15.12:0/2246347852 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(75 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7efc0c014070 con 0x56264bab9890 2023-10-05T19:41:27.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.272+0000 7efc2448de40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:41:27.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.272+0000 7efc2448de40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:41:27.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.272+0000 7efc2448de40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:41:27.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.272+0000 7efc2448de40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:41:27.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.272+0000 7efc2448de40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.272+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac1bc0 con 0x56264ba88320 2023-10-05T19:41:27.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.276+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7efc14014040 con 0x56264ba88320 2023-10-05T19:41:27.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.276+0000 7efc2448de40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:27.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.276+0000 7efc2448de40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.276+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac1f70 con 0x56264bab9890 2023-10-05T19:41:27.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.276+0000 7efc21fc2700 1 -- 172.21.15.12:0/2246347852 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7efc0c014070 con 0x56264bab9890 2023-10-05T19:41:27.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.276+0000 7efc2448de40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:41:27.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.276+0000 7efc2448de40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.276+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac1090 con 0x56264ba88320 2023-10-05T19:41:27.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.276+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7efc14014040 con 0x56264ba88320 2023-10-05T19:41:27.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.276+0000 7efc2448de40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:27.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.276+0000 7efc2448de40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.276+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac10a0 con 0x56264babe210 2023-10-05T19:41:27.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.276+0000 7efc217c1700 1 -- 172.21.15.12:0/2246347852 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7efc1c00fcc0 con 0x56264babe210 2023-10-05T19:41:27.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.276+0000 7efc2448de40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:41:27.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.276+0000 7efc2448de40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.276+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac2e10 con 0x7efc0c016730 2023-10-05T19:41:27.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.280+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7efc14014040 con 0x7efc0c016730 2023-10-05T19:41:27.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.280+0000 7efc2448de40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.280+0000 7efc2448de40 20 zone default found 2023-10-05T19:41:27.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.280+0000 7efc2448de40 4 Realm: () 2023-10-05T19:41:27.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.280+0000 7efc2448de40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:41:27.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.280+0000 7efc2448de40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:41:27.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.280+0000 7efc2448de40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:41:27.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.280+0000 7efc2448de40 20 zonegroup default 2023-10-05T19:41:27.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.280+0000 7efc2448de40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.280+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac2e10 con 0x7efc0c016730 2023-10-05T19:41:27.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.280+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7efc14014040 con 0x7efc0c016730 2023-10-05T19:41:27.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.280+0000 7efc2448de40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.280+0000 7efc2448de40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.280+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac4790 con 0x56264babe210 2023-10-05T19:41:27.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.280+0000 7efc217c1700 1 -- 172.21.15.12:0/2246347852 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7efc1c00fcc0 con 0x56264babe210 2023-10-05T19:41:27.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.280+0000 7efc2448de40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.280+0000 7efc2448de40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.280+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac4b40 con 0x56264babe210 2023-10-05T19:41:27.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.280+0000 7efc217c1700 1 -- 172.21.15.12:0/2246347852 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7efc1c00fcc0 con 0x56264babe210 2023-10-05T19:41:27.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.280+0000 7efc2448de40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.280+0000 7efc2448de40 20 started sync module instance, tier type = 2023-10-05T19:41:27.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.280+0000 7efc2448de40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:41:27.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.280+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac3e50 con 0x56264bab9890 2023-10-05T19:41:27.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.284+0000 7efc21fc2700 1 -- 172.21.15.12:0/2246347852 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(84 notify.0 [create] v30'228 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc0c014070 con 0x56264bab9890 2023-10-05T19:41:27.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.284+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94722478324064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac4ef0 con 0x56264bab9890 2023-10-05T19:41:27.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.284+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac5790 con 0x56264ba88320 2023-10-05T19:41:27.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.288+0000 7efc21fc2700 1 -- 172.21.15.12:0/2246347852 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(85 notify.0 [watch watch cookie 94722478324064] v30'229 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc0c014070 con 0x56264bab9890 2023-10-05T19:41:27.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.288+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(86 notify.1 [create] v30'228 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc14014040 con 0x56264ba88320 2023-10-05T19:41:27.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.288+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94722478330528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac43b0 con 0x56264ba88320 2023-10-05T19:41:27.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.288+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac6570 con 0x56264babe210 2023-10-05T19:41:27.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.292+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(87 notify.1 [watch watch cookie 94722478330528] v30'229 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc14014040 con 0x56264ba88320 2023-10-05T19:41:27.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.292+0000 7efc217c1700 1 -- 172.21.15.12:0/2246347852 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(88 notify.2 [create] v30'228 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc1c00fcc0 con 0x56264babe210 2023-10-05T19:41:27.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.292+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94722478332816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac6920 con 0x56264babe210 2023-10-05T19:41:27.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.292+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac71c0 con 0x56264bab9890 2023-10-05T19:41:27.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.296+0000 7efc21fc2700 1 -- 172.21.15.12:0/2246347852 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(90 notify.3 [create] v30'228 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc0c014070 con 0x56264bab9890 2023-10-05T19:41:27.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.296+0000 7efc217c1700 1 -- 172.21.15.12:0/2246347852 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(89 notify.2 [watch watch cookie 94722478332816] v30'229 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc1c00fcc0 con 0x56264babe210 2023-10-05T19:41:27.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.296+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94722478339440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac6df0 con 0x56264bab9890 2023-10-05T19:41:27.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.296+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac6df0 con 0x56264ba88320 2023-10-05T19:41:27.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.296+0000 7efc21fc2700 1 -- 172.21.15.12:0/2246347852 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(91 notify.3 [watch watch cookie 94722478339440] v30'229 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc0c014070 con 0x56264bab9890 2023-10-05T19:41:27.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.296+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(92 notify.4 [create] v30'228 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc14014040 con 0x56264ba88320 2023-10-05T19:41:27.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.296+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94722478340544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac8130 con 0x56264ba88320 2023-10-05T19:41:27.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.296+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac89d0 con 0x7efc0c016730 2023-10-05T19:41:27.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.300+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(94 notify.5 [create] v30'228 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc14014040 con 0x7efc0c016730 2023-10-05T19:41:27.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.300+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94722478343008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac8dd0 con 0x7efc0c016730 2023-10-05T19:41:27.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.300+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac9900 con 0x7efc0c016730 2023-10-05T19:41:27.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.300+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(93 notify.4 [watch watch cookie 94722478340544] v30'229 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc14014040 con 0x56264ba88320 2023-10-05T19:41:27.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.304+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(95 notify.5 [watch watch cookie 94722478343008] v30'229 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc14014040 con 0x7efc0c016730 2023-10-05T19:41:27.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.304+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(96 notify.6 [create] v30'228 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc14014040 con 0x7efc0c016730 2023-10-05T19:41:27.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.304+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94722478333920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac93d0 con 0x7efc0c016730 2023-10-05T19:41:27.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.304+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac9de0 con 0x7efc0c016730 2023-10-05T19:41:27.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.308+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(97 notify.6 [watch watch cookie 94722478333920] v30'229 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc14014040 con 0x7efc0c016730 2023-10-05T19:41:27.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.308+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(98 notify.7 [create] v30'228 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc14014040 con 0x7efc0c016730 2023-10-05T19:41:27.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.308+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94722478344112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac9de0 con 0x7efc0c016730 2023-10-05T19:41:27.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.308+0000 7efc2448de40 20 add_watcher() i=0 2023-10-05T19:41:27.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.308+0000 7efc2448de40 20 add_watcher() i=1 2023-10-05T19:41:27.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.308+0000 7efc2448de40 20 add_watcher() i=2 2023-10-05T19:41:27.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.308+0000 7efc2448de40 20 add_watcher() i=3 2023-10-05T19:41:27.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.308+0000 7efc2448de40 20 add_watcher() i=4 2023-10-05T19:41:27.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.308+0000 7efc2448de40 20 add_watcher() i=5 2023-10-05T19:41:27.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.308+0000 7efc2448de40 20 add_watcher() i=6 2023-10-05T19:41:27.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.308+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(99 notify.7 [watch watch cookie 94722478344112] v30'229 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc14014040 con 0x7efc0c016730 2023-10-05T19:41:27.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.308+0000 7efc2448de40 20 add_watcher() i=7 2023-10-05T19:41:27.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.308+0000 7efc2448de40 2 all 8 watchers are set, enabling cache 2023-10-05T19:41:27.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.308+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264baccf80 con 0x56264ba88320 2023-10-05T19:41:27.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.308+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7efc14014040 con 0x56264ba88320 2023-10-05T19:41:27.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.308+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94722478348896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264baca4c0 con 0x56264ba88320 2023-10-05T19:41:27.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.312+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94722478348896] v30'223 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7efc14014040 con 0x56264ba88320 2023-10-05T19:41:27.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.312+0000 7efbf27ec700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:41:27.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.312+0000 7efc2448de40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:41:27.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.312+0000 7efc2448de40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:41:27.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.312+0000 7efc2448de40 1 Processor -- start 2023-10-05T19:41:27.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.312+0000 7efc2448de40 1 -- start start 2023-10-05T19:41:27.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.312+0000 7efc2448de40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56264bad7400 0x56264bad77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:27.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.312+0000 7efc2448de40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56264bad7d10 con 0x56264bad7400 2023-10-05T19:41:27.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.316+0000 7efc217c1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56264bad7400 0x56264bad77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:27.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.316+0000 7efc217c1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56264bad7400 0x56264bad77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:50520/0 (socket says 172.21.15.12:50520) 2023-10-05T19:41:27.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.316+0000 7efc217c1700 1 -- 172.21.15.12:0/2541210949 learned_addr learned my addr 172.21.15.12:0/2541210949 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:27.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.316+0000 7efc217c1700 1 -- 172.21.15.12:0/2541210949 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56264bad7e50 con 0x56264bad7400 2023-10-05T19:41:27.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.316+0000 7efc217c1700 1 --2- 172.21.15.12:0/2541210949 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56264bad7400 0x56264bad77d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x56264b9f73e0 tx=0x7efc1c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46366701227097e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:27.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.316+0000 7efbf1feb700 1 -- 172.21.15.12:0/2541210949 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc1c01ccf0 con 0x56264bad7400 2023-10-05T19:41:27.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.316+0000 7efbf1feb700 1 -- 172.21.15.12:0/2541210949 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc1c01ce50 con 0x56264bad7400 2023-10-05T19:41:27.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.316+0000 7efbf1feb700 1 -- 172.21.15.12:0/2541210949 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc1c044440 con 0x56264bad7400 2023-10-05T19:41:27.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.316+0000 7efc2448de40 1 -- 172.21.15.12:0/2541210949 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56264bad7400 msgr2=0x56264bad77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.316+0000 7efc2448de40 1 --2- 172.21.15.12:0/2541210949 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56264bad7400 0x56264bad77d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x56264b9f73e0 tx=0x7efc1c011000 comp rx=0 tx=0).stop 2023-10-05T19:41:27.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.316+0000 7efc2448de40 1 -- 172.21.15.12:0/2541210949 shutdown_connections 2023-10-05T19:41:27.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.316+0000 7efc2448de40 1 --2- 172.21.15.12:0/2541210949 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56264bad7400 0x56264bad77d0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:27.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.316+0000 7efc2448de40 1 -- 172.21.15.12:0/2541210949 >> 172.21.15.12:0/2541210949 conn(0x56264bad27c0 msgr2=0x56264bad0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:27.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.316+0000 7efc2448de40 1 -- 172.21.15.12:0/2541210949 shutdown_connections 2023-10-05T19:41:27.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.316+0000 7efc2448de40 1 -- 172.21.15.12:0/2541210949 wait complete. 2023-10-05T19:41:27.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.316+0000 7efc2448de40 1 Processor -- start 2023-10-05T19:41:27.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.316+0000 7efc2448de40 1 -- start start 2023-10-05T19:41:27.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.320+0000 7efc2448de40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56264bad7400 0x56264badcf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:27.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.320+0000 7efc2448de40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56264baddf00 con 0x56264bad7400 2023-10-05T19:41:27.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.320+0000 7efc217c1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56264bad7400 0x56264badcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:27.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.320+0000 7efc217c1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56264bad7400 0x56264badcf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:50524/0 (socket says 172.21.15.12:50524) 2023-10-05T19:41:27.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.320+0000 7efc217c1700 1 -- 172.21.15.12:0/753893938 learned_addr learned my addr 172.21.15.12:0/753893938 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:27.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.320+0000 7efc217c1700 1 -- 172.21.15.12:0/753893938 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc1c02e040 con 0x56264bad7400 2023-10-05T19:41:27.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.320+0000 7efc217c1700 1 --2- 172.21.15.12:0/753893938 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56264bad7400 0x56264badcf00 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00fb80 tx=0x7efc1c048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:27.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.320+0000 7efb63fff700 1 -- 172.21.15.12:0/753893938 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc1c04f480 con 0x56264bad7400 2023-10-05T19:41:27.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.320+0000 7efb63fff700 1 -- 172.21.15.12:0/753893938 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc1c04f5e0 con 0x56264bad7400 2023-10-05T19:41:27.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.320+0000 7efc2448de40 1 -- 172.21.15.12:0/753893938 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56264badd500 con 0x56264bad7400 2023-10-05T19:41:27.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.320+0000 7efb63fff700 1 -- 172.21.15.12:0/753893938 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc1c04f770 con 0x56264bad7400 2023-10-05T19:41:27.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.320+0000 7efc2448de40 1 -- 172.21.15.12:0/753893938 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56264badd940 con 0x56264bad7400 2023-10-05T19:41:27.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.320+0000 7efb63fff700 1 -- 172.21.15.12:0/753893938 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7efc1c04f8d0 con 0x56264bad7400 2023-10-05T19:41:27.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.324+0000 7efb63fff700 1 --2- 172.21.15.12:0/753893938 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7efb68030ca0 0x7efb68033150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:27.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.324+0000 7efb63fff700 1 -- 172.21.15.12:0/753893938 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7efc1c059c30 con 0x56264bad7400 2023-10-05T19:41:27.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.324+0000 7efc227c3700 1 --2- 172.21.15.12:0/753893938 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7efb68030ca0 0x7efb68033150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:27.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.324+0000 7efc2448de40 5 note: GC not initialized 2023-10-05T19:41:27.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.324+0000 7efc227c3700 1 --2- 172.21.15.12:0/753893938 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7efb68030ca0 0x7efb68033150 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x56264baca420 tx=0x7efc1400ab60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:27.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.324+0000 7efb617fa700 20 reqs_thread_entry: start 2023-10-05T19:41:27.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.324+0000 7efb5affd700 20 reqs_thread_entry: start 2023-10-05T19:41:27.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.324+0000 7efc2448de40 20 init_complete bucket index max shards: 11 2023-10-05T19:41:27.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.328+0000 7efc2448de40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:41:27.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.328+0000 7efb597fa700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:41:57 2023 2023-10-05T19:41:27.332 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:41:27.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.328+0000 7efb58ff9700 20 reqs_thread_entry: start 2023-10-05T19:41:27.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.328+0000 7efc2448de40 20 get_system_obj_state: rctx=0x7ffce8e6b108 obj=default.rgw.meta:users.uid:foo state=0x56264b95d0b0 s->prefetch_data=0 2023-10-05T19:41:27.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.328+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bb45650 con 0x56264ba88320 2023-10-05T19:41:27.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.328+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(102 foo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv15 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7efc14014040 con 0x56264ba88320 2023-10-05T19:41:27.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.328+0000 7efc2448de40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:41:27.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.328+0000 7efc2448de40 20 Read xattr: user.rgw.idtag 2023-10-05T19:41:27.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.328+0000 7efc2448de40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.328+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bb45640 con 0x56264ba88320 2023-10-05T19:41:27.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.328+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(103 foo [call,call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv15 ondisk = 0) v8 ==== 273+0+480 (crc 0 0 0) 0x7efc14014040 con 0x56264ba88320 2023-10-05T19:41:27.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.328+0000 7efc2448de40 20 rados_obj.operate() r=0 bl.length=357 2023-10-05T19:41:27.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.328+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:104 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bb470d0 con 0x7efc0c016730 2023-10-05T19:41:27.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.328+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(104 usage.10 [call out=1268b] v0'0 uv32 ondisk = 0) v8 ==== 152+0+1268 (crc 0 0 0) 0x7efc14014040 con 0x7efc0c016730 2023-10-05T19:41:27.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.332+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:105 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94722478324064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bb03c00 con 0x56264bab9890 2023-10-05T19:41:27.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.332+0000 7efc21fc2700 1 -- 172.21.15.12:0/2246347852 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(105 notify.0 [watch unwatch cookie 94722478324064] v30'230 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc0c014070 con 0x56264bab9890 2023-10-05T19:41:27.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.332+0000 7efc2448de40 20 remove_watcher() i=0 2023-10-05T19:41:27.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.332+0000 7efc2448de40 2 removed watcher, disabling cache 2023-10-05T19:41:27.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.332+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:106 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94722478330528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264ba995e0 con 0x56264ba88320 2023-10-05T19:41:27.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.336+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(106 notify.1 [watch unwatch cookie 94722478330528] v30'230 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc14014040 con 0x56264ba88320 2023-10-05T19:41:27.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.336+0000 7efc2448de40 20 remove_watcher() i=1 2023-10-05T19:41:27.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.336+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:107 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94722478332816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac52a0 con 0x56264babe210 2023-10-05T19:41:27.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.340+0000 7efc217c1700 1 -- 172.21.15.12:0/2246347852 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(107 notify.2 [watch unwatch cookie 94722478332816] v30'230 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc1c00fcc0 con 0x56264babe210 2023-10-05T19:41:27.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.340+0000 7efc2448de40 20 remove_watcher() i=2 2023-10-05T19:41:27.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.340+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:108 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94722478339440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac5b90 con 0x56264bab9890 2023-10-05T19:41:27.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.344+0000 7efc21fc2700 1 -- 172.21.15.12:0/2246347852 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(108 notify.3 [watch unwatch cookie 94722478339440] v30'230 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc0c014070 con 0x56264bab9890 2023-10-05T19:41:27.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.344+0000 7efc2448de40 20 remove_watcher() i=3 2023-10-05T19:41:27.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.344+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:109 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94722478340544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac52a0 con 0x56264ba88320 2023-10-05T19:41:27.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.344+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== osd_op_reply(109 notify.4 [watch unwatch cookie 94722478340544] v30'230 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc14014040 con 0x56264ba88320 2023-10-05T19:41:27.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.344+0000 7efc2448de40 20 remove_watcher() i=4 2023-10-05T19:41:27.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.344+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:110 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94722478343008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac7a60 con 0x7efc0c016730 2023-10-05T19:41:27.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.348+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(110 notify.5 [watch unwatch cookie 94722478343008] v30'230 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc14014040 con 0x7efc0c016730 2023-10-05T19:41:27.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.348+0000 7efc2448de40 20 remove_watcher() i=5 2023-10-05T19:41:27.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.348+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:111 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94722478333920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac7a60 con 0x7efc0c016730 2023-10-05T19:41:27.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.352+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(111 notify.6 [watch unwatch cookie 94722478333920] v30'230 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc14014040 con 0x7efc0c016730 2023-10-05T19:41:27.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.352+0000 7efc2448de40 20 remove_watcher() i=6 2023-10-05T19:41:27.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.352+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:112 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94722478344112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac7a60 con 0x7efc0c016730 2023-10-05T19:41:27.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.356+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(112 notify.7 [watch unwatch cookie 94722478344112] v30'230 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc14014040 con 0x7efc0c016730 2023-10-05T19:41:27.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.356+0000 7efc2448de40 20 remove_watcher() i=7 2023-10-05T19:41:27.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.356+0000 7efbf27ec700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:41:27.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.356+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:113 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94722478348896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56264bac6080 con 0x56264ba88320 2023-10-05T19:41:27.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.360+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 <== osd.1 v2:172.21.15.12:6808/29933 39 ==== osd_op_reply(113 data_loggenerations_metadata [watch unwatch cookie 94722478348896] v30'224 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7efc14014040 con 0x56264ba88320 2023-10-05T19:41:27.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.360+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7efc0c016730 msgr2=0x7efc0c042260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.360+0000 7efc2448de40 1 --2- 172.21.15.12:0/2246347852 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7efc0c016730 0x7efc0c042260 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:27.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.360+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x56264ba88320 msgr2=0x56264bab77b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.360+0000 7efc2448de40 1 --2- 172.21.15.12:0/2246347852 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x56264ba88320 0x56264bab77b0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:27.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.360+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x56264bab9890 msgr2=0x56264babbc70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.360+0000 7efc2448de40 1 --2- 172.21.15.12:0/2246347852 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x56264bab9890 0x56264babbc70 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:27.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.360+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x56264babe210 msgr2=0x56264bac05f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.360+0000 7efc2448de40 1 --2- 172.21.15.12:0/2246347852 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x56264babe210 0x56264bac05f0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:27.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.360+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7efc08030c50 msgr2=0x7efc08033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.360+0000 7efc2448de40 1 --2- 172.21.15.12:0/2246347852 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7efc08030c50 0x7efc08033100 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7efc1c005d80 tx=0x7efc1c014000 comp rx=0 tx=0).stop 2023-10-05T19:41:27.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.360+0000 7efc227c3700 1 -- 172.21.15.12:0/2246347852 reap_dead start 2023-10-05T19:41:27.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.360+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56264b9f7b50 msgr2=0x56264baabd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.360+0000 7efc2448de40 1 --2- 172.21.15.12:0/2246347852 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56264b9f7b50 0x56264baabd20 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7efc0c0017d0 tx=0x7efc0c016000 comp rx=0 tx=0).stop 2023-10-05T19:41:27.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.360+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 shutdown_connections 2023-10-05T19:41:27.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.360+0000 7efc2448de40 1 --2- 172.21.15.12:0/2246347852 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56264b9f7b50 0x56264baabd20 secure :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7efc0c0017d0 tx=0x7efc0c016000 comp rx=0 tx=0).stop 2023-10-05T19:41:27.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.360+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 >> 172.21.15.12:0/2246347852 conn(0x56264ba7e750 msgr2=0x56264ba9d340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:27.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.360+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 shutdown_connections 2023-10-05T19:41:27.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.360+0000 7efc2448de40 1 -- 172.21.15.12:0/2246347852 wait complete. 2023-10-05T19:41:27.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.364+0000 7efc2448de40 1 -- 172.21.15.12:0/753893938 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7efb68030ca0 msgr2=0x7efb68033150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.364+0000 7efc2448de40 1 --2- 172.21.15.12:0/753893938 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7efb68030ca0 0x7efb68033150 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x56264baca420 tx=0x7efc1400ab60 comp rx=0 tx=0).stop 2023-10-05T19:41:27.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.364+0000 7efc2448de40 1 -- 172.21.15.12:0/753893938 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56264bad7400 msgr2=0x56264badcf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.364+0000 7efc2448de40 1 --2- 172.21.15.12:0/753893938 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56264bad7400 0x56264badcf00 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00fb80 tx=0x7efc1c048000 comp rx=0 tx=0).stop 2023-10-05T19:41:27.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.364+0000 7efc2448de40 1 -- 172.21.15.12:0/753893938 shutdown_connections 2023-10-05T19:41:27.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.364+0000 7efc2448de40 1 --2- 172.21.15.12:0/753893938 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7efb68030ca0 0x7efb68033150 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:27.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.364+0000 7efc2448de40 1 --2- 172.21.15.12:0/753893938 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56264bad7400 0x56264badcf00 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:27.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.364+0000 7efc2448de40 1 -- 172.21.15.12:0/753893938 >> 172.21.15.12:0/753893938 conn(0x56264bad27c0 msgr2=0x56264bad0880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:27.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.364+0000 7efc2448de40 1 -- 172.21.15.12:0/753893938 shutdown_connections 2023-10-05T19:41:27.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.364+0000 7efc2448de40 1 -- 172.21.15.12:0/753893938 wait complete. 2023-10-05T19:41:27.371 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:41:27.372 INFO:teuthology.orchestra.run.smithi012.stdout: "entries": [ 2023-10-05T19:41:27.372 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.372 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:41:27.372 INFO:teuthology.orchestra.run.smithi012.stdout: "buckets": [ 2023-10-05T19:41:27.372 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.372 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "", 2023-10-05T19:41:27.372 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:00:00.000000Z", 2023-10-05T19:41:27.372 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 1696532400, 2023-10-05T19:41:27.372 INFO:teuthology.orchestra.run.smithi012.stdout: "owner": "foo", 2023-10-05T19:41:27.372 INFO:teuthology.orchestra.run.smithi012.stdout: "categories": [] 2023-10-05T19:41:27.372 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:27.372 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.373 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo", 2023-10-05T19:41:27.373 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:00:00.000000Z", 2023-10-05T19:41:27.373 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 1696532400, 2023-10-05T19:41:27.373 INFO:teuthology.orchestra.run.smithi012.stdout: "owner": "foo", 2023-10-05T19:41:27.373 INFO:teuthology.orchestra.run.smithi012.stdout: "categories": [ 2023-10-05T19:41:27.373 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.373 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "create_bucket", 2023-10-05T19:41:27.373 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:27.373 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:27.373 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 5, 2023-10-05T19:41:27.373 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 5 2023-10-05T19:41:27.374 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:27.374 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2023-10-05T19:41:27.374 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:27.374 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.374 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo2", 2023-10-05T19:41:27.374 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:00:00.000000Z", 2023-10-05T19:41:27.374 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 1696532400, 2023-10-05T19:41:27.374 INFO:teuthology.orchestra.run.smithi012.stdout: "owner": "foo", 2023-10-05T19:41:27.374 INFO:teuthology.orchestra.run.smithi012.stdout: "categories": [ 2023-10-05T19:41:27.374 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.374 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "create_bucket", 2023-10-05T19:41:27.375 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:27.375 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:27.375 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 1, 2023-10-05T19:41:27.375 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 1 2023-10-05T19:41:27.375 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:27.375 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2023-10-05T19:41:27.375 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:27.375 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.375 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo3", 2023-10-05T19:41:27.375 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:00:00.000000Z", 2023-10-05T19:41:27.375 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 1696532400, 2023-10-05T19:41:27.375 INFO:teuthology.orchestra.run.smithi012.stdout: "owner": "foo", 2023-10-05T19:41:27.376 INFO:teuthology.orchestra.run.smithi012.stdout: "categories": [ 2023-10-05T19:41:27.376 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.376 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "create_bucket", 2023-10-05T19:41:27.376 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:27.376 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:27.376 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 1, 2023-10-05T19:41:27.376 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 1 2023-10-05T19:41:27.376 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:27.376 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2023-10-05T19:41:27.376 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:27.376 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.376 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo4", 2023-10-05T19:41:27.377 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:00:00.000000Z", 2023-10-05T19:41:27.377 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 1696532400, 2023-10-05T19:41:27.377 INFO:teuthology.orchestra.run.smithi012.stdout: "owner": "foo", 2023-10-05T19:41:27.377 INFO:teuthology.orchestra.run.smithi012.stdout: "categories": [ 2023-10-05T19:41:27.377 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.377 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "create_bucket", 2023-10-05T19:41:27.377 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:27.377 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:27.377 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 1, 2023-10-05T19:41:27.377 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 1 2023-10-05T19:41:27.377 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:27.377 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2023-10-05T19:41:27.378 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:27.378 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2023-10-05T19:41:27.378 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:27.378 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:41:27.378 INFO:teuthology.orchestra.run.smithi012.stdout: "summary": [ 2023-10-05T19:41:27.378 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.378 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:41:27.378 INFO:teuthology.orchestra.run.smithi012.stdout: "categories": [ 2023-10-05T19:41:27.378 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.378 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "create_bucket", 2023-10-05T19:41:27.378 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:27.379 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:27.379 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 8, 2023-10-05T19:41:27.379 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 8 2023-10-05T19:41:27.379 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:27.379 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:41:27.379 INFO:teuthology.orchestra.run.smithi012.stdout: "total": { 2023-10-05T19:41:27.379 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:27.379 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:27.379 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 8, 2023-10-05T19:41:27.379 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 8 2023-10-05T19:41:27.379 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:27.379 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:27.380 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2023-10-05T19:41:27.380 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:41:27.381 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': '', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 5, 'successful_ops': 5}]}, {'bucket': 'myfoo2', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo3', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo4', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 8, 'successful_ops': 8}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 8, 'successful_ops': 8}}]} 2023-10-05T19:41:27.381 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo', '--categories', 'put_obj'] 2023-10-05T19:41:27.381 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo', '--categories', 'put_obj'] 2023-10-05T19:41:27.381 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo --categories put_obj 2023-10-05T19:41:27.423 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:41:27.423 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:41:27.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.416+0000 7f3de2bd9e40 1 Processor -- start 2023-10-05T19:41:27.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.416+0000 7f3de2bd9e40 1 -- start start 2023-10-05T19:41:27.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.420+0000 7f3de2bd9e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55d3d2ec8420 0x55d3d2eca800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:27.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.420+0000 7f3de2bd9e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55d3d2e10bd0 con 0x55d3d2ec3e40 2023-10-05T19:41:27.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.420+0000 7f3de2bd9e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x55d3d2dcdee0 con 0x55d3d2ec8420 2023-10-05T19:41:27.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.420+0000 7f3ddff0d700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55d3d2ec8420 0x55d3d2eca800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:27.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.420+0000 7f3de070e700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x55d3d2ec3e40 0x55d3d2ec4210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:57670/0 (socket says 172.21.15.12:57670) 2023-10-05T19:41:27.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.420+0000 7f3ddff0d700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55d3d2ec8420 0x55d3d2eca800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:50538/0 (socket says 172.21.15.12:50538) 2023-10-05T19:41:27.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.420+0000 7f3de070e700 1 -- 172.21.15.12:0/1704540851 learned_addr learned my addr 172.21.15.12:0/1704540851 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:41:27.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.420+0000 7f3ddff0d700 1 -- 172.21.15.12:0/1704540851 >> v1:172.21.15.8:6789/0 conn(0x55d3d2ec3e40 legacy=0x55d3d2ec4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.420+0000 7f3ddf70c700 1 -- 172.21.15.12:0/1704540851 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3963501960 0 0) 0x55d3d2e10bd0 con 0x55d3d2ec3e40 2023-10-05T19:41:27.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.420+0000 7f3ddff0d700 1 -- 172.21.15.12:0/1704540851 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3d2ec4a50 con 0x55d3d2ec8420 2023-10-05T19:41:27.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.420+0000 7f3ddff0d700 1 --2- 172.21.15.12:0/1704540851 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d3d2ec8420 0x55d3d2eca800 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f3dc800a5d0 tx=0x7f3dc8005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62070d6871e7c4db server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:27.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.420+0000 7f3ddf70c700 1 -- 172.21.15.12:0/1704540851 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3dc80086a0 con 0x55d3d2ec8420 2023-10-05T19:41:27.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.420+0000 7f3ddf70c700 1 -- 172.21.15.12:0/1704540851 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc8008800 con 0x55d3d2ec8420 2023-10-05T19:41:27.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.420+0000 7f3ddf70c700 1 -- 172.21.15.12:0/1704540851 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3dc8008ad0 con 0x55d3d2ec8420 2023-10-05T19:41:27.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.420+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/1704540851 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d3d2ec8420 msgr2=0x55d3d2eca800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.420+0000 7f3de2bd9e40 1 --2- 172.21.15.12:0/1704540851 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d3d2ec8420 0x55d3d2eca800 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f3dc800a5d0 tx=0x7f3dc8005a50 comp rx=0 tx=0).stop 2023-10-05T19:41:27.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.420+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/1704540851 shutdown_connections 2023-10-05T19:41:27.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.420+0000 7f3de2bd9e40 1 --2- 172.21.15.12:0/1704540851 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d3d2ec8420 0x55d3d2eca800 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:27.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.420+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/1704540851 >> 172.21.15.12:0/1704540851 conn(0x55d3d2ebf860 msgr2=0x55d3d2ec1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:27.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.420+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/1704540851 shutdown_connections 2023-10-05T19:41:27.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.420+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/1704540851 wait complete. 2023-10-05T19:41:27.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.424+0000 7f3de2bd9e40 1 Processor -- start 2023-10-05T19:41:27.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.424+0000 7f3de2bd9e40 1 -- start start 2023-10-05T19:41:27.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.424+0000 7f3de2bd9e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d3d2ec3e40 0x55d3d2ee2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:27.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.424+0000 7f3de2bd9e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3d2d91e30 con 0x55d3d2ec3e40 2023-10-05T19:41:27.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.424+0000 7f3de070e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d3d2ec3e40 0x55d3d2ee2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:27.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.424+0000 7f3de070e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d3d2ec3e40 0x55d3d2ee2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:50546/0 (socket says 172.21.15.12:50546) 2023-10-05T19:41:27.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.424+0000 7f3de070e700 1 -- 172.21.15.12:0/822525643 learned_addr learned my addr 172.21.15.12:0/822525643 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:27.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.424+0000 7f3de070e700 1 -- 172.21.15.12:0/822525643 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc8005cf0 con 0x55d3d2ec3e40 2023-10-05T19:41:27.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.424+0000 7f3de070e700 1 --2- 172.21.15.12:0/822525643 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d3d2ec3e40 0x55d3d2ee2db0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f3dd00016b0 tx=0x7f3dd0006a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0a9da8e7bf3e1b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:27.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.424+0000 7f3dde70a700 1 -- 172.21.15.12:0/822525643 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3dd0006320 con 0x55d3d2ec3e40 2023-10-05T19:41:27.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.424+0000 7f3dde70a700 1 -- 172.21.15.12:0/822525643 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd0004520 con 0x55d3d2ec3e40 2023-10-05T19:41:27.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.424+0000 7f3dde70a700 1 -- 172.21.15.12:0/822525643 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3dd00047f0 con 0x55d3d2ec3e40 2023-10-05T19:41:27.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.424+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/822525643 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d3d2ec3e40 msgr2=0x55d3d2ee2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.424+0000 7f3de2bd9e40 1 --2- 172.21.15.12:0/822525643 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d3d2ec3e40 0x55d3d2ee2db0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f3dd00016b0 tx=0x7f3dd0006a10 comp rx=0 tx=0).stop 2023-10-05T19:41:27.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.424+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/822525643 shutdown_connections 2023-10-05T19:41:27.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.424+0000 7f3de2bd9e40 1 --2- 172.21.15.12:0/822525643 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d3d2ec3e40 0x55d3d2ee2db0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:27.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.424+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/822525643 >> 172.21.15.12:0/822525643 conn(0x55d3d2ebf860 msgr2=0x55d3d2ede970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:27.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.424+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/822525643 shutdown_connections 2023-10-05T19:41:27.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.424+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/822525643 wait complete. 2023-10-05T19:41:27.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.424+0000 7f3de2bd9e40 1 Processor -- start 2023-10-05T19:41:27.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.424+0000 7f3de2bd9e40 1 -- start start 2023-10-05T19:41:27.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.424+0000 7f3de2bd9e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d3d2ec8420 0x55d3d2e39b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:27.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.424+0000 7f3de2bd9e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3d2d64ed0 con 0x55d3d2ec8420 2023-10-05T19:41:27.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.428+0000 7f3de070e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d3d2ec8420 0x55d3d2e39b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:27.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.428+0000 7f3de070e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d3d2ec8420 0x55d3d2e39b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:50562/0 (socket says 172.21.15.12:50562) 2023-10-05T19:41:27.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.428+0000 7f3de070e700 1 -- 172.21.15.12:0/3730364971 learned_addr learned my addr 172.21.15.12:0/3730364971 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:27.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.428+0000 7f3de070e700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd0006cf0 con 0x55d3d2ec8420 2023-10-05T19:41:27.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.428+0000 7f3de070e700 1 --2- 172.21.15.12:0/3730364971 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d3d2ec8420 0x55d3d2e39b00 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0000f80 tx=0x7f3dd0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:27.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.428+0000 7f3ddcf07700 1 -- 172.21.15.12:0/3730364971 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3dd00047e0 con 0x55d3d2ec8420 2023-10-05T19:41:27.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.428+0000 7f3ddcf07700 1 -- 172.21.15.12:0/3730364971 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd0013b80 con 0x55d3d2ec8420 2023-10-05T19:41:27.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.428+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d3d2e3a040 con 0x55d3d2ec8420 2023-10-05T19:41:27.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.428+0000 7f3ddcf07700 1 -- 172.21.15.12:0/3730364971 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3dd0013e70 con 0x55d3d2ec8420 2023-10-05T19:41:27.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.428+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3d2e38400 con 0x55d3d2ec8420 2023-10-05T19:41:27.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.428+0000 7f3de2bd9e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.428+0000 7f3ddcf07700 1 -- 172.21.15.12:0/3730364971 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f3dd0028460 con 0x55d3d2ec8420 2023-10-05T19:41:27.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.428+0000 7f3ddcf07700 1 --2- 172.21.15.12:0/3730364971 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3dc0030c00 0x7f3dc00330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:27.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.428+0000 7f3ddcf07700 1 -- 172.21.15.12:0/3730364971 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7f3dd0045370 con 0x55d3d2ec8420 2023-10-05T19:41:27.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.432+0000 7f3de2bd9e40 1 --2- 172.21.15.12:0/3730364971 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55d3d2ef59e0 0x55d3d2ef7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:27.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.432+0000 7f3ddff0d700 1 --2- 172.21.15.12:0/3730364971 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3dc0030c00 0x7f3dc00330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:27.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.432+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2ef8470 con 0x55d3d2ef59e0 2023-10-05T19:41:27.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.432+0000 7f3de0f0f700 1 --2- 172.21.15.12:0/3730364971 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55d3d2ef59e0 0x55d3d2ef7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:27.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.432+0000 7f3ddff0d700 1 --2- 172.21.15.12:0/3730364971 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3dc0030c00 0x7f3dc00330b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8005a90 tx=0x7f3dc8015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:27.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.432+0000 7f3de0f0f700 1 --2- 172.21.15.12:0/3730364971 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55d3d2ef59e0 0x55d3d2ef7dc0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:27.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.432+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3dd800a500 con 0x55d3d2ef59e0 2023-10-05T19:41:27.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.432+0000 7f3de2bd9e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.432+0000 7f3de2bd9e40 20 realm 2023-10-05T19:41:27.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.432+0000 7f3de2bd9e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.432+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2ef5050 con 0x55d3d2ef59e0 2023-10-05T19:41:27.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.432+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3dd800a500 con 0x55d3d2ef59e0 2023-10-05T19:41:27.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.432+0000 7f3de2bd9e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.432+0000 7f3de2bd9e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:41:27.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.432+0000 7f3de2bd9e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.432+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2ef8f50 con 0x55d3d2ef59e0 2023-10-05T19:41:27.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.432+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3dd800a500 con 0x55d3d2ef59e0 2023-10-05T19:41:27.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.432+0000 7f3de2bd9e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.432+0000 7f3de2bd9e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.432+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2ef8b30 con 0x55d3d2ef59e0 2023-10-05T19:41:27.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.432+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f3dd800a500 con 0x55d3d2ef59e0 2023-10-05T19:41:27.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.432+0000 7f3de2bd9e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:27.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.432+0000 7f3de2bd9e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.432+0000 7f3de2bd9e40 1 --2- 172.21.15.12:0/3730364971 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55d3d2ef9ee0 0x55d3d2efc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:27.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.432+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2efca30 con 0x55d3d2ef9ee0 2023-10-05T19:41:27.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.432+0000 7f3de070e700 1 --2- 172.21.15.12:0/3730364971 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55d3d2ef9ee0 0x55d3d2efc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:27.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.436+0000 7f3de070e700 1 --2- 172.21.15.12:0/3730364971 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55d3d2ef9ee0 0x55d3d2efc2c0 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:27.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.436+0000 7f3de070e700 1 -- 172.21.15.12:0/3730364971 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f3dd0024020 con 0x55d3d2ef9ee0 2023-10-05T19:41:27.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.436+0000 7f3de2bd9e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:41:27.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.436+0000 7f3de2bd9e40 20 searching for the correct realm 2023-10-05T19:41:27.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.436+0000 7f3de2bd9e40 1 --2- 172.21.15.12:0/3730364971 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55d3d2efe860 0x55d3d2f00c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:27.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.436+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01180 con 0x55d3d2efe860 2023-10-05T19:41:27.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.436+0000 7f3ddff0d700 1 --2- 172.21.15.12:0/3730364971 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55d3d2efe860 0x55d3d2f00c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:27.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.436+0000 7f3ddff0d700 1 --2- 172.21.15.12:0/3730364971 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55d3d2efe860 0x55d3d2f00c40 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:27.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.436+0000 7f3ddff0d700 1 -- 172.21.15.12:0/3730364971 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f3dc8008a40 con 0x55d3d2efe860 2023-10-05T19:41:27.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.436+0000 7f3ddff0d700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01180 con 0x55d3d2ef9ee0 2023-10-05T19:41:27.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.440+0000 7f3de070e700 1 -- 172.21.15.12:0/3730364971 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dd0024020 con 0x55d3d2ef9ee0 2023-10-05T19:41:27.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.440+0000 7f3de070e700 1 --2- 172.21.15.12:0/3730364971 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55d3d2ec3e40 0x7f3dd0012810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:27.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.440+0000 7f3de070e700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01180 con 0x55d3d2ec3e40 2023-10-05T19:41:27.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.440+0000 7f3de0f0f700 1 --2- 172.21.15.12:0/3730364971 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55d3d2ec3e40 0x7f3dd0012810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:27.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.440+0000 7f3de0f0f700 1 --2- 172.21.15.12:0/3730364971 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55d3d2ec3e40 0x7f3dd0012810 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:27.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.440+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ec3e40 2023-10-05T19:41:27.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.440+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01180 con 0x55d3d2ef59e0 2023-10-05T19:41:27.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.440+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ef59e0 2023-10-05T19:41:27.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.440+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01180 con 0x55d3d2ef59e0 2023-10-05T19:41:27.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.440+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ef59e0 2023-10-05T19:41:27.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.440+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01180 con 0x55d3d2ef59e0 2023-10-05T19:41:27.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.440+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(11 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ef59e0 2023-10-05T19:41:27.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.444+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01180 con 0x55d3d2ec3e40 2023-10-05T19:41:27.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.444+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ec3e40 2023-10-05T19:41:27.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.444+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01180 con 0x55d3d2ef9ee0 2023-10-05T19:41:27.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.444+0000 7f3de070e700 1 -- 172.21.15.12:0/3730364971 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dd0024020 con 0x55d3d2ef9ee0 2023-10-05T19:41:27.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.444+0000 7f3de070e700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01180 con 0x55d3d2ef9ee0 2023-10-05T19:41:27.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.444+0000 7f3de070e700 1 -- 172.21.15.12:0/3730364971 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dd0024020 con 0x55d3d2ef9ee0 2023-10-05T19:41:27.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.444+0000 7f3de070e700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01180 con 0x55d3d2ef59e0 2023-10-05T19:41:27.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.444+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ef59e0 2023-10-05T19:41:27.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.444+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01180 con 0x55d3d2ec3e40 2023-10-05T19:41:27.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.444+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ec3e40 2023-10-05T19:41:27.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.448+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01180 con 0x55d3d2ef59e0 2023-10-05T19:41:27.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.448+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ef59e0 2023-10-05T19:41:27.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.448+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01180 con 0x55d3d2ef9ee0 2023-10-05T19:41:27.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.448+0000 7f3de070e700 1 -- 172.21.15.12:0/3730364971 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(18 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dd0024020 con 0x55d3d2ef9ee0 2023-10-05T19:41:27.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.448+0000 7f3de070e700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01180 con 0x55d3d2efe860 2023-10-05T19:41:27.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.448+0000 7f3ddff0d700 1 -- 172.21.15.12:0/3730364971 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dc8012450 con 0x55d3d2efe860 2023-10-05T19:41:27.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.448+0000 7f3ddff0d700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01180 con 0x55d3d2efe860 2023-10-05T19:41:27.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.448+0000 7f3ddff0d700 1 -- 172.21.15.12:0/3730364971 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dc8012450 con 0x55d3d2efe860 2023-10-05T19:41:27.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.448+0000 7f3ddff0d700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01180 con 0x55d3d2ec3e40 2023-10-05T19:41:27.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.448+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ec3e40 2023-10-05T19:41:27.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.448+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01180 con 0x55d3d2efe860 2023-10-05T19:41:27.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.452+0000 7f3ddff0d700 1 -- 172.21.15.12:0/3730364971 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dc8012450 con 0x55d3d2efe860 2023-10-05T19:41:27.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.452+0000 7f3ddff0d700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01180 con 0x55d3d2efe860 2023-10-05T19:41:27.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.452+0000 7f3ddff0d700 1 -- 172.21.15.12:0/3730364971 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dc8012450 con 0x55d3d2efe860 2023-10-05T19:41:27.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.452+0000 7f3ddff0d700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01180 con 0x55d3d2ef59e0 2023-10-05T19:41:27.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.452+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ef59e0 2023-10-05T19:41:27.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.452+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01180 con 0x55d3d2ef59e0 2023-10-05T19:41:27.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.452+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ef59e0 2023-10-05T19:41:27.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.452+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01180 con 0x55d3d2ec3e40 2023-10-05T19:41:27.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.452+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ec3e40 2023-10-05T19:41:27.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.452+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01180 con 0x55d3d2efe860 2023-10-05T19:41:27.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.456+0000 7f3ddff0d700 1 -- 172.21.15.12:0/3730364971 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dc8012450 con 0x55d3d2efe860 2023-10-05T19:41:27.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.456+0000 7f3ddff0d700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01180 con 0x55d3d2ef9ee0 2023-10-05T19:41:27.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.456+0000 7f3de070e700 1 -- 172.21.15.12:0/3730364971 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dd0024020 con 0x55d3d2ef9ee0 2023-10-05T19:41:27.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.456+0000 7f3de070e700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01180 con 0x55d3d2ef59e0 2023-10-05T19:41:27.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.456+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ef59e0 2023-10-05T19:41:27.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.456+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01180 con 0x55d3d2ec3e40 2023-10-05T19:41:27.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.456+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ec3e40 2023-10-05T19:41:27.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.456+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01180 con 0x55d3d2efe860 2023-10-05T19:41:27.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.456+0000 7f3ddff0d700 1 -- 172.21.15.12:0/3730364971 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dc8012450 con 0x55d3d2efe860 2023-10-05T19:41:27.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.456+0000 7f3ddff0d700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01180 con 0x55d3d2ec3e40 2023-10-05T19:41:27.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.460+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ec3e40 2023-10-05T19:41:27.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.460+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01180 con 0x55d3d2ef9ee0 2023-10-05T19:41:27.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.460+0000 7f3de070e700 1 -- 172.21.15.12:0/3730364971 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dd0024020 con 0x55d3d2ef9ee0 2023-10-05T19:41:27.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.460+0000 7f3de070e700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01180 con 0x55d3d2efe860 2023-10-05T19:41:27.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.460+0000 7f3ddff0d700 1 -- 172.21.15.12:0/3730364971 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dc8012450 con 0x55d3d2efe860 2023-10-05T19:41:27.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.460+0000 7f3ddff0d700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01180 con 0x55d3d2ec3e40 2023-10-05T19:41:27.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.460+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ec3e40 2023-10-05T19:41:27.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.460+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01180 con 0x55d3d2ef59e0 2023-10-05T19:41:27.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.460+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ef59e0 2023-10-05T19:41:27.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.464+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01180 con 0x55d3d2ef9ee0 2023-10-05T19:41:27.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.464+0000 7f3de070e700 1 -- 172.21.15.12:0/3730364971 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(37 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f3dd0024020 con 0x55d3d2ef9ee0 2023-10-05T19:41:27.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.464+0000 7f3de2bd9e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:41:27.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.464+0000 7f3de2bd9e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:41:27.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.464+0000 7f3de2bd9e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:41:27.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.464+0000 7f3de2bd9e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:41:27.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.464+0000 7f3de2bd9e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.464+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2efd5e0 con 0x55d3d2ef59e0 2023-10-05T19:41:27.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.464+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ef59e0 2023-10-05T19:41:27.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.464+0000 7f3de2bd9e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.464+0000 7f3de2bd9e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.464+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2efdf30 con 0x55d3d2ef59e0 2023-10-05T19:41:27.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.464+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ef59e0 2023-10-05T19:41:27.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.464+0000 7f3de2bd9e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:27.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.464+0000 7f3de2bd9e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.464+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2efd1a0 con 0x55d3d2efe860 2023-10-05T19:41:27.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.464+0000 7f3ddff0d700 1 -- 172.21.15.12:0/3730364971 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f3dc8012450 con 0x55d3d2efe860 2023-10-05T19:41:27.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.464+0000 7f3de2bd9e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:41:27.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.464+0000 7f3de2bd9e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.464+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01ab0 con 0x55d3d2efe860 2023-10-05T19:41:27.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.464+0000 7f3ddff0d700 1 -- 172.21.15.12:0/3730364971 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f3dc8012450 con 0x55d3d2efe860 2023-10-05T19:41:27.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.464+0000 7f3de2bd9e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.464+0000 7f3de2bd9e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.464+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01e60 con 0x55d3d2ef59e0 2023-10-05T19:41:27.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.468+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ef59e0 2023-10-05T19:41:27.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.468+0000 7f3de2bd9e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.468+0000 7f3de2bd9e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.468+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f02210 con 0x55d3d2ef59e0 2023-10-05T19:41:27.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.468+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ef59e0 2023-10-05T19:41:27.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.468+0000 7f3de2bd9e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:27.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.468+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f03220 con 0x55d3d2efe860 2023-10-05T19:41:27.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.468+0000 7f3ddff0d700 1 -- 172.21.15.12:0/3730364971 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(44 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f3dc8012450 con 0x55d3d2efe860 2023-10-05T19:41:27.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.468+0000 7f3ddff0d700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f03220 con 0x55d3d2ef9ee0 2023-10-05T19:41:27.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.472+0000 7f3de070e700 1 -- 172.21.15.12:0/3730364971 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(45 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dd0024020 con 0x55d3d2ef9ee0 2023-10-05T19:41:27.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.472+0000 7f3de070e700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f03220 con 0x55d3d2ec3e40 2023-10-05T19:41:27.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.472+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(46 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ec3e40 2023-10-05T19:41:27.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.472+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01e60 con 0x55d3d2ef59e0 2023-10-05T19:41:27.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.472+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(47 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ef59e0 2023-10-05T19:41:27.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.472+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01e60 con 0x55d3d2ef59e0 2023-10-05T19:41:27.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.472+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(48 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ef59e0 2023-10-05T19:41:27.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.472+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01e60 con 0x55d3d2ef59e0 2023-10-05T19:41:27.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.472+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(49 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ef59e0 2023-10-05T19:41:27.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.472+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01e60 con 0x55d3d2ec3e40 2023-10-05T19:41:27.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.476+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(50 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ec3e40 2023-10-05T19:41:27.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.476+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01e60 con 0x55d3d2ef9ee0 2023-10-05T19:41:27.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.476+0000 7f3de070e700 1 -- 172.21.15.12:0/3730364971 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(51 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dd0024020 con 0x55d3d2ef9ee0 2023-10-05T19:41:27.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.476+0000 7f3de070e700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01e60 con 0x55d3d2ef9ee0 2023-10-05T19:41:27.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.476+0000 7f3de070e700 1 -- 172.21.15.12:0/3730364971 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(52 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dd0024020 con 0x55d3d2ef9ee0 2023-10-05T19:41:27.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.476+0000 7f3de070e700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01e60 con 0x55d3d2ef59e0 2023-10-05T19:41:27.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.476+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(53 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ef59e0 2023-10-05T19:41:27.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.476+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01e60 con 0x55d3d2ec3e40 2023-10-05T19:41:27.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.480+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(54 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ec3e40 2023-10-05T19:41:27.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.480+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01e60 con 0x55d3d2ef59e0 2023-10-05T19:41:27.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.480+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(55 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ef59e0 2023-10-05T19:41:27.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.480+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01e60 con 0x55d3d2ef9ee0 2023-10-05T19:41:27.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.480+0000 7f3de070e700 1 -- 172.21.15.12:0/3730364971 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(56 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dd0024020 con 0x55d3d2ef9ee0 2023-10-05T19:41:27.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.480+0000 7f3de070e700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01e60 con 0x55d3d2efe860 2023-10-05T19:41:27.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.480+0000 7f3ddff0d700 1 -- 172.21.15.12:0/3730364971 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(57 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dc8012450 con 0x55d3d2efe860 2023-10-05T19:41:27.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.480+0000 7f3ddff0d700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01e60 con 0x55d3d2efe860 2023-10-05T19:41:27.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.480+0000 7f3ddff0d700 1 -- 172.21.15.12:0/3730364971 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(58 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dc8012450 con 0x55d3d2efe860 2023-10-05T19:41:27.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.480+0000 7f3ddff0d700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01e60 con 0x55d3d2ec3e40 2023-10-05T19:41:27.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.484+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(59 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ec3e40 2023-10-05T19:41:27.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.484+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01e60 con 0x55d3d2efe860 2023-10-05T19:41:27.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.484+0000 7f3ddff0d700 1 -- 172.21.15.12:0/3730364971 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(60 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dc8012450 con 0x55d3d2efe860 2023-10-05T19:41:27.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.484+0000 7f3ddff0d700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01e60 con 0x55d3d2efe860 2023-10-05T19:41:27.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.484+0000 7f3ddff0d700 1 -- 172.21.15.12:0/3730364971 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(61 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dc8012450 con 0x55d3d2efe860 2023-10-05T19:41:27.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.484+0000 7f3ddff0d700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01e60 con 0x55d3d2ef59e0 2023-10-05T19:41:27.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.484+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(62 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ef59e0 2023-10-05T19:41:27.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.484+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01e60 con 0x55d3d2ef59e0 2023-10-05T19:41:27.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.488+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(63 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ef59e0 2023-10-05T19:41:27.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.488+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01e60 con 0x55d3d2ec3e40 2023-10-05T19:41:27.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.488+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(64 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ec3e40 2023-10-05T19:41:27.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.488+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01e60 con 0x55d3d2efe860 2023-10-05T19:41:27.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.488+0000 7f3ddff0d700 1 -- 172.21.15.12:0/3730364971 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(65 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dc8012450 con 0x55d3d2efe860 2023-10-05T19:41:27.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.488+0000 7f3ddff0d700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01e60 con 0x55d3d2ef9ee0 2023-10-05T19:41:27.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.488+0000 7f3de070e700 1 -- 172.21.15.12:0/3730364971 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(66 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dd0024020 con 0x55d3d2ef9ee0 2023-10-05T19:41:27.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.488+0000 7f3de070e700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01e60 con 0x55d3d2ef59e0 2023-10-05T19:41:27.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.488+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(67 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ef59e0 2023-10-05T19:41:27.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.488+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01e60 con 0x55d3d2ec3e40 2023-10-05T19:41:27.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.492+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(68 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ec3e40 2023-10-05T19:41:27.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.492+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01e60 con 0x55d3d2efe860 2023-10-05T19:41:27.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.492+0000 7f3ddff0d700 1 -- 172.21.15.12:0/3730364971 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(69 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dc8012450 con 0x55d3d2efe860 2023-10-05T19:41:27.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.492+0000 7f3ddff0d700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01e60 con 0x55d3d2ec3e40 2023-10-05T19:41:27.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.492+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(70 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ec3e40 2023-10-05T19:41:27.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.492+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01e60 con 0x55d3d2ef9ee0 2023-10-05T19:41:27.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.492+0000 7f3de070e700 1 -- 172.21.15.12:0/3730364971 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(71 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dd0024020 con 0x55d3d2ef9ee0 2023-10-05T19:41:27.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.492+0000 7f3de070e700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01e60 con 0x55d3d2efe860 2023-10-05T19:41:27.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.496+0000 7f3ddff0d700 1 -- 172.21.15.12:0/3730364971 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(72 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dc8012450 con 0x55d3d2efe860 2023-10-05T19:41:27.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.496+0000 7f3ddff0d700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01e60 con 0x55d3d2ec3e40 2023-10-05T19:41:27.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.496+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(73 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ec3e40 2023-10-05T19:41:27.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.496+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01e60 con 0x55d3d2ef59e0 2023-10-05T19:41:27.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.496+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(74 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ef59e0 2023-10-05T19:41:27.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.496+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01e60 con 0x55d3d2ef9ee0 2023-10-05T19:41:27.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.496+0000 7f3de070e700 1 -- 172.21.15.12:0/3730364971 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(75 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f3dd0024020 con 0x55d3d2ef9ee0 2023-10-05T19:41:27.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.496+0000 7f3de2bd9e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:41:27.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.496+0000 7f3de2bd9e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:41:27.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.496+0000 7f3de2bd9e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:41:27.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.496+0000 7f3de2bd9e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:41:27.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.496+0000 7f3de2bd9e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.496+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f02210 con 0x55d3d2ef59e0 2023-10-05T19:41:27.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.496+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ef59e0 2023-10-05T19:41:27.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.496+0000 7f3de2bd9e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:27.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.496+0000 7f3de2bd9e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.496+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f025c0 con 0x55d3d2ef9ee0 2023-10-05T19:41:27.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.500+0000 7f3de070e700 1 -- 172.21.15.12:0/3730364971 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f3dd0024020 con 0x55d3d2ef9ee0 2023-10-05T19:41:27.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.500+0000 7f3de2bd9e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:41:27.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.500+0000 7f3de2bd9e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.500+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f01680 con 0x55d3d2ef59e0 2023-10-05T19:41:27.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.500+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ef59e0 2023-10-05T19:41:27.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.500+0000 7f3de2bd9e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:27.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.500+0000 7f3de2bd9e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.500+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f02e60 con 0x55d3d2efe860 2023-10-05T19:41:27.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.500+0000 7f3ddff0d700 1 -- 172.21.15.12:0/3730364971 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f3dc8012450 con 0x55d3d2efe860 2023-10-05T19:41:27.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.500+0000 7f3de2bd9e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:41:27.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.500+0000 7f3de2bd9e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.500+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f03210 con 0x55d3d2ec3e40 2023-10-05T19:41:27.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.500+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ec3e40 2023-10-05T19:41:27.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.500+0000 7f3de2bd9e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.500+0000 7f3de2bd9e40 20 zone default found 2023-10-05T19:41:27.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.500+0000 7f3de2bd9e40 4 Realm: () 2023-10-05T19:41:27.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.500+0000 7f3de2bd9e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:41:27.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.500+0000 7f3de2bd9e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:41:27.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.500+0000 7f3de2bd9e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:41:27.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.500+0000 7f3de2bd9e40 20 zonegroup default 2023-10-05T19:41:27.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.500+0000 7f3de2bd9e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.500+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f03210 con 0x55d3d2ec3e40 2023-10-05T19:41:27.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.500+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ec3e40 2023-10-05T19:41:27.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.500+0000 7f3de2bd9e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.500+0000 7f3de2bd9e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.500+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f04b90 con 0x55d3d2efe860 2023-10-05T19:41:27.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.504+0000 7f3ddff0d700 1 -- 172.21.15.12:0/3730364971 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f3dc8012450 con 0x55d3d2efe860 2023-10-05T19:41:27.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.504+0000 7f3de2bd9e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.504+0000 7f3de2bd9e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.504+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f04f40 con 0x55d3d2efe860 2023-10-05T19:41:27.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.504+0000 7f3ddff0d700 1 -- 172.21.15.12:0/3730364971 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f3dc8012450 con 0x55d3d2efe860 2023-10-05T19:41:27.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.504+0000 7f3de2bd9e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.504+0000 7f3de2bd9e40 20 started sync module instance, tier type = 2023-10-05T19:41:27.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.504+0000 7f3de2bd9e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:41:27.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.504+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f04250 con 0x55d3d2ef9ee0 2023-10-05T19:41:27.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.508+0000 7f3de070e700 1 -- 172.21.15.12:0/3730364971 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(84 notify.0 [create] v30'231 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3dd0024020 con 0x55d3d2ef9ee0 2023-10-05T19:41:27.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.508+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94368265422416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f02a90 con 0x55d3d2ef9ee0 2023-10-05T19:41:27.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.508+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f05da0 con 0x55d3d2ef59e0 2023-10-05T19:41:27.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.508+0000 7f3de070e700 1 -- 172.21.15.12:0/3730364971 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(85 notify.0 [watch watch cookie 94368265422416] v30'232 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3dd0024020 con 0x55d3d2ef9ee0 2023-10-05T19:41:27.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.508+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(86 notify.1 [create] v30'231 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ef59e0 2023-10-05T19:41:27.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.508+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94368265425648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f061a0 con 0x55d3d2ef59e0 2023-10-05T19:41:27.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.508+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f06bb0 con 0x55d3d2efe860 2023-10-05T19:41:27.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.512+0000 7f3ddff0d700 1 -- 172.21.15.12:0/3730364971 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(88 notify.2 [create] v30'231 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3dc8012450 con 0x55d3d2efe860 2023-10-05T19:41:27.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.512+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94368265430720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f07450 con 0x55d3d2efe860 2023-10-05T19:41:27.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.512+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f07d30 con 0x55d3d2ef9ee0 2023-10-05T19:41:27.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.512+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(87 notify.1 [watch watch cookie 94368265425648] v30'232 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ef59e0 2023-10-05T19:41:27.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.516+0000 7f3de070e700 1 -- 172.21.15.12:0/3730364971 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(90 notify.3 [create] v30'231 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3dd0024020 con 0x55d3d2ef9ee0 2023-10-05T19:41:27.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.516+0000 7f3ddff0d700 1 -- 172.21.15.12:0/3730364971 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(89 notify.2 [watch watch cookie 94368265430720] v30'232 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3dc8012450 con 0x55d3d2efe860 2023-10-05T19:41:27.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.516+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94368265427056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f07920 con 0x55d3d2ef9ee0 2023-10-05T19:41:27.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.516+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f08620 con 0x55d3d2ef59e0 2023-10-05T19:41:27.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.516+0000 7f3de070e700 1 -- 172.21.15.12:0/3730364971 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(91 notify.3 [watch watch cookie 94368265427056] v30'232 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3dd0024020 con 0x55d3d2ef9ee0 2023-10-05T19:41:27.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.516+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(92 notify.4 [create] v30'231 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ef59e0 2023-10-05T19:41:27.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.516+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94368265437488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f07070 con 0x55d3d2ef59e0 2023-10-05T19:41:27.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.520+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f091d0 con 0x55d3d2ec3e40 2023-10-05T19:41:27.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.520+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(93 notify.4 [watch watch cookie 94368265437488] v30'232 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ef59e0 2023-10-05T19:41:27.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.520+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(94 notify.5 [create] v30'231 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ec3e40 2023-10-05T19:41:27.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.520+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94368265438592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f09190 con 0x55d3d2ec3e40 2023-10-05T19:41:27.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.520+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f09c30 con 0x55d3d2ec3e40 2023-10-05T19:41:27.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.524+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(95 notify.5 [watch watch cookie 94368265438592] v30'232 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ec3e40 2023-10-05T19:41:27.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.524+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(96 notify.6 [create] v30'231 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ec3e40 2023-10-05T19:41:27.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.524+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94368265440976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f09c30 con 0x55d3d2ec3e40 2023-10-05T19:41:27.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.524+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f0a640 con 0x55d3d2ec3e40 2023-10-05T19:41:27.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.528+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(97 notify.6 [watch watch cookie 94368265440976] v30'232 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ec3e40 2023-10-05T19:41:27.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.528+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(98 notify.7 [create] v30'231 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ec3e40 2023-10-05T19:41:27.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.528+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94368265443408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f0a640 con 0x55d3d2ec3e40 2023-10-05T19:41:27.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.528+0000 7f3de2bd9e40 20 add_watcher() i=0 2023-10-05T19:41:27.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.528+0000 7f3de2bd9e40 20 add_watcher() i=1 2023-10-05T19:41:27.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.528+0000 7f3de2bd9e40 20 add_watcher() i=2 2023-10-05T19:41:27.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.528+0000 7f3de2bd9e40 20 add_watcher() i=3 2023-10-05T19:41:27.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.528+0000 7f3de2bd9e40 20 add_watcher() i=4 2023-10-05T19:41:27.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.528+0000 7f3de2bd9e40 20 add_watcher() i=5 2023-10-05T19:41:27.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.528+0000 7f3de2bd9e40 20 add_watcher() i=6 2023-10-05T19:41:27.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.532+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(99 notify.7 [watch watch cookie 94368265443408] v30'232 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ec3e40 2023-10-05T19:41:27.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.532+0000 7f3de2bd9e40 20 add_watcher() i=7 2023-10-05T19:41:27.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.532+0000 7f3de2bd9e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:41:27.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.532+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f0d710 con 0x55d3d2ef59e0 2023-10-05T19:41:27.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.532+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ef59e0 2023-10-05T19:41:27.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.532+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94368265445712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f0aa90 con 0x55d3d2ef59e0 2023-10-05T19:41:27.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.532+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94368265445712] v30'225 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ef59e0 2023-10-05T19:41:27.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.536+0000 7f3db1fe3700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:41:27.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.536+0000 7f3de2bd9e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:41:27.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.536+0000 7f3de2bd9e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:41:27.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.536+0000 7f3de2bd9e40 1 Processor -- start 2023-10-05T19:41:27.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.536+0000 7f3de2bd9e40 1 -- start start 2023-10-05T19:41:27.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.536+0000 7f3de2bd9e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d3d2f17c30 0x55d3d2f18000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:27.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.536+0000 7f3de2bd9e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3d2f18540 con 0x55d3d2f17c30 2023-10-05T19:41:27.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.536+0000 7f3ddff0d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d3d2f17c30 0x55d3d2f18000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:27.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.536+0000 7f3ddff0d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d3d2f17c30 0x55d3d2f18000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:50570/0 (socket says 172.21.15.12:50570) 2023-10-05T19:41:27.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.536+0000 7f3ddff0d700 1 -- 172.21.15.12:0/2304813194 learned_addr learned my addr 172.21.15.12:0/2304813194 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:27.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.536+0000 7f3ddff0d700 1 -- 172.21.15.12:0/2304813194 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3d2f18680 con 0x55d3d2f17c30 2023-10-05T19:41:27.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.536+0000 7f3ddff0d700 1 --2- 172.21.15.12:0/2304813194 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d3d2f17c30 0x55d3d2f18000 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x55d3d2ec4ec0 tx=0x7f3dc8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2e4f70040971a47 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:27.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.536+0000 7f3db17e2700 1 -- 172.21.15.12:0/2304813194 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3dc8012c00 con 0x55d3d2f17c30 2023-10-05T19:41:27.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.536+0000 7f3db17e2700 1 -- 172.21.15.12:0/2304813194 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc8030c10 con 0x55d3d2f17c30 2023-10-05T19:41:27.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.536+0000 7f3db17e2700 1 -- 172.21.15.12:0/2304813194 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3dc8030e00 con 0x55d3d2f17c30 2023-10-05T19:41:27.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.540+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/2304813194 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d3d2f17c30 msgr2=0x55d3d2f18000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.540+0000 7f3de2bd9e40 1 --2- 172.21.15.12:0/2304813194 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d3d2f17c30 0x55d3d2f18000 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x55d3d2ec4ec0 tx=0x7f3dc8012000 comp rx=0 tx=0).stop 2023-10-05T19:41:27.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.540+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/2304813194 shutdown_connections 2023-10-05T19:41:27.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.540+0000 7f3de2bd9e40 1 --2- 172.21.15.12:0/2304813194 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d3d2f17c30 0x55d3d2f18000 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:27.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.540+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/2304813194 >> 172.21.15.12:0/2304813194 conn(0x55d3d2f12ff0 msgr2=0x55d3d2f111a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:27.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.540+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/2304813194 shutdown_connections 2023-10-05T19:41:27.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.540+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/2304813194 wait complete. 2023-10-05T19:41:27.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.540+0000 7f3de2bd9e40 1 Processor -- start 2023-10-05T19:41:27.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.540+0000 7f3de2bd9e40 1 -- start start 2023-10-05T19:41:27.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.540+0000 7f3de2bd9e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d3d2f17c30 0x55d3d2f1d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:27.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.540+0000 7f3de2bd9e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3d2f1e6b0 con 0x55d3d2f17c30 2023-10-05T19:41:27.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.540+0000 7f3ddff0d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d3d2f17c30 0x55d3d2f1d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:27.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.540+0000 7f3ddff0d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d3d2f17c30 0x55d3d2f1d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:50572/0 (socket says 172.21.15.12:50572) 2023-10-05T19:41:27.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.540+0000 7f3ddff0d700 1 -- 172.21.15.12:0/3211949211 learned_addr learned my addr 172.21.15.12:0/3211949211 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:27.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.540+0000 7f3ddff0d700 1 -- 172.21.15.12:0/3211949211 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3d2ec4a50 con 0x55d3d2f17c30 2023-10-05T19:41:27.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.540+0000 7f3ddff0d700 1 --2- 172.21.15.12:0/3211949211 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d3d2f17c30 0x55d3d2f1d6b0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f3dc803a040 tx=0x7f3dc8032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:27.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.540+0000 7f3d2b7fe700 1 -- 172.21.15.12:0/3211949211 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3dc8027ea0 con 0x55d3d2f17c30 2023-10-05T19:41:27.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.540+0000 7f3d2b7fe700 1 -- 172.21.15.12:0/3211949211 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc80403f0 con 0x55d3d2f17c30 2023-10-05T19:41:27.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.540+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3211949211 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d3d2f1dcb0 con 0x55d3d2f17c30 2023-10-05T19:41:27.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.540+0000 7f3d2b7fe700 1 -- 172.21.15.12:0/3211949211 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3dc80405e0 con 0x55d3d2f17c30 2023-10-05T19:41:27.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.540+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3211949211 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3d2f1e0f0 con 0x55d3d2f17c30 2023-10-05T19:41:27.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.544+0000 7f3d2b7fe700 1 -- 172.21.15.12:0/3211949211 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f3dc8036040 con 0x55d3d2f17c30 2023-10-05T19:41:27.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.544+0000 7f3d2b7fe700 1 --2- 172.21.15.12:0/3211949211 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3d20030ca0 0x7f3d20033150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:27.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.544+0000 7f3d2b7fe700 1 -- 172.21.15.12:0/3211949211 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7f3dc8033040 con 0x55d3d2f17c30 2023-10-05T19:41:27.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.544+0000 7f3de2bd9e40 5 note: GC not initialized 2023-10-05T19:41:27.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.544+0000 7f3de0f0f700 1 --2- 172.21.15.12:0/3211949211 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3d20030ca0 0x7f3d20033150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:27.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.544+0000 7f3de0f0f700 1 --2- 172.21.15.12:0/3211949211 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3d20030ca0 0x7f3d20033150 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x55d3d2f0d7e0 tx=0x7f3dd8032600 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:27.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.548+0000 7f3d28ff9700 20 reqs_thread_entry: start 2023-10-05T19:41:27.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.548+0000 7f3d1e7fc700 20 reqs_thread_entry: start 2023-10-05T19:41:27.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.548+0000 7f3de2bd9e40 20 init_complete bucket index max shards: 11 2023-10-05T19:41:27.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.548+0000 7f3de2bd9e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:41:27.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.548+0000 7f3d1cff9700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:41:57 2023 2023-10-05T19:41:27.554 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:41:27.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.548+0000 7f3d03fff700 20 reqs_thread_entry: start 2023-10-05T19:41:27.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.548+0000 7f3de2bd9e40 20 get_system_obj_state: rctx=0x7ffc6b30f7b8 obj=default.rgw.meta:users.uid:foo state=0x55d3d2d9d0b0 s->prefetch_data=0 2023-10-05T19:41:27.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.548+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f85e90 con 0x55d3d2ef59e0 2023-10-05T19:41:27.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.548+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(102 foo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv15 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ef59e0 2023-10-05T19:41:27.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.548+0000 7f3de2bd9e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:41:27.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.548+0000 7f3de2bd9e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:41:27.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.548+0000 7f3de2bd9e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.548+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f88920 con 0x55d3d2ef59e0 2023-10-05T19:41:27.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.548+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(103 foo [call,call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv15 ondisk = 0) v8 ==== 273+0+480 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ef59e0 2023-10-05T19:41:27.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.548+0000 7f3de2bd9e40 20 rados_obj.operate() r=0 bl.length=357 2023-10-05T19:41:27.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.548+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:104 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f87910 con 0x55d3d2ec3e40 2023-10-05T19:41:27.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.552+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(104 usage.10 [call out=1268b] v0'0 uv32 ondisk = 0) v8 ==== 152+0+1268 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ec3e40 2023-10-05T19:41:27.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.552+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:105 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94368265422416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f44280 con 0x55d3d2ef9ee0 2023-10-05T19:41:27.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.556+0000 7f3de070e700 1 -- 172.21.15.12:0/3730364971 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(105 notify.0 [watch unwatch cookie 94368265422416] v30'233 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3dd0024020 con 0x55d3d2ef9ee0 2023-10-05T19:41:27.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.556+0000 7f3de2bd9e40 20 remove_watcher() i=0 2023-10-05T19:41:27.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.556+0000 7f3de2bd9e40 2 removed watcher, disabling cache 2023-10-05T19:41:27.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.556+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:106 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94368265425648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f04650 con 0x55d3d2ef59e0 2023-10-05T19:41:27.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.556+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(106 notify.1 [watch unwatch cookie 94368265425648] v30'233 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ef59e0 2023-10-05T19:41:27.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.556+0000 7f3de2bd9e40 20 remove_watcher() i=1 2023-10-05T19:41:27.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.556+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:107 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94368265430720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f052f0 con 0x55d3d2efe860 2023-10-05T19:41:27.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.560+0000 7f3ddff0d700 1 -- 172.21.15.12:0/3730364971 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(107 notify.2 [watch unwatch cookie 94368265430720] v30'233 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3dc8012450 con 0x55d3d2efe860 2023-10-05T19:41:27.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.560+0000 7f3de2bd9e40 20 remove_watcher() i=2 2023-10-05T19:41:27.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.560+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:108 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94368265427056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f052f0 con 0x55d3d2ef9ee0 2023-10-05T19:41:27.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.564+0000 7f3de070e700 1 -- 172.21.15.12:0/3730364971 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(108 notify.3 [watch unwatch cookie 94368265427056] v30'233 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3dd0024020 con 0x55d3d2ef9ee0 2023-10-05T19:41:27.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.564+0000 7f3de2bd9e40 20 remove_watcher() i=3 2023-10-05T19:41:27.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.564+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:109 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94368265437488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f05870 con 0x55d3d2ef59e0 2023-10-05T19:41:27.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.564+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== osd_op_reply(109 notify.4 [watch unwatch cookie 94368265437488] v30'233 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ef59e0 2023-10-05T19:41:27.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.568+0000 7f3de2bd9e40 20 remove_watcher() i=4 2023-10-05T19:41:27.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.568+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:110 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94368265438592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f052f0 con 0x55d3d2ec3e40 2023-10-05T19:41:27.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.568+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(110 notify.5 [watch unwatch cookie 94368265438592] v30'233 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ec3e40 2023-10-05T19:41:27.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.568+0000 7f3de2bd9e40 20 remove_watcher() i=5 2023-10-05T19:41:27.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.568+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:111 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94368265440976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f085d0 con 0x55d3d2ec3e40 2023-10-05T19:41:27.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.572+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(111 notify.6 [watch unwatch cookie 94368265440976] v30'233 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ec3e40 2023-10-05T19:41:27.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.572+0000 7f3de2bd9e40 20 remove_watcher() i=6 2023-10-05T19:41:27.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.572+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:112 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94368265443408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f085d0 con 0x55d3d2ec3e40 2023-10-05T19:41:27.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.576+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(112 notify.7 [watch unwatch cookie 94368265443408] v30'233 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ec3e40 2023-10-05T19:41:27.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.576+0000 7f3de2bd9e40 20 remove_watcher() i=7 2023-10-05T19:41:27.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.576+0000 7f3db1fe3700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:41:27.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.576+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:113 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94368265445712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55d3d2f052f0 con 0x55d3d2ef59e0 2023-10-05T19:41:27.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.580+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 <== osd.1 v2:172.21.15.12:6808/29933 39 ==== osd_op_reply(113 data_loggenerations_metadata [watch unwatch cookie 94368265445712] v30'226 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f3dd8014040 con 0x55d3d2ef59e0 2023-10-05T19:41:27.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.580+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55d3d2ec3e40 msgr2=0x7f3dd0012810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.580+0000 7f3de2bd9e40 1 --2- 172.21.15.12:0/3730364971 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55d3d2ec3e40 0x7f3dd0012810 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:27.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.580+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55d3d2ef59e0 msgr2=0x55d3d2ef7dc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.580+0000 7f3de2bd9e40 1 --2- 172.21.15.12:0/3730364971 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55d3d2ef59e0 0x55d3d2ef7dc0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:27.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.580+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55d3d2ef9ee0 msgr2=0x55d3d2efc2c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.580+0000 7f3de2bd9e40 1 --2- 172.21.15.12:0/3730364971 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55d3d2ef9ee0 0x55d3d2efc2c0 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:27.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.580+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55d3d2efe860 msgr2=0x55d3d2f00c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.580+0000 7f3de2bd9e40 1 --2- 172.21.15.12:0/3730364971 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55d3d2efe860 0x55d3d2f00c40 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:27.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.580+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3dc0030c00 msgr2=0x7f3dc00330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.580+0000 7f3de2bd9e40 1 --2- 172.21.15.12:0/3730364971 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3dc0030c00 0x7f3dc00330b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8005a90 tx=0x7f3dc8015000 comp rx=0 tx=0).stop 2023-10-05T19:41:27.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.580+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d3d2ec8420 msgr2=0x55d3d2e39b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.580+0000 7f3de0f0f700 1 -- 172.21.15.12:0/3730364971 reap_dead start 2023-10-05T19:41:27.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.580+0000 7f3de2bd9e40 1 --2- 172.21.15.12:0/3730364971 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d3d2ec8420 0x55d3d2e39b00 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0000f80 tx=0x7f3dd0014000 comp rx=0 tx=0).stop 2023-10-05T19:41:27.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.580+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 shutdown_connections 2023-10-05T19:41:27.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.580+0000 7f3de2bd9e40 1 --2- 172.21.15.12:0/3730364971 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d3d2ec8420 0x55d3d2e39b00 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:27.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.580+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 >> 172.21.15.12:0/3730364971 conn(0x55d3d2ebf860 msgr2=0x55d3d2edd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:27.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.580+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 shutdown_connections 2023-10-05T19:41:27.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.580+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3730364971 wait complete. 2023-10-05T19:41:27.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.580+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3211949211 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3d20030ca0 msgr2=0x7f3d20033150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.580+0000 7f3de2bd9e40 1 --2- 172.21.15.12:0/3211949211 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3d20030ca0 0x7f3d20033150 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x55d3d2f0d7e0 tx=0x7f3dd8032600 comp rx=0 tx=0).stop 2023-10-05T19:41:27.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.580+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3211949211 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d3d2f17c30 msgr2=0x55d3d2f1d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.580+0000 7f3de2bd9e40 1 --2- 172.21.15.12:0/3211949211 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d3d2f17c30 0x55d3d2f1d6b0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f3dc803a040 tx=0x7f3dc8032000 comp rx=0 tx=0).stop 2023-10-05T19:41:27.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.584+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3211949211 shutdown_connections 2023-10-05T19:41:27.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.584+0000 7f3de2bd9e40 1 --2- 172.21.15.12:0/3211949211 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3d20030ca0 0x7f3d20033150 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:27.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.584+0000 7f3de2bd9e40 1 --2- 172.21.15.12:0/3211949211 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d3d2f17c30 0x55d3d2f1d6b0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:27.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.584+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3211949211 >> 172.21.15.12:0/3211949211 conn(0x55d3d2f12ff0 msgr2=0x55d3d2f110f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:27.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.584+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3211949211 shutdown_connections 2023-10-05T19:41:27.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.584+0000 7f3de2bd9e40 1 -- 172.21.15.12:0/3211949211 wait complete. 2023-10-05T19:41:27.591 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:41:27.591 INFO:teuthology.orchestra.run.smithi012.stdout: "entries": [ 2023-10-05T19:41:27.591 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.591 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:41:27.591 INFO:teuthology.orchestra.run.smithi012.stdout: "buckets": [ 2023-10-05T19:41:27.591 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.591 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "", 2023-10-05T19:41:27.591 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:00:00.000000Z", 2023-10-05T19:41:27.592 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 1696532400, 2023-10-05T19:41:27.592 INFO:teuthology.orchestra.run.smithi012.stdout: "owner": "foo", 2023-10-05T19:41:27.592 INFO:teuthology.orchestra.run.smithi012.stdout: "categories": [] 2023-10-05T19:41:27.592 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:27.592 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.592 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo", 2023-10-05T19:41:27.592 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:00:00.000000Z", 2023-10-05T19:41:27.592 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 1696532400, 2023-10-05T19:41:27.592 INFO:teuthology.orchestra.run.smithi012.stdout: "owner": "foo", 2023-10-05T19:41:27.592 INFO:teuthology.orchestra.run.smithi012.stdout: "categories": [ 2023-10-05T19:41:27.592 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.592 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "put_obj", 2023-10-05T19:41:27.593 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:27.593 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 30000042, 2023-10-05T19:41:27.593 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 11, 2023-10-05T19:41:27.593 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 11 2023-10-05T19:41:27.593 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:27.593 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2023-10-05T19:41:27.593 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:27.593 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.593 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo2", 2023-10-05T19:41:27.593 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:00:00.000000Z", 2023-10-05T19:41:27.593 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 1696532400, 2023-10-05T19:41:27.593 INFO:teuthology.orchestra.run.smithi012.stdout: "owner": "foo", 2023-10-05T19:41:27.594 INFO:teuthology.orchestra.run.smithi012.stdout: "categories": [] 2023-10-05T19:41:27.594 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:27.594 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.594 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo3", 2023-10-05T19:41:27.594 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:00:00.000000Z", 2023-10-05T19:41:27.594 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 1696532400, 2023-10-05T19:41:27.594 INFO:teuthology.orchestra.run.smithi012.stdout: "owner": "foo", 2023-10-05T19:41:27.594 INFO:teuthology.orchestra.run.smithi012.stdout: "categories": [] 2023-10-05T19:41:27.594 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:27.594 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.594 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo4", 2023-10-05T19:41:27.594 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:00:00.000000Z", 2023-10-05T19:41:27.595 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 1696532400, 2023-10-05T19:41:27.595 INFO:teuthology.orchestra.run.smithi012.stdout: "owner": "foo", 2023-10-05T19:41:27.595 INFO:teuthology.orchestra.run.smithi012.stdout: "categories": [] 2023-10-05T19:41:27.595 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:27.595 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2023-10-05T19:41:27.595 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:27.595 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:41:27.595 INFO:teuthology.orchestra.run.smithi012.stdout: "summary": [ 2023-10-05T19:41:27.595 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.595 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:41:27.595 INFO:teuthology.orchestra.run.smithi012.stdout: "categories": [ 2023-10-05T19:41:27.595 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.596 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "put_obj", 2023-10-05T19:41:27.596 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:27.596 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 30000042, 2023-10-05T19:41:27.596 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 11, 2023-10-05T19:41:27.596 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 11 2023-10-05T19:41:27.596 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:27.596 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:41:27.596 INFO:teuthology.orchestra.run.smithi012.stdout: "total": { 2023-10-05T19:41:27.596 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:27.596 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 30000042, 2023-10-05T19:41:27.596 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 11, 2023-10-05T19:41:27.596 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 11 2023-10-05T19:41:27.596 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:27.597 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:27.597 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2023-10-05T19:41:27.597 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:41:27.598 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': '', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': [{'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}]}, {'bucket': 'myfoo2', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo3', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo4', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': []}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}], 'total': {'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}}]} 2023-10-05T19:41:27.598 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo', '--categories', 'delete_obj'] 2023-10-05T19:41:27.598 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo', '--categories', 'delete_obj'] 2023-10-05T19:41:27.598 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo --categories delete_obj 2023-10-05T19:41:27.680 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:41:27.680 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:41:27.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.676+0000 7fb61a038e40 1 Processor -- start 2023-10-05T19:41:27.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.676+0000 7fb61a038e40 1 -- start start 2023-10-05T19:41:27.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.676+0000 7fb61a038e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x560da0907e40 0x560da0908210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:27.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.676+0000 7fb61a038e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x560da0854bd0 con 0x560da0908750 2023-10-05T19:41:27.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.676+0000 7fb61a038e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x560da0811ee0 con 0x560da0907e40 2023-10-05T19:41:27.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.676+0000 7fb617b6d700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x560da0907e40 0x560da0908210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:27.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.676+0000 7fb617b6d700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x560da0907e40 0x560da0908210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:50574/0 (socket says 172.21.15.12:50574) 2023-10-05T19:41:27.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.676+0000 7fb617b6d700 1 -- 172.21.15.12:0/3230279404 learned_addr learned my addr 172.21.15.12:0/3230279404 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:27.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.676+0000 7fb61736c700 1 --1- 172.21.15.12:0/3230279404 >> v1:172.21.15.8:6789/0 conn(0x560da0908750 0x560da09091f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:57674/0 (socket says 172.21.15.12:57674) 2023-10-05T19:41:27.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.676+0000 7fb617b6d700 1 -- 172.21.15.12:0/3230279404 >> v1:172.21.15.8:6789/0 conn(0x560da0908750 legacy=0x560da09091f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.676+0000 7fb617b6d700 1 -- 172.21.15.12:0/3230279404 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560da0909900 con 0x560da0907e40 2023-10-05T19:41:27.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.676+0000 7fb617b6d700 1 --2- 172.21.15.12:0/3230279404 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560da0907e40 0x560da0908210 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb60800a650 tx=0x7fb608005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cc1aa5dea29b852 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:27.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.676+0000 7fb616b6b700 1 -- 172.21.15.12:0/3230279404 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb608008760 con 0x560da0907e40 2023-10-05T19:41:27.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.676+0000 7fb616b6b700 1 -- 172.21.15.12:0/3230279404 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6080088c0 con 0x560da0907e40 2023-10-05T19:41:27.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.676+0000 7fb616b6b700 1 -- 172.21.15.12:0/3230279404 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb608008bb0 con 0x560da0907e40 2023-10-05T19:41:27.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.680+0000 7fb61a038e40 1 -- 172.21.15.12:0/3230279404 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560da0907e40 msgr2=0x560da0908210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.680+0000 7fb61a038e40 1 --2- 172.21.15.12:0/3230279404 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560da0907e40 0x560da0908210 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb60800a650 tx=0x7fb608005b10 comp rx=0 tx=0).stop 2023-10-05T19:41:27.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.680+0000 7fb61a038e40 1 -- 172.21.15.12:0/3230279404 shutdown_connections 2023-10-05T19:41:27.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.680+0000 7fb61a038e40 1 --2- 172.21.15.12:0/3230279404 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560da0907e40 0x560da0908210 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:27.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.680+0000 7fb61a038e40 1 -- 172.21.15.12:0/3230279404 >> 172.21.15.12:0/3230279404 conn(0x560da0903860 msgr2=0x560da0905c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:27.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.680+0000 7fb61a038e40 1 -- 172.21.15.12:0/3230279404 shutdown_connections 2023-10-05T19:41:27.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.680+0000 7fb61a038e40 1 -- 172.21.15.12:0/3230279404 wait complete. 2023-10-05T19:41:27.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.680+0000 7fb61a038e40 1 Processor -- start 2023-10-05T19:41:27.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.680+0000 7fb61a038e40 1 -- start start 2023-10-05T19:41:27.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.680+0000 7fb61a038e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560da0907e40 0x560da0926f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:27.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.680+0000 7fb61a038e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560da07d5e30 con 0x560da0907e40 2023-10-05T19:41:27.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.680+0000 7fb617b6d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560da0907e40 0x560da0926f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:27.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.684+0000 7fb617b6d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560da0907e40 0x560da0926f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:50580/0 (socket says 172.21.15.12:50580) 2023-10-05T19:41:27.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.684+0000 7fb617b6d700 1 -- 172.21.15.12:0/3280799605 learned_addr learned my addr 172.21.15.12:0/3280799605 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:27.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.684+0000 7fb617b6d700 1 -- 172.21.15.12:0/3280799605 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb608005db0 con 0x560da0907e40 2023-10-05T19:41:27.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.684+0000 7fb617b6d700 1 --2- 172.21.15.12:0/3280799605 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560da0907e40 0x560da0926f40 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fb60800a5a0 tx=0x7fb608015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ce47a5004a4137a server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:27.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.684+0000 7fb615b69700 1 -- 172.21.15.12:0/3280799605 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6080055f0 con 0x560da0907e40 2023-10-05T19:41:27.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.684+0000 7fb615b69700 1 -- 172.21.15.12:0/3280799605 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb608008a30 con 0x560da0907e40 2023-10-05T19:41:27.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.684+0000 7fb615b69700 1 -- 172.21.15.12:0/3280799605 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb608008d00 con 0x560da0907e40 2023-10-05T19:41:27.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.684+0000 7fb61a038e40 1 -- 172.21.15.12:0/3280799605 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560da0907e40 msgr2=0x560da0926f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.684+0000 7fb61a038e40 1 --2- 172.21.15.12:0/3280799605 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560da0907e40 0x560da0926f40 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fb60800a5a0 tx=0x7fb608015000 comp rx=0 tx=0).stop 2023-10-05T19:41:27.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.684+0000 7fb61a038e40 1 -- 172.21.15.12:0/3280799605 shutdown_connections 2023-10-05T19:41:27.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.684+0000 7fb61a038e40 1 --2- 172.21.15.12:0/3280799605 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560da0907e40 0x560da0926f40 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:27.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.684+0000 7fb61a038e40 1 -- 172.21.15.12:0/3280799605 >> 172.21.15.12:0/3280799605 conn(0x560da0903860 msgr2=0x560da0922b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:27.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.684+0000 7fb61a038e40 1 -- 172.21.15.12:0/3280799605 shutdown_connections 2023-10-05T19:41:27.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.684+0000 7fb61a038e40 1 -- 172.21.15.12:0/3280799605 wait complete. 2023-10-05T19:41:27.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.684+0000 7fb61a038e40 1 Processor -- start 2023-10-05T19:41:27.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.684+0000 7fb61a038e40 1 -- start start 2023-10-05T19:41:27.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.688+0000 7fb61a038e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560da0907e40 0x560da092dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:27.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.688+0000 7fb61a038e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560da07a8ed0 con 0x560da0907e40 2023-10-05T19:41:27.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.688+0000 7fb617b6d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560da0907e40 0x560da092dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:27.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.688+0000 7fb617b6d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560da0907e40 0x560da092dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:50594/0 (socket says 172.21.15.12:50594) 2023-10-05T19:41:27.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.688+0000 7fb617b6d700 1 -- 172.21.15.12:0/1081639402 learned_addr learned my addr 172.21.15.12:0/1081639402 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:27.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.688+0000 7fb617b6d700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560da0909900 con 0x560da0907e40 2023-10-05T19:41:27.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.688+0000 7fb617b6d700 1 --2- 172.21.15.12:0/1081639402 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560da0907e40 0x560da092dba0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fb60800a5a0 tx=0x7fb60801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:27.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.688+0000 7fb60ffff700 1 -- 172.21.15.12:0/1081639402 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb608026b60 con 0x560da0907e40 2023-10-05T19:41:27.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.688+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560da092e0e0 con 0x560da0907e40 2023-10-05T19:41:27.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.688+0000 7fb60ffff700 1 -- 172.21.15.12:0/1081639402 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb608026cc0 con 0x560da0907e40 2023-10-05T19:41:27.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.688+0000 7fb60ffff700 1 -- 172.21.15.12:0/1081639402 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb608012450 con 0x560da0907e40 2023-10-05T19:41:27.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.688+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560da092e580 con 0x560da0907e40 2023-10-05T19:41:27.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.692+0000 7fb60ffff700 1 -- 172.21.15.12:0/1081639402 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fb608019070 con 0x560da0907e40 2023-10-05T19:41:27.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.692+0000 7fb61a038e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.692+0000 7fb60ffff700 1 --2- 172.21.15.12:0/1081639402 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fb5f8030c50 0x7fb5f8033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:27.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.692+0000 7fb60ffff700 1 -- 172.21.15.12:0/1081639402 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7fb6080466d0 con 0x560da0907e40 2023-10-05T19:41:27.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.692+0000 7fb61736c700 1 --2- 172.21.15.12:0/1081639402 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fb5f8030c50 0x7fb5f8033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:27.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.692+0000 7fb61a038e40 1 --2- 172.21.15.12:0/1081639402 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x560da0908750 0x560da0939690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:27.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.692+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0939d00 con 0x560da0908750 2023-10-05T19:41:27.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.692+0000 7fb61836e700 1 --2- 172.21.15.12:0/1081639402 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x560da0908750 0x560da0939690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:27.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.692+0000 7fb61836e700 1 --2- 172.21.15.12:0/1081639402 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x560da0908750 0x560da0939690 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:27.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.692+0000 7fb61736c700 1 --2- 172.21.15.12:0/1081639402 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fb5f8030c50 0x7fb5f8033100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb60000b300 tx=0x7fb600001430 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:27.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.692+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb61000a5a0 con 0x560da0908750 2023-10-05T19:41:27.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.692+0000 7fb61a038e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.692+0000 7fb61a038e40 20 realm 2023-10-05T19:41:27.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.692+0000 7fb61a038e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.692+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0936cf0 con 0x560da0908750 2023-10-05T19:41:27.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.692+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb61000a5a0 con 0x560da0908750 2023-10-05T19:41:27.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.692+0000 7fb61a038e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.692+0000 7fb61a038e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:41:27.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.692+0000 7fb61a038e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.692+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da093a7e0 con 0x560da0908750 2023-10-05T19:41:27.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.696+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb61000a5a0 con 0x560da0908750 2023-10-05T19:41:27.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.696+0000 7fb61a038e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.696+0000 7fb61a038e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.696+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da093a3c0 con 0x560da0908750 2023-10-05T19:41:27.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.696+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fb61000a5a0 con 0x560da0908750 2023-10-05T19:41:27.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.696+0000 7fb61a038e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:27.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.696+0000 7fb61a038e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.696+0000 7fb61a038e40 1 --2- 172.21.15.12:0/1081639402 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x560da093b770 0x560da093db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:27.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.696+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da093e2c0 con 0x560da093b770 2023-10-05T19:41:27.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.696+0000 7fb617b6d700 1 --2- 172.21.15.12:0/1081639402 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x560da093b770 0x560da093db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:27.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.696+0000 7fb617b6d700 1 --2- 172.21.15.12:0/1081639402 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x560da093b770 0x560da093db50 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:27.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.696+0000 7fb617b6d700 1 -- 172.21.15.12:0/1081639402 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fb608012c50 con 0x560da093b770 2023-10-05T19:41:27.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.696+0000 7fb61a038e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:41:27.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.696+0000 7fb61a038e40 20 searching for the correct realm 2023-10-05T19:41:27.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.696+0000 7fb61a038e40 1 --2- 172.21.15.12:0/1081639402 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x560da09400f0 0x560da09424d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:27.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.696+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0942a10 con 0x560da09400f0 2023-10-05T19:41:27.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.696+0000 7fb61736c700 1 --2- 172.21.15.12:0/1081639402 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x560da09400f0 0x560da09424d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:27.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.696+0000 7fb61736c700 1 --2- 172.21.15.12:0/1081639402 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x560da09400f0 0x560da09424d0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:27.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.700+0000 7fb61736c700 1 -- 172.21.15.12:0/1081639402 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fb60000f560 con 0x560da09400f0 2023-10-05T19:41:27.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.700+0000 7fb61736c700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0942a10 con 0x560da093b770 2023-10-05T19:41:27.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.700+0000 7fb617b6d700 1 -- 172.21.15.12:0/1081639402 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb608012c50 con 0x560da093b770 2023-10-05T19:41:27.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.700+0000 7fb617b6d700 1 --2- 172.21.15.12:0/1081639402 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fb608007030 0x7fb608001250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:27.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.700+0000 7fb617b6d700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0942a10 con 0x7fb608007030 2023-10-05T19:41:27.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.700+0000 7fb61836e700 1 --2- 172.21.15.12:0/1081639402 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fb608007030 0x7fb608001250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:27.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.700+0000 7fb61836e700 1 --2- 172.21.15.12:0/1081639402 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fb608007030 0x7fb608001250 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:27.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.704+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb61000a5a0 con 0x7fb608007030 2023-10-05T19:41:27.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.704+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0942a10 con 0x560da0908750 2023-10-05T19:41:27.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.704+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb61000a5a0 con 0x560da0908750 2023-10-05T19:41:27.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.704+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0942a10 con 0x560da0908750 2023-10-05T19:41:27.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.704+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb61000a5a0 con 0x560da0908750 2023-10-05T19:41:27.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.704+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0942a10 con 0x560da0908750 2023-10-05T19:41:27.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.704+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(11 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb61000a5a0 con 0x560da0908750 2023-10-05T19:41:27.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.704+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0942a10 con 0x7fb608007030 2023-10-05T19:41:27.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.704+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb61000a5a0 con 0x7fb608007030 2023-10-05T19:41:27.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.704+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0942a10 con 0x560da093b770 2023-10-05T19:41:27.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.704+0000 7fb617b6d700 1 -- 172.21.15.12:0/1081639402 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb608012c50 con 0x560da093b770 2023-10-05T19:41:27.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.708+0000 7fb617b6d700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0942a10 con 0x560da093b770 2023-10-05T19:41:27.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.708+0000 7fb617b6d700 1 -- 172.21.15.12:0/1081639402 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb608012c50 con 0x560da093b770 2023-10-05T19:41:27.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.708+0000 7fb617b6d700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0942a10 con 0x560da0908750 2023-10-05T19:41:27.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.708+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb61000a5a0 con 0x560da0908750 2023-10-05T19:41:27.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.708+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0942a10 con 0x7fb608007030 2023-10-05T19:41:27.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.708+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb61000a5a0 con 0x7fb608007030 2023-10-05T19:41:27.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.708+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0942a10 con 0x560da0908750 2023-10-05T19:41:27.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.708+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb61000a5a0 con 0x560da0908750 2023-10-05T19:41:27.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.708+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0942a10 con 0x560da093b770 2023-10-05T19:41:27.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.708+0000 7fb617b6d700 1 -- 172.21.15.12:0/1081639402 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(18 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb608012c50 con 0x560da093b770 2023-10-05T19:41:27.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.708+0000 7fb617b6d700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0942a10 con 0x560da09400f0 2023-10-05T19:41:27.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.712+0000 7fb61736c700 1 -- 172.21.15.12:0/1081639402 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb60000f560 con 0x560da09400f0 2023-10-05T19:41:27.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.712+0000 7fb61736c700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0942a10 con 0x560da09400f0 2023-10-05T19:41:27.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.712+0000 7fb61736c700 1 -- 172.21.15.12:0/1081639402 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb60000f560 con 0x560da09400f0 2023-10-05T19:41:27.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.712+0000 7fb61736c700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0942a10 con 0x7fb608007030 2023-10-05T19:41:27.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.712+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb61000a5a0 con 0x7fb608007030 2023-10-05T19:41:27.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.712+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0942a10 con 0x560da09400f0 2023-10-05T19:41:27.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.712+0000 7fb61736c700 1 -- 172.21.15.12:0/1081639402 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb60000f560 con 0x560da09400f0 2023-10-05T19:41:27.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.712+0000 7fb61736c700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0942a10 con 0x560da09400f0 2023-10-05T19:41:27.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.712+0000 7fb61736c700 1 -- 172.21.15.12:0/1081639402 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb60000f560 con 0x560da09400f0 2023-10-05T19:41:27.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.712+0000 7fb61736c700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0942a10 con 0x560da0908750 2023-10-05T19:41:27.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.716+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb61000a5a0 con 0x560da0908750 2023-10-05T19:41:27.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.716+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0942a10 con 0x560da0908750 2023-10-05T19:41:27.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.716+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb61000a5a0 con 0x560da0908750 2023-10-05T19:41:27.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.716+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0942a10 con 0x7fb608007030 2023-10-05T19:41:27.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.716+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb61000a5a0 con 0x7fb608007030 2023-10-05T19:41:27.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.716+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0942a10 con 0x560da09400f0 2023-10-05T19:41:27.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.716+0000 7fb61736c700 1 -- 172.21.15.12:0/1081639402 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb60000f560 con 0x560da09400f0 2023-10-05T19:41:27.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.716+0000 7fb61736c700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0942a10 con 0x560da093b770 2023-10-05T19:41:27.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.720+0000 7fb617b6d700 1 -- 172.21.15.12:0/1081639402 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb608012c50 con 0x560da093b770 2023-10-05T19:41:27.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.720+0000 7fb617b6d700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0942a10 con 0x560da0908750 2023-10-05T19:41:27.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.720+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb61000a5a0 con 0x560da0908750 2023-10-05T19:41:27.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.720+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0942a10 con 0x7fb608007030 2023-10-05T19:41:27.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.720+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb61000a5a0 con 0x7fb608007030 2023-10-05T19:41:27.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.720+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0942a10 con 0x560da09400f0 2023-10-05T19:41:27.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.720+0000 7fb61736c700 1 -- 172.21.15.12:0/1081639402 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb60000f560 con 0x560da09400f0 2023-10-05T19:41:27.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.720+0000 7fb61736c700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0942a10 con 0x7fb608007030 2023-10-05T19:41:27.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.724+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb61000a5a0 con 0x7fb608007030 2023-10-05T19:41:27.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.724+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0942a10 con 0x560da093b770 2023-10-05T19:41:27.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.724+0000 7fb617b6d700 1 -- 172.21.15.12:0/1081639402 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb608012c50 con 0x560da093b770 2023-10-05T19:41:27.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.724+0000 7fb617b6d700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0942a10 con 0x560da09400f0 2023-10-05T19:41:27.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.724+0000 7fb61736c700 1 -- 172.21.15.12:0/1081639402 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb60000f560 con 0x560da09400f0 2023-10-05T19:41:27.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.724+0000 7fb61736c700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0942a10 con 0x7fb608007030 2023-10-05T19:41:27.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.724+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb61000a5a0 con 0x7fb608007030 2023-10-05T19:41:27.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.724+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0942a10 con 0x560da0908750 2023-10-05T19:41:27.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.724+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fb61000a5a0 con 0x560da0908750 2023-10-05T19:41:27.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.724+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0942a10 con 0x560da093b770 2023-10-05T19:41:27.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.728+0000 7fb617b6d700 1 -- 172.21.15.12:0/1081639402 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(37 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fb608012c50 con 0x560da093b770 2023-10-05T19:41:27.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.728+0000 7fb61a038e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:41:27.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.728+0000 7fb61a038e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:41:27.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.728+0000 7fb61a038e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:41:27.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.728+0000 7fb61a038e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:41:27.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.728+0000 7fb61a038e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.728+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da093ee70 con 0x560da0908750 2023-10-05T19:41:27.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.728+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb61000a5a0 con 0x560da0908750 2023-10-05T19:41:27.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.728+0000 7fb61a038e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.728+0000 7fb61a038e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.728+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da093f7c0 con 0x560da0908750 2023-10-05T19:41:27.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.728+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fb61000a5a0 con 0x560da0908750 2023-10-05T19:41:27.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.728+0000 7fb61a038e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:27.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.728+0000 7fb61a038e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.728+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da093ea30 con 0x560da09400f0 2023-10-05T19:41:27.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.728+0000 7fb61736c700 1 -- 172.21.15.12:0/1081639402 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fb60000f560 con 0x560da09400f0 2023-10-05T19:41:27.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.728+0000 7fb61a038e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:41:27.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.728+0000 7fb61a038e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.728+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0943340 con 0x560da09400f0 2023-10-05T19:41:27.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.728+0000 7fb61736c700 1 -- 172.21.15.12:0/1081639402 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7fb60000f560 con 0x560da09400f0 2023-10-05T19:41:27.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.728+0000 7fb61a038e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.728+0000 7fb61a038e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.728+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da09436f0 con 0x560da0908750 2023-10-05T19:41:27.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.732+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb61000a5a0 con 0x560da0908750 2023-10-05T19:41:27.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.732+0000 7fb61a038e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.732+0000 7fb61a038e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.732+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0943aa0 con 0x560da0908750 2023-10-05T19:41:27.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.732+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fb61000a5a0 con 0x560da0908750 2023-10-05T19:41:27.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.732+0000 7fb61a038e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:27.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.732+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0944ab0 con 0x560da09400f0 2023-10-05T19:41:27.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.736+0000 7fb61736c700 1 -- 172.21.15.12:0/1081639402 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(44 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fb600027080 con 0x560da09400f0 2023-10-05T19:41:27.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.736+0000 7fb61736c700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0944ab0 con 0x560da093b770 2023-10-05T19:41:27.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.736+0000 7fb617b6d700 1 -- 172.21.15.12:0/1081639402 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(45 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb608012c50 con 0x560da093b770 2023-10-05T19:41:27.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.736+0000 7fb617b6d700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0944ab0 con 0x7fb608007030 2023-10-05T19:41:27.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.736+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(46 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb61000a5a0 con 0x7fb608007030 2023-10-05T19:41:27.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.736+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da09436f0 con 0x560da0908750 2023-10-05T19:41:27.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.736+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(47 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb61000a5a0 con 0x560da0908750 2023-10-05T19:41:27.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.736+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da09436f0 con 0x560da0908750 2023-10-05T19:41:27.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.736+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(48 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb61000a5a0 con 0x560da0908750 2023-10-05T19:41:27.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.736+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da09436f0 con 0x560da0908750 2023-10-05T19:41:27.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.740+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(49 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb61000a5a0 con 0x560da0908750 2023-10-05T19:41:27.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.740+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da09436f0 con 0x7fb608007030 2023-10-05T19:41:27.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.740+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(50 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb61000a5a0 con 0x7fb608007030 2023-10-05T19:41:27.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.740+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da09436f0 con 0x560da093b770 2023-10-05T19:41:27.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.740+0000 7fb617b6d700 1 -- 172.21.15.12:0/1081639402 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(51 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb608012c50 con 0x560da093b770 2023-10-05T19:41:27.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.740+0000 7fb617b6d700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da09436f0 con 0x560da093b770 2023-10-05T19:41:27.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.740+0000 7fb617b6d700 1 -- 172.21.15.12:0/1081639402 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(52 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb608012c50 con 0x560da093b770 2023-10-05T19:41:27.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.740+0000 7fb617b6d700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da09436f0 con 0x560da0908750 2023-10-05T19:41:27.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.740+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(53 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb61000a5a0 con 0x560da0908750 2023-10-05T19:41:27.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.740+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da09436f0 con 0x7fb608007030 2023-10-05T19:41:27.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.744+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(54 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb61000a5a0 con 0x7fb608007030 2023-10-05T19:41:27.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.744+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da09436f0 con 0x560da0908750 2023-10-05T19:41:27.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.744+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(55 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb61000a5a0 con 0x560da0908750 2023-10-05T19:41:27.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.744+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da09436f0 con 0x560da093b770 2023-10-05T19:41:27.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.744+0000 7fb617b6d700 1 -- 172.21.15.12:0/1081639402 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(56 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb608012c50 con 0x560da093b770 2023-10-05T19:41:27.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.744+0000 7fb617b6d700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da09436f0 con 0x560da09400f0 2023-10-05T19:41:27.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.744+0000 7fb61736c700 1 -- 172.21.15.12:0/1081639402 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(57 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb600027080 con 0x560da09400f0 2023-10-05T19:41:27.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.744+0000 7fb61736c700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da09436f0 con 0x560da09400f0 2023-10-05T19:41:27.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.744+0000 7fb61736c700 1 -- 172.21.15.12:0/1081639402 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(58 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb600027080 con 0x560da09400f0 2023-10-05T19:41:27.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.744+0000 7fb61736c700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da09436f0 con 0x7fb608007030 2023-10-05T19:41:27.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.748+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(59 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb61000a5a0 con 0x7fb608007030 2023-10-05T19:41:27.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.748+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da09436f0 con 0x560da09400f0 2023-10-05T19:41:27.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.748+0000 7fb61736c700 1 -- 172.21.15.12:0/1081639402 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(60 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb600027080 con 0x560da09400f0 2023-10-05T19:41:27.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.748+0000 7fb61736c700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da09436f0 con 0x560da09400f0 2023-10-05T19:41:27.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.748+0000 7fb61736c700 1 -- 172.21.15.12:0/1081639402 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(61 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb600027080 con 0x560da09400f0 2023-10-05T19:41:27.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.748+0000 7fb61736c700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da09436f0 con 0x560da0908750 2023-10-05T19:41:27.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.748+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(62 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb61000a5a0 con 0x560da0908750 2023-10-05T19:41:27.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.748+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da09436f0 con 0x560da0908750 2023-10-05T19:41:27.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.748+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(63 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb61000a5a0 con 0x560da0908750 2023-10-05T19:41:27.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.748+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da09436f0 con 0x7fb608007030 2023-10-05T19:41:27.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.752+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(64 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb61000a5a0 con 0x7fb608007030 2023-10-05T19:41:27.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.752+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da09436f0 con 0x560da09400f0 2023-10-05T19:41:27.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.752+0000 7fb61736c700 1 -- 172.21.15.12:0/1081639402 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(65 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb600027080 con 0x560da09400f0 2023-10-05T19:41:27.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.752+0000 7fb61736c700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da09436f0 con 0x560da093b770 2023-10-05T19:41:27.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.752+0000 7fb617b6d700 1 -- 172.21.15.12:0/1081639402 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(66 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb608012c50 con 0x560da093b770 2023-10-05T19:41:27.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.752+0000 7fb617b6d700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da09436f0 con 0x560da0908750 2023-10-05T19:41:27.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.752+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(67 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb61000a5a0 con 0x560da0908750 2023-10-05T19:41:27.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.752+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da09436f0 con 0x7fb608007030 2023-10-05T19:41:27.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.756+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(68 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb61000a5a0 con 0x7fb608007030 2023-10-05T19:41:27.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.756+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da09436f0 con 0x560da09400f0 2023-10-05T19:41:27.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.756+0000 7fb61736c700 1 -- 172.21.15.12:0/1081639402 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(69 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb600027080 con 0x560da09400f0 2023-10-05T19:41:27.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.756+0000 7fb61736c700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da09436f0 con 0x7fb608007030 2023-10-05T19:41:27.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.756+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(70 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb61000a5a0 con 0x7fb608007030 2023-10-05T19:41:27.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.756+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da09436f0 con 0x560da093b770 2023-10-05T19:41:27.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.756+0000 7fb617b6d700 1 -- 172.21.15.12:0/1081639402 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(71 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb608012c50 con 0x560da093b770 2023-10-05T19:41:27.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.756+0000 7fb617b6d700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da09436f0 con 0x560da09400f0 2023-10-05T19:41:27.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.760+0000 7fb61736c700 1 -- 172.21.15.12:0/1081639402 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(72 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb600027080 con 0x560da09400f0 2023-10-05T19:41:27.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.760+0000 7fb61736c700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da09436f0 con 0x7fb608007030 2023-10-05T19:41:27.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.760+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(73 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb61000a5a0 con 0x7fb608007030 2023-10-05T19:41:27.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.760+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da09436f0 con 0x560da0908750 2023-10-05T19:41:27.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.760+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(74 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fb61000a5a0 con 0x560da0908750 2023-10-05T19:41:27.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.760+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da09436f0 con 0x560da093b770 2023-10-05T19:41:27.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.760+0000 7fb617b6d700 1 -- 172.21.15.12:0/1081639402 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(75 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fb608012c50 con 0x560da093b770 2023-10-05T19:41:27.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.760+0000 7fb61a038e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:41:27.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.760+0000 7fb61a038e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:41:27.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.760+0000 7fb61a038e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:41:27.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.760+0000 7fb61a038e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:41:27.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.760+0000 7fb61a038e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.760+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0943aa0 con 0x560da0908750 2023-10-05T19:41:27.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.764+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fb61000a5a0 con 0x560da0908750 2023-10-05T19:41:27.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.764+0000 7fb61a038e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:27.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.764+0000 7fb61a038e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.764+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0943e50 con 0x560da093b770 2023-10-05T19:41:27.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.764+0000 7fb617b6d700 1 -- 172.21.15.12:0/1081639402 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fb608012c50 con 0x560da093b770 2023-10-05T19:41:27.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.764+0000 7fb61a038e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:41:27.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.764+0000 7fb61a038e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.764+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da09446f0 con 0x560da0908750 2023-10-05T19:41:27.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.764+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fb61000a5a0 con 0x560da0908750 2023-10-05T19:41:27.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.764+0000 7fb61a038e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:27.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.764+0000 7fb61a038e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.764+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da09446f0 con 0x560da09400f0 2023-10-05T19:41:27.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.764+0000 7fb61736c700 1 -- 172.21.15.12:0/1081639402 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fb600027080 con 0x560da09400f0 2023-10-05T19:41:27.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.764+0000 7fb61a038e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:41:27.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.764+0000 7fb61a038e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.764+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da09442b0 con 0x7fb608007030 2023-10-05T19:41:27.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.764+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7fb61000a5a0 con 0x7fb608007030 2023-10-05T19:41:27.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.764+0000 7fb61a038e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.764+0000 7fb61a038e40 20 zone default found 2023-10-05T19:41:27.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.764+0000 7fb61a038e40 4 Realm: () 2023-10-05T19:41:27.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.764+0000 7fb61a038e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:41:27.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.764+0000 7fb61a038e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:41:27.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.764+0000 7fb61a038e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:41:27.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.764+0000 7fb61a038e40 20 zonegroup default 2023-10-05T19:41:27.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.764+0000 7fb61a038e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.764+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da09442b0 con 0x7fb608007030 2023-10-05T19:41:27.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.768+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fb61000a5a0 con 0x7fb608007030 2023-10-05T19:41:27.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.768+0000 7fb61a038e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.768+0000 7fb61a038e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.768+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da09442b0 con 0x560da09400f0 2023-10-05T19:41:27.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.768+0000 7fb61736c700 1 -- 172.21.15.12:0/1081639402 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fb600027080 con 0x560da09400f0 2023-10-05T19:41:27.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.768+0000 7fb61a038e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.768+0000 7fb61a038e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.768+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0946910 con 0x560da09400f0 2023-10-05T19:41:27.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.768+0000 7fb61736c700 1 -- 172.21.15.12:0/1081639402 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fb600027080 con 0x560da09400f0 2023-10-05T19:41:27.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.768+0000 7fb61a038e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.768+0000 7fb61a038e40 20 started sync module instance, tier type = 2023-10-05T19:41:27.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.768+0000 7fb61a038e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:41:27.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.768+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0945fd0 con 0x560da093b770 2023-10-05T19:41:27.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.772+0000 7fb617b6d700 1 -- 172.21.15.12:0/1081639402 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(84 notify.0 [create] v30'234 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb608012c50 con 0x560da093b770 2023-10-05T19:41:27.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.772+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94616528640736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0946d00 con 0x560da093b770 2023-10-05T19:41:27.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.772+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0947750 con 0x560da0908750 2023-10-05T19:41:27.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.776+0000 7fb617b6d700 1 -- 172.21.15.12:0/1081639402 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(85 notify.0 [watch watch cookie 94616528640736] v30'235 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb608012c50 con 0x560da093b770 2023-10-05T19:41:27.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.776+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(86 notify.1 [create] v30'234 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb61000a5a0 con 0x560da0908750 2023-10-05T19:41:27.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.776+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94616528646688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0947b50 con 0x560da0908750 2023-10-05T19:41:27.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.776+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0948430 con 0x560da09400f0 2023-10-05T19:41:27.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.776+0000 7fb61736c700 1 -- 172.21.15.12:0/1081639402 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(88 notify.2 [create] v30'234 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb600027080 con 0x560da09400f0 2023-10-05T19:41:27.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.776+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94616528649328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0948d20 con 0x560da09400f0 2023-10-05T19:41:27.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.776+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0949730 con 0x560da093b770 2023-10-05T19:41:27.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.780+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(87 notify.1 [watch watch cookie 94616528646688] v30'235 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb61000a5a0 con 0x560da0908750 2023-10-05T19:41:27.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.780+0000 7fb61736c700 1 -- 172.21.15.12:0/1081639402 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(89 notify.2 [watch watch cookie 94616528649328] v30'235 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb600027080 con 0x560da09400f0 2023-10-05T19:41:27.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.780+0000 7fb617b6d700 1 -- 172.21.15.12:0/1081639402 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(90 notify.3 [create] v30'234 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb608012c50 con 0x560da093b770 2023-10-05T19:41:27.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.780+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94616528654912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0948950 con 0x560da093b770 2023-10-05T19:41:27.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.780+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da094a140 con 0x560da0908750 2023-10-05T19:41:27.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.784+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(92 notify.4 [create] v30'234 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb61000a5a0 con 0x560da0908750 2023-10-05T19:41:27.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.784+0000 7fb617b6d700 1 -- 172.21.15.12:0/1081639402 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(91 notify.3 [watch watch cookie 94616528654912] v30'235 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb608012c50 con 0x560da093b770 2023-10-05T19:41:27.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.784+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94616528657488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da094a0a0 con 0x560da0908750 2023-10-05T19:41:27.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.784+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da094aab0 con 0x7fb608007030 2023-10-05T19:41:27.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.788+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(93 notify.4 [watch watch cookie 94616528657488] v30'235 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb61000a5a0 con 0x560da0908750 2023-10-05T19:41:27.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.788+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(94 notify.5 [create] v30'234 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb61000a5a0 con 0x7fb608007030 2023-10-05T19:41:27.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.788+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94616528642944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da094aab0 con 0x7fb608007030 2023-10-05T19:41:27.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.788+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da094b350 con 0x7fb608007030 2023-10-05T19:41:27.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.792+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(96 notify.6 [create] v30'234 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb61000a5a0 con 0x7fb608007030 2023-10-05T19:41:27.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.792+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94616528661728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da094b660 con 0x7fb608007030 2023-10-05T19:41:27.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.792+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da094c1f0 con 0x7fb608007030 2023-10-05T19:41:27.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.792+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(95 notify.5 [watch watch cookie 94616528642944] v30'235 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb61000a5a0 con 0x7fb608007030 2023-10-05T19:41:27.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.792+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(97 notify.6 [watch watch cookie 94616528661728] v30'235 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb61000a5a0 con 0x7fb608007030 2023-10-05T19:41:27.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.792+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(98 notify.7 [create] v30'234 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb61000a5a0 con 0x7fb608007030 2023-10-05T19:41:27.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.792+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94616528659904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da094bcc0 con 0x7fb608007030 2023-10-05T19:41:27.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.792+0000 7fb61a038e40 20 add_watcher() i=0 2023-10-05T19:41:27.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.792+0000 7fb61a038e40 20 add_watcher() i=1 2023-10-05T19:41:27.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.792+0000 7fb61a038e40 20 add_watcher() i=2 2023-10-05T19:41:27.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.792+0000 7fb61a038e40 20 add_watcher() i=3 2023-10-05T19:41:27.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.792+0000 7fb61a038e40 20 add_watcher() i=4 2023-10-05T19:41:27.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.792+0000 7fb61a038e40 20 add_watcher() i=5 2023-10-05T19:41:27.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.792+0000 7fb61a038e40 20 add_watcher() i=6 2023-10-05T19:41:27.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.796+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(99 notify.7 [watch watch cookie 94616528659904] v30'235 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb61000a5a0 con 0x7fb608007030 2023-10-05T19:41:27.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.796+0000 7fb61a038e40 20 add_watcher() i=7 2023-10-05T19:41:27.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.796+0000 7fb61a038e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:41:27.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.796+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da094f1e0 con 0x560da0908750 2023-10-05T19:41:27.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.796+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fb61000a5a0 con 0x560da0908750 2023-10-05T19:41:27.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.796+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94616528665792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da094c600 con 0x560da0908750 2023-10-05T19:41:27.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.800+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94616528665792] v30'227 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fb61000a5a0 con 0x560da0908750 2023-10-05T19:41:27.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.800+0000 7fb5e97e2700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:41:27.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.800+0000 7fb61a038e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:41:27.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.800+0000 7fb61a038e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:41:27.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.800+0000 7fb61a038e40 1 Processor -- start 2023-10-05T19:41:27.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.800+0000 7fb61a038e40 1 -- start start 2023-10-05T19:41:27.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.800+0000 7fb61a038e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560da0959480 0x560da0959850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:27.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.800+0000 7fb61a038e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560da0959d90 con 0x560da0959480 2023-10-05T19:41:27.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.800+0000 7fb61736c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560da0959480 0x560da0959850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:27.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.800+0000 7fb61736c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560da0959480 0x560da0959850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:50610/0 (socket says 172.21.15.12:50610) 2023-10-05T19:41:27.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.800+0000 7fb61736c700 1 -- 172.21.15.12:0/1101601642 learned_addr learned my addr 172.21.15.12:0/1101601642 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.800+0000 7fb61736c700 1 -- 172.21.15.12:0/1101601642 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560da0959ed0 con 0x560da0959480 2023-10-05T19:41:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.800+0000 7fb61736c700 1 --2- 172.21.15.12:0/1101601642 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560da0959480 0x560da0959850 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fb60000bf50 tx=0x7fb600010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6463a97204afcce server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.800+0000 7fb5e8fe1700 1 -- 172.21.15.12:0/1101601642 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb600040070 con 0x560da0959480 2023-10-05T19:41:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.800+0000 7fb5e8fe1700 1 -- 172.21.15.12:0/1101601642 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb600025040 con 0x560da0959480 2023-10-05T19:41:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.800+0000 7fb5e8fe1700 1 -- 172.21.15.12:0/1101601642 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb600024040 con 0x560da0959480 2023-10-05T19:41:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.804+0000 7fb61a038e40 1 -- 172.21.15.12:0/1101601642 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560da0959480 msgr2=0x560da0959850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.804+0000 7fb61a038e40 1 --2- 172.21.15.12:0/1101601642 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560da0959480 0x560da0959850 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fb60000bf50 tx=0x7fb600010000 comp rx=0 tx=0).stop 2023-10-05T19:41:27.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.804+0000 7fb61a038e40 1 -- 172.21.15.12:0/1101601642 shutdown_connections 2023-10-05T19:41:27.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.804+0000 7fb61a038e40 1 --2- 172.21.15.12:0/1101601642 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560da0959480 0x560da0959850 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:27.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.804+0000 7fb61a038e40 1 -- 172.21.15.12:0/1101601642 >> 172.21.15.12:0/1101601642 conn(0x560da0954840 msgr2=0x560da09529f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:27.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.804+0000 7fb61a038e40 1 -- 172.21.15.12:0/1101601642 shutdown_connections 2023-10-05T19:41:27.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.804+0000 7fb61a038e40 1 -- 172.21.15.12:0/1101601642 wait complete. 2023-10-05T19:41:27.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.804+0000 7fb61a038e40 1 Processor -- start 2023-10-05T19:41:27.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.804+0000 7fb61a038e40 1 -- start start 2023-10-05T19:41:27.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.804+0000 7fb61a038e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560da0959480 0x560da095efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:27.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.804+0000 7fb61a038e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560da095ffe0 con 0x560da0959480 2023-10-05T19:41:27.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.804+0000 7fb61736c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560da0959480 0x560da095efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:27.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.804+0000 7fb61736c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560da0959480 0x560da095efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:50614/0 (socket says 172.21.15.12:50614) 2023-10-05T19:41:27.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.804+0000 7fb61736c700 1 -- 172.21.15.12:0/2144673840 learned_addr learned my addr 172.21.15.12:0/2144673840 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:27.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.804+0000 7fb61736c700 1 -- 172.21.15.12:0/2144673840 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb600006070 con 0x560da0959480 2023-10-05T19:41:27.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.804+0000 7fb61736c700 1 --2- 172.21.15.12:0/2144673840 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560da0959480 0x560da095efe0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x560da0953560 tx=0x7fb60003c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:27.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.804+0000 7fb562ffd700 1 -- 172.21.15.12:0/2144673840 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb600024040 con 0x560da0959480 2023-10-05T19:41:27.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.804+0000 7fb562ffd700 1 -- 172.21.15.12:0/2144673840 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb600039930 con 0x560da0959480 2023-10-05T19:41:27.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.804+0000 7fb562ffd700 1 -- 172.21.15.12:0/2144673840 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb600039ae0 con 0x560da0959480 2023-10-05T19:41:27.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.804+0000 7fb61a038e40 1 -- 172.21.15.12:0/2144673840 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560da095f5e0 con 0x560da0959480 2023-10-05T19:41:27.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.804+0000 7fb61a038e40 1 -- 172.21.15.12:0/2144673840 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560da095fa80 con 0x560da0959480 2023-10-05T19:41:27.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.808+0000 7fb562ffd700 1 -- 172.21.15.12:0/2144673840 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fb600043020 con 0x560da0959480 2023-10-05T19:41:27.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.808+0000 7fb562ffd700 1 --2- 172.21.15.12:0/2144673840 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fb558030c50 0x7fb558033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:27.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.808+0000 7fb562ffd700 1 -- 172.21.15.12:0/2144673840 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7fb600022040 con 0x560da0959480 2023-10-05T19:41:27.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.808+0000 7fb61a038e40 5 note: GC not initialized 2023-10-05T19:41:27.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.808+0000 7fb61836e700 1 --2- 172.21.15.12:0/2144673840 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fb558030c50 0x7fb558033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:27.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.808+0000 7fb61836e700 1 --2- 172.21.15.12:0/2144673840 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fb558030c50 0x7fb558033100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x560da094b3f0 tx=0x7fb61000ae00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:27.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.808+0000 7fb557fff700 20 reqs_thread_entry: start 2023-10-05T19:41:27.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.812+0000 7fb61a038e40 20 init_complete bucket index max shards: 11 2023-10-05T19:41:27.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.812+0000 7fb555ffb700 20 reqs_thread_entry: start 2023-10-05T19:41:27.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.812+0000 7fb61a038e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:41:27.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.812+0000 7fb53ffff700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:41:58 2023 2023-10-05T19:41:27.817 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:41:27.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.812+0000 7fb53f7fe700 20 reqs_thread_entry: start 2023-10-05T19:41:27.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.812+0000 7fb61a038e40 20 get_system_obj_state: rctx=0x7ffcfe6bfcc8 obj=default.rgw.meta:users.uid:foo state=0x560da07e10b0 s->prefetch_data=0 2023-10-05T19:41:27.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.812+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da09c7890 con 0x560da0908750 2023-10-05T19:41:27.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.812+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(102 foo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv15 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7fb61000a5a0 con 0x560da0908750 2023-10-05T19:41:27.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.812+0000 7fb61a038e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:41:27.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.812+0000 7fb61a038e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:41:27.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.812+0000 7fb61a038e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.812+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da09c7880 con 0x560da0908750 2023-10-05T19:41:27.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.812+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(103 foo [call,call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv15 ondisk = 0) v8 ==== 273+0+480 (crc 0 0 0) 0x7fb61000a5a0 con 0x560da0908750 2023-10-05T19:41:27.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.812+0000 7fb61a038e40 20 rados_obj.operate() r=0 bl.length=357 2023-10-05T19:41:27.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.812+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:104 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da09c9310 con 0x7fb608007030 2023-10-05T19:41:27.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.812+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(104 usage.10 [call out=1268b] v0'0 uv32 ondisk = 0) v8 ==== 152+0+1268 (crc 0 0 0) 0x7fb61000a5a0 con 0x7fb608007030 2023-10-05T19:41:27.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.816+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:105 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94616528640736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0985c80 con 0x560da093b770 2023-10-05T19:41:27.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.816+0000 7fb617b6d700 1 -- 172.21.15.12:0/1081639402 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(105 notify.0 [watch unwatch cookie 94616528640736] v30'236 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb608012c50 con 0x560da093b770 2023-10-05T19:41:27.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.816+0000 7fb61a038e40 20 remove_watcher() i=0 2023-10-05T19:41:27.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.816+0000 7fb61a038e40 2 removed watcher, disabling cache 2023-10-05T19:41:27.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.816+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:106 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94616528646688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0945ae0 con 0x560da0908750 2023-10-05T19:41:27.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.820+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(106 notify.1 [watch unwatch cookie 94616528646688] v30'236 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb61000a5a0 con 0x560da0908750 2023-10-05T19:41:27.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.820+0000 7fb61a038e40 20 remove_watcher() i=1 2023-10-05T19:41:27.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.820+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:107 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94616528649328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0945ae0 con 0x560da09400f0 2023-10-05T19:41:27.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.824+0000 7fb61736c700 1 -- 172.21.15.12:0/1081639402 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(107 notify.2 [watch unwatch cookie 94616528649328] v30'236 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb600027080 con 0x560da09400f0 2023-10-05T19:41:27.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.824+0000 7fb61a038e40 20 remove_watcher() i=2 2023-10-05T19:41:27.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.824+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:108 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94616528654912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0947c70 con 0x560da093b770 2023-10-05T19:41:27.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.824+0000 7fb617b6d700 1 -- 172.21.15.12:0/1081639402 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(108 notify.3 [watch unwatch cookie 94616528654912] v30'236 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb608012c50 con 0x560da093b770 2023-10-05T19:41:27.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.824+0000 7fb61a038e40 20 remove_watcher() i=3 2023-10-05T19:41:27.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.824+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:109 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94616528657488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0945ae0 con 0x560da0908750 2023-10-05T19:41:27.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.828+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== osd_op_reply(109 notify.4 [watch unwatch cookie 94616528657488] v30'236 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb61000a5a0 con 0x560da0908750 2023-10-05T19:41:27.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.828+0000 7fb61a038e40 20 remove_watcher() i=4 2023-10-05T19:41:27.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.828+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:110 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94616528642944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0949240 con 0x7fb608007030 2023-10-05T19:41:27.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.832+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(110 notify.5 [watch unwatch cookie 94616528642944] v30'236 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb61000a5a0 con 0x7fb608007030 2023-10-05T19:41:27.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.832+0000 7fb61a038e40 20 remove_watcher() i=5 2023-10-05T19:41:27.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.832+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:111 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94616528661728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0947c70 con 0x7fb608007030 2023-10-05T19:41:27.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.836+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(111 notify.6 [watch unwatch cookie 94616528661728] v30'236 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb61000a5a0 con 0x7fb608007030 2023-10-05T19:41:27.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.836+0000 7fb61a038e40 20 remove_watcher() i=6 2023-10-05T19:41:27.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.836+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:112 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94616528659904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0949c50 con 0x7fb608007030 2023-10-05T19:41:27.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.836+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(112 notify.7 [watch unwatch cookie 94616528659904] v30'236 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb61000a5a0 con 0x7fb608007030 2023-10-05T19:41:27.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.840+0000 7fb61a038e40 20 remove_watcher() i=7 2023-10-05T19:41:27.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.840+0000 7fb5e97e2700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:41:27.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.840+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:113 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94616528665792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x560da0986c30 con 0x560da0908750 2023-10-05T19:41:27.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.844+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 <== osd.1 v2:172.21.15.12:6808/29933 39 ==== osd_op_reply(113 data_loggenerations_metadata [watch unwatch cookie 94616528665792] v30'228 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fb61000a5a0 con 0x560da0908750 2023-10-05T19:41:27.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.844+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fb608007030 msgr2=0x7fb608001250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.844+0000 7fb61a038e40 1 --2- 172.21.15.12:0/1081639402 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fb608007030 0x7fb608001250 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:27.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.844+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x560da0908750 msgr2=0x560da0939690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.844+0000 7fb61a038e40 1 --2- 172.21.15.12:0/1081639402 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x560da0908750 0x560da0939690 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:27.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.844+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x560da093b770 msgr2=0x560da093db50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.844+0000 7fb61a038e40 1 --2- 172.21.15.12:0/1081639402 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x560da093b770 0x560da093db50 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:27.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.844+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x560da09400f0 msgr2=0x560da09424d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.844+0000 7fb61a038e40 1 --2- 172.21.15.12:0/1081639402 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x560da09400f0 0x560da09424d0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:27.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.844+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fb5f8030c50 msgr2=0x7fb5f8033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.844+0000 7fb61a038e40 1 --2- 172.21.15.12:0/1081639402 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fb5f8030c50 0x7fb5f8033100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb60000b300 tx=0x7fb600001430 comp rx=0 tx=0).stop 2023-10-05T19:41:27.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.844+0000 7fb61836e700 1 -- 172.21.15.12:0/1081639402 reap_dead start 2023-10-05T19:41:27.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.844+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560da0907e40 msgr2=0x560da092dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.844+0000 7fb61a038e40 1 --2- 172.21.15.12:0/1081639402 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560da0907e40 0x560da092dba0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fb60800a5a0 tx=0x7fb60801c000 comp rx=0 tx=0).stop 2023-10-05T19:41:27.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.844+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 shutdown_connections 2023-10-05T19:41:27.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.844+0000 7fb61a038e40 1 --2- 172.21.15.12:0/1081639402 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560da0907e40 0x560da092dba0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:27.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.844+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 >> 172.21.15.12:0/1081639402 conn(0x560da0903860 msgr2=0x560da0921580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:27.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.844+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 shutdown_connections 2023-10-05T19:41:27.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.844+0000 7fb61a038e40 1 -- 172.21.15.12:0/1081639402 wait complete. 2023-10-05T19:41:27.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.844+0000 7fb61a038e40 1 -- 172.21.15.12:0/2144673840 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fb558030c50 msgr2=0x7fb558033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.844+0000 7fb61a038e40 1 --2- 172.21.15.12:0/2144673840 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fb558030c50 0x7fb558033100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x560da094b3f0 tx=0x7fb61000ae00 comp rx=0 tx=0).stop 2023-10-05T19:41:27.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.844+0000 7fb61a038e40 1 -- 172.21.15.12:0/2144673840 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560da0959480 msgr2=0x560da095efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.844+0000 7fb61a038e40 1 --2- 172.21.15.12:0/2144673840 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560da0959480 0x560da095efe0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x560da0953560 tx=0x7fb60003c000 comp rx=0 tx=0).stop 2023-10-05T19:41:27.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.844+0000 7fb61a038e40 1 -- 172.21.15.12:0/2144673840 shutdown_connections 2023-10-05T19:41:27.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.844+0000 7fb61a038e40 1 --2- 172.21.15.12:0/2144673840 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fb558030c50 0x7fb558033100 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:27.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.844+0000 7fb61a038e40 1 --2- 172.21.15.12:0/2144673840 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560da0959480 0x560da095efe0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:27.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.844+0000 7fb61a038e40 1 -- 172.21.15.12:0/2144673840 >> 172.21.15.12:0/2144673840 conn(0x560da0954840 msgr2=0x560da0952940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:27.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.844+0000 7fb61a038e40 1 -- 172.21.15.12:0/2144673840 shutdown_connections 2023-10-05T19:41:27.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.844+0000 7fb61a038e40 1 -- 172.21.15.12:0/2144673840 wait complete. 2023-10-05T19:41:27.854 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:41:27.854 INFO:teuthology.orchestra.run.smithi012.stdout: "entries": [ 2023-10-05T19:41:27.854 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.854 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:41:27.855 INFO:teuthology.orchestra.run.smithi012.stdout: "buckets": [ 2023-10-05T19:41:27.855 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.855 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "", 2023-10-05T19:41:27.855 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:00:00.000000Z", 2023-10-05T19:41:27.855 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 1696532400, 2023-10-05T19:41:27.855 INFO:teuthology.orchestra.run.smithi012.stdout: "owner": "foo", 2023-10-05T19:41:27.855 INFO:teuthology.orchestra.run.smithi012.stdout: "categories": [] 2023-10-05T19:41:27.855 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:27.855 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.855 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo", 2023-10-05T19:41:27.855 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:00:00.000000Z", 2023-10-05T19:41:27.855 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 1696532400, 2023-10-05T19:41:27.856 INFO:teuthology.orchestra.run.smithi012.stdout: "owner": "foo", 2023-10-05T19:41:27.856 INFO:teuthology.orchestra.run.smithi012.stdout: "categories": [ 2023-10-05T19:41:27.856 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.856 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "delete_obj", 2023-10-05T19:41:27.856 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:27.856 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:27.856 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 4, 2023-10-05T19:41:27.856 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 4 2023-10-05T19:41:27.856 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:27.856 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2023-10-05T19:41:27.856 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:27.856 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.857 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo2", 2023-10-05T19:41:27.857 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:00:00.000000Z", 2023-10-05T19:41:27.857 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 1696532400, 2023-10-05T19:41:27.857 INFO:teuthology.orchestra.run.smithi012.stdout: "owner": "foo", 2023-10-05T19:41:27.857 INFO:teuthology.orchestra.run.smithi012.stdout: "categories": [] 2023-10-05T19:41:27.857 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:27.857 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.857 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo3", 2023-10-05T19:41:27.857 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:00:00.000000Z", 2023-10-05T19:41:27.857 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 1696532400, 2023-10-05T19:41:27.857 INFO:teuthology.orchestra.run.smithi012.stdout: "owner": "foo", 2023-10-05T19:41:27.858 INFO:teuthology.orchestra.run.smithi012.stdout: "categories": [] 2023-10-05T19:41:27.858 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:27.858 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.858 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo4", 2023-10-05T19:41:27.858 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:00:00.000000Z", 2023-10-05T19:41:27.858 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 1696532400, 2023-10-05T19:41:27.858 INFO:teuthology.orchestra.run.smithi012.stdout: "owner": "foo", 2023-10-05T19:41:27.858 INFO:teuthology.orchestra.run.smithi012.stdout: "categories": [] 2023-10-05T19:41:27.858 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:27.858 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2023-10-05T19:41:27.858 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:27.858 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:41:27.859 INFO:teuthology.orchestra.run.smithi012.stdout: "summary": [ 2023-10-05T19:41:27.859 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.859 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:41:27.859 INFO:teuthology.orchestra.run.smithi012.stdout: "categories": [ 2023-10-05T19:41:27.859 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:27.859 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "delete_obj", 2023-10-05T19:41:27.859 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:27.859 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:27.859 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 4, 2023-10-05T19:41:27.859 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 4 2023-10-05T19:41:27.859 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:27.859 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:41:27.859 INFO:teuthology.orchestra.run.smithi012.stdout: "total": { 2023-10-05T19:41:27.860 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:27.860 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:27.860 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 4, 2023-10-05T19:41:27.860 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 4 2023-10-05T19:41:27.860 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:27.860 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:27.860 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2023-10-05T19:41:27.860 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:41:27.861 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': '', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}]}, {'bucket': 'myfoo2', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo3', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo4', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': []}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}}]} 2023-10-05T19:41:27.861 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo', '--categories', 'delete_bucket'] 2023-10-05T19:41:27.861 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo', '--categories', 'delete_bucket'] 2023-10-05T19:41:27.862 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo --categories delete_bucket 2023-10-05T19:41:27.900 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:41:27.900 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:41:27.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.896+0000 7fec4d674e40 1 Processor -- start 2023-10-05T19:41:27.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.896+0000 7fec4d674e40 1 -- start start 2023-10-05T19:41:27.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.896+0000 7fec4d674e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55e228f0c7f0 0x55e228f0b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:27.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.896+0000 7fec4d674e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55e228e58bd0 con 0x55e228f0c420 2023-10-05T19:41:27.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.896+0000 7fec4d674e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x55e228e15ee0 con 0x55e228f0c7f0 2023-10-05T19:41:27.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.896+0000 7fec4a9a8700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55e228f0c7f0 0x55e228f0b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:27.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.896+0000 7fec4b1a9700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x55e228f0c420 0x55e228f0afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:57688/0 (socket says 172.21.15.12:57688) 2023-10-05T19:41:27.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.896+0000 7fec4b1a9700 1 -- 172.21.15.12:0/4073461200 learned_addr learned my addr 172.21.15.12:0/4073461200 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:41:27.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.896+0000 7fec4a9a8700 1 -- 172.21.15.12:0/4073461200 >> v1:172.21.15.8:6789/0 conn(0x55e228f0c420 legacy=0x55e228f0afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.896+0000 7fec4a1a7700 1 -- 172.21.15.12:0/4073461200 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1642881386 0 0) 0x55e228e58bd0 con 0x55e228f0c420 2023-10-05T19:41:27.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.896+0000 7fec4a9a8700 1 -- 172.21.15.12:0/4073461200 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e228f0bce0 con 0x55e228f0c7f0 2023-10-05T19:41:27.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.896+0000 7fec4a9a8700 1 --2- 172.21.15.12:0/4073461200 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e228f0c7f0 0x55e228f0b6e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fec3400a520 tx=0x7fec34005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66f70f3eb039156d server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:27.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.900+0000 7fec4a1a7700 1 -- 172.21.15.12:0/4073461200 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec3400f590 con 0x55e228f0c7f0 2023-10-05T19:41:27.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.900+0000 7fec4a1a7700 1 -- 172.21.15.12:0/4073461200 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec3400f6f0 con 0x55e228f0c7f0 2023-10-05T19:41:27.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.900+0000 7fec4a1a7700 1 -- 172.21.15.12:0/4073461200 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec3400f880 con 0x55e228f0c7f0 2023-10-05T19:41:27.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.900+0000 7fec4d674e40 1 -- 172.21.15.12:0/4073461200 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e228f0c7f0 msgr2=0x55e228f0b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.900+0000 7fec4d674e40 1 --2- 172.21.15.12:0/4073461200 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e228f0c7f0 0x55e228f0b6e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fec3400a520 tx=0x7fec34005d40 comp rx=0 tx=0).stop 2023-10-05T19:41:27.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.900+0000 7fec4d674e40 1 -- 172.21.15.12:0/4073461200 shutdown_connections 2023-10-05T19:41:27.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.900+0000 7fec4d674e40 1 --2- 172.21.15.12:0/4073461200 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e228f0c7f0 0x55e228f0b6e0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:27.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.900+0000 7fec4d674e40 1 -- 172.21.15.12:0/4073461200 >> 172.21.15.12:0/4073461200 conn(0x55e228f06730 msgr2=0x55e228f08b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:27.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.900+0000 7fec4d674e40 1 -- 172.21.15.12:0/4073461200 shutdown_connections 2023-10-05T19:41:27.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.900+0000 7fec4d674e40 1 -- 172.21.15.12:0/4073461200 wait complete. 2023-10-05T19:41:27.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.900+0000 7fec4d674e40 1 Processor -- start 2023-10-05T19:41:27.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.900+0000 7fec4d674e40 1 -- start start 2023-10-05T19:41:27.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.900+0000 7fec4d674e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e228f0c420 0x55e228f2cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:27.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.900+0000 7fec4d674e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e228dd9e30 con 0x55e228f0c420 2023-10-05T19:41:27.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.900+0000 7fec4b1a9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e228f0c420 0x55e228f2cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:27.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.900+0000 7fec4b1a9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e228f0c420 0x55e228f2cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:50636/0 (socket says 172.21.15.12:50636) 2023-10-05T19:41:27.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.900+0000 7fec4b1a9700 1 -- 172.21.15.12:0/302552218 learned_addr learned my addr 172.21.15.12:0/302552218 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:27.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.900+0000 7fec4b1a9700 1 -- 172.21.15.12:0/302552218 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec3400a0e0 con 0x55e228f0c420 2023-10-05T19:41:27.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.900+0000 7fec4b1a9700 1 --2- 172.21.15.12:0/302552218 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e228f0c420 0x55e228f2cff0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fec3c001dc0 tx=0x7fec3c001790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15451ce503c3e0aa server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:27.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.900+0000 7fec491a5700 1 -- 172.21.15.12:0/302552218 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec3c00bce0 con 0x55e228f0c420 2023-10-05T19:41:27.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.900+0000 7fec491a5700 1 -- 172.21.15.12:0/302552218 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec3c00be40 con 0x55e228f0c420 2023-10-05T19:41:27.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.900+0000 7fec491a5700 1 -- 172.21.15.12:0/302552218 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec3c00e5d0 con 0x55e228f0c420 2023-10-05T19:41:27.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.904+0000 7fec4d674e40 1 -- 172.21.15.12:0/302552218 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e228f0c420 msgr2=0x55e228f2cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:27.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.904+0000 7fec4d674e40 1 --2- 172.21.15.12:0/302552218 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e228f0c420 0x55e228f2cff0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fec3c001dc0 tx=0x7fec3c001790 comp rx=0 tx=0).stop 2023-10-05T19:41:27.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.904+0000 7fec4d674e40 1 -- 172.21.15.12:0/302552218 shutdown_connections 2023-10-05T19:41:27.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.904+0000 7fec4d674e40 1 --2- 172.21.15.12:0/302552218 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e228f0c420 0x55e228f2cff0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:27.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.904+0000 7fec4d674e40 1 -- 172.21.15.12:0/302552218 >> 172.21.15.12:0/302552218 conn(0x55e228f06730 msgr2=0x55e228f28bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:27.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.904+0000 7fec4d674e40 1 -- 172.21.15.12:0/302552218 shutdown_connections 2023-10-05T19:41:27.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.904+0000 7fec4d674e40 1 -- 172.21.15.12:0/302552218 wait complete. 2023-10-05T19:41:27.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.904+0000 7fec4d674e40 1 Processor -- start 2023-10-05T19:41:27.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.904+0000 7fec4d674e40 1 -- start start 2023-10-05T19:41:27.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.904+0000 7fec4d674e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e228f0c420 0x55e228f33d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:27.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.904+0000 7fec4d674e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e228daced0 con 0x55e228f0c420 2023-10-05T19:41:27.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.904+0000 7fec4b1a9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e228f0c420 0x55e228f33d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:27.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.904+0000 7fec4b1a9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e228f0c420 0x55e228f33d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:50648/0 (socket says 172.21.15.12:50648) 2023-10-05T19:41:27.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.904+0000 7fec4b1a9700 1 -- 172.21.15.12:0/681061727 learned_addr learned my addr 172.21.15.12:0/681061727 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:27.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.904+0000 7fec4b1a9700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec3c001a70 con 0x55e228f0c420 2023-10-05T19:41:27.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.904+0000 7fec4b1a9700 1 --2- 172.21.15.12:0/681061727 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e228f0c420 0x55e228f33d80 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fec3c0017d0 tx=0x7fec3c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:27.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.904+0000 7fec437fe700 1 -- 172.21.15.12:0/681061727 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec3c01f070 con 0x55e228f0c420 2023-10-05T19:41:27.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.904+0000 7fec437fe700 1 -- 172.21.15.12:0/681061727 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec3c01a4d0 con 0x55e228f0c420 2023-10-05T19:41:27.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.904+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e228f342c0 con 0x55e228f0c420 2023-10-05T19:41:27.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.904+0000 7fec437fe700 1 -- 172.21.15.12:0/681061727 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec3c01a7a0 con 0x55e228f0c420 2023-10-05T19:41:27.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.904+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e228f34700 con 0x55e228f0c420 2023-10-05T19:41:27.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.908+0000 7fec437fe700 1 -- 172.21.15.12:0/681061727 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fec3c01a900 con 0x55e228f0c420 2023-10-05T19:41:27.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.908+0000 7fec4d674e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.908+0000 7fec437fe700 1 --2- 172.21.15.12:0/681061727 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fec2c030c00 0x7fec2c0330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:27.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.908+0000 7fec437fe700 1 -- 172.21.15.12:0/681061727 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7fec3c0452b0 con 0x55e228f0c420 2023-10-05T19:41:27.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.908+0000 7fec4d674e40 1 --2- 172.21.15.12:0/681061727 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55e228f0c7f0 0x55e228f3f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:27.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.908+0000 7fec4a9a8700 1 --2- 172.21.15.12:0/681061727 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fec2c030c00 0x7fec2c0330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:27.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.908+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f3fec0 con 0x55e228f0c7f0 2023-10-05T19:41:27.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.908+0000 7fec4b9aa700 1 --2- 172.21.15.12:0/681061727 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55e228f0c7f0 0x55e228f3f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:27.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.908+0000 7fec4a9a8700 1 --2- 172.21.15.12:0/681061727 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fec2c030c00 0x7fec2c0330b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fec34005d80 tx=0x7fec34014000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:27.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.908+0000 7fec4b9aa700 1 --2- 172.21.15.12:0/681061727 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55e228f0c7f0 0x55e228f3f810 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:27.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.908+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fec4400a500 con 0x55e228f0c7f0 2023-10-05T19:41:27.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.908+0000 7fec4d674e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.908+0000 7fec4d674e40 20 realm 2023-10-05T19:41:27.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.908+0000 7fec4d674e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.908+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f3ce70 con 0x55e228f0c7f0 2023-10-05T19:41:27.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.912+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fec4400a500 con 0x55e228f0c7f0 2023-10-05T19:41:27.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.912+0000 7fec4d674e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.912+0000 7fec4d674e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:41:27.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.912+0000 7fec4d674e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.912+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f409a0 con 0x55e228f0c7f0 2023-10-05T19:41:27.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.912+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fec4400a500 con 0x55e228f0c7f0 2023-10-05T19:41:27.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.912+0000 7fec4d674e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.912+0000 7fec4d674e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.912+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f40580 con 0x55e228f0c7f0 2023-10-05T19:41:27.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.912+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fec4400a500 con 0x55e228f0c7f0 2023-10-05T19:41:27.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.912+0000 7fec4d674e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:27.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.912+0000 7fec4d674e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.912+0000 7fec4d674e40 1 --2- 172.21.15.12:0/681061727 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55e228f41930 0x55e228f43d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:27.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.912+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f44480 con 0x55e228f41930 2023-10-05T19:41:27.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.912+0000 7fec4b1a9700 1 --2- 172.21.15.12:0/681061727 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55e228f41930 0x55e228f43d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:27.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.912+0000 7fec4b1a9700 1 --2- 172.21.15.12:0/681061727 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55e228f41930 0x55e228f43d10 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:27.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.912+0000 7fec4b1a9700 1 -- 172.21.15.12:0/681061727 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fec3c014070 con 0x55e228f41930 2023-10-05T19:41:27.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.912+0000 7fec4d674e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:41:27.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.912+0000 7fec4d674e40 20 searching for the correct realm 2023-10-05T19:41:27.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.912+0000 7fec4d674e40 1 --2- 172.21.15.12:0/681061727 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55e228f462b0 0x55e228f48690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:27.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.912+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f48bd0 con 0x55e228f462b0 2023-10-05T19:41:27.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.912+0000 7fec4a9a8700 1 --2- 172.21.15.12:0/681061727 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55e228f462b0 0x55e228f48690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:27.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.916+0000 7fec4a9a8700 1 --2- 172.21.15.12:0/681061727 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55e228f462b0 0x55e228f48690 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:27.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.916+0000 7fec4a9a8700 1 -- 172.21.15.12:0/681061727 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fec3400fd40 con 0x55e228f462b0 2023-10-05T19:41:27.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.916+0000 7fec4a9a8700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f48bd0 con 0x55e228f41930 2023-10-05T19:41:27.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.916+0000 7fec4b1a9700 1 -- 172.21.15.12:0/681061727 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec3c014070 con 0x55e228f41930 2023-10-05T19:41:27.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.916+0000 7fec4b1a9700 1 --2- 172.21.15.12:0/681061727 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fec3c02d020 0x7fec3c027120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:27.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.916+0000 7fec4b1a9700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f48bd0 con 0x7fec3c02d020 2023-10-05T19:41:27.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.916+0000 7fec4b9aa700 1 --2- 172.21.15.12:0/681061727 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fec3c02d020 0x7fec3c027120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:27.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.916+0000 7fec4b9aa700 1 --2- 172.21.15.12:0/681061727 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fec3c02d020 0x7fec3c027120 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:27.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.920+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec44014040 con 0x7fec3c02d020 2023-10-05T19:41:27.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.920+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f48bd0 con 0x55e228f0c7f0 2023-10-05T19:41:27.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.920+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec44014040 con 0x55e228f0c7f0 2023-10-05T19:41:27.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.920+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f48bd0 con 0x55e228f0c7f0 2023-10-05T19:41:27.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.920+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec44014040 con 0x55e228f0c7f0 2023-10-05T19:41:27.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.920+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f48bd0 con 0x55e228f0c7f0 2023-10-05T19:41:27.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.920+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(11 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec44014040 con 0x55e228f0c7f0 2023-10-05T19:41:27.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.920+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f48bd0 con 0x7fec3c02d020 2023-10-05T19:41:27.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.920+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec44014040 con 0x7fec3c02d020 2023-10-05T19:41:27.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.920+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f48bd0 con 0x55e228f41930 2023-10-05T19:41:27.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.920+0000 7fec4b1a9700 1 -- 172.21.15.12:0/681061727 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec3c014070 con 0x55e228f41930 2023-10-05T19:41:27.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.924+0000 7fec4b1a9700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f48bd0 con 0x55e228f41930 2023-10-05T19:41:27.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.924+0000 7fec4b1a9700 1 -- 172.21.15.12:0/681061727 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec3c014070 con 0x55e228f41930 2023-10-05T19:41:27.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.924+0000 7fec4b1a9700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f48bd0 con 0x55e228f0c7f0 2023-10-05T19:41:27.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.924+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec44014040 con 0x55e228f0c7f0 2023-10-05T19:41:27.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.924+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f48bd0 con 0x7fec3c02d020 2023-10-05T19:41:27.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.924+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec44014040 con 0x7fec3c02d020 2023-10-05T19:41:27.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.924+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f48bd0 con 0x55e228f0c7f0 2023-10-05T19:41:27.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.924+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec44014040 con 0x55e228f0c7f0 2023-10-05T19:41:27.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.924+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f48bd0 con 0x55e228f41930 2023-10-05T19:41:27.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.924+0000 7fec4b1a9700 1 -- 172.21.15.12:0/681061727 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(18 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec3c014070 con 0x55e228f41930 2023-10-05T19:41:27.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.924+0000 7fec4b1a9700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f48bd0 con 0x55e228f462b0 2023-10-05T19:41:27.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.928+0000 7fec4a9a8700 1 -- 172.21.15.12:0/681061727 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec3400fd40 con 0x55e228f462b0 2023-10-05T19:41:27.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.928+0000 7fec4a9a8700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f48bd0 con 0x55e228f462b0 2023-10-05T19:41:27.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.928+0000 7fec4a9a8700 1 -- 172.21.15.12:0/681061727 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec3400fd40 con 0x55e228f462b0 2023-10-05T19:41:27.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.928+0000 7fec4a9a8700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f48bd0 con 0x7fec3c02d020 2023-10-05T19:41:27.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.928+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec44014040 con 0x7fec3c02d020 2023-10-05T19:41:27.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.928+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f48bd0 con 0x55e228f462b0 2023-10-05T19:41:27.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.928+0000 7fec4a9a8700 1 -- 172.21.15.12:0/681061727 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec3400fd40 con 0x55e228f462b0 2023-10-05T19:41:27.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.928+0000 7fec4a9a8700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f48bd0 con 0x55e228f462b0 2023-10-05T19:41:27.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.928+0000 7fec4a9a8700 1 -- 172.21.15.12:0/681061727 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec3400fd40 con 0x55e228f462b0 2023-10-05T19:41:27.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.928+0000 7fec4a9a8700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f48bd0 con 0x55e228f0c7f0 2023-10-05T19:41:27.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.928+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec44014040 con 0x55e228f0c7f0 2023-10-05T19:41:27.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.928+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f48bd0 con 0x55e228f0c7f0 2023-10-05T19:41:27.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.932+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec44014040 con 0x55e228f0c7f0 2023-10-05T19:41:27.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.932+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f48bd0 con 0x7fec3c02d020 2023-10-05T19:41:27.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.932+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec44014040 con 0x7fec3c02d020 2023-10-05T19:41:27.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.932+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f48bd0 con 0x55e228f462b0 2023-10-05T19:41:27.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.932+0000 7fec4a9a8700 1 -- 172.21.15.12:0/681061727 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec3400fd40 con 0x55e228f462b0 2023-10-05T19:41:27.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.932+0000 7fec4a9a8700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f48bd0 con 0x55e228f41930 2023-10-05T19:41:27.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.932+0000 7fec4b1a9700 1 -- 172.21.15.12:0/681061727 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec3c014070 con 0x55e228f41930 2023-10-05T19:41:27.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.932+0000 7fec4b1a9700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f48bd0 con 0x55e228f0c7f0 2023-10-05T19:41:27.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.932+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec44014040 con 0x55e228f0c7f0 2023-10-05T19:41:27.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.932+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f48bd0 con 0x7fec3c02d020 2023-10-05T19:41:27.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.936+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec44014040 con 0x7fec3c02d020 2023-10-05T19:41:27.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.936+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f48bd0 con 0x55e228f462b0 2023-10-05T19:41:27.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.936+0000 7fec4a9a8700 1 -- 172.21.15.12:0/681061727 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec3400fd40 con 0x55e228f462b0 2023-10-05T19:41:27.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.936+0000 7fec4a9a8700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f48bd0 con 0x7fec3c02d020 2023-10-05T19:41:27.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.936+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec44014040 con 0x7fec3c02d020 2023-10-05T19:41:27.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.936+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f48bd0 con 0x55e228f41930 2023-10-05T19:41:27.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.936+0000 7fec4b1a9700 1 -- 172.21.15.12:0/681061727 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec3c014070 con 0x55e228f41930 2023-10-05T19:41:27.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.936+0000 7fec4b1a9700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f48bd0 con 0x55e228f462b0 2023-10-05T19:41:27.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.940+0000 7fec4a9a8700 1 -- 172.21.15.12:0/681061727 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec3400fd40 con 0x55e228f462b0 2023-10-05T19:41:27.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.940+0000 7fec4a9a8700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f48bd0 con 0x7fec3c02d020 2023-10-05T19:41:27.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.940+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec44014040 con 0x7fec3c02d020 2023-10-05T19:41:27.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.940+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f48bd0 con 0x55e228f0c7f0 2023-10-05T19:41:27.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.940+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fec44014040 con 0x55e228f0c7f0 2023-10-05T19:41:27.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.940+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f48bd0 con 0x55e228f41930 2023-10-05T19:41:27.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.940+0000 7fec4b1a9700 1 -- 172.21.15.12:0/681061727 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(37 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fec3c014070 con 0x55e228f41930 2023-10-05T19:41:27.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.940+0000 7fec4d674e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:41:27.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.940+0000 7fec4d674e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:41:27.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.940+0000 7fec4d674e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:41:27.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.940+0000 7fec4d674e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:41:27.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.940+0000 7fec4d674e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.940+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f45000 con 0x55e228f0c7f0 2023-10-05T19:41:27.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.940+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fec44014040 con 0x55e228f0c7f0 2023-10-05T19:41:27.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.940+0000 7fec4d674e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.940+0000 7fec4d674e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.940+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f44bd0 con 0x55e228f0c7f0 2023-10-05T19:41:27.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.944+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fec44014040 con 0x55e228f0c7f0 2023-10-05T19:41:27.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.944+0000 7fec4d674e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:27.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.944+0000 7fec4d674e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.944+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f45930 con 0x55e228f462b0 2023-10-05T19:41:27.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.944+0000 7fec4a9a8700 1 -- 172.21.15.12:0/681061727 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fec3400fd40 con 0x55e228f462b0 2023-10-05T19:41:27.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.944+0000 7fec4d674e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:41:27.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.944+0000 7fec4d674e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.944+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f49500 con 0x55e228f462b0 2023-10-05T19:41:27.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.944+0000 7fec4a9a8700 1 -- 172.21.15.12:0/681061727 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7fec3400fd40 con 0x55e228f462b0 2023-10-05T19:41:27.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.944+0000 7fec4d674e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.944+0000 7fec4d674e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.944+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f498b0 con 0x55e228f0c7f0 2023-10-05T19:41:27.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.944+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fec44014040 con 0x55e228f0c7f0 2023-10-05T19:41:27.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.944+0000 7fec4d674e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.944+0000 7fec4d674e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.944+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f49c60 con 0x55e228f0c7f0 2023-10-05T19:41:27.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.944+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fec44014040 con 0x55e228f0c7f0 2023-10-05T19:41:27.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.944+0000 7fec4d674e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:27.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.944+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f4ac70 con 0x55e228f462b0 2023-10-05T19:41:27.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.948+0000 7fec4a9a8700 1 -- 172.21.15.12:0/681061727 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(44 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fec3400fd40 con 0x55e228f462b0 2023-10-05T19:41:27.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.948+0000 7fec4a9a8700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f4ac70 con 0x55e228f41930 2023-10-05T19:41:27.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.948+0000 7fec4b1a9700 1 -- 172.21.15.12:0/681061727 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(45 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec3c014070 con 0x55e228f41930 2023-10-05T19:41:27.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.948+0000 7fec4b1a9700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f4ac70 con 0x7fec3c02d020 2023-10-05T19:41:27.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.952+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(46 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec44014040 con 0x7fec3c02d020 2023-10-05T19:41:27.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.952+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f498b0 con 0x55e228f0c7f0 2023-10-05T19:41:27.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.952+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(47 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec44014040 con 0x55e228f0c7f0 2023-10-05T19:41:27.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.952+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f498b0 con 0x55e228f0c7f0 2023-10-05T19:41:27.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.952+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(48 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec44014040 con 0x55e228f0c7f0 2023-10-05T19:41:27.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.952+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f498b0 con 0x55e228f0c7f0 2023-10-05T19:41:27.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.952+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(49 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec44014040 con 0x55e228f0c7f0 2023-10-05T19:41:27.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.952+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f498b0 con 0x7fec3c02d020 2023-10-05T19:41:27.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.952+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(50 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec44014040 con 0x7fec3c02d020 2023-10-05T19:41:27.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.952+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f498b0 con 0x55e228f41930 2023-10-05T19:41:27.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.956+0000 7fec4b1a9700 1 -- 172.21.15.12:0/681061727 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(51 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec3c014070 con 0x55e228f41930 2023-10-05T19:41:27.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.956+0000 7fec4b1a9700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f498b0 con 0x55e228f41930 2023-10-05T19:41:27.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.956+0000 7fec4b1a9700 1 -- 172.21.15.12:0/681061727 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(52 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec3c014070 con 0x55e228f41930 2023-10-05T19:41:27.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.956+0000 7fec4b1a9700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f498b0 con 0x55e228f0c7f0 2023-10-05T19:41:27.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.956+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(53 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec44014040 con 0x55e228f0c7f0 2023-10-05T19:41:27.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.956+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f498b0 con 0x7fec3c02d020 2023-10-05T19:41:27.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.956+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(54 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec44014040 con 0x7fec3c02d020 2023-10-05T19:41:27.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.956+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f498b0 con 0x55e228f0c7f0 2023-10-05T19:41:27.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.956+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(55 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec44014040 con 0x55e228f0c7f0 2023-10-05T19:41:27.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.956+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f498b0 con 0x55e228f41930 2023-10-05T19:41:27.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.956+0000 7fec4b1a9700 1 -- 172.21.15.12:0/681061727 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(56 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec3c014070 con 0x55e228f41930 2023-10-05T19:41:27.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.956+0000 7fec4b1a9700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f498b0 con 0x55e228f462b0 2023-10-05T19:41:27.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.960+0000 7fec4a9a8700 1 -- 172.21.15.12:0/681061727 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(57 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec3400fd40 con 0x55e228f462b0 2023-10-05T19:41:27.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.960+0000 7fec4a9a8700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f498b0 con 0x55e228f462b0 2023-10-05T19:41:27.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.960+0000 7fec4a9a8700 1 -- 172.21.15.12:0/681061727 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(58 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec3400fd40 con 0x55e228f462b0 2023-10-05T19:41:27.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.960+0000 7fec4a9a8700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f498b0 con 0x7fec3c02d020 2023-10-05T19:41:27.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.960+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(59 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec44014040 con 0x7fec3c02d020 2023-10-05T19:41:27.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.960+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f498b0 con 0x55e228f462b0 2023-10-05T19:41:27.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.960+0000 7fec4a9a8700 1 -- 172.21.15.12:0/681061727 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(60 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec3400fd40 con 0x55e228f462b0 2023-10-05T19:41:27.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.960+0000 7fec4a9a8700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f498b0 con 0x55e228f462b0 2023-10-05T19:41:27.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.960+0000 7fec4a9a8700 1 -- 172.21.15.12:0/681061727 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(61 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec3400fd40 con 0x55e228f462b0 2023-10-05T19:41:27.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.960+0000 7fec4a9a8700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f498b0 con 0x55e228f0c7f0 2023-10-05T19:41:27.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.964+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(62 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec44014040 con 0x55e228f0c7f0 2023-10-05T19:41:27.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.964+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f498b0 con 0x55e228f0c7f0 2023-10-05T19:41:27.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.964+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(63 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec44014040 con 0x55e228f0c7f0 2023-10-05T19:41:27.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.964+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f498b0 con 0x7fec3c02d020 2023-10-05T19:41:27.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.964+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(64 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec44014040 con 0x7fec3c02d020 2023-10-05T19:41:27.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.964+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f498b0 con 0x55e228f462b0 2023-10-05T19:41:27.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.964+0000 7fec4a9a8700 1 -- 172.21.15.12:0/681061727 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(65 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec3400fd40 con 0x55e228f462b0 2023-10-05T19:41:27.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.964+0000 7fec4a9a8700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f498b0 con 0x55e228f41930 2023-10-05T19:41:27.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.964+0000 7fec4b1a9700 1 -- 172.21.15.12:0/681061727 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(66 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec3c014070 con 0x55e228f41930 2023-10-05T19:41:27.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.964+0000 7fec4b1a9700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f498b0 con 0x55e228f0c7f0 2023-10-05T19:41:27.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.968+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(67 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec44014040 con 0x55e228f0c7f0 2023-10-05T19:41:27.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.968+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f498b0 con 0x7fec3c02d020 2023-10-05T19:41:27.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.968+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(68 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec44014040 con 0x7fec3c02d020 2023-10-05T19:41:27.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.968+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f498b0 con 0x55e228f462b0 2023-10-05T19:41:27.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.968+0000 7fec4a9a8700 1 -- 172.21.15.12:0/681061727 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(69 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec3400fd40 con 0x55e228f462b0 2023-10-05T19:41:27.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.968+0000 7fec4a9a8700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f498b0 con 0x7fec3c02d020 2023-10-05T19:41:27.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.968+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(70 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec44014040 con 0x7fec3c02d020 2023-10-05T19:41:27.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.968+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f498b0 con 0x55e228f41930 2023-10-05T19:41:27.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.972+0000 7fec4b1a9700 1 -- 172.21.15.12:0/681061727 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(71 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec3c014070 con 0x55e228f41930 2023-10-05T19:41:27.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.972+0000 7fec4b1a9700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f498b0 con 0x55e228f462b0 2023-10-05T19:41:27.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.972+0000 7fec4a9a8700 1 -- 172.21.15.12:0/681061727 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(72 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec3400fd40 con 0x55e228f462b0 2023-10-05T19:41:27.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.972+0000 7fec4a9a8700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f498b0 con 0x7fec3c02d020 2023-10-05T19:41:27.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.972+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(73 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fec44014040 con 0x7fec3c02d020 2023-10-05T19:41:27.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.972+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f498b0 con 0x55e228f0c7f0 2023-10-05T19:41:27.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.972+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(74 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fec44014040 con 0x55e228f0c7f0 2023-10-05T19:41:27.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.972+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f498b0 con 0x55e228f41930 2023-10-05T19:41:27.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.976+0000 7fec4b1a9700 1 -- 172.21.15.12:0/681061727 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(75 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fec3c014070 con 0x55e228f41930 2023-10-05T19:41:27.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.976+0000 7fec4d674e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:41:27.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.976+0000 7fec4d674e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:41:27.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.976+0000 7fec4d674e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:41:27.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.976+0000 7fec4d674e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:41:27.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.976+0000 7fec4d674e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.976+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f49c60 con 0x55e228f0c7f0 2023-10-05T19:41:27.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.976+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fec44014040 con 0x55e228f0c7f0 2023-10-05T19:41:27.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.976+0000 7fec4d674e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:27.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.976+0000 7fec4d674e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.976+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f4a010 con 0x55e228f41930 2023-10-05T19:41:27.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.976+0000 7fec4b1a9700 1 -- 172.21.15.12:0/681061727 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fec3c014070 con 0x55e228f41930 2023-10-05T19:41:27.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.976+0000 7fec4d674e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:41:27.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.976+0000 7fec4d674e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.976+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f4a8b0 con 0x55e228f0c7f0 2023-10-05T19:41:27.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.976+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fec44014040 con 0x55e228f0c7f0 2023-10-05T19:41:27.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.976+0000 7fec4d674e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:27.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.976+0000 7fec4d674e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.976+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f4a8b0 con 0x55e228f462b0 2023-10-05T19:41:27.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.976+0000 7fec4a9a8700 1 -- 172.21.15.12:0/681061727 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fec3400fd40 con 0x55e228f462b0 2023-10-05T19:41:27.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.976+0000 7fec4d674e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:41:27.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.976+0000 7fec4d674e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.976+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f4a440 con 0x7fec3c02d020 2023-10-05T19:41:27.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.980+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7fec44014040 con 0x7fec3c02d020 2023-10-05T19:41:27.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.980+0000 7fec4d674e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.980+0000 7fec4d674e40 20 zone default found 2023-10-05T19:41:27.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.980+0000 7fec4d674e40 4 Realm: () 2023-10-05T19:41:27.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.980+0000 7fec4d674e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:41:27.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.980+0000 7fec4d674e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:41:27.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.980+0000 7fec4d674e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:41:27.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.980+0000 7fec4d674e40 20 zonegroup default 2023-10-05T19:41:27.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.980+0000 7fec4d674e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.980+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f4a440 con 0x7fec3c02d020 2023-10-05T19:41:27.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.980+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fec44014040 con 0x7fec3c02d020 2023-10-05T19:41:27.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.980+0000 7fec4d674e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.980+0000 7fec4d674e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.980+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f4a440 con 0x55e228f462b0 2023-10-05T19:41:27.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.980+0000 7fec4a9a8700 1 -- 172.21.15.12:0/681061727 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fec3400fd40 con 0x55e228f462b0 2023-10-05T19:41:27.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.980+0000 7fec4d674e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.980+0000 7fec4d674e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:27.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.980+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f4cad0 con 0x55e228f462b0 2023-10-05T19:41:27.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.980+0000 7fec4a9a8700 1 -- 172.21.15.12:0/681061727 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fec3400fd40 con 0x55e228f462b0 2023-10-05T19:41:27.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.980+0000 7fec4d674e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:27.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.980+0000 7fec4d674e40 20 started sync module instance, tier type = 2023-10-05T19:41:27.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.980+0000 7fec4d674e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:41:27.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.980+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f4c190 con 0x55e228f41930 2023-10-05T19:41:27.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.984+0000 7fec4b1a9700 1 -- 172.21.15.12:0/681061727 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(84 notify.0 [create] v30'237 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fec3c014070 con 0x55e228f41930 2023-10-05T19:41:27.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.984+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94429838099856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f4bdc0 con 0x55e228f41930 2023-10-05T19:41:27.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.984+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f4d860 con 0x55e228f0c7f0 2023-10-05T19:41:27.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.988+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(86 notify.1 [create] v30'237 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fec44014040 con 0x55e228f0c7f0 2023-10-05T19:41:27.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.988+0000 7fec4b1a9700 1 -- 172.21.15.12:0/681061727 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(85 notify.0 [watch watch cookie 94429838099856] v30'238 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fec3c014070 con 0x55e228f41930 2023-10-05T19:41:27.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.988+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94429838103696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f4e150 con 0x55e228f0c7f0 2023-10-05T19:41:27.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.988+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f4ce80 con 0x55e228f462b0 2023-10-05T19:41:27.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.992+0000 7fec4a9a8700 1 -- 172.21.15.12:0/681061727 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(88 notify.2 [create] v30'237 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fec3400fd40 con 0x55e228f462b0 2023-10-05T19:41:27.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.992+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94429838108272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f4efb0 con 0x55e228f462b0 2023-10-05T19:41:27.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.992+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f4f850 con 0x55e228f41930 2023-10-05T19:41:27.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.992+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(87 notify.1 [watch watch cookie 94429838103696] v30'238 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fec44014040 con 0x55e228f0c7f0 2023-10-05T19:41:27.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.992+0000 7fec4a9a8700 1 -- 172.21.15.12:0/681061727 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(89 notify.2 [watch watch cookie 94429838108272] v30'238 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fec3400fd40 con 0x55e228f462b0 2023-10-05T19:41:27.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.992+0000 7fec4b1a9700 1 -- 172.21.15.12:0/681061727 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(90 notify.3 [create] v30'237 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fec3c014070 con 0x55e228f41930 2023-10-05T19:41:28.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.992+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94429838109376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f4dd80 con 0x55e228f41930 2023-10-05T19:41:28.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.992+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f500f0 con 0x55e228f0c7f0 2023-10-05T19:41:28.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.996+0000 7fec4b1a9700 1 -- 172.21.15.12:0/681061727 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(91 notify.3 [watch watch cookie 94429838109376] v30'238 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fec3c014070 con 0x55e228f41930 2023-10-05T19:41:28.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.996+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(92 notify.4 [create] v30'237 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fec44014040 con 0x55e228f0c7f0 2023-10-05T19:41:28.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.996+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94429838110480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f500f0 con 0x55e228f0c7f0 2023-10-05T19:41:28.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:27.996+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f50990 con 0x7fec3c02d020 2023-10-05T19:41:28.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.000+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(93 notify.4 [watch watch cookie 94429838110480] v30'238 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fec44014040 con 0x55e228f0c7f0 2023-10-05T19:41:28.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.000+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(94 notify.5 [create] v30'237 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fec44014040 con 0x7fec3c02d020 2023-10-05T19:41:28.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.000+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94429838114352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f50770 con 0x7fec3c02d020 2023-10-05T19:41:28.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.000+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f512b0 con 0x7fec3c02d020 2023-10-05T19:41:28.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.004+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(96 notify.6 [create] v30'237 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fec44014040 con 0x7fec3c02d020 2023-10-05T19:41:28.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.004+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94429838118272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f516c0 con 0x7fec3c02d020 2023-10-05T19:41:28.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.004+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f52130 con 0x7fec3c02d020 2023-10-05T19:41:28.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.004+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(95 notify.5 [watch watch cookie 94429838114352] v30'238 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fec44014040 con 0x7fec3c02d020 2023-10-05T19:41:28.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.004+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(98 notify.7 [create] v30'237 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fec44014040 con 0x7fec3c02d020 2023-10-05T19:41:28.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.004+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94429838115456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f52130 con 0x7fec3c02d020 2023-10-05T19:41:28.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.004+0000 7fec4d674e40 20 add_watcher() i=0 2023-10-05T19:41:28.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.004+0000 7fec4d674e40 20 add_watcher() i=1 2023-10-05T19:41:28.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.004+0000 7fec4d674e40 20 add_watcher() i=2 2023-10-05T19:41:28.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.004+0000 7fec4d674e40 20 add_watcher() i=3 2023-10-05T19:41:28.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.004+0000 7fec4d674e40 20 add_watcher() i=4 2023-10-05T19:41:28.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.004+0000 7fec4d674e40 20 add_watcher() i=5 2023-10-05T19:41:28.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.004+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(97 notify.6 [watch watch cookie 94429838118272] v30'238 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fec44014040 con 0x7fec3c02d020 2023-10-05T19:41:28.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.008+0000 7fec4d674e40 20 add_watcher() i=6 2023-10-05T19:41:28.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.008+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(99 notify.7 [watch watch cookie 94429838115456] v30'238 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fec44014040 con 0x7fec3c02d020 2023-10-05T19:41:28.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.008+0000 7fec4d674e40 20 add_watcher() i=7 2023-10-05T19:41:28.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.008+0000 7fec4d674e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:41:28.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.008+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f55160 con 0x55e228f0c7f0 2023-10-05T19:41:28.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.008+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fec44014040 con 0x55e228f0c7f0 2023-10-05T19:41:28.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.008+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94429838122048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f52580 con 0x55e228f0c7f0 2023-10-05T19:41:28.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.012+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94429838122048] v30'229 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fec44014040 con 0x55e228f0c7f0 2023-10-05T19:41:28.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.012+0000 7fec1cfe1700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:41:28.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.012+0000 7fec4d674e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:41:28.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.012+0000 7fec4d674e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:41:28.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.012+0000 7fec4d674e40 1 Processor -- start 2023-10-05T19:41:28.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.012+0000 7fec4d674e40 1 -- start start 2023-10-05T19:41:28.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.012+0000 7fec4d674e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e228f5f2b0 0x55e228f5f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:28.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.012+0000 7fec4d674e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e228f5fbc0 con 0x55e228f5f2b0 2023-10-05T19:41:28.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.012+0000 7fec4a9a8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e228f5f2b0 0x55e228f5f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:28.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.012+0000 7fec4a9a8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e228f5f2b0 0x55e228f5f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:50656/0 (socket says 172.21.15.12:50656) 2023-10-05T19:41:28.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.012+0000 7fec4a9a8700 1 -- 172.21.15.12:0/805958149 learned_addr learned my addr 172.21.15.12:0/805958149 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:28.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.012+0000 7fec4a9a8700 1 -- 172.21.15.12:0/805958149 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e228f5fd00 con 0x55e228f5f2b0 2023-10-05T19:41:28.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.012+0000 7fec4a9a8700 1 --2- 172.21.15.12:0/805958149 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e228f5f2b0 0x55e228f5f680 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fec34003000 tx=0x7fec34011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2212d3af7bc4a57b server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:28.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.012+0000 7feb97fff700 1 -- 172.21.15.12:0/805958149 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec3401cce0 con 0x55e228f5f2b0 2023-10-05T19:41:28.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.016+0000 7feb97fff700 1 -- 172.21.15.12:0/805958149 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec3401ce40 con 0x55e228f5f2b0 2023-10-05T19:41:28.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.016+0000 7feb97fff700 1 -- 172.21.15.12:0/805958149 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec34044440 con 0x55e228f5f2b0 2023-10-05T19:41:28.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.016+0000 7fec4d674e40 1 -- 172.21.15.12:0/805958149 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e228f5f2b0 msgr2=0x55e228f5f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:28.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.016+0000 7fec4d674e40 1 --2- 172.21.15.12:0/805958149 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e228f5f2b0 0x55e228f5f680 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fec34003000 tx=0x7fec34011000 comp rx=0 tx=0).stop 2023-10-05T19:41:28.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.016+0000 7fec4d674e40 1 -- 172.21.15.12:0/805958149 shutdown_connections 2023-10-05T19:41:28.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.016+0000 7fec4d674e40 1 --2- 172.21.15.12:0/805958149 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e228f5f2b0 0x55e228f5f680 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:28.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.016+0000 7fec4d674e40 1 -- 172.21.15.12:0/805958149 >> 172.21.15.12:0/805958149 conn(0x55e228f5a670 msgr2=0x55e228f58820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:28.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.016+0000 7fec4d674e40 1 -- 172.21.15.12:0/805958149 shutdown_connections 2023-10-05T19:41:28.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.016+0000 7fec4d674e40 1 -- 172.21.15.12:0/805958149 wait complete. 2023-10-05T19:41:28.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.016+0000 7fec4d674e40 1 Processor -- start 2023-10-05T19:41:28.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.016+0000 7fec4d674e40 1 -- start start 2023-10-05T19:41:28.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.016+0000 7fec4d674e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e228f5f2b0 0x55e228f64e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:28.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.016+0000 7fec4d674e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e228f65e20 con 0x55e228f5f2b0 2023-10-05T19:41:28.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.016+0000 7fec4a9a8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e228f5f2b0 0x55e228f64e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:28.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.016+0000 7fec4a9a8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e228f5f2b0 0x55e228f64e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:50666/0 (socket says 172.21.15.12:50666) 2023-10-05T19:41:28.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.016+0000 7fec4a9a8700 1 -- 172.21.15.12:0/1541142333 learned_addr learned my addr 172.21.15.12:0/1541142333 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:28.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.016+0000 7fec4a9a8700 1 -- 172.21.15.12:0/1541142333 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec3402e040 con 0x55e228f5f2b0 2023-10-05T19:41:28.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.020+0000 7fec4a9a8700 1 --2- 172.21.15.12:0/1541142333 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e228f5f2b0 0x55e228f64e20 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x55e228f0bcb0 tx=0x7fec34048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:28.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.020+0000 7feb967fc700 1 -- 172.21.15.12:0/1541142333 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec3401cce0 con 0x55e228f5f2b0 2023-10-05T19:41:28.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.020+0000 7feb967fc700 1 -- 172.21.15.12:0/1541142333 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec3404f660 con 0x55e228f5f2b0 2023-10-05T19:41:28.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.020+0000 7feb967fc700 1 -- 172.21.15.12:0/1541142333 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec3404f810 con 0x55e228f5f2b0 2023-10-05T19:41:28.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.020+0000 7fec4d674e40 1 -- 172.21.15.12:0/1541142333 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e228f65420 con 0x55e228f5f2b0 2023-10-05T19:41:28.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.020+0000 7fec4d674e40 1 -- 172.21.15.12:0/1541142333 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e228f658c0 con 0x55e228f5f2b0 2023-10-05T19:41:28.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.020+0000 7feb967fc700 1 -- 172.21.15.12:0/1541142333 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fec3404f970 con 0x55e228f5f2b0 2023-10-05T19:41:28.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.020+0000 7feb967fc700 1 --2- 172.21.15.12:0/1541142333 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7feb8c030c50 0x7feb8c033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:28.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.020+0000 7feb967fc700 1 -- 172.21.15.12:0/1541142333 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7fec3402d160 con 0x55e228f5f2b0 2023-10-05T19:41:28.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.020+0000 7fec4d674e40 5 note: GC not initialized 2023-10-05T19:41:28.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.020+0000 7fec4b9aa700 1 --2- 172.21.15.12:0/1541142333 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7feb8c030c50 0x7feb8c033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:28.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.020+0000 7fec4b9aa700 1 --2- 172.21.15.12:0/1541142333 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7feb8c030c50 0x7feb8c033100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x55e228f513d0 tx=0x7fec440325c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:28.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.024+0000 7feb8b7fe700 20 reqs_thread_entry: start 2023-10-05T19:41:28.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.024+0000 7feb897fa700 20 reqs_thread_entry: start 2023-10-05T19:41:28.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.024+0000 7fec4d674e40 20 init_complete bucket index max shards: 11 2023-10-05T19:41:28.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.024+0000 7fec4d674e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:41:28.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.024+0000 7feb777fe700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:41:58 2023 2023-10-05T19:41:28.030 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:41:28.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.024+0000 7feb76ffd700 20 reqs_thread_entry: start 2023-10-05T19:41:28.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.024+0000 7fec4d674e40 20 get_system_obj_state: rctx=0x7ffec8a6f8a8 obj=default.rgw.meta:users.uid:foo state=0x55e228de50b0 s->prefetch_data=0 2023-10-05T19:41:28.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.024+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228fcd410 con 0x55e228f0c7f0 2023-10-05T19:41:28.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.024+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(102 foo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv15 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7fec44014040 con 0x55e228f0c7f0 2023-10-05T19:41:28.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.024+0000 7fec4d674e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:41:28.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.024+0000 7fec4d674e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:41:28.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.024+0000 7fec4d674e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:28.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.024+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228fcd3d0 con 0x55e228f0c7f0 2023-10-05T19:41:28.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.028+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(103 foo [call,call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv15 ondisk = 0) v8 ==== 273+0+480 (crc 0 0 0) 0x7fec44014040 con 0x55e228f0c7f0 2023-10-05T19:41:28.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.028+0000 7fec4d674e40 20 rados_obj.operate() r=0 bl.length=357 2023-10-05T19:41:28.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.028+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:104 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228fcd3d0 con 0x7fec3c02d020 2023-10-05T19:41:28.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.028+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(104 usage.10 [call out=1268b] v0'0 uv32 ondisk = 0) v8 ==== 152+0+1268 (crc 0 0 0) 0x7fec44014040 con 0x7fec3c02d020 2023-10-05T19:41:28.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.028+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:105 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94429838099856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228fcfa20 con 0x55e228f41930 2023-10-05T19:41:28.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.032+0000 7fec4b1a9700 1 -- 172.21.15.12:0/681061727 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(105 notify.0 [watch unwatch cookie 94429838099856] v30'239 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fec3c014070 con 0x55e228f41930 2023-10-05T19:41:28.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.032+0000 7fec4d674e40 20 remove_watcher() i=0 2023-10-05T19:41:28.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.032+0000 7fec4d674e40 2 removed watcher, disabling cache 2023-10-05T19:41:28.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.032+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:106 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94429838103696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f23910 con 0x55e228f0c7f0 2023-10-05T19:41:28.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.032+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(106 notify.1 [watch unwatch cookie 94429838103696] v30'239 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fec44014040 con 0x55e228f0c7f0 2023-10-05T19:41:28.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.032+0000 7fec4d674e40 20 remove_watcher() i=1 2023-10-05T19:41:28.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.032+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:107 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94429838108272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f23910 con 0x55e228f462b0 2023-10-05T19:41:28.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.036+0000 7fec4a9a8700 1 -- 172.21.15.12:0/681061727 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(107 notify.2 [watch unwatch cookie 94429838108272] v30'239 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fec3400fd40 con 0x55e228f462b0 2023-10-05T19:41:28.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.036+0000 7fec4d674e40 20 remove_watcher() i=2 2023-10-05T19:41:28.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.036+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:108 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94429838109376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f23910 con 0x55e228f41930 2023-10-05T19:41:28.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.040+0000 7fec4b1a9700 1 -- 172.21.15.12:0/681061727 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(108 notify.3 [watch unwatch cookie 94429838109376] v30'239 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fec3c014070 con 0x55e228f41930 2023-10-05T19:41:28.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.040+0000 7fec4d674e40 20 remove_watcher() i=3 2023-10-05T19:41:28.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.040+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:109 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94429838110480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f4eb60 con 0x55e228f0c7f0 2023-10-05T19:41:28.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.040+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== osd_op_reply(109 notify.4 [watch unwatch cookie 94429838110480] v30'239 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fec44014040 con 0x55e228f0c7f0 2023-10-05T19:41:28.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.040+0000 7fec4d674e40 20 remove_watcher() i=4 2023-10-05T19:41:28.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.040+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:110 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94429838114352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f23910 con 0x7fec3c02d020 2023-10-05T19:41:28.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.044+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(110 notify.5 [watch unwatch cookie 94429838114352] v30'239 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fec44014040 con 0x7fec3c02d020 2023-10-05T19:41:28.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.044+0000 7fec4d674e40 20 remove_watcher() i=5 2023-10-05T19:41:28.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.044+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:111 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94429838118272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f23910 con 0x7fec3c02d020 2023-10-05T19:41:28.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.048+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(111 notify.6 [watch unwatch cookie 94429838118272] v30'239 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fec44014040 con 0x7fec3c02d020 2023-10-05T19:41:28.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.048+0000 7fec4d674e40 20 remove_watcher() i=6 2023-10-05T19:41:28.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.048+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:112 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94429838115456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f23910 con 0x7fec3c02d020 2023-10-05T19:41:28.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.052+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(112 notify.7 [watch unwatch cookie 94429838115456] v30'239 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fec44014040 con 0x7fec3c02d020 2023-10-05T19:41:28.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.052+0000 7fec4d674e40 20 remove_watcher() i=7 2023-10-05T19:41:28.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.052+0000 7fec1cfe1700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:41:28.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.052+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:113 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94429838122048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e228f50320 con 0x55e228f0c7f0 2023-10-05T19:41:28.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.056+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 <== osd.1 v2:172.21.15.12:6808/29933 39 ==== osd_op_reply(113 data_loggenerations_metadata [watch unwatch cookie 94429838122048] v30'230 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fec44014040 con 0x55e228f0c7f0 2023-10-05T19:41:28.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.056+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fec3c02d020 msgr2=0x7fec3c027120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:28.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.056+0000 7fec4d674e40 1 --2- 172.21.15.12:0/681061727 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fec3c02d020 0x7fec3c027120 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:28.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.056+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55e228f0c7f0 msgr2=0x55e228f3f810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:28.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.056+0000 7fec4d674e40 1 --2- 172.21.15.12:0/681061727 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55e228f0c7f0 0x55e228f3f810 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:28.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.056+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55e228f41930 msgr2=0x55e228f43d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:28.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.056+0000 7fec4d674e40 1 --2- 172.21.15.12:0/681061727 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55e228f41930 0x55e228f43d10 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:28.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.056+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55e228f462b0 msgr2=0x55e228f48690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:28.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.056+0000 7fec4d674e40 1 --2- 172.21.15.12:0/681061727 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55e228f462b0 0x55e228f48690 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:28.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.056+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fec2c030c00 msgr2=0x7fec2c0330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:28.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.056+0000 7fec4d674e40 1 --2- 172.21.15.12:0/681061727 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fec2c030c00 0x7fec2c0330b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fec34005d80 tx=0x7fec34014000 comp rx=0 tx=0).stop 2023-10-05T19:41:28.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.056+0000 7fec4b9aa700 1 -- 172.21.15.12:0/681061727 reap_dead start 2023-10-05T19:41:28.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.056+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e228f0c420 msgr2=0x55e228f33d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:28.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.056+0000 7fec4d674e40 1 --2- 172.21.15.12:0/681061727 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e228f0c420 0x55e228f33d80 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fec3c0017d0 tx=0x7fec3c016000 comp rx=0 tx=0).stop 2023-10-05T19:41:28.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.056+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 shutdown_connections 2023-10-05T19:41:28.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.056+0000 7fec4d674e40 1 --2- 172.21.15.12:0/681061727 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e228f0c420 0x55e228f33d80 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:28.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.056+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 >> 172.21.15.12:0/681061727 conn(0x55e228f06730 msgr2=0x55e228f27630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:28.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.056+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 shutdown_connections 2023-10-05T19:41:28.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.056+0000 7fec4d674e40 1 -- 172.21.15.12:0/681061727 wait complete. 2023-10-05T19:41:28.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.056+0000 7fec4d674e40 1 -- 172.21.15.12:0/1541142333 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7feb8c030c50 msgr2=0x7feb8c033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:28.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.056+0000 7fec4d674e40 1 --2- 172.21.15.12:0/1541142333 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7feb8c030c50 0x7feb8c033100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x55e228f513d0 tx=0x7fec440325c0 comp rx=0 tx=0).stop 2023-10-05T19:41:28.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.056+0000 7fec4d674e40 1 -- 172.21.15.12:0/1541142333 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e228f5f2b0 msgr2=0x55e228f64e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:28.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.056+0000 7fec4d674e40 1 --2- 172.21.15.12:0/1541142333 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e228f5f2b0 0x55e228f64e20 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x55e228f0bcb0 tx=0x7fec34048000 comp rx=0 tx=0).stop 2023-10-05T19:41:28.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.056+0000 7fec4d674e40 1 -- 172.21.15.12:0/1541142333 shutdown_connections 2023-10-05T19:41:28.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.056+0000 7fec4d674e40 1 --2- 172.21.15.12:0/1541142333 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7feb8c030c50 0x7feb8c033100 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:28.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.056+0000 7fec4d674e40 1 --2- 172.21.15.12:0/1541142333 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e228f5f2b0 0x55e228f64e20 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:28.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.056+0000 7fec4d674e40 1 -- 172.21.15.12:0/1541142333 >> 172.21.15.12:0/1541142333 conn(0x55e228f5a670 msgr2=0x55e228f58770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:28.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.056+0000 7fec4d674e40 1 -- 172.21.15.12:0/1541142333 shutdown_connections 2023-10-05T19:41:28.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.056+0000 7fec4d674e40 1 -- 172.21.15.12:0/1541142333 wait complete. 2023-10-05T19:41:28.066 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:41:28.067 INFO:teuthology.orchestra.run.smithi012.stdout: "entries": [ 2023-10-05T19:41:28.067 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:28.067 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:41:28.067 INFO:teuthology.orchestra.run.smithi012.stdout: "buckets": [ 2023-10-05T19:41:28.067 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:28.067 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "", 2023-10-05T19:41:28.067 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:00:00.000000Z", 2023-10-05T19:41:28.067 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 1696532400, 2023-10-05T19:41:28.067 INFO:teuthology.orchestra.run.smithi012.stdout: "owner": "foo", 2023-10-05T19:41:28.067 INFO:teuthology.orchestra.run.smithi012.stdout: "categories": [] 2023-10-05T19:41:28.067 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:28.067 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:28.068 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo", 2023-10-05T19:41:28.068 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:00:00.000000Z", 2023-10-05T19:41:28.068 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 1696532400, 2023-10-05T19:41:28.068 INFO:teuthology.orchestra.run.smithi012.stdout: "owner": "foo", 2023-10-05T19:41:28.068 INFO:teuthology.orchestra.run.smithi012.stdout: "categories": [ 2023-10-05T19:41:28.068 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:28.068 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "delete_bucket", 2023-10-05T19:41:28.068 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 217, 2023-10-05T19:41:28.068 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:28.068 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 2, 2023-10-05T19:41:28.068 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 1 2023-10-05T19:41:28.069 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:28.069 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2023-10-05T19:41:28.069 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:28.069 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:28.069 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo2", 2023-10-05T19:41:28.069 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:00:00.000000Z", 2023-10-05T19:41:28.069 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 1696532400, 2023-10-05T19:41:28.069 INFO:teuthology.orchestra.run.smithi012.stdout: "owner": "foo", 2023-10-05T19:41:28.069 INFO:teuthology.orchestra.run.smithi012.stdout: "categories": [ 2023-10-05T19:41:28.069 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:28.069 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "delete_bucket", 2023-10-05T19:41:28.069 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:28.070 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:28.070 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 1, 2023-10-05T19:41:28.070 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 1 2023-10-05T19:41:28.070 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:28.070 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2023-10-05T19:41:28.070 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:28.070 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:28.070 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo3", 2023-10-05T19:41:28.070 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:00:00.000000Z", 2023-10-05T19:41:28.070 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 1696532400, 2023-10-05T19:41:28.070 INFO:teuthology.orchestra.run.smithi012.stdout: "owner": "foo", 2023-10-05T19:41:28.071 INFO:teuthology.orchestra.run.smithi012.stdout: "categories": [ 2023-10-05T19:41:28.071 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:28.071 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "delete_bucket", 2023-10-05T19:41:28.071 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:28.071 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:28.071 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 1, 2023-10-05T19:41:28.071 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 1 2023-10-05T19:41:28.071 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:28.071 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2023-10-05T19:41:28.071 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:28.071 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:28.072 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket": "myfoo4", 2023-10-05T19:41:28.072 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2023-10-05T19:00:00.000000Z", 2023-10-05T19:41:28.072 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 1696532400, 2023-10-05T19:41:28.072 INFO:teuthology.orchestra.run.smithi012.stdout: "owner": "foo", 2023-10-05T19:41:28.072 INFO:teuthology.orchestra.run.smithi012.stdout: "categories": [ 2023-10-05T19:41:28.072 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:28.072 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "delete_bucket", 2023-10-05T19:41:28.072 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 0, 2023-10-05T19:41:28.072 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:28.072 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 1, 2023-10-05T19:41:28.072 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 1 2023-10-05T19:41:28.073 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:28.073 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2023-10-05T19:41:28.073 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:28.073 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2023-10-05T19:41:28.073 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:28.073 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:41:28.073 INFO:teuthology.orchestra.run.smithi012.stdout: "summary": [ 2023-10-05T19:41:28.073 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:28.073 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "foo", 2023-10-05T19:41:28.074 INFO:teuthology.orchestra.run.smithi012.stdout: "categories": [ 2023-10-05T19:41:28.074 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:28.074 INFO:teuthology.orchestra.run.smithi012.stdout: "category": "delete_bucket", 2023-10-05T19:41:28.074 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 217, 2023-10-05T19:41:28.074 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:28.074 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 5, 2023-10-05T19:41:28.074 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 4 2023-10-05T19:41:28.074 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:28.074 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:41:28.075 INFO:teuthology.orchestra.run.smithi012.stdout: "total": { 2023-10-05T19:41:28.075 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_sent": 217, 2023-10-05T19:41:28.075 INFO:teuthology.orchestra.run.smithi012.stdout: "bytes_received": 0, 2023-10-05T19:41:28.075 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": 5, 2023-10-05T19:41:28.075 INFO:teuthology.orchestra.run.smithi012.stdout: "successful_ops": 4 2023-10-05T19:41:28.075 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:28.075 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:28.075 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2023-10-05T19:41:28.075 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:41:28.076 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': '', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 217, 'bytes_received': 0, 'ops': 2, 'successful_ops': 1}]}, {'bucket': 'myfoo2', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo3', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo4', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 217, 'bytes_received': 0, 'ops': 5, 'successful_ops': 4}], 'total': {'bytes_sent': 217, 'bytes_received': 0, 'ops': 5, 'successful_ops': 4}}]} 2023-10-05T19:41:28.076 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'bar', '--display-name', 'Bar', '--access-key', 'NX5QOQKC6BH2IDN8HC7A', '--secret', 'LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn', '--max-buckets', '4'] 2023-10-05T19:41:28.077 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'bar', '--display-name', 'Bar', '--access-key', 'NX5QOQKC6BH2IDN8HC7A', '--secret', 'LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn', '--max-buckets', '4'] 2023-10-05T19:41:28.077 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid bar --display-name Bar --access-key NX5QOQKC6BH2IDN8HC7A --secret LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn --max-buckets 4 2023-10-05T19:41:28.125 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:41:28.125 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:41:28.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.120+0000 7f33b679fe40 1 Processor -- start 2023-10-05T19:41:28.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.120+0000 7f33b679fe40 1 -- start start 2023-10-05T19:41:28.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.120+0000 7f33b679fe40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55e41bc9ab80 0x55e41bd29c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:28.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.120+0000 7f33b679fe40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55e41bc73bd0 con 0x55e41bd2a170 2023-10-05T19:41:28.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.120+0000 7f33b679fe40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x55e41bc30ee0 con 0x55e41bc9ab80 2023-10-05T19:41:28.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.120+0000 7f33b42d4700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55e41bc9ab80 0x55e41bd29c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:28.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.120+0000 7f33b3ad3700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x55e41bd2a170 0x55e41bd2c550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:57694/0 (socket says 172.21.15.12:57694) 2023-10-05T19:41:28.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.120+0000 7f33b42d4700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55e41bc9ab80 0x55e41bd29c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:50678/0 (socket says 172.21.15.12:50678) 2023-10-05T19:41:28.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.120+0000 7f33b3ad3700 1 -- 172.21.15.12:0/4203114275 learned_addr learned my addr 172.21.15.12:0/4203114275 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:41:28.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.124+0000 7f33b42d4700 1 -- 172.21.15.12:0/4203114275 >> v1:172.21.15.8:6789/0 conn(0x55e41bd2a170 legacy=0x55e41bd2c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:28.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.124+0000 7f33b32d2700 1 -- 172.21.15.12:0/4203114275 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 973549288 0 0) 0x55e41bc73bd0 con 0x55e41bd2a170 2023-10-05T19:41:28.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.124+0000 7f33b42d4700 1 -- 172.21.15.12:0/4203114275 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e41bd2ec80 con 0x55e41bc9ab80 2023-10-05T19:41:28.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.124+0000 7f33b42d4700 1 --2- 172.21.15.12:0/4203114275 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e41bc9ab80 0x55e41bd29c30 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f33a400a5d0 tx=0x7f33a4005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa53d5eac2b6e9d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:28.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.124+0000 7f33b32d2700 1 -- 172.21.15.12:0/4203114275 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f33a40086a0 con 0x55e41bc9ab80 2023-10-05T19:41:28.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.124+0000 7f33b32d2700 1 -- 172.21.15.12:0/4203114275 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a4008800 con 0x55e41bc9ab80 2023-10-05T19:41:28.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.124+0000 7f33b32d2700 1 -- 172.21.15.12:0/4203114275 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f33a4008ad0 con 0x55e41bc9ab80 2023-10-05T19:41:28.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.124+0000 7f33b679fe40 1 -- 172.21.15.12:0/4203114275 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e41bc9ab80 msgr2=0x55e41bd29c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:28.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.124+0000 7f33b679fe40 1 --2- 172.21.15.12:0/4203114275 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e41bc9ab80 0x55e41bd29c30 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f33a400a5d0 tx=0x7f33a4005a50 comp rx=0 tx=0).stop 2023-10-05T19:41:28.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.124+0000 7f33b679fe40 1 -- 172.21.15.12:0/4203114275 shutdown_connections 2023-10-05T19:41:28.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.124+0000 7f33b679fe40 1 --2- 172.21.15.12:0/4203114275 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e41bc9ab80 0x55e41bd29c30 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:28.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.124+0000 7f33b679fe40 1 -- 172.21.15.12:0/4203114275 >> 172.21.15.12:0/4203114275 conn(0x55e41bd21780 msgr2=0x55e41bd23b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:28.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.124+0000 7f33b679fe40 1 -- 172.21.15.12:0/4203114275 shutdown_connections 2023-10-05T19:41:28.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.124+0000 7f33b679fe40 1 -- 172.21.15.12:0/4203114275 wait complete. 2023-10-05T19:41:28.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.124+0000 7f33b679fe40 1 Processor -- start 2023-10-05T19:41:28.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.124+0000 7f33b679fe40 1 -- start start 2023-10-05T19:41:28.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.124+0000 7f33b679fe40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e41bc9ab80 0x55e41bd271b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:28.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.124+0000 7f33b679fe40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e41bbf4e30 con 0x55e41bc9ab80 2023-10-05T19:41:28.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.124+0000 7f33b42d4700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e41bc9ab80 0x55e41bd271b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:28.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.124+0000 7f33b42d4700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e41bc9ab80 0x55e41bd271b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:50680/0 (socket says 172.21.15.12:50680) 2023-10-05T19:41:28.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.124+0000 7f33b42d4700 1 -- 172.21.15.12:0/2016650182 learned_addr learned my addr 172.21.15.12:0/2016650182 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:28.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.124+0000 7f33b42d4700 1 -- 172.21.15.12:0/2016650182 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a4005cf0 con 0x55e41bc9ab80 2023-10-05T19:41:28.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.128+0000 7f33b42d4700 1 --2- 172.21.15.12:0/2016650182 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e41bc9ab80 0x55e41bd271b0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f33a4005a90 tx=0x7f33a4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe20af752f3faf46 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:28.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.128+0000 7f33b22d0700 1 -- 172.21.15.12:0/2016650182 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f33a4008a40 con 0x55e41bc9ab80 2023-10-05T19:41:28.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.128+0000 7f33b22d0700 1 -- 172.21.15.12:0/2016650182 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a4008bc0 con 0x55e41bc9ab80 2023-10-05T19:41:28.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.128+0000 7f33b22d0700 1 -- 172.21.15.12:0/2016650182 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f33a4012450 con 0x55e41bc9ab80 2023-10-05T19:41:28.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.128+0000 7f33b679fe40 1 -- 172.21.15.12:0/2016650182 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e41bc9ab80 msgr2=0x55e41bd271b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:28.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.128+0000 7f33b679fe40 1 --2- 172.21.15.12:0/2016650182 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e41bc9ab80 0x55e41bd271b0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f33a4005a90 tx=0x7f33a4015000 comp rx=0 tx=0).stop 2023-10-05T19:41:28.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.128+0000 7f33b679fe40 1 -- 172.21.15.12:0/2016650182 shutdown_connections 2023-10-05T19:41:28.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.128+0000 7f33b679fe40 1 --2- 172.21.15.12:0/2016650182 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e41bc9ab80 0x55e41bd271b0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:28.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.128+0000 7f33b679fe40 1 -- 172.21.15.12:0/2016650182 >> 172.21.15.12:0/2016650182 conn(0x55e41bd21780 msgr2=0x55e41bd41930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:28.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.128+0000 7f33b679fe40 1 -- 172.21.15.12:0/2016650182 shutdown_connections 2023-10-05T19:41:28.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.128+0000 7f33b679fe40 1 -- 172.21.15.12:0/2016650182 wait complete. 2023-10-05T19:41:28.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.128+0000 7f33b679fe40 1 Processor -- start 2023-10-05T19:41:28.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.128+0000 7f33b679fe40 1 -- start start 2023-10-05T19:41:28.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.128+0000 7f33b679fe40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e41bc9ab80 0x55e41bd4ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:28.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.128+0000 7f33b679fe40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e41bbc7ed0 con 0x55e41bc9ab80 2023-10-05T19:41:28.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.128+0000 7f33b42d4700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e41bc9ab80 0x55e41bd4ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:28.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.128+0000 7f33b42d4700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e41bc9ab80 0x55e41bd4ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:50692/0 (socket says 172.21.15.12:50692) 2023-10-05T19:41:28.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.128+0000 7f33b42d4700 1 -- 172.21.15.12:0/2032180726 learned_addr learned my addr 172.21.15.12:0/2032180726 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:28.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.128+0000 7f33b42d4700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e41bd2ec80 con 0x55e41bc9ab80 2023-10-05T19:41:28.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.128+0000 7f33b42d4700 1 --2- 172.21.15.12:0/2032180726 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e41bc9ab80 0x55e41bd4ebb0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f33a400c010 tx=0x7f33a401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:28.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.132+0000 7f33b0acd700 1 -- 172.21.15.12:0/2032180726 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f33a4008e50 con 0x55e41bc9ab80 2023-10-05T19:41:28.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.132+0000 7f33b0acd700 1 -- 172.21.15.12:0/2032180726 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a4012660 con 0x55e41bc9ab80 2023-10-05T19:41:28.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.132+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e41bd4f0f0 con 0x55e41bc9ab80 2023-10-05T19:41:28.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.132+0000 7f33b0acd700 1 -- 172.21.15.12:0/2032180726 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f33a4012950 con 0x55e41bc9ab80 2023-10-05T19:41:28.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.132+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e41bd4f590 con 0x55e41bc9ab80 2023-10-05T19:41:28.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.132+0000 7f33b679fe40 20 rados->read ofs=0 len=0 2023-10-05T19:41:28.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.132+0000 7f33b0acd700 1 -- 172.21.15.12:0/2032180726 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f33a4019070 con 0x55e41bc9ab80 2023-10-05T19:41:28.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.132+0000 7f33b0acd700 1 --2- 172.21.15.12:0/2032180726 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3394030c50 0x7f3394033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:28.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.132+0000 7f33b0acd700 1 -- 172.21.15.12:0/2032180726 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7f33a40383c0 con 0x55e41bc9ab80 2023-10-05T19:41:28.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.132+0000 7f33b3ad3700 1 --2- 172.21.15.12:0/2032180726 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3394030c50 0x7f3394033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:28.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.132+0000 7f33b679fe40 1 --2- 172.21.15.12:0/2032180726 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55e41bd2a170 0x55e41bd5a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:28.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.132+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd5ad90 con 0x55e41bd2a170 2023-10-05T19:41:28.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.132+0000 7f33b4ad5700 1 --2- 172.21.15.12:0/2032180726 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55e41bd2a170 0x55e41bd5a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:28.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.132+0000 7f33b3ad3700 1 --2- 172.21.15.12:0/2032180726 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3394030c50 0x7f3394033100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f339c001560 tx=0x7f339c0065f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:28.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.136+0000 7f33b4ad5700 1 --2- 172.21.15.12:0/2032180726 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55e41bd2a170 0x55e41bd5a770 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:28.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.136+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f33ac00a500 con 0x55e41bd2a170 2023-10-05T19:41:28.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.136+0000 7f33b679fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:28.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.136+0000 7f33b679fe40 20 realm 2023-10-05T19:41:28.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.136+0000 7f33b679fe40 20 rados->read ofs=0 len=0 2023-10-05T19:41:28.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.136+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd57dd0 con 0x55e41bd2a170 2023-10-05T19:41:28.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.136+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f33ac00a500 con 0x55e41bd2a170 2023-10-05T19:41:28.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.136+0000 7f33b679fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:28.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.136+0000 7f33b679fe40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:41:28.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.136+0000 7f33b679fe40 20 rados->read ofs=0 len=0 2023-10-05T19:41:28.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.136+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd5b870 con 0x55e41bd2a170 2023-10-05T19:41:28.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.136+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f33ac00a500 con 0x55e41bd2a170 2023-10-05T19:41:28.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.136+0000 7f33b679fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:28.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.136+0000 7f33b679fe40 20 rados->read ofs=0 len=0 2023-10-05T19:41:28.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.136+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd5b480 con 0x55e41bd2a170 2023-10-05T19:41:28.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.136+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f33ac00a500 con 0x55e41bd2a170 2023-10-05T19:41:28.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.136+0000 7f33b679fe40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:28.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.136+0000 7f33b679fe40 20 rados->read ofs=0 len=0 2023-10-05T19:41:28.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.136+0000 7f33b679fe40 1 --2- 172.21.15.12:0/2032180726 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55e41bd5cc10 0x55e41bd5eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:28.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.136+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd5f530 con 0x55e41bd5cc10 2023-10-05T19:41:28.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.136+0000 7f33b42d4700 1 --2- 172.21.15.12:0/2032180726 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55e41bd5cc10 0x55e41bd5eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:28.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.136+0000 7f33b42d4700 1 --2- 172.21.15.12:0/2032180726 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55e41bd5cc10 0x55e41bd5eff0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:28.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.136+0000 7f33b42d4700 1 -- 172.21.15.12:0/2032180726 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f33a4012ab0 con 0x55e41bd5cc10 2023-10-05T19:41:28.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.136+0000 7f33b679fe40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:41:28.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.140+0000 7f33b679fe40 20 searching for the correct realm 2023-10-05T19:41:28.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.140+0000 7f33b679fe40 1 --2- 172.21.15.12:0/2032180726 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55e41bd613b0 0x55e41bd637c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:28.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.140+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd63e70 con 0x55e41bd613b0 2023-10-05T19:41:28.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.140+0000 7f33b3ad3700 1 --2- 172.21.15.12:0/2032180726 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55e41bd613b0 0x55e41bd637c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:28.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.140+0000 7f33b3ad3700 1 --2- 172.21.15.12:0/2032180726 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55e41bd613b0 0x55e41bd637c0 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:28.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.140+0000 7f33b3ad3700 1 -- 172.21.15.12:0/2032180726 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f339c004520 con 0x55e41bd613b0 2023-10-05T19:41:28.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.140+0000 7f33b3ad3700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd63e70 con 0x55e41bd5cc10 2023-10-05T19:41:28.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.144+0000 7f33b42d4700 1 -- 172.21.15.12:0/2032180726 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f33a4012ab0 con 0x55e41bd5cc10 2023-10-05T19:41:28.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.144+0000 7f33b42d4700 1 --2- 172.21.15.12:0/2032180726 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f33a4007030 0x7f33a400c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:28.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.144+0000 7f33b42d4700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd63e70 con 0x7f33a4007030 2023-10-05T19:41:28.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.144+0000 7f33b4ad5700 1 --2- 172.21.15.12:0/2032180726 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f33a4007030 0x7f33a400c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:28.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.144+0000 7f33b4ad5700 1 --2- 172.21.15.12:0/2032180726 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f33a4007030 0x7f33a400c0b0 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:28.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.144+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f33ac014040 con 0x7f33a4007030 2023-10-05T19:41:28.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.144+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd63e70 con 0x55e41bd2a170 2023-10-05T19:41:28.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.144+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f33ac014040 con 0x55e41bd2a170 2023-10-05T19:41:28.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.144+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd63e70 con 0x55e41bd2a170 2023-10-05T19:41:28.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.148+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f33ac014040 con 0x55e41bd2a170 2023-10-05T19:41:28.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.148+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd63e70 con 0x55e41bd2a170 2023-10-05T19:41:28.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.148+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(11 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f33ac014040 con 0x55e41bd2a170 2023-10-05T19:41:28.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.148+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd63e70 con 0x7f33a4007030 2023-10-05T19:41:28.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.148+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f33ac014040 con 0x7f33a4007030 2023-10-05T19:41:28.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.148+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd63e70 con 0x55e41bd5cc10 2023-10-05T19:41:28.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.148+0000 7f33b42d4700 1 -- 172.21.15.12:0/2032180726 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f33a4012ab0 con 0x55e41bd5cc10 2023-10-05T19:41:28.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.148+0000 7f33b42d4700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd63e70 con 0x55e41bd5cc10 2023-10-05T19:41:28.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.148+0000 7f33b42d4700 1 -- 172.21.15.12:0/2032180726 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f33a4012ab0 con 0x55e41bd5cc10 2023-10-05T19:41:28.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.148+0000 7f33b42d4700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd63e70 con 0x55e41bd2a170 2023-10-05T19:41:28.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.148+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f33ac014040 con 0x55e41bd2a170 2023-10-05T19:41:28.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.148+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd63e70 con 0x7f33a4007030 2023-10-05T19:41:28.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.152+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f33ac014040 con 0x7f33a4007030 2023-10-05T19:41:28.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.152+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd63e70 con 0x55e41bd2a170 2023-10-05T19:41:28.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.152+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f33ac014040 con 0x55e41bd2a170 2023-10-05T19:41:28.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.152+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd63e70 con 0x55e41bd5cc10 2023-10-05T19:41:28.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.152+0000 7f33b42d4700 1 -- 172.21.15.12:0/2032180726 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(18 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f33a4012ab0 con 0x55e41bd5cc10 2023-10-05T19:41:28.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.152+0000 7f33b42d4700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd63e70 con 0x55e41bd613b0 2023-10-05T19:41:28.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.152+0000 7f33b3ad3700 1 -- 172.21.15.12:0/2032180726 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f339c004520 con 0x55e41bd613b0 2023-10-05T19:41:28.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.152+0000 7f33b3ad3700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd63e70 con 0x55e41bd613b0 2023-10-05T19:41:28.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.152+0000 7f33b3ad3700 1 -- 172.21.15.12:0/2032180726 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f339c004520 con 0x55e41bd613b0 2023-10-05T19:41:28.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.152+0000 7f33b3ad3700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd63e70 con 0x7f33a4007030 2023-10-05T19:41:28.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.156+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f33ac014040 con 0x7f33a4007030 2023-10-05T19:41:28.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.156+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd63e70 con 0x55e41bd613b0 2023-10-05T19:41:28.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.156+0000 7f33b3ad3700 1 -- 172.21.15.12:0/2032180726 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f339c01d040 con 0x55e41bd613b0 2023-10-05T19:41:28.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.156+0000 7f33b3ad3700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd63e70 con 0x55e41bd613b0 2023-10-05T19:41:28.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.156+0000 7f33b3ad3700 1 -- 172.21.15.12:0/2032180726 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f339c01d040 con 0x55e41bd613b0 2023-10-05T19:41:28.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.156+0000 7f33b3ad3700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd63e70 con 0x55e41bd2a170 2023-10-05T19:41:28.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.156+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f33ac014040 con 0x55e41bd2a170 2023-10-05T19:41:28.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.156+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd63e70 con 0x55e41bd2a170 2023-10-05T19:41:28.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.156+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f33ac014040 con 0x55e41bd2a170 2023-10-05T19:41:28.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.156+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd63e70 con 0x7f33a4007030 2023-10-05T19:41:28.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.160+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f33ac014040 con 0x7f33a4007030 2023-10-05T19:41:28.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.160+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd63e70 con 0x55e41bd613b0 2023-10-05T19:41:28.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.160+0000 7f33b3ad3700 1 -- 172.21.15.12:0/2032180726 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f339c01d040 con 0x55e41bd613b0 2023-10-05T19:41:28.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.160+0000 7f33b3ad3700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd63e70 con 0x55e41bd5cc10 2023-10-05T19:41:28.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.160+0000 7f33b42d4700 1 -- 172.21.15.12:0/2032180726 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f33a4012ab0 con 0x55e41bd5cc10 2023-10-05T19:41:28.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.160+0000 7f33b42d4700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd63e70 con 0x55e41bd2a170 2023-10-05T19:41:28.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.160+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f33ac014040 con 0x55e41bd2a170 2023-10-05T19:41:28.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.160+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd63e70 con 0x7f33a4007030 2023-10-05T19:41:28.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.164+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f33ac014040 con 0x7f33a4007030 2023-10-05T19:41:28.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.164+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd63e70 con 0x55e41bd613b0 2023-10-05T19:41:28.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.164+0000 7f33b3ad3700 1 -- 172.21.15.12:0/2032180726 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f339c01d040 con 0x55e41bd613b0 2023-10-05T19:41:28.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.164+0000 7f33b3ad3700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd63e70 con 0x7f33a4007030 2023-10-05T19:41:28.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.164+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f33ac014040 con 0x7f33a4007030 2023-10-05T19:41:28.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.164+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd63e70 con 0x55e41bd5cc10 2023-10-05T19:41:28.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.164+0000 7f33b42d4700 1 -- 172.21.15.12:0/2032180726 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f33a4012ab0 con 0x55e41bd5cc10 2023-10-05T19:41:28.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.164+0000 7f33b42d4700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd63e70 con 0x55e41bd613b0 2023-10-05T19:41:28.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.168+0000 7f33b3ad3700 1 -- 172.21.15.12:0/2032180726 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f339c01d040 con 0x55e41bd613b0 2023-10-05T19:41:28.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.168+0000 7f33b3ad3700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd63e70 con 0x7f33a4007030 2023-10-05T19:41:28.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.168+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f33ac014040 con 0x7f33a4007030 2023-10-05T19:41:28.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.168+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd63e70 con 0x55e41bd2a170 2023-10-05T19:41:28.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.168+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f33ac014040 con 0x55e41bd2a170 2023-10-05T19:41:28.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.168+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd63e70 con 0x55e41bd5cc10 2023-10-05T19:41:28.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.168+0000 7f33b42d4700 1 -- 172.21.15.12:0/2032180726 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(37 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f33a4012ab0 con 0x55e41bd5cc10 2023-10-05T19:41:28.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.168+0000 7f33b679fe40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:41:28.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.168+0000 7f33b679fe40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:41:28.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.168+0000 7f33b679fe40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:41:28.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.168+0000 7f33b679fe40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:41:28.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.168+0000 7f33b679fe40 20 rados->read ofs=0 len=0 2023-10-05T19:41:28.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.168+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd60100 con 0x55e41bd2a170 2023-10-05T19:41:28.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.168+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f33ac014040 con 0x55e41bd2a170 2023-10-05T19:41:28.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.168+0000 7f33b679fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:28.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.168+0000 7f33b679fe40 20 rados->read ofs=0 len=0 2023-10-05T19:41:28.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.168+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd5fd30 con 0x55e41bd2a170 2023-10-05T19:41:28.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.172+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f33ac014040 con 0x55e41bd2a170 2023-10-05T19:41:28.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.172+0000 7f33b679fe40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:28.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.172+0000 7f33b679fe40 20 rados->read ofs=0 len=0 2023-10-05T19:41:28.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.172+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd642e0 con 0x55e41bd613b0 2023-10-05T19:41:28.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.172+0000 7f33b3ad3700 1 -- 172.21.15.12:0/2032180726 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f339c01d040 con 0x55e41bd613b0 2023-10-05T19:41:28.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.172+0000 7f33b679fe40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:41:28.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.172+0000 7f33b679fe40 20 rados->read ofs=0 len=0 2023-10-05T19:41:28.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.172+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd64690 con 0x55e41bd613b0 2023-10-05T19:41:28.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.172+0000 7f33b3ad3700 1 -- 172.21.15.12:0/2032180726 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f339c01d040 con 0x55e41bd613b0 2023-10-05T19:41:28.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.172+0000 7f33b679fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:28.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.172+0000 7f33b679fe40 20 rados->read ofs=0 len=0 2023-10-05T19:41:28.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.172+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd64a40 con 0x55e41bd2a170 2023-10-05T19:41:28.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.172+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f33ac014040 con 0x55e41bd2a170 2023-10-05T19:41:28.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.172+0000 7f33b679fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:28.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.172+0000 7f33b679fe40 20 rados->read ofs=0 len=0 2023-10-05T19:41:28.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.172+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd60620 con 0x55e41bd2a170 2023-10-05T19:41:28.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.172+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f33ac014040 con 0x55e41bd2a170 2023-10-05T19:41:28.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.172+0000 7f33b679fe40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:28.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.172+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd60620 con 0x55e41bd613b0 2023-10-05T19:41:28.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.176+0000 7f33b3ad3700 1 -- 172.21.15.12:0/2032180726 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(44 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f339c01d040 con 0x55e41bd613b0 2023-10-05T19:41:28.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.176+0000 7f33b3ad3700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd60620 con 0x55e41bd5cc10 2023-10-05T19:41:28.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.176+0000 7f33b42d4700 1 -- 172.21.15.12:0/2032180726 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(45 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f33a4012ab0 con 0x55e41bd5cc10 2023-10-05T19:41:28.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.176+0000 7f33b42d4700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd60620 con 0x7f33a4007030 2023-10-05T19:41:28.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.180+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(46 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f33ac014040 con 0x7f33a4007030 2023-10-05T19:41:28.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.180+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd64a40 con 0x55e41bd2a170 2023-10-05T19:41:28.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.180+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(47 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f33ac014040 con 0x55e41bd2a170 2023-10-05T19:41:28.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.180+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd64a40 con 0x55e41bd2a170 2023-10-05T19:41:28.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.180+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(48 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f33ac014040 con 0x55e41bd2a170 2023-10-05T19:41:28.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.180+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd64a40 con 0x55e41bd2a170 2023-10-05T19:41:28.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.180+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(49 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f33ac014040 con 0x55e41bd2a170 2023-10-05T19:41:28.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.180+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd64a40 con 0x7f33a4007030 2023-10-05T19:41:28.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.180+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(50 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f33ac014040 con 0x7f33a4007030 2023-10-05T19:41:28.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.180+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd64a40 con 0x55e41bd5cc10 2023-10-05T19:41:28.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.184+0000 7f33b42d4700 1 -- 172.21.15.12:0/2032180726 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(51 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f33a4012ab0 con 0x55e41bd5cc10 2023-10-05T19:41:28.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.184+0000 7f33b42d4700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd64a40 con 0x55e41bd5cc10 2023-10-05T19:41:28.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.184+0000 7f33b42d4700 1 -- 172.21.15.12:0/2032180726 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(52 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f33a4012ab0 con 0x55e41bd5cc10 2023-10-05T19:41:28.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.184+0000 7f33b42d4700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd64a40 con 0x55e41bd2a170 2023-10-05T19:41:28.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.184+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(53 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f33ac014040 con 0x55e41bd2a170 2023-10-05T19:41:28.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.184+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd64a40 con 0x7f33a4007030 2023-10-05T19:41:28.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.184+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(54 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f33ac014040 con 0x7f33a4007030 2023-10-05T19:41:28.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.184+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd64a40 con 0x55e41bd2a170 2023-10-05T19:41:28.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.184+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(55 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f33ac014040 con 0x55e41bd2a170 2023-10-05T19:41:28.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.184+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd64a40 con 0x55e41bd5cc10 2023-10-05T19:41:28.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.188+0000 7f33b42d4700 1 -- 172.21.15.12:0/2032180726 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(56 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f33a4012ab0 con 0x55e41bd5cc10 2023-10-05T19:41:28.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.188+0000 7f33b42d4700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd64a40 con 0x55e41bd613b0 2023-10-05T19:41:28.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.188+0000 7f33b3ad3700 1 -- 172.21.15.12:0/2032180726 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(57 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f339c01d040 con 0x55e41bd613b0 2023-10-05T19:41:28.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.188+0000 7f33b3ad3700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd64a40 con 0x55e41bd613b0 2023-10-05T19:41:28.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.188+0000 7f33b3ad3700 1 -- 172.21.15.12:0/2032180726 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(58 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f339c02d040 con 0x55e41bd613b0 2023-10-05T19:41:28.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.188+0000 7f33b3ad3700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd64a40 con 0x7f33a4007030 2023-10-05T19:41:28.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.188+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(59 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f33ac014040 con 0x7f33a4007030 2023-10-05T19:41:28.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.188+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd64a40 con 0x55e41bd613b0 2023-10-05T19:41:28.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.192+0000 7f33b3ad3700 1 -- 172.21.15.12:0/2032180726 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(60 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f339c02d040 con 0x55e41bd613b0 2023-10-05T19:41:28.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.192+0000 7f33b3ad3700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd64a40 con 0x55e41bd613b0 2023-10-05T19:41:28.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.192+0000 7f33b3ad3700 1 -- 172.21.15.12:0/2032180726 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(61 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f339c02d040 con 0x55e41bd613b0 2023-10-05T19:41:28.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.192+0000 7f33b3ad3700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd64a40 con 0x55e41bd2a170 2023-10-05T19:41:28.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.192+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(62 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f33ac014040 con 0x55e41bd2a170 2023-10-05T19:41:28.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.192+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd64a40 con 0x55e41bd2a170 2023-10-05T19:41:28.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.192+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(63 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f33ac014040 con 0x55e41bd2a170 2023-10-05T19:41:28.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.192+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd64a40 con 0x7f33a4007030 2023-10-05T19:41:28.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.192+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(64 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f33ac014040 con 0x7f33a4007030 2023-10-05T19:41:28.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.192+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd64a40 con 0x55e41bd613b0 2023-10-05T19:41:28.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.196+0000 7f33b3ad3700 1 -- 172.21.15.12:0/2032180726 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(65 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f339c02d040 con 0x55e41bd613b0 2023-10-05T19:41:28.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.196+0000 7f33b3ad3700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd64a40 con 0x55e41bd5cc10 2023-10-05T19:41:28.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.196+0000 7f33b42d4700 1 -- 172.21.15.12:0/2032180726 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(66 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f33a4012ab0 con 0x55e41bd5cc10 2023-10-05T19:41:28.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.196+0000 7f33b42d4700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd64a40 con 0x55e41bd2a170 2023-10-05T19:41:28.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.196+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(67 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f33ac014040 con 0x55e41bd2a170 2023-10-05T19:41:28.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.196+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd64a40 con 0x7f33a4007030 2023-10-05T19:41:28.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.196+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(68 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f33ac014040 con 0x7f33a4007030 2023-10-05T19:41:28.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.196+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd64a40 con 0x55e41bd613b0 2023-10-05T19:41:28.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.200+0000 7f33b3ad3700 1 -- 172.21.15.12:0/2032180726 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(69 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f339c02d040 con 0x55e41bd613b0 2023-10-05T19:41:28.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.200+0000 7f33b3ad3700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd64a40 con 0x7f33a4007030 2023-10-05T19:41:28.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.200+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(70 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f33ac014040 con 0x7f33a4007030 2023-10-05T19:41:28.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.200+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd64a40 con 0x55e41bd5cc10 2023-10-05T19:41:28.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.200+0000 7f33b42d4700 1 -- 172.21.15.12:0/2032180726 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(71 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f33a4012ab0 con 0x55e41bd5cc10 2023-10-05T19:41:28.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.200+0000 7f33b42d4700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd64a40 con 0x55e41bd613b0 2023-10-05T19:41:28.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.200+0000 7f33b3ad3700 1 -- 172.21.15.12:0/2032180726 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(72 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f339c02d040 con 0x55e41bd613b0 2023-10-05T19:41:28.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.200+0000 7f33b3ad3700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd64a40 con 0x7f33a4007030 2023-10-05T19:41:28.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.200+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(73 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f33ac014040 con 0x7f33a4007030 2023-10-05T19:41:28.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.200+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd64a40 con 0x55e41bd2a170 2023-10-05T19:41:28.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.204+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(74 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f33ac014040 con 0x55e41bd2a170 2023-10-05T19:41:28.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.204+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd64a40 con 0x55e41bd5cc10 2023-10-05T19:41:28.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.204+0000 7f33b42d4700 1 -- 172.21.15.12:0/2032180726 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(75 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f33a4012ab0 con 0x55e41bd5cc10 2023-10-05T19:41:28.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.204+0000 7f33b679fe40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:41:28.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.204+0000 7f33b679fe40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:41:28.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.204+0000 7f33b679fe40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:41:28.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.204+0000 7f33b679fe40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:41:28.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.204+0000 7f33b679fe40 20 rados->read ofs=0 len=0 2023-10-05T19:41:28.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.204+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd60620 con 0x55e41bd2a170 2023-10-05T19:41:28.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.204+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f33ac014040 con 0x55e41bd2a170 2023-10-05T19:41:28.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.204+0000 7f33b679fe40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:28.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.204+0000 7f33b679fe40 20 rados->read ofs=0 len=0 2023-10-05T19:41:28.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.204+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd65390 con 0x55e41bd5cc10 2023-10-05T19:41:28.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.204+0000 7f33b42d4700 1 -- 172.21.15.12:0/2032180726 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f33a4012ab0 con 0x55e41bd5cc10 2023-10-05T19:41:28.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.204+0000 7f33b679fe40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:41:28.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.204+0000 7f33b679fe40 20 rados->read ofs=0 len=0 2023-10-05T19:41:28.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.204+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd65c80 con 0x55e41bd2a170 2023-10-05T19:41:28.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.208+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f33ac014040 con 0x55e41bd2a170 2023-10-05T19:41:28.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.208+0000 7f33b679fe40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:28.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.208+0000 7f33b679fe40 20 rados->read ofs=0 len=0 2023-10-05T19:41:28.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.208+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd65c80 con 0x55e41bd613b0 2023-10-05T19:41:28.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.208+0000 7f33b3ad3700 1 -- 172.21.15.12:0/2032180726 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f339c02d040 con 0x55e41bd613b0 2023-10-05T19:41:28.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.208+0000 7f33b679fe40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:41:28.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.208+0000 7f33b679fe40 20 rados->read ofs=0 len=0 2023-10-05T19:41:28.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.208+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd663f0 con 0x7f33a4007030 2023-10-05T19:41:28.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.208+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f33ac014040 con 0x7f33a4007030 2023-10-05T19:41:28.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.208+0000 7f33b679fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:28.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.208+0000 7f33b679fe40 20 zone default found 2023-10-05T19:41:28.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.208+0000 7f33b679fe40 4 Realm: () 2023-10-05T19:41:28.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.208+0000 7f33b679fe40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:41:28.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.208+0000 7f33b679fe40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:41:28.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.208+0000 7f33b679fe40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:41:28.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.208+0000 7f33b679fe40 20 zonegroup default 2023-10-05T19:41:28.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.208+0000 7f33b679fe40 20 rados->read ofs=0 len=0 2023-10-05T19:41:28.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.208+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd663f0 con 0x7f33a4007030 2023-10-05T19:41:28.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.208+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f33ac014040 con 0x7f33a4007030 2023-10-05T19:41:28.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.208+0000 7f33b679fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:28.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.208+0000 7f33b679fe40 20 rados->read ofs=0 len=0 2023-10-05T19:41:28.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.208+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd67d70 con 0x55e41bd613b0 2023-10-05T19:41:28.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.212+0000 7f33b3ad3700 1 -- 172.21.15.12:0/2032180726 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f339c02d040 con 0x55e41bd613b0 2023-10-05T19:41:28.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.212+0000 7f33b679fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:28.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.212+0000 7f33b679fe40 20 rados->read ofs=0 len=0 2023-10-05T19:41:28.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.212+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd68120 con 0x55e41bd613b0 2023-10-05T19:41:28.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.212+0000 7f33b3ad3700 1 -- 172.21.15.12:0/2032180726 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f339c02d040 con 0x55e41bd613b0 2023-10-05T19:41:28.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.212+0000 7f33b679fe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:28.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.212+0000 7f33b679fe40 20 started sync module instance, tier type = 2023-10-05T19:41:28.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.212+0000 7f33b679fe40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:41:28.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.212+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd67430 con 0x55e41bd5cc10 2023-10-05T19:41:28.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.216+0000 7f33b42d4700 1 -- 172.21.15.12:0/2032180726 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(84 notify.0 [create] v30'240 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f33a4012ab0 con 0x55e41bd5cc10 2023-10-05T19:41:28.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.216+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94438207936400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd684d0 con 0x55e41bd5cc10 2023-10-05T19:41:28.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.216+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd68d70 con 0x55e41bd2a170 2023-10-05T19:41:28.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.216+0000 7f33b42d4700 1 -- 172.21.15.12:0/2032180726 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(85 notify.0 [watch watch cookie 94438207936400] v30'241 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f33a4012ab0 con 0x55e41bd5cc10 2023-10-05T19:41:28.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.216+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(86 notify.1 [create] v30'240 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f33ac014040 con 0x55e41bd2a170 2023-10-05T19:41:28.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.216+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94438207948928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd691c0 con 0x55e41bd2a170 2023-10-05T19:41:28.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.216+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd69a60 con 0x55e41bd613b0 2023-10-05T19:41:28.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.220+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(87 notify.1 [watch watch cookie 94438207948928] v30'241 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f33ac014040 con 0x55e41bd2a170 2023-10-05T19:41:28.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.220+0000 7f33b3ad3700 1 -- 172.21.15.12:0/2032180726 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(88 notify.2 [create] v30'240 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f339c02d040 con 0x55e41bd613b0 2023-10-05T19:41:28.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.220+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94438207950320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd69e10 con 0x55e41bd613b0 2023-10-05T19:41:28.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.220+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd6a6b0 con 0x55e41bd5cc10 2023-10-05T19:41:28.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.224+0000 7f33b42d4700 1 -- 172.21.15.12:0/2032180726 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(90 notify.3 [create] v30'240 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f33a4012ab0 con 0x55e41bd5cc10 2023-10-05T19:41:28.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.224+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94438207957600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd6b3a0 con 0x55e41bd5cc10 2023-10-05T19:41:28.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.224+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd6bc40 con 0x55e41bd2a170 2023-10-05T19:41:28.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.224+0000 7f33b3ad3700 1 -- 172.21.15.12:0/2032180726 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(89 notify.2 [watch watch cookie 94438207950320] v30'241 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f339c02d040 con 0x55e41bd613b0 2023-10-05T19:41:28.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.228+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(92 notify.4 [create] v30'240 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f33ac014040 con 0x55e41bd2a170 2023-10-05T19:41:28.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.228+0000 7f33b42d4700 1 -- 172.21.15.12:0/2032180726 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(91 notify.3 [watch watch cookie 94438207957600] v30'241 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f33a4012ab0 con 0x55e41bd5cc10 2023-10-05T19:41:28.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.228+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94438207959968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd6bce0 con 0x55e41bd2a170 2023-10-05T19:41:28.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.228+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd6aeb0 con 0x7f33a4007030 2023-10-05T19:41:28.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.232+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(94 notify.5 [create] v30'240 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f33ac014040 con 0x7f33a4007030 2023-10-05T19:41:28.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.232+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94438207958704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd6c6f0 con 0x7f33a4007030 2023-10-05T19:41:28.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.232+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd6cf90 con 0x7f33a4007030 2023-10-05T19:41:28.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.232+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(93 notify.4 [watch watch cookie 94438207959968] v30'241 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f33ac014040 con 0x55e41bd2a170 2023-10-05T19:41:28.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.232+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(95 notify.5 [watch watch cookie 94438207958704] v30'241 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f33ac014040 con 0x7f33a4007030 2023-10-05T19:41:28.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.232+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(96 notify.6 [create] v30'240 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f33ac014040 con 0x7f33a4007030 2023-10-05T19:41:28.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.232+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94438207963648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd6cb40 con 0x7f33a4007030 2023-10-05T19:41:28.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.232+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd6d3e0 con 0x7f33a4007030 2023-10-05T19:41:28.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.236+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(97 notify.6 [watch watch cookie 94438207963648] v30'241 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f33ac014040 con 0x7f33a4007030 2023-10-05T19:41:28.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.236+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(98 notify.7 [create] v30'240 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f33ac014040 con 0x7f33a4007030 2023-10-05T19:41:28.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.236+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94438207965312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd6d1c0 con 0x7f33a4007030 2023-10-05T19:41:28.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.236+0000 7f33b679fe40 20 add_watcher() i=0 2023-10-05T19:41:28.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.236+0000 7f33b679fe40 20 add_watcher() i=1 2023-10-05T19:41:28.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.236+0000 7f33b679fe40 20 add_watcher() i=2 2023-10-05T19:41:28.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.236+0000 7f33b679fe40 20 add_watcher() i=3 2023-10-05T19:41:28.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.236+0000 7f33b679fe40 20 add_watcher() i=4 2023-10-05T19:41:28.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.236+0000 7f33b679fe40 20 add_watcher() i=5 2023-10-05T19:41:28.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.236+0000 7f33b679fe40 20 add_watcher() i=6 2023-10-05T19:41:28.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.240+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(99 notify.7 [watch watch cookie 94438207965312] v30'241 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f33ac014040 con 0x7f33a4007030 2023-10-05T19:41:28.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.240+0000 7f33b679fe40 20 add_watcher() i=7 2023-10-05T19:41:28.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.240+0000 7f33b679fe40 2 all 8 watchers are set, enabling cache 2023-10-05T19:41:28.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.240+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd705d0 con 0x55e41bd2a170 2023-10-05T19:41:28.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.240+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f33ac014040 con 0x55e41bd2a170 2023-10-05T19:41:28.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.240+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94438207966880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd6da90 con 0x55e41bd2a170 2023-10-05T19:41:28.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.240+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94438207966880] v30'231 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f33ac014040 con 0x55e41bd2a170 2023-10-05T19:41:28.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.240+0000 7f3385fe3700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:41:28.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.240+0000 7f33b679fe40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:41:28.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.240+0000 7f33b679fe40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:41:28.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.244+0000 7f33b679fe40 1 Processor -- start 2023-10-05T19:41:28.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.244+0000 7f33b679fe40 1 -- start start 2023-10-05T19:41:28.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.244+0000 7f33b679fe40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e41bd7ac90 0x55e41bd7b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:28.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.244+0000 7f33b679fe40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e41bd7b5a0 con 0x55e41bd7ac90 2023-10-05T19:41:28.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.244+0000 7f33b3ad3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e41bd7ac90 0x55e41bd7b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:28.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.244+0000 7f33b3ad3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e41bd7ac90 0x55e41bd7b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:50704/0 (socket says 172.21.15.12:50704) 2023-10-05T19:41:28.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.244+0000 7f33b3ad3700 1 -- 172.21.15.12:0/4066661237 learned_addr learned my addr 172.21.15.12:0/4066661237 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:28.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.244+0000 7f33b3ad3700 1 -- 172.21.15.12:0/4066661237 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e41bd7b6e0 con 0x55e41bd7ac90 2023-10-05T19:41:28.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.244+0000 7f33b3ad3700 1 --2- 172.21.15.12:0/4066661237 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e41bd7ac90 0x55e41bd7b060 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f33a4036ef0 tx=0x7f339c027180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a30793ab00c3e4d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:28.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.244+0000 7f33857e2700 1 -- 172.21.15.12:0/4066661237 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f339c039040 con 0x55e41bd7ac90 2023-10-05T19:41:28.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.244+0000 7f33857e2700 1 -- 172.21.15.12:0/4066661237 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f339c046070 con 0x55e41bd7ac90 2023-10-05T19:41:28.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.244+0000 7f33857e2700 1 -- 172.21.15.12:0/4066661237 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f339c02a040 con 0x55e41bd7ac90 2023-10-05T19:41:28.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.244+0000 7f33b679fe40 1 -- 172.21.15.12:0/4066661237 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e41bd7ac90 msgr2=0x55e41bd7b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:28.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.244+0000 7f33b679fe40 1 --2- 172.21.15.12:0/4066661237 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e41bd7ac90 0x55e41bd7b060 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f33a4036ef0 tx=0x7f339c027180 comp rx=0 tx=0).stop 2023-10-05T19:41:28.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.244+0000 7f33b679fe40 1 -- 172.21.15.12:0/4066661237 shutdown_connections 2023-10-05T19:41:28.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.244+0000 7f33b679fe40 1 --2- 172.21.15.12:0/4066661237 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e41bd7ac90 0x55e41bd7b060 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:28.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.244+0000 7f33b679fe40 1 -- 172.21.15.12:0/4066661237 >> 172.21.15.12:0/4066661237 conn(0x55e41bd76050 msgr2=0x55e41bd741a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:28.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.244+0000 7f33b679fe40 1 -- 172.21.15.12:0/4066661237 shutdown_connections 2023-10-05T19:41:28.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.244+0000 7f33b679fe40 1 -- 172.21.15.12:0/4066661237 wait complete. 2023-10-05T19:41:28.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.248+0000 7f33b679fe40 1 Processor -- start 2023-10-05T19:41:28.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.248+0000 7f33b679fe40 1 -- start start 2023-10-05T19:41:28.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.248+0000 7f33b679fe40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e41bd7ac90 0x55e41bd807f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:28.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.248+0000 7f33b679fe40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e41bd817f0 con 0x55e41bd7ac90 2023-10-05T19:41:28.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.248+0000 7f33b3ad3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e41bd7ac90 0x55e41bd807f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:28.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.248+0000 7f33b3ad3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e41bd7ac90 0x55e41bd807f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:50708/0 (socket says 172.21.15.12:50708) 2023-10-05T19:41:28.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.248+0000 7f33b3ad3700 1 -- 172.21.15.12:0/2044834204 learned_addr learned my addr 172.21.15.12:0/2044834204 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:28.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.248+0000 7f33b3ad3700 1 -- 172.21.15.12:0/2044834204 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f339c004c90 con 0x55e41bd7ac90 2023-10-05T19:41:28.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.248+0000 7f33b3ad3700 1 --2- 172.21.15.12:0/2044834204 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e41bd7ac90 0x55e41bd807f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f33ac0058f0 tx=0x7f339c004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:28.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.248+0000 7f32ff7fe700 1 -- 172.21.15.12:0/2044834204 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f339c039070 con 0x55e41bd7ac90 2023-10-05T19:41:28.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.248+0000 7f32ff7fe700 1 -- 172.21.15.12:0/2044834204 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f339c01b040 con 0x55e41bd7ac90 2023-10-05T19:41:28.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.248+0000 7f32ff7fe700 1 -- 172.21.15.12:0/2044834204 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f339c02a030 con 0x55e41bd7ac90 2023-10-05T19:41:28.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.248+0000 7f33b679fe40 1 -- 172.21.15.12:0/2044834204 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e41bd80df0 con 0x55e41bd7ac90 2023-10-05T19:41:28.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.248+0000 7f33b679fe40 1 -- 172.21.15.12:0/2044834204 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e41bd81230 con 0x55e41bd7ac90 2023-10-05T19:41:28.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.252+0000 7f32ff7fe700 1 -- 172.21.15.12:0/2044834204 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f339c03c030 con 0x55e41bd7ac90 2023-10-05T19:41:28.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.252+0000 7f32ff7fe700 1 --2- 172.21.15.12:0/2044834204 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f32f4030c00 0x7f32f40330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:28.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.252+0000 7f32ff7fe700 1 -- 172.21.15.12:0/2044834204 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7f339c02a190 con 0x55e41bd7ac90 2023-10-05T19:41:28.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.252+0000 7f33b4ad5700 1 --2- 172.21.15.12:0/2044834204 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f32f4030c00 0x7f32f40330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:28.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.252+0000 7f33b679fe40 5 note: GC not initialized 2023-10-05T19:41:28.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.252+0000 7f33b4ad5700 1 --2- 172.21.15.12:0/2044834204 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f32f4030c00 0x7f32f40330b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x55e41bd6d3f0 tx=0x7f33ac0335d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:28.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.252+0000 7f32fcff9700 20 reqs_thread_entry: start 2023-10-05T19:41:28.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.256+0000 7f32f27fc700 20 reqs_thread_entry: start 2023-10-05T19:41:28.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.256+0000 7f33b679fe40 20 init_complete bucket index max shards: 11 2023-10-05T19:41:28.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.256+0000 7f33b679fe40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:41:28.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.256+0000 7f32f0ff9700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:41:58 2023 2023-10-05T19:41:28.261 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:41:28.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.256+0000 7f33b679fe40 20 get_system_obj_state: rctx=0x7fff19d93258 obj=default.rgw.meta:users.uid:bar state=0x55e41bc000b0 s->prefetch_data=0 2023-10-05T19:41:28.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.256+0000 7f33b679fe40 10 cache get: name=default.rgw.meta+users.uid+bar : miss 2023-10-05T19:41:28.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.256+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:102 8.e 8:718f6a54:users.uid::bar:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bde9100 con 0x7f33a4007030 2023-10-05T19:41:28.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.256+0000 7f32d7fff700 20 reqs_thread_entry: start 2023-10-05T19:41:28.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.256+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(102 bar [call,getxattrs,stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x7f33ac014040 con 0x7f33a4007030 2023-10-05T19:41:28.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.256+0000 7f33b679fe40 10 cache put: name=default.rgw.meta+users.uid+bar info.flags=0x0 2023-10-05T19:41:28.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.256+0000 7f33b679fe40 10 adding default.rgw.meta+users.uid+bar to cache LRU end 2023-10-05T19:41:28.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.256+0000 7f33b679fe40 20 get_system_obj_state: rctx=0x7fff19d92f48 obj=default.rgw.meta:users.keys:NX5QOQKC6BH2IDN8HC7A state=0x55e41bc000b0 s->prefetch_data=0 2023-10-05T19:41:28.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.256+0000 7f33b679fe40 10 cache get: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A : miss 2023-10-05T19:41:28.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.256+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 8.a 8:50c8eddb:users.keys::NX5QOQKC6BH2IDN8HC7A:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bde9100 con 0x55e41bd2a170 2023-10-05T19:41:28.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.256+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(103 NX5QOQKC6BH2IDN8HC7A [getxattrs,stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 206+0+0 (crc 0 0 0) 0x7f33ac014040 con 0x55e41bd2a170 2023-10-05T19:41:28.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.256+0000 7f33b679fe40 10 cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x0 2023-10-05T19:41:28.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.256+0000 7f33b679fe40 10 adding default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2023-10-05T19:41:28.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.256+0000 7f33b679fe40 20 get_system_obj_state: rctx=0x7fff19d927e8 obj=default.rgw.meta:users.keys:NX5QOQKC6BH2IDN8HC7A state=0x55e41bc000b0 s->prefetch_data=0 2023-10-05T19:41:28.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.256+0000 7f33b679fe40 10 cache get: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A : hit (negative entry) 2023-10-05T19:41:28.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.256+0000 7f33b679fe40 20 get_system_obj_state: rctx=0x7fff19d92e88 obj=default.rgw.meta:users.keys:NX5QOQKC6BH2IDN8HC7A state=0x55e41bc000b0 s->prefetch_data=0 2023-10-05T19:41:28.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.256+0000 7f33b679fe40 10 cache get: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A : hit (negative entry) 2023-10-05T19:41:28.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.256+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:104 8.e 8:718f6a54:users.uid::bar:head [delete,create,call version.set in=58b,writefull 0~346 in=346b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bdeb6a0 con 0x7f33a4007030 2023-10-05T19:41:28.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.260+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(104 bar [delete,create,call,writefull 0~346] v30'12 uv12 ondisk = 0) v8 ==== 273+0+0 (crc 0 0 0) 0x7f33ac014040 con 0x7f33a4007030 2023-10-05T19:41:28.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.260+0000 7f33b679fe40 10 cache put: name=default.rgw.meta+users.uid+bar info.flags=0x17 2023-10-05T19:41:28.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.260+0000 7f33b679fe40 10 moving default.rgw.meta+users.uid+bar to cache LRU end 2023-10-05T19:41:28.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.260+0000 7f33b679fe40 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.uid:bar, ofs0, ns] 2023-10-05T19:41:28.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.260+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:105 7.9 7:93e5b521:::notify.7:head [notify cookie 94438208480912 in=534b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bdeb010 con 0x7f33a4007030 2023-10-05T19:41:28.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.260+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== watch-notify(notify (1) cookie 94438207965312 notify 128849018906 ret 0) v3 ==== 564+0+0 (crc 0 0 0) 0x7f33ac0315b0 con 0x7f33a4007030 2023-10-05T19:41:28.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.264+0000 7f33b22d0700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018906 cookie 94438207965312 notifier 4705 bl.length()=522 2023-10-05T19:41:28.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.264+0000 7f33b22d0700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+bar info.flags=0x17 2023-10-05T19:41:28.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.264+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(105 notify.7 [notify cookie 94438208480912 out=8b] v0'0 uv240 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f33ac014040 con 0x7f33a4007030 2023-10-05T19:41:28.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.264+0000 7f33b22d0700 10 rgw watcher librados: moving default.rgw.meta+users.uid+bar to cache LRU end 2023-10-05T19:41:28.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.264+0000 7f33b22d0700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:106 7.9 7:93e5b521:::notify.7:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f3398003d90 con 0x7f33a4007030 2023-10-05T19:41:28.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.264+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(106 notify.7 [notify-ack] v0'0 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f33ac014040 con 0x7f33a4007030 2023-10-05T19:41:28.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.264+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.0 v2:172.21.15.8:6800/27563 30 ==== watch-notify(notify_complete (2) cookie 94438208480912 notify 128849018906 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f33ac031740 con 0x7f33a4007030 2023-10-05T19:41:28.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.264+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:107 8.a 8:50c8eddb:users.keys::NX5QOQKC6BH2IDN8HC7A:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bde9680 con 0x55e41bd2a170 2023-10-05T19:41:28.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.268+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(107 NX5QOQKC6BH2IDN8HC7A [delete,create,writefull 0~7] v30'1 uv1 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x7f33ac014040 con 0x55e41bd2a170 2023-10-05T19:41:28.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.268+0000 7f33b679fe40 10 cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x7 2023-10-05T19:41:28.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.268+0000 7f33b679fe40 10 moving default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2023-10-05T19:41:28.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.268+0000 7f33b679fe40 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.keys:NX5QOQKC6BH2IDN8HC7A, ofs0, ns] 2023-10-05T19:41:28.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.268+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:108 7.3 7:c609908c:::notify.5:head [notify cookie 94438208484464 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bdea9b0 con 0x7f33a4007030 2023-10-05T19:41:28.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.268+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.0 v2:172.21.15.8:6800/27563 31 ==== watch-notify(notify (1) cookie 94438207958704 notify 128849018907 ret 0) v3 ==== 219+0+0 (crc 0 0 0) 0x7f33ac031740 con 0x7f33a4007030 2023-10-05T19:41:28.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.268+0000 7f33b22d0700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018907 cookie 94438207958704 notifier 4705 bl.length()=177 2023-10-05T19:41:28.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.268+0000 7f33b22d0700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x7 2023-10-05T19:41:28.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.268+0000 7f33b22d0700 10 rgw watcher librados: moving default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2023-10-05T19:41:28.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.268+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.0 v2:172.21.15.8:6800/27563 32 ==== osd_op_reply(108 notify.5 [notify cookie 94438208484464 out=8b] v0'0 uv240 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f33ac014040 con 0x7f33a4007030 2023-10-05T19:41:28.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.268+0000 7f33b22d0700 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:109 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f3398003d90 con 0x7f33a4007030 2023-10-05T19:41:28.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.268+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.0 v2:172.21.15.8:6800/27563 33 ==== watch-notify(notify_complete (2) cookie 94438208484464 notify 128849018907 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f33ac031970 con 0x7f33a4007030 2023-10-05T19:41:28.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.268+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.0 v2:172.21.15.8:6800/27563 34 ==== osd_op_reply(109 notify.5 [notify-ack] v0'0 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f33ac014040 con 0x7f33a4007030 2023-10-05T19:41:28.275 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:41:28.275 INFO:teuthology.orchestra.run.smithi012.stdout: "user_id": "bar", 2023-10-05T19:41:28.276 INFO:teuthology.orchestra.run.smithi012.stdout: "display_name": "Bar", 2023-10-05T19:41:28.276 INFO:teuthology.orchestra.run.smithi012.stdout: "email": "", 2023-10-05T19:41:28.276 INFO:teuthology.orchestra.run.smithi012.stdout: "suspended": 0, 2023-10-05T19:41:28.276 INFO:teuthology.orchestra.run.smithi012.stdout: "max_buckets": 4, 2023-10-05T19:41:28.276 INFO:teuthology.orchestra.run.smithi012.stdout: "subusers": [], 2023-10-05T19:41:28.276 INFO:teuthology.orchestra.run.smithi012.stdout: "keys": [ 2023-10-05T19:41:28.276 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:28.276 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "bar", 2023-10-05T19:41:28.276 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key": "NX5QOQKC6BH2IDN8HC7A", 2023-10-05T19:41:28.276 INFO:teuthology.orchestra.run.smithi012.stdout: "secret_key": "LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn" 2023-10-05T19:41:28.276 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:28.277 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:41:28.277 INFO:teuthology.orchestra.run.smithi012.stdout: "swift_keys": [], 2023-10-05T19:41:28.277 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": [], 2023-10-05T19:41:28.277 INFO:teuthology.orchestra.run.smithi012.stdout: "op_mask": "read, write, delete", 2023-10-05T19:41:28.277 INFO:teuthology.orchestra.run.smithi012.stdout: "default_placement": "", 2023-10-05T19:41:28.277 INFO:teuthology.orchestra.run.smithi012.stdout: "default_storage_class": "", 2023-10-05T19:41:28.277 INFO:teuthology.orchestra.run.smithi012.stdout: "placement_tags": [], 2023-10-05T19:41:28.277 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket_quota": { 2023-10-05T19:41:28.277 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:41:28.277 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:41:28.277 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:41:28.277 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:41:28.277 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:41:28.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:28.278 INFO:teuthology.orchestra.run.smithi012.stdout: "user_quota": { 2023-10-05T19:41:28.278 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:41:28.278 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:41:28.278 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:41:28.278 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:41:28.278 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:41:28.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:28.278 INFO:teuthology.orchestra.run.smithi012.stdout: "temp_url_keys": [], 2023-10-05T19:41:28.278 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "rgw", 2023-10-05T19:41:28.279 INFO:teuthology.orchestra.run.smithi012.stdout: "mfa_ids": [] 2023-10-05T19:41:28.279 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:41:28.279 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:41:28.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.268+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:110 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94438207936400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd3c610 con 0x55e41bd5cc10 2023-10-05T19:41:28.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.272+0000 7f33b42d4700 1 -- 172.21.15.12:0/2032180726 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(110 notify.0 [watch unwatch cookie 94438207936400] v30'242 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f33a4012ab0 con 0x55e41bd5cc10 2023-10-05T19:41:28.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.272+0000 7f33b679fe40 20 remove_watcher() i=0 2023-10-05T19:41:28.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.272+0000 7f33b679fe40 2 removed watcher, disabling cache 2023-10-05T19:41:28.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.272+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:111 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94438207948928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd3c610 con 0x55e41bd2a170 2023-10-05T19:41:28.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.276+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(111 notify.1 [watch unwatch cookie 94438207948928] v30'242 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f33ac014040 con 0x55e41bd2a170 2023-10-05T19:41:28.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.276+0000 7f33b679fe40 20 remove_watcher() i=1 2023-10-05T19:41:28.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.276+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:112 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94438207950320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd3c610 con 0x55e41bd613b0 2023-10-05T19:41:28.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.280+0000 7f33b3ad3700 1 -- 172.21.15.12:0/2032180726 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(112 notify.2 [watch unwatch cookie 94438207950320] v30'242 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f339c02d040 con 0x55e41bd613b0 2023-10-05T19:41:28.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.280+0000 7f33b679fe40 20 remove_watcher() i=2 2023-10-05T19:41:28.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.280+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:113 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94438207957600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd3c610 con 0x55e41bd5cc10 2023-10-05T19:41:28.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.284+0000 7f33b42d4700 1 -- 172.21.15.12:0/2032180726 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(113 notify.3 [watch unwatch cookie 94438207957600] v30'242 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f33a4012ab0 con 0x55e41bd5cc10 2023-10-05T19:41:28.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.284+0000 7f33b679fe40 20 remove_watcher() i=3 2023-10-05T19:41:28.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.284+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:114 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94438207959968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd3c610 con 0x55e41bd2a170 2023-10-05T19:41:28.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.284+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== osd_op_reply(114 notify.4 [watch unwatch cookie 94438207959968] v30'242 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f33ac047040 con 0x55e41bd2a170 2023-10-05T19:41:28.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.284+0000 7f33b679fe40 20 remove_watcher() i=4 2023-10-05T19:41:28.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.284+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:115 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94438207958704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd3c610 con 0x7f33a4007030 2023-10-05T19:41:28.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.288+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.0 v2:172.21.15.8:6800/27563 35 ==== osd_op_reply(115 notify.5 [watch unwatch cookie 94438207958704] v30'242 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f33ac047040 con 0x7f33a4007030 2023-10-05T19:41:28.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.288+0000 7f33b679fe40 20 remove_watcher() i=5 2023-10-05T19:41:28.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.288+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:116 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94438207963648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd6af50 con 0x7f33a4007030 2023-10-05T19:41:28.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.292+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.0 v2:172.21.15.8:6800/27563 36 ==== osd_op_reply(116 notify.6 [watch unwatch cookie 94438207963648] v30'242 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f33ac047040 con 0x7f33a4007030 2023-10-05T19:41:28.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.292+0000 7f33b679fe40 20 remove_watcher() i=6 2023-10-05T19:41:28.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.292+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:117 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94438207965312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd6af50 con 0x7f33a4007030 2023-10-05T19:41:28.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.296+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.0 v2:172.21.15.8:6800/27563 37 ==== osd_op_reply(117 notify.7 [watch unwatch cookie 94438207965312] v30'242 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f33ac047040 con 0x7f33a4007030 2023-10-05T19:41:28.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.296+0000 7f33b679fe40 20 remove_watcher() i=7 2023-10-05T19:41:28.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.296+0000 7f3385fe3700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:41:28.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.296+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:118 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94438207966880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55e41bd3c610 con 0x55e41bd2a170 2023-10-05T19:41:28.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.300+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 <== osd.1 v2:172.21.15.12:6808/29933 39 ==== osd_op_reply(118 data_loggenerations_metadata [watch unwatch cookie 94438207966880] v30'232 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f33ac047040 con 0x55e41bd2a170 2023-10-05T19:41:28.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.300+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f33a4007030 msgr2=0x7f33a400c0b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:28.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.300+0000 7f33b679fe40 1 --2- 172.21.15.12:0/2032180726 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f33a4007030 0x7f33a400c0b0 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:28.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.300+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55e41bd2a170 msgr2=0x55e41bd5a770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:28.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.300+0000 7f33b679fe40 1 --2- 172.21.15.12:0/2032180726 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55e41bd2a170 0x55e41bd5a770 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:28.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.300+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55e41bd5cc10 msgr2=0x55e41bd5eff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:28.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.300+0000 7f33b679fe40 1 --2- 172.21.15.12:0/2032180726 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55e41bd5cc10 0x55e41bd5eff0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:28.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.300+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55e41bd613b0 msgr2=0x55e41bd637c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:28.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.300+0000 7f33b679fe40 1 --2- 172.21.15.12:0/2032180726 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55e41bd613b0 0x55e41bd637c0 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:28.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.300+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3394030c50 msgr2=0x7f3394033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:28.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.300+0000 7f33b679fe40 1 --2- 172.21.15.12:0/2032180726 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3394030c50 0x7f3394033100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f339c001560 tx=0x7f339c0065f0 comp rx=0 tx=0).stop 2023-10-05T19:41:28.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.300+0000 7f33b4ad5700 1 -- 172.21.15.12:0/2032180726 reap_dead start 2023-10-05T19:41:28.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.300+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e41bc9ab80 msgr2=0x55e41bd4ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:28.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.300+0000 7f33b679fe40 1 --2- 172.21.15.12:0/2032180726 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e41bc9ab80 0x55e41bd4ebb0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f33a400c010 tx=0x7f33a401c000 comp rx=0 tx=0).stop 2023-10-05T19:41:28.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.300+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 shutdown_connections 2023-10-05T19:41:28.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.300+0000 7f33b679fe40 1 --2- 172.21.15.12:0/2032180726 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e41bc9ab80 0x55e41bd4ebb0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:28.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.300+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 >> 172.21.15.12:0/2032180726 conn(0x55e41bd21780 msgr2=0x55e41bd403b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:28.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.300+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 shutdown_connections 2023-10-05T19:41:28.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.300+0000 7f33b679fe40 1 -- 172.21.15.12:0/2032180726 wait complete. 2023-10-05T19:41:28.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.304+0000 7f33b679fe40 1 -- 172.21.15.12:0/2044834204 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f32f4030c00 msgr2=0x7f32f40330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:28.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.304+0000 7f33b679fe40 1 --2- 172.21.15.12:0/2044834204 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f32f4030c00 0x7f32f40330b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x55e41bd6d3f0 tx=0x7f33ac0335d0 comp rx=0 tx=0).stop 2023-10-05T19:41:28.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.304+0000 7f33b679fe40 1 -- 172.21.15.12:0/2044834204 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e41bd7ac90 msgr2=0x55e41bd807f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:28.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.304+0000 7f33b679fe40 1 --2- 172.21.15.12:0/2044834204 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e41bd7ac90 0x55e41bd807f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f33ac0058f0 tx=0x7f339c004f90 comp rx=0 tx=0).stop 2023-10-05T19:41:28.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.304+0000 7f33b679fe40 1 -- 172.21.15.12:0/2044834204 shutdown_connections 2023-10-05T19:41:28.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.304+0000 7f33b679fe40 1 --2- 172.21.15.12:0/2044834204 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f32f4030c00 0x7f32f40330b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:28.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.304+0000 7f33b679fe40 1 --2- 172.21.15.12:0/2044834204 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e41bd7ac90 0x55e41bd807f0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:28.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.304+0000 7f33b679fe40 1 -- 172.21.15.12:0/2044834204 >> 172.21.15.12:0/2044834204 conn(0x55e41bd76050 msgr2=0x55e41bd740f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:28.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.304+0000 7f33b679fe40 1 -- 172.21.15.12:0/2044834204 shutdown_connections 2023-10-05T19:41:28.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.304+0000 7f33b679fe40 1 -- 172.21.15.12:0/2044834204 wait complete. 2023-10-05T19:41:28.315 DEBUG:tasks.util.rgw: json result: {'user_id': 'bar', 'display_name': 'Bar', 'email': '', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'bar', 'access_key': 'NX5QOQKC6BH2IDN8HC7A', 'secret_key': 'LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:41:28.338 INFO:tasks.radosgw_admin:RL: create_bucket myfoo6 bar bytes_out=0 bytes_in=0 failed=False 2023-10-05T19:41:28.346 INFO:tasks.radosgw_admin:RL: put_obj myfoo6 bar bytes_out=8 bytes_in=0 failed=False 2023-10-05T19:41:28.346 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rename', '--uid', 'bar', '--new-uid', 'bud'] 2023-10-05T19:41:28.346 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rename', '--uid', 'bar', '--new-uid', 'bud'] 2023-10-05T19:41:28.347 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rename --uid bar --new-uid bud 2023-10-05T19:41:28.386 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:41:28.386 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:41:28.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.380+0000 7fb8a9028e40 1 Processor -- start 2023-10-05T19:41:28.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.380+0000 7fb8a9028e40 1 -- start start 2023-10-05T19:41:28.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.380+0000 7fb8a9028e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55badc8dae20 0x55badc8db1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:28.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.380+0000 7fb8a9028e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55badc827bd0 con 0x55badc8db730 2023-10-05T19:41:28.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.380+0000 7fb8a9028e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x55badc7e4ee0 con 0x55badc8dae20 2023-10-05T19:41:28.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.380+0000 7fb8a635c700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x55badc8db730 0x55badc8dc1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:57708/0 (socket says 172.21.15.12:57708) 2023-10-05T19:41:28.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.380+0000 7fb8a6b5d700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55badc8dae20 0x55badc8db1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:28.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.380+0000 7fb8a635c700 1 -- 172.21.15.12:0/880896716 learned_addr learned my addr 172.21.15.12:0/880896716 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:41:28.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.384+0000 7fb8a5b5b700 1 -- 172.21.15.12:0/880896716 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 365577569 0 0) 0x55badc827bd0 con 0x55badc8db730 2023-10-05T19:41:28.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.384+0000 7fb8a5b5b700 1 -- 172.21.15.12:0/880896716 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb888001e10 con 0x55badc8db730 2023-10-05T19:41:28.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.384+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/880896716 >> v1:172.21.15.8:6789/0 conn(0x55badc8db730 legacy=0x55badc8dc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:28.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.384+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/880896716 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55badc8dc8e0 con 0x55badc8dae20 2023-10-05T19:41:28.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.384+0000 7fb8a6b5d700 1 --2- 172.21.15.12:0/880896716 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55badc8dae20 0x55badc8db1f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb8a000a5d0 tx=0x7fb8a0005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a6c9a16b63eac68 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:28.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.384+0000 7fb8a5b5b700 1 -- 172.21.15.12:0/880896716 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb8a0007610 con 0x55badc8dae20 2023-10-05T19:41:28.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.384+0000 7fb8a5b5b700 1 -- 172.21.15.12:0/880896716 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a0007770 con 0x55badc8dae20 2023-10-05T19:41:28.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.384+0000 7fb8a5b5b700 1 -- 172.21.15.12:0/880896716 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb8a0007a40 con 0x55badc8dae20 2023-10-05T19:41:28.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.384+0000 7fb8a9028e40 1 -- 172.21.15.12:0/880896716 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55badc8dae20 msgr2=0x55badc8db1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:28.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.384+0000 7fb8a9028e40 1 --2- 172.21.15.12:0/880896716 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55badc8dae20 0x55badc8db1f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb8a000a5d0 tx=0x7fb8a0005a50 comp rx=0 tx=0).stop 2023-10-05T19:41:28.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.384+0000 7fb8a9028e40 1 -- 172.21.15.12:0/880896716 shutdown_connections 2023-10-05T19:41:28.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.384+0000 7fb8a9028e40 1 --2- 172.21.15.12:0/880896716 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55badc8dae20 0x55badc8db1f0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:28.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.384+0000 7fb8a9028e40 1 -- 172.21.15.12:0/880896716 >> 172.21.15.12:0/880896716 conn(0x55badc8d6840 msgr2=0x55badc8d8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:28.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.384+0000 7fb8a9028e40 1 -- 172.21.15.12:0/880896716 shutdown_connections 2023-10-05T19:41:28.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.384+0000 7fb8a9028e40 1 -- 172.21.15.12:0/880896716 wait complete. 2023-10-05T19:41:28.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.384+0000 7fb8a9028e40 1 Processor -- start 2023-10-05T19:41:28.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.384+0000 7fb8a9028e40 1 -- start start 2023-10-05T19:41:28.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.384+0000 7fb8a9028e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55badc8dae20 0x55badc851180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:28.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.384+0000 7fb8a9028e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55badc7a8e30 con 0x55badc8dae20 2023-10-05T19:41:28.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.384+0000 7fb8a6b5d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55badc8dae20 0x55badc851180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:28.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.384+0000 7fb8a6b5d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55badc8dae20 0x55badc851180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:50736/0 (socket says 172.21.15.12:50736) 2023-10-05T19:41:28.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.384+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/195114985 learned_addr learned my addr 172.21.15.12:0/195114985 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:28.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.388+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/195114985 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a0005cf0 con 0x55badc8dae20 2023-10-05T19:41:28.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.388+0000 7fb8a6b5d700 1 --2- 172.21.15.12:0/195114985 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55badc8dae20 0x55badc851180 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0000c00 tx=0x7fb8a0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5466914fe2034e4c server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:28.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.388+0000 7fb8a4b59700 1 -- 172.21.15.12:0/195114985 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb8a00078e0 con 0x55badc8dae20 2023-10-05T19:41:28.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.388+0000 7fb8a4b59700 1 -- 172.21.15.12:0/195114985 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a0007a40 con 0x55badc8dae20 2023-10-05T19:41:28.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.388+0000 7fb8a4b59700 1 -- 172.21.15.12:0/195114985 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb8a0007bd0 con 0x55badc8dae20 2023-10-05T19:41:28.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.388+0000 7fb8a9028e40 1 -- 172.21.15.12:0/195114985 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55badc8dae20 msgr2=0x55badc851180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:28.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.388+0000 7fb8a9028e40 1 --2- 172.21.15.12:0/195114985 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55badc8dae20 0x55badc851180 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0000c00 tx=0x7fb8a0014000 comp rx=0 tx=0).stop 2023-10-05T19:41:28.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.388+0000 7fb8a9028e40 1 -- 172.21.15.12:0/195114985 shutdown_connections 2023-10-05T19:41:28.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.388+0000 7fb8a9028e40 1 --2- 172.21.15.12:0/195114985 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55badc8dae20 0x55badc851180 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:28.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.388+0000 7fb8a9028e40 1 -- 172.21.15.12:0/195114985 >> 172.21.15.12:0/195114985 conn(0x55badc8d6840 msgr2=0x55badc8f58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:28.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.388+0000 7fb8a9028e40 1 -- 172.21.15.12:0/195114985 shutdown_connections 2023-10-05T19:41:28.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.388+0000 7fb8a9028e40 1 -- 172.21.15.12:0/195114985 wait complete. 2023-10-05T19:41:28.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.388+0000 7fb8a9028e40 1 Processor -- start 2023-10-05T19:41:28.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.388+0000 7fb8a9028e40 1 -- start start 2023-10-05T19:41:28.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.388+0000 7fb8a9028e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55badc8dae20 0x55badc902bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:28.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.388+0000 7fb8a9028e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55badc77bed0 con 0x55badc8dae20 2023-10-05T19:41:28.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.388+0000 7fb8a6b5d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55badc8dae20 0x55badc902bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:28.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.388+0000 7fb8a6b5d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55badc8dae20 0x55badc902bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:50742/0 (socket says 172.21.15.12:50742) 2023-10-05T19:41:28.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.388+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 learned_addr learned my addr 172.21.15.12:0/1873315814 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:28.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.388+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55badc8dc8e0 con 0x55badc8dae20 2023-10-05T19:41:28.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.392+0000 7fb8a6b5d700 1 --2- 172.21.15.12:0/1873315814 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55badc8dae20 0x55badc902bc0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fb8a001f040 tx=0x7fb8a001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:28.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.392+0000 7fb89effd700 1 -- 172.21.15.12:0/1873315814 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb8a0007e50 con 0x55badc8dae20 2023-10-05T19:41:28.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.392+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55badc903100 con 0x55badc8dae20 2023-10-05T19:41:28.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.392+0000 7fb89effd700 1 -- 172.21.15.12:0/1873315814 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a0029070 con 0x55badc8dae20 2023-10-05T19:41:28.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.392+0000 7fb89effd700 1 -- 172.21.15.12:0/1873315814 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb8a0030480 con 0x55badc8dae20 2023-10-05T19:41:28.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.392+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55badc903540 con 0x55badc8dae20 2023-10-05T19:41:28.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.392+0000 7fb89effd700 1 -- 172.21.15.12:0/1873315814 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fb8a0012400 con 0x55badc8dae20 2023-10-05T19:41:28.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.392+0000 7fb8a9028e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:28.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.392+0000 7fb89effd700 1 --2- 172.21.15.12:0/1873315814 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fb88c030bf0 0x7fb88c0330a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:28.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.392+0000 7fb89effd700 1 -- 172.21.15.12:0/1873315814 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7fb8a002b070 con 0x55badc8dae20 2023-10-05T19:41:28.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.392+0000 7fb8a635c700 1 --2- 172.21.15.12:0/1873315814 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fb88c030bf0 0x7fb88c0330a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:28.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.392+0000 7fb8a9028e40 1 --2- 172.21.15.12:0/1873315814 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55badc8db730 0x55badc90e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:28.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.392+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc90ed30 con 0x55badc8db730 2023-10-05T19:41:28.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.392+0000 7fb8a735e700 1 --2- 172.21.15.12:0/1873315814 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55badc8db730 0x55badc90e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:28.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.392+0000 7fb8a635c700 1 --2- 172.21.15.12:0/1873315814 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fb88c030bf0 0x7fb88c0330a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb8900044f0 tx=0x7fb890006690 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:28.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.396+0000 7fb8a735e700 1 --2- 172.21.15.12:0/1873315814 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55badc8db730 0x55badc90e680 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:28.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.396+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb89800a500 con 0x55badc8db730 2023-10-05T19:41:28.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.396+0000 7fb8a9028e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:28.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.396+0000 7fb8a9028e40 20 realm 2023-10-05T19:41:28.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.396+0000 7fb8a9028e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:28.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.396+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc90bce0 con 0x55badc8db730 2023-10-05T19:41:28.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.396+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb89800a500 con 0x55badc8db730 2023-10-05T19:41:28.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.396+0000 7fb8a9028e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:28.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.396+0000 7fb8a9028e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:41:28.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.396+0000 7fb8a9028e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:28.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.396+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc90f810 con 0x55badc8db730 2023-10-05T19:41:28.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.396+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb89800a500 con 0x55badc8db730 2023-10-05T19:41:28.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.396+0000 7fb8a9028e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:28.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.396+0000 7fb8a9028e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:28.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.396+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc90f420 con 0x55badc8db730 2023-10-05T19:41:28.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.396+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fb89800a500 con 0x55badc8db730 2023-10-05T19:41:28.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.396+0000 7fb8a9028e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:28.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.396+0000 7fb8a9028e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:28.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.396+0000 7fb8a9028e40 1 --2- 172.21.15.12:0/1873315814 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55badc910bb0 0x55badc912f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:28.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.396+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9134d0 con 0x55badc910bb0 2023-10-05T19:41:28.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.396+0000 7fb8a6b5d700 1 --2- 172.21.15.12:0/1873315814 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55badc910bb0 0x55badc912f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:28.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.396+0000 7fb8a6b5d700 1 --2- 172.21.15.12:0/1873315814 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55badc910bb0 0x55badc912f90 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:28.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.400+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fb8a001c7f0 con 0x55badc910bb0 2023-10-05T19:41:28.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.400+0000 7fb8a9028e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:41:28.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.400+0000 7fb8a9028e40 20 searching for the correct realm 2023-10-05T19:41:28.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.400+0000 7fb8a9028e40 1 --2- 172.21.15.12:0/1873315814 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55badc915350 0x55badc917760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:28.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.400+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc917e10 con 0x55badc915350 2023-10-05T19:41:28.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.400+0000 7fb8a635c700 1 --2- 172.21.15.12:0/1873315814 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55badc915350 0x55badc917760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:28.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.400+0000 7fb8a635c700 1 --2- 172.21.15.12:0/1873315814 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55badc915350 0x55badc917760 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:28.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.400+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fb890004520 con 0x55badc915350 2023-10-05T19:41:28.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.400+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc917e10 con 0x55badc910bb0 2023-10-05T19:41:28.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.400+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb8a001c7f0 con 0x55badc910bb0 2023-10-05T19:41:28.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.400+0000 7fb8a6b5d700 1 --2- 172.21.15.12:0/1873315814 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fb8a0007030 0x7fb8a000c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:28.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.404+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc917e10 con 0x7fb8a0007030 2023-10-05T19:41:28.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.404+0000 7fb8a735e700 1 --2- 172.21.15.12:0/1873315814 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fb8a0007030 0x7fb8a000c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:28.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.404+0000 7fb8a735e700 1 --2- 172.21.15.12:0/1873315814 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fb8a0007030 0x7fb8a000c9c0 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:28.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.404+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb898014040 con 0x7fb8a0007030 2023-10-05T19:41:28.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.404+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc917e10 con 0x55badc8db730 2023-10-05T19:41:28.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.404+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb898014040 con 0x55badc8db730 2023-10-05T19:41:28.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.404+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc917e10 con 0x55badc8db730 2023-10-05T19:41:28.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.404+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb898014040 con 0x55badc8db730 2023-10-05T19:41:28.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.404+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc917e10 con 0x55badc8db730 2023-10-05T19:41:28.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.408+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(11 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb898014040 con 0x55badc8db730 2023-10-05T19:41:28.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.408+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc917e10 con 0x7fb8a0007030 2023-10-05T19:41:28.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.408+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb898014040 con 0x7fb8a0007030 2023-10-05T19:41:28.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.408+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc917e10 con 0x55badc910bb0 2023-10-05T19:41:28.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.408+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb8a001c7f0 con 0x55badc910bb0 2023-10-05T19:41:28.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.408+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc917e10 con 0x55badc910bb0 2023-10-05T19:41:28.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.408+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb8a001c7f0 con 0x55badc910bb0 2023-10-05T19:41:28.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.408+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc917e10 con 0x55badc8db730 2023-10-05T19:41:28.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.408+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb898014040 con 0x55badc8db730 2023-10-05T19:41:28.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.408+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc917e10 con 0x7fb8a0007030 2023-10-05T19:41:28.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.412+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb898014040 con 0x7fb8a0007030 2023-10-05T19:41:28.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.412+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc917e10 con 0x55badc8db730 2023-10-05T19:41:28.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.412+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb898014040 con 0x55badc8db730 2023-10-05T19:41:28.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.412+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc917e10 con 0x55badc910bb0 2023-10-05T19:41:28.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.412+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(18 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb8a001c7f0 con 0x55badc910bb0 2023-10-05T19:41:28.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.412+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc917e10 con 0x55badc915350 2023-10-05T19:41:28.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.412+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb890004520 con 0x55badc915350 2023-10-05T19:41:28.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.412+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc917e10 con 0x55badc915350 2023-10-05T19:41:28.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.412+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb890004520 con 0x55badc915350 2023-10-05T19:41:28.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.412+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc917e10 con 0x7fb8a0007030 2023-10-05T19:41:28.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.416+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb898014040 con 0x7fb8a0007030 2023-10-05T19:41:28.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.416+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc917e10 con 0x55badc915350 2023-10-05T19:41:28.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.416+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb89001d040 con 0x55badc915350 2023-10-05T19:41:28.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.416+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc917e10 con 0x55badc915350 2023-10-05T19:41:28.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.416+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb89001d040 con 0x55badc915350 2023-10-05T19:41:28.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.416+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc917e10 con 0x55badc8db730 2023-10-05T19:41:28.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.416+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb898014040 con 0x55badc8db730 2023-10-05T19:41:28.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.416+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc917e10 con 0x55badc8db730 2023-10-05T19:41:28.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.416+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb898014040 con 0x55badc8db730 2023-10-05T19:41:28.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.416+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc917e10 con 0x7fb8a0007030 2023-10-05T19:41:28.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.420+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb898014040 con 0x7fb8a0007030 2023-10-05T19:41:28.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.420+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc917e10 con 0x55badc915350 2023-10-05T19:41:28.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.420+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb89001d040 con 0x55badc915350 2023-10-05T19:41:28.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.420+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc917e10 con 0x55badc910bb0 2023-10-05T19:41:28.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.420+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb8a001c7f0 con 0x55badc910bb0 2023-10-05T19:41:28.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.420+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc917e10 con 0x55badc8db730 2023-10-05T19:41:28.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.420+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb898014040 con 0x55badc8db730 2023-10-05T19:41:28.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.420+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc917e10 con 0x7fb8a0007030 2023-10-05T19:41:28.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.424+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb898014040 con 0x7fb8a0007030 2023-10-05T19:41:28.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.424+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc917e10 con 0x55badc915350 2023-10-05T19:41:28.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.424+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb89001d040 con 0x55badc915350 2023-10-05T19:41:28.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.424+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc917e10 con 0x7fb8a0007030 2023-10-05T19:41:28.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.424+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb898014040 con 0x7fb8a0007030 2023-10-05T19:41:28.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.424+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc917e10 con 0x55badc910bb0 2023-10-05T19:41:28.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.424+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb8a001c7f0 con 0x55badc910bb0 2023-10-05T19:41:28.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.424+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc917e10 con 0x55badc915350 2023-10-05T19:41:28.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.428+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb89001d040 con 0x55badc915350 2023-10-05T19:41:28.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.428+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc917e10 con 0x7fb8a0007030 2023-10-05T19:41:28.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.428+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb898014040 con 0x7fb8a0007030 2023-10-05T19:41:28.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.428+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc917e10 con 0x55badc8db730 2023-10-05T19:41:28.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.428+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fb898014040 con 0x55badc8db730 2023-10-05T19:41:28.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.428+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc917e10 con 0x55badc910bb0 2023-10-05T19:41:28.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.428+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(37 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fb8a001c7f0 con 0x55badc910bb0 2023-10-05T19:41:28.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.428+0000 7fb8a9028e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:41:28.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.428+0000 7fb8a9028e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:41:28.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.428+0000 7fb8a9028e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:41:28.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.428+0000 7fb8a9028e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:41:28.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.428+0000 7fb8a9028e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:28.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.428+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9140d0 con 0x55badc8db730 2023-10-05T19:41:28.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.428+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb898014040 con 0x55badc8db730 2023-10-05T19:41:28.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.428+0000 7fb8a9028e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:28.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.428+0000 7fb8a9028e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:28.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.428+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc913d00 con 0x55badc8db730 2023-10-05T19:41:28.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.432+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fb898014040 con 0x55badc8db730 2023-10-05T19:41:28.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.432+0000 7fb8a9028e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:28.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.432+0000 7fb8a9028e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:28.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.432+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc918280 con 0x55badc915350 2023-10-05T19:41:28.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.432+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fb89001d040 con 0x55badc915350 2023-10-05T19:41:28.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.432+0000 7fb8a9028e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:41:28.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.432+0000 7fb8a9028e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:28.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.432+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc918630 con 0x55badc915350 2023-10-05T19:41:28.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.432+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7fb89001d040 con 0x55badc915350 2023-10-05T19:41:28.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.432+0000 7fb8a9028e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:28.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.432+0000 7fb8a9028e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:28.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.432+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9189e0 con 0x55badc8db730 2023-10-05T19:41:28.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.432+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb898014040 con 0x55badc8db730 2023-10-05T19:41:28.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.432+0000 7fb8a9028e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:28.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.432+0000 7fb8a9028e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:28.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.432+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc914560 con 0x55badc8db730 2023-10-05T19:41:28.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.432+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fb898014040 con 0x55badc8db730 2023-10-05T19:41:28.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.432+0000 7fb8a9028e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:28.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.432+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc914530 con 0x55badc915350 2023-10-05T19:41:28.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.436+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(44 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fb89001d040 con 0x55badc915350 2023-10-05T19:41:28.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.436+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc914530 con 0x55badc910bb0 2023-10-05T19:41:28.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.436+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(45 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb8a001c7f0 con 0x55badc910bb0 2023-10-05T19:41:28.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.436+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc914530 con 0x7fb8a0007030 2023-10-05T19:41:28.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.440+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(46 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb898014040 con 0x7fb8a0007030 2023-10-05T19:41:28.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.440+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9189e0 con 0x55badc8db730 2023-10-05T19:41:28.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.440+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(47 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb898014040 con 0x55badc8db730 2023-10-05T19:41:28.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.440+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9189e0 con 0x55badc8db730 2023-10-05T19:41:28.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.440+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(48 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb898014040 con 0x55badc8db730 2023-10-05T19:41:28.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.440+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9189e0 con 0x55badc8db730 2023-10-05T19:41:28.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.440+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(49 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb898014040 con 0x55badc8db730 2023-10-05T19:41:28.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.440+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9189e0 con 0x7fb8a0007030 2023-10-05T19:41:28.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.440+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(50 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb898014040 con 0x7fb8a0007030 2023-10-05T19:41:28.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.444+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9189e0 con 0x55badc910bb0 2023-10-05T19:41:28.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.444+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(51 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb8a001c7f0 con 0x55badc910bb0 2023-10-05T19:41:28.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.444+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9189e0 con 0x55badc910bb0 2023-10-05T19:41:28.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.444+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(52 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb8a001c7f0 con 0x55badc910bb0 2023-10-05T19:41:28.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.444+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9189e0 con 0x55badc8db730 2023-10-05T19:41:28.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.444+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(53 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb898014040 con 0x55badc8db730 2023-10-05T19:41:28.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.444+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9189e0 con 0x7fb8a0007030 2023-10-05T19:41:28.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.444+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(54 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb898014040 con 0x7fb8a0007030 2023-10-05T19:41:28.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.444+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9189e0 con 0x55badc8db730 2023-10-05T19:41:28.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.448+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(55 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb898014040 con 0x55badc8db730 2023-10-05T19:41:28.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.448+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9189e0 con 0x55badc910bb0 2023-10-05T19:41:28.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.448+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(56 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb8a001c7f0 con 0x55badc910bb0 2023-10-05T19:41:28.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.448+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9189e0 con 0x55badc915350 2023-10-05T19:41:28.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.448+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(57 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb89001d040 con 0x55badc915350 2023-10-05T19:41:28.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.448+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9189e0 con 0x55badc915350 2023-10-05T19:41:28.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.448+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(58 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb89002d040 con 0x55badc915350 2023-10-05T19:41:28.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.448+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9189e0 con 0x7fb8a0007030 2023-10-05T19:41:28.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.452+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(59 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb898014040 con 0x7fb8a0007030 2023-10-05T19:41:28.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.452+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9189e0 con 0x55badc915350 2023-10-05T19:41:28.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.452+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(60 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb89002d040 con 0x55badc915350 2023-10-05T19:41:28.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.452+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9189e0 con 0x55badc915350 2023-10-05T19:41:28.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.452+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(61 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb89002d040 con 0x55badc915350 2023-10-05T19:41:28.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.452+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9189e0 con 0x55badc8db730 2023-10-05T19:41:28.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.452+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(62 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb898014040 con 0x55badc8db730 2023-10-05T19:41:28.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.452+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9189e0 con 0x55badc8db730 2023-10-05T19:41:28.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.452+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(63 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb898014040 con 0x55badc8db730 2023-10-05T19:41:28.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.452+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9189e0 con 0x7fb8a0007030 2023-10-05T19:41:28.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.456+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(64 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb898014040 con 0x7fb8a0007030 2023-10-05T19:41:28.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.456+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9189e0 con 0x55badc915350 2023-10-05T19:41:28.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.456+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(65 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb89002d040 con 0x55badc915350 2023-10-05T19:41:28.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.456+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9189e0 con 0x55badc910bb0 2023-10-05T19:41:28.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.456+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(66 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb8a001c7f0 con 0x55badc910bb0 2023-10-05T19:41:28.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.456+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9189e0 con 0x55badc8db730 2023-10-05T19:41:28.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.456+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(67 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb898014040 con 0x55badc8db730 2023-10-05T19:41:28.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.456+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9189e0 con 0x7fb8a0007030 2023-10-05T19:41:28.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.460+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(68 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb898014040 con 0x7fb8a0007030 2023-10-05T19:41:28.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.460+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9189e0 con 0x55badc915350 2023-10-05T19:41:28.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.460+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(69 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb89002d040 con 0x55badc915350 2023-10-05T19:41:28.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.460+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9189e0 con 0x7fb8a0007030 2023-10-05T19:41:28.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.460+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(70 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb898014040 con 0x7fb8a0007030 2023-10-05T19:41:28.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.460+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9189e0 con 0x55badc910bb0 2023-10-05T19:41:28.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.460+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(71 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb8a001c7f0 con 0x55badc910bb0 2023-10-05T19:41:28.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.460+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9189e0 con 0x55badc915350 2023-10-05T19:41:28.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.464+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(72 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb89002d040 con 0x55badc915350 2023-10-05T19:41:28.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.464+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9189e0 con 0x7fb8a0007030 2023-10-05T19:41:28.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.464+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(73 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb898014040 con 0x7fb8a0007030 2023-10-05T19:41:28.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.464+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9189e0 con 0x55badc8db730 2023-10-05T19:41:28.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.464+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(74 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fb898014040 con 0x55badc8db730 2023-10-05T19:41:28.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.464+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9189e0 con 0x55badc910bb0 2023-10-05T19:41:28.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.464+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(75 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fb8a001c7f0 con 0x55badc910bb0 2023-10-05T19:41:28.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.464+0000 7fb8a9028e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:41:28.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.464+0000 7fb8a9028e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:41:28.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.464+0000 7fb8a9028e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:41:28.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.464+0000 7fb8a9028e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:41:28.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.464+0000 7fb8a9028e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:28.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.464+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc914530 con 0x55badc8db730 2023-10-05T19:41:28.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.464+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fb898014040 con 0x55badc8db730 2023-10-05T19:41:28.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.464+0000 7fb8a9028e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:28.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.464+0000 7fb8a9028e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:28.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.464+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9192d0 con 0x55badc910bb0 2023-10-05T19:41:28.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.468+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fb8a001c7f0 con 0x55badc910bb0 2023-10-05T19:41:28.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.468+0000 7fb8a9028e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:41:28.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.468+0000 7fb8a9028e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:28.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.468+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc919bc0 con 0x55badc8db730 2023-10-05T19:41:28.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.468+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fb898014040 con 0x55badc8db730 2023-10-05T19:41:28.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.468+0000 7fb8a9028e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:28.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.468+0000 7fb8a9028e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:28.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.468+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc919bc0 con 0x55badc915350 2023-10-05T19:41:28.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.468+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fb89002d040 con 0x55badc915350 2023-10-05T19:41:28.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.468+0000 7fb8a9028e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:41:28.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.468+0000 7fb8a9028e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:28.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.468+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc91a330 con 0x7fb8a0007030 2023-10-05T19:41:28.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.468+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7fb898014040 con 0x7fb8a0007030 2023-10-05T19:41:28.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.468+0000 7fb8a9028e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:28.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.468+0000 7fb8a9028e40 20 zone default found 2023-10-05T19:41:28.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.468+0000 7fb8a9028e40 4 Realm: () 2023-10-05T19:41:28.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.468+0000 7fb8a9028e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:41:28.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.468+0000 7fb8a9028e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:41:28.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.468+0000 7fb8a9028e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:41:28.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.468+0000 7fb8a9028e40 20 zonegroup default 2023-10-05T19:41:28.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.468+0000 7fb8a9028e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:28.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.468+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc91a330 con 0x7fb8a0007030 2023-10-05T19:41:28.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.472+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fb898014040 con 0x7fb8a0007030 2023-10-05T19:41:28.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.472+0000 7fb8a9028e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:28.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.472+0000 7fb8a9028e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:28.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.472+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc91bcb0 con 0x55badc915350 2023-10-05T19:41:28.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.472+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fb89002d040 con 0x55badc915350 2023-10-05T19:41:28.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.472+0000 7fb8a9028e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:28.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.472+0000 7fb8a9028e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:28.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.472+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc91c060 con 0x55badc915350 2023-10-05T19:41:28.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.472+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fb89002d040 con 0x55badc915350 2023-10-05T19:41:28.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.472+0000 7fb8a9028e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:28.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.472+0000 7fb8a9028e40 20 started sync module instance, tier type = 2023-10-05T19:41:28.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.472+0000 7fb8a9028e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:41:28.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.472+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc91b370 con 0x55badc910bb0 2023-10-05T19:41:28.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.476+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(84 notify.0 [create] v30'243 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb8a001c7f0 con 0x55badc910bb0 2023-10-05T19:41:28.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.476+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94261052806864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc91c410 con 0x55badc910bb0 2023-10-05T19:41:28.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.476+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc91ccb0 con 0x55badc8db730 2023-10-05T19:41:28.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.480+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(85 notify.0 [watch watch cookie 94261052806864] v30'244 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb8a001c7f0 con 0x55badc910bb0 2023-10-05T19:41:28.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.480+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(86 notify.1 [create] v30'243 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb898014040 con 0x55badc8db730 2023-10-05T19:41:28.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.480+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94261052819392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc91b8d0 con 0x55badc8db730 2023-10-05T19:41:28.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.480+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc91d860 con 0x55badc915350 2023-10-05T19:41:28.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.480+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(87 notify.1 [watch watch cookie 94261052819392] v30'244 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb898014040 con 0x55badc8db730 2023-10-05T19:41:28.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.480+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(88 notify.2 [create] v30'243 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb89002d040 con 0x55badc915350 2023-10-05T19:41:28.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.484+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94261052820496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc91dc60 con 0x55badc915350 2023-10-05T19:41:28.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.484+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc91e500 con 0x55badc910bb0 2023-10-05T19:41:28.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.484+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(90 notify.3 [create] v30'243 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb8a001c7f0 con 0x55badc910bb0 2023-10-05T19:41:28.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.484+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94261052825904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc91edf0 con 0x55badc910bb0 2023-10-05T19:41:28.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.484+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc91f800 con 0x55badc8db730 2023-10-05T19:41:28.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.484+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(89 notify.2 [watch watch cookie 94261052820496] v30'244 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb89002d040 con 0x55badc915350 2023-10-05T19:41:28.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.488+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(91 notify.3 [watch watch cookie 94261052825904] v30'244 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb8a001c7f0 con 0x55badc910bb0 2023-10-05T19:41:28.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.488+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(92 notify.4 [create] v30'243 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb898014040 con 0x55badc8db730 2023-10-05T19:41:28.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.488+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94261052827904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc91ed50 con 0x55badc8db730 2023-10-05T19:41:28.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.488+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc91fcf0 con 0x7fb8a0007030 2023-10-05T19:41:28.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.492+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(93 notify.4 [watch watch cookie 94261052827904] v30'244 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb898014040 con 0x55badc8db730 2023-10-05T19:41:28.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.492+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(94 notify.5 [create] v30'243 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb898014040 con 0x7fb8a0007030 2023-10-05T19:41:28.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.492+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94261052830480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc91fc50 con 0x7fb8a0007030 2023-10-05T19:41:28.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.492+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc920660 con 0x7fb8a0007030 2023-10-05T19:41:28.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.492+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(95 notify.5 [watch watch cookie 94261052830480] v30'244 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb898014040 con 0x7fb8a0007030 2023-10-05T19:41:28.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.492+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(96 notify.6 [create] v30'243 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb898014040 con 0x7fb8a0007030 2023-10-05T19:41:28.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.492+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94261052831856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc920660 con 0x7fb8a0007030 2023-10-05T19:41:28.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.492+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc920f00 con 0x7fb8a0007030 2023-10-05T19:41:28.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.496+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(97 notify.6 [watch watch cookie 94261052831856] v30'244 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb898014040 con 0x7fb8a0007030 2023-10-05T19:41:28.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.496+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(98 notify.7 [create] v30'243 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb898014040 con 0x7fb8a0007030 2023-10-05T19:41:28.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.496+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94261052834720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc920ce0 con 0x7fb8a0007030 2023-10-05T19:41:28.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.496+0000 7fb8a9028e40 20 add_watcher() i=0 2023-10-05T19:41:28.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.496+0000 7fb8a9028e40 20 add_watcher() i=1 2023-10-05T19:41:28.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.496+0000 7fb8a9028e40 20 add_watcher() i=2 2023-10-05T19:41:28.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.496+0000 7fb8a9028e40 20 add_watcher() i=3 2023-10-05T19:41:28.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.496+0000 7fb8a9028e40 20 add_watcher() i=4 2023-10-05T19:41:28.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.496+0000 7fb8a9028e40 20 add_watcher() i=5 2023-10-05T19:41:28.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.496+0000 7fb8a9028e40 20 add_watcher() i=6 2023-10-05T19:41:28.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.500+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(99 notify.7 [watch watch cookie 94261052834720] v30'244 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb898014040 con 0x7fb8a0007030 2023-10-05T19:41:28.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.500+0000 7fb8a9028e40 20 add_watcher() i=7 2023-10-05T19:41:28.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.500+0000 7fb8a9028e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:41:28.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.500+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc924330 con 0x55badc8db730 2023-10-05T19:41:28.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.500+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fb898014040 con 0x55badc8db730 2023-10-05T19:41:28.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.500+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94261052836288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc921780 con 0x55badc8db730 2023-10-05T19:41:28.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.500+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94261052836288] v30'233 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fb898014040 con 0x55badc8db730 2023-10-05T19:41:28.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.500+0000 7fb8777ee700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:41:28.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.500+0000 7fb8a9028e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:41:28.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.500+0000 7fb8a9028e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:41:28.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.504+0000 7fb8a9028e40 1 Processor -- start 2023-10-05T19:41:28.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.504+0000 7fb8a9028e40 1 -- start start 2023-10-05T19:41:28.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.504+0000 7fb8a9028e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55badc92e8f0 0x55badc92ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:28.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.504+0000 7fb8a9028e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55badc92f200 con 0x55badc92e8f0 2023-10-05T19:41:28.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.504+0000 7fb8a635c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55badc92e8f0 0x55badc92ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:28.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.504+0000 7fb8a635c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55badc92e8f0 0x55badc92ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:50756/0 (socket says 172.21.15.12:50756) 2023-10-05T19:41:28.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.504+0000 7fb8a635c700 1 -- 172.21.15.12:0/3552942378 learned_addr learned my addr 172.21.15.12:0/3552942378 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:28.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.504+0000 7fb8a635c700 1 -- 172.21.15.12:0/3552942378 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55badc92f340 con 0x55badc92e8f0 2023-10-05T19:41:28.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.504+0000 7fb8a635c700 1 --2- 172.21.15.12:0/3552942378 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55badc92e8f0 0x55badc92ecc0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x55badc9138e0 tx=0x7fb890027180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f3df21f1b0af1f server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:28.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.504+0000 7fb876fed700 1 -- 172.21.15.12:0/3552942378 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb890039040 con 0x55badc92e8f0 2023-10-05T19:41:28.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.504+0000 7fb876fed700 1 -- 172.21.15.12:0/3552942378 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb890046070 con 0x55badc92e8f0 2023-10-05T19:41:28.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.504+0000 7fb876fed700 1 -- 172.21.15.12:0/3552942378 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb89002a040 con 0x55badc92e8f0 2023-10-05T19:41:28.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.504+0000 7fb8a9028e40 1 -- 172.21.15.12:0/3552942378 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55badc92e8f0 msgr2=0x55badc92ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:28.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.504+0000 7fb8a9028e40 1 --2- 172.21.15.12:0/3552942378 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55badc92e8f0 0x55badc92ecc0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x55badc9138e0 tx=0x7fb890027180 comp rx=0 tx=0).stop 2023-10-05T19:41:28.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.504+0000 7fb8a9028e40 1 -- 172.21.15.12:0/3552942378 shutdown_connections 2023-10-05T19:41:28.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.504+0000 7fb8a9028e40 1 --2- 172.21.15.12:0/3552942378 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55badc92e8f0 0x55badc92ecc0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:28.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.504+0000 7fb8a9028e40 1 -- 172.21.15.12:0/3552942378 >> 172.21.15.12:0/3552942378 conn(0x55badc929cb0 msgr2=0x55badc927e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:28.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.508+0000 7fb8a9028e40 1 -- 172.21.15.12:0/3552942378 shutdown_connections 2023-10-05T19:41:28.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.508+0000 7fb8a9028e40 1 -- 172.21.15.12:0/3552942378 wait complete. 2023-10-05T19:41:28.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.508+0000 7fb8a9028e40 1 Processor -- start 2023-10-05T19:41:28.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.508+0000 7fb8a9028e40 1 -- start start 2023-10-05T19:41:28.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.508+0000 7fb8a9028e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55badc92e8f0 0x55badc9344d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:28.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.508+0000 7fb8a9028e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55badc9354d0 con 0x55badc92e8f0 2023-10-05T19:41:28.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.508+0000 7fb8a635c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55badc92e8f0 0x55badc9344d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:28.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.508+0000 7fb8a635c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55badc92e8f0 0x55badc9344d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:50760/0 (socket says 172.21.15.12:50760) 2023-10-05T19:41:28.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.508+0000 7fb8a635c700 1 -- 172.21.15.12:0/385242213 learned_addr learned my addr 172.21.15.12:0/385242213 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:28.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.508+0000 7fb8a635c700 1 -- 172.21.15.12:0/385242213 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb888001e10 con 0x55badc92e8f0 2023-10-05T19:41:28.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.508+0000 7fb8a635c700 1 --2- 172.21.15.12:0/385242213 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55badc92e8f0 0x55badc9344d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x55badc90c110 tx=0x7fb890044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:28.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.508+0000 7fb8757ea700 1 -- 172.21.15.12:0/385242213 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb890039070 con 0x55badc92e8f0 2023-10-05T19:41:28.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.508+0000 7fb8757ea700 1 -- 172.21.15.12:0/385242213 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb89001b040 con 0x55badc92e8f0 2023-10-05T19:41:28.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.508+0000 7fb8a9028e40 1 -- 172.21.15.12:0/385242213 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55badc934ad0 con 0x55badc92e8f0 2023-10-05T19:41:28.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.508+0000 7fb8757ea700 1 -- 172.21.15.12:0/385242213 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb89002a030 con 0x55badc92e8f0 2023-10-05T19:41:28.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.508+0000 7fb8a9028e40 1 -- 172.21.15.12:0/385242213 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55badc934f10 con 0x55badc92e8f0 2023-10-05T19:41:28.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.512+0000 7fb8757ea700 1 -- 172.21.15.12:0/385242213 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fb89003c030 con 0x55badc92e8f0 2023-10-05T19:41:28.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.512+0000 7fb8757ea700 1 --2- 172.21.15.12:0/385242213 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fb7ec030c50 0x7fb7ec033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:28.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.512+0000 7fb8757ea700 1 -- 172.21.15.12:0/385242213 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7fb890057c20 con 0x55badc92e8f0 2023-10-05T19:41:28.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.512+0000 7fb8a9028e40 5 note: GC not initialized 2023-10-05T19:41:28.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.512+0000 7fb8a735e700 1 --2- 172.21.15.12:0/385242213 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fb7ec030c50 0x7fb7ec033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:28.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.512+0000 7fb8a735e700 1 --2- 172.21.15.12:0/385242213 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fb7ec030c50 0x7fb7ec033100 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x55badc920ec0 tx=0x7fb8980335d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:28.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.516+0000 7fb7dffff700 20 reqs_thread_entry: start 2023-10-05T19:41:28.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.516+0000 7fb8a9028e40 20 init_complete bucket index max shards: 11 2023-10-05T19:41:28.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.516+0000 7fb7e67fc700 20 reqs_thread_entry: start 2023-10-05T19:41:28.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.516+0000 7fb8a9028e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:41:28.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.516+0000 7fb7de7fc700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:41:58 2023 2023-10-05T19:41:28.521 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:41:28.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.516+0000 7fb7ddffb700 20 reqs_thread_entry: start 2023-10-05T19:41:28.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.516+0000 7fb8a9028e40 20 get_system_obj_state: rctx=0x7ffd53b69848 obj=default.rgw.meta:users.uid:bar state=0x55badc7b40b0 s->prefetch_data=0 2023-10-05T19:41:28.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.516+0000 7fb8a9028e40 10 cache get: name=default.rgw.meta+users.uid+bar : miss 2023-10-05T19:41:28.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.516+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:102 8.e 8:718f6a54:users.uid::bar:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc99cd80 con 0x7fb8a0007030 2023-10-05T19:41:28.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.516+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(102 bar [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7fb898014040 con 0x7fb8a0007030 2023-10-05T19:41:28.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.516+0000 7fb8a9028e40 10 cache put: name=default.rgw.meta+users.uid+bar info.flags=0x16 2023-10-05T19:41:28.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.516+0000 7fb8a9028e40 10 adding default.rgw.meta+users.uid+bar to cache LRU end 2023-10-05T19:41:28.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.516+0000 7fb8a9028e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:41:28.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.516+0000 7fb8a9028e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:41:28.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.516+0000 7fb8a9028e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:41:28.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.516+0000 7fb8a9028e40 10 cache get: name=default.rgw.meta+users.uid+bar : type miss (requested=0x13, cached=0x16) 2023-10-05T19:41:28.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.516+0000 7fb8a9028e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:28.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.516+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:103 8.e 8:718f6a54:users.uid::bar:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc99f810 con 0x7fb8a0007030 2023-10-05T19:41:28.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.520+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(103 bar [call,call out=48b,read 0~346 out=346b,getxattrs out=75b] v0'0 uv12 ondisk = 0) v8 ==== 273+0+469 (crc 0 0 0) 0x7fb898014040 con 0x7fb8a0007030 2023-10-05T19:41:28.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.520+0000 7fb8a9028e40 20 rados_obj.operate() r=0 bl.length=346 2023-10-05T19:41:28.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.520+0000 7fb8a9028e40 10 cache put: name=default.rgw.meta+users.uid+bar info.flags=0x13 2023-10-05T19:41:28.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.520+0000 7fb8a9028e40 10 moving default.rgw.meta+users.uid+bar to cache LRU end 2023-10-05T19:41:28.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.520+0000 7fb8a9028e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:41:28.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.520+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:104 8.4 8:267425a8:users.uid::bud:head [create,call version.set in=58b,writefull 0~181 in=181b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9a0e60 con 0x55badc910bb0 2023-10-05T19:41:28.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.520+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(104 bud [create,call,writefull 0~181] v30'1 uv1 ondisk = 0) v8 ==== 231+0+0 (crc 0 0 0) 0x7fb8a001c7f0 con 0x55badc910bb0 2023-10-05T19:41:28.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.520+0000 7fb8a9028e40 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x17 2023-10-05T19:41:28.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.520+0000 7fb8a9028e40 10 adding default.rgw.meta+users.uid+bud to cache LRU end 2023-10-05T19:41:28.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.520+0000 7fb8a9028e40 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:bud, ofs0, ns] 2023-10-05T19:41:28.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.520+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:105 7.3 7:c609908c:::notify.5:head [notify cookie 94261053356384 in=369b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9a18d0 con 0x7fb8a0007030 2023-10-05T19:41:28.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.524+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== watch-notify(notify (1) cookie 94261052830480 notify 128849018909 ret 0) v3 ==== 399+0+0 (crc 0 0 0) 0x7fb8980332d0 con 0x7fb8a0007030 2023-10-05T19:41:28.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.524+0000 7fb8a4b59700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018909 cookie 94261052830480 notifier 4711 bl.length()=357 2023-10-05T19:41:28.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.524+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(105 notify.5 [notify cookie 94261053356384 out=8b] v0'0 uv243 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fb898014040 con 0x7fb8a0007030 2023-10-05T19:41:28.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.524+0000 7fb8a4b59700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+bud info.flags=0x17 2023-10-05T19:41:28.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.524+0000 7fb8a4b59700 10 rgw watcher librados: moving default.rgw.meta+users.uid+bud to cache LRU end 2023-10-05T19:41:28.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.524+0000 7fb8a4b59700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:106 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x7fb880003d90 con 0x7fb8a0007030 2023-10-05T19:41:28.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.524+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== watch-notify(notify_complete (2) cookie 94261053356384 notify 128849018909 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7fb8980315b0 con 0x7fb8a0007030 2023-10-05T19:41:28.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.524+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 30 ==== osd_op_reply(106 notify.5 [notify-ack] v0'0 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb898014040 con 0x7fb8a0007030 2023-10-05T19:41:28.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.524+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:107 8.5 8:a00000ee:users.uid::bar.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc99fe50 con 0x55badc8db730 2023-10-05T19:41:28.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.524+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(107 bar.buckets [call out=184b] v0'0 uv3 ondisk = 0) v8 ==== 155+0+184 (crc 0 0 0) 0x7fb898014040 con 0x55badc8db730 2023-10-05T19:41:28.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.524+0000 7fb8a9028e40 20 get_system_obj_state: rctx=0x7ffd53b697b0 obj=default.rgw.meta:root:.bucket.meta.myfoo6:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7 state=0x55badc7b40b0 s->prefetch_data=0 2023-10-05T19:41:28.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.524+0000 7fb8a9028e40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo6:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7 : miss 2023-10-05T19:41:28.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.524+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:108 8.9 8:9736f005:root::.bucket.meta.myfoo6%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc99fa40 con 0x55badc910bb0 2023-10-05T19:41:28.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.528+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(108 .bucket.meta.myfoo6:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7 [call out=48b,getxattrs out=218b,stat out=16b] v0'0 uv42 ondisk = 0) v8 ==== 291+0+282 (crc 0 0 0) 0x7fb8a001c7f0 con 0x55badc910bb0 2023-10-05T19:41:28.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.528+0000 7fb8a9028e40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7 info.flags=0x16 2023-10-05T19:41:28.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.528+0000 7fb8a9028e40 10 adding default.rgw.meta+root+.bucket.meta.myfoo6:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7 to cache LRU end 2023-10-05T19:41:28.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.528+0000 7fb8a9028e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:41:28.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.528+0000 7fb8a9028e40 10 updating xattr: name=user.rgw.acl bl.length()=123 2023-10-05T19:41:28.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.528+0000 7fb8a9028e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:41:28.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.528+0000 7fb8a9028e40 20 Read xattr: user.rgw.acl 2023-10-05T19:41:28.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.528+0000 7fb8a9028e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:41:28.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.528+0000 7fb8a9028e40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo6:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7 : type miss (requested=0x13, cached=0x16) 2023-10-05T19:41:28.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.528+0000 7fb8a9028e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:28.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.528+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:109 8.9 8:9736f005:root::.bucket.meta.myfoo6%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9a05f0 con 0x55badc910bb0 2023-10-05T19:41:28.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.528+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 <== osd.2 v2:172.21.15.12:6800/29934 23 ==== osd_op_reply(109 .bucket.meta.myfoo6:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7 [call,call out=48b,read 0~353 out=353b,getxattrs out=218b] v0'0 uv42 ondisk = 0) v8 ==== 333+0+619 (crc 0 0 0) 0x7fb8a001c7f0 con 0x55badc910bb0 2023-10-05T19:41:28.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.528+0000 7fb8a9028e40 20 rados_obj.operate() r=0 bl.length=353 2023-10-05T19:41:28.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.528+0000 7fb8a9028e40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7 info.flags=0x13 2023-10-05T19:41:28.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.528+0000 7fb8a9028e40 10 moving default.rgw.meta+root+.bucket.meta.myfoo6:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7 to cache LRU end 2023-10-05T19:41:28.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.528+0000 7fb8a9028e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:41:28.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.528+0000 7fb8a9028e40 10 updating xattr: name=user.rgw.acl bl.length()=123 2023-10-05T19:41:28.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.528+0000 7fb8a9028e40 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo6:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7 2023-10-05T19:41:28.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.528+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:110 8.9 8:9736f005:root::.bucket.meta.myfoo6%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~353 in=353b,setxattr user.rgw.acl (117) in=129b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9a92e0 con 0x55badc910bb0 2023-10-05T19:41:28.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.532+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 <== osd.2 v2:172.21.15.12:6800/29934 24 ==== osd_op_reply(110 .bucket.meta.myfoo6:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7 [delete,create,call,call,writefull 0~353,setxattr (117)] v30'43 uv43 ondisk = 0) v8 ==== 417+0+0 (crc 0 0 0) 0x7fb8a001c7f0 con 0x55badc910bb0 2023-10-05T19:41:28.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.532+0000 7fb8a9028e40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7 info.flags=0x17 2023-10-05T19:41:28.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.532+0000 7fb8a9028e40 10 moving default.rgw.meta+root+.bucket.meta.myfoo6:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7 to cache LRU end 2023-10-05T19:41:28.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.532+0000 7fb8a9028e40 10 updating xattr: name=user.rgw.acl bl.length()=117 2023-10-05T19:41:28.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.532+0000 7fb8a9028e40 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo6:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7, ofs0, ns] 2023-10-05T19:41:28.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.532+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:111 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94261053395536 in=733b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9a3930 con 0x7fb8a0007030 2023-10-05T19:41:28.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.532+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 31 ==== watch-notify(notify (1) cookie 94261052831856 notify 128849018910 ret 0) v3 ==== 763+0+0 (crc 0 0 0) 0x7fb8980315b0 con 0x7fb8a0007030 2023-10-05T19:41:28.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.532+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 32 ==== osd_op_reply(111 notify.6 [notify cookie 94261053395536 out=8b] v0'0 uv243 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fb898014040 con 0x7fb8a0007030 2023-10-05T19:41:28.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.532+0000 7fb8a4b59700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018910 cookie 94261052831856 notifier 4711 bl.length()=721 2023-10-05T19:41:28.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.532+0000 7fb8a4b59700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7 info.flags=0x17 2023-10-05T19:41:28.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.532+0000 7fb8a4b59700 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo6:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7 to cache LRU end 2023-10-05T19:41:28.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.532+0000 7fb8a4b59700 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=117 2023-10-05T19:41:28.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.532+0000 7fb8a4b59700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:112 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x7fb880003d90 con 0x7fb8a0007030 2023-10-05T19:41:28.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.532+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 33 ==== watch-notify(notify_complete (2) cookie 94261053395536 notify 128849018910 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7fb898031b50 con 0x7fb8a0007030 2023-10-05T19:41:28.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.532+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 34 ==== osd_op_reply(112 notify.6 [notify-ack] v0'0 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb898014040 con 0x7fb8a0007030 2023-10-05T19:41:28.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.532+0000 7fb8a9028e40 20 handle_bi_update(): bucket=:myfoo6[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7]): orig_sources= new_sources= 2023-10-05T19:41:28.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.532+0000 7fb8a9028e40 20 handle_bi_update(): bucket=:myfoo6[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7]): potential sources added=[] removed=[] 2023-10-05T19:41:28.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.532+0000 7fb8a9028e40 20 handle_bi_update(): bucket=:myfoo6[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7]): orig_dests= new_dests= 2023-10-05T19:41:28.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.532+0000 7fb8a9028e40 20 handle_bi_update(): bucket=:myfoo6[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7]): potential dests added=[] removed=[] 2023-10-05T19:41:28.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.532+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:113 8.5 8:a00000ee:users.uid::bar.buckets:head [call user.remove_bucket in=59b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9a8e80 con 0x55badc8db730 2023-10-05T19:41:28.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.536+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(113 bar.buckets [call] v30'4 uv4 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x7fb898014040 con 0x55badc8db730 2023-10-05T19:41:28.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.536+0000 7fb8a9028e40 20 get_system_obj_state: rctx=0x7ffd53b69068 obj=default.rgw.meta:root:myfoo6 state=0x55badc7b40b0 s->prefetch_data=0 2023-10-05T19:41:28.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.536+0000 7fb8a9028e40 10 cache get: name=default.rgw.meta+root+myfoo6 : miss 2023-10-05T19:41:28.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.536+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:114 8.18 8:1944a257:root::myfoo6:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9a8e80 con 0x55badc8db730 2023-10-05T19:41:28.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.536+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(114 myfoo6 [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv16 ondisk = 0) v8 ==== 234+0+139 (crc 0 0 0) 0x7fb898014040 con 0x55badc8db730 2023-10-05T19:41:28.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.536+0000 7fb8a9028e40 10 cache put: name=default.rgw.meta+root+myfoo6 info.flags=0x16 2023-10-05T19:41:28.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.536+0000 7fb8a9028e40 10 adding default.rgw.meta+root+myfoo6 to cache LRU end 2023-10-05T19:41:28.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.536+0000 7fb8a9028e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:41:28.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.536+0000 7fb8a9028e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:41:28.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.536+0000 7fb8a9028e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:41:28.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.536+0000 7fb8a9028e40 10 cache get: name=default.rgw.meta+root+myfoo6 : type miss (requested=0x13, cached=0x16) 2023-10-05T19:41:28.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.536+0000 7fb8a9028e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:28.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.536+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:115 8.18 8:1944a257:root::myfoo6:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9a8e60 con 0x55badc8db730 2023-10-05T19:41:28.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.536+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== osd_op_reply(115 myfoo6 [call,call out=48b,read 0~166 out=166b,getxattrs out=75b] v0'0 uv16 ondisk = 0) v8 ==== 276+0+289 (crc 0 0 0) 0x7fb898014040 con 0x55badc8db730 2023-10-05T19:41:28.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.536+0000 7fb8a9028e40 20 rados_obj.operate() r=0 bl.length=166 2023-10-05T19:41:28.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.536+0000 7fb8a9028e40 10 cache put: name=default.rgw.meta+root+myfoo6 info.flags=0x13 2023-10-05T19:41:28.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.536+0000 7fb8a9028e40 10 moving default.rgw.meta+root+myfoo6 to cache LRU end 2023-10-05T19:41:28.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.536+0000 7fb8a9028e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:41:28.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.540+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:116 8.18 8:1944a257:root::myfoo6:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~166 in=166b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9a6e50 con 0x55badc8db730 2023-10-05T19:41:28.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.540+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 39 ==== osd_op_reply(116 myfoo6 [delete,create,call,call,writefull 0~166] v30'17 uv17 ondisk = 0) v8 ==== 318+0+0 (crc 0 0 0) 0x7fb898014040 con 0x55badc8db730 2023-10-05T19:41:28.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.540+0000 7fb8a9028e40 10 cache put: name=default.rgw.meta+root+myfoo6 info.flags=0x17 2023-10-05T19:41:28.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.540+0000 7fb8a9028e40 10 moving default.rgw.meta+root+myfoo6 to cache LRU end 2023-10-05T19:41:28.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.540+0000 7fb8a9028e40 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:myfoo6, ofs0, ns] 2023-10-05T19:41:28.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.540+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:117 7.e 7:7759931f:::notify.3:head [notify cookie 94261053368608 in=352b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9a4270 con 0x55badc910bb0 2023-10-05T19:41:28.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.540+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 <== osd.2 v2:172.21.15.12:6800/29934 25 ==== watch-notify(notify (1) cookie 94261052825904 notify 128849018908 ret 0) v3 ==== 382+0+0 (crc 0 0 0) 0x7fb8a0046040 con 0x55badc910bb0 2023-10-05T19:41:28.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.540+0000 7fb8a4b59700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018908 cookie 94261052825904 notifier 4711 bl.length()=340 2023-10-05T19:41:28.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.540+0000 7fb8a4b59700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo6 info.flags=0x17 2023-10-05T19:41:28.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.540+0000 7fb8a4b59700 10 rgw watcher librados: moving default.rgw.meta+root+myfoo6 to cache LRU end 2023-10-05T19:41:28.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.540+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 <== osd.2 v2:172.21.15.12:6800/29934 26 ==== osd_op_reply(117 notify.3 [notify cookie 94261053368608 out=8b] v0'0 uv243 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fb8a001c7f0 con 0x55badc910bb0 2023-10-05T19:41:28.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.540+0000 7fb8a4b59700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:118 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x7fb880003d90 con 0x55badc910bb0 2023-10-05T19:41:28.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.544+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 <== osd.2 v2:172.21.15.12:6800/29934 27 ==== osd_op_reply(118 notify.3 [notify-ack] v0'0 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb8a001c7f0 con 0x55badc910bb0 2023-10-05T19:41:28.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.544+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 <== osd.2 v2:172.21.15.12:6800/29934 28 ==== watch-notify(notify_complete (2) cookie 94261053368608 notify 128849018908 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55badc903540 con 0x55badc910bb0 2023-10-05T19:41:28.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.544+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:119 8.19 8:99adfd5d:users.uid::bud.buckets:head [call user.set_buckets_info in=208b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9a2e10 con 0x55badc8db730 2023-10-05T19:41:28.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.544+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 40 ==== osd_op_reply(119 bud.buckets [call] v30'6 uv6 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x7fb898014040 con 0x55badc8db730 2023-10-05T19:41:28.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.544+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:120 8.18 8:1944a257:root::myfoo6:head [delete,create,call version.inc in=38b,writefull 0~166 in=166b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9a3e20 con 0x55badc8db730 2023-10-05T19:41:28.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.548+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 41 ==== osd_op_reply(120 myfoo6 [delete,create,call,writefull 0~166] v30'18 uv18 ondisk = 0) v8 ==== 276+0+0 (crc 0 0 0) 0x7fb898014040 con 0x55badc8db730 2023-10-05T19:41:28.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.548+0000 7fb8a9028e40 10 cache put: name=default.rgw.meta+root+myfoo6 info.flags=0x7 2023-10-05T19:41:28.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.548+0000 7fb8a9028e40 10 moving default.rgw.meta+root+myfoo6 to cache LRU end 2023-10-05T19:41:28.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.548+0000 7fb8a9028e40 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:myfoo6, ofs0, ns] 2023-10-05T19:41:28.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.548+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:121 7.e 7:7759931f:::notify.3:head [notify cookie 94261053368608 in=328b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9a75a0 con 0x55badc910bb0 2023-10-05T19:41:28.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.548+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 <== osd.2 v2:172.21.15.12:6800/29934 29 ==== watch-notify(notify (1) cookie 94261052825904 notify 128849018909 ret 0) v3 ==== 358+0+0 (crc 0 0 0) 0x55badc903540 con 0x55badc910bb0 2023-10-05T19:41:28.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.548+0000 7fb89ffff700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018909 cookie 94261052825904 notifier 4711 bl.length()=316 2023-10-05T19:41:28.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.548+0000 7fb89ffff700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo6 info.flags=0x7 2023-10-05T19:41:28.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.548+0000 7fb89ffff700 10 rgw watcher librados: moving default.rgw.meta+root+myfoo6 to cache LRU end 2023-10-05T19:41:28.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.548+0000 7fb89ffff700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:122 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x7fb7f4003390 con 0x55badc910bb0 2023-10-05T19:41:28.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.548+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 <== osd.2 v2:172.21.15.12:6800/29934 30 ==== osd_op_reply(121 notify.3 [notify cookie 94261053368608 out=8b] v0'0 uv243 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fb8a001c7f0 con 0x55badc910bb0 2023-10-05T19:41:28.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.552+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 <== osd.2 v2:172.21.15.12:6800/29934 31 ==== watch-notify(notify_complete (2) cookie 94261053368608 notify 128849018909 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55badc903100 con 0x55badc910bb0 2023-10-05T19:41:28.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.552+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 <== osd.2 v2:172.21.15.12:6800/29934 32 ==== osd_op_reply(122 notify.3 [notify-ack] v0'0 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb8a001c7f0 con 0x55badc910bb0 2023-10-05T19:41:28.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.552+0000 7fb8a9028e40 10 int RGWRados::cls_bucket_list_unordered(const DoutPrefixProvider*, RGWBucketInfo&, int, const rgw_obj_index_key&, const string&, uint32_t, bool, std::vector&, bool*, rgw_obj_index_key*, optional_yield, RGWBucketListNameFilter) :myfoo6[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7]) start_after="", prefix="", shard_id=-1", num_entries=1100, list_versions=1, force_check_filter is unset 2023-10-05T19:41:28.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.552+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:123 4.4 4:23f8e882:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9a75a0 con 0x7fb8a0007030 2023-10-05T19:41:28.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.552+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 35 ==== osd_op_reply(123 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.0 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fb898014040 con 0x7fb8a0007030 2023-10-05T19:41:28.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.552+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:124 4.1d 4:ba39143d:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9a75a0 con 0x55badc915350 2023-10-05T19:41:28.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.552+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(124 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.1 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fb89002d040 con 0x55badc915350 2023-10-05T19:41:28.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.552+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:125 4.c 4:301ec9e9:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9a75a0 con 0x55badc8db730 2023-10-05T19:41:28.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.552+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 42 ==== osd_op_reply(125 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.2 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fb898014040 con 0x55badc8db730 2023-10-05T19:41:28.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.552+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:126 4.2e 4:7620d31e:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9a75a0 con 0x55badc915350 2023-10-05T19:41:28.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.556+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(126 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.3 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fb89002d040 con 0x55badc915350 2023-10-05T19:41:28.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.556+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:127 4.15 4:aa925a53:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9a75a0 con 0x55badc915350 2023-10-05T19:41:28.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.556+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(127 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.4 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fb89002d040 con 0x55badc915350 2023-10-05T19:41:28.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.556+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:128 4.19 4:9845dac2:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9a75a0 con 0x55badc915350 2023-10-05T19:41:28.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.556+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 <== osd.3 v2:172.21.15.12:6816/29951 27 ==== osd_op_reply(128 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.5 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fb89002d040 con 0x55badc915350 2023-10-05T19:41:28.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.556+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:129 4.a 4:502210e1:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9a75a0 con 0x55badc910bb0 2023-10-05T19:41:28.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.556+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 <== osd.2 v2:172.21.15.12:6800/29934 33 ==== osd_op_reply(129 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.6 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fb8a001c7f0 con 0x55badc910bb0 2023-10-05T19:41:28.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.556+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:130 4.15 4:a86bce80:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9a75a0 con 0x55badc915350 2023-10-05T19:41:28.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.560+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 <== osd.3 v2:172.21.15.12:6816/29951 28 ==== osd_op_reply(130 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.7 [call out=85b] v0'0 uv4 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fb89002d040 con 0x55badc915350 2023-10-05T19:41:28.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.560+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:131 4.22 4:4713563e:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9a75a0 con 0x55badc8db730 2023-10-05T19:41:28.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.560+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 43 ==== osd_op_reply(131 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.8 [call out=85b] v0'0 uv4 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fb898051060 con 0x55badc8db730 2023-10-05T19:41:28.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.560+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:132 4.2 4:42519106:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9a75a0 con 0x55badc8db730 2023-10-05T19:41:28.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.560+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 44 ==== osd_op_reply(132 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.9 [call out=377b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+377 (crc 0 0 0) 0x7fb898051060 con 0x55badc8db730 2023-10-05T19:41:28.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.560+0000 7fb8a9028e40 10 cls_bucket_list_unordered: got thirteen 2023-10-05T19:41:28.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.560+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:133 4.3c 4:3cd1d09f:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9a75a0 con 0x55badc8db730 2023-10-05T19:41:28.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.560+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 45 ==== osd_op_reply(133 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7fb898051060 con 0x55badc8db730 2023-10-05T19:41:28.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.560+0000 7fb8a9028e40 20 get_obj_state: rctx=0x7ffd53b690c0 obj=myfoo6:thirteen state=0x55badc99e348 s->prefetch_data=0 2023-10-05T19:41:28.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.560+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:134 3.2f 3:f7da5efc:::93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7_thirteen:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9a7560 con 0x55badc915350 2023-10-05T19:41:28.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.564+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 <== osd.3 v2:172.21.15.12:6816/29951 29 ==== osd_op_reply(134 93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7_thirteen [getxattrs out=854b,stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 238+0+870 (crc 0 0 0) 0x7fb89002d040 con 0x55badc915350 2023-10-05T19:41:28.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.564+0000 7fb8a9028e40 10 manifest: total_size = 8 2023-10-05T19:41:28.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.564+0000 7fb8a9028e40 20 get_obj_state: setting s->obj_tag to 93ac49f4-ff49-4f99-811b-6453b28decd5.4219.3784713100582726063 2023-10-05T19:41:28.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.564+0000 7fb8a9028e40 20 Read xattr rgw_rados: user.rgw.acl 2023-10-05T19:41:28.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.564+0000 7fb8a9028e40 20 Read xattr rgw_rados: user.rgw.content_type 2023-10-05T19:41:28.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.564+0000 7fb8a9028e40 20 Read xattr rgw_rados: user.rgw.etag 2023-10-05T19:41:28.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.564+0000 7fb8a9028e40 20 Read xattr rgw_rados: user.rgw.idtag 2023-10-05T19:41:28.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.564+0000 7fb8a9028e40 20 Read xattr rgw_rados: user.rgw.manifest 2023-10-05T19:41:28.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.564+0000 7fb8a9028e40 20 Read xattr rgw_rados: user.rgw.pg_ver 2023-10-05T19:41:28.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.564+0000 7fb8a9028e40 20 Read xattr rgw_rados: user.rgw.source_zone 2023-10-05T19:41:28.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.564+0000 7fb8a9028e40 20 Read xattr rgw_rados: user.rgw.tail_tag 2023-10-05T19:41:28.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.564+0000 7fb8a9028e40 20 get_obj_state: rctx=0x7ffd53b690c0 obj=myfoo6:thirteen state=0x55badc99e348 s->prefetch_data=0 2023-10-05T19:41:28.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.564+0000 7fb8a9028e40 20 get_system_obj_state: rctx=0x7ffd53b67790 obj=default.rgw.meta:root:.bucket.meta.myfoo6:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7 state=0x55badc7b40b0 s->prefetch_data=0 2023-10-05T19:41:28.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.564+0000 7fb8a9028e40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo6:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7 : hit (requested=0x16, cached=0x17) 2023-10-05T19:41:28.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.564+0000 7fb8a9028e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:41:28.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.564+0000 7fb8a9028e40 20 Read xattr: user.rgw.acl 2023-10-05T19:41:28.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.564+0000 7fb8a9028e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:41:28.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.564+0000 7fb8a9028e40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo6:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7 : hit (requested=0x13, cached=0x17) 2023-10-05T19:41:28.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.564+0000 7fb8a9028e40 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo6:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7 2023-10-05T19:41:28.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.564+0000 7fb8a9028e40 20 bucket index object: default.rgw.buckets.index:.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.9 2023-10-05T19:41:28.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.564+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:135 4.2 4:42519106:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.9:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=187b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9a6db0 con 0x55badc8db730 2023-10-05T19:41:28.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.564+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 46 ==== osd_op_reply(135 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.9 [stat,call,call] v30'6 uv6 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x7fb898051060 con 0x55badc8db730 2023-10-05T19:41:28.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.564+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:136 3.2f 3:f7da5efc:::93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7_thirteen:head [cmpxattr user.rgw.idtag (62) op 1 mode 1 in=76b,setxattr user.rgw.acl (117) in=129b,setxattr user.rgw.idtag (33) in=47b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9a6db0 con 0x55badc915350 2023-10-05T19:41:28.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.568+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 <== osd.3 v2:172.21.15.12:6816/29951 30 ==== osd_op_reply(136 93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7_thirteen [cmpxattr (62) op 1 mode 1,setxattr (117),setxattr (33)] v30'4 uv4 ondisk = 0) v8 ==== 280+0+0 (crc 0 0 0) 0x7fb89002d040 con 0x55badc915350 2023-10-05T19:41:28.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.568+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:137 4.2 4:42519106:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.9:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=267b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9ac730 con 0x55badc8db730 2023-10-05T19:41:28.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.568+0000 7fb8a9028e40 10 cache get: name=default.rgw.log++bucket.sync-source-hints.myfoo6 : miss 2023-10-05T19:41:28.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.568+0000 7fb8a9028e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:28.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.568+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:138 6.9 6:936e6ff6:::bucket.sync-source-hints.myfoo6:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9af490 con 0x55badc915350 2023-10-05T19:41:28.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.568+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 <== osd.3 v2:172.21.15.12:6816/29951 31 ==== osd_op_reply(138 bucket.sync-source-hints.myfoo6 [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 217+0+0 (crc 0 0 0) 0x7fb89002d040 con 0x55badc915350 2023-10-05T19:41:28.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.568+0000 7fb8a9028e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:28.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.568+0000 7fb8a9028e40 10 cache put: name=default.rgw.log++bucket.sync-source-hints.myfoo6 info.flags=0x0 2023-10-05T19:41:28.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.568+0000 7fb8a9028e40 10 adding default.rgw.log++bucket.sync-source-hints.myfoo6 to cache LRU end 2023-10-05T19:41:28.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.568+0000 7fb8a9028e40 10 cache get: name=default.rgw.log++bucket.sync-target-hints.myfoo6 : miss 2023-10-05T19:41:28.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.568+0000 7fb8a9028e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:28.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.568+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:139 6.19 6:9e3df8ab:::bucket.sync-target-hints.myfoo6:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9ac730 con 0x55badc8db730 2023-10-05T19:41:28.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.568+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 47 ==== osd_op_reply(139 bucket.sync-target-hints.myfoo6 [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 217+0+0 (crc 0 0 0) 0x7fb898051060 con 0x55badc8db730 2023-10-05T19:41:28.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.568+0000 7fb8a9028e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:28.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.568+0000 7fb8a9028e40 10 cache put: name=default.rgw.log++bucket.sync-target-hints.myfoo6 info.flags=0x0 2023-10-05T19:41:28.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.568+0000 7fb8a9028e40 10 adding default.rgw.log++bucket.sync-target-hints.myfoo6 to cache LRU end 2023-10-05T19:41:28.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.568+0000 7fb8a9028e40 10 chain_cache_entry: cache_locator= 2023-10-05T19:41:28.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.568+0000 7fb8a9028e40 20 chain_cache_entry: couldn't find cache locator 2023-10-05T19:41:28.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.568+0000 7fb8a9028e40 20 couldn't put bucket_sync_policy cache entry, might have raced with data changes 2023-10-05T19:41:28.577 INFO:teuthology.orchestra.run.smithi012.stderr:1 objects processed in :myfoo6[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7]). Next marker thirteen 2023-10-05T19:41:28.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.568+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:140 8.4 8:267425a8:users.uid::bud:head [delete,create,call version.set in=58b,writefull 0~346 in=346b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9af460 con 0x55badc910bb0 2023-10-05T19:41:28.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.572+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 48 ==== osd_op_reply(137 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.9 [stat,call,call] v30'7 uv7 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x7fb898051060 con 0x55badc8db730 2023-10-05T19:41:28.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.572+0000 7fb8a4b59700 20 handle_completion(): completion ok for obj=thirteen 2023-10-05T19:41:28.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.572+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 <== osd.2 v2:172.21.15.12:6800/29934 34 ==== osd_op_reply(140 bud [delete,create,call,writefull 0~346] v30'2 uv2 ondisk = 0) v8 ==== 273+0+0 (crc 0 0 0) 0x7fb8a001c7f0 con 0x55badc910bb0 2023-10-05T19:41:28.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.572+0000 7fb8a9028e40 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x17 2023-10-05T19:41:28.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.572+0000 7fb8a9028e40 10 moving default.rgw.meta+users.uid+bud to cache LRU end 2023-10-05T19:41:28.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.572+0000 7fb8a9028e40 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:bud, ofs0, ns] 2023-10-05T19:41:28.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.572+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:141 7.3 7:c609908c:::notify.5:head [notify cookie 94261053419104 in=534b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9af460 con 0x7fb8a0007030 2023-10-05T19:41:28.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.572+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 36 ==== watch-notify(notify (1) cookie 94261052830480 notify 128849018911 ret 0) v3 ==== 564+0+0 (crc 0 0 0) 0x7fb898053000 con 0x7fb8a0007030 2023-10-05T19:41:28.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.572+0000 7fb89ffff700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018911 cookie 94261052830480 notifier 4711 bl.length()=522 2023-10-05T19:41:28.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.572+0000 7fb89ffff700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+bud info.flags=0x17 2023-10-05T19:41:28.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.572+0000 7fb89ffff700 10 rgw watcher librados: moving default.rgw.meta+users.uid+bud to cache LRU end 2023-10-05T19:41:28.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.572+0000 7fb89ffff700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:142 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x7fb7f40033f0 con 0x7fb8a0007030 2023-10-05T19:41:28.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.572+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 37 ==== osd_op_reply(141 notify.5 [notify cookie 94261053419104 out=8b] v0'0 uv243 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fb898051060 con 0x7fb8a0007030 2023-10-05T19:41:28.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.576+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 38 ==== watch-notify(notify_complete (2) cookie 94261053419104 notify 128849018911 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7fb898031b50 con 0x7fb8a0007030 2023-10-05T19:41:28.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.576+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 39 ==== osd_op_reply(142 notify.5 [notify-ack] v0'0 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb898051060 con 0x7fb8a0007030 2023-10-05T19:41:28.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.576+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:143 8.a 8:50c8eddb:users.keys::NX5QOQKC6BH2IDN8HC7A:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9ad420 con 0x55badc8db730 2023-10-05T19:41:28.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.576+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 49 ==== osd_op_reply(143 NX5QOQKC6BH2IDN8HC7A [delete,create,writefull 0~7] v30'2 uv2 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x7fb898051060 con 0x55badc8db730 2023-10-05T19:41:28.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.576+0000 7fb8a9028e40 10 cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x7 2023-10-05T19:41:28.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.576+0000 7fb8a9028e40 10 adding default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2023-10-05T19:41:28.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.576+0000 7fb8a9028e40 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.keys:NX5QOQKC6BH2IDN8HC7A, ofs0, ns] 2023-10-05T19:41:28.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.576+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:144 7.3 7:c609908c:::notify.5:head [notify cookie 94261053419056 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc99e160 con 0x7fb8a0007030 2023-10-05T19:41:28.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.580+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 40 ==== watch-notify(notify (1) cookie 94261052830480 notify 128849018912 ret 0) v3 ==== 219+0+0 (crc 0 0 0) 0x7fb898031b50 con 0x7fb8a0007030 2023-10-05T19:41:28.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.580+0000 7fb8a4b59700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018912 cookie 94261052830480 notifier 4711 bl.length()=177 2023-10-05T19:41:28.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.580+0000 7fb8a4b59700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x7 2023-10-05T19:41:28.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.580+0000 7fb8a4b59700 10 rgw watcher librados: moving default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2023-10-05T19:41:28.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.580+0000 7fb8a4b59700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:145 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x7fb880003d90 con 0x7fb8a0007030 2023-10-05T19:41:28.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.580+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 41 ==== osd_op_reply(144 notify.5 [notify cookie 94261053419056 out=8b] v0'0 uv243 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fb898051060 con 0x7fb8a0007030 2023-10-05T19:41:28.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.580+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 42 ==== osd_op_reply(145 notify.5 [notify-ack] v0'0 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb898051060 con 0x7fb8a0007030 2023-10-05T19:41:28.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.580+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 43 ==== watch-notify(notify_complete (2) cookie 94261053419056 notify 128849018912 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7fb89803e470 con 0x7fb8a0007030 2023-10-05T19:41:28.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.580+0000 7fb8a9028e40 10 removing user index: bar 2023-10-05T19:41:28.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.580+0000 7fb8a9028e40 10 removing default.rgw.meta+users.uid+bar from cache 2023-10-05T19:41:28.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.580+0000 7fb8a9028e40 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 1, obj: default.rgw.meta:users.uid:bar, ofs0, ns] 2023-10-05T19:41:28.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.580+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:146 7.9 7:93e5b521:::notify.7:head [notify cookie 94261053419056 in=164b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc99e160 con 0x7fb8a0007030 2023-10-05T19:41:28.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.580+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 44 ==== watch-notify(notify (1) cookie 94261052834720 notify 128849018913 ret 0) v3 ==== 194+0+0 (crc 0 0 0) 0x7fb89803e470 con 0x7fb8a0007030 2023-10-05T19:41:28.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.580+0000 7fb89ffff700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018913 cookie 94261052834720 notifier 4711 bl.length()=152 2023-10-05T19:41:28.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.580+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 45 ==== osd_op_reply(146 notify.7 [notify cookie 94261053419056 out=8b] v0'0 uv243 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fb898060020 con 0x7fb8a0007030 2023-10-05T19:41:28.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.580+0000 7fb89ffff700 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:147 7.9 7:93e5b521:::notify.7:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x7fb7f40033f0 con 0x7fb8a0007030 2023-10-05T19:41:28.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.580+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 46 ==== watch-notify(notify_complete (2) cookie 94261053419056 notify 128849018913 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7fb89803e6a0 con 0x7fb8a0007030 2023-10-05T19:41:28.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.580+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 47 ==== osd_op_reply(147 notify.7 [notify-ack] v0'0 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb898060020 con 0x7fb8a0007030 2023-10-05T19:41:28.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.580+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:148 8.e 8:718f6a54:users.uid::bar:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9ad420 con 0x7fb8a0007030 2023-10-05T19:41:28.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.584+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 48 ==== osd_op_reply(148 bar [delete] v30'13 uv13 ondisk = 0) v8 ==== 147+0+0 (crc 0 0 0) 0x7fb898060020 con 0x7fb8a0007030 2023-10-05T19:41:28.590 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:41:28.591 INFO:teuthology.orchestra.run.smithi012.stdout: "user_id": "bud", 2023-10-05T19:41:28.591 INFO:teuthology.orchestra.run.smithi012.stdout: "display_name": "Bar", 2023-10-05T19:41:28.591 INFO:teuthology.orchestra.run.smithi012.stdout: "email": "", 2023-10-05T19:41:28.591 INFO:teuthology.orchestra.run.smithi012.stdout: "suspended": 0, 2023-10-05T19:41:28.591 INFO:teuthology.orchestra.run.smithi012.stdout: "max_buckets": 4, 2023-10-05T19:41:28.591 INFO:teuthology.orchestra.run.smithi012.stdout: "subusers": [], 2023-10-05T19:41:28.591 INFO:teuthology.orchestra.run.smithi012.stdout: "keys": [ 2023-10-05T19:41:28.591 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:28.591 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "bud", 2023-10-05T19:41:28.591 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key": "NX5QOQKC6BH2IDN8HC7A", 2023-10-05T19:41:28.592 INFO:teuthology.orchestra.run.smithi012.stdout: "secret_key": "LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn" 2023-10-05T19:41:28.592 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:28.592 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:41:28.592 INFO:teuthology.orchestra.run.smithi012.stdout: "swift_keys": [], 2023-10-05T19:41:28.592 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": [], 2023-10-05T19:41:28.592 INFO:teuthology.orchestra.run.smithi012.stdout: "op_mask": "read, write, delete", 2023-10-05T19:41:28.592 INFO:teuthology.orchestra.run.smithi012.stdout: "default_placement": "", 2023-10-05T19:41:28.592 INFO:teuthology.orchestra.run.smithi012.stdout: "default_storage_class": "", 2023-10-05T19:41:28.592 INFO:teuthology.orchestra.run.smithi012.stdout: "placement_tags": [], 2023-10-05T19:41:28.592 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket_quota": { 2023-10-05T19:41:28.592 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:41:28.592 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:41:28.593 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:41:28.593 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:41:28.593 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:41:28.593 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:28.593 INFO:teuthology.orchestra.run.smithi012.stdout: "user_quota": { 2023-10-05T19:41:28.593 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:41:28.593 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:41:28.593 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:41:28.593 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:41:28.593 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:41:28.593 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:28.593 INFO:teuthology.orchestra.run.smithi012.stdout: "temp_url_keys": [], 2023-10-05T19:41:28.594 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "rgw", 2023-10-05T19:41:28.594 INFO:teuthology.orchestra.run.smithi012.stdout: "mfa_ids": [] 2023-10-05T19:41:28.594 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:41:28.594 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:41:28.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.584+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:149 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94261052806864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc8f05f0 con 0x55badc910bb0 2023-10-05T19:41:28.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.588+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 <== osd.2 v2:172.21.15.12:6800/29934 35 ==== osd_op_reply(149 notify.0 [watch unwatch cookie 94261052806864] v30'245 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb8a001c7f0 con 0x55badc910bb0 2023-10-05T19:41:28.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.588+0000 7fb8a9028e40 20 remove_watcher() i=0 2023-10-05T19:41:28.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.588+0000 7fb8a9028e40 2 removed watcher, disabling cache 2023-10-05T19:41:28.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.588+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:150 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94261052819392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9196d0 con 0x55badc8db730 2023-10-05T19:41:28.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.592+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 50 ==== osd_op_reply(150 notify.1 [watch unwatch cookie 94261052819392] v30'245 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb898060020 con 0x55badc8db730 2023-10-05T19:41:28.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.592+0000 7fb8a9028e40 20 remove_watcher() i=1 2023-10-05T19:41:28.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.592+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:151 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94261052820496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9196d0 con 0x55badc915350 2023-10-05T19:41:28.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.592+0000 7fb8a635c700 1 -- 172.21.15.12:0/1873315814 <== osd.3 v2:172.21.15.12:6816/29951 32 ==== osd_op_reply(151 notify.2 [watch unwatch cookie 94261052820496] v30'245 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb89002d040 con 0x55badc915350 2023-10-05T19:41:28.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.592+0000 7fb8a9028e40 20 remove_watcher() i=2 2023-10-05T19:41:28.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.592+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:152 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94261052825904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9196d0 con 0x55badc910bb0 2023-10-05T19:41:28.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.596+0000 7fb8a6b5d700 1 -- 172.21.15.12:0/1873315814 <== osd.2 v2:172.21.15.12:6800/29934 36 ==== osd_op_reply(152 notify.3 [watch unwatch cookie 94261052825904] v30'245 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb8a001c7f0 con 0x55badc910bb0 2023-10-05T19:41:28.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.596+0000 7fb8a9028e40 20 remove_watcher() i=3 2023-10-05T19:41:28.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.596+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:153 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94261052827904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9196d0 con 0x55badc8db730 2023-10-05T19:41:28.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.600+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 51 ==== osd_op_reply(153 notify.4 [watch unwatch cookie 94261052827904] v30'245 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb898060020 con 0x55badc8db730 2023-10-05T19:41:28.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.600+0000 7fb8a9028e40 20 remove_watcher() i=4 2023-10-05T19:41:28.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.600+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:154 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94261052830480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc9196d0 con 0x7fb8a0007030 2023-10-05T19:41:28.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.604+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 49 ==== osd_op_reply(154 notify.5 [watch unwatch cookie 94261052830480] v30'245 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb898060020 con 0x7fb8a0007030 2023-10-05T19:41:28.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.604+0000 7fb8a9028e40 20 remove_watcher() i=5 2023-10-05T19:41:28.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.604+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:155 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94261052831856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc91f310 con 0x7fb8a0007030 2023-10-05T19:41:28.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.604+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 50 ==== osd_op_reply(155 notify.6 [watch unwatch cookie 94261052831856] v30'245 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb898060020 con 0x7fb8a0007030 2023-10-05T19:41:28.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.604+0000 7fb8a9028e40 20 remove_watcher() i=6 2023-10-05T19:41:28.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.604+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:156 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94261052834720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc91f310 con 0x7fb8a0007030 2023-10-05T19:41:28.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.608+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.0 v2:172.21.15.8:6800/27563 51 ==== osd_op_reply(156 notify.7 [watch unwatch cookie 94261052834720] v30'245 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb898060020 con 0x7fb8a0007030 2023-10-05T19:41:28.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.608+0000 7fb8a9028e40 20 remove_watcher() i=7 2023-10-05T19:41:28.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.612+0000 7fb8777ee700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:41:28.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.612+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:157 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94261052836288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55badc95ce30 con 0x55badc8db730 2023-10-05T19:41:28.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.612+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 <== osd.1 v2:172.21.15.12:6808/29933 52 ==== osd_op_reply(157 data_loggenerations_metadata [watch unwatch cookie 94261052836288] v30'234 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fb898060020 con 0x55badc8db730 2023-10-05T19:41:28.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.612+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fb8a0007030 msgr2=0x7fb8a000c9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:28.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.612+0000 7fb8a9028e40 1 --2- 172.21.15.12:0/1873315814 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fb8a0007030 0x7fb8a000c9c0 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:28.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.612+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55badc8db730 msgr2=0x55badc90e680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:28.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.612+0000 7fb8a9028e40 1 --2- 172.21.15.12:0/1873315814 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55badc8db730 0x55badc90e680 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:28.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.612+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55badc910bb0 msgr2=0x55badc912f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:28.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.612+0000 7fb8a9028e40 1 --2- 172.21.15.12:0/1873315814 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55badc910bb0 0x55badc912f90 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:28.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.612+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55badc915350 msgr2=0x55badc917760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:28.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.612+0000 7fb8a9028e40 1 --2- 172.21.15.12:0/1873315814 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55badc915350 0x55badc917760 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:28.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.612+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fb88c030bf0 msgr2=0x7fb88c0330a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:28.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.612+0000 7fb8a9028e40 1 --2- 172.21.15.12:0/1873315814 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fb88c030bf0 0x7fb88c0330a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb8900044f0 tx=0x7fb890006690 comp rx=0 tx=0).stop 2023-10-05T19:41:28.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.612+0000 7fb8a735e700 1 -- 172.21.15.12:0/1873315814 reap_dead start 2023-10-05T19:41:28.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.612+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55badc8dae20 msgr2=0x55badc902bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:28.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.612+0000 7fb8a9028e40 1 --2- 172.21.15.12:0/1873315814 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55badc8dae20 0x55badc902bc0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fb8a001f040 tx=0x7fb8a001c000 comp rx=0 tx=0).stop 2023-10-05T19:41:28.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.616+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 shutdown_connections 2023-10-05T19:41:28.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.616+0000 7fb8a9028e40 1 --2- 172.21.15.12:0/1873315814 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55badc8dae20 0x55badc902bc0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:28.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.616+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 >> 172.21.15.12:0/1873315814 conn(0x55badc8d6840 msgr2=0x55badc8f4350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:28.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.616+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 shutdown_connections 2023-10-05T19:41:28.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.616+0000 7fb8a9028e40 1 -- 172.21.15.12:0/1873315814 wait complete. 2023-10-05T19:41:28.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.616+0000 7fb8a9028e40 1 -- 172.21.15.12:0/385242213 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fb7ec030c50 msgr2=0x7fb7ec033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:28.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.616+0000 7fb8a9028e40 1 --2- 172.21.15.12:0/385242213 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fb7ec030c50 0x7fb7ec033100 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x55badc920ec0 tx=0x7fb8980335d0 comp rx=0 tx=0).stop 2023-10-05T19:41:28.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.616+0000 7fb8a9028e40 1 -- 172.21.15.12:0/385242213 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55badc92e8f0 msgr2=0x55badc9344d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:28.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.616+0000 7fb8a9028e40 1 --2- 172.21.15.12:0/385242213 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55badc92e8f0 0x55badc9344d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x55badc90c110 tx=0x7fb890044000 comp rx=0 tx=0).stop 2023-10-05T19:41:28.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.616+0000 7fb8a9028e40 1 -- 172.21.15.12:0/385242213 shutdown_connections 2023-10-05T19:41:28.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.616+0000 7fb8a9028e40 1 --2- 172.21.15.12:0/385242213 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fb7ec030c50 0x7fb7ec033100 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:28.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.616+0000 7fb8a9028e40 1 --2- 172.21.15.12:0/385242213 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55badc92e8f0 0x55badc9344d0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:28.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.616+0000 7fb8a9028e40 1 -- 172.21.15.12:0/385242213 >> 172.21.15.12:0/385242213 conn(0x55badc929cb0 msgr2=0x55badc927d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:28.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.616+0000 7fb8a9028e40 1 -- 172.21.15.12:0/385242213 shutdown_connections 2023-10-05T19:41:28.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:28.616+0000 7fb8a9028e40 1 -- 172.21.15.12:0/385242213 wait complete. 2023-10-05T19:41:28.626 DEBUG:tasks.util.rgw: json result: {'user_id': 'bud', 'display_name': 'Bar', 'email': '', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'bud', 'access_key': 'NX5QOQKC6BH2IDN8HC7A', 'secret_key': 'LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:41:33.637 INFO:tasks.radosgw_admin:RL: get_obj myfoo6 bud bytes_out=0 bytes_in=0 failed=False 2023-10-05T19:41:33.637 INFO:tasks.radosgw_admin:RL: get_obj myfoo6 bud bytes_out=0 bytes_in=8 failed=False 2023-10-05T19:41:33.637 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '4'] 2023-10-05T19:41:33.637 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '4'] 2023-10-05T19:41:33.638 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid fud --display-name Fud --access-key p5YnriCv1nAtykxBrupQ --secret Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh --max-buckets 4 2023-10-05T19:41:33.683 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:41:33.684 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:41:33.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.680+0000 7fd5c3782e40 1 Processor -- start 2023-10-05T19:41:33.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.680+0000 7fd5c3782e40 1 -- start start 2023-10-05T19:41:33.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.680+0000 7fd5c3782e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x563c91978330 0x563c9197a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:33.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.680+0000 7fd5c3782e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x563c918c0bd0 con 0x563c918e8410 2023-10-05T19:41:33.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.680+0000 7fd5c3782e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x563c9187dee0 con 0x563c91978330 2023-10-05T19:41:33.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.680+0000 7fd5c0ab6700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x563c91978330 0x563c9197a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:33.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.680+0000 7fd5c12b7700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x563c918e8410 0x563c91976c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:44306/0 (socket says 172.21.15.12:44306) 2023-10-05T19:41:33.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.680+0000 7fd5c12b7700 1 -- 172.21.15.12:0/3824166360 learned_addr learned my addr 172.21.15.12:0/3824166360 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:41:33.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.680+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/3824166360 >> v1:172.21.15.8:6789/0 conn(0x563c918e8410 legacy=0x563c91976c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:33.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.680+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/3824166360 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c9197bc60 con 0x563c91978330 2023-10-05T19:41:33.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.680+0000 7fd5c0ab6700 1 --2- 172.21.15.12:0/3824166360 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c91978330 0x563c9197a710 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800a5d0 tx=0x7fd5b8005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d53190087d0b6721 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:33.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.680+0000 7fd5b3fff700 1 -- 172.21.15.12:0/3824166360 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd5b80086a0 con 0x563c91978330 2023-10-05T19:41:33.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.680+0000 7fd5b3fff700 1 -- 172.21.15.12:0/3824166360 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b8008800 con 0x563c91978330 2023-10-05T19:41:33.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.680+0000 7fd5b3fff700 1 -- 172.21.15.12:0/3824166360 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd5b8008ad0 con 0x563c91978330 2023-10-05T19:41:33.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.680+0000 7fd5c3782e40 1 -- 172.21.15.12:0/3824166360 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c91978330 msgr2=0x563c9197a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:33.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.680+0000 7fd5c3782e40 1 --2- 172.21.15.12:0/3824166360 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c91978330 0x563c9197a710 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800a5d0 tx=0x7fd5b8005a50 comp rx=0 tx=0).stop 2023-10-05T19:41:33.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.680+0000 7fd5c3782e40 1 -- 172.21.15.12:0/3824166360 shutdown_connections 2023-10-05T19:41:33.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.680+0000 7fd5c3782e40 1 --2- 172.21.15.12:0/3824166360 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c91978330 0x563c9197a710 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:33.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.680+0000 7fd5c3782e40 1 -- 172.21.15.12:0/3824166360 >> 172.21.15.12:0/3824166360 conn(0x563c9196e760 msgr2=0x563c91970b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:33.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.680+0000 7fd5c3782e40 1 -- 172.21.15.12:0/3824166360 shutdown_connections 2023-10-05T19:41:33.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.680+0000 7fd5c3782e40 1 -- 172.21.15.12:0/3824166360 wait complete. 2023-10-05T19:41:33.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.684+0000 7fd5c3782e40 1 Processor -- start 2023-10-05T19:41:33.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.684+0000 7fd5c3782e40 1 -- start start 2023-10-05T19:41:33.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.684+0000 7fd5c3782e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c918e8410 0x563c91974190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:33.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.684+0000 7fd5c3782e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c91841e30 con 0x563c918e8410 2023-10-05T19:41:33.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.684+0000 7fd5c12b7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c918e8410 0x563c91974190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:33.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.684+0000 7fd5c12b7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c918e8410 0x563c91974190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:60762/0 (socket says 172.21.15.12:60762) 2023-10-05T19:41:33.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.684+0000 7fd5c12b7700 1 -- 172.21.15.12:0/3238284150 learned_addr learned my addr 172.21.15.12:0/3238284150 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:33.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.684+0000 7fd5c12b7700 1 -- 172.21.15.12:0/3238284150 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b8005cf0 con 0x563c918e8410 2023-10-05T19:41:33.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.684+0000 7fd5c12b7700 1 --2- 172.21.15.12:0/3238284150 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c918e8410 0x563c91974190 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc0014a0 tx=0x7fd5bc006b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=121a5a0cb7b302fb server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:33.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.684+0000 7fd5b2ffd700 1 -- 172.21.15.12:0/3238284150 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd5bc006110 con 0x563c918e8410 2023-10-05T19:41:33.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.684+0000 7fd5b2ffd700 1 -- 172.21.15.12:0/3238284150 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5bc006270 con 0x563c918e8410 2023-10-05T19:41:33.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.684+0000 7fd5b2ffd700 1 -- 172.21.15.12:0/3238284150 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd5bc004520 con 0x563c918e8410 2023-10-05T19:41:33.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.684+0000 7fd5c3782e40 1 -- 172.21.15.12:0/3238284150 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c918e8410 msgr2=0x563c91974190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:33.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.684+0000 7fd5c3782e40 1 --2- 172.21.15.12:0/3238284150 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c918e8410 0x563c91974190 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc0014a0 tx=0x7fd5bc006b90 comp rx=0 tx=0).stop 2023-10-05T19:41:33.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.684+0000 7fd5c3782e40 1 -- 172.21.15.12:0/3238284150 shutdown_connections 2023-10-05T19:41:33.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.684+0000 7fd5c3782e40 1 --2- 172.21.15.12:0/3238284150 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c918e8410 0x563c91974190 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:33.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.684+0000 7fd5c3782e40 1 -- 172.21.15.12:0/3238284150 >> 172.21.15.12:0/3238284150 conn(0x563c9196e760 msgr2=0x563c9198ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:33.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.684+0000 7fd5c3782e40 1 -- 172.21.15.12:0/3238284150 shutdown_connections 2023-10-05T19:41:33.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.684+0000 7fd5c3782e40 1 -- 172.21.15.12:0/3238284150 wait complete. 2023-10-05T19:41:33.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.688+0000 7fd5c3782e40 1 Processor -- start 2023-10-05T19:41:33.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.688+0000 7fd5c3782e40 1 -- start start 2023-10-05T19:41:33.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.688+0000 7fd5c3782e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c918e8410 0x563c9199c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:33.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.688+0000 7fd5c3782e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c91814ed0 con 0x563c918e8410 2023-10-05T19:41:33.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.688+0000 7fd5c12b7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c918e8410 0x563c9199c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:33.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.688+0000 7fd5c12b7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c918e8410 0x563c9199c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:60772/0 (socket says 172.21.15.12:60772) 2023-10-05T19:41:33.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.688+0000 7fd5c12b7700 1 -- 172.21.15.12:0/2345853771 learned_addr learned my addr 172.21.15.12:0/2345853771 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:33.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.688+0000 7fd5c12b7700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c918c0bd0 con 0x563c918e8410 2023-10-05T19:41:33.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.688+0000 7fd5c12b7700 1 --2- 172.21.15.12:0/2345853771 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c918e8410 0x563c9199c040 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc006bd0 tx=0x7fd5bc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:33.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.688+0000 7fd5b17fa700 1 -- 172.21.15.12:0/2345853771 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd5bc013750 con 0x563c918e8410 2023-10-05T19:41:33.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.688+0000 7fd5b17fa700 1 -- 172.21.15.12:0/2345853771 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5bc0138b0 con 0x563c918e8410 2023-10-05T19:41:33.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.688+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563c9199c580 con 0x563c918e8410 2023-10-05T19:41:33.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.688+0000 7fd5b17fa700 1 -- 172.21.15.12:0/2345853771 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd5bc013b80 con 0x563c918e8410 2023-10-05T19:41:33.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.688+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563c9199c9c0 con 0x563c918e8410 2023-10-05T19:41:33.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.692+0000 7fd5b17fa700 1 -- 172.21.15.12:0/2345853771 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fd5bc013d30 con 0x563c918e8410 2023-10-05T19:41:33.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.692+0000 7fd5c3782e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:33.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.692+0000 7fd5b17fa700 1 --2- 172.21.15.12:0/2345853771 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd5a4030c50 0x7fd5a4033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:33.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.692+0000 7fd5b17fa700 1 -- 172.21.15.12:0/2345853771 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7fd5bc045150 con 0x563c918e8410 2023-10-05T19:41:33.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.692+0000 7fd5c0ab6700 1 --2- 172.21.15.12:0/2345853771 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd5a4030c50 0x7fd5a4033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:33.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.692+0000 7fd5c3782e40 1 --2- 172.21.15.12:0/2345853771 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x563c91978330 0x563c919a7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:33.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.692+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919a8230 con 0x563c91978330 2023-10-05T19:41:33.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.692+0000 7fd5c1ab8700 1 --2- 172.21.15.12:0/2345853771 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x563c91978330 0x563c919a7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:33.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.692+0000 7fd5c0ab6700 1 --2- 172.21.15.12:0/2345853771 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd5a4030c50 0x7fd5a4033100 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8005a90 tx=0x7fd5b8015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:33.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.692+0000 7fd5c1ab8700 1 --2- 172.21.15.12:0/2345853771 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x563c91978330 0x563c919a7b30 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:33.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.692+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd5b400a500 con 0x563c91978330 2023-10-05T19:41:33.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.692+0000 7fd5c3782e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:33.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.692+0000 7fd5c3782e40 20 realm 2023-10-05T19:41:33.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.692+0000 7fd5c3782e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:33.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.692+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919a5190 con 0x563c91978330 2023-10-05T19:41:33.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.692+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd5b400a500 con 0x563c91978330 2023-10-05T19:41:33.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.692+0000 7fd5c3782e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:33.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.692+0000 7fd5c3782e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:41:33.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.692+0000 7fd5c3782e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:33.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.692+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919a8d10 con 0x563c91978330 2023-10-05T19:41:33.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.696+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd5b400a500 con 0x563c91978330 2023-10-05T19:41:33.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.696+0000 7fd5c3782e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:33.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.696+0000 7fd5c3782e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:33.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.696+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919a8920 con 0x563c91978330 2023-10-05T19:41:33.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.696+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fd5b400a500 con 0x563c91978330 2023-10-05T19:41:33.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.696+0000 7fd5c3782e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:33.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.696+0000 7fd5c3782e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:33.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.696+0000 7fd5c3782e40 1 --2- 172.21.15.12:0/2345853771 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x563c919aa0b0 0x563c919ac490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:33.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.696+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919ac9d0 con 0x563c919aa0b0 2023-10-05T19:41:33.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.696+0000 7fd5c12b7700 1 --2- 172.21.15.12:0/2345853771 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x563c919aa0b0 0x563c919ac490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:33.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.696+0000 7fd5c12b7700 1 --2- 172.21.15.12:0/2345853771 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x563c919aa0b0 0x563c919ac490 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:33.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.696+0000 7fd5c12b7700 1 -- 172.21.15.12:0/2345853771 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fd5bc024020 con 0x563c919aa0b0 2023-10-05T19:41:33.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.696+0000 7fd5c3782e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:41:33.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.696+0000 7fd5c3782e40 20 searching for the correct realm 2023-10-05T19:41:33.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.696+0000 7fd5c3782e40 1 --2- 172.21.15.12:0/2345853771 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x563c919ae850 0x563c919b0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:33.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.696+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1310 con 0x563c919ae850 2023-10-05T19:41:33.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.696+0000 7fd5c0ab6700 1 --2- 172.21.15.12:0/2345853771 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x563c919ae850 0x563c919b0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:33.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.696+0000 7fd5c0ab6700 1 --2- 172.21.15.12:0/2345853771 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x563c919ae850 0x563c919b0c60 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:33.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.700+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/2345853771 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fd5b8008a80 con 0x563c919ae850 2023-10-05T19:41:33.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.700+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1310 con 0x563c919aa0b0 2023-10-05T19:41:33.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.700+0000 7fd5c12b7700 1 -- 172.21.15.12:0/2345853771 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5bc024020 con 0x563c919aa0b0 2023-10-05T19:41:33.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.700+0000 7fd5c12b7700 1 --2- 172.21.15.12:0/2345853771 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fd5bc027be0 0x7fd5bc028460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:33.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.700+0000 7fd5c12b7700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1310 con 0x7fd5bc027be0 2023-10-05T19:41:33.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.700+0000 7fd5c1ab8700 1 --2- 172.21.15.12:0/2345853771 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fd5bc027be0 0x7fd5bc028460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:33.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.700+0000 7fd5c1ab8700 1 --2- 172.21.15.12:0/2345853771 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fd5bc027be0 0x7fd5bc028460 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:33.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.704+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5b4014040 con 0x7fd5bc027be0 2023-10-05T19:41:33.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.704+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1310 con 0x563c91978330 2023-10-05T19:41:33.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.704+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5b4014040 con 0x563c91978330 2023-10-05T19:41:33.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.704+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1310 con 0x563c91978330 2023-10-05T19:41:33.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.704+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5b4014040 con 0x563c91978330 2023-10-05T19:41:33.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.704+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1310 con 0x563c91978330 2023-10-05T19:41:33.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.704+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(11 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5b4014040 con 0x563c91978330 2023-10-05T19:41:33.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.704+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1310 con 0x7fd5bc027be0 2023-10-05T19:41:33.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.704+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5b4014040 con 0x7fd5bc027be0 2023-10-05T19:41:33.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.704+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1310 con 0x563c919aa0b0 2023-10-05T19:41:33.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.704+0000 7fd5c12b7700 1 -- 172.21.15.12:0/2345853771 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5bc024020 con 0x563c919aa0b0 2023-10-05T19:41:33.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.704+0000 7fd5c12b7700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1310 con 0x563c919aa0b0 2023-10-05T19:41:33.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.708+0000 7fd5c12b7700 1 -- 172.21.15.12:0/2345853771 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5bc024020 con 0x563c919aa0b0 2023-10-05T19:41:33.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.708+0000 7fd5c12b7700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1310 con 0x563c91978330 2023-10-05T19:41:33.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.708+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5b4014040 con 0x563c91978330 2023-10-05T19:41:33.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.708+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1310 con 0x7fd5bc027be0 2023-10-05T19:41:33.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.708+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5b4014040 con 0x7fd5bc027be0 2023-10-05T19:41:33.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.708+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1310 con 0x563c91978330 2023-10-05T19:41:33.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.708+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5b4014040 con 0x563c91978330 2023-10-05T19:41:33.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.708+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1310 con 0x563c919aa0b0 2023-10-05T19:41:33.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.708+0000 7fd5c12b7700 1 -- 172.21.15.12:0/2345853771 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(18 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5bc024020 con 0x563c919aa0b0 2023-10-05T19:41:33.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.708+0000 7fd5c12b7700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1310 con 0x563c919ae850 2023-10-05T19:41:33.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.712+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/2345853771 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5b8008a80 con 0x563c919ae850 2023-10-05T19:41:33.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.712+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1310 con 0x563c919ae850 2023-10-05T19:41:33.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.712+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/2345853771 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5b8008a80 con 0x563c919ae850 2023-10-05T19:41:33.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.712+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1310 con 0x7fd5bc027be0 2023-10-05T19:41:33.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.712+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5b4014040 con 0x7fd5bc027be0 2023-10-05T19:41:33.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.712+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1310 con 0x563c919ae850 2023-10-05T19:41:33.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.712+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/2345853771 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5b8008a80 con 0x563c919ae850 2023-10-05T19:41:33.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.712+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1310 con 0x563c919ae850 2023-10-05T19:41:33.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.712+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/2345853771 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5b8008a80 con 0x563c919ae850 2023-10-05T19:41:33.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.712+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1310 con 0x563c91978330 2023-10-05T19:41:33.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.716+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5b4014040 con 0x563c91978330 2023-10-05T19:41:33.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.716+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1310 con 0x563c91978330 2023-10-05T19:41:33.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.716+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5b4014040 con 0x563c91978330 2023-10-05T19:41:33.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.716+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1310 con 0x7fd5bc027be0 2023-10-05T19:41:33.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.716+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5b4014040 con 0x7fd5bc027be0 2023-10-05T19:41:33.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.716+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1310 con 0x563c919ae850 2023-10-05T19:41:33.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.716+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/2345853771 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5b8008a80 con 0x563c919ae850 2023-10-05T19:41:33.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.716+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1310 con 0x563c919aa0b0 2023-10-05T19:41:33.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.716+0000 7fd5c12b7700 1 -- 172.21.15.12:0/2345853771 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5bc024020 con 0x563c919aa0b0 2023-10-05T19:41:33.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.716+0000 7fd5c12b7700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1310 con 0x563c91978330 2023-10-05T19:41:33.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.720+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5b4014040 con 0x563c91978330 2023-10-05T19:41:33.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.720+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1310 con 0x7fd5bc027be0 2023-10-05T19:41:33.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.720+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5b4014040 con 0x7fd5bc027be0 2023-10-05T19:41:33.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.720+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1310 con 0x563c919ae850 2023-10-05T19:41:33.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.720+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/2345853771 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5b8008a80 con 0x563c919ae850 2023-10-05T19:41:33.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.720+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1310 con 0x7fd5bc027be0 2023-10-05T19:41:33.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.720+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5b4014040 con 0x7fd5bc027be0 2023-10-05T19:41:33.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.720+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1310 con 0x563c919aa0b0 2023-10-05T19:41:33.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.724+0000 7fd5c12b7700 1 -- 172.21.15.12:0/2345853771 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5bc024020 con 0x563c919aa0b0 2023-10-05T19:41:33.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.724+0000 7fd5c12b7700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1310 con 0x563c919ae850 2023-10-05T19:41:33.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.724+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/2345853771 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5b8008a80 con 0x563c919ae850 2023-10-05T19:41:33.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.724+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1310 con 0x7fd5bc027be0 2023-10-05T19:41:33.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.724+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5b4014040 con 0x7fd5bc027be0 2023-10-05T19:41:33.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.724+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1310 con 0x563c91978330 2023-10-05T19:41:33.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.724+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fd5b4014040 con 0x563c91978330 2023-10-05T19:41:33.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.724+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1310 con 0x563c919aa0b0 2023-10-05T19:41:33.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.728+0000 7fd5c12b7700 1 -- 172.21.15.12:0/2345853771 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(37 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fd5bc024020 con 0x563c919aa0b0 2023-10-05T19:41:33.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.728+0000 7fd5c3782e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:41:33.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.728+0000 7fd5c3782e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:41:33.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.728+0000 7fd5c3782e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:41:33.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.728+0000 7fd5c3782e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:41:33.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.728+0000 7fd5c3782e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:33.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.728+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919ad5a0 con 0x563c91978330 2023-10-05T19:41:33.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.728+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd5b4014040 con 0x563c91978330 2023-10-05T19:41:33.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.728+0000 7fd5c3782e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:33.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.728+0000 7fd5c3782e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:33.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.728+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919ad1d0 con 0x563c91978330 2023-10-05T19:41:33.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.728+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fd5b4014040 con 0x563c91978330 2023-10-05T19:41:33.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.728+0000 7fd5c3782e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:33.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.728+0000 7fd5c3782e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:33.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.728+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1780 con 0x563c919ae850 2023-10-05T19:41:33.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.728+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/2345853771 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fd5b8008a80 con 0x563c919ae850 2023-10-05T19:41:33.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.728+0000 7fd5c3782e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:41:33.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.728+0000 7fd5c3782e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:33.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.728+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1b30 con 0x563c919ae850 2023-10-05T19:41:33.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.728+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/2345853771 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7fd5b8008a80 con 0x563c919ae850 2023-10-05T19:41:33.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.728+0000 7fd5c3782e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:33.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.728+0000 7fd5c3782e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:33.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.728+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1ee0 con 0x563c91978330 2023-10-05T19:41:33.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.728+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd5b4014040 con 0x563c91978330 2023-10-05T19:41:33.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.732+0000 7fd5c3782e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:33.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.732+0000 7fd5c3782e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:33.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.732+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b22f0 con 0x563c91978330 2023-10-05T19:41:33.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.732+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fd5b4014040 con 0x563c91978330 2023-10-05T19:41:33.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.732+0000 7fd5c3782e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:33.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.732+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b22c0 con 0x563c919ae850 2023-10-05T19:41:33.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.732+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/2345853771 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(44 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fd5b8008a80 con 0x563c919ae850 2023-10-05T19:41:33.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.732+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b22c0 con 0x563c919aa0b0 2023-10-05T19:41:33.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.736+0000 7fd5c12b7700 1 -- 172.21.15.12:0/2345853771 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(45 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5bc024020 con 0x563c919aa0b0 2023-10-05T19:41:33.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.736+0000 7fd5c12b7700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b22c0 con 0x7fd5bc027be0 2023-10-05T19:41:33.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.736+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(46 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5b4014040 con 0x7fd5bc027be0 2023-10-05T19:41:33.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.736+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1ee0 con 0x563c91978330 2023-10-05T19:41:33.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.736+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(47 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5b4014040 con 0x563c91978330 2023-10-05T19:41:33.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.736+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1ee0 con 0x563c91978330 2023-10-05T19:41:33.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.736+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(48 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5b4014040 con 0x563c91978330 2023-10-05T19:41:33.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.736+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1ee0 con 0x563c91978330 2023-10-05T19:41:33.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.736+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(49 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5b4014040 con 0x563c91978330 2023-10-05T19:41:33.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.736+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1ee0 con 0x7fd5bc027be0 2023-10-05T19:41:33.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.740+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(50 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5b4014040 con 0x7fd5bc027be0 2023-10-05T19:41:33.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.740+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1ee0 con 0x563c919aa0b0 2023-10-05T19:41:33.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.740+0000 7fd5c12b7700 1 -- 172.21.15.12:0/2345853771 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(51 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5bc024020 con 0x563c919aa0b0 2023-10-05T19:41:33.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.740+0000 7fd5c12b7700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1ee0 con 0x563c919aa0b0 2023-10-05T19:41:33.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.740+0000 7fd5c12b7700 1 -- 172.21.15.12:0/2345853771 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(52 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5bc024020 con 0x563c919aa0b0 2023-10-05T19:41:33.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.740+0000 7fd5c12b7700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1ee0 con 0x563c91978330 2023-10-05T19:41:33.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.740+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(53 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5b4014040 con 0x563c91978330 2023-10-05T19:41:33.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.740+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1ee0 con 0x7fd5bc027be0 2023-10-05T19:41:33.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.744+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(54 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5b4014040 con 0x7fd5bc027be0 2023-10-05T19:41:33.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.744+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1ee0 con 0x563c91978330 2023-10-05T19:41:33.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.744+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(55 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5b4014040 con 0x563c91978330 2023-10-05T19:41:33.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.744+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1ee0 con 0x563c919aa0b0 2023-10-05T19:41:33.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.744+0000 7fd5c12b7700 1 -- 172.21.15.12:0/2345853771 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(56 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5bc024020 con 0x563c919aa0b0 2023-10-05T19:41:33.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.744+0000 7fd5c12b7700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1ee0 con 0x563c919ae850 2023-10-05T19:41:33.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.744+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/2345853771 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(57 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5b8008a80 con 0x563c919ae850 2023-10-05T19:41:33.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.744+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1ee0 con 0x563c919ae850 2023-10-05T19:41:33.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.744+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/2345853771 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(58 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5b8008a80 con 0x563c919ae850 2023-10-05T19:41:33.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.744+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1ee0 con 0x7fd5bc027be0 2023-10-05T19:41:33.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.748+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(59 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5b4014040 con 0x7fd5bc027be0 2023-10-05T19:41:33.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.748+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1ee0 con 0x563c919ae850 2023-10-05T19:41:33.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.748+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/2345853771 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(60 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5b8008a80 con 0x563c919ae850 2023-10-05T19:41:33.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.748+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1ee0 con 0x563c919ae850 2023-10-05T19:41:33.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.748+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/2345853771 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(61 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5b8008a80 con 0x563c919ae850 2023-10-05T19:41:33.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.748+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1ee0 con 0x563c91978330 2023-10-05T19:41:33.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.748+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(62 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5b4014040 con 0x563c91978330 2023-10-05T19:41:33.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.748+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1ee0 con 0x563c91978330 2023-10-05T19:41:33.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.748+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(63 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5b4014040 con 0x563c91978330 2023-10-05T19:41:33.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.748+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1ee0 con 0x7fd5bc027be0 2023-10-05T19:41:33.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.752+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(64 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5b4014040 con 0x7fd5bc027be0 2023-10-05T19:41:33.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.752+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1ee0 con 0x563c919ae850 2023-10-05T19:41:33.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.752+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/2345853771 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(65 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5b8008a80 con 0x563c919ae850 2023-10-05T19:41:33.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.752+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1ee0 con 0x563c919aa0b0 2023-10-05T19:41:33.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.752+0000 7fd5c12b7700 1 -- 172.21.15.12:0/2345853771 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(66 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5bc024020 con 0x563c919aa0b0 2023-10-05T19:41:33.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.752+0000 7fd5c12b7700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1ee0 con 0x563c91978330 2023-10-05T19:41:33.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.752+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(67 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5b4014040 con 0x563c91978330 2023-10-05T19:41:33.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.752+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1ee0 con 0x7fd5bc027be0 2023-10-05T19:41:33.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.752+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(68 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5b4014040 con 0x7fd5bc027be0 2023-10-05T19:41:33.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.752+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1ee0 con 0x563c919ae850 2023-10-05T19:41:33.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.756+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/2345853771 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(69 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5b8008a80 con 0x563c919ae850 2023-10-05T19:41:33.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.756+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1ee0 con 0x7fd5bc027be0 2023-10-05T19:41:33.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.756+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(70 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5b4014040 con 0x7fd5bc027be0 2023-10-05T19:41:33.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.756+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1ee0 con 0x563c919aa0b0 2023-10-05T19:41:33.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.756+0000 7fd5c12b7700 1 -- 172.21.15.12:0/2345853771 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(71 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5bc024020 con 0x563c919aa0b0 2023-10-05T19:41:33.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.756+0000 7fd5c12b7700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1ee0 con 0x563c919ae850 2023-10-05T19:41:33.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.756+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/2345853771 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(72 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5b8008a80 con 0x563c919ae850 2023-10-05T19:41:33.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.756+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1ee0 con 0x7fd5bc027be0 2023-10-05T19:41:33.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.760+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(73 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd5b4014040 con 0x7fd5bc027be0 2023-10-05T19:41:33.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.760+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1ee0 con 0x563c91978330 2023-10-05T19:41:33.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.760+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(74 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fd5b4014040 con 0x563c91978330 2023-10-05T19:41:33.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.760+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b1ee0 con 0x563c919aa0b0 2023-10-05T19:41:33.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.760+0000 7fd5c12b7700 1 -- 172.21.15.12:0/2345853771 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(75 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fd5bc024020 con 0x563c919aa0b0 2023-10-05T19:41:33.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.760+0000 7fd5c3782e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:41:33.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.760+0000 7fd5c3782e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:41:33.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.760+0000 7fd5c3782e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:41:33.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.760+0000 7fd5c3782e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:41:33.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.760+0000 7fd5c3782e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:33.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.760+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b22c0 con 0x563c91978330 2023-10-05T19:41:33.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.760+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fd5b4014040 con 0x563c91978330 2023-10-05T19:41:33.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.760+0000 7fd5c3782e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:33.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.760+0000 7fd5c3782e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:33.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.760+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b2720 con 0x563c919aa0b0 2023-10-05T19:41:33.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.760+0000 7fd5c12b7700 1 -- 172.21.15.12:0/2345853771 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fd5bc024020 con 0x563c919aa0b0 2023-10-05T19:41:33.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.760+0000 7fd5c3782e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:41:33.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.760+0000 7fd5c3782e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:33.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.760+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b3010 con 0x563c91978330 2023-10-05T19:41:33.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.764+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fd5b4014040 con 0x563c91978330 2023-10-05T19:41:33.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.764+0000 7fd5c3782e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:33.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.764+0000 7fd5c3782e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:33.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.764+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b3010 con 0x563c919ae850 2023-10-05T19:41:33.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.764+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/2345853771 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fd5b8008a80 con 0x563c919ae850 2023-10-05T19:41:33.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.764+0000 7fd5c3782e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:41:33.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.764+0000 7fd5c3782e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:33.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.764+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b3780 con 0x7fd5bc027be0 2023-10-05T19:41:33.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.764+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7fd5b4014040 con 0x7fd5bc027be0 2023-10-05T19:41:33.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.764+0000 7fd5c3782e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:33.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.764+0000 7fd5c3782e40 20 zone default found 2023-10-05T19:41:33.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.764+0000 7fd5c3782e40 4 Realm: () 2023-10-05T19:41:33.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.764+0000 7fd5c3782e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:41:33.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.764+0000 7fd5c3782e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:41:33.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.764+0000 7fd5c3782e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:41:33.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.764+0000 7fd5c3782e40 20 zonegroup default 2023-10-05T19:41:33.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.764+0000 7fd5c3782e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:33.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.764+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b3780 con 0x7fd5bc027be0 2023-10-05T19:41:33.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.764+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fd5b4014040 con 0x7fd5bc027be0 2023-10-05T19:41:33.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.764+0000 7fd5c3782e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:33.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.764+0000 7fd5c3782e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:33.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.764+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b5100 con 0x563c919ae850 2023-10-05T19:41:33.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.768+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/2345853771 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fd5b8008a80 con 0x563c919ae850 2023-10-05T19:41:33.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.768+0000 7fd5c3782e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:33.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.768+0000 7fd5c3782e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:33.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.768+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b54b0 con 0x563c919ae850 2023-10-05T19:41:33.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.768+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/2345853771 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fd5b8008a80 con 0x563c919ae850 2023-10-05T19:41:33.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.768+0000 7fd5c3782e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:33.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.768+0000 7fd5c3782e40 20 started sync module instance, tier type = 2023-10-05T19:41:33.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.768+0000 7fd5c3782e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:41:33.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.768+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b47c0 con 0x563c919aa0b0 2023-10-05T19:41:33.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.768+0000 7fd5c12b7700 1 -- 172.21.15.12:0/2345853771 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(84 notify.0 [create] v30'246 uv246 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd5bc024020 con 0x563c919aa0b0 2023-10-05T19:41:33.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.768+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94818140891936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b5860 con 0x563c919aa0b0 2023-10-05T19:41:33.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.768+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b6100 con 0x563c91978330 2023-10-05T19:41:33.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.772+0000 7fd5c12b7700 1 -- 172.21.15.12:0/2345853771 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(85 notify.0 [watch watch cookie 94818140891936] v30'247 uv246 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd5bc024020 con 0x563c919aa0b0 2023-10-05T19:41:33.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.772+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(86 notify.1 [create] v30'246 uv246 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd5b4014040 con 0x563c91978330 2023-10-05T19:41:33.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.772+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94818140904464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b6060 con 0x563c91978330 2023-10-05T19:41:33.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.772+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b6ee0 con 0x563c919ae850 2023-10-05T19:41:33.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.776+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(87 notify.1 [watch watch cookie 94818140904464] v30'247 uv246 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd5b4014040 con 0x563c91978330 2023-10-05T19:41:33.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.776+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/2345853771 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(88 notify.2 [create] v30'246 uv246 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd5b8008a80 con 0x563c919ae850 2023-10-05T19:41:33.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.776+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94818140906752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b7290 con 0x563c919ae850 2023-10-05T19:41:33.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.776+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b7b30 con 0x563c919aa0b0 2023-10-05T19:41:33.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.780+0000 7fd5c12b7700 1 -- 172.21.15.12:0/2345853771 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(90 notify.3 [create] v30'246 uv246 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd5bc024020 con 0x563c919aa0b0 2023-10-05T19:41:33.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.780+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94818140911456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b8120 con 0x563c919aa0b0 2023-10-05T19:41:33.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.780+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b8c90 con 0x563c91978330 2023-10-05T19:41:33.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.780+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/2345853771 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(89 notify.2 [watch watch cookie 94818140906752] v30'247 uv246 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd5b8008a80 con 0x563c919ae850 2023-10-05T19:41:33.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.784+0000 7fd5c12b7700 1 -- 172.21.15.12:0/2345853771 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(91 notify.3 [watch watch cookie 94818140911456] v30'247 uv246 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd5bc024020 con 0x563c919aa0b0 2023-10-05T19:41:33.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.784+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(92 notify.4 [create] v30'246 uv246 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd5b4014040 con 0x563c91978330 2023-10-05T19:41:33.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.784+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94818140907856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b8760 con 0x563c91978330 2023-10-05T19:41:33.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.784+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b9580 con 0x7fd5bc027be0 2023-10-05T19:41:33.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.784+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(93 notify.4 [watch watch cookie 94818140907856] v30'247 uv246 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd5b4014040 con 0x563c91978330 2023-10-05T19:41:33.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.784+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(94 notify.5 [create] v30'246 uv246 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd5b4014040 con 0x7fd5bc027be0 2023-10-05T19:41:33.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.784+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94818140912560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b9280 con 0x7fd5bc027be0 2023-10-05T19:41:33.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.784+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b9b20 con 0x7fd5bc027be0 2023-10-05T19:41:33.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.788+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(96 notify.6 [create] v30'246 uv246 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd5b4014040 con 0x7fd5bc027be0 2023-10-05T19:41:33.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.788+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94818140918672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b9e20 con 0x7fd5bc027be0 2023-10-05T19:41:33.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.788+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919ba980 con 0x7fd5bc027be0 2023-10-05T19:41:33.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.788+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(95 notify.5 [watch watch cookie 94818140912560] v30'247 uv246 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd5b4014040 con 0x7fd5bc027be0 2023-10-05T19:41:33.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.792+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(97 notify.6 [watch watch cookie 94818140918672] v30'247 uv246 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd5b4014040 con 0x7fd5bc027be0 2023-10-05T19:41:33.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.792+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(98 notify.7 [create] v30'246 uv246 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd5b4014040 con 0x7fd5bc027be0 2023-10-05T19:41:33.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.792+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94818140917136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919ba450 con 0x7fd5bc027be0 2023-10-05T19:41:33.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.792+0000 7fd5c3782e40 20 add_watcher() i=0 2023-10-05T19:41:33.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.792+0000 7fd5c3782e40 20 add_watcher() i=1 2023-10-05T19:41:33.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.792+0000 7fd5c3782e40 20 add_watcher() i=2 2023-10-05T19:41:33.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.792+0000 7fd5c3782e40 20 add_watcher() i=3 2023-10-05T19:41:33.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.792+0000 7fd5c3782e40 20 add_watcher() i=4 2023-10-05T19:41:33.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.792+0000 7fd5c3782e40 20 add_watcher() i=5 2023-10-05T19:41:33.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.792+0000 7fd5c3782e40 20 add_watcher() i=6 2023-10-05T19:41:33.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.792+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(99 notify.7 [watch watch cookie 94818140917136] v30'247 uv246 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd5b4014040 con 0x7fd5bc027be0 2023-10-05T19:41:33.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.792+0000 7fd5c3782e40 20 add_watcher() i=7 2023-10-05T19:41:33.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.792+0000 7fd5c3782e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:41:33.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.792+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919bd970 con 0x563c91978330 2023-10-05T19:41:33.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.796+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fd5b4014040 con 0x563c91978330 2023-10-05T19:41:33.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.796+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94818140922960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919bad90 con 0x563c91978330 2023-10-05T19:41:33.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.796+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94818140922960] v30'235 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fd5b4014040 con 0x563c91978330 2023-10-05T19:41:33.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.796+0000 7fd5927e4700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:41:33.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.796+0000 7fd5c3782e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:41:33.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.796+0000 7fd5c3782e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:41:33.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.800+0000 7fd5c3782e40 1 Processor -- start 2023-10-05T19:41:33.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.800+0000 7fd5c3782e40 1 -- start start 2023-10-05T19:41:33.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.800+0000 7fd5c3782e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c919c7de0 0x563c919c81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:33.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.800+0000 7fd5c3782e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c919c86f0 con 0x563c919c7de0 2023-10-05T19:41:33.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.800+0000 7fd5c0ab6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c919c7de0 0x563c919c81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:33.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.800+0000 7fd5c0ab6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c919c7de0 0x563c919c81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:60774/0 (socket says 172.21.15.12:60774) 2023-10-05T19:41:33.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.800+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/268024989 learned_addr learned my addr 172.21.15.12:0/268024989 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:33.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.800+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/268024989 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c919c8830 con 0x563c919c7de0 2023-10-05T19:41:33.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.800+0000 7fd5c0ab6700 1 --2- 172.21.15.12:0/268024989 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c919c7de0 0x563c919c81b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x563c918e7ca0 tx=0x7fd5b8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c634bc6e47b0ae2 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:33.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.800+0000 7fd591fe3700 1 -- 172.21.15.12:0/268024989 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd5b8030930 con 0x563c919c7de0 2023-10-05T19:41:33.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.800+0000 7fd591fe3700 1 -- 172.21.15.12:0/268024989 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b8030a90 con 0x563c919c7de0 2023-10-05T19:41:33.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.800+0000 7fd591fe3700 1 -- 172.21.15.12:0/268024989 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd5b8030c40 con 0x563c919c7de0 2023-10-05T19:41:33.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.800+0000 7fd5c3782e40 1 -- 172.21.15.12:0/268024989 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c919c7de0 msgr2=0x563c919c81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:33.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.800+0000 7fd5c3782e40 1 --2- 172.21.15.12:0/268024989 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c919c7de0 0x563c919c81b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x563c918e7ca0 tx=0x7fd5b8012000 comp rx=0 tx=0).stop 2023-10-05T19:41:33.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.800+0000 7fd5c3782e40 1 -- 172.21.15.12:0/268024989 shutdown_connections 2023-10-05T19:41:33.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.800+0000 7fd5c3782e40 1 --2- 172.21.15.12:0/268024989 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c919c7de0 0x563c919c81b0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:33.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.800+0000 7fd5c3782e40 1 -- 172.21.15.12:0/268024989 >> 172.21.15.12:0/268024989 conn(0x563c919c31a0 msgr2=0x563c919c12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:33.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.800+0000 7fd5c3782e40 1 -- 172.21.15.12:0/268024989 shutdown_connections 2023-10-05T19:41:33.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.800+0000 7fd5c3782e40 1 -- 172.21.15.12:0/268024989 wait complete. 2023-10-05T19:41:33.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.804+0000 7fd5c3782e40 1 Processor -- start 2023-10-05T19:41:33.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.804+0000 7fd5c3782e40 1 -- start start 2023-10-05T19:41:33.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.804+0000 7fd5c3782e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c919c7de0 0x563c919cd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:33.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.804+0000 7fd5c3782e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c919ce880 con 0x563c919c7de0 2023-10-05T19:41:33.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.804+0000 7fd5c0ab6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c919c7de0 0x563c919cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:33.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.804+0000 7fd5c0ab6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c919c7de0 0x563c919cd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:60788/0 (socket says 172.21.15.12:60788) 2023-10-05T19:41:33.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.804+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/1892840556 learned_addr learned my addr 172.21.15.12:0/1892840556 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:33.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.804+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/1892840556 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c9197bc60 con 0x563c919c7de0 2023-10-05T19:41:33.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.804+0000 7fd5c0ab6700 1 --2- 172.21.15.12:0/1892840556 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c919c7de0 0x563c919cd880 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x563c919c1e60 tx=0x7fd5b8033000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:33.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.804+0000 7fd503fff700 1 -- 172.21.15.12:0/1892840556 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd5b8027610 con 0x563c919c7de0 2023-10-05T19:41:33.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.804+0000 7fd503fff700 1 -- 172.21.15.12:0/1892840556 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b8027770 con 0x563c919c7de0 2023-10-05T19:41:33.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.804+0000 7fd5c3782e40 1 -- 172.21.15.12:0/1892840556 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563c919cde80 con 0x563c919c7de0 2023-10-05T19:41:33.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.804+0000 7fd503fff700 1 -- 172.21.15.12:0/1892840556 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd5b8027920 con 0x563c919c7de0 2023-10-05T19:41:33.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.804+0000 7fd5c3782e40 1 -- 172.21.15.12:0/1892840556 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563c919ce2c0 con 0x563c919c7de0 2023-10-05T19:41:33.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.808+0000 7fd503fff700 1 -- 172.21.15.12:0/1892840556 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fd5b8036040 con 0x563c919c7de0 2023-10-05T19:41:33.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.808+0000 7fd503fff700 1 --2- 172.21.15.12:0/1892840556 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd508030c50 0x7fd508033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:33.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.808+0000 7fd503fff700 1 -- 172.21.15.12:0/1892840556 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7fd5b804abd0 con 0x563c919c7de0 2023-10-05T19:41:33.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.808+0000 7fd5c1ab8700 1 --2- 172.21.15.12:0/1892840556 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd508030c50 0x7fd508033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:33.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.808+0000 7fd5c3782e40 5 note: GC not initialized 2023-10-05T19:41:33.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.808+0000 7fd5c1ab8700 1 --2- 172.21.15.12:0/1892840556 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd508030c50 0x7fd508033100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x563c919b9af0 tx=0x7fd5b40325e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:33.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.808+0000 7fd5017fa700 20 reqs_thread_entry: start 2023-10-05T19:41:33.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.812+0000 7fd4faffd700 20 reqs_thread_entry: start 2023-10-05T19:41:33.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.812+0000 7fd5c3782e40 20 init_complete bucket index max shards: 11 2023-10-05T19:41:33.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.812+0000 7fd5c3782e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:41:33.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.812+0000 7fd4f97fa700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:42:04 2023 2023-10-05T19:41:33.817 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:41:33.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.812+0000 7fd5c3782e40 20 get_system_obj_state: rctx=0x7ffd724cf398 obj=default.rgw.meta:users.uid:fud state=0x563c9184d0b0 s->prefetch_data=0 2023-10-05T19:41:33.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.812+0000 7fd5c3782e40 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2023-10-05T19:41:33.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.812+0000 7fd4f8ff9700 20 reqs_thread_entry: start 2023-10-05T19:41:33.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.812+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:102 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c91a360d0 con 0x7fd5bc027be0 2023-10-05T19:41:33.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.812+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(102 fud [call,getxattrs,stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x7fd5b4014040 con 0x7fd5bc027be0 2023-10-05T19:41:33.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.812+0000 7fd5c3782e40 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x0 2023-10-05T19:41:33.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.812+0000 7fd5c3782e40 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2023-10-05T19:41:33.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.812+0000 7fd5c3782e40 20 get_system_obj_state: rctx=0x7ffd724cf088 obj=default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ state=0x563c9184d0b0 s->prefetch_data=0 2023-10-05T19:41:33.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.812+0000 7fd5c3782e40 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : miss 2023-10-05T19:41:33.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.812+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:103 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c91a360d0 con 0x7fd5bc027be0 2023-10-05T19:41:33.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.812+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(103 p5YnriCv1nAtykxBrupQ [getxattrs,stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 206+0+0 (crc 0 0 0) 0x7fd5b4014040 con 0x7fd5bc027be0 2023-10-05T19:41:33.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.812+0000 7fd5c3782e40 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x0 2023-10-05T19:41:33.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.812+0000 7fd5c3782e40 10 adding default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2023-10-05T19:41:33.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.812+0000 7fd5c3782e40 20 get_system_obj_state: rctx=0x7ffd724ce928 obj=default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ state=0x563c9184d0b0 s->prefetch_data=0 2023-10-05T19:41:33.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.812+0000 7fd5c3782e40 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2023-10-05T19:41:33.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.812+0000 7fd5c3782e40 20 get_system_obj_state: rctx=0x7ffd724cefc8 obj=default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ state=0x563c9184d0b0 s->prefetch_data=0 2023-10-05T19:41:33.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.812+0000 7fd5c3782e40 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2023-10-05T19:41:33.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.812+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:104 8.e 8:75f48285:users.uid::fud:head [delete,create,call version.set in=58b,writefull 0~346 in=346b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c91a38670 con 0x7fd5bc027be0 2023-10-05T19:41:33.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.816+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(104 fud [delete,create,call,writefull 0~346] v30'14 uv14 ondisk = 0) v8 ==== 273+0+0 (crc 0 0 0) 0x7fd5b4014040 con 0x7fd5bc027be0 2023-10-05T19:41:33.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.816+0000 7fd5c3782e40 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x17 2023-10-05T19:41:33.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.816+0000 7fd5c3782e40 10 moving default.rgw.meta+users.uid+fud to cache LRU end 2023-10-05T19:41:33.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.816+0000 7fd5c3782e40 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:fud, ofs0, ns] 2023-10-05T19:41:33.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.816+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:105 7.3 7:c609908c:::notify.5:head [notify cookie 94818141435488 in=534b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c91a37fe0 con 0x7fd5bc027be0 2023-10-05T19:41:33.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.816+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== watch-notify(notify (1) cookie 94818140912560 notify 128849018914 ret 0) v3 ==== 564+0+0 (crc 0 0 0) 0x7fd5b40266e0 con 0x7fd5bc027be0 2023-10-05T19:41:33.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.816+0000 7fd5b2ffd700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018914 cookie 94818140912560 notifier 4717 bl.length()=522 2023-10-05T19:41:33.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.816+0000 7fd5b2ffd700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+fud info.flags=0x17 2023-10-05T19:41:33.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.816+0000 7fd5b2ffd700 10 rgw watcher librados: moving default.rgw.meta+users.uid+fud to cache LRU end 2023-10-05T19:41:33.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.816+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(105 notify.5 [notify cookie 94818141435488 out=8b] v0'0 uv246 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fd5b4014040 con 0x7fd5bc027be0 2023-10-05T19:41:33.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.820+0000 7fd5b2ffd700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:106 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x7fd5a0003d90 con 0x7fd5bc027be0 2023-10-05T19:41:33.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.820+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.0 v2:172.21.15.8:6800/27563 30 ==== watch-notify(notify_complete (2) cookie 94818141435488 notify 128849018914 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7fd5b4026870 con 0x7fd5bc027be0 2023-10-05T19:41:33.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.820+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.0 v2:172.21.15.8:6800/27563 31 ==== osd_op_reply(106 notify.5 [notify-ack] v0'0 uv246 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd5b4014040 con 0x7fd5bc027be0 2023-10-05T19:41:33.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.820+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:107 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c91a36650 con 0x7fd5bc027be0 2023-10-05T19:41:33.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.824+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.0 v2:172.21.15.8:6800/27563 32 ==== osd_op_reply(107 p5YnriCv1nAtykxBrupQ [delete,create,writefull 0~7] v30'9 uv9 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x7fd5b4014040 con 0x7fd5bc027be0 2023-10-05T19:41:33.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.824+0000 7fd5c3782e40 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2023-10-05T19:41:33.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.824+0000 7fd5c3782e40 10 moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2023-10-05T19:41:33.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.824+0000 7fd5c3782e40 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2023-10-05T19:41:33.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.824+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:108 7.15 7:a93a5511:::notify.2:head [notify cookie 94818141439040 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c91a37980 con 0x563c919ae850 2023-10-05T19:41:33.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.824+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/2345853771 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== watch-notify(notify (1) cookie 94818140906752 notify 128849018890 ret 0) v3 ==== 219+0+0 (crc 0 0 0) 0x563c919ce2c0 con 0x563c919ae850 2023-10-05T19:41:33.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.824+0000 7fd5b27fc700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018890 cookie 94818140906752 notifier 4717 bl.length()=177 2023-10-05T19:41:33.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.824+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/2345853771 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(108 notify.2 [notify cookie 94818141439040 out=8b] v0'0 uv246 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fd5b8008a80 con 0x563c919ae850 2023-10-05T19:41:33.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.824+0000 7fd5b27fc700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2023-10-05T19:41:33.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.824+0000 7fd5b27fc700 10 rgw watcher librados: moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2023-10-05T19:41:33.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.824+0000 7fd5b27fc700 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:109 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x7fd510003450 con 0x563c919ae850 2023-10-05T19:41:33.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.824+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/2345853771 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== watch-notify(notify_complete (2) cookie 94818141439040 notify 128849018890 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x563c919cde80 con 0x563c919ae850 2023-10-05T19:41:33.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.824+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/2345853771 <== osd.3 v2:172.21.15.12:6816/29951 27 ==== osd_op_reply(109 notify.2 [notify-ack] v0'0 uv246 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd5b8008a80 con 0x563c919ae850 2023-10-05T19:41:33.831 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:41:33.832 INFO:teuthology.orchestra.run.smithi012.stdout: "user_id": "fud", 2023-10-05T19:41:33.832 INFO:teuthology.orchestra.run.smithi012.stdout: "display_name": "Fud", 2023-10-05T19:41:33.832 INFO:teuthology.orchestra.run.smithi012.stdout: "email": "", 2023-10-05T19:41:33.832 INFO:teuthology.orchestra.run.smithi012.stdout: "suspended": 0, 2023-10-05T19:41:33.832 INFO:teuthology.orchestra.run.smithi012.stdout: "max_buckets": 4, 2023-10-05T19:41:33.832 INFO:teuthology.orchestra.run.smithi012.stdout: "subusers": [], 2023-10-05T19:41:33.832 INFO:teuthology.orchestra.run.smithi012.stdout: "keys": [ 2023-10-05T19:41:33.832 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:41:33.832 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "fud", 2023-10-05T19:41:33.832 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key": "p5YnriCv1nAtykxBrupQ", 2023-10-05T19:41:33.832 INFO:teuthology.orchestra.run.smithi012.stdout: "secret_key": "Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh" 2023-10-05T19:41:33.833 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:41:33.833 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:41:33.833 INFO:teuthology.orchestra.run.smithi012.stdout: "swift_keys": [], 2023-10-05T19:41:33.833 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": [], 2023-10-05T19:41:33.833 INFO:teuthology.orchestra.run.smithi012.stdout: "op_mask": "read, write, delete", 2023-10-05T19:41:33.833 INFO:teuthology.orchestra.run.smithi012.stdout: "default_placement": "", 2023-10-05T19:41:33.833 INFO:teuthology.orchestra.run.smithi012.stdout: "default_storage_class": "", 2023-10-05T19:41:33.833 INFO:teuthology.orchestra.run.smithi012.stdout: "placement_tags": [], 2023-10-05T19:41:33.833 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket_quota": { 2023-10-05T19:41:33.833 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:41:33.833 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:41:33.833 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:41:33.834 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:41:33.834 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:41:33.834 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:33.834 INFO:teuthology.orchestra.run.smithi012.stdout: "user_quota": { 2023-10-05T19:41:33.834 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:41:33.834 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:41:33.834 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:41:33.834 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:41:33.834 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:41:33.834 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:41:33.834 INFO:teuthology.orchestra.run.smithi012.stdout: "temp_url_keys": [], 2023-10-05T19:41:33.835 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "rgw", 2023-10-05T19:41:33.835 INFO:teuthology.orchestra.run.smithi012.stdout: "mfa_ids": [] 2023-10-05T19:41:33.835 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:41:33.835 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:41:33.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.824+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:110 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94818140891936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919898d0 con 0x563c919aa0b0 2023-10-05T19:41:33.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.828+0000 7fd5c12b7700 1 -- 172.21.15.12:0/2345853771 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(110 notify.0 [watch unwatch cookie 94818140891936] v30'248 uv246 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd5bc024020 con 0x563c919aa0b0 2023-10-05T19:41:33.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.828+0000 7fd5c3782e40 20 remove_watcher() i=0 2023-10-05T19:41:33.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.828+0000 7fd5c3782e40 2 removed watcher, disabling cache 2023-10-05T19:41:33.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.828+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:111 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94818140904464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b2b20 con 0x563c91978330 2023-10-05T19:41:33.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.832+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(111 notify.1 [watch unwatch cookie 94818140904464] v30'248 uv246 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd5b4014040 con 0x563c91978330 2023-10-05T19:41:33.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.832+0000 7fd5c3782e40 20 remove_watcher() i=1 2023-10-05T19:41:33.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.832+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:112 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94818140906752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b2b20 con 0x563c919ae850 2023-10-05T19:41:33.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.836+0000 7fd5c0ab6700 1 -- 172.21.15.12:0/2345853771 <== osd.3 v2:172.21.15.12:6816/29951 28 ==== osd_op_reply(112 notify.2 [watch unwatch cookie 94818140906752] v30'248 uv246 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd5b8008a80 con 0x563c919ae850 2023-10-05T19:41:33.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.836+0000 7fd5c3782e40 20 remove_watcher() i=2 2023-10-05T19:41:33.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.836+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:113 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94818140911456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b2b20 con 0x563c919aa0b0 2023-10-05T19:41:33.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.836+0000 7fd5c12b7700 1 -- 172.21.15.12:0/2345853771 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(113 notify.3 [watch unwatch cookie 94818140911456] v30'248 uv246 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd5bc024020 con 0x563c919aa0b0 2023-10-05T19:41:33.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.836+0000 7fd5c3782e40 20 remove_watcher() i=3 2023-10-05T19:41:33.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.836+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:114 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94818140907856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b7760 con 0x563c91978330 2023-10-05T19:41:33.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.840+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(114 notify.4 [watch unwatch cookie 94818140907856] v30'248 uv246 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd5b4014040 con 0x563c91978330 2023-10-05T19:41:33.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.840+0000 7fd5c3782e40 20 remove_watcher() i=4 2023-10-05T19:41:33.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.840+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:115 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94818140912560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b6980 con 0x7fd5bc027be0 2023-10-05T19:41:33.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.844+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.0 v2:172.21.15.8:6800/27563 33 ==== osd_op_reply(115 notify.5 [watch unwatch cookie 94818140912560] v30'248 uv246 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd5b4014040 con 0x7fd5bc027be0 2023-10-05T19:41:33.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.844+0000 7fd5c3782e40 20 remove_watcher() i=5 2023-10-05T19:41:33.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.844+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:116 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94818140918672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b7c50 con 0x7fd5bc027be0 2023-10-05T19:41:33.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.848+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.0 v2:172.21.15.8:6800/27563 34 ==== osd_op_reply(116 notify.6 [watch unwatch cookie 94818140918672] v30'248 uv246 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd5b4014040 con 0x7fd5bc027be0 2023-10-05T19:41:33.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.848+0000 7fd5c3782e40 20 remove_watcher() i=6 2023-10-05T19:41:33.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.848+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:117 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94818140917136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b7c50 con 0x7fd5bc027be0 2023-10-05T19:41:33.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.852+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.0 v2:172.21.15.8:6800/27563 35 ==== osd_op_reply(117 notify.7 [watch unwatch cookie 94818140917136] v30'248 uv246 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd5b4014040 con 0x7fd5bc027be0 2023-10-05T19:41:33.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.852+0000 7fd5c3782e40 20 remove_watcher() i=7 2023-10-05T19:41:33.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.852+0000 7fd5927e4700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:41:33.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.852+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:118 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94818140922960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563c919b7c50 con 0x563c91978330 2023-10-05T19:41:33.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.856+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(118 data_loggenerations_metadata [watch unwatch cookie 94818140922960] v30'236 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fd5b4014040 con 0x563c91978330 2023-10-05T19:41:33.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.856+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fd5bc027be0 msgr2=0x7fd5bc028460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:33.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.856+0000 7fd5c3782e40 1 --2- 172.21.15.12:0/2345853771 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fd5bc027be0 0x7fd5bc028460 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:33.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.856+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x563c91978330 msgr2=0x563c919a7b30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:33.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.856+0000 7fd5c3782e40 1 --2- 172.21.15.12:0/2345853771 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x563c91978330 0x563c919a7b30 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:33.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.856+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x563c919aa0b0 msgr2=0x563c919ac490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:33.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.856+0000 7fd5c3782e40 1 --2- 172.21.15.12:0/2345853771 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x563c919aa0b0 0x563c919ac490 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:33.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.856+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x563c919ae850 msgr2=0x563c919b0c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:33.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.856+0000 7fd5c3782e40 1 --2- 172.21.15.12:0/2345853771 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x563c919ae850 0x563c919b0c60 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:33.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.856+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd5a4030c50 msgr2=0x7fd5a4033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:33.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.856+0000 7fd5c3782e40 1 --2- 172.21.15.12:0/2345853771 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd5a4030c50 0x7fd5a4033100 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8005a90 tx=0x7fd5b8015000 comp rx=0 tx=0).stop 2023-10-05T19:41:33.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.856+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c918e8410 msgr2=0x563c9199c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:33.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.856+0000 7fd5c1ab8700 1 -- 172.21.15.12:0/2345853771 reap_dead start 2023-10-05T19:41:33.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.856+0000 7fd5c3782e40 1 --2- 172.21.15.12:0/2345853771 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c918e8410 0x563c9199c040 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc006bd0 tx=0x7fd5bc014000 comp rx=0 tx=0).stop 2023-10-05T19:41:33.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.856+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 shutdown_connections 2023-10-05T19:41:33.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.856+0000 7fd5c3782e40 1 --2- 172.21.15.12:0/2345853771 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c918e8410 0x563c9199c040 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:33.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.856+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 >> 172.21.15.12:0/2345853771 conn(0x563c9196e760 msgr2=0x563c9198d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:33.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.856+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 shutdown_connections 2023-10-05T19:41:33.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.856+0000 7fd5c3782e40 1 -- 172.21.15.12:0/2345853771 wait complete. 2023-10-05T19:41:33.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.856+0000 7fd5c3782e40 1 -- 172.21.15.12:0/1892840556 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd508030c50 msgr2=0x7fd508033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:33.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.856+0000 7fd5c3782e40 1 --2- 172.21.15.12:0/1892840556 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd508030c50 0x7fd508033100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x563c919b9af0 tx=0x7fd5b40325e0 comp rx=0 tx=0).stop 2023-10-05T19:41:33.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.856+0000 7fd5c3782e40 1 -- 172.21.15.12:0/1892840556 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c919c7de0 msgr2=0x563c919cd880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:33.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.856+0000 7fd5c3782e40 1 --2- 172.21.15.12:0/1892840556 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c919c7de0 0x563c919cd880 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x563c919c1e60 tx=0x7fd5b8033000 comp rx=0 tx=0).stop 2023-10-05T19:41:33.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.856+0000 7fd5c3782e40 1 -- 172.21.15.12:0/1892840556 shutdown_connections 2023-10-05T19:41:33.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.856+0000 7fd5c3782e40 1 --2- 172.21.15.12:0/1892840556 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd508030c50 0x7fd508033100 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:33.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.856+0000 7fd5c3782e40 1 --2- 172.21.15.12:0/1892840556 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563c919c7de0 0x563c919cd880 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:33.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.856+0000 7fd5c3782e40 1 -- 172.21.15.12:0/1892840556 >> 172.21.15.12:0/1892840556 conn(0x563c919c31a0 msgr2=0x563c919c1240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:33.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.856+0000 7fd5c3782e40 1 -- 172.21.15.12:0/1892840556 shutdown_connections 2023-10-05T19:41:33.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.856+0000 7fd5c3782e40 1 -- 172.21.15.12:0/1892840556 wait complete. 2023-10-05T19:41:33.869 DEBUG:tasks.util.rgw: json result: {'user_id': 'fud', 'display_name': 'Fud', 'email': '', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'fud', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:41:33.897 INFO:tasks.radosgw_admin:RL: create_bucket myfoo7 fud bytes_out=0 bytes_in=0 failed=False 2023-10-05T19:41:33.908 INFO:tasks.radosgw_admin:RL: put_obj myfoo7 fud bytes_out=8 bytes_in=0 failed=False 2023-10-05T19:41:33.908 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rename', '--uid', 'bud', '--new-uid', 'fud'] 2023-10-05T19:41:33.908 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rename', '--uid', 'bud', '--new-uid', 'fud'] 2023-10-05T19:41:33.909 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rename --uid bud --new-uid fud 2023-10-05T19:41:33.951 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:41:33.951 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:41:33.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.948+0000 7f5e50e56e40 1 Processor -- start 2023-10-05T19:41:33.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.948+0000 7f5e50e56e40 1 -- start start 2023-10-05T19:41:33.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.948+0000 7f5e50e56e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55a9aeada320 0x55a9aeadc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:33.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.948+0000 7f5e50e56e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55a9aea22bd0 con 0x55a9aea49b50 2023-10-05T19:41:33.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.948+0000 7f5e50e56e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x55a9ae9dfee0 con 0x55a9aeada320 2023-10-05T19:41:33.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.948+0000 7f5e4e18a700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55a9aeada320 0x55a9aeadc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:33.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.948+0000 7f5e4e98b700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x55a9aea49b50 0x55a9aead8c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:44322/0 (socket says 172.21.15.12:44322) 2023-10-05T19:41:33.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.948+0000 7f5e4e18a700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55a9aeada320 0x55a9aeadc700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:60790/0 (socket says 172.21.15.12:60790) 2023-10-05T19:41:33.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.948+0000 7f5e4e98b700 1 -- 172.21.15.12:0/3442753220 learned_addr learned my addr 172.21.15.12:0/3442753220 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:41:33.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.948+0000 7f5e4e18a700 1 -- 172.21.15.12:0/3442753220 >> v1:172.21.15.8:6789/0 conn(0x55a9aea49b50 legacy=0x55a9aead8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:33.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.948+0000 7f5e4e18a700 1 -- 172.21.15.12:0/3442753220 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9aeaddc50 con 0x55a9aeada320 2023-10-05T19:41:33.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.948+0000 7f5e4e18a700 1 --2- 172.21.15.12:0/3442753220 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a9aeada320 0x55a9aeadc700 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f5e4400a5d0 tx=0x7f5e44005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93cae3ef0b4b933a server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:33.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.948+0000 7f5e4d989700 1 -- 172.21.15.12:0/3442753220 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e440086a0 con 0x55a9aeada320 2023-10-05T19:41:33.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.948+0000 7f5e4d989700 1 -- 172.21.15.12:0/3442753220 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e44008800 con 0x55a9aeada320 2023-10-05T19:41:33.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.948+0000 7f5e4d989700 1 -- 172.21.15.12:0/3442753220 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e44008ad0 con 0x55a9aeada320 2023-10-05T19:41:33.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.948+0000 7f5e50e56e40 1 -- 172.21.15.12:0/3442753220 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a9aeada320 msgr2=0x55a9aeadc700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:33.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.948+0000 7f5e50e56e40 1 --2- 172.21.15.12:0/3442753220 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a9aeada320 0x55a9aeadc700 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f5e4400a5d0 tx=0x7f5e44005a50 comp rx=0 tx=0).stop 2023-10-05T19:41:33.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.948+0000 7f5e50e56e40 1 -- 172.21.15.12:0/3442753220 shutdown_connections 2023-10-05T19:41:33.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.948+0000 7f5e50e56e40 1 --2- 172.21.15.12:0/3442753220 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a9aeada320 0x55a9aeadc700 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:33.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.948+0000 7f5e50e56e40 1 -- 172.21.15.12:0/3442753220 >> 172.21.15.12:0/3442753220 conn(0x55a9aead0750 msgr2=0x55a9aead2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:33.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.948+0000 7f5e50e56e40 1 -- 172.21.15.12:0/3442753220 shutdown_connections 2023-10-05T19:41:33.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.948+0000 7f5e50e56e40 1 -- 172.21.15.12:0/3442753220 wait complete. 2023-10-05T19:41:33.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.952+0000 7f5e50e56e40 1 Processor -- start 2023-10-05T19:41:33.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.952+0000 7f5e50e56e40 1 -- start start 2023-10-05T19:41:33.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.952+0000 7f5e50e56e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a9aea49b50 0x55a9aead6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:33.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.952+0000 7f5e50e56e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9ae9a3e30 con 0x55a9aea49b50 2023-10-05T19:41:33.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.952+0000 7f5e4e98b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a9aea49b50 0x55a9aead6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:33.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.952+0000 7f5e4e98b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a9aea49b50 0x55a9aead6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:60798/0 (socket says 172.21.15.12:60798) 2023-10-05T19:41:33.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.952+0000 7f5e4e98b700 1 -- 172.21.15.12:0/2176832931 learned_addr learned my addr 172.21.15.12:0/2176832931 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:33.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.952+0000 7f5e4e98b700 1 -- 172.21.15.12:0/2176832931 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e44005cf0 con 0x55a9aea49b50 2023-10-05T19:41:33.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.952+0000 7f5e4e98b700 1 --2- 172.21.15.12:0/2176832931 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a9aea49b50 0x55a9aead6180 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f5e400014a0 tx=0x7f5e40006b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45d6acf68a9170fd server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:33.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.952+0000 7f5e4c987700 1 -- 172.21.15.12:0/2176832931 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e40006110 con 0x55a9aea49b50 2023-10-05T19:41:33.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.952+0000 7f5e4c987700 1 -- 172.21.15.12:0/2176832931 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e40006270 con 0x55a9aea49b50 2023-10-05T19:41:33.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.952+0000 7f5e4c987700 1 -- 172.21.15.12:0/2176832931 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e40004520 con 0x55a9aea49b50 2023-10-05T19:41:33.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.952+0000 7f5e50e56e40 1 -- 172.21.15.12:0/2176832931 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a9aea49b50 msgr2=0x55a9aead6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:33.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.952+0000 7f5e50e56e40 1 --2- 172.21.15.12:0/2176832931 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a9aea49b50 0x55a9aead6180 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f5e400014a0 tx=0x7f5e40006b90 comp rx=0 tx=0).stop 2023-10-05T19:41:33.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.952+0000 7f5e50e56e40 1 -- 172.21.15.12:0/2176832931 shutdown_connections 2023-10-05T19:41:33.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.952+0000 7f5e50e56e40 1 --2- 172.21.15.12:0/2176832931 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a9aea49b50 0x55a9aead6180 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:33.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.952+0000 7f5e50e56e40 1 -- 172.21.15.12:0/2176832931 >> 172.21.15.12:0/2176832931 conn(0x55a9aead0750 msgr2=0x55a9aeaf0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:33.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.952+0000 7f5e50e56e40 1 -- 172.21.15.12:0/2176832931 shutdown_connections 2023-10-05T19:41:33.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.952+0000 7f5e50e56e40 1 -- 172.21.15.12:0/2176832931 wait complete. 2023-10-05T19:41:33.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.952+0000 7f5e50e56e40 1 Processor -- start 2023-10-05T19:41:33.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.952+0000 7f5e50e56e40 1 -- start start 2023-10-05T19:41:33.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.952+0000 7f5e50e56e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a9aea49b50 0x55a9aeafdef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:33.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.952+0000 7f5e50e56e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9ae976ed0 con 0x55a9aea49b50 2023-10-05T19:41:33.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.952+0000 7f5e4e98b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a9aea49b50 0x55a9aeafdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:33.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.956+0000 7f5e4e98b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a9aea49b50 0x55a9aeafdef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:60800/0 (socket says 172.21.15.12:60800) 2023-10-05T19:41:33.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.956+0000 7f5e4e98b700 1 -- 172.21.15.12:0/488197165 learned_addr learned my addr 172.21.15.12:0/488197165 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:33.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.956+0000 7f5e4e98b700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9aea22bd0 con 0x55a9aea49b50 2023-10-05T19:41:33.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.956+0000 7f5e4e98b700 1 --2- 172.21.15.12:0/488197165 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a9aea49b50 0x55a9aeafdef0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f5e40006bd0 tx=0x7f5e40014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:33.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.956+0000 7f5e3effd700 1 -- 172.21.15.12:0/488197165 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e40013750 con 0x55a9aea49b50 2023-10-05T19:41:33.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.956+0000 7f5e3effd700 1 -- 172.21.15.12:0/488197165 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e400138b0 con 0x55a9aea49b50 2023-10-05T19:41:33.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.956+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a9aeafe430 con 0x55a9aea49b50 2023-10-05T19:41:33.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.956+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a9aeafe870 con 0x55a9aea49b50 2023-10-05T19:41:33.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.956+0000 7f5e3effd700 1 -- 172.21.15.12:0/488197165 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e40013b80 con 0x55a9aea49b50 2023-10-05T19:41:33.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.956+0000 7f5e50e56e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:33.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.956+0000 7f5e3effd700 1 -- 172.21.15.12:0/488197165 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f5e40013d30 con 0x55a9aea49b50 2023-10-05T19:41:33.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.956+0000 7f5e3effd700 1 --2- 172.21.15.12:0/488197165 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5e34030c50 0x7f5e34033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:33.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.956+0000 7f5e3effd700 1 -- 172.21.15.12:0/488197165 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7f5e40045050 con 0x55a9aea49b50 2023-10-05T19:41:33.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.960+0000 7f5e4e18a700 1 --2- 172.21.15.12:0/488197165 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5e34030c50 0x7f5e34033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:33.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.960+0000 7f5e50e56e40 1 --2- 172.21.15.12:0/488197165 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55a9aeada320 0x55a9aeb099e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:33.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.960+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb0a0e0 con 0x55a9aeada320 2023-10-05T19:41:33.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.960+0000 7f5e4f18c700 1 --2- 172.21.15.12:0/488197165 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55a9aeada320 0x55a9aeb099e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:33.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.960+0000 7f5e4e18a700 1 --2- 172.21.15.12:0/488197165 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5e34030c50 0x7f5e34033100 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5e44005a90 tx=0x7f5e44015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:33.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.960+0000 7f5e4f18c700 1 --2- 172.21.15.12:0/488197165 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55a9aeada320 0x55a9aeb099e0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:33.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.960+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5e4800a500 con 0x55a9aeada320 2023-10-05T19:41:33.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.960+0000 7f5e50e56e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:33.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.960+0000 7f5e50e56e40 20 realm 2023-10-05T19:41:33.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.960+0000 7f5e50e56e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:33.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.960+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb07040 con 0x55a9aeada320 2023-10-05T19:41:33.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.960+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5e4800a500 con 0x55a9aeada320 2023-10-05T19:41:33.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.960+0000 7f5e50e56e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:33.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.960+0000 7f5e50e56e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:41:33.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.960+0000 7f5e50e56e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:33.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.960+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb0abc0 con 0x55a9aeada320 2023-10-05T19:41:33.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.960+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5e4800a500 con 0x55a9aeada320 2023-10-05T19:41:33.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.960+0000 7f5e50e56e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:33.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.960+0000 7f5e50e56e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:33.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.960+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb0a7d0 con 0x55a9aeada320 2023-10-05T19:41:33.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.960+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f5e4800a500 con 0x55a9aeada320 2023-10-05T19:41:33.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.960+0000 7f5e50e56e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:33.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.960+0000 7f5e50e56e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:33.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.960+0000 7f5e50e56e40 1 --2- 172.21.15.12:0/488197165 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55a9aeb0bf60 0x55a9aeb0e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:33.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.960+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb0e880 con 0x55a9aeb0bf60 2023-10-05T19:41:33.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.960+0000 7f5e4e98b700 1 --2- 172.21.15.12:0/488197165 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55a9aeb0bf60 0x55a9aeb0e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:33.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.964+0000 7f5e4e98b700 1 --2- 172.21.15.12:0/488197165 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55a9aeb0bf60 0x55a9aeb0e340 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:33.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.964+0000 7f5e4e98b700 1 -- 172.21.15.12:0/488197165 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f5e40024020 con 0x55a9aeb0bf60 2023-10-05T19:41:33.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.964+0000 7f5e50e56e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:41:33.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.964+0000 7f5e50e56e40 20 searching for the correct realm 2023-10-05T19:41:33.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.964+0000 7f5e50e56e40 1 --2- 172.21.15.12:0/488197165 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55a9aeb10700 0x55a9aeb12b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:33.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.964+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb131c0 con 0x55a9aeb10700 2023-10-05T19:41:33.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.964+0000 7f5e4e18a700 1 --2- 172.21.15.12:0/488197165 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55a9aeb10700 0x55a9aeb12b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:33.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.964+0000 7f5e4e18a700 1 --2- 172.21.15.12:0/488197165 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55a9aeb10700 0x55a9aeb12b10 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:33.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.968+0000 7f5e4e18a700 1 -- 172.21.15.12:0/488197165 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f5e44008a80 con 0x55a9aeb10700 2023-10-05T19:41:33.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.968+0000 7f5e4e18a700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb131c0 con 0x55a9aeb0bf60 2023-10-05T19:41:33.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.968+0000 7f5e4e98b700 1 -- 172.21.15.12:0/488197165 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e40024020 con 0x55a9aeb0bf60 2023-10-05T19:41:33.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.968+0000 7f5e4e98b700 1 --2- 172.21.15.12:0/488197165 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f5e40027be0 0x7f5e40028460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:33.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.968+0000 7f5e4e98b700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb131c0 con 0x7f5e40027be0 2023-10-05T19:41:33.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.968+0000 7f5e4f18c700 1 --2- 172.21.15.12:0/488197165 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f5e40027be0 0x7f5e40028460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:33.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.968+0000 7f5e4f18c700 1 --2- 172.21.15.12:0/488197165 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f5e40027be0 0x7f5e40028460 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:33.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.972+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e48014040 con 0x7f5e40027be0 2023-10-05T19:41:33.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.972+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb131c0 con 0x55a9aeada320 2023-10-05T19:41:33.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.972+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e48014040 con 0x55a9aeada320 2023-10-05T19:41:33.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.972+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb131c0 con 0x55a9aeada320 2023-10-05T19:41:33.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.972+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e48014040 con 0x55a9aeada320 2023-10-05T19:41:33.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.972+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb131c0 con 0x55a9aeada320 2023-10-05T19:41:33.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.972+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(11 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e48014040 con 0x55a9aeada320 2023-10-05T19:41:33.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.972+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb131c0 con 0x7f5e40027be0 2023-10-05T19:41:33.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.972+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e48014040 con 0x7f5e40027be0 2023-10-05T19:41:33.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.972+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb131c0 con 0x55a9aeb0bf60 2023-10-05T19:41:33.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.972+0000 7f5e4e98b700 1 -- 172.21.15.12:0/488197165 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e40024020 con 0x55a9aeb0bf60 2023-10-05T19:41:33.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.976+0000 7f5e4e98b700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb131c0 con 0x55a9aeb0bf60 2023-10-05T19:41:33.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.976+0000 7f5e4e98b700 1 -- 172.21.15.12:0/488197165 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e40024020 con 0x55a9aeb0bf60 2023-10-05T19:41:33.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.976+0000 7f5e4e98b700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb131c0 con 0x55a9aeada320 2023-10-05T19:41:33.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.976+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e48014040 con 0x55a9aeada320 2023-10-05T19:41:33.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.976+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb131c0 con 0x7f5e40027be0 2023-10-05T19:41:33.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.976+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e48014040 con 0x7f5e40027be0 2023-10-05T19:41:33.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.976+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb131c0 con 0x55a9aeada320 2023-10-05T19:41:33.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.976+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e48014040 con 0x55a9aeada320 2023-10-05T19:41:33.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.976+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb131c0 con 0x55a9aeb0bf60 2023-10-05T19:41:33.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.976+0000 7f5e4e98b700 1 -- 172.21.15.12:0/488197165 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(18 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e40024020 con 0x55a9aeb0bf60 2023-10-05T19:41:33.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.976+0000 7f5e4e98b700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb131c0 con 0x55a9aeb10700 2023-10-05T19:41:33.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.980+0000 7f5e4e18a700 1 -- 172.21.15.12:0/488197165 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e44008a80 con 0x55a9aeb10700 2023-10-05T19:41:33.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.980+0000 7f5e4e18a700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb131c0 con 0x55a9aeb10700 2023-10-05T19:41:33.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.980+0000 7f5e4e18a700 1 -- 172.21.15.12:0/488197165 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e44008a80 con 0x55a9aeb10700 2023-10-05T19:41:33.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.980+0000 7f5e4e18a700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb131c0 con 0x7f5e40027be0 2023-10-05T19:41:33.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.980+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e48014040 con 0x7f5e40027be0 2023-10-05T19:41:33.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.980+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb131c0 con 0x55a9aeb10700 2023-10-05T19:41:33.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.980+0000 7f5e4e18a700 1 -- 172.21.15.12:0/488197165 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e44008a80 con 0x55a9aeb10700 2023-10-05T19:41:33.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.980+0000 7f5e4e18a700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb131c0 con 0x55a9aeb10700 2023-10-05T19:41:33.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.980+0000 7f5e4e18a700 1 -- 172.21.15.12:0/488197165 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e44008a80 con 0x55a9aeb10700 2023-10-05T19:41:33.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.980+0000 7f5e4e18a700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb131c0 con 0x55a9aeada320 2023-10-05T19:41:33.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.984+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e48014040 con 0x55a9aeada320 2023-10-05T19:41:33.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.984+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb131c0 con 0x55a9aeada320 2023-10-05T19:41:33.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.984+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e48014040 con 0x55a9aeada320 2023-10-05T19:41:33.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.984+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb131c0 con 0x7f5e40027be0 2023-10-05T19:41:33.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.984+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e48014040 con 0x7f5e40027be0 2023-10-05T19:41:33.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.984+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb131c0 con 0x55a9aeb10700 2023-10-05T19:41:33.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.984+0000 7f5e4e18a700 1 -- 172.21.15.12:0/488197165 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e44008a80 con 0x55a9aeb10700 2023-10-05T19:41:33.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.984+0000 7f5e4e18a700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb131c0 con 0x55a9aeb0bf60 2023-10-05T19:41:33.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.984+0000 7f5e4e98b700 1 -- 172.21.15.12:0/488197165 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e40024020 con 0x55a9aeb0bf60 2023-10-05T19:41:33.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.984+0000 7f5e4e98b700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb131c0 con 0x55a9aeada320 2023-10-05T19:41:33.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.988+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e48014040 con 0x55a9aeada320 2023-10-05T19:41:33.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.988+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb131c0 con 0x7f5e40027be0 2023-10-05T19:41:33.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.988+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e48014040 con 0x7f5e40027be0 2023-10-05T19:41:33.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.988+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb131c0 con 0x55a9aeb10700 2023-10-05T19:41:33.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.988+0000 7f5e4e18a700 1 -- 172.21.15.12:0/488197165 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e44008a80 con 0x55a9aeb10700 2023-10-05T19:41:33.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.988+0000 7f5e4e18a700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb131c0 con 0x7f5e40027be0 2023-10-05T19:41:33.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.988+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e48014040 con 0x7f5e40027be0 2023-10-05T19:41:33.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.988+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb131c0 con 0x55a9aeb0bf60 2023-10-05T19:41:33.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.988+0000 7f5e4e98b700 1 -- 172.21.15.12:0/488197165 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e40024020 con 0x55a9aeb0bf60 2023-10-05T19:41:33.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.988+0000 7f5e4e98b700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb131c0 con 0x55a9aeb10700 2023-10-05T19:41:33.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.992+0000 7f5e4e18a700 1 -- 172.21.15.12:0/488197165 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e44008a80 con 0x55a9aeb10700 2023-10-05T19:41:33.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.992+0000 7f5e4e18a700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb131c0 con 0x7f5e40027be0 2023-10-05T19:41:33.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.992+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e48014040 con 0x7f5e40027be0 2023-10-05T19:41:33.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.992+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb131c0 con 0x55a9aeada320 2023-10-05T19:41:33.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.992+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f5e48014040 con 0x55a9aeada320 2023-10-05T19:41:33.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.992+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb131c0 con 0x55a9aeb0bf60 2023-10-05T19:41:33.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.992+0000 7f5e4e98b700 1 -- 172.21.15.12:0/488197165 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(37 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f5e40024020 con 0x55a9aeb0bf60 2023-10-05T19:41:33.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.992+0000 7f5e50e56e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:41:33.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.992+0000 7f5e50e56e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:41:33.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.992+0000 7f5e50e56e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:41:34.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.992+0000 7f5e50e56e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:41:34.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.992+0000 7f5e50e56e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.992+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb0f480 con 0x55a9aeada320 2023-10-05T19:41:34.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.992+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5e48014040 con 0x55a9aeada320 2023-10-05T19:41:34.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.996+0000 7f5e50e56e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:34.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.996+0000 7f5e50e56e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.996+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb0f0b0 con 0x55a9aeada320 2023-10-05T19:41:34.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.996+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f5e48014040 con 0x55a9aeada320 2023-10-05T19:41:34.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.996+0000 7f5e50e56e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:34.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.996+0000 7f5e50e56e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.996+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb13630 con 0x55a9aeb10700 2023-10-05T19:41:34.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.996+0000 7f5e4e18a700 1 -- 172.21.15.12:0/488197165 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f5e44008a80 con 0x55a9aeb10700 2023-10-05T19:41:34.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.996+0000 7f5e50e56e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:41:34.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.996+0000 7f5e50e56e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.996+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb139e0 con 0x55a9aeb10700 2023-10-05T19:41:34.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.996+0000 7f5e4e18a700 1 -- 172.21.15.12:0/488197165 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f5e44008a80 con 0x55a9aeb10700 2023-10-05T19:41:34.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.996+0000 7f5e50e56e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:34.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.996+0000 7f5e50e56e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.996+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb13d90 con 0x55a9aeada320 2023-10-05T19:41:34.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.996+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5e48014040 con 0x55a9aeada320 2023-10-05T19:41:34.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.996+0000 7f5e50e56e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:34.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.996+0000 7f5e50e56e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.996+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb141a0 con 0x55a9aeada320 2023-10-05T19:41:34.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.996+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f5e48014040 con 0x55a9aeada320 2023-10-05T19:41:34.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.996+0000 7f5e50e56e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:34.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:33.996+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb14170 con 0x55a9aeb10700 2023-10-05T19:41:34.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.000+0000 7f5e4e18a700 1 -- 172.21.15.12:0/488197165 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(44 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f5e44008a80 con 0x55a9aeb10700 2023-10-05T19:41:34.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.000+0000 7f5e4e18a700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb14170 con 0x55a9aeb0bf60 2023-10-05T19:41:34.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.000+0000 7f5e4e98b700 1 -- 172.21.15.12:0/488197165 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(45 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e40024020 con 0x55a9aeb0bf60 2023-10-05T19:41:34.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.000+0000 7f5e4e98b700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb14170 con 0x7f5e40027be0 2023-10-05T19:41:34.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.004+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(46 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e48014040 con 0x7f5e40027be0 2023-10-05T19:41:34.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.004+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb13d90 con 0x55a9aeada320 2023-10-05T19:41:34.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.004+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(47 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e48014040 con 0x55a9aeada320 2023-10-05T19:41:34.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.004+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb13d90 con 0x55a9aeada320 2023-10-05T19:41:34.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.004+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(48 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e48014040 con 0x55a9aeada320 2023-10-05T19:41:34.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.004+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb13d90 con 0x55a9aeada320 2023-10-05T19:41:34.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.004+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(49 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e48014040 con 0x55a9aeada320 2023-10-05T19:41:34.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.004+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb13d90 con 0x7f5e40027be0 2023-10-05T19:41:34.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.004+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(50 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e48014040 con 0x7f5e40027be0 2023-10-05T19:41:34.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.004+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb13d90 con 0x55a9aeb0bf60 2023-10-05T19:41:34.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.008+0000 7f5e4e98b700 1 -- 172.21.15.12:0/488197165 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(51 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e40024020 con 0x55a9aeb0bf60 2023-10-05T19:41:34.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.008+0000 7f5e4e98b700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb13d90 con 0x55a9aeb0bf60 2023-10-05T19:41:34.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.008+0000 7f5e4e98b700 1 -- 172.21.15.12:0/488197165 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(52 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e40024020 con 0x55a9aeb0bf60 2023-10-05T19:41:34.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.008+0000 7f5e4e98b700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb13d90 con 0x55a9aeada320 2023-10-05T19:41:34.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.008+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(53 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e48014040 con 0x55a9aeada320 2023-10-05T19:41:34.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.008+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb13d90 con 0x7f5e40027be0 2023-10-05T19:41:34.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.008+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(54 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e48014040 con 0x7f5e40027be0 2023-10-05T19:41:34.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.008+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb13d90 con 0x55a9aeada320 2023-10-05T19:41:34.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.008+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(55 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e48014040 con 0x55a9aeada320 2023-10-05T19:41:34.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.008+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb13d90 con 0x55a9aeb0bf60 2023-10-05T19:41:34.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.012+0000 7f5e4e98b700 1 -- 172.21.15.12:0/488197165 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(56 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e40024020 con 0x55a9aeb0bf60 2023-10-05T19:41:34.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.012+0000 7f5e4e98b700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb13d90 con 0x55a9aeb10700 2023-10-05T19:41:34.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.012+0000 7f5e4e18a700 1 -- 172.21.15.12:0/488197165 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(57 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e44008a80 con 0x55a9aeb10700 2023-10-05T19:41:34.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.012+0000 7f5e4e18a700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb13d90 con 0x55a9aeb10700 2023-10-05T19:41:34.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.012+0000 7f5e4e18a700 1 -- 172.21.15.12:0/488197165 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(58 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e44008a80 con 0x55a9aeb10700 2023-10-05T19:41:34.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.012+0000 7f5e4e18a700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb13d90 con 0x7f5e40027be0 2023-10-05T19:41:34.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.012+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(59 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e48014040 con 0x7f5e40027be0 2023-10-05T19:41:34.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.012+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb13d90 con 0x55a9aeb10700 2023-10-05T19:41:34.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.012+0000 7f5e4e18a700 1 -- 172.21.15.12:0/488197165 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(60 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e44008a80 con 0x55a9aeb10700 2023-10-05T19:41:34.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.012+0000 7f5e4e18a700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb13d90 con 0x55a9aeb10700 2023-10-05T19:41:34.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.016+0000 7f5e4e18a700 1 -- 172.21.15.12:0/488197165 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(61 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e44008a80 con 0x55a9aeb10700 2023-10-05T19:41:34.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.016+0000 7f5e4e18a700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb13d90 con 0x55a9aeada320 2023-10-05T19:41:34.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.016+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(62 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e48014040 con 0x55a9aeada320 2023-10-05T19:41:34.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.016+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb13d90 con 0x55a9aeada320 2023-10-05T19:41:34.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.016+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(63 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e48014040 con 0x55a9aeada320 2023-10-05T19:41:34.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.016+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb13d90 con 0x7f5e40027be0 2023-10-05T19:41:34.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.016+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(64 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e48014040 con 0x7f5e40027be0 2023-10-05T19:41:34.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.016+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb13d90 con 0x55a9aeb10700 2023-10-05T19:41:34.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.016+0000 7f5e4e18a700 1 -- 172.21.15.12:0/488197165 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(65 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e44008a80 con 0x55a9aeb10700 2023-10-05T19:41:34.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.016+0000 7f5e4e18a700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb13d90 con 0x55a9aeb0bf60 2023-10-05T19:41:34.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.020+0000 7f5e4e98b700 1 -- 172.21.15.12:0/488197165 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(66 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e40024020 con 0x55a9aeb0bf60 2023-10-05T19:41:34.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.020+0000 7f5e4e98b700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb13d90 con 0x55a9aeada320 2023-10-05T19:41:34.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.020+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(67 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e48014040 con 0x55a9aeada320 2023-10-05T19:41:34.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.020+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb13d90 con 0x7f5e40027be0 2023-10-05T19:41:34.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.020+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(68 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e48014040 con 0x7f5e40027be0 2023-10-05T19:41:34.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.020+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb13d90 con 0x55a9aeb10700 2023-10-05T19:41:34.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.020+0000 7f5e4e18a700 1 -- 172.21.15.12:0/488197165 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(69 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e44008a80 con 0x55a9aeb10700 2023-10-05T19:41:34.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.020+0000 7f5e4e18a700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb13d90 con 0x7f5e40027be0 2023-10-05T19:41:34.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.024+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(70 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e48014040 con 0x7f5e40027be0 2023-10-05T19:41:34.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.024+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb13d90 con 0x55a9aeb0bf60 2023-10-05T19:41:34.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.024+0000 7f5e4e98b700 1 -- 172.21.15.12:0/488197165 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(71 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e40024020 con 0x55a9aeb0bf60 2023-10-05T19:41:34.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.024+0000 7f5e4e98b700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb13d90 con 0x55a9aeb10700 2023-10-05T19:41:34.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.024+0000 7f5e4e18a700 1 -- 172.21.15.12:0/488197165 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(72 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e44008a80 con 0x55a9aeb10700 2023-10-05T19:41:34.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.024+0000 7f5e4e18a700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb13d90 con 0x7f5e40027be0 2023-10-05T19:41:34.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.024+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(73 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5e48014040 con 0x7f5e40027be0 2023-10-05T19:41:34.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.024+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb13d90 con 0x55a9aeada320 2023-10-05T19:41:34.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.028+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(74 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f5e48014040 con 0x55a9aeada320 2023-10-05T19:41:34.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.028+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb13d90 con 0x55a9aeb0bf60 2023-10-05T19:41:34.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.028+0000 7f5e4e98b700 1 -- 172.21.15.12:0/488197165 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(75 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f5e40024020 con 0x55a9aeb0bf60 2023-10-05T19:41:34.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.028+0000 7f5e50e56e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:41:34.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.028+0000 7f5e50e56e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:41:34.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.028+0000 7f5e50e56e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:41:34.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.028+0000 7f5e50e56e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:41:34.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.028+0000 7f5e50e56e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.028+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb14170 con 0x55a9aeada320 2023-10-05T19:41:34.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.028+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f5e48014040 con 0x55a9aeada320 2023-10-05T19:41:34.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.028+0000 7f5e50e56e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:34.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.028+0000 7f5e50e56e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.028+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb145d0 con 0x55a9aeb0bf60 2023-10-05T19:41:34.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.028+0000 7f5e4e98b700 1 -- 172.21.15.12:0/488197165 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f5e40024020 con 0x55a9aeb0bf60 2023-10-05T19:41:34.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.028+0000 7f5e50e56e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:41:34.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.028+0000 7f5e50e56e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.028+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb14ec0 con 0x55a9aeada320 2023-10-05T19:41:34.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.028+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f5e48014040 con 0x55a9aeada320 2023-10-05T19:41:34.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.028+0000 7f5e50e56e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:34.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.028+0000 7f5e50e56e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.028+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb14ec0 con 0x55a9aeb10700 2023-10-05T19:41:34.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.032+0000 7f5e4e18a700 1 -- 172.21.15.12:0/488197165 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f5e44008a80 con 0x55a9aeb10700 2023-10-05T19:41:34.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.032+0000 7f5e50e56e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:41:34.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.032+0000 7f5e50e56e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.032+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb15630 con 0x7f5e40027be0 2023-10-05T19:41:34.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.032+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f5e48014040 con 0x7f5e40027be0 2023-10-05T19:41:34.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.032+0000 7f5e50e56e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:34.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.032+0000 7f5e50e56e40 20 zone default found 2023-10-05T19:41:34.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.032+0000 7f5e50e56e40 4 Realm: () 2023-10-05T19:41:34.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.032+0000 7f5e50e56e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:41:34.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.032+0000 7f5e50e56e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:41:34.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.032+0000 7f5e50e56e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:41:34.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.032+0000 7f5e50e56e40 20 zonegroup default 2023-10-05T19:41:34.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.032+0000 7f5e50e56e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.032+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb15630 con 0x7f5e40027be0 2023-10-05T19:41:34.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.032+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f5e48014040 con 0x7f5e40027be0 2023-10-05T19:41:34.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.032+0000 7f5e50e56e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:34.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.032+0000 7f5e50e56e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.032+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb16fb0 con 0x55a9aeb10700 2023-10-05T19:41:34.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.032+0000 7f5e4e18a700 1 -- 172.21.15.12:0/488197165 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f5e44008a80 con 0x55a9aeb10700 2023-10-05T19:41:34.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.032+0000 7f5e50e56e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:34.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.032+0000 7f5e50e56e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.032+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb17360 con 0x55a9aeb10700 2023-10-05T19:41:34.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.036+0000 7f5e4e18a700 1 -- 172.21.15.12:0/488197165 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f5e44008a80 con 0x55a9aeb10700 2023-10-05T19:41:34.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.036+0000 7f5e50e56e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:34.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.036+0000 7f5e50e56e40 20 started sync module instance, tier type = 2023-10-05T19:41:34.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.036+0000 7f5e50e56e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:41:34.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.036+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb16670 con 0x55a9aeb0bf60 2023-10-05T19:41:34.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.036+0000 7f5e4e98b700 1 -- 172.21.15.12:0/488197165 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(84 notify.0 [create] v30'249 uv249 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e40024020 con 0x55a9aeb0bf60 2023-10-05T19:41:34.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.040+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94187268688336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb17710 con 0x55a9aeb0bf60 2023-10-05T19:41:34.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.040+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb17fb0 con 0x55a9aeada320 2023-10-05T19:41:34.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.040+0000 7f5e4e98b700 1 -- 172.21.15.12:0/488197165 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(85 notify.0 [watch watch cookie 94187268688336] v30'250 uv249 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e40024020 con 0x55a9aeb0bf60 2023-10-05T19:41:34.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.040+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(86 notify.1 [create] v30'249 uv249 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e48014040 con 0x55a9aeada320 2023-10-05T19:41:34.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.040+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94187268700864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb17f10 con 0x55a9aeada320 2023-10-05T19:41:34.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.040+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb18d90 con 0x55a9aeb10700 2023-10-05T19:41:34.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.044+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(87 notify.1 [watch watch cookie 94187268700864] v30'250 uv249 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e48014040 con 0x55a9aeada320 2023-10-05T19:41:34.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.044+0000 7f5e4e18a700 1 -- 172.21.15.12:0/488197165 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(88 notify.2 [create] v30'249 uv249 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e44008a80 con 0x55a9aeb10700 2023-10-05T19:41:34.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.044+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94187268703152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb19140 con 0x55a9aeb10700 2023-10-05T19:41:34.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.044+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb199e0 con 0x55a9aeb0bf60 2023-10-05T19:41:34.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.048+0000 7f5e4e98b700 1 -- 172.21.15.12:0/488197165 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(90 notify.3 [create] v30'249 uv249 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e40024020 con 0x55a9aeb0bf60 2023-10-05T19:41:34.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.048+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94187268709856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb1a720 con 0x55a9aeb0bf60 2023-10-05T19:41:34.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.048+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb1afc0 con 0x55a9aeada320 2023-10-05T19:41:34.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.048+0000 7f5e4e18a700 1 -- 172.21.15.12:0/488197165 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(89 notify.2 [watch watch cookie 94187268703152] v30'250 uv249 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e44008a80 con 0x55a9aeb10700 2023-10-05T19:41:34.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.052+0000 7f5e4e98b700 1 -- 172.21.15.12:0/488197165 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(91 notify.3 [watch watch cookie 94187268709856] v30'250 uv249 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e40024020 con 0x55a9aeb0bf60 2023-10-05T19:41:34.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.052+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(92 notify.4 [create] v30'249 uv249 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e48014040 con 0x55a9aeada320 2023-10-05T19:41:34.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.052+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94187268710960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb1ab70 con 0x55a9aeada320 2023-10-05T19:41:34.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.052+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb1b8b0 con 0x7f5e40027be0 2023-10-05T19:41:34.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.052+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(93 notify.4 [watch watch cookie 94187268710960] v30'250 uv249 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e48014040 con 0x55a9aeada320 2023-10-05T19:41:34.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.052+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(94 notify.5 [create] v30'249 uv249 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e48014040 con 0x7f5e40027be0 2023-10-05T19:41:34.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.052+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94187268712064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb1aad0 con 0x7f5e40027be0 2023-10-05T19:41:34.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.052+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb1bcf0 con 0x7f5e40027be0 2023-10-05T19:41:34.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.056+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(95 notify.5 [watch watch cookie 94187268712064] v30'250 uv249 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e48014040 con 0x7f5e40027be0 2023-10-05T19:41:34.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.056+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(96 notify.6 [create] v30'249 uv249 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e48014040 con 0x7f5e40027be0 2023-10-05T19:41:34.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.056+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94187268714912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb1bcf0 con 0x7f5e40027be0 2023-10-05T19:41:34.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.056+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb1c750 con 0x7f5e40027be0 2023-10-05T19:41:34.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.060+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(97 notify.6 [watch watch cookie 94187268714912] v30'250 uv249 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e48014040 con 0x7f5e40027be0 2023-10-05T19:41:34.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.060+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(98 notify.7 [create] v30'249 uv249 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e48014040 con 0x7f5e40027be0 2023-10-05T19:41:34.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.060+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94187268716816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb1c750 con 0x7f5e40027be0 2023-10-05T19:41:34.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.060+0000 7f5e50e56e40 20 add_watcher() i=0 2023-10-05T19:41:34.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.060+0000 7f5e50e56e40 20 add_watcher() i=1 2023-10-05T19:41:34.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.060+0000 7f5e50e56e40 20 add_watcher() i=2 2023-10-05T19:41:34.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.060+0000 7f5e50e56e40 20 add_watcher() i=3 2023-10-05T19:41:34.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.060+0000 7f5e50e56e40 20 add_watcher() i=4 2023-10-05T19:41:34.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.060+0000 7f5e50e56e40 20 add_watcher() i=5 2023-10-05T19:41:34.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.060+0000 7f5e50e56e40 20 add_watcher() i=6 2023-10-05T19:41:34.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.060+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(99 notify.7 [watch watch cookie 94187268716816] v30'250 uv249 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e48014040 con 0x7f5e40027be0 2023-10-05T19:41:34.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.060+0000 7f5e50e56e40 20 add_watcher() i=7 2023-10-05T19:41:34.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.060+0000 7f5e50e56e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:41:34.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.060+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb1f7f0 con 0x55a9aeada320 2023-10-05T19:41:34.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.060+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f5e48014040 con 0x55a9aeada320 2023-10-05T19:41:34.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.064+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94187268719200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb1cba0 con 0x55a9aeada320 2023-10-05T19:41:34.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.064+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94187268719200] v30'237 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f5e48014040 con 0x55a9aeada320 2023-10-05T19:41:34.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.064+0000 7f5e1ffe7700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:41:34.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.064+0000 7f5e50e56e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:41:34.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.064+0000 7f5e50e56e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:41:34.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.064+0000 7f5e50e56e40 1 Processor -- start 2023-10-05T19:41:34.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.064+0000 7f5e50e56e40 1 -- start start 2023-10-05T19:41:34.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.064+0000 7f5e50e56e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a9aeb2a010 0x55a9aeb2a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:34.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.064+0000 7f5e50e56e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9aeb2a920 con 0x55a9aeb2a010 2023-10-05T19:41:34.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.064+0000 7f5e4e18a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a9aeb2a010 0x55a9aeb2a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:34.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.064+0000 7f5e4e18a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a9aeb2a010 0x55a9aeb2a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:60816/0 (socket says 172.21.15.12:60816) 2023-10-05T19:41:34.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.064+0000 7f5e4e18a700 1 -- 172.21.15.12:0/3480845223 learned_addr learned my addr 172.21.15.12:0/3480845223 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:34.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.068+0000 7f5e4e18a700 1 -- 172.21.15.12:0/3480845223 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9aeb2aa60 con 0x55a9aeb2a010 2023-10-05T19:41:34.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.068+0000 7f5e4e18a700 1 --2- 172.21.15.12:0/3480845223 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a9aeb2a010 0x55a9aeb2a3e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x55a9aea493e0 tx=0x7f5e44012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faa9e268e3fb009b server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:34.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.068+0000 7f5e1f7e6700 1 -- 172.21.15.12:0/3480845223 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e44030930 con 0x55a9aeb2a010 2023-10-05T19:41:34.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.068+0000 7f5e1f7e6700 1 -- 172.21.15.12:0/3480845223 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e44030a90 con 0x55a9aeb2a010 2023-10-05T19:41:34.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.068+0000 7f5e1f7e6700 1 -- 172.21.15.12:0/3480845223 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e44030c80 con 0x55a9aeb2a010 2023-10-05T19:41:34.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.068+0000 7f5e50e56e40 1 -- 172.21.15.12:0/3480845223 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a9aeb2a010 msgr2=0x55a9aeb2a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:34.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.068+0000 7f5e50e56e40 1 --2- 172.21.15.12:0/3480845223 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a9aeb2a010 0x55a9aeb2a3e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x55a9aea493e0 tx=0x7f5e44012000 comp rx=0 tx=0).stop 2023-10-05T19:41:34.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.068+0000 7f5e50e56e40 1 -- 172.21.15.12:0/3480845223 shutdown_connections 2023-10-05T19:41:34.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.068+0000 7f5e50e56e40 1 --2- 172.21.15.12:0/3480845223 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a9aeb2a010 0x55a9aeb2a3e0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:34.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.068+0000 7f5e50e56e40 1 -- 172.21.15.12:0/3480845223 >> 172.21.15.12:0/3480845223 conn(0x55a9aeb253d0 msgr2=0x55a9aeb23520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:34.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.068+0000 7f5e50e56e40 1 -- 172.21.15.12:0/3480845223 shutdown_connections 2023-10-05T19:41:34.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.068+0000 7f5e50e56e40 1 -- 172.21.15.12:0/3480845223 wait complete. 2023-10-05T19:41:34.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.068+0000 7f5e50e56e40 1 Processor -- start 2023-10-05T19:41:34.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.068+0000 7f5e50e56e40 1 -- start start 2023-10-05T19:41:34.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.068+0000 7f5e50e56e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a9aeb2a010 0x55a9aeb2fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:34.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.068+0000 7f5e50e56e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9aeb30b10 con 0x55a9aeb2a010 2023-10-05T19:41:34.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.072+0000 7f5e4e18a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a9aeb2a010 0x55a9aeb2fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:34.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.072+0000 7f5e4e18a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a9aeb2a010 0x55a9aeb2fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:60828/0 (socket says 172.21.15.12:60828) 2023-10-05T19:41:34.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.072+0000 7f5e4e18a700 1 -- 172.21.15.12:0/1508232210 learned_addr learned my addr 172.21.15.12:0/1508232210 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:34.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.072+0000 7f5e4e18a700 1 -- 172.21.15.12:0/1508232210 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9aeaddc50 con 0x55a9aeb2a010 2023-10-05T19:41:34.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.072+0000 7f5e4e18a700 1 --2- 172.21.15.12:0/1508232210 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a9aeb2a010 0x55a9aeb2fb10 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x55a9aeb24090 tx=0x7f5e44033000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:34.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.072+0000 7f5e1dfe3700 1 -- 172.21.15.12:0/1508232210 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e44030930 con 0x55a9aeb2a010 2023-10-05T19:41:34.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.072+0000 7f5e1dfe3700 1 -- 172.21.15.12:0/1508232210 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e44027400 con 0x55a9aeb2a010 2023-10-05T19:41:34.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.072+0000 7f5e50e56e40 1 -- 172.21.15.12:0/1508232210 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a9aeb30110 con 0x55a9aeb2a010 2023-10-05T19:41:34.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.072+0000 7f5e1dfe3700 1 -- 172.21.15.12:0/1508232210 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e440275b0 con 0x55a9aeb2a010 2023-10-05T19:41:34.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.072+0000 7f5e50e56e40 1 -- 172.21.15.12:0/1508232210 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a9aeb30550 con 0x55a9aeb2a010 2023-10-05T19:41:34.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.072+0000 7f5e1dfe3700 1 -- 172.21.15.12:0/1508232210 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f5e44036040 con 0x55a9aeb2a010 2023-10-05T19:41:34.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.072+0000 7f5e1dfe3700 1 --2- 172.21.15.12:0/1508232210 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5d94030c50 0x7f5d94033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:34.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.072+0000 7f5e1dfe3700 1 -- 172.21.15.12:0/1508232210 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7f5e44055bd0 con 0x55a9aeb2a010 2023-10-05T19:41:34.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.072+0000 7f5e4f18c700 1 --2- 172.21.15.12:0/1508232210 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5d94030c50 0x7f5d94033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:34.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.076+0000 7f5e50e56e40 5 note: GC not initialized 2023-10-05T19:41:34.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.076+0000 7f5e4f18c700 1 --2- 172.21.15.12:0/1508232210 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5d94030c50 0x7f5d94033100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x55a9aeb1f900 tx=0x7f5e480335f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:34.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.076+0000 7f5d8effd700 20 reqs_thread_entry: start 2023-10-05T19:41:34.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.076+0000 7f5d8cff9700 20 reqs_thread_entry: start 2023-10-05T19:41:34.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.076+0000 7f5e50e56e40 20 init_complete bucket index max shards: 11 2023-10-05T19:41:34.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.076+0000 7f5e50e56e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:41:34.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.076+0000 7f5d7effd700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:42:04 2023 2023-10-05T19:41:34.086 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:41:34.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.076+0000 7f5d7e7fc700 20 reqs_thread_entry: start 2023-10-05T19:41:34.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.076+0000 7f5e50e56e40 20 get_system_obj_state: rctx=0x7ffd1260a4d8 obj=default.rgw.meta:users.uid:bud state=0x55a9ae9af0b0 s->prefetch_data=0 2023-10-05T19:41:34.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.076+0000 7f5e50e56e40 10 cache get: name=default.rgw.meta+users.uid+bud : miss 2023-10-05T19:41:34.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.076+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:102 8.4 8:267425a8:users.uid::bud:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb98460 con 0x55a9aeb0bf60 2023-10-05T19:41:34.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.080+0000 7f5e4e98b700 1 -- 172.21.15.12:0/488197165 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(102 bud [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7f5e40024020 con 0x55a9aeb0bf60 2023-10-05T19:41:34.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.080+0000 7f5e50e56e40 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x16 2023-10-05T19:41:34.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.080+0000 7f5e50e56e40 10 adding default.rgw.meta+users.uid+bud to cache LRU end 2023-10-05T19:41:34.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.080+0000 7f5e50e56e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:41:34.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.080+0000 7f5e50e56e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:41:34.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.080+0000 7f5e50e56e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:41:34.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.080+0000 7f5e50e56e40 10 cache get: name=default.rgw.meta+users.uid+bud : type miss (requested=0x13, cached=0x16) 2023-10-05T19:41:34.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.080+0000 7f5e50e56e40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.080+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:103 8.4 8:267425a8:users.uid::bud:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb9aef0 con 0x55a9aeb0bf60 2023-10-05T19:41:34.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.080+0000 7f5e4e98b700 1 -- 172.21.15.12:0/488197165 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(103 bud [call,call out=48b,read 0~346 out=346b,getxattrs out=75b] v0'0 uv2 ondisk = 0) v8 ==== 273+0+469 (crc 0 0 0) 0x7f5e40024020 con 0x55a9aeb0bf60 2023-10-05T19:41:34.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.080+0000 7f5e50e56e40 20 rados_obj.operate() r=0 bl.length=346 2023-10-05T19:41:34.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.080+0000 7f5e50e56e40 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x13 2023-10-05T19:41:34.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.080+0000 7f5e50e56e40 10 moving default.rgw.meta+users.uid+bud to cache LRU end 2023-10-05T19:41:34.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.080+0000 7f5e50e56e40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:41:34.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.080+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:104 8.e 8:75f48285:users.uid::fud:head [create,call version.set in=58b,writefull 0~181 in=181b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeb9c540 con 0x7f5e40027be0 2023-10-05T19:41:34.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.084+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(104 fud [create,call,writefull 0~181] v30'15 uv14 ondisk = -17 ((17) File exists)) v8 ==== 231+0+0 (crc 0 0 0) 0x7f5e48014040 con 0x7f5e40027be0 2023-10-05T19:41:34.089 INFO:teuthology.orchestra.run.smithi012.stderr:could not rename user: unable to rename user, user name given by --new-uid already exists. to overwrite this user, add --yes-i-really-mean-it 2023-10-05T19:41:34.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.084+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:105 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94187268688336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeaeb7c0 con 0x55a9aeb0bf60 2023-10-05T19:41:34.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.084+0000 7f5e4e98b700 1 -- 172.21.15.12:0/488197165 <== osd.2 v2:172.21.15.12:6800/29934 23 ==== osd_op_reply(105 notify.0 [watch unwatch cookie 94187268688336] v30'251 uv249 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e40024020 con 0x55a9aeb0bf60 2023-10-05T19:41:34.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.084+0000 7f5e50e56e40 20 remove_watcher() i=0 2023-10-05T19:41:34.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.084+0000 7f5e50e56e40 2 removed watcher, disabling cache 2023-10-05T19:41:34.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.084+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:106 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94187268700864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeaeb7c0 con 0x55a9aeada320 2023-10-05T19:41:34.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.088+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(106 notify.1 [watch unwatch cookie 94187268700864] v30'251 uv249 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e48014040 con 0x55a9aeada320 2023-10-05T19:41:34.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.088+0000 7f5e50e56e40 20 remove_watcher() i=1 2023-10-05T19:41:34.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.088+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:107 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94187268703152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeaeb7c0 con 0x55a9aeb10700 2023-10-05T19:41:34.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.092+0000 7f5e4e18a700 1 -- 172.21.15.12:0/488197165 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(107 notify.2 [watch unwatch cookie 94187268703152] v30'251 uv249 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e44008a80 con 0x55a9aeb10700 2023-10-05T19:41:34.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.092+0000 7f5e50e56e40 20 remove_watcher() i=2 2023-10-05T19:41:34.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.092+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:108 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94187268709856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeaeb7c0 con 0x55a9aeb0bf60 2023-10-05T19:41:34.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.092+0000 7f5e4e98b700 1 -- 172.21.15.12:0/488197165 <== osd.2 v2:172.21.15.12:6800/29934 24 ==== osd_op_reply(108 notify.3 [watch unwatch cookie 94187268709856] v30'251 uv249 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e40024020 con 0x55a9aeb0bf60 2023-10-05T19:41:34.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.092+0000 7f5e50e56e40 20 remove_watcher() i=3 2023-10-05T19:41:34.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.092+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:109 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94187268710960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeaeb7c0 con 0x55a9aeada320 2023-10-05T19:41:34.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.096+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(109 notify.4 [watch unwatch cookie 94187268710960] v30'251 uv249 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e48014040 con 0x55a9aeada320 2023-10-05T19:41:34.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.096+0000 7f5e50e56e40 20 remove_watcher() i=4 2023-10-05T19:41:34.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.096+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:110 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94187268712064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeaeb7c0 con 0x7f5e40027be0 2023-10-05T19:41:34.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.100+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(110 notify.5 [watch unwatch cookie 94187268712064] v30'251 uv249 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e48014040 con 0x7f5e40027be0 2023-10-05T19:41:34.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.100+0000 7f5e50e56e40 20 remove_watcher() i=5 2023-10-05T19:41:34.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.100+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:111 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94187268714912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeaeb7c0 con 0x7f5e40027be0 2023-10-05T19:41:34.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.104+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(111 notify.6 [watch unwatch cookie 94187268714912] v30'251 uv249 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e48014040 con 0x7f5e40027be0 2023-10-05T19:41:34.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.104+0000 7f5e50e56e40 20 remove_watcher() i=6 2023-10-05T19:41:34.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.104+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:112 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94187268716816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeaeb7c0 con 0x7f5e40027be0 2023-10-05T19:41:34.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.108+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(112 notify.7 [watch unwatch cookie 94187268716816] v30'251 uv249 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5e48014040 con 0x7f5e40027be0 2023-10-05T19:41:34.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.108+0000 7f5e50e56e40 20 remove_watcher() i=7 2023-10-05T19:41:34.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.108+0000 7f5e1ffe7700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:41:34.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.108+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:113 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94187268719200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55a9aeaeb7c0 con 0x55a9aeada320 2023-10-05T19:41:34.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.112+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(113 data_loggenerations_metadata [watch unwatch cookie 94187268719200] v30'238 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f5e48014040 con 0x55a9aeada320 2023-10-05T19:41:34.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.112+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f5e40027be0 msgr2=0x7f5e40028460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:34.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.112+0000 7f5e50e56e40 1 --2- 172.21.15.12:0/488197165 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f5e40027be0 0x7f5e40028460 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:34.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.112+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55a9aeada320 msgr2=0x55a9aeb099e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:34.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.112+0000 7f5e50e56e40 1 --2- 172.21.15.12:0/488197165 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55a9aeada320 0x55a9aeb099e0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:34.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.112+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55a9aeb0bf60 msgr2=0x55a9aeb0e340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:34.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.112+0000 7f5e50e56e40 1 --2- 172.21.15.12:0/488197165 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55a9aeb0bf60 0x55a9aeb0e340 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:34.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.112+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55a9aeb10700 msgr2=0x55a9aeb12b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:34.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.112+0000 7f5e50e56e40 1 --2- 172.21.15.12:0/488197165 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55a9aeb10700 0x55a9aeb12b10 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:34.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.112+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5e34030c50 msgr2=0x7f5e34033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:34.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.112+0000 7f5e50e56e40 1 --2- 172.21.15.12:0/488197165 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5e34030c50 0x7f5e34033100 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5e44005a90 tx=0x7f5e44015000 comp rx=0 tx=0).stop 2023-10-05T19:41:34.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.112+0000 7f5e4f18c700 1 -- 172.21.15.12:0/488197165 reap_dead start 2023-10-05T19:41:34.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.112+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a9aea49b50 msgr2=0x55a9aeafdef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:34.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.112+0000 7f5e50e56e40 1 --2- 172.21.15.12:0/488197165 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a9aea49b50 0x55a9aeafdef0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f5e40006bd0 tx=0x7f5e40014000 comp rx=0 tx=0).stop 2023-10-05T19:41:34.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.112+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 shutdown_connections 2023-10-05T19:41:34.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.112+0000 7f5e50e56e40 1 --2- 172.21.15.12:0/488197165 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a9aea49b50 0x55a9aeafdef0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:34.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.112+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 >> 172.21.15.12:0/488197165 conn(0x55a9aead0750 msgr2=0x55a9aeaef560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:34.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.112+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 shutdown_connections 2023-10-05T19:41:34.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.112+0000 7f5e50e56e40 1 -- 172.21.15.12:0/488197165 wait complete. 2023-10-05T19:41:34.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.112+0000 7f5e50e56e40 1 -- 172.21.15.12:0/1508232210 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5d94030c50 msgr2=0x7f5d94033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:34.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.112+0000 7f5e50e56e40 1 --2- 172.21.15.12:0/1508232210 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5d94030c50 0x7f5d94033100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x55a9aeb1f900 tx=0x7f5e480335f0 comp rx=0 tx=0).stop 2023-10-05T19:41:34.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.112+0000 7f5e50e56e40 1 -- 172.21.15.12:0/1508232210 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a9aeb2a010 msgr2=0x55a9aeb2fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:34.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.116+0000 7f5e50e56e40 1 --2- 172.21.15.12:0/1508232210 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a9aeb2a010 0x55a9aeb2fb10 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x55a9aeb24090 tx=0x7f5e44033000 comp rx=0 tx=0).stop 2023-10-05T19:41:34.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.116+0000 7f5e50e56e40 1 -- 172.21.15.12:0/1508232210 shutdown_connections 2023-10-05T19:41:34.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.116+0000 7f5e50e56e40 1 --2- 172.21.15.12:0/1508232210 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5d94030c50 0x7f5d94033100 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:34.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.116+0000 7f5e50e56e40 1 --2- 172.21.15.12:0/1508232210 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55a9aeb2a010 0x55a9aeb2fb10 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:34.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.116+0000 7f5e50e56e40 1 -- 172.21.15.12:0/1508232210 >> 172.21.15.12:0/1508232210 conn(0x55a9aeb253d0 msgr2=0x55a9aeb23470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:34.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.116+0000 7f5e50e56e40 1 -- 172.21.15.12:0/1508232210 shutdown_connections 2023-10-05T19:41:34.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.116+0000 7f5e50e56e40 1 -- 172.21.15.12:0/1508232210 wait complete. 2023-10-05T19:41:34.126 DEBUG:teuthology.orchestra.run:got remote process result: 17 2023-10-05T19:41:34.174 INFO:tasks.radosgw_admin:RL: get_obj myfoo6 bud bytes_out=0 bytes_in=0 failed=False 2023-10-05T19:41:34.174 INFO:tasks.radosgw_admin:RL: get_obj myfoo6 bud bytes_out=0 bytes_in=8 failed=False 2023-10-05T19:41:34.182 INFO:tasks.radosgw_admin:RL: get_obj myfoo7 fud bytes_out=0 bytes_in=0 failed=False 2023-10-05T19:41:34.183 INFO:tasks.radosgw_admin:RL: get_obj myfoo7 fud bytes_out=0 bytes_in=8 failed=False 2023-10-05T19:41:34.183 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'bud', '--purge-data'] 2023-10-05T19:41:34.183 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'bud', '--purge-data'] 2023-10-05T19:41:34.183 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid bud --purge-data 2023-10-05T19:41:34.223 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:41:34.223 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:41:34.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.220+0000 7f215353ae40 1 Processor -- start 2023-10-05T19:41:34.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.220+0000 7f215353ae40 1 -- start start 2023-10-05T19:41:34.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.220+0000 7f215353ae40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55fce8d6a420 0x55fce8d68fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:34.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.220+0000 7f215353ae40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55fce8cb6bd0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.220+0000 7f215353ae40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x55fce8c73ee0 con 0x55fce8d6a420 2023-10-05T19:41:34.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.220+0000 7f215106f700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55fce8d6a420 0x55fce8d68fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:34.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.220+0000 7f215106f700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55fce8d6a420 0x55fce8d68fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:60830/0 (socket says 172.21.15.12:60830) 2023-10-05T19:41:34.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.220+0000 7f215106f700 1 -- 172.21.15.12:0/1425547629 learned_addr learned my addr 172.21.15.12:0/1425547629 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:34.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.220+0000 7f215086e700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x55fce8d6a7f0 0x55fce8d69510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:44330/0 (socket says 172.21.15.12:44330) 2023-10-05T19:41:34.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.220+0000 7f215106f700 1 -- 172.21.15.12:0/1425547629 >> v1:172.21.15.8:6789/0 conn(0x55fce8d6a7f0 legacy=0x55fce8d69510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:34.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.220+0000 7f215106f700 1 -- 172.21.15.12:0/1425547629 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fce8d69ce0 con 0x55fce8d6a420 2023-10-05T19:41:34.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.220+0000 7f215106f700 1 --2- 172.21.15.12:0/1425547629 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fce8d6a420 0x55fce8d68fd0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f214c00a650 tx=0x7f214c005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c19db2f5aae514 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:34.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.220+0000 7f214bfff700 1 -- 172.21.15.12:0/1425547629 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f214c008760 con 0x55fce8d6a420 2023-10-05T19:41:34.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.220+0000 7f214bfff700 1 -- 172.21.15.12:0/1425547629 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f214c0088c0 con 0x55fce8d6a420 2023-10-05T19:41:34.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.220+0000 7f214bfff700 1 -- 172.21.15.12:0/1425547629 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f214c008bb0 con 0x55fce8d6a420 2023-10-05T19:41:34.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.220+0000 7f215353ae40 1 -- 172.21.15.12:0/1425547629 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fce8d6a420 msgr2=0x55fce8d68fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:34.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.220+0000 7f215353ae40 1 --2- 172.21.15.12:0/1425547629 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fce8d6a420 0x55fce8d68fd0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f214c00a650 tx=0x7f214c005b10 comp rx=0 tx=0).stop 2023-10-05T19:41:34.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.220+0000 7f215353ae40 1 -- 172.21.15.12:0/1425547629 shutdown_connections 2023-10-05T19:41:34.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.220+0000 7f215353ae40 1 --2- 172.21.15.12:0/1425547629 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fce8d6a420 0x55fce8d68fd0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:34.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.220+0000 7f215353ae40 1 -- 172.21.15.12:0/1425547629 >> 172.21.15.12:0/1425547629 conn(0x55fce8d64730 msgr2=0x55fce8d66b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:34.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.220+0000 7f215353ae40 1 -- 172.21.15.12:0/1425547629 shutdown_connections 2023-10-05T19:41:34.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.220+0000 7f215353ae40 1 -- 172.21.15.12:0/1425547629 wait complete. 2023-10-05T19:41:34.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.220+0000 7f215353ae40 1 Processor -- start 2023-10-05T19:41:34.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.220+0000 7f215353ae40 1 -- start start 2023-10-05T19:41:34.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.224+0000 7f215353ae40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fce8d6a420 0x55fce8d8b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:34.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.224+0000 7f215353ae40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fce8c37e30 con 0x55fce8d6a420 2023-10-05T19:41:34.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.224+0000 7f215106f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fce8d6a420 0x55fce8d8b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:34.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.224+0000 7f215106f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fce8d6a420 0x55fce8d8b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:60834/0 (socket says 172.21.15.12:60834) 2023-10-05T19:41:34.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.224+0000 7f215106f700 1 -- 172.21.15.12:0/2525062487 learned_addr learned my addr 172.21.15.12:0/2525062487 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:34.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.224+0000 7f215106f700 1 -- 172.21.15.12:0/2525062487 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f214c005db0 con 0x55fce8d6a420 2023-10-05T19:41:34.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.224+0000 7f215106f700 1 --2- 172.21.15.12:0/2525062487 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fce8d6a420 0x55fce8d8b160 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f214c005b50 tx=0x7f214c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cab813d2cb643c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:34.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.224+0000 7f214affd700 1 -- 172.21.15.12:0/2525062487 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f214c0055f0 con 0x55fce8d6a420 2023-10-05T19:41:34.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.224+0000 7f214affd700 1 -- 172.21.15.12:0/2525062487 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f214c008a30 con 0x55fce8d6a420 2023-10-05T19:41:34.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.224+0000 7f214affd700 1 -- 172.21.15.12:0/2525062487 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f214c008d00 con 0x55fce8d6a420 2023-10-05T19:41:34.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.224+0000 7f215353ae40 1 -- 172.21.15.12:0/2525062487 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fce8d6a420 msgr2=0x55fce8d8b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:34.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.224+0000 7f215353ae40 1 --2- 172.21.15.12:0/2525062487 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fce8d6a420 0x55fce8d8b160 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f214c005b50 tx=0x7f214c015000 comp rx=0 tx=0).stop 2023-10-05T19:41:34.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.224+0000 7f215353ae40 1 -- 172.21.15.12:0/2525062487 shutdown_connections 2023-10-05T19:41:34.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.224+0000 7f215353ae40 1 --2- 172.21.15.12:0/2525062487 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fce8d6a420 0x55fce8d8b160 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:34.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.224+0000 7f215353ae40 1 -- 172.21.15.12:0/2525062487 >> 172.21.15.12:0/2525062487 conn(0x55fce8d64730 msgr2=0x55fce8d86d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:34.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.224+0000 7f215353ae40 1 -- 172.21.15.12:0/2525062487 shutdown_connections 2023-10-05T19:41:34.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.224+0000 7f215353ae40 1 -- 172.21.15.12:0/2525062487 wait complete. 2023-10-05T19:41:34.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.224+0000 7f215353ae40 1 Processor -- start 2023-10-05T19:41:34.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.224+0000 7f215353ae40 1 -- start start 2023-10-05T19:41:34.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.224+0000 7f215353ae40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fce8d6a420 0x55fce8d91d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:34.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.224+0000 7f215353ae40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fce8c0aed0 con 0x55fce8d6a420 2023-10-05T19:41:34.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.224+0000 7f215106f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fce8d6a420 0x55fce8d91d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:34.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.224+0000 7f215106f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fce8d6a420 0x55fce8d91d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:60844/0 (socket says 172.21.15.12:60844) 2023-10-05T19:41:34.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.224+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 learned_addr learned my addr 172.21.15.12:0/3689692740 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:34.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.224+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fce8d69ce0 con 0x55fce8d6a420 2023-10-05T19:41:34.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.228+0000 7f215106f700 1 --2- 172.21.15.12:0/3689692740 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fce8d6a420 0x55fce8d91d70 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f214c00a650 tx=0x7f214c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:34.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.228+0000 7f21497fa700 1 -- 172.21.15.12:0/3689692740 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f214c026b80 con 0x55fce8d6a420 2023-10-05T19:41:34.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.228+0000 7f21497fa700 1 -- 172.21.15.12:0/3689692740 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f214c026d00 con 0x55fce8d6a420 2023-10-05T19:41:34.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.228+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55fce8d922b0 con 0x55fce8d6a420 2023-10-05T19:41:34.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.228+0000 7f21497fa700 1 -- 172.21.15.12:0/3689692740 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f214c012450 con 0x55fce8d6a420 2023-10-05T19:41:34.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.228+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fce8d926f0 con 0x55fce8d6a420 2023-10-05T19:41:34.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.228+0000 7f21497fa700 1 -- 172.21.15.12:0/3689692740 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f214c019070 con 0x55fce8d6a420 2023-10-05T19:41:34.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.228+0000 7f215353ae40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.228+0000 7f21497fa700 1 --2- 172.21.15.12:0/3689692740 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2134030ca0 0x7f2134033150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:34.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.228+0000 7f21497fa700 1 -- 172.21.15.12:0/3689692740 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7f214c031b60 con 0x55fce8d6a420 2023-10-05T19:41:34.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.228+0000 7f215086e700 1 --2- 172.21.15.12:0/3689692740 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2134030ca0 0x7f2134033150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:34.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.228+0000 7f215353ae40 1 --2- 172.21.15.12:0/3689692740 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55fce8d6a7f0 0x55fce8d9d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:34.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.228+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8d9dee0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.228+0000 7f2151870700 1 --2- 172.21.15.12:0/3689692740 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55fce8d6a7f0 0x55fce8d9d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:34.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.228+0000 7f215086e700 1 --2- 172.21.15.12:0/3689692740 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2134030ca0 0x7f2134033150 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f213c00b300 tx=0x7f213c001470 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:34.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.232+0000 7f2151870700 1 --2- 172.21.15.12:0/3689692740 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55fce8d6a7f0 0x55fce8d9d830 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:34.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.232+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.232+0000 7f215353ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:34.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.232+0000 7f215353ae40 20 realm 2023-10-05T19:41:34.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.232+0000 7f215353ae40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.232+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8d9ae90 con 0x55fce8d6a7f0 2023-10-05T19:41:34.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.232+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.232+0000 7f215353ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:34.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.232+0000 7f215353ae40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:41:34.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.232+0000 7f215353ae40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.232+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8d9e9c0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.232+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.232+0000 7f215353ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:34.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.232+0000 7f215353ae40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.232+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8d9e5d0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.232+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.232+0000 7f215353ae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:34.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.232+0000 7f215353ae40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.232+0000 7f215353ae40 1 --2- 172.21.15.12:0/3689692740 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55fce8d9fd60 0x55fce8da2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:34.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.232+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da2680 con 0x55fce8d9fd60 2023-10-05T19:41:34.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.232+0000 7f215106f700 1 --2- 172.21.15.12:0/3689692740 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55fce8d9fd60 0x55fce8da2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:34.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.232+0000 7f215106f700 1 --2- 172.21.15.12:0/3689692740 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55fce8d9fd60 0x55fce8da2140 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:34.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.236+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f214c036070 con 0x55fce8d9fd60 2023-10-05T19:41:34.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.236+0000 7f215353ae40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:41:34.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.236+0000 7f215353ae40 20 searching for the correct realm 2023-10-05T19:41:34.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.236+0000 7f215353ae40 1 --2- 172.21.15.12:0/3689692740 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55fce8da4500 0x55fce8da6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:34.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.236+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da6fc0 con 0x55fce8da4500 2023-10-05T19:41:34.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.236+0000 7f215086e700 1 --2- 172.21.15.12:0/3689692740 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55fce8da4500 0x55fce8da6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:34.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.236+0000 7f215086e700 1 --2- 172.21.15.12:0/3689692740 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55fce8da4500 0x55fce8da6910 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:34.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.236+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f213c00f560 con 0x55fce8da4500 2023-10-05T19:41:34.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.236+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da6fc0 con 0x55fce8d9fd60 2023-10-05T19:41:34.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.240+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f214c036070 con 0x55fce8d9fd60 2023-10-05T19:41:34.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.240+0000 7f215106f700 1 --2- 172.21.15.12:0/3689692740 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f214c039030 0x7f214c01b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:34.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.240+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da6fc0 con 0x7f214c039030 2023-10-05T19:41:34.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.240+0000 7f2151870700 1 --2- 172.21.15.12:0/3689692740 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f214c039030 0x7f214c01b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:34.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.240+0000 7f2151870700 1 --2- 172.21.15.12:0/3689692740 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f214c039030 0x7f214c01b9b0 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:34.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.240+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f214400a5a0 con 0x7f214c039030 2023-10-05T19:41:34.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.240+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da6fc0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.240+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.240+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da6fc0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.244+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.244+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da6fc0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.244+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(11 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.244+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da6fc0 con 0x7f214c039030 2023-10-05T19:41:34.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.244+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f214400a5a0 con 0x7f214c039030 2023-10-05T19:41:34.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.244+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da6fc0 con 0x55fce8d9fd60 2023-10-05T19:41:34.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.244+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f214c036070 con 0x55fce8d9fd60 2023-10-05T19:41:34.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.244+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da6fc0 con 0x55fce8d9fd60 2023-10-05T19:41:34.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.244+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f214c036070 con 0x55fce8d9fd60 2023-10-05T19:41:34.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.244+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da6fc0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.244+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.244+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da6fc0 con 0x7f214c039030 2023-10-05T19:41:34.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.248+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f214400a5a0 con 0x7f214c039030 2023-10-05T19:41:34.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.248+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da6fc0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.248+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.248+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da6fc0 con 0x55fce8d9fd60 2023-10-05T19:41:34.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.248+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(18 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f214c036070 con 0x55fce8d9fd60 2023-10-05T19:41:34.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.248+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da6fc0 con 0x55fce8da4500 2023-10-05T19:41:34.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.248+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f213c00f560 con 0x55fce8da4500 2023-10-05T19:41:34.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.248+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da6fc0 con 0x55fce8da4500 2023-10-05T19:41:34.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.252+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f213c00f560 con 0x55fce8da4500 2023-10-05T19:41:34.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.252+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da6fc0 con 0x7f214c039030 2023-10-05T19:41:34.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.252+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f214400a5a0 con 0x7f214c039030 2023-10-05T19:41:34.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.252+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da6fc0 con 0x55fce8da4500 2023-10-05T19:41:34.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.252+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f213c00f560 con 0x55fce8da4500 2023-10-05T19:41:34.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.252+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da6fc0 con 0x55fce8da4500 2023-10-05T19:41:34.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.252+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f213c00f560 con 0x55fce8da4500 2023-10-05T19:41:34.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.252+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da6fc0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.252+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.252+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da6fc0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.256+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.256+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da6fc0 con 0x7f214c039030 2023-10-05T19:41:34.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.256+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f214400a5a0 con 0x7f214c039030 2023-10-05T19:41:34.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.256+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da6fc0 con 0x55fce8da4500 2023-10-05T19:41:34.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.256+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f213c00f560 con 0x55fce8da4500 2023-10-05T19:41:34.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.256+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da6fc0 con 0x55fce8d9fd60 2023-10-05T19:41:34.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.256+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f214c036070 con 0x55fce8d9fd60 2023-10-05T19:41:34.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.256+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da6fc0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.256+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.256+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da6fc0 con 0x7f214c039030 2023-10-05T19:41:34.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.260+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f214400a5a0 con 0x7f214c039030 2023-10-05T19:41:34.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.260+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da6fc0 con 0x55fce8da4500 2023-10-05T19:41:34.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.260+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f213c00f560 con 0x55fce8da4500 2023-10-05T19:41:34.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.260+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da6fc0 con 0x7f214c039030 2023-10-05T19:41:34.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.260+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f214400a5a0 con 0x7f214c039030 2023-10-05T19:41:34.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.260+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da6fc0 con 0x55fce8d9fd60 2023-10-05T19:41:34.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.260+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f214c036070 con 0x55fce8d9fd60 2023-10-05T19:41:34.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.260+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da6fc0 con 0x55fce8da4500 2023-10-05T19:41:34.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.264+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f213c00f560 con 0x55fce8da4500 2023-10-05T19:41:34.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.264+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da6fc0 con 0x7f214c039030 2023-10-05T19:41:34.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.264+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f214400a5a0 con 0x7f214c039030 2023-10-05T19:41:34.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.264+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da6fc0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.264+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.264+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da6fc0 con 0x55fce8d9fd60 2023-10-05T19:41:34.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.264+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(37 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f214c036070 con 0x55fce8d9fd60 2023-10-05T19:41:34.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.264+0000 7f215353ae40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:41:34.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.264+0000 7f215353ae40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:41:34.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.264+0000 7f215353ae40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:41:34.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.264+0000 7f215353ae40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:41:34.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.264+0000 7f215353ae40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.264+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da3250 con 0x55fce8d6a7f0 2023-10-05T19:41:34.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.268+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.268+0000 7f215353ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:34.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.268+0000 7f215353ae40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.268+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da2e80 con 0x55fce8d6a7f0 2023-10-05T19:41:34.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.268+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.268+0000 7f215353ae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:34.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.268+0000 7f215353ae40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.268+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da7430 con 0x55fce8da4500 2023-10-05T19:41:34.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.268+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f213c00f560 con 0x55fce8da4500 2023-10-05T19:41:34.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.268+0000 7f215353ae40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:41:34.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.268+0000 7f215353ae40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.268+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da77e0 con 0x55fce8da4500 2023-10-05T19:41:34.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.268+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f213c00f560 con 0x55fce8da4500 2023-10-05T19:41:34.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.268+0000 7f215353ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:34.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.268+0000 7f215353ae40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.268+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da7b90 con 0x55fce8d6a7f0 2023-10-05T19:41:34.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.268+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.268+0000 7f215353ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:34.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.268+0000 7f215353ae40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.268+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da3770 con 0x55fce8d6a7f0 2023-10-05T19:41:34.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.272+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.272+0000 7f215353ae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:34.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.272+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da3770 con 0x55fce8da4500 2023-10-05T19:41:34.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.272+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(44 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f213c027080 con 0x55fce8da4500 2023-10-05T19:41:34.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.272+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da3770 con 0x55fce8d9fd60 2023-10-05T19:41:34.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.272+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(45 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f214c036070 con 0x55fce8d9fd60 2023-10-05T19:41:34.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.272+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da3770 con 0x7f214c039030 2023-10-05T19:41:34.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.276+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(46 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f214400a5a0 con 0x7f214c039030 2023-10-05T19:41:34.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.276+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da7b90 con 0x55fce8d6a7f0 2023-10-05T19:41:34.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.276+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(47 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.276+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da7b90 con 0x55fce8d6a7f0 2023-10-05T19:41:34.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.276+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(48 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.276+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da7b90 con 0x55fce8d6a7f0 2023-10-05T19:41:34.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.276+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(49 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.276+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da7b90 con 0x7f214c039030 2023-10-05T19:41:34.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.280+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(50 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f214400a5a0 con 0x7f214c039030 2023-10-05T19:41:34.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.280+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da7b90 con 0x55fce8d9fd60 2023-10-05T19:41:34.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.280+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(51 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f214c036070 con 0x55fce8d9fd60 2023-10-05T19:41:34.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.280+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da7b90 con 0x55fce8d9fd60 2023-10-05T19:41:34.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.280+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(52 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f214c036070 con 0x55fce8d9fd60 2023-10-05T19:41:34.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.280+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da7b90 con 0x55fce8d6a7f0 2023-10-05T19:41:34.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.280+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(53 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.280+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da7b90 con 0x7f214c039030 2023-10-05T19:41:34.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.284+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(54 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f214400a5a0 con 0x7f214c039030 2023-10-05T19:41:34.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.284+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da7b90 con 0x55fce8d6a7f0 2023-10-05T19:41:34.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.284+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(55 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.284+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da7b90 con 0x55fce8d9fd60 2023-10-05T19:41:34.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.284+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(56 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f214c036070 con 0x55fce8d9fd60 2023-10-05T19:41:34.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.284+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da7b90 con 0x55fce8da4500 2023-10-05T19:41:34.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.284+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(57 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f213c027080 con 0x55fce8da4500 2023-10-05T19:41:34.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.284+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da7b90 con 0x55fce8da4500 2023-10-05T19:41:34.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.288+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(58 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f213c027080 con 0x55fce8da4500 2023-10-05T19:41:34.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.288+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da7b90 con 0x7f214c039030 2023-10-05T19:41:34.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.288+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(59 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f214400a5a0 con 0x7f214c039030 2023-10-05T19:41:34.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.288+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da7b90 con 0x55fce8da4500 2023-10-05T19:41:34.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.288+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(60 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f213c027080 con 0x55fce8da4500 2023-10-05T19:41:34.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.288+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da7b90 con 0x55fce8da4500 2023-10-05T19:41:34.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.288+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(61 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f213c027080 con 0x55fce8da4500 2023-10-05T19:41:34.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.288+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da7b90 con 0x55fce8d6a7f0 2023-10-05T19:41:34.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.292+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(62 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.292+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da7b90 con 0x55fce8d6a7f0 2023-10-05T19:41:34.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.292+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(63 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.292+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da7b90 con 0x7f214c039030 2023-10-05T19:41:34.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.292+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(64 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f214400a5a0 con 0x7f214c039030 2023-10-05T19:41:34.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.292+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da7b90 con 0x55fce8da4500 2023-10-05T19:41:34.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.292+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(65 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f213c027080 con 0x55fce8da4500 2023-10-05T19:41:34.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.292+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da7b90 con 0x55fce8d9fd60 2023-10-05T19:41:34.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.296+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(66 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f214c036070 con 0x55fce8d9fd60 2023-10-05T19:41:34.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.296+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da7b90 con 0x55fce8d6a7f0 2023-10-05T19:41:34.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.296+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(67 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.296+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da7b90 con 0x7f214c039030 2023-10-05T19:41:34.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.296+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(68 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f214400a5a0 con 0x7f214c039030 2023-10-05T19:41:34.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.296+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da7b90 con 0x55fce8da4500 2023-10-05T19:41:34.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.296+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(69 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f213c027080 con 0x55fce8da4500 2023-10-05T19:41:34.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.296+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da7b90 con 0x7f214c039030 2023-10-05T19:41:34.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.300+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(70 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f214400a5a0 con 0x7f214c039030 2023-10-05T19:41:34.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.300+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da7b90 con 0x55fce8d9fd60 2023-10-05T19:41:34.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.300+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(71 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f214c036070 con 0x55fce8d9fd60 2023-10-05T19:41:34.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.300+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da7b90 con 0x55fce8da4500 2023-10-05T19:41:34.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.300+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(72 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f213c027080 con 0x55fce8da4500 2023-10-05T19:41:34.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.300+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da7b90 con 0x7f214c039030 2023-10-05T19:41:34.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.300+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(73 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f214400a5a0 con 0x7f214c039030 2023-10-05T19:41:34.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.300+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da7b90 con 0x55fce8d6a7f0 2023-10-05T19:41:34.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.304+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(74 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.304+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da7b90 con 0x55fce8d9fd60 2023-10-05T19:41:34.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.304+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(75 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f214c036070 con 0x55fce8d9fd60 2023-10-05T19:41:34.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.304+0000 7f215353ae40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:41:34.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.304+0000 7f215353ae40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:41:34.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.304+0000 7f215353ae40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:41:34.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.304+0000 7f215353ae40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:41:34.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.304+0000 7f215353ae40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.304+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da3770 con 0x55fce8d6a7f0 2023-10-05T19:41:34.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.304+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.304+0000 7f215353ae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:34.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.304+0000 7f215353ae40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.304+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da84e0 con 0x55fce8d9fd60 2023-10-05T19:41:34.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.304+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f214c036070 con 0x55fce8d9fd60 2023-10-05T19:41:34.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.304+0000 7f215353ae40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:41:34.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.304+0000 7f215353ae40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.304+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da8dd0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.304+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.304+0000 7f215353ae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:34.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.304+0000 7f215353ae40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.304+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da8dd0 con 0x55fce8da4500 2023-10-05T19:41:34.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.308+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f213c027080 con 0x55fce8da4500 2023-10-05T19:41:34.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.308+0000 7f215353ae40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:41:34.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.308+0000 7f215353ae40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.308+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da9540 con 0x7f214c039030 2023-10-05T19:41:34.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.308+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f214400a5a0 con 0x7f214c039030 2023-10-05T19:41:34.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.308+0000 7f215353ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:34.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.308+0000 7f215353ae40 20 zone default found 2023-10-05T19:41:34.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.308+0000 7f215353ae40 4 Realm: () 2023-10-05T19:41:34.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.308+0000 7f215353ae40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:41:34.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.308+0000 7f215353ae40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:41:34.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.308+0000 7f215353ae40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:41:34.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.308+0000 7f215353ae40 20 zonegroup default 2023-10-05T19:41:34.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.308+0000 7f215353ae40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.308+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8da9540 con 0x7f214c039030 2023-10-05T19:41:34.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.308+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f214400a5a0 con 0x7f214c039030 2023-10-05T19:41:34.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.308+0000 7f215353ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:34.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.308+0000 7f215353ae40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.308+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8daaec0 con 0x55fce8da4500 2023-10-05T19:41:34.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.308+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f213c027080 con 0x55fce8da4500 2023-10-05T19:41:34.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.308+0000 7f215353ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:34.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.308+0000 7f215353ae40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.308+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8dab270 con 0x55fce8da4500 2023-10-05T19:41:34.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.312+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f213c027080 con 0x55fce8da4500 2023-10-05T19:41:34.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.312+0000 7f215353ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:34.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.312+0000 7f215353ae40 20 started sync module instance, tier type = 2023-10-05T19:41:34.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.312+0000 7f215353ae40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:41:34.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.312+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8daa580 con 0x55fce8d9fd60 2023-10-05T19:41:34.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.312+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(84 notify.0 [create] v30'252 uv252 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f214c036070 con 0x55fce8d9fd60 2023-10-05T19:41:34.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.312+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94544726755552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8dab620 con 0x55fce8d9fd60 2023-10-05T19:41:34.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.312+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8dabec0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.316+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(85 notify.0 [watch watch cookie 94544726755552] v30'253 uv252 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f214c036070 con 0x55fce8d9fd60 2023-10-05T19:41:34.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.316+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(86 notify.1 [create] v30'252 uv252 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.316+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94544726768080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8dac310 con 0x55fce8d6a7f0 2023-10-05T19:41:34.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.316+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8dacbb0 con 0x55fce8da4500 2023-10-05T19:41:34.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.320+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(88 notify.2 [create] v30'252 uv252 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f213c027080 con 0x55fce8da4500 2023-10-05T19:41:34.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.320+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94544726770736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8dad450 con 0x55fce8da4500 2023-10-05T19:41:34.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.320+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8dade60 con 0x55fce8d9fd60 2023-10-05T19:41:34.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.320+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(87 notify.1 [watch watch cookie 94544726768080] v30'253 uv252 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.324+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(90 notify.3 [create] v30'252 uv252 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f214c036070 con 0x55fce8d9fd60 2023-10-05T19:41:34.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.324+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94544726769184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8dada90 con 0x55fce8d9fd60 2023-10-05T19:41:34.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.324+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8daedc0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.324+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(89 notify.2 [watch watch cookie 94544726770736] v30'253 uv252 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f213c027080 con 0x55fce8da4500 2023-10-05T19:41:34.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.324+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(91 notify.3 [watch watch cookie 94544726769184] v30'253 uv252 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f214c036070 con 0x55fce8d9fd60 2023-10-05T19:41:34.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.324+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(92 notify.4 [create] v30'252 uv252 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.324+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94544726778384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8dae8d0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.324+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8daf6b0 con 0x7f214c039030 2023-10-05T19:41:34.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.328+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(93 notify.4 [watch watch cookie 94544726778384] v30'253 uv252 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.328+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(94 notify.5 [create] v30'252 uv252 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f214400a5a0 con 0x7f214c039030 2023-10-05T19:41:34.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.328+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94544726773600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8daf3f0 con 0x7f214c039030 2023-10-05T19:41:34.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.328+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8dafc90 con 0x7f214c039030 2023-10-05T19:41:34.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.332+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(95 notify.5 [watch watch cookie 94544726773600] v30'253 uv252 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f214400a5a0 con 0x7f214c039030 2023-10-05T19:41:34.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.332+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(96 notify.6 [create] v30'252 uv252 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f214400a5a0 con 0x7f214c039030 2023-10-05T19:41:34.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.332+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94544726781968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8daf990 con 0x7f214c039030 2023-10-05T19:41:34.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.332+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8db04f0 con 0x7f214c039030 2023-10-05T19:41:34.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.332+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(97 notify.6 [watch watch cookie 94544726781968] v30'253 uv252 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f214400a5a0 con 0x7f214c039030 2023-10-05T19:41:34.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.336+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(98 notify.7 [create] v30'252 uv252 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f214400a5a0 con 0x7f214c039030 2023-10-05T19:41:34.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.336+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94544726783072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8db04b0 con 0x7f214c039030 2023-10-05T19:41:34.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.336+0000 7f215353ae40 20 add_watcher() i=0 2023-10-05T19:41:34.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.336+0000 7f215353ae40 20 add_watcher() i=1 2023-10-05T19:41:34.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.336+0000 7f215353ae40 20 add_watcher() i=2 2023-10-05T19:41:34.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.336+0000 7f215353ae40 20 add_watcher() i=3 2023-10-05T19:41:34.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.336+0000 7f215353ae40 20 add_watcher() i=4 2023-10-05T19:41:34.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.336+0000 7f215353ae40 20 add_watcher() i=5 2023-10-05T19:41:34.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.336+0000 7f215353ae40 20 add_watcher() i=6 2023-10-05T19:41:34.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.336+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(99 notify.7 [watch watch cookie 94544726783072] v30'253 uv252 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f214400a5a0 con 0x7f214c039030 2023-10-05T19:41:34.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.336+0000 7f215353ae40 20 add_watcher() i=7 2023-10-05T19:41:34.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.336+0000 7f215353ae40 2 all 8 watchers are set, enabling cache 2023-10-05T19:41:34.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.336+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8db36c0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.336+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.336+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94544726786352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8db0a70 con 0x55fce8d6a7f0 2023-10-05T19:41:34.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.340+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94544726786352] v30'239 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.340+0000 7f21227e4700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:41:34.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.340+0000 7f215353ae40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:41:34.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.340+0000 7f215353ae40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:41:34.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.340+0000 7f215353ae40 1 Processor -- start 2023-10-05T19:41:34.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.340+0000 7f215353ae40 1 -- start start 2023-10-05T19:41:34.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.340+0000 7f215353ae40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fce8dbdee0 0x55fce8dbe2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:34.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.340+0000 7f215353ae40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fce8dbe7f0 con 0x55fce8dbdee0 2023-10-05T19:41:34.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.340+0000 7f215086e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fce8dbdee0 0x55fce8dbe2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:34.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.340+0000 7f215086e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fce8dbdee0 0x55fce8dbe2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:60858/0 (socket says 172.21.15.12:60858) 2023-10-05T19:41:34.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.340+0000 7f215086e700 1 -- 172.21.15.12:0/827178420 learned_addr learned my addr 172.21.15.12:0/827178420 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:34.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.344+0000 7f215086e700 1 -- 172.21.15.12:0/827178420 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fce8dbe930 con 0x55fce8dbdee0 2023-10-05T19:41:34.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.344+0000 7f215086e700 1 --2- 172.21.15.12:0/827178420 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fce8dbdee0 0x55fce8dbe2b0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f214c000c00 tx=0x7f213c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b55aeab3f269dc4 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:34.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.344+0000 7f2121fe3700 1 -- 172.21.15.12:0/827178420 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f213c040070 con 0x55fce8dbdee0 2023-10-05T19:41:34.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.344+0000 7f2121fe3700 1 -- 172.21.15.12:0/827178420 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f213c025040 con 0x55fce8dbdee0 2023-10-05T19:41:34.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.344+0000 7f2121fe3700 1 -- 172.21.15.12:0/827178420 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f213c024040 con 0x55fce8dbdee0 2023-10-05T19:41:34.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.344+0000 7f215353ae40 1 -- 172.21.15.12:0/827178420 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fce8dbdee0 msgr2=0x55fce8dbe2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:34.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.344+0000 7f215353ae40 1 --2- 172.21.15.12:0/827178420 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fce8dbdee0 0x55fce8dbe2b0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f214c000c00 tx=0x7f213c010000 comp rx=0 tx=0).stop 2023-10-05T19:41:34.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.344+0000 7f215353ae40 1 -- 172.21.15.12:0/827178420 shutdown_connections 2023-10-05T19:41:34.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.344+0000 7f215353ae40 1 --2- 172.21.15.12:0/827178420 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fce8dbdee0 0x55fce8dbe2b0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:34.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.344+0000 7f215353ae40 1 -- 172.21.15.12:0/827178420 >> 172.21.15.12:0/827178420 conn(0x55fce8db92a0 msgr2=0x55fce8db73f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:34.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.344+0000 7f215353ae40 1 -- 172.21.15.12:0/827178420 shutdown_connections 2023-10-05T19:41:34.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.344+0000 7f215353ae40 1 -- 172.21.15.12:0/827178420 wait complete. 2023-10-05T19:41:34.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.344+0000 7f215353ae40 1 Processor -- start 2023-10-05T19:41:34.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.344+0000 7f215353ae40 1 -- start start 2023-10-05T19:41:34.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.344+0000 7f215353ae40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fce8dbdee0 0x55fce8dc39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:34.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.344+0000 7f215353ae40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fce8dc49c0 con 0x55fce8dbdee0 2023-10-05T19:41:34.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.348+0000 7f215086e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fce8dbdee0 0x55fce8dc39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:34.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.348+0000 7f215086e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fce8dbdee0 0x55fce8dc39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:60866/0 (socket says 172.21.15.12:60866) 2023-10-05T19:41:34.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.348+0000 7f215086e700 1 -- 172.21.15.12:0/1802509420 learned_addr learned my addr 172.21.15.12:0/1802509420 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:34.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.348+0000 7f215086e700 1 -- 172.21.15.12:0/1802509420 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f213c006070 con 0x55fce8dbdee0 2023-10-05T19:41:34.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.348+0000 7f215086e700 1 --2- 172.21.15.12:0/1802509420 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fce8dbdee0 0x55fce8dc39c0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f213c000fd0 tx=0x7f213c03c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:34.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.348+0000 7f2093fff700 1 -- 172.21.15.12:0/1802509420 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f213c024040 con 0x55fce8dbdee0 2023-10-05T19:41:34.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.348+0000 7f2093fff700 1 -- 172.21.15.12:0/1802509420 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f213c0398a0 con 0x55fce8dbdee0 2023-10-05T19:41:34.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.348+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55fce8dc3fc0 con 0x55fce8dbdee0 2023-10-05T19:41:34.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.348+0000 7f2093fff700 1 -- 172.21.15.12:0/1802509420 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f213c039a50 con 0x55fce8dbdee0 2023-10-05T19:41:34.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.348+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fce8dc4460 con 0x55fce8dbdee0 2023-10-05T19:41:34.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.348+0000 7f2093fff700 1 -- 172.21.15.12:0/1802509420 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f213c043020 con 0x55fce8dbdee0 2023-10-05T19:41:34.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.348+0000 7f2093fff700 1 --2- 172.21.15.12:0/1802509420 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2098030c00 0x7f20980330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:34.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.348+0000 7f2093fff700 1 -- 172.21.15.12:0/1802509420 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7f213c022040 con 0x55fce8dbdee0 2023-10-05T19:41:34.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.348+0000 7f2151870700 1 --2- 172.21.15.12:0/1802509420 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2098030c00 0x7f20980330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:34.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.348+0000 7f215353ae40 1 --2- 172.21.15.12:0/1802509420 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55fce8dce5e0 0x55fce8dd09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:34.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.348+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8dd1090 con 0x55fce8dce5e0 2023-10-05T19:41:34.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.352+0000 7f215106f700 1 --2- 172.21.15.12:0/1802509420 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55fce8dce5e0 0x55fce8dd09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:34.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.352+0000 7f2151870700 1 --2- 172.21.15.12:0/1802509420 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2098030c00 0x7f20980330b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x55fce8db37d0 tx=0x7f214400aed0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:34.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.352+0000 7f215106f700 1 --2- 172.21.15.12:0/1802509420 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55fce8dce5e0 0x55fce8dd09f0 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:34.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.352+0000 7f215106f700 1 -- 172.21.15.12:0/1802509420 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v30'35 uv30 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f214c036070 con 0x55fce8dce5e0 2023-10-05T19:41:34.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.352+0000 7f215353ae40 1 --2- 172.21.15.12:0/1802509420 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55fce8dd1d60 0x55fce8dd4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:34.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.352+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8dd4870 con 0x55fce8dd1d60 2023-10-05T19:41:34.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.356+0000 7f215086e700 1 --2- 172.21.15.12:0/1802509420 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55fce8dd1d60 0x55fce8dd4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:34.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.356+0000 7f215086e700 1 --2- 172.21.15.12:0/1802509420 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55fce8dd1d60 0x55fce8dd4140 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:34.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.360+0000 7f215086e700 1 -- 172.21.15.12:0/1802509420 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v30'26 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f213c027080 con 0x55fce8dd1d60 2023-10-05T19:41:34.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.360+0000 7f215353ae40 1 --2- 172.21.15.12:0/1802509420 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55fce8dd5540 0x55fce8dd7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:34.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.360+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8dd8080 con 0x55fce8dd5540 2023-10-05T19:41:34.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.360+0000 7f2151870700 1 --2- 172.21.15.12:0/1802509420 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55fce8dd5540 0x55fce8dd7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:34.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.360+0000 7f2151870700 1 --2- 172.21.15.12:0/1802509420 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55fce8dd5540 0x55fce8dd7950 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:34.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.360+0000 7f2151870700 1 -- 172.21.15.12:0/1802509420 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v30'26 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8dd5540 2023-10-05T19:41:34.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.360+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8dd86d0 con 0x55fce8dce5e0 2023-10-05T19:41:34.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.364+0000 7f215106f700 1 -- 172.21.15.12:0/1802509420 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v30'240 uv185 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f214c036070 con 0x55fce8dce5e0 2023-10-05T19:41:34.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.364+0000 7f215353ae40 1 --2- 172.21.15.12:0/1802509420 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55fce8dd9370 0x55fce8ddb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:34.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.364+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8ddbdb0 con 0x55fce8dd9370 2023-10-05T19:41:34.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.364+0000 7f215106f700 1 --2- 172.21.15.12:0/1802509420 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55fce8dd9370 0x55fce8ddb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:34.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.364+0000 7f215106f700 1 --2- 172.21.15.12:0/1802509420 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55fce8dd9370 0x55fce8ddb750 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:34.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.368+0000 7f215106f700 1 -- 172.21.15.12:0/1802509420 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v30'68 uv51 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f214c036070 con 0x55fce8dd9370 2023-10-05T19:41:34.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.368+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8ddc400 con 0x55fce8dd5540 2023-10-05T19:41:34.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.372+0000 7f2151870700 1 -- 172.21.15.12:0/1802509420 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v30'48 uv39 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8dd5540 2023-10-05T19:41:34.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.372+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8ddca20 con 0x55fce8dce5e0 2023-10-05T19:41:34.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.376+0000 7f215106f700 1 -- 172.21.15.12:0/1802509420 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v30'28 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f214c036070 con 0x55fce8dce5e0 2023-10-05T19:41:34.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.376+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8ddca20 con 0x55fce8dd1d60 2023-10-05T19:41:34.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.376+0000 7f215086e700 1 -- 172.21.15.12:0/1802509420 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v30'45 uv36 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f213c027080 con 0x55fce8dd1d60 2023-10-05T19:41:34.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.376+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8dc9ce0 con 0x55fce8dd9370 2023-10-05T19:41:34.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.380+0000 7f215106f700 1 -- 172.21.15.12:0/1802509420 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v30'26 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f214c036070 con 0x55fce8dd9370 2023-10-05T19:41:34.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.380+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8dc9ce0 con 0x55fce8dd5540 2023-10-05T19:41:34.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.384+0000 7f2151870700 1 -- 172.21.15.12:0/1802509420 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v30'32 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8dd5540 2023-10-05T19:41:34.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.384+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8dca390 con 0x55fce8dd9370 2023-10-05T19:41:34.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.384+0000 7f215106f700 1 -- 172.21.15.12:0/1802509420 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v30'69 uv53 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f214c036070 con 0x55fce8dd9370 2023-10-05T19:41:34.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.384+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8dca390 con 0x55fce8dd1d60 2023-10-05T19:41:34.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.388+0000 7f215086e700 1 -- 172.21.15.12:0/1802509420 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v30'33 uv24 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f213c027080 con 0x55fce8dd1d60 2023-10-05T19:41:34.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.388+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8ddda30 con 0x55fce8dd9370 2023-10-05T19:41:34.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.392+0000 7f215106f700 1 -- 172.21.15.12:0/1802509420 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v30'41 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f214c036070 con 0x55fce8dd9370 2023-10-05T19:41:34.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.392+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8ddda30 con 0x55fce8dd1d60 2023-10-05T19:41:34.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.392+0000 7f215086e700 1 -- 172.21.15.12:0/1802509420 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v30'54 uv45 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f213c027080 con 0x55fce8dd1d60 2023-10-05T19:41:34.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.392+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8dddf20 con 0x55fce8dce5e0 2023-10-05T19:41:34.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.396+0000 7f215106f700 1 -- 172.21.15.12:0/1802509420 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v30'241 uv187 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f214c036070 con 0x55fce8dce5e0 2023-10-05T19:41:34.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.396+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8dddf20 con 0x55fce8dd5540 2023-10-05T19:41:34.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.400+0000 7f2151870700 1 -- 172.21.15.12:0/1802509420 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v30'68 uv49 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8dd5540 2023-10-05T19:41:34.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.400+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8dddf20 con 0x55fce8dd9370 2023-10-05T19:41:34.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.400+0000 7f215106f700 1 -- 172.21.15.12:0/1802509420 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v30'70 uv55 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f214c036070 con 0x55fce8dd9370 2023-10-05T19:41:34.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.400+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8dddf20 con 0x55fce8dd5540 2023-10-05T19:41:34.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.404+0000 7f2151870700 1 -- 172.21.15.12:0/1802509420 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v30'26 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8dd5540 2023-10-05T19:41:34.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.404+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8dddf20 con 0x55fce8dce5e0 2023-10-05T19:41:34.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.408+0000 7f215106f700 1 -- 172.21.15.12:0/1802509420 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v30'50 uv41 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f214c036070 con 0x55fce8dce5e0 2023-10-05T19:41:34.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.408+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8dddf20 con 0x55fce8dce5e0 2023-10-05T19:41:34.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.412+0000 7f215106f700 1 -- 172.21.15.12:0/1802509420 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v30'242 uv183 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f214c036070 con 0x55fce8dce5e0 2023-10-05T19:41:34.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.412+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8dddf20 con 0x55fce8dd5540 2023-10-05T19:41:34.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.412+0000 7f2151870700 1 -- 172.21.15.12:0/1802509420 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v30'69 uv45 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8dd5540 2023-10-05T19:41:34.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.412+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8dddfb0 con 0x55fce8dd9370 2023-10-05T19:41:34.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.416+0000 7f215106f700 1 -- 172.21.15.12:0/1802509420 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v30'36 uv31 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f214c036070 con 0x55fce8dd9370 2023-10-05T19:41:34.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.416+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8dddfb0 con 0x55fce8dd5540 2023-10-05T19:41:34.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.420+0000 7f2151870700 1 -- 172.21.15.12:0/1802509420 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v30'30 uv25 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8dd5540 2023-10-05T19:41:34.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.420+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8dddfb0 con 0x55fce8dd5540 2023-10-05T19:41:34.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.420+0000 7f2151870700 1 -- 172.21.15.12:0/1802509420 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v30'35 uv30 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8dd5540 2023-10-05T19:41:34.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.420+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8dddfb0 con 0x55fce8dd5540 2023-10-05T19:41:34.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.424+0000 7f2151870700 1 -- 172.21.15.12:0/1802509420 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v30'29 uv18 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8dd5540 2023-10-05T19:41:34.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.424+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8dddfb0 con 0x55fce8dd5540 2023-10-05T19:41:34.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.428+0000 7f2151870700 1 -- 172.21.15.12:0/1802509420 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v30'49 uv37 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8dd5540 2023-10-05T19:41:34.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.428+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8dddfb0 con 0x55fce8dd1d60 2023-10-05T19:41:34.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.432+0000 7f215086e700 1 -- 172.21.15.12:0/1802509420 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v30'32 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f213c027080 con 0x55fce8dd1d60 2023-10-05T19:41:34.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.432+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8dddfb0 con 0x55fce8dd5540 2023-10-05T19:41:34.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.432+0000 7f2151870700 1 -- 172.21.15.12:0/1802509420 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v30'26 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8dd5540 2023-10-05T19:41:34.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.432+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8dddfb0 con 0x55fce8dd1d60 2023-10-05T19:41:34.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.436+0000 7f215086e700 1 -- 172.21.15.12:0/1802509420 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v30'46 uv34 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f213c027080 con 0x55fce8dd1d60 2023-10-05T19:41:34.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.436+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8dddfb0 con 0x55fce8dd5540 2023-10-05T19:41:34.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.440+0000 7f2151870700 1 -- 172.21.15.12:0/1802509420 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v30'70 uv47 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8dd5540 2023-10-05T19:41:34.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.440+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8dddfb0 con 0x55fce8dd1d60 2023-10-05T19:41:34.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.444+0000 7f215086e700 1 -- 172.21.15.12:0/1802509420 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v30'55 uv43 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f213c027080 con 0x55fce8dd1d60 2023-10-05T19:41:34.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.444+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8dddfb0 con 0x55fce8dd9370 2023-10-05T19:41:34.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.444+0000 7f215106f700 1 -- 172.21.15.12:0/1802509420 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v30'32 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f214c036070 con 0x55fce8dd9370 2023-10-05T19:41:34.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.448+0000 7f20917fa700 20 reqs_thread_entry: start 2023-10-05T19:41:34.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.448+0000 7f215353ae40 20 init_complete bucket index max shards: 11 2023-10-05T19:41:34.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.448+0000 7f208affd700 20 reqs_thread_entry: start 2023-10-05T19:41:34.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.448+0000 7f215353ae40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:41:34.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.448+0000 7f2088ff9700 20 reqs_thread_entry: start 2023-10-05T19:41:34.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.448+0000 7f20897fa700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:42:04 2023 2023-10-05T19:41:34.455 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:41:34.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.448+0000 7f215353ae40 20 get_system_obj_state: rctx=0x7fff51ca7df8 obj=default.rgw.meta:users.uid:bud state=0x55fce8c430b0 s->prefetch_data=0 2023-10-05T19:41:34.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.448+0000 7f215353ae40 10 cache get: name=default.rgw.meta+users.uid+bud : miss 2023-10-05T19:41:34.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.448+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:102 8.4 8:267425a8:users.uid::bud:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e3c8b0 con 0x55fce8d9fd60 2023-10-05T19:41:34.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.452+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(102 bud [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7f214c036070 con 0x55fce8d9fd60 2023-10-05T19:41:34.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.452+0000 7f215353ae40 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x16 2023-10-05T19:41:34.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.452+0000 7f215353ae40 10 adding default.rgw.meta+users.uid+bud to cache LRU end 2023-10-05T19:41:34.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.452+0000 7f215353ae40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:41:34.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.452+0000 7f215353ae40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:41:34.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.452+0000 7f215353ae40 20 Read xattr: user.rgw.idtag 2023-10-05T19:41:34.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.452+0000 7f215353ae40 10 cache get: name=default.rgw.meta+users.uid+bud : type miss (requested=0x13, cached=0x16) 2023-10-05T19:41:34.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.452+0000 7f215353ae40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.452+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:103 8.4 8:267425a8:users.uid::bud:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e3f340 con 0x55fce8d9fd60 2023-10-05T19:41:34.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.452+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(103 bud [call,call out=48b,read 0~346 out=346b,getxattrs out=75b] v0'0 uv2 ondisk = 0) v8 ==== 273+0+469 (crc 0 0 0) 0x7f214c036070 con 0x55fce8d9fd60 2023-10-05T19:41:34.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.452+0000 7f215353ae40 20 rados_obj.operate() r=0 bl.length=346 2023-10-05T19:41:34.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.452+0000 7f215353ae40 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x13 2023-10-05T19:41:34.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.452+0000 7f215353ae40 10 moving default.rgw.meta+users.uid+bud to cache LRU end 2023-10-05T19:41:34.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.452+0000 7f215353ae40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:41:34.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.452+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:104 8.19 8:99adfd5d:users.uid::bud.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e3f340 con 0x55fce8d6a7f0 2023-10-05T19:41:34.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.452+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(104 bud.buckets [call out=184b] v0'0 uv6 ondisk = 0) v8 ==== 155+0+184 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.452+0000 7f215353ae40 20 get_system_obj_state: rctx=0x7fff51ca7aa0 obj=default.rgw.meta:root:.bucket.meta.myfoo6:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7 state=0x55fce8c430b0 s->prefetch_data=0 2023-10-05T19:41:34.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.452+0000 7f215353ae40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo6:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7 : miss 2023-10-05T19:41:34.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.452+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:105 8.9 8:9736f005:root::.bucket.meta.myfoo6%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e3eb50 con 0x55fce8d9fd60 2023-10-05T19:41:34.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.452+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 <== osd.2 v2:172.21.15.12:6800/29934 23 ==== osd_op_reply(105 .bucket.meta.myfoo6:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7 [call out=48b,getxattrs out=212b,stat out=16b] v0'0 uv43 ondisk = 0) v8 ==== 291+0+276 (crc 0 0 0) 0x7f214c036070 con 0x55fce8d9fd60 2023-10-05T19:41:34.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.452+0000 7f215353ae40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7 info.flags=0x16 2023-10-05T19:41:34.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.452+0000 7f215353ae40 10 adding default.rgw.meta+root+.bucket.meta.myfoo6:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7 to cache LRU end 2023-10-05T19:41:34.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.452+0000 7f215353ae40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:41:34.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.452+0000 7f215353ae40 10 updating xattr: name=user.rgw.acl bl.length()=117 2023-10-05T19:41:34.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.452+0000 7f215353ae40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:41:34.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.452+0000 7f215353ae40 20 Read xattr: user.rgw.acl 2023-10-05T19:41:34.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.452+0000 7f215353ae40 20 Read xattr: user.rgw.idtag 2023-10-05T19:41:34.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.452+0000 7f215353ae40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo6:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7 : type miss (requested=0x13, cached=0x16) 2023-10-05T19:41:34.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.452+0000 7f215353ae40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.452+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:106 8.9 8:9736f005:root::.bucket.meta.myfoo6%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e41600 con 0x55fce8d9fd60 2023-10-05T19:41:34.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.456+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 <== osd.2 v2:172.21.15.12:6800/29934 24 ==== osd_op_reply(106 .bucket.meta.myfoo6:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7 [call,call out=48b,read 0~353 out=353b,getxattrs out=212b] v0'0 uv43 ondisk = 0) v8 ==== 333+0+613 (crc 0 0 0) 0x7f214c036070 con 0x55fce8d9fd60 2023-10-05T19:41:34.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.456+0000 7f215353ae40 20 rados_obj.operate() r=0 bl.length=353 2023-10-05T19:41:34.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.456+0000 7f215353ae40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7 info.flags=0x13 2023-10-05T19:41:34.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.456+0000 7f215353ae40 10 moving default.rgw.meta+root+.bucket.meta.myfoo6:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7 to cache LRU end 2023-10-05T19:41:34.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.456+0000 7f215353ae40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:41:34.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.456+0000 7f215353ae40 10 updating xattr: name=user.rgw.acl bl.length()=117 2023-10-05T19:41:34.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.456+0000 7f215353ae40 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo6:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7 2023-10-05T19:41:34.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.456+0000 7f215353ae40 10 int RGWRados::cls_bucket_list_unordered(const DoutPrefixProvider*, RGWBucketInfo&, int, const rgw_obj_index_key&, const string&, uint32_t, bool, std::vector&, bool*, rgw_obj_index_key*, optional_yield, RGWBucketListNameFilter) :myfoo6[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7]) start_after="", prefix="", shard_id=-1", num_entries=1100, list_versions=1, force_check_filter is unset 2023-10-05T19:41:34.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.456+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:107 4.4 4:23f8e882:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8ebc420 con 0x7f214c039030 2023-10-05T19:41:34.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.456+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(107 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.0 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f214400a5a0 con 0x7f214c039030 2023-10-05T19:41:34.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.456+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:108 4.1d 4:ba39143d:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8ebc420 con 0x55fce8da4500 2023-10-05T19:41:34.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.456+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(108 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.1 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f213c027080 con 0x55fce8da4500 2023-10-05T19:41:34.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.456+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:109 4.c 4:301ec9e9:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8ebc420 con 0x55fce8d6a7f0 2023-10-05T19:41:34.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.460+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(109 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.2 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.460+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:110 4.2e 4:7620d31e:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8ebc420 con 0x55fce8da4500 2023-10-05T19:41:34.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.460+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(110 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.3 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f213c027080 con 0x55fce8da4500 2023-10-05T19:41:34.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.460+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:111 4.15 4:aa925a53:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8ebc420 con 0x55fce8da4500 2023-10-05T19:41:34.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.460+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(111 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.4 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f213c027080 con 0x55fce8da4500 2023-10-05T19:41:34.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.460+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:112 4.19 4:9845dac2:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8ebc420 con 0x55fce8da4500 2023-10-05T19:41:34.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.460+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 27 ==== osd_op_reply(112 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.5 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f213c027080 con 0x55fce8da4500 2023-10-05T19:41:34.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.460+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:113 4.a 4:502210e1:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8ebc420 con 0x55fce8d9fd60 2023-10-05T19:41:34.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.464+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 <== osd.2 v2:172.21.15.12:6800/29934 25 ==== osd_op_reply(113 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.6 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f214c036070 con 0x55fce8d9fd60 2023-10-05T19:41:34.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.464+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:114 4.15 4:a86bce80:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8ebc420 con 0x55fce8da4500 2023-10-05T19:41:34.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.464+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 28 ==== osd_op_reply(114 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.7 [call out=85b] v0'0 uv4 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f213c027080 con 0x55fce8da4500 2023-10-05T19:41:34.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.464+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:115 4.22 4:4713563e:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8ebc420 con 0x55fce8d6a7f0 2023-10-05T19:41:34.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.464+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(115 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.8 [call out=85b] v0'0 uv4 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.464+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:116 4.2 4:42519106:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8ebc3e0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.464+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== osd_op_reply(116 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.9 [call out=289b] v0'0 uv7 ondisk = 0) v8 ==== 194+0+289 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.464+0000 7f215353ae40 10 cls_bucket_list_unordered: got thirteen 2023-10-05T19:41:34.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.464+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:117 4.3c 4:3cd1d09f:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8ebc3e0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.468+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 39 ==== osd_op_reply(117 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.468+0000 7f215353ae40 20 get_obj_state: rctx=0x7fff51ca7ac0 obj=myfoo6:thirteen state=0x55fce8eb9fb8 s->prefetch_data=0 2023-10-05T19:41:34.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.468+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:118 3.2f 3:f7da5efc:::93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7_thirteen:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8ebae10 con 0x55fce8da4500 2023-10-05T19:41:34.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.468+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 29 ==== osd_op_reply(118 93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7_thirteen [getxattrs out=819b,stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 238+0+835 (crc 0 0 0) 0x7f213c027080 con 0x55fce8da4500 2023-10-05T19:41:34.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.468+0000 7f215353ae40 10 manifest: total_size = 8 2023-10-05T19:41:34.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.468+0000 7f215353ae40 20 get_obj_state: setting s->obj_tag to _jQrh3P_tU_n84jgutDFLxSwWWUY9UvJ 2023-10-05T19:41:34.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.468+0000 7f215353ae40 20 get_obj_state: rctx=0x7fff51ca7ac0 obj=myfoo6:thirteen state=0x55fce8eb9fb8 s->prefetch_data=0 2023-10-05T19:41:34.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.468+0000 7f215353ae40 20 prepare_atomic_modification: state is not atomic. state=0x55fce8eb9fb8 2023-10-05T19:41:34.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.468+0000 7f215353ae40 20 bucket index object: default.rgw.buckets.index:.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.9 2023-10-05T19:41:34.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.468+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:119 4.2 4:42519106:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.9:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=187b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e42b40 con 0x55fce8d6a7f0 2023-10-05T19:41:34.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.472+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 40 ==== osd_op_reply(119 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.9 [stat,call,call] v30'9 uv9 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.472+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:120 3.2f 3:f7da5efc:::93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7_thirteen:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e40b20 con 0x55fce8da4500 2023-10-05T19:41:34.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.472+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 30 ==== osd_op_reply(120 93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7_thirteen [call] v30'5 uv5 ondisk = 0) v8 ==== 196+0+0 (crc 0 0 0) 0x7f213c027080 con 0x55fce8da4500 2023-10-05T19:41:34.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.472+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:121 4.2 4:42519106:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.9:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=264b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e43670 con 0x55fce8d6a7f0 2023-10-05T19:41:34.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.472+0000 7f215353ae40 10 cache get: name=default.rgw.log++bucket.sync-source-hints.myfoo6 : miss 2023-10-05T19:41:34.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.472+0000 7f215353ae40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.472+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:122 6.9 6:936e6ff6:::bucket.sync-source-hints.myfoo6:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e46090 con 0x55fce8da4500 2023-10-05T19:41:34.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.476+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 31 ==== osd_op_reply(122 bucket.sync-source-hints.myfoo6 [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 217+0+0 (crc 0 0 0) 0x7f213c027080 con 0x55fce8da4500 2023-10-05T19:41:34.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.476+0000 7f215353ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:34.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.476+0000 7f215353ae40 10 cache put: name=default.rgw.log++bucket.sync-source-hints.myfoo6 info.flags=0x0 2023-10-05T19:41:34.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.476+0000 7f215353ae40 10 adding default.rgw.log++bucket.sync-source-hints.myfoo6 to cache LRU end 2023-10-05T19:41:34.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.476+0000 7f215353ae40 10 cache get: name=default.rgw.log++bucket.sync-target-hints.myfoo6 : miss 2023-10-05T19:41:34.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.476+0000 7f215353ae40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.476+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:123 6.19 6:9e3df8ab:::bucket.sync-target-hints.myfoo6:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e46030 con 0x55fce8d6a7f0 2023-10-05T19:41:34.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.476+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 41 ==== osd_op_reply(123 bucket.sync-target-hints.myfoo6 [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 217+0+0 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.476+0000 7f215353ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:34.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.476+0000 7f215353ae40 10 cache put: name=default.rgw.log++bucket.sync-target-hints.myfoo6 info.flags=0x0 2023-10-05T19:41:34.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.476+0000 7f215353ae40 10 adding default.rgw.log++bucket.sync-target-hints.myfoo6 to cache LRU end 2023-10-05T19:41:34.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.476+0000 7f215353ae40 10 chain_cache_entry: cache_locator= 2023-10-05T19:41:34.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.476+0000 7f215353ae40 20 chain_cache_entry: couldn't find cache locator 2023-10-05T19:41:34.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.476+0000 7f215353ae40 20 couldn't put bucket_sync_policy cache entry, might have raced with data changes 2023-10-05T19:41:34.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.476+0000 7f215353ae40 20 int RGWRados::Bucket::List::list_objects_ordered(const DoutPrefixProvider*, int64_t, std::vector*, std::map, bool>*, bool*, optional_yield): starting attempt 1 2023-10-05T19:41:34.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.476+0000 7f215353ae40 10 int RGWRados::cls_bucket_list_ordered(const DoutPrefixProvider*, RGWBucketInfo&, int, const rgw_obj_index_key&, const string&, const string&, uint32_t, bool, uint16_t, RGWRados::ent_map_t&, bool*, bool*, rgw_obj_index_key*, optional_yield, RGWBucketListNameFilter): :myfoo6[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7]) start_after="_multipart_", prefix="_multipart_, delimiter="", shard_id=-1", num_entries=1001, list_versions=0, expansion_factor=1, force_check_filter is unset 2023-10-05T19:41:34.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.476+0000 7f215353ae40 10 int RGWRados::cls_bucket_list_ordered(const DoutPrefixProvider*, RGWBucketInfo&, int, const rgw_obj_index_key&, const string&, const string&, uint32_t, bool, uint16_t, RGWRados::ent_map_t&, bool*, bool*, rgw_obj_index_key*, optional_yield, RGWBucketListNameFilter): request from each of 11 shard(s) for 112 entries to get 1001 total entries 2023-10-05T19:41:34.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.476+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:124 4.4 4:23f8e882:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.0:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e45040 con 0x7f214c039030 2023-10-05T19:41:34.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.476+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:125 4.1d 4:ba39143d:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.1:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e45580 con 0x55fce8da4500 2023-10-05T19:41:34.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.476+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:126 4.c 4:301ec9e9:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.2:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e45040 con 0x55fce8d6a7f0 2023-10-05T19:41:34.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.476+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:127 4.2e 4:7620d31e:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.3:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e4e3f0 con 0x55fce8da4500 2023-10-05T19:41:34.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.476+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:128 4.15 4:aa925a53:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.4:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e50da0 con 0x55fce8da4500 2023-10-05T19:41:34.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.476+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:129 4.19 4:9845dac2:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.5:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e53a40 con 0x55fce8da4500 2023-10-05T19:41:34.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.476+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:130 4.a 4:502210e1:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.6:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e56890 con 0x55fce8d9fd60 2023-10-05T19:41:34.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.476+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:131 4.15 4:a86bce80:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.7:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e596c0 con 0x55fce8da4500 2023-10-05T19:41:34.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.476+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:132 4.22 4:4713563e:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.8:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e5c490 con 0x55fce8d6a7f0 2023-10-05T19:41:34.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.476+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:133 4.2 4:42519106:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.9:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e5ef60 con 0x55fce8d6a7f0 2023-10-05T19:41:34.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.476+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:134 4.3c 4:3cd1d09f:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.10:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e619f0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.476+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(124 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.0 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f214400a5a0 con 0x7f214c039030 2023-10-05T19:41:34.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.476+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 42 ==== osd_op_reply(121 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.9 [stat,call,call] v30'10 uv10 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.476+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 <== osd.2 v2:172.21.15.12:6800/29934 26 ==== osd_op_reply(130 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.6 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f214c036070 con 0x55fce8d9fd60 2023-10-05T19:41:34.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.476+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 43 ==== osd_op_reply(126 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.2 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.476+0000 7f214a7fc700 20 handle_completion(): completion ok for obj=thirteen 2023-10-05T19:41:34.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.476+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 44 ==== osd_op_reply(132 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.8 [call out=85b] v0'0 uv4 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.476+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 32 ==== osd_op_reply(125 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.1 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f213c027080 con 0x55fce8da4500 2023-10-05T19:41:34.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.476+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 45 ==== osd_op_reply(133 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.9 [call out=124b] v0'0 uv10 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.480+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 46 ==== osd_op_reply(134 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.480+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 33 ==== osd_op_reply(128 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.4 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f213c027080 con 0x55fce8da4500 2023-10-05T19:41:34.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.480+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 34 ==== osd_op_reply(127 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.3 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f213c027080 con 0x55fce8da4500 2023-10-05T19:41:34.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.480+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 35 ==== osd_op_reply(129 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.5 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f213c027080 con 0x55fce8da4500 2023-10-05T19:41:34.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.480+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 36 ==== osd_op_reply(131 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.7 [call out=85b] v0'0 uv4 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f213c027080 con 0x55fce8da4500 2023-10-05T19:41:34.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.480+0000 7f215353ae40 20 int RGWRados::cls_bucket_list_ordered(const DoutPrefixProvider*, RGWBucketInfo&, int, const rgw_obj_index_key&, const string&, const string&, uint32_t, bool, uint16_t, RGWRados::ent_map_t&, bool*, bool*, rgw_obj_index_key*, optional_yield, RGWBucketListNameFilter): returning, count=0, is_truncated=0 2023-10-05T19:41:34.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.480+0000 7f215353ae40 20 int RGWRados::cls_bucket_list_ordered(const DoutPrefixProvider*, RGWBucketInfo&, int, const rgw_obj_index_key&, const string&, const string&, uint32_t, bool, uint16_t, RGWRados::ent_map_t&, bool*, bool*, rgw_obj_index_key*, optional_yield, RGWBucketListNameFilter): returning, last_entry NOT SET 2023-10-05T19:41:34.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.480+0000 7f215353ae40 10 int RGWRados::Bucket::List::list_objects_ordered(const DoutPrefixProvider*, int64_t, std::vector*, std::map, bool>*, bool*, optional_yield): end of outer loop, truncated=0, count=0, attempt=1 2023-10-05T19:41:34.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.480+0000 7f215353ae40 10 int RGWRados::Bucket::List::list_objects_ordered(const DoutPrefixProvider*, int64_t, std::vector*, std::map, bool>*, bool*, optional_yield): exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2023-10-05T19:41:34.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.480+0000 7f215353ae40 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":myfoo6[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7])"; uploads.size()=0; is_truncated=0 2023-10-05T19:41:34.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.480+0000 7f215353ae40 10 lifecycle: RGWRados::convert_old_bucket_info(): bucket=:myfoo6[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7]) 2023-10-05T19:41:34.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.480+0000 7f215353ae40 20 lifecycle: get_system_obj_state: rctx=0x7fff51ca7558 obj=default.rgw.meta:root:myfoo6 state=0x55fce8c430b0 s->prefetch_data=0 2023-10-05T19:41:34.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.480+0000 7f215353ae40 10 lifecycle: cache get: name=default.rgw.meta+root+myfoo6 : miss 2023-10-05T19:41:34.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.480+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:135 8.18 8:1944a257:root::myfoo6:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e5efc0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.480+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 47 ==== osd_op_reply(135 myfoo6 [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv18 ondisk = 0) v8 ==== 234+0+139 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.480+0000 7f215353ae40 10 lifecycle: cache put: name=default.rgw.meta+root+myfoo6 info.flags=0x16 2023-10-05T19:41:34.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.480+0000 7f215353ae40 10 lifecycle: adding default.rgw.meta+root+myfoo6 to cache LRU end 2023-10-05T19:41:34.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.480+0000 7f215353ae40 10 lifecycle: updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:41:34.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.480+0000 7f215353ae40 20 lifecycle: get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:41:34.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.480+0000 7f215353ae40 20 lifecycle: Read xattr: user.rgw.idtag 2023-10-05T19:41:34.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.480+0000 7f215353ae40 10 lifecycle: cache get: name=default.rgw.meta+root+myfoo6 : type miss (requested=0x13, cached=0x16) 2023-10-05T19:41:34.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.480+0000 7f215353ae40 20 lifecycle: rados->read ofs=0 len=0 2023-10-05T19:41:34.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.480+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:136 8.18 8:1944a257:root::myfoo6:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e53a40 con 0x55fce8d6a7f0 2023-10-05T19:41:34.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.480+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 48 ==== osd_op_reply(136 myfoo6 [call,call out=48b,read 0~166 out=166b,getxattrs out=75b] v0'0 uv18 ondisk = 0) v8 ==== 276+0+289 (crc 0 0 0) 0x7f214400a5a0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.480+0000 7f215353ae40 20 lifecycle: rados_obj.operate() r=0 bl.length=166 2023-10-05T19:41:34.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.480+0000 7f215353ae40 10 lifecycle: cache put: name=default.rgw.meta+root+myfoo6 info.flags=0x13 2023-10-05T19:41:34.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.480+0000 7f215353ae40 10 lifecycle: moving default.rgw.meta+root+myfoo6 to cache LRU end 2023-10-05T19:41:34.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.480+0000 7f215353ae40 10 lifecycle: updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:41:34.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.480+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:137 8.9 8:9736f005:root::.bucket.meta.myfoo6%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~353 in=353b,setxattr user.rgw.acl (117) in=129b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e596c0 con 0x55fce8d9fd60 2023-10-05T19:41:34.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.484+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 <== osd.2 v2:172.21.15.12:6800/29934 27 ==== osd_op_reply(137 .bucket.meta.myfoo6:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7 [delete,create,call,call,writefull 0~353,setxattr (117)] v30'45 uv45 ondisk = 0) v8 ==== 417+0+0 (crc 0 0 0) 0x7f214c036070 con 0x55fce8d9fd60 2023-10-05T19:41:34.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.484+0000 7f215353ae40 10 lifecycle: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7 info.flags=0x17 2023-10-05T19:41:34.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.484+0000 7f215353ae40 10 lifecycle: moving default.rgw.meta+root+.bucket.meta.myfoo6:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7 to cache LRU end 2023-10-05T19:41:34.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.484+0000 7f215353ae40 10 lifecycle: updating xattr: name=user.rgw.acl bl.length()=117 2023-10-05T19:41:34.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.484+0000 7f215353ae40 10 lifecycle: distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo6:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7, ofs0, ns] 2023-10-05T19:41:34.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.484+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:138 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94544727480000 in=733b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e56200 con 0x7f214c039030 2023-10-05T19:41:34.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.484+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== watch-notify(notify (1) cookie 94544726781968 notify 128849018916 ret 0) v3 ==== 763+0+0 (crc 0 0 0) 0x7f2144059040 con 0x7f214c039030 2023-10-05T19:41:34.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.484+0000 7f214a7fc700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018916 cookie 94544726781968 notifier 4729 bl.length()=721 2023-10-05T19:41:34.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.484+0000 7f214a7fc700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7 info.flags=0x17 2023-10-05T19:41:34.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.484+0000 7f214a7fc700 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo6:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7 to cache LRU end 2023-10-05T19:41:34.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.484+0000 7f214a7fc700 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=117 2023-10-05T19:41:34.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.484+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(138 notify.6 [notify cookie 94544727480000 out=8b] v0'0 uv252 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f214400a5a0 con 0x7f214c039030 2023-10-05T19:41:34.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.484+0000 7f214a7fc700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:139 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f20a00034e0 con 0x7f214c039030 2023-10-05T19:41:34.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.484+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(139 notify.6 [notify-ack] v0'0 uv252 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f214400a5a0 con 0x7f214c039030 2023-10-05T19:41:34.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.484+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 30 ==== watch-notify(notify_complete (2) cookie 94544727480000 notify 128849018916 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f214402f040 con 0x7f214c039030 2023-10-05T19:41:34.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.488+0000 7f215353ae40 20 lifecycle: handle_bi_update(): bucket=:myfoo6[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7]): orig_sources= new_sources= 2023-10-05T19:41:34.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.488+0000 7f215353ae40 20 lifecycle: handle_bi_update(): bucket=:myfoo6[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7]): potential sources added=[] removed=[] 2023-10-05T19:41:34.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.488+0000 7f215353ae40 20 lifecycle: handle_bi_update(): bucket=:myfoo6[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7]): orig_dests= new_dests= 2023-10-05T19:41:34.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.488+0000 7f215353ae40 20 lifecycle: handle_bi_update(): bucket=:myfoo6[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7]): potential dests added=[] removed=[] 2023-10-05T19:41:34.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.488+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 6.14 6:2e7dc4d8:lc::lc.22:head [call lock.lock in=65b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e596c0 con 0x55fce8dd9370 2023-10-05T19:41:34.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.488+0000 7f215106f700 1 -- 172.21.15.12:0/1802509420 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(33 lc.22 [call] v30'11 uv11 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f214c036070 con 0x55fce8dd9370 2023-10-05T19:41:34.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.488+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:34 6.14 6:2e7dc4d8:lc::lc.22:head [call rgw.lc_rm_entry in=93b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e596c0 con 0x55fce8dd9370 2023-10-05T19:41:34.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.492+0000 7f215106f700 1 -- 172.21.15.12:0/1802509420 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(34 lc.22 [call] v30'12 uv12 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f214c036070 con 0x55fce8dd9370 2023-10-05T19:41:34.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.492+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:35 6.14 6:2e7dc4d8:lc::lc.22:head [call lock.unlock in=49b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e596c0 con 0x55fce8dd9370 2023-10-05T19:41:34.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.492+0000 7f215106f700 1 -- 172.21.15.12:0/1802509420 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(35 lc.22 [call] v30'13 uv13 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f214c036070 con 0x55fce8dd9370 2023-10-05T19:41:34.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.492+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:140 4.4 4:23f8e882:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e596c0 con 0x7f214c039030 2023-10-05T19:41:34.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.492+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:141 4.1d 4:ba39143d:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e56200 con 0x55fce8da4500 2023-10-05T19:41:34.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.492+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:142 4.c 4:301ec9e9:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e5ad70 con 0x55fce8d6a7f0 2023-10-05T19:41:34.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.492+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:143 4.2e 4:7620d31e:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e596c0 con 0x55fce8da4500 2023-10-05T19:41:34.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.492+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:144 4.15 4:aa925a53:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e56200 con 0x55fce8da4500 2023-10-05T19:41:34.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.492+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:145 4.19 4:9845dac2:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e5ad70 con 0x55fce8da4500 2023-10-05T19:41:34.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.492+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:146 4.a 4:502210e1:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e596c0 con 0x55fce8d9fd60 2023-10-05T19:41:34.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.492+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:147 4.15 4:a86bce80:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e5ad70 con 0x55fce8da4500 2023-10-05T19:41:34.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.492+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:148 4.22 4:4713563e:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e56200 con 0x55fce8d6a7f0 2023-10-05T19:41:34.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.492+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:149 4.2 4:42519106:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e5ad70 con 0x55fce8d6a7f0 2023-10-05T19:41:34.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.492+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:150 4.3c 4:3cd1d09f:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e50d80 con 0x55fce8d6a7f0 2023-10-05T19:41:34.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.492+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 37 ==== osd_op_reply(141 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.1 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f213c027080 con 0x55fce8da4500 2023-10-05T19:41:34.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.492+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 49 ==== osd_op_reply(142 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.2 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f2144068060 con 0x55fce8d6a7f0 2023-10-05T19:41:34.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.492+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 31 ==== osd_op_reply(140 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.0 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f2144068060 con 0x7f214c039030 2023-10-05T19:41:34.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.492+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 38 ==== osd_op_reply(143 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.3 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f213c027080 con 0x55fce8da4500 2023-10-05T19:41:34.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.492+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 39 ==== osd_op_reply(145 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.5 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f213c027080 con 0x55fce8da4500 2023-10-05T19:41:34.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.492+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 <== osd.2 v2:172.21.15.12:6800/29934 28 ==== osd_op_reply(146 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.6 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f214c036070 con 0x55fce8d9fd60 2023-10-05T19:41:34.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.496+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 50 ==== osd_op_reply(148 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.8 [call out=85b] v0'0 uv4 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f2144068060 con 0x55fce8d6a7f0 2023-10-05T19:41:34.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.496+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 51 ==== osd_op_reply(150 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7f2144068060 con 0x55fce8d6a7f0 2023-10-05T19:41:34.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.496+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 40 ==== osd_op_reply(144 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.4 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f213c027080 con 0x55fce8da4500 2023-10-05T19:41:34.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.496+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 41 ==== osd_op_reply(147 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.7 [call out=85b] v0'0 uv4 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f213c027080 con 0x55fce8da4500 2023-10-05T19:41:34.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.496+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 52 ==== osd_op_reply(149 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.9 [call out=124b] v0'0 uv10 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x7f2144068060 con 0x55fce8d6a7f0 2023-10-05T19:41:34.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.496+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:151 8.19 8:99adfd5d:users.uid::bud.buckets:head [call user.set_buckets_info in=208b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e5ad70 con 0x55fce8d6a7f0 2023-10-05T19:41:34.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.496+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 53 ==== osd_op_reply(151 bud.buckets [call] v30'7 uv7 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x7f2144068060 con 0x55fce8d6a7f0 2023-10-05T19:41:34.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.496+0000 7f215353ae40 20 get_system_obj_state: rctx=0x7fff51ca6fd8 obj=default.rgw.meta:root:myfoo6 state=0x55fce8c430b0 s->prefetch_data=0 2023-10-05T19:41:34.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.496+0000 7f215353ae40 10 cache get: name=default.rgw.meta+root+myfoo6 : type miss (requested=0x16, cached=0x13) 2023-10-05T19:41:34.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.496+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:152 8.18 8:1944a257:root::myfoo6:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e5ad70 con 0x55fce8d6a7f0 2023-10-05T19:41:34.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.496+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 54 ==== osd_op_reply(152 myfoo6 [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv18 ondisk = 0) v8 ==== 234+0+139 (crc 0 0 0) 0x7f2144068060 con 0x55fce8d6a7f0 2023-10-05T19:41:34.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.496+0000 7f215353ae40 10 cache put: name=default.rgw.meta+root+myfoo6 info.flags=0x16 2023-10-05T19:41:34.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.496+0000 7f215353ae40 10 moving default.rgw.meta+root+myfoo6 to cache LRU end 2023-10-05T19:41:34.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.496+0000 7f215353ae40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:41:34.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.496+0000 7f215353ae40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:41:34.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.496+0000 7f215353ae40 10 cache get: name=default.rgw.meta+root+myfoo6 : hit (requested=0x11, cached=0x17) 2023-10-05T19:41:34.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.496+0000 7f215353ae40 10 removing default.rgw.meta+root+myfoo6 from cache 2023-10-05T19:41:34.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.496+0000 7f215353ae40 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:myfoo6, ofs0, ns] 2023-10-05T19:41:34.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.496+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:153 7.e 7:7759931f:::notify.3:head [notify cookie 94544727490304 in=162b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e5ad70 con 0x55fce8d9fd60 2023-10-05T19:41:34.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.496+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 <== osd.2 v2:172.21.15.12:6800/29934 29 ==== watch-notify(notify (1) cookie 94544726769184 notify 128849018911 ret 0) v3 ==== 192+0+0 (crc 0 0 0) 0x7f214c06e040 con 0x55fce8d9fd60 2023-10-05T19:41:34.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.496+0000 7f214a7fc700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018911 cookie 94544726769184 notifier 4729 bl.length()=150 2023-10-05T19:41:34.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.496+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 <== osd.2 v2:172.21.15.12:6800/29934 30 ==== osd_op_reply(153 notify.3 [notify cookie 94544727490304 out=8b] v0'0 uv252 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f214c036070 con 0x55fce8d9fd60 2023-10-05T19:41:34.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.496+0000 7f214a7fc700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:154 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f20a00034c0 con 0x55fce8d9fd60 2023-10-05T19:41:34.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.500+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 <== osd.2 v2:172.21.15.12:6800/29934 31 ==== osd_op_reply(154 notify.3 [notify-ack] v0'0 uv252 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f214c036070 con 0x55fce8d9fd60 2023-10-05T19:41:34.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.500+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 <== osd.2 v2:172.21.15.12:6800/29934 32 ==== watch-notify(notify_complete (2) cookie 94544727490304 notify 128849018911 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f214c04f040 con 0x55fce8d9fd60 2023-10-05T19:41:34.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.500+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:155 8.18 8:1944a257:root::myfoo6:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e45440 con 0x55fce8d6a7f0 2023-10-05T19:41:34.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.500+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 55 ==== osd_op_reply(155 myfoo6 [call,call,delete] v30'19 uv19 ondisk = 0) v8 ==== 234+0+0 (crc 0 0 0) 0x7f2144068060 con 0x55fce8d6a7f0 2023-10-05T19:41:34.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.500+0000 7f215353ae40 10 removing default.rgw.meta+root+.bucket.meta.myfoo6:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7 from cache 2023-10-05T19:41:34.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.500+0000 7f215353ae40 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.myfoo6:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7, ofs0, ns] 2023-10-05T19:41:34.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.500+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:156 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94544727490304 in=219b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e45440 con 0x7f214c039030 2023-10-05T19:41:34.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.500+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 32 ==== watch-notify(notify (1) cookie 94544726781968 notify 128849018917 ret 0) v3 ==== 249+0+0 (crc 0 0 0) 0x7f214402f040 con 0x7f214c039030 2023-10-05T19:41:34.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.500+0000 7f214affd700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018917 cookie 94544726781968 notifier 4729 bl.length()=207 2023-10-05T19:41:34.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.500+0000 7f214affd700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:157 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f2130003c50 con 0x7f214c039030 2023-10-05T19:41:34.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.500+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 33 ==== osd_op_reply(156 notify.6 [notify cookie 94544727490304 out=8b] v0'0 uv252 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f2144068060 con 0x7f214c039030 2023-10-05T19:41:34.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.504+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 34 ==== osd_op_reply(157 notify.6 [notify-ack] v0'0 uv252 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2144068060 con 0x7f214c039030 2023-10-05T19:41:34.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.504+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 35 ==== watch-notify(notify_complete (2) cookie 94544727490304 notify 128849018917 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f214403a310 con 0x7f214c039030 2023-10-05T19:41:34.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.504+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:158 8.9 8:9736f005:root::.bucket.meta.myfoo6%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7:head [call version.check_conds in=122b,call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e45440 con 0x55fce8d9fd60 2023-10-05T19:41:34.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.504+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 <== osd.2 v2:172.21.15.12:6800/29934 33 ==== osd_op_reply(158 .bucket.meta.myfoo6:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7 [call,call,delete] v30'46 uv46 ondisk = 0) v8 ==== 291+0+0 (crc 0 0 0) 0x7f214c036070 con 0x55fce8d9fd60 2023-10-05T19:41:34.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.504+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:159 4.4 4:23f8e882:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e45440 con 0x7f214c039030 2023-10-05T19:41:34.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.504+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:160 4.1d 4:ba39143d:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e41d70 con 0x55fce8da4500 2023-10-05T19:41:34.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.504+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:161 4.c 4:301ec9e9:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e45440 con 0x55fce8d6a7f0 2023-10-05T19:41:34.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.504+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:162 4.2e 4:7620d31e:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e3dc20 con 0x55fce8da4500 2023-10-05T19:41:34.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.504+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:163 4.15 4:aa925a53:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.4:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e41d70 con 0x55fce8da4500 2023-10-05T19:41:34.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.504+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:164 4.19 4:9845dac2:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e45440 con 0x55fce8da4500 2023-10-05T19:41:34.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.504+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:165 4.a 4:502210e1:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8ebba50 con 0x55fce8d9fd60 2023-10-05T19:41:34.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.504+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:166 4.15 4:a86bce80:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e44520 con 0x55fce8da4500 2023-10-05T19:41:34.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.504+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:167 4.22 4:4713563e:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e3dc20 con 0x55fce8d6a7f0 2023-10-05T19:41:34.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.504+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:168 4.2 4:42519106:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e41d70 con 0x55fce8d6a7f0 2023-10-05T19:41:34.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.504+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:169 4.3c 4:3cd1d09f:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.10:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e45440 con 0x55fce8d6a7f0 2023-10-05T19:41:34.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.508+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 56 ==== osd_op_reply(161 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.2 [delete] v30'4 uv4 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f2144068060 con 0x55fce8d6a7f0 2023-10-05T19:41:34.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.508+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 57 ==== osd_op_reply(169 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.10 [delete] v30'2 uv2 ondisk = 0) v8 ==== 195+0+0 (crc 0 0 0) 0x7f2144068060 con 0x55fce8d6a7f0 2023-10-05T19:41:34.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.512+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 <== osd.2 v2:172.21.15.12:6800/29934 34 ==== osd_op_reply(165 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.6 [delete] v30'6 uv6 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f214c036070 con 0x55fce8d9fd60 2023-10-05T19:41:34.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.512+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 42 ==== osd_op_reply(162 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.3 [delete] v30'6 uv6 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f213c027080 con 0x55fce8da4500 2023-10-05T19:41:34.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.512+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 43 ==== osd_op_reply(164 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.5 [delete] v30'4 uv4 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f213c027080 con 0x55fce8da4500 2023-10-05T19:41:34.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.512+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 44 ==== osd_op_reply(160 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.1 [delete] v30'5 uv5 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f213c027080 con 0x55fce8da4500 2023-10-05T19:41:34.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.512+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 36 ==== osd_op_reply(159 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.0 [delete] v30'4 uv4 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f2144068060 con 0x7f214c039030 2023-10-05T19:41:34.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.512+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 58 ==== osd_op_reply(168 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.9 [delete] v30'11 uv11 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f2144068060 con 0x55fce8d6a7f0 2023-10-05T19:41:34.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.512+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 45 ==== osd_op_reply(163 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.4 [delete] v30'5 uv5 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f213c027080 con 0x55fce8da4500 2023-10-05T19:41:34.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.512+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 46 ==== osd_op_reply(166 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.7 [delete] v30'6 uv6 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f213c027080 con 0x55fce8da4500 2023-10-05T19:41:34.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.512+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 59 ==== osd_op_reply(167 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7.8 [delete] v30'6 uv6 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f2144068060 con 0x55fce8d6a7f0 2023-10-05T19:41:34.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.512+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:170 6.1b 6:de681bf8:::pubsub..bucket.myfoo6%2f93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8ebba50 con 0x55fce8da4500 2023-10-05T19:41:34.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.512+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 47 ==== osd_op_reply(170 pubsub..bucket.myfoo6/93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7 [getxattrs,stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 251+0+0 (crc 0 0 0) 0x7f213c027080 con 0x55fce8da4500 2023-10-05T19:41:34.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.512+0000 7f215353ae40 10 removing default.rgw.log++pubsub..bucket.myfoo6/93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7 from cache 2023-10-05T19:41:34.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.512+0000 7f215353ae40 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.log:pubsub..bucket.myfoo6/93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7, ofs0, ns] 2023-10-05T19:41:34.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.512+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:171 7.3 7:c609908c:::notify.5:head [notify cookie 94544727490304 in=216b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e45440 con 0x7f214c039030 2023-10-05T19:41:34.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.516+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 37 ==== watch-notify(notify (1) cookie 94544726773600 notify 128849018918 ret 0) v3 ==== 246+0+0 (crc 0 0 0) 0x7f214403a310 con 0x7f214c039030 2023-10-05T19:41:34.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.516+0000 7f214affd700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018918 cookie 94544726773600 notifier 4729 bl.length()=204 2023-10-05T19:41:34.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.516+0000 7f214affd700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:172 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f2130003c50 con 0x7f214c039030 2023-10-05T19:41:34.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.516+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 38 ==== osd_op_reply(171 notify.5 [notify cookie 94544727490304 out=8b] v0'0 uv252 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f2144068060 con 0x7f214c039030 2023-10-05T19:41:34.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.516+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 39 ==== osd_op_reply(172 notify.5 [notify-ack] v0'0 uv252 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2144068060 con 0x7f214c039030 2023-10-05T19:41:34.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.516+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 40 ==== watch-notify(notify_complete (2) cookie 94544727490304 notify 128849018918 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f214403a4a0 con 0x7f214c039030 2023-10-05T19:41:34.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.516+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:173 6.1b 6:de681bf8:::pubsub..bucket.myfoo6%2f93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e453f0 con 0x55fce8da4500 2023-10-05T19:41:34.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.516+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 48 ==== osd_op_reply(173 pubsub..bucket.myfoo6/93ac49f4-ff49-4f99-811b-6453b28decd5.4221.7 [delete] v30'30 uv28 ondisk = -2 ((2) No such file or directory)) v8 ==== 209+0+0 (crc 0 0 0) 0x7f213c027080 con 0x55fce8da4500 2023-10-05T19:41:34.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.516+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:174 8.19 8:99adfd5d:users.uid::bud.buckets:head [call user.remove_bucket in=59b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e453f0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.520+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 60 ==== osd_op_reply(174 bud.buckets [call] v30'8 uv8 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x7f2144068060 con 0x55fce8d6a7f0 2023-10-05T19:41:34.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.520+0000 7f215353ae40 10 removing key index: NX5QOQKC6BH2IDN8HC7A 2023-10-05T19:41:34.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.520+0000 7f215353ae40 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.keys:NX5QOQKC6BH2IDN8HC7A, ofs0, ns] 2023-10-05T19:41:34.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.520+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:175 7.3 7:c609908c:::notify.5:head [notify cookie 94544727490304 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e453f0 con 0x7f214c039030 2023-10-05T19:41:34.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.520+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 41 ==== watch-notify(notify (1) cookie 94544726773600 notify 128849018919 ret 0) v3 ==== 212+0+0 (crc 0 0 0) 0x7f214403a4a0 con 0x7f214c039030 2023-10-05T19:41:34.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.520+0000 7f214a7fc700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018919 cookie 94544726773600 notifier 4729 bl.length()=170 2023-10-05T19:41:34.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.520+0000 7f214a7fc700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:176 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f20a00034c0 con 0x7f214c039030 2023-10-05T19:41:34.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.520+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 42 ==== osd_op_reply(175 notify.5 [notify cookie 94544727490304 out=8b] v0'0 uv252 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f2144068060 con 0x7f214c039030 2023-10-05T19:41:34.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.520+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 43 ==== osd_op_reply(176 notify.5 [notify-ack] v0'0 uv252 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2144068060 con 0x7f214c039030 2023-10-05T19:41:34.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.520+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 44 ==== watch-notify(notify_complete (2) cookie 94544727490304 notify 128849018919 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f2144038440 con 0x7f214c039030 2023-10-05T19:41:34.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.520+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:177 8.a 8:50c8eddb:users.keys::NX5QOQKC6BH2IDN8HC7A:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e453d0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.524+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 61 ==== osd_op_reply(177 NX5QOQKC6BH2IDN8HC7A [delete] v30'3 uv3 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2144068060 con 0x55fce8d6a7f0 2023-10-05T19:41:34.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.524+0000 7f215353ae40 10 removing email index: 2023-10-05T19:41:34.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.524+0000 7f215353ae40 10 removing user buckets index 2023-10-05T19:41:34.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.524+0000 7f215353ae40 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:users.uid:bud.buckets, ofs0, ns] 2023-10-05T19:41:34.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.524+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:178 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94544727397328 in=172b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e44970 con 0x7f214c039030 2023-10-05T19:41:34.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.524+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 45 ==== watch-notify(notify (1) cookie 94544726781968 notify 128849018920 ret 0) v3 ==== 202+0+0 (crc 0 0 0) 0x7f2144038440 con 0x7f214c039030 2023-10-05T19:41:34.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.524+0000 7f214affd700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018920 cookie 94544726781968 notifier 4729 bl.length()=160 2023-10-05T19:41:34.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.524+0000 7f214affd700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:179 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f2130003c50 con 0x7f214c039030 2023-10-05T19:41:34.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.524+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 46 ==== osd_op_reply(178 notify.6 [notify cookie 94544727397328 out=8b] v0'0 uv252 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f2144068060 con 0x7f214c039030 2023-10-05T19:41:34.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.524+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 47 ==== watch-notify(notify_complete (2) cookie 94544727397328 notify 128849018920 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f21440387b0 con 0x7f214c039030 2023-10-05T19:41:34.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.524+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 48 ==== osd_op_reply(179 notify.6 [notify-ack] v0'0 uv252 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2144068060 con 0x7f214c039030 2023-10-05T19:41:34.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.524+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:180 8.19 8:99adfd5d:users.uid::bud.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e453d0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.528+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 62 ==== osd_op_reply(180 bud.buckets [delete] v30'9 uv9 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x7f2144068060 con 0x55fce8d6a7f0 2023-10-05T19:41:34.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.528+0000 7f215353ae40 10 removing user index: bud 2023-10-05T19:41:34.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.528+0000 7f215353ae40 10 removing default.rgw.meta+users.uid+bud from cache 2023-10-05T19:41:34.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.528+0000 7f215353ae40 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.uid:bud, ofs0, ns] 2023-10-05T19:41:34.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.528+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:181 7.3 7:c609908c:::notify.5:head [notify cookie 94544727397328 in=164b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e44970 con 0x7f214c039030 2023-10-05T19:41:34.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.528+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 49 ==== watch-notify(notify (1) cookie 94544726773600 notify 128849018921 ret 0) v3 ==== 194+0+0 (crc 0 0 0) 0x7f21440387b0 con 0x7f214c039030 2023-10-05T19:41:34.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.528+0000 7f214a7fc700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018921 cookie 94544726773600 notifier 4729 bl.length()=152 2023-10-05T19:41:34.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.528+0000 7f214a7fc700 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:182 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f20a00034c0 con 0x7f214c039030 2023-10-05T19:41:34.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.528+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 50 ==== osd_op_reply(181 notify.5 [notify cookie 94544727397328 out=8b] v0'0 uv252 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f2144068060 con 0x7f214c039030 2023-10-05T19:41:34.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.528+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 51 ==== osd_op_reply(182 notify.5 [notify-ack] v0'0 uv252 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2144068060 con 0x7f214c039030 2023-10-05T19:41:34.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.528+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 52 ==== watch-notify(notify_complete (2) cookie 94544727397328 notify 128849018921 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f21440389e0 con 0x7f214c039030 2023-10-05T19:41:34.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.528+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:183 8.4 8:267425a8:users.uid::bud:head [call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8e453d0 con 0x55fce8d9fd60 2023-10-05T19:41:34.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.532+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 <== osd.2 v2:172.21.15.12:6800/29934 35 ==== osd_op_reply(183 bud [call,delete] v30'3 uv3 ondisk = 0) v8 ==== 189+0+0 (crc 0 0 0) 0x7f214c036070 con 0x55fce8d9fd60 2023-10-05T19:41:34.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.532+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:184 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94544726755552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8d81a40 con 0x55fce8d9fd60 2023-10-05T19:41:34.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.536+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 <== osd.2 v2:172.21.15.12:6800/29934 36 ==== osd_op_reply(184 notify.0 [watch unwatch cookie 94544726755552] v30'254 uv252 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f214c036070 con 0x55fce8d9fd60 2023-10-05T19:41:34.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.536+0000 7f215353ae40 20 remove_watcher() i=0 2023-10-05T19:41:34.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.536+0000 7f215353ae40 2 removed watcher, disabling cache 2023-10-05T19:41:34.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.536+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:185 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94544726768080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8d81a40 con 0x55fce8d6a7f0 2023-10-05T19:41:34.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.536+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 63 ==== osd_op_reply(185 notify.1 [watch unwatch cookie 94544726768080] v30'254 uv252 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2144068060 con 0x55fce8d6a7f0 2023-10-05T19:41:34.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.536+0000 7f215353ae40 20 remove_watcher() i=1 2023-10-05T19:41:34.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.536+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:186 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94544726770736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8d81a40 con 0x55fce8da4500 2023-10-05T19:41:34.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.540+0000 7f215086e700 1 -- 172.21.15.12:0/3689692740 <== osd.3 v2:172.21.15.12:6816/29951 49 ==== osd_op_reply(186 notify.2 [watch unwatch cookie 94544726770736] v30'254 uv252 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f213c027080 con 0x55fce8da4500 2023-10-05T19:41:34.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.540+0000 7f215353ae40 20 remove_watcher() i=2 2023-10-05T19:41:34.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.540+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:187 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94544726769184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8d81a40 con 0x55fce8d9fd60 2023-10-05T19:41:34.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.544+0000 7f215106f700 1 -- 172.21.15.12:0/3689692740 <== osd.2 v2:172.21.15.12:6800/29934 37 ==== osd_op_reply(187 notify.3 [watch unwatch cookie 94544726769184] v30'254 uv252 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f214c036070 con 0x55fce8d9fd60 2023-10-05T19:41:34.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.544+0000 7f215353ae40 20 remove_watcher() i=3 2023-10-05T19:41:34.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.544+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:188 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94544726778384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8d81a40 con 0x55fce8d6a7f0 2023-10-05T19:41:34.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.544+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 64 ==== osd_op_reply(188 notify.4 [watch unwatch cookie 94544726778384] v30'254 uv252 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2144068060 con 0x55fce8d6a7f0 2023-10-05T19:41:34.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.544+0000 7f215353ae40 20 remove_watcher() i=4 2023-10-05T19:41:34.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.544+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:189 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94544726773600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8d81a40 con 0x7f214c039030 2023-10-05T19:41:34.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.548+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 53 ==== osd_op_reply(189 notify.5 [watch unwatch cookie 94544726773600] v30'254 uv252 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2144068060 con 0x7f214c039030 2023-10-05T19:41:34.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.548+0000 7f215353ae40 20 remove_watcher() i=5 2023-10-05T19:41:34.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.548+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:190 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94544726781968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8d81a40 con 0x7f214c039030 2023-10-05T19:41:34.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.552+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 54 ==== osd_op_reply(190 notify.6 [watch unwatch cookie 94544726781968] v30'254 uv252 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2144068060 con 0x7f214c039030 2023-10-05T19:41:34.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.552+0000 7f215353ae40 20 remove_watcher() i=6 2023-10-05T19:41:34.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.552+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:191 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94544726783072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8d81a40 con 0x7f214c039030 2023-10-05T19:41:34.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.556+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.0 v2:172.21.15.8:6800/27563 55 ==== osd_op_reply(191 notify.7 [watch unwatch cookie 94544726783072] v30'254 uv252 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2144068060 con 0x7f214c039030 2023-10-05T19:41:34.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.556+0000 7f215353ae40 20 remove_watcher() i=7 2023-10-05T19:41:34.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.556+0000 7f21227e4700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:41:34.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.556+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:192 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94544726786352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fce8dfd5e0 con 0x55fce8d6a7f0 2023-10-05T19:41:34.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.560+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 <== osd.1 v2:172.21.15.12:6808/29933 65 ==== osd_op_reply(192 data_loggenerations_metadata [watch unwatch cookie 94544726786352] v30'243 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f2144068060 con 0x55fce8d6a7f0 2023-10-05T19:41:34.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.560+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f214c039030 msgr2=0x7f214c01b9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:34.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.560+0000 7f215353ae40 1 --2- 172.21.15.12:0/3689692740 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f214c039030 0x7f214c01b9b0 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:34.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.560+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55fce8d6a7f0 msgr2=0x55fce8d9d830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:34.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.560+0000 7f215353ae40 1 --2- 172.21.15.12:0/3689692740 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55fce8d6a7f0 0x55fce8d9d830 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:34.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.560+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55fce8d9fd60 msgr2=0x55fce8da2140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:34.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.560+0000 7f215353ae40 1 --2- 172.21.15.12:0/3689692740 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55fce8d9fd60 0x55fce8da2140 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:34.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.560+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55fce8da4500 msgr2=0x55fce8da6910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:34.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.560+0000 7f215353ae40 1 --2- 172.21.15.12:0/3689692740 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55fce8da4500 0x55fce8da6910 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:34.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.560+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2134030ca0 msgr2=0x7f2134033150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:34.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.560+0000 7f215353ae40 1 --2- 172.21.15.12:0/3689692740 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2134030ca0 0x7f2134033150 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f213c00b300 tx=0x7f213c001470 comp rx=0 tx=0).stop 2023-10-05T19:41:34.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.560+0000 7f2151870700 1 -- 172.21.15.12:0/3689692740 reap_dead start 2023-10-05T19:41:34.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.560+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fce8d6a420 msgr2=0x55fce8d91d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:34.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.560+0000 7f215353ae40 1 --2- 172.21.15.12:0/3689692740 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fce8d6a420 0x55fce8d91d70 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f214c00a650 tx=0x7f214c01c000 comp rx=0 tx=0).stop 2023-10-05T19:41:34.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.560+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 shutdown_connections 2023-10-05T19:41:34.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.560+0000 7f215353ae40 1 --2- 172.21.15.12:0/3689692740 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fce8d6a420 0x55fce8d91d70 secure :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f214c00a650 tx=0x7f214c01c000 comp rx=0 tx=0).stop 2023-10-05T19:41:34.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.560+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 >> 172.21.15.12:0/3689692740 conn(0x55fce8d64730 msgr2=0x55fce8d857a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:34.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.560+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 shutdown_connections 2023-10-05T19:41:34.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.560+0000 7f215353ae40 1 -- 172.21.15.12:0/3689692740 wait complete. 2023-10-05T19:41:34.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.560+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55fce8dd1d60 msgr2=0x55fce8dd4140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:34.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.560+0000 7f215353ae40 1 --2- 172.21.15.12:0/1802509420 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55fce8dd1d60 0x55fce8dd4140 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:34.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.560+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55fce8dce5e0 msgr2=0x55fce8dd09f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:34.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.560+0000 7f215353ae40 1 --2- 172.21.15.12:0/1802509420 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55fce8dce5e0 0x55fce8dd09f0 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:34.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.560+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55fce8dd9370 msgr2=0x55fce8ddb750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:34.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.560+0000 7f215353ae40 1 --2- 172.21.15.12:0/1802509420 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55fce8dd9370 0x55fce8ddb750 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:34.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.560+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55fce8dd5540 msgr2=0x55fce8dd7950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:34.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.564+0000 7f215353ae40 1 --2- 172.21.15.12:0/1802509420 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55fce8dd5540 0x55fce8dd7950 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:34.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.564+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2098030c00 msgr2=0x7f20980330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:34.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.564+0000 7f215353ae40 1 --2- 172.21.15.12:0/1802509420 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f2098030c00 0x7f20980330b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x55fce8db37d0 tx=0x7f214400aed0 comp rx=0 tx=0).stop 2023-10-05T19:41:34.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.564+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fce8dbdee0 msgr2=0x55fce8dc39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:34.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.564+0000 7f215353ae40 1 --2- 172.21.15.12:0/1802509420 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fce8dbdee0 0x55fce8dc39c0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f213c000fd0 tx=0x7f213c03c000 comp rx=0 tx=0).stop 2023-10-05T19:41:34.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.564+0000 7f215106f700 1 -- 172.21.15.12:0/1802509420 reap_dead start 2023-10-05T19:41:34.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.564+0000 7f215106f700 1 -- 172.21.15.12:0/1802509420 reap_dead start 2023-10-05T19:41:34.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.564+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 shutdown_connections 2023-10-05T19:41:34.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.564+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 >> 172.21.15.12:0/1802509420 conn(0x55fce8db92a0 msgr2=0x55fce8db7340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:34.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.564+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 shutdown_connections 2023-10-05T19:41:34.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.564+0000 7f215353ae40 1 -- 172.21.15.12:0/1802509420 wait complete. 2023-10-05T19:41:34.574 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'fud', '--purge-data'] 2023-10-05T19:41:34.574 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'fud', '--purge-data'] 2023-10-05T19:41:34.574 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid fud --purge-data 2023-10-05T19:41:34.615 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:41:34.615 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:41:34.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.612+0000 7f9ce83fde40 1 Processor -- start 2023-10-05T19:41:34.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.612+0000 7f9ce83fde40 1 -- start start 2023-10-05T19:41:34.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.612+0000 7f9ce83fde40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55f447c19c60 0x55f447c1a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:34.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.612+0000 7f9ce83fde40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55f447bf2bd0 con 0x55f447c1a570 2023-10-05T19:41:34.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.612+0000 7f9ce83fde40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x55f447bafee0 con 0x55f447c19c60 2023-10-05T19:41:34.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.612+0000 7f9ce5f32700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55f447c19c60 0x55f447c1a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:34.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.612+0000 7f9ce5f32700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55f447c19c60 0x55f447c1a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:60870/0 (socket says 172.21.15.12:60870) 2023-10-05T19:41:34.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.612+0000 7f9ce5f32700 1 -- 172.21.15.12:0/659007681 learned_addr learned my addr 172.21.15.12:0/659007681 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:34.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.612+0000 7f9ce5731700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x55f447c1a570 0x55f447ca71e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:44334/0 (socket says 172.21.15.12:44334) 2023-10-05T19:41:34.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.612+0000 7f9ce5f32700 1 -- 172.21.15.12:0/659007681 >> v1:172.21.15.8:6789/0 conn(0x55f447c1a570 legacy=0x55f447ca71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:34.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.612+0000 7f9ce5f32700 1 -- 172.21.15.12:0/659007681 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f447ca78f0 con 0x55f447c19c60 2023-10-05T19:41:34.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.612+0000 7f9ce5f32700 1 --2- 172.21.15.12:0/659007681 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f447c19c60 0x55f447c1a030 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f9ce000a5c0 tx=0x7f9ce0005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6473b6904009f1a server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:34.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.612+0000 7f9ce4f30700 1 -- 172.21.15.12:0/659007681 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ce000e510 con 0x55f447c19c60 2023-10-05T19:41:34.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.612+0000 7f9ce4f30700 1 -- 172.21.15.12:0/659007681 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce000e670 con 0x55f447c19c60 2023-10-05T19:41:34.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.612+0000 7f9ce4f30700 1 -- 172.21.15.12:0/659007681 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ce000e920 con 0x55f447c19c60 2023-10-05T19:41:34.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.612+0000 7f9ce83fde40 1 -- 172.21.15.12:0/659007681 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f447c19c60 msgr2=0x55f447c1a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:34.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.612+0000 7f9ce83fde40 1 --2- 172.21.15.12:0/659007681 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f447c19c60 0x55f447c1a030 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f9ce000a5c0 tx=0x7f9ce0005e00 comp rx=0 tx=0).stop 2023-10-05T19:41:34.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.612+0000 7f9ce83fde40 1 -- 172.21.15.12:0/659007681 shutdown_connections 2023-10-05T19:41:34.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.612+0000 7f9ce83fde40 1 --2- 172.21.15.12:0/659007681 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f447c19c60 0x55f447c1a030 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:34.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.612+0000 7f9ce83fde40 1 -- 172.21.15.12:0/659007681 >> 172.21.15.12:0/659007681 conn(0x55f447ca0730 msgr2=0x55f447ca2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:34.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.612+0000 7f9ce83fde40 1 -- 172.21.15.12:0/659007681 shutdown_connections 2023-10-05T19:41:34.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.612+0000 7f9ce83fde40 1 -- 172.21.15.12:0/659007681 wait complete. 2023-10-05T19:41:34.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.612+0000 7f9ce83fde40 1 Processor -- start 2023-10-05T19:41:34.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.616+0000 7f9ce83fde40 1 -- start start 2023-10-05T19:41:34.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.616+0000 7f9ce83fde40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f447c19c60 0x55f447cc9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:34.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.616+0000 7f9ce83fde40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f447b73e30 con 0x55f447c19c60 2023-10-05T19:41:34.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.616+0000 7f9ce5f32700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f447c19c60 0x55f447cc9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:34.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.616+0000 7f9ce5f32700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f447c19c60 0x55f447cc9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:60874/0 (socket says 172.21.15.12:60874) 2023-10-05T19:41:34.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.616+0000 7f9ce5f32700 1 -- 172.21.15.12:0/2095802080 learned_addr learned my addr 172.21.15.12:0/2095802080 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:34.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.616+0000 7f9ce5f32700 1 -- 172.21.15.12:0/2095802080 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce000a180 con 0x55f447c19c60 2023-10-05T19:41:34.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.616+0000 7f9ce5f32700 1 --2- 172.21.15.12:0/2095802080 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f447c19c60 0x55f447cc9a00 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x55f447ca6f00 tx=0x7f9ce0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73262f7bd357ec34 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:34.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.616+0000 7f9cd77fe700 1 -- 172.21.15.12:0/2095802080 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ce00032f0 con 0x55f447c19c60 2023-10-05T19:41:34.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.616+0000 7f9cd77fe700 1 -- 172.21.15.12:0/2095802080 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce000e7e0 con 0x55f447c19c60 2023-10-05T19:41:34.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.616+0000 7f9cd77fe700 1 -- 172.21.15.12:0/2095802080 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ce000ead0 con 0x55f447c19c60 2023-10-05T19:41:34.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.616+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2095802080 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f447c19c60 msgr2=0x55f447cc9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:34.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.616+0000 7f9ce83fde40 1 --2- 172.21.15.12:0/2095802080 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f447c19c60 0x55f447cc9a00 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x55f447ca6f00 tx=0x7f9ce0013000 comp rx=0 tx=0).stop 2023-10-05T19:41:34.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.616+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2095802080 shutdown_connections 2023-10-05T19:41:34.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.616+0000 7f9ce83fde40 1 --2- 172.21.15.12:0/2095802080 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f447c19c60 0x55f447cc9a00 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:34.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.616+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2095802080 >> 172.21.15.12:0/2095802080 conn(0x55f447ca0730 msgr2=0x55f447ca4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:34.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.616+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2095802080 shutdown_connections 2023-10-05T19:41:34.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.616+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2095802080 wait complete. 2023-10-05T19:41:34.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.616+0000 7f9ce83fde40 1 Processor -- start 2023-10-05T19:41:34.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.616+0000 7f9ce83fde40 1 -- start start 2023-10-05T19:41:34.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.616+0000 7f9ce83fde40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f447c19c60 0x55f447ccddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:34.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.616+0000 7f9ce83fde40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f447b46ed0 con 0x55f447c19c60 2023-10-05T19:41:34.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.616+0000 7f9ce5f32700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f447c19c60 0x55f447ccddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:34.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.616+0000 7f9ce5f32700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f447c19c60 0x55f447ccddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:60890/0 (socket says 172.21.15.12:60890) 2023-10-05T19:41:34.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.616+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 learned_addr learned my addr 172.21.15.12:0/819885356 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:34.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.620+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f447ca78f0 con 0x55f447c19c60 2023-10-05T19:41:34.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.620+0000 7f9ce5f32700 1 --2- 172.21.15.12:0/819885356 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f447c19c60 0x55f447ccddb0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0005860 tx=0x7f9ce001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:34.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.620+0000 7f9cd5ffb700 1 -- 172.21.15.12:0/819885356 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ce00109f0 con 0x55f447c19c60 2023-10-05T19:41:34.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.620+0000 7f9cd5ffb700 1 -- 172.21.15.12:0/819885356 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce0010b70 con 0x55f447c19c60 2023-10-05T19:41:34.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.620+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f447cce2f0 con 0x55f447c19c60 2023-10-05T19:41:34.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.620+0000 7f9cd5ffb700 1 -- 172.21.15.12:0/819885356 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ce0010e60 con 0x55f447c19c60 2023-10-05T19:41:34.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.620+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f447cce730 con 0x55f447c19c60 2023-10-05T19:41:34.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.620+0000 7f9cd5ffb700 1 -- 172.21.15.12:0/819885356 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f9ce001c430 con 0x55f447c19c60 2023-10-05T19:41:34.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.620+0000 7f9cd5ffb700 1 --2- 172.21.15.12:0/819885356 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f9cc8030ca0 0x7f9cc8033150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:34.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.620+0000 7f9cd5ffb700 1 -- 172.21.15.12:0/819885356 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7f9ce0046200 con 0x55f447c19c60 2023-10-05T19:41:34.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.620+0000 7f9ce83fde40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.620+0000 7f9ce5731700 1 --2- 172.21.15.12:0/819885356 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f9cc8030ca0 0x7f9cc8033150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:34.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.620+0000 7f9ce83fde40 1 --2- 172.21.15.12:0/819885356 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55f447c1a570 0x55f447cd97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:34.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.620+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447cd9ee0 con 0x55f447c1a570 2023-10-05T19:41:34.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.624+0000 7f9ce6733700 1 --2- 172.21.15.12:0/819885356 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55f447c1a570 0x55f447cd97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:34.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.624+0000 7f9ce5731700 1 --2- 172.21.15.12:0/819885356 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f9cc8030ca0 0x7f9cc8033150 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc001e70 tx=0x7f9cdc001390 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:34.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.624+0000 7f9ce6733700 1 --2- 172.21.15.12:0/819885356 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55f447c1a570 0x55f447cd97b0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:34.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.624+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9cd800a500 con 0x55f447c1a570 2023-10-05T19:41:34.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.624+0000 7f9ce83fde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:34.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.624+0000 7f9ce83fde40 20 realm 2023-10-05T19:41:34.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.624+0000 7f9ce83fde40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.624+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447cd6d50 con 0x55f447c1a570 2023-10-05T19:41:34.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.624+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9cd800a500 con 0x55f447c1a570 2023-10-05T19:41:34.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.624+0000 7f9ce83fde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:34.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.624+0000 7f9ce83fde40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:41:34.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.624+0000 7f9ce83fde40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.624+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447cda9c0 con 0x55f447c1a570 2023-10-05T19:41:34.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.624+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9cd800a500 con 0x55f447c1a570 2023-10-05T19:41:34.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.624+0000 7f9ce83fde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:34.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.624+0000 7f9ce83fde40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.624+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447cda5d0 con 0x55f447c1a570 2023-10-05T19:41:34.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.624+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f9cd800a500 con 0x55f447c1a570 2023-10-05T19:41:34.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.624+0000 7f9ce83fde40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:34.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.624+0000 7f9ce83fde40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.624+0000 7f9ce83fde40 1 --2- 172.21.15.12:0/819885356 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55f447cdbd60 0x55f447cde140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:34.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.624+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447cde680 con 0x55f447cdbd60 2023-10-05T19:41:34.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.628+0000 7f9ce5f32700 1 --2- 172.21.15.12:0/819885356 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55f447cdbd60 0x55f447cde140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:34.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.628+0000 7f9ce5f32700 1 --2- 172.21.15.12:0/819885356 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55f447cdbd60 0x55f447cde140 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:34.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.628+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447cdbd60 2023-10-05T19:41:34.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.628+0000 7f9ce83fde40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:41:34.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.628+0000 7f9ce83fde40 20 searching for the correct realm 2023-10-05T19:41:34.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.628+0000 7f9ce83fde40 1 --2- 172.21.15.12:0/819885356 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55f447ce0500 0x55f447ce2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:34.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.628+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce2fc0 con 0x55f447ce0500 2023-10-05T19:41:34.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.628+0000 7f9ce5731700 1 --2- 172.21.15.12:0/819885356 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55f447ce0500 0x55f447ce2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:34.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.628+0000 7f9ce5731700 1 --2- 172.21.15.12:0/819885356 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55f447ce0500 0x55f447ce2910 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:34.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.632+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f9cdc00f560 con 0x55f447ce0500 2023-10-05T19:41:34.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.632+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce2fc0 con 0x55f447cdbd60 2023-10-05T19:41:34.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.632+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447cdbd60 2023-10-05T19:41:34.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.632+0000 7f9ce5f32700 1 --2- 172.21.15.12:0/819885356 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f9ce0010070 0x7f9ce001c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:34.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.632+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce2fc0 con 0x7f9ce0010070 2023-10-05T19:41:34.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.632+0000 7f9ce6733700 1 --2- 172.21.15.12:0/819885356 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f9ce0010070 0x7f9ce001c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:34.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.632+0000 7f9ce6733700 1 --2- 172.21.15.12:0/819885356 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f9ce0010070 0x7f9ce001c6e0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:34.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.632+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9cd8014040 con 0x7f9ce0010070 2023-10-05T19:41:34.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.632+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce2fc0 con 0x55f447c1a570 2023-10-05T19:41:34.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.632+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447c1a570 2023-10-05T19:41:34.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.632+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce2fc0 con 0x55f447c1a570 2023-10-05T19:41:34.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.636+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447c1a570 2023-10-05T19:41:34.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.636+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce2fc0 con 0x55f447c1a570 2023-10-05T19:41:34.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.636+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(11 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447c1a570 2023-10-05T19:41:34.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.636+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce2fc0 con 0x7f9ce0010070 2023-10-05T19:41:34.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.636+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9cd8014040 con 0x7f9ce0010070 2023-10-05T19:41:34.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.636+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce2fc0 con 0x55f447cdbd60 2023-10-05T19:41:34.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.636+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447cdbd60 2023-10-05T19:41:34.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.636+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce2fc0 con 0x55f447cdbd60 2023-10-05T19:41:34.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.636+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447cdbd60 2023-10-05T19:41:34.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.636+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce2fc0 con 0x55f447c1a570 2023-10-05T19:41:34.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.636+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447c1a570 2023-10-05T19:41:34.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.636+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce2fc0 con 0x7f9ce0010070 2023-10-05T19:41:34.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.640+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9cd8014040 con 0x7f9ce0010070 2023-10-05T19:41:34.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.640+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce2fc0 con 0x55f447c1a570 2023-10-05T19:41:34.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.640+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447c1a570 2023-10-05T19:41:34.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.640+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce2fc0 con 0x55f447cdbd60 2023-10-05T19:41:34.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.640+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(18 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447cdbd60 2023-10-05T19:41:34.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.640+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce2fc0 con 0x55f447ce0500 2023-10-05T19:41:34.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.640+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9cdc00f560 con 0x55f447ce0500 2023-10-05T19:41:34.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.640+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce2fc0 con 0x55f447ce0500 2023-10-05T19:41:34.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.640+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9cdc00f560 con 0x55f447ce0500 2023-10-05T19:41:34.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.640+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce2fc0 con 0x7f9ce0010070 2023-10-05T19:41:34.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.640+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9cd8014040 con 0x7f9ce0010070 2023-10-05T19:41:34.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.640+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce2fc0 con 0x55f447ce0500 2023-10-05T19:41:34.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.644+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9cdc00f560 con 0x55f447ce0500 2023-10-05T19:41:34.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.644+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce2fc0 con 0x55f447ce0500 2023-10-05T19:41:34.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.644+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9cdc00f560 con 0x55f447ce0500 2023-10-05T19:41:34.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.644+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce2fc0 con 0x55f447c1a570 2023-10-05T19:41:34.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.644+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447c1a570 2023-10-05T19:41:34.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.644+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce2fc0 con 0x55f447c1a570 2023-10-05T19:41:34.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.644+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447c1a570 2023-10-05T19:41:34.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.644+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce2fc0 con 0x7f9ce0010070 2023-10-05T19:41:34.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.644+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9cd8014040 con 0x7f9ce0010070 2023-10-05T19:41:34.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.644+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce2fc0 con 0x55f447ce0500 2023-10-05T19:41:34.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.644+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9cdc00f560 con 0x55f447ce0500 2023-10-05T19:41:34.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.644+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce2fc0 con 0x55f447cdbd60 2023-10-05T19:41:34.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.648+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447cdbd60 2023-10-05T19:41:34.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.648+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce2fc0 con 0x55f447c1a570 2023-10-05T19:41:34.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.648+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447c1a570 2023-10-05T19:41:34.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.648+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce2fc0 con 0x7f9ce0010070 2023-10-05T19:41:34.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.648+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9cd8014040 con 0x7f9ce0010070 2023-10-05T19:41:34.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.648+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce2fc0 con 0x55f447ce0500 2023-10-05T19:41:34.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.648+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9cdc00f560 con 0x55f447ce0500 2023-10-05T19:41:34.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.648+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce2fc0 con 0x7f9ce0010070 2023-10-05T19:41:34.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.648+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9cd8014040 con 0x7f9ce0010070 2023-10-05T19:41:34.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.648+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce2fc0 con 0x55f447cdbd60 2023-10-05T19:41:34.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.648+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447cdbd60 2023-10-05T19:41:34.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.648+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce2fc0 con 0x55f447ce0500 2023-10-05T19:41:34.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.652+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9cdc00f560 con 0x55f447ce0500 2023-10-05T19:41:34.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.652+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce2fc0 con 0x7f9ce0010070 2023-10-05T19:41:34.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.652+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9cd8014040 con 0x7f9ce0010070 2023-10-05T19:41:34.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.652+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce2fc0 con 0x55f447c1a570 2023-10-05T19:41:34.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.652+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447c1a570 2023-10-05T19:41:34.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.652+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce2fc0 con 0x55f447cdbd60 2023-10-05T19:41:34.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.652+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(37 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447cdbd60 2023-10-05T19:41:34.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.652+0000 7f9ce83fde40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:41:34.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.652+0000 7f9ce83fde40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:41:34.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.652+0000 7f9ce83fde40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:41:34.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.652+0000 7f9ce83fde40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:41:34.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.652+0000 7f9ce83fde40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.652+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447cdf280 con 0x55f447c1a570 2023-10-05T19:41:34.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.652+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447c1a570 2023-10-05T19:41:34.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.652+0000 7f9ce83fde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:34.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.652+0000 7f9ce83fde40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.652+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447cdeeb0 con 0x55f447c1a570 2023-10-05T19:41:34.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.656+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447c1a570 2023-10-05T19:41:34.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.656+0000 7f9ce83fde40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:34.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.656+0000 7f9ce83fde40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.656+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce3430 con 0x55f447ce0500 2023-10-05T19:41:34.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.656+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f9cdc00f560 con 0x55f447ce0500 2023-10-05T19:41:34.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.656+0000 7f9ce83fde40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:41:34.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.656+0000 7f9ce83fde40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.656+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce37e0 con 0x55f447ce0500 2023-10-05T19:41:34.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.656+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f9cdc00f560 con 0x55f447ce0500 2023-10-05T19:41:34.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.656+0000 7f9ce83fde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:34.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.656+0000 7f9ce83fde40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.656+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce3b90 con 0x55f447c1a570 2023-10-05T19:41:34.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.656+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447c1a570 2023-10-05T19:41:34.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.656+0000 7f9ce83fde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:34.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.656+0000 7f9ce83fde40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.656+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447cdf7a0 con 0x55f447c1a570 2023-10-05T19:41:34.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.656+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447c1a570 2023-10-05T19:41:34.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.656+0000 7f9ce83fde40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:34.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.656+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447cdf7a0 con 0x55f447ce0500 2023-10-05T19:41:34.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.660+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(44 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447ce0500 2023-10-05T19:41:34.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.660+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447cdf7a0 con 0x55f447cdbd60 2023-10-05T19:41:34.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.660+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(45 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447cdbd60 2023-10-05T19:41:34.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.660+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447cdf7a0 con 0x7f9ce0010070 2023-10-05T19:41:34.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.660+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(46 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9cd8014040 con 0x7f9ce0010070 2023-10-05T19:41:34.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.660+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce3b90 con 0x55f447c1a570 2023-10-05T19:41:34.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.660+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(47 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447c1a570 2023-10-05T19:41:34.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.660+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce3b90 con 0x55f447c1a570 2023-10-05T19:41:34.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.660+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(48 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447c1a570 2023-10-05T19:41:34.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.664+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce3b90 con 0x55f447c1a570 2023-10-05T19:41:34.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.664+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(49 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447c1a570 2023-10-05T19:41:34.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.664+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce3b90 con 0x7f9ce0010070 2023-10-05T19:41:34.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.664+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(50 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9cd8014040 con 0x7f9ce0010070 2023-10-05T19:41:34.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.664+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce3b90 con 0x55f447cdbd60 2023-10-05T19:41:34.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.664+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(51 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447cdbd60 2023-10-05T19:41:34.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.664+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce3b90 con 0x55f447cdbd60 2023-10-05T19:41:34.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.664+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(52 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447cdbd60 2023-10-05T19:41:34.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.664+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce3b90 con 0x55f447c1a570 2023-10-05T19:41:34.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.664+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(53 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447c1a570 2023-10-05T19:41:34.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.664+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce3b90 con 0x7f9ce0010070 2023-10-05T19:41:34.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.668+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(54 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9cd8014040 con 0x7f9ce0010070 2023-10-05T19:41:34.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.668+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce3b90 con 0x55f447c1a570 2023-10-05T19:41:34.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.668+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(55 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447c1a570 2023-10-05T19:41:34.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.668+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce3b90 con 0x55f447cdbd60 2023-10-05T19:41:34.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.668+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(56 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447cdbd60 2023-10-05T19:41:34.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.668+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce3b90 con 0x55f447ce0500 2023-10-05T19:41:34.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.668+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(57 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447ce0500 2023-10-05T19:41:34.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.668+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce3b90 con 0x55f447ce0500 2023-10-05T19:41:34.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.668+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(58 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447ce0500 2023-10-05T19:41:34.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.668+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce3b90 con 0x7f9ce0010070 2023-10-05T19:41:34.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.672+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(59 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9cd8014040 con 0x7f9ce0010070 2023-10-05T19:41:34.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.672+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce3b90 con 0x55f447ce0500 2023-10-05T19:41:34.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.672+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(60 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447ce0500 2023-10-05T19:41:34.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.672+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce3b90 con 0x55f447ce0500 2023-10-05T19:41:34.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.672+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(61 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447ce0500 2023-10-05T19:41:34.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.672+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce3b90 con 0x55f447c1a570 2023-10-05T19:41:34.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.672+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(62 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447c1a570 2023-10-05T19:41:34.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.672+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce3b90 con 0x55f447c1a570 2023-10-05T19:41:34.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.672+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(63 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447c1a570 2023-10-05T19:41:34.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.672+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce3b90 con 0x7f9ce0010070 2023-10-05T19:41:34.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.676+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(64 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9cd8014040 con 0x7f9ce0010070 2023-10-05T19:41:34.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.676+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce3b90 con 0x55f447ce0500 2023-10-05T19:41:34.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.676+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(65 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447ce0500 2023-10-05T19:41:34.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.676+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce3b90 con 0x55f447cdbd60 2023-10-05T19:41:34.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.676+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(66 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447cdbd60 2023-10-05T19:41:34.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.676+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce3b90 con 0x55f447c1a570 2023-10-05T19:41:34.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.676+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(67 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447c1a570 2023-10-05T19:41:34.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.676+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce3b90 con 0x7f9ce0010070 2023-10-05T19:41:34.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.676+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(68 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9cd8014040 con 0x7f9ce0010070 2023-10-05T19:41:34.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.676+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce3b90 con 0x55f447ce0500 2023-10-05T19:41:34.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.680+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(69 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447ce0500 2023-10-05T19:41:34.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.680+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce3b90 con 0x7f9ce0010070 2023-10-05T19:41:34.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.680+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(70 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9cd8014040 con 0x7f9ce0010070 2023-10-05T19:41:34.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.680+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce3b90 con 0x55f447cdbd60 2023-10-05T19:41:34.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.680+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(71 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447cdbd60 2023-10-05T19:41:34.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.680+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce3b90 con 0x55f447ce0500 2023-10-05T19:41:34.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.680+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(72 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447ce0500 2023-10-05T19:41:34.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.680+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce3b90 con 0x7f9ce0010070 2023-10-05T19:41:34.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.684+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(73 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9cd8014040 con 0x7f9ce0010070 2023-10-05T19:41:34.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.684+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce3b90 con 0x55f447c1a570 2023-10-05T19:41:34.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.684+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(74 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447c1a570 2023-10-05T19:41:34.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.684+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce3b90 con 0x55f447cdbd60 2023-10-05T19:41:34.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.684+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(75 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447cdbd60 2023-10-05T19:41:34.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.684+0000 7f9ce83fde40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:41:34.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.684+0000 7f9ce83fde40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:41:34.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.684+0000 7f9ce83fde40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:41:34.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.684+0000 7f9ce83fde40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:41:34.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.684+0000 7f9ce83fde40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.684+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447cdf7a0 con 0x55f447c1a570 2023-10-05T19:41:34.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.684+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447c1a570 2023-10-05T19:41:34.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.684+0000 7f9ce83fde40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:34.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.684+0000 7f9ce83fde40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.684+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce4530 con 0x55f447cdbd60 2023-10-05T19:41:34.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.684+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447cdbd60 2023-10-05T19:41:34.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.684+0000 7f9ce83fde40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:41:34.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.684+0000 7f9ce83fde40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.684+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce4e20 con 0x55f447c1a570 2023-10-05T19:41:34.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.688+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447c1a570 2023-10-05T19:41:34.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.688+0000 7f9ce83fde40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:41:34.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.688+0000 7f9ce83fde40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.688+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce4e20 con 0x55f447ce0500 2023-10-05T19:41:34.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.688+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447ce0500 2023-10-05T19:41:34.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.688+0000 7f9ce83fde40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:41:34.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.688+0000 7f9ce83fde40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.688+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce5590 con 0x7f9ce0010070 2023-10-05T19:41:34.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.688+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f9cd8014040 con 0x7f9ce0010070 2023-10-05T19:41:34.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.688+0000 7f9ce83fde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:34.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.688+0000 7f9ce83fde40 20 zone default found 2023-10-05T19:41:34.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.688+0000 7f9ce83fde40 4 Realm: () 2023-10-05T19:41:34.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.688+0000 7f9ce83fde40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:41:34.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.688+0000 7f9ce83fde40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:41:34.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.688+0000 7f9ce83fde40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:41:34.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.688+0000 7f9ce83fde40 20 zonegroup default 2023-10-05T19:41:34.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.688+0000 7f9ce83fde40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.688+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce5590 con 0x7f9ce0010070 2023-10-05T19:41:34.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.688+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f9cd8014040 con 0x7f9ce0010070 2023-10-05T19:41:34.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.688+0000 7f9ce83fde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:34.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.688+0000 7f9ce83fde40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.688+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce6f10 con 0x55f447ce0500 2023-10-05T19:41:34.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.692+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447ce0500 2023-10-05T19:41:34.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.692+0000 7f9ce83fde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:34.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.692+0000 7f9ce83fde40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.692+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce72c0 con 0x55f447ce0500 2023-10-05T19:41:34.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.692+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447ce0500 2023-10-05T19:41:34.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.692+0000 7f9ce83fde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:34.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.692+0000 7f9ce83fde40 20 started sync module instance, tier type = 2023-10-05T19:41:34.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.692+0000 7f9ce83fde40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:41:34.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.692+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce65d0 con 0x55f447cdbd60 2023-10-05T19:41:34.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.696+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(84 notify.0 [create] v30'255 uv255 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447cdbd60 2023-10-05T19:41:34.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.696+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94507665082672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce7670 con 0x55f447cdbd60 2023-10-05T19:41:34.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.696+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce7f10 con 0x55f447c1a570 2023-10-05T19:41:34.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.700+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(86 notify.1 [create] v30'255 uv255 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447c1a570 2023-10-05T19:41:34.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.700+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(85 notify.0 [watch watch cookie 94507665082672] v30'256 uv255 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447cdbd60 2023-10-05T19:41:34.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.700+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94507665095200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce6aa0 con 0x55f447c1a570 2023-10-05T19:41:34.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.700+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce8a60 con 0x55f447ce0500 2023-10-05T19:41:34.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.700+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(88 notify.2 [create] v30'255 uv255 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447ce0500 2023-10-05T19:41:34.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.700+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94507665098032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce9350 con 0x55f447ce0500 2023-10-05T19:41:34.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.700+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce9d60 con 0x55f447cdbd60 2023-10-05T19:41:34.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.704+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(87 notify.1 [watch watch cookie 94507665095200] v30'256 uv255 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447c1a570 2023-10-05T19:41:34.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.704+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(90 notify.3 [create] v30'255 uv255 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447cdbd60 2023-10-05T19:41:34.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.704+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94507665102960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ce7f90 con 0x55f447cdbd60 2023-10-05T19:41:34.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.704+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447cea870 con 0x55f447c1a570 2023-10-05T19:41:34.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.704+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(89 notify.2 [watch watch cookie 94507665098032] v30'256 uv255 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447ce0500 2023-10-05T19:41:34.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.704+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(91 notify.3 [watch watch cookie 94507665102960] v30'256 uv255 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447cdbd60 2023-10-05T19:41:34.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.704+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(92 notify.4 [create] v30'255 uv255 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447c1a570 2023-10-05T19:41:34.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.704+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94507665104064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447cea380 con 0x55f447c1a570 2023-10-05T19:41:34.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.704+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ceb160 con 0x7f9ce0010070 2023-10-05T19:41:34.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.708+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(94 notify.5 [create] v30'255 uv255 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9cd8014040 con 0x7f9ce0010070 2023-10-05T19:41:34.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.708+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94507665107376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ceb2f0 con 0x7f9ce0010070 2023-10-05T19:41:34.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.708+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447cebd00 con 0x7f9ce0010070 2023-10-05T19:41:34.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.708+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(93 notify.4 [watch watch cookie 94507665104064] v30'256 uv255 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447c1a570 2023-10-05T19:41:34.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.712+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(96 notify.6 [create] v30'255 uv255 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9cd8014040 con 0x7f9ce0010070 2023-10-05T19:41:34.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.712+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94507665100384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447cebd00 con 0x7f9ce0010070 2023-10-05T19:41:34.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.712+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447cec710 con 0x7f9ce0010070 2023-10-05T19:41:34.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.712+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(95 notify.5 [watch watch cookie 94507665107376] v30'256 uv255 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9cd8014040 con 0x7f9ce0010070 2023-10-05T19:41:34.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.712+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(97 notify.6 [watch watch cookie 94507665100384] v30'256 uv255 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9cd8014040 con 0x7f9ce0010070 2023-10-05T19:41:34.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.712+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(98 notify.7 [create] v30'255 uv255 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9cd8014040 con 0x7f9ce0010070 2023-10-05T19:41:34.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.712+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94507665111056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ceaf20 con 0x7f9ce0010070 2023-10-05T19:41:34.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.712+0000 7f9ce83fde40 20 add_watcher() i=0 2023-10-05T19:41:34.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.712+0000 7f9ce83fde40 20 add_watcher() i=1 2023-10-05T19:41:34.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.712+0000 7f9ce83fde40 20 add_watcher() i=2 2023-10-05T19:41:34.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.712+0000 7f9ce83fde40 20 add_watcher() i=3 2023-10-05T19:41:34.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.712+0000 7f9ce83fde40 20 add_watcher() i=4 2023-10-05T19:41:34.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.712+0000 7f9ce83fde40 20 add_watcher() i=5 2023-10-05T19:41:34.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.712+0000 7f9ce83fde40 20 add_watcher() i=6 2023-10-05T19:41:34.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.716+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(99 notify.7 [watch watch cookie 94507665111056] v30'256 uv255 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9cd8014040 con 0x7f9ce0010070 2023-10-05T19:41:34.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.716+0000 7f9ce83fde40 20 add_watcher() i=7 2023-10-05T19:41:34.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.716+0000 7f9ce83fde40 2 all 8 watchers are set, enabling cache 2023-10-05T19:41:34.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.716+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447ceaf20 con 0x55f447c1a570 2023-10-05T19:41:34.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.716+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447c1a570 2023-10-05T19:41:34.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.716+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94507665113632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447cecb60 con 0x55f447c1a570 2023-10-05T19:41:34.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.720+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94507665113632] v30'244 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447c1a570 2023-10-05T19:41:34.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.720+0000 7f9cb6fe5700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:41:34.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.720+0000 7f9ce83fde40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:41:34.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.720+0000 7f9ce83fde40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:41:34.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.720+0000 7f9ce83fde40 1 Processor -- start 2023-10-05T19:41:34.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.720+0000 7f9ce83fde40 1 -- start start 2023-10-05T19:41:34.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.720+0000 7f9ce83fde40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f447cf9a90 0x55f447cf9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:34.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.720+0000 7f9ce83fde40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f447cfa3a0 con 0x55f447cf9a90 2023-10-05T19:41:34.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.720+0000 7f9ce5731700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f447cf9a90 0x55f447cf9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:34.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.720+0000 7f9ce5731700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f447cf9a90 0x55f447cf9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:60892/0 (socket says 172.21.15.12:60892) 2023-10-05T19:41:34.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.720+0000 7f9ce5731700 1 -- 172.21.15.12:0/1777610090 learned_addr learned my addr 172.21.15.12:0/1777610090 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:34.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.720+0000 7f9ce5731700 1 -- 172.21.15.12:0/1777610090 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f447cfa4e0 con 0x55f447cf9a90 2023-10-05T19:41:34.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.720+0000 7f9ce5731700 1 --2- 172.21.15.12:0/1777610090 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f447cf9a90 0x55f447cf9e60 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc000fd0 tx=0x7f9cdc010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e231c085d7e9b13e server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:34.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.720+0000 7f9cb67e4700 1 -- 172.21.15.12:0/1777610090 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9cdc040070 con 0x55f447cf9a90 2023-10-05T19:41:34.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.720+0000 7f9cb67e4700 1 -- 172.21.15.12:0/1777610090 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cdc025040 con 0x55f447cf9a90 2023-10-05T19:41:34.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.720+0000 7f9cb67e4700 1 -- 172.21.15.12:0/1777610090 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9cdc024040 con 0x55f447cf9a90 2023-10-05T19:41:34.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.720+0000 7f9ce83fde40 1 -- 172.21.15.12:0/1777610090 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f447cf9a90 msgr2=0x55f447cf9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:34.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.720+0000 7f9ce83fde40 1 --2- 172.21.15.12:0/1777610090 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f447cf9a90 0x55f447cf9e60 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc000fd0 tx=0x7f9cdc010000 comp rx=0 tx=0).stop 2023-10-05T19:41:34.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.720+0000 7f9ce83fde40 1 -- 172.21.15.12:0/1777610090 shutdown_connections 2023-10-05T19:41:34.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.720+0000 7f9ce83fde40 1 --2- 172.21.15.12:0/1777610090 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f447cf9a90 0x55f447cf9e60 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:34.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.720+0000 7f9ce83fde40 1 -- 172.21.15.12:0/1777610090 >> 172.21.15.12:0/1777610090 conn(0x55f447cf4e50 msgr2=0x55f447cf2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:34.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.720+0000 7f9ce83fde40 1 -- 172.21.15.12:0/1777610090 shutdown_connections 2023-10-05T19:41:34.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.720+0000 7f9ce83fde40 1 -- 172.21.15.12:0/1777610090 wait complete. 2023-10-05T19:41:34.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.724+0000 7f9ce83fde40 1 Processor -- start 2023-10-05T19:41:34.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.724+0000 7f9ce83fde40 1 -- start start 2023-10-05T19:41:34.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.724+0000 7f9ce83fde40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f447cf9a90 0x55f447cff5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:34.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.724+0000 7f9ce83fde40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f447d005e0 con 0x55f447cf9a90 2023-10-05T19:41:34.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.724+0000 7f9ce5731700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f447cf9a90 0x55f447cff5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:34.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.724+0000 7f9ce5731700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f447cf9a90 0x55f447cff5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:60898/0 (socket says 172.21.15.12:60898) 2023-10-05T19:41:34.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.724+0000 7f9ce5731700 1 -- 172.21.15.12:0/2605150000 learned_addr learned my addr 172.21.15.12:0/2605150000 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:41:34.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.724+0000 7f9ce5731700 1 -- 172.21.15.12:0/2605150000 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cdc006070 con 0x55f447cf9a90 2023-10-05T19:41:34.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.724+0000 7f9ce5731700 1 --2- 172.21.15.12:0/2605150000 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f447cf9a90 0x55f447cff5e0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f9ce00461b0 tx=0x7f9cdc03c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:34.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.724+0000 7f9cb4fe1700 1 -- 172.21.15.12:0/2605150000 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9cdc024040 con 0x55f447cf9a90 2023-10-05T19:41:34.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.724+0000 7f9cb4fe1700 1 -- 172.21.15.12:0/2605150000 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cdc039990 con 0x55f447cf9a90 2023-10-05T19:41:34.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.724+0000 7f9cb4fe1700 1 -- 172.21.15.12:0/2605150000 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9cdc039b20 con 0x55f447cf9a90 2023-10-05T19:41:34.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.724+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f447cffbe0 con 0x55f447cf9a90 2023-10-05T19:41:34.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.724+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f447d00080 con 0x55f447cf9a90 2023-10-05T19:41:34.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.724+0000 7f9cb4fe1700 1 -- 172.21.15.12:0/2605150000 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f9cdc043020 con 0x55f447cf9a90 2023-10-05T19:41:34.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.728+0000 7f9cb4fe1700 1 --2- 172.21.15.12:0/2605150000 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f9c2c030c50 0x7f9c2c033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:34.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.728+0000 7f9cb4fe1700 1 -- 172.21.15.12:0/2605150000 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7f9cdc022040 con 0x55f447cf9a90 2023-10-05T19:41:34.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.728+0000 7f9ce6733700 1 --2- 172.21.15.12:0/2605150000 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f9c2c030c50 0x7f9c2c033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:34.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.728+0000 7f9ce83fde40 1 --2- 172.21.15.12:0/2605150000 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55f447d0a200 0x55f447d0c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:34.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.728+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d0cc60 con 0x55f447d0a200 2023-10-05T19:41:34.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.728+0000 7f9ce5f32700 1 --2- 172.21.15.12:0/2605150000 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55f447d0a200 0x55f447d0c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:34.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.728+0000 7f9ce6733700 1 --2- 172.21.15.12:0/2605150000 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f9c2c030c50 0x7f9c2c033100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x55f447cec060 tx=0x7f9cd80325c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:34.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.728+0000 7f9ce5f32700 1 --2- 172.21.15.12:0/2605150000 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55f447d0a200 0x55f447d0c610 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:34.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.728+0000 7f9ce5f32700 1 -- 172.21.15.12:0/2605150000 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v30'36 uv30 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447d0a200 2023-10-05T19:41:34.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.728+0000 7f9ce83fde40 1 --2- 172.21.15.12:0/2605150000 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55f447d0d930 0x55f447d0fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:34.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.732+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d10440 con 0x55f447d0d930 2023-10-05T19:41:34.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.732+0000 7f9ce5731700 1 --2- 172.21.15.12:0/2605150000 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55f447d0d930 0x55f447d0fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:34.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.732+0000 7f9ce5731700 1 --2- 172.21.15.12:0/2605150000 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55f447d0d930 0x55f447d0fd10 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:34.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.732+0000 7f9ce5731700 1 -- 172.21.15.12:0/2605150000 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v30'27 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447d0d930 2023-10-05T19:41:34.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.732+0000 7f9ce83fde40 1 --2- 172.21.15.12:0/2605150000 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55f447d11110 0x55f447d134f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:34.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.732+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d13c20 con 0x55f447d11110 2023-10-05T19:41:34.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.732+0000 7f9ce6733700 1 --2- 172.21.15.12:0/2605150000 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55f447d11110 0x55f447d134f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:34.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.736+0000 7f9ce6733700 1 --2- 172.21.15.12:0/2605150000 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55f447d11110 0x55f447d134f0 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:34.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.736+0000 7f9ce6733700 1 -- 172.21.15.12:0/2605150000 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v30'27 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447d11110 2023-10-05T19:41:34.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.736+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d14270 con 0x55f447d0a200 2023-10-05T19:41:34.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.740+0000 7f9ce5f32700 1 -- 172.21.15.12:0/2605150000 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v30'245 uv185 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447d0a200 2023-10-05T19:41:34.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.740+0000 7f9ce83fde40 1 --2- 172.21.15.12:0/2605150000 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55f447d14f10 0x55f447d172f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:41:34.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.740+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d17950 con 0x55f447d14f10 2023-10-05T19:41:34.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.740+0000 7f9ce5f32700 1 --2- 172.21.15.12:0/2605150000 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55f447d14f10 0x55f447d172f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:41:34.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.740+0000 7f9ce5f32700 1 --2- 172.21.15.12:0/2605150000 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55f447d14f10 0x55f447d172f0 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:41:34.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.744+0000 7f9ce5f32700 1 -- 172.21.15.12:0/2605150000 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v30'71 uv51 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447d14f10 2023-10-05T19:41:34.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.744+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d17fa0 con 0x55f447d11110 2023-10-05T19:41:34.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.748+0000 7f9ce6733700 1 -- 172.21.15.12:0/2605150000 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v30'50 uv39 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447d11110 2023-10-05T19:41:34.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.748+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d185c0 con 0x55f447d0a200 2023-10-05T19:41:34.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.748+0000 7f9ce5f32700 1 -- 172.21.15.12:0/2605150000 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v30'29 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447d0a200 2023-10-05T19:41:34.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.748+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d185c0 con 0x55f447d0d930 2023-10-05T19:41:34.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.752+0000 7f9ce5731700 1 -- 172.21.15.12:0/2605150000 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v30'47 uv36 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447d0d930 2023-10-05T19:41:34.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.752+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d05900 con 0x55f447d14f10 2023-10-05T19:41:34.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.756+0000 7f9ce5f32700 1 -- 172.21.15.12:0/2605150000 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v30'27 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447d14f10 2023-10-05T19:41:34.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.756+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d05900 con 0x55f447d11110 2023-10-05T19:41:34.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.756+0000 7f9ce6733700 1 -- 172.21.15.12:0/2605150000 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v30'33 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447d11110 2023-10-05T19:41:34.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.756+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d05900 con 0x55f447d14f10 2023-10-05T19:41:34.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.760+0000 7f9ce5f32700 1 -- 172.21.15.12:0/2605150000 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v30'72 uv53 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447d14f10 2023-10-05T19:41:34.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.760+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d05900 con 0x55f447d0d930 2023-10-05T19:41:34.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.764+0000 7f9ce5731700 1 -- 172.21.15.12:0/2605150000 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v30'34 uv24 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447d0d930 2023-10-05T19:41:34.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.764+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d19840 con 0x55f447d14f10 2023-10-05T19:41:34.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.764+0000 7f9ce5f32700 1 -- 172.21.15.12:0/2605150000 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v30'42 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447d14f10 2023-10-05T19:41:34.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.764+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d19840 con 0x55f447d0d930 2023-10-05T19:41:34.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.768+0000 7f9ce5731700 1 -- 172.21.15.12:0/2605150000 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v30'56 uv45 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447d0d930 2023-10-05T19:41:34.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.768+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d19840 con 0x55f447d0a200 2023-10-05T19:41:34.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.772+0000 7f9ce5f32700 1 -- 172.21.15.12:0/2605150000 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v30'246 uv187 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447d0a200 2023-10-05T19:41:34.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.772+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d19840 con 0x55f447d11110 2023-10-05T19:41:34.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.776+0000 7f9ce6733700 1 -- 172.21.15.12:0/2605150000 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v30'71 uv49 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447d11110 2023-10-05T19:41:34.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.776+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d19d30 con 0x55f447d14f10 2023-10-05T19:41:34.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.776+0000 7f9ce5f32700 1 -- 172.21.15.12:0/2605150000 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v30'73 uv55 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447d14f10 2023-10-05T19:41:34.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.776+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d19d30 con 0x55f447d11110 2023-10-05T19:41:34.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.780+0000 7f9ce6733700 1 -- 172.21.15.12:0/2605150000 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v30'27 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447d11110 2023-10-05T19:41:34.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.780+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d19d30 con 0x55f447d0a200 2023-10-05T19:41:34.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.784+0000 7f9ce5f32700 1 -- 172.21.15.12:0/2605150000 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v30'51 uv41 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447d0a200 2023-10-05T19:41:34.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.784+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d19d30 con 0x55f447d0a200 2023-10-05T19:41:34.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.788+0000 7f9ce5f32700 1 -- 172.21.15.12:0/2605150000 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v30'247 uv183 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447d0a200 2023-10-05T19:41:34.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.788+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d19d30 con 0x55f447d11110 2023-10-05T19:41:34.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.788+0000 7f9ce6733700 1 -- 172.21.15.12:0/2605150000 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v30'72 uv45 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447d11110 2023-10-05T19:41:34.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.788+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d19dc0 con 0x55f447d14f10 2023-10-05T19:41:34.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.792+0000 7f9ce5f32700 1 -- 172.21.15.12:0/2605150000 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v30'37 uv31 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447d14f10 2023-10-05T19:41:34.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.792+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d19dc0 con 0x55f447d11110 2023-10-05T19:41:34.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.796+0000 7f9ce6733700 1 -- 172.21.15.12:0/2605150000 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v30'31 uv25 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447d11110 2023-10-05T19:41:34.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.796+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d19e50 con 0x55f447d11110 2023-10-05T19:41:34.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.796+0000 7f9ce6733700 1 -- 172.21.15.12:0/2605150000 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v30'36 uv30 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447d11110 2023-10-05T19:41:34.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.796+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d19e50 con 0x55f447d11110 2023-10-05T19:41:34.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.800+0000 7f9ce6733700 1 -- 172.21.15.12:0/2605150000 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v30'31 uv18 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447d11110 2023-10-05T19:41:34.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.800+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d19e50 con 0x55f447d11110 2023-10-05T19:41:34.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.804+0000 7f9ce6733700 1 -- 172.21.15.12:0/2605150000 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v30'51 uv37 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447d11110 2023-10-05T19:41:34.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.804+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d19e50 con 0x55f447d0d930 2023-10-05T19:41:34.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.804+0000 7f9ce5731700 1 -- 172.21.15.12:0/2605150000 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v30'33 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447d0d930 2023-10-05T19:41:34.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.808+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d19e50 con 0x55f447d11110 2023-10-05T19:41:34.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.808+0000 7f9ce6733700 1 -- 172.21.15.12:0/2605150000 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v30'27 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447d11110 2023-10-05T19:41:34.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.808+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d19e50 con 0x55f447d0d930 2023-10-05T19:41:34.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.812+0000 7f9ce5731700 1 -- 172.21.15.12:0/2605150000 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v30'48 uv34 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447d0d930 2023-10-05T19:41:34.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.812+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d19e50 con 0x55f447d11110 2023-10-05T19:41:34.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.812+0000 7f9ce6733700 1 -- 172.21.15.12:0/2605150000 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v30'73 uv47 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447d11110 2023-10-05T19:41:34.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.812+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d19e50 con 0x55f447d0d930 2023-10-05T19:41:34.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.816+0000 7f9ce5731700 1 -- 172.21.15.12:0/2605150000 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v30'57 uv43 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447d0d930 2023-10-05T19:41:34.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.816+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d19e50 con 0x55f447d14f10 2023-10-05T19:41:34.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.820+0000 7f9ce5f32700 1 -- 172.21.15.12:0/2605150000 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v30'33 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447d14f10 2023-10-05T19:41:34.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.824+0000 7f9c25ffb700 20 reqs_thread_entry: start 2023-10-05T19:41:34.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.824+0000 7f9c177fe700 20 reqs_thread_entry: start 2023-10-05T19:41:34.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.824+0000 7f9ce83fde40 20 init_complete bucket index max shards: 11 2023-10-05T19:41:34.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.824+0000 7f9ce83fde40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:41:34.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.824+0000 7f9c15ffb700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:42:05 2023 2023-10-05T19:41:34.829 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:41:34.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.824+0000 7f9ce83fde40 20 get_system_obj_state: rctx=0x7ffc751eb578 obj=default.rgw.meta:users.uid:fud state=0x55f447b7f0b0 s->prefetch_data=0 2023-10-05T19:41:34.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.824+0000 7f9ce83fde40 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2023-10-05T19:41:34.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.824+0000 7f9c157fa700 20 reqs_thread_entry: start 2023-10-05T19:41:34.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.824+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:102 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d788a0 con 0x7f9ce0010070 2023-10-05T19:41:34.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.824+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(102 fud [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7f9cd8014040 con 0x7f9ce0010070 2023-10-05T19:41:34.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.824+0000 7f9ce83fde40 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x16 2023-10-05T19:41:34.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.824+0000 7f9ce83fde40 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2023-10-05T19:41:34.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.824+0000 7f9ce83fde40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:41:34.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.824+0000 7f9ce83fde40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:41:34.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.824+0000 7f9ce83fde40 20 Read xattr: user.rgw.idtag 2023-10-05T19:41:34.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.824+0000 7f9ce83fde40 10 cache get: name=default.rgw.meta+users.uid+fud : type miss (requested=0x13, cached=0x16) 2023-10-05T19:41:34.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.824+0000 7f9ce83fde40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.824+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:103 8.e 8:75f48285:users.uid::fud:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d7b330 con 0x7f9ce0010070 2023-10-05T19:41:34.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.824+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(103 fud [call,call out=48b,read 0~346 out=346b,getxattrs out=75b] v0'0 uv14 ondisk = 0) v8 ==== 273+0+469 (crc 0 0 0) 0x7f9cd8014040 con 0x7f9ce0010070 2023-10-05T19:41:34.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.824+0000 7f9ce83fde40 20 rados_obj.operate() r=0 bl.length=346 2023-10-05T19:41:34.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.824+0000 7f9ce83fde40 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x13 2023-10-05T19:41:34.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.824+0000 7f9ce83fde40 10 moving default.rgw.meta+users.uid+fud to cache LRU end 2023-10-05T19:41:34.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.824+0000 7f9ce83fde40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:41:34.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.824+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:104 8.1 8:810902b1:users.uid::fud.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d7b330 con 0x7f9ce0010070 2023-10-05T19:41:34.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.828+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(104 fud.buckets [call out=184b] v0'0 uv5 ondisk = 0) v8 ==== 155+0+184 (crc 0 0 0) 0x7f9cd8014040 con 0x7f9ce0010070 2023-10-05T19:41:34.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.828+0000 7f9ce83fde40 20 get_system_obj_state: rctx=0x7ffc751eb220 obj=default.rgw.meta:root:.bucket.meta.myfoo7:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8 state=0x55f447b7f0b0 s->prefetch_data=0 2023-10-05T19:41:34.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.828+0000 7f9ce83fde40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo7:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8 : miss 2023-10-05T19:41:34.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.828+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:105 8.1b 8:db019024:root::.bucket.meta.myfoo7%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d7ab40 con 0x55f447cdbd60 2023-10-05T19:41:34.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.828+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(105 .bucket.meta.myfoo7:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8 [call out=48b,getxattrs out=218b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 291+0+282 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447cdbd60 2023-10-05T19:41:34.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.828+0000 7f9ce83fde40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo7:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8 info.flags=0x16 2023-10-05T19:41:34.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.828+0000 7f9ce83fde40 10 adding default.rgw.meta+root+.bucket.meta.myfoo7:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8 to cache LRU end 2023-10-05T19:41:34.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.828+0000 7f9ce83fde40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:41:34.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.828+0000 7f9ce83fde40 10 updating xattr: name=user.rgw.acl bl.length()=123 2023-10-05T19:41:34.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.828+0000 7f9ce83fde40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:41:34.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.828+0000 7f9ce83fde40 20 Read xattr: user.rgw.acl 2023-10-05T19:41:34.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.828+0000 7f9ce83fde40 20 Read xattr: user.rgw.idtag 2023-10-05T19:41:34.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.828+0000 7f9ce83fde40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo7:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8 : type miss (requested=0x13, cached=0x16) 2023-10-05T19:41:34.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.828+0000 7f9ce83fde40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.828+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:106 8.1b 8:db019024:root::.bucket.meta.myfoo7%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d7d450 con 0x55f447cdbd60 2023-10-05T19:41:34.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.828+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(106 .bucket.meta.myfoo7:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8 [call,call out=48b,read 0~353 out=353b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 333+0+619 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447cdbd60 2023-10-05T19:41:34.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.828+0000 7f9ce83fde40 20 rados_obj.operate() r=0 bl.length=353 2023-10-05T19:41:34.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.828+0000 7f9ce83fde40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo7:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8 info.flags=0x13 2023-10-05T19:41:34.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.828+0000 7f9ce83fde40 10 moving default.rgw.meta+root+.bucket.meta.myfoo7:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8 to cache LRU end 2023-10-05T19:41:34.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.828+0000 7f9ce83fde40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:41:34.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.828+0000 7f9ce83fde40 10 updating xattr: name=user.rgw.acl bl.length()=123 2023-10-05T19:41:34.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.828+0000 7f9ce83fde40 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo7:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8 2023-10-05T19:41:34.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.828+0000 7f9ce83fde40 10 int RGWRados::cls_bucket_list_unordered(const DoutPrefixProvider*, RGWBucketInfo&, int, const rgw_obj_index_key&, const string&, uint32_t, bool, std::vector&, bool*, rgw_obj_index_key*, optional_yield, RGWBucketListNameFilter) :myfoo7[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8]) start_after="", prefix="", shard_id=-1", num_entries=1100, list_versions=1, force_check_filter is unset 2023-10-05T19:41:34.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.828+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:107 4.35 4:ae976f7d:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447df8760 con 0x55f447c1a570 2023-10-05T19:41:34.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.828+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(107 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.0 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447c1a570 2023-10-05T19:41:34.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.832+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:108 4.16 4:6979bfc1:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447df8760 con 0x55f447ce0500 2023-10-05T19:41:34.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.832+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(108 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.1 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447ce0500 2023-10-05T19:41:34.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.832+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:109 4.24 4:24316731:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447df8760 con 0x7f9ce0010070 2023-10-05T19:41:34.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.832+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(109 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.2 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f9cd8014040 con 0x7f9ce0010070 2023-10-05T19:41:34.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.832+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:110 4.22 4:44b64489:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447df8760 con 0x55f447c1a570 2023-10-05T19:41:34.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.832+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(110 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.3 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447c1a570 2023-10-05T19:41:34.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.832+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:111 4.11 4:8845c530:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447df8760 con 0x7f9ce0010070 2023-10-05T19:41:34.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.836+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(111 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f9cd8014040 con 0x7f9ce0010070 2023-10-05T19:41:34.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.836+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:112 4.25 4:a776f692:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447df8760 con 0x55f447ce0500 2023-10-05T19:41:34.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.836+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(112 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.5 [call out=378b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+378 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447ce0500 2023-10-05T19:41:34.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.836+0000 7f9ce83fde40 10 cls_bucket_list_unordered: got fourteen 2023-10-05T19:41:34.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.836+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:113 4.2 4:41f5dc17:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447df8760 con 0x55f447c1a570 2023-10-05T19:41:34.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.836+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(113 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.6 [call out=85b] v0'0 uv8 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447c1a570 2023-10-05T19:41:34.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.836+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:114 4.3a 4:5dc1d38a:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447df8760 con 0x55f447ce0500 2023-10-05T19:41:34.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.836+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(114 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.7 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447ce0500 2023-10-05T19:41:34.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.836+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:115 4.1d 4:b96c601a:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447df8b30 con 0x55f447ce0500 2023-10-05T19:41:34.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.840+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 27 ==== osd_op_reply(115 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.8 [call out=85b] v0'0 uv4 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447ce0500 2023-10-05T19:41:34.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.840+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:116 4.0 4:03d7646b:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447df8b30 con 0x55f447cdbd60 2023-10-05T19:41:34.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.840+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 <== osd.2 v2:172.21.15.12:6800/29934 23 ==== osd_op_reply(116 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.9 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447cdbd60 2023-10-05T19:41:34.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.840+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:117 4.39 4:9e5b4203:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447df8b30 con 0x55f447ce0500 2023-10-05T19:41:34.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.840+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 28 ==== osd_op_reply(117 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.10 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447ce0500 2023-10-05T19:41:34.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.840+0000 7f9ce83fde40 20 get_obj_state: rctx=0x7ffc751eb240 obj=myfoo7:fourteen state=0x55f447d7b808 s->prefetch_data=0 2023-10-05T19:41:34.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.840+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:118 3.8 3:10cf92bc:::93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8_fourteen:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d7c660 con 0x55f447ce0500 2023-10-05T19:41:34.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.840+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 29 ==== osd_op_reply(118 93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8_fourteen [getxattrs out=856b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 238+0+872 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447ce0500 2023-10-05T19:41:34.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.840+0000 7f9ce83fde40 10 manifest: total_size = 8 2023-10-05T19:41:34.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.840+0000 7f9ce83fde40 20 get_obj_state: setting s->obj_tag to 93ac49f4-ff49-4f99-811b-6453b28decd5.4219.10389688365620015751 2023-10-05T19:41:34.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.840+0000 7f9ce83fde40 20 get_obj_state: rctx=0x7ffc751eb240 obj=myfoo7:fourteen state=0x55f447d7b808 s->prefetch_data=0 2023-10-05T19:41:34.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.840+0000 7f9ce83fde40 20 prepare_atomic_modification: state is not atomic. state=0x55f447d7b808 2023-10-05T19:41:34.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.840+0000 7f9ce83fde40 20 bucket index object: default.rgw.buckets.index:.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.5 2023-10-05T19:41:34.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.840+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:119 4.25 4:a776f692:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.5:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=187b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d7cf70 con 0x55f447ce0500 2023-10-05T19:41:34.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.844+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 30 ==== osd_op_reply(119 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.5 [stat,call,call] v30'4 uv4 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447ce0500 2023-10-05T19:41:34.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.844+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:120 3.8 3:10cf92bc:::93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8_fourteen:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d7cf70 con 0x55f447ce0500 2023-10-05T19:41:34.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.848+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 31 ==== osd_op_reply(120 93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8_fourteen [call] v30'2 uv2 ondisk = 0) v8 ==== 196+0+0 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447ce0500 2023-10-05T19:41:34.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.848+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:121 4.25 4:a776f692:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.5:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=264b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d82030 con 0x55f447ce0500 2023-10-05T19:41:34.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.848+0000 7f9ce83fde40 10 cache get: name=default.rgw.log++bucket.sync-source-hints.myfoo7 : miss 2023-10-05T19:41:34.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.848+0000 7f9ce83fde40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.848+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:122 6.11 6:8826b99d:::bucket.sync-source-hints.myfoo7:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d84840 con 0x55f447ce0500 2023-10-05T19:41:34.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.848+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 32 ==== osd_op_reply(122 bucket.sync-source-hints.myfoo7 [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 217+0+0 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447ce0500 2023-10-05T19:41:34.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.848+0000 7f9ce83fde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:34.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.848+0000 7f9ce83fde40 10 cache put: name=default.rgw.log++bucket.sync-source-hints.myfoo7 info.flags=0x0 2023-10-05T19:41:34.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.848+0000 7f9ce83fde40 10 adding default.rgw.log++bucket.sync-source-hints.myfoo7 to cache LRU end 2023-10-05T19:41:34.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.848+0000 7f9ce83fde40 10 cache get: name=default.rgw.log++bucket.sync-target-hints.myfoo7 : miss 2023-10-05T19:41:34.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.848+0000 7f9ce83fde40 20 rados->read ofs=0 len=0 2023-10-05T19:41:34.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.848+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:123 6.17 6:e82d5341:::bucket.sync-target-hints.myfoo7:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d84840 con 0x55f447c1a570 2023-10-05T19:41:34.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.848+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== osd_op_reply(123 bucket.sync-target-hints.myfoo7 [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 217+0+0 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447c1a570 2023-10-05T19:41:34.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.848+0000 7f9ce83fde40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:41:34.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.848+0000 7f9ce83fde40 10 cache put: name=default.rgw.log++bucket.sync-target-hints.myfoo7 info.flags=0x0 2023-10-05T19:41:34.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.848+0000 7f9ce83fde40 10 adding default.rgw.log++bucket.sync-target-hints.myfoo7 to cache LRU end 2023-10-05T19:41:34.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.848+0000 7f9ce83fde40 10 chain_cache_entry: cache_locator= 2023-10-05T19:41:34.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.848+0000 7f9ce83fde40 20 chain_cache_entry: couldn't find cache locator 2023-10-05T19:41:34.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.848+0000 7f9ce83fde40 20 couldn't put bucket_sync_policy cache entry, might have raced with data changes 2023-10-05T19:41:34.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.848+0000 7f9ce83fde40 20 int RGWRados::Bucket::List::list_objects_ordered(const DoutPrefixProvider*, int64_t, std::vector*, std::map, bool>*, bool*, optional_yield): starting attempt 1 2023-10-05T19:41:34.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.848+0000 7f9ce83fde40 10 int RGWRados::cls_bucket_list_ordered(const DoutPrefixProvider*, RGWBucketInfo&, int, const rgw_obj_index_key&, const string&, const string&, uint32_t, bool, uint16_t, RGWRados::ent_map_t&, bool*, bool*, rgw_obj_index_key*, optional_yield, RGWBucketListNameFilter): :myfoo7[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8]) start_after="_multipart_", prefix="_multipart_, delimiter="", shard_id=-1", num_entries=1001, list_versions=0, expansion_factor=1, force_check_filter is unset 2023-10-05T19:41:34.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.848+0000 7f9ce83fde40 10 int RGWRados::cls_bucket_list_ordered(const DoutPrefixProvider*, RGWBucketInfo&, int, const rgw_obj_index_key&, const string&, const string&, uint32_t, bool, uint16_t, RGWRados::ent_map_t&, bool*, bool*, rgw_obj_index_key*, optional_yield, RGWBucketListNameFilter): request from each of 11 shard(s) for 112 entries to get 1001 total entries 2023-10-05T19:41:34.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.848+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:124 4.35 4:ae976f7d:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.0:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d82c10 con 0x55f447c1a570 2023-10-05T19:41:34.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.852+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:125 4.16 4:6979bfc1:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.1:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d7bad0 con 0x55f447ce0500 2023-10-05T19:41:34.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.852+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:126 4.24 4:24316731:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.2:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d8a180 con 0x7f9ce0010070 2023-10-05T19:41:34.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.852+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:127 4.22 4:44b64489:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.3:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d82c10 con 0x55f447c1a570 2023-10-05T19:41:34.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.852+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:128 4.11 4:8845c530:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.4:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d8f440 con 0x7f9ce0010070 2023-10-05T19:41:34.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.852+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:129 4.25 4:a776f692:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.5:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d91f40 con 0x55f447ce0500 2023-10-05T19:41:34.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.852+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:130 4.2 4:41f5dc17:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.6:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d94d70 con 0x55f447c1a570 2023-10-05T19:41:34.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.852+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:131 4.3a 4:5dc1d38a:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.7:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d97790 con 0x55f447ce0500 2023-10-05T19:41:34.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.852+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:132 4.1d 4:b96c601a:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.8:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d9a660 con 0x55f447ce0500 2023-10-05T19:41:34.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.852+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:133 4.0 4:03d7646b:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.9:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d9d4b0 con 0x55f447cdbd60 2023-10-05T19:41:34.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.852+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:134 4.39 4:9e5b4203:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.10:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d9ff30 con 0x55f447ce0500 2023-10-05T19:41:34.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.852+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 39 ==== osd_op_reply(124 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.0 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447c1a570 2023-10-05T19:41:34.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.852+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 <== osd.2 v2:172.21.15.12:6800/29934 24 ==== osd_op_reply(133 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.9 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447cdbd60 2023-10-05T19:41:34.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.852+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 40 ==== osd_op_reply(130 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.6 [call out=85b] v0'0 uv8 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447c1a570 2023-10-05T19:41:34.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.852+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 41 ==== osd_op_reply(127 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.3 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f9cd8014040 con 0x55f447c1a570 2023-10-05T19:41:34.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.852+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 30 ==== osd_op_reply(126 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.2 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f9cd8014040 con 0x7f9ce0010070 2023-10-05T19:41:34.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.852+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 31 ==== osd_op_reply(128 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f9cd8014040 con 0x7f9ce0010070 2023-10-05T19:41:34.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.852+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 33 ==== osd_op_reply(125 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.1 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447ce0500 2023-10-05T19:41:34.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.852+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 34 ==== osd_op_reply(134 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.10 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447ce0500 2023-10-05T19:41:34.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.852+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 35 ==== osd_op_reply(132 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.8 [call out=85b] v0'0 uv4 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447ce0500 2023-10-05T19:41:34.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.852+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 36 ==== osd_op_reply(121 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.5 [stat,call,call] v30'5 uv5 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447ce0500 2023-10-05T19:41:34.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.852+0000 7f9cd6ffd700 20 handle_completion(): completion ok for obj=fourteen 2023-10-05T19:41:34.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.852+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 37 ==== osd_op_reply(131 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.7 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447ce0500 2023-10-05T19:41:34.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.852+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 38 ==== osd_op_reply(129 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.5 [call out=124b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447ce0500 2023-10-05T19:41:34.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.852+0000 7f9ce83fde40 20 int RGWRados::cls_bucket_list_ordered(const DoutPrefixProvider*, RGWBucketInfo&, int, const rgw_obj_index_key&, const string&, const string&, uint32_t, bool, uint16_t, RGWRados::ent_map_t&, bool*, bool*, rgw_obj_index_key*, optional_yield, RGWBucketListNameFilter): returning, count=0, is_truncated=0 2023-10-05T19:41:34.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.852+0000 7f9ce83fde40 20 int RGWRados::cls_bucket_list_ordered(const DoutPrefixProvider*, RGWBucketInfo&, int, const rgw_obj_index_key&, const string&, const string&, uint32_t, bool, uint16_t, RGWRados::ent_map_t&, bool*, bool*, rgw_obj_index_key*, optional_yield, RGWBucketListNameFilter): returning, last_entry NOT SET 2023-10-05T19:41:34.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.852+0000 7f9ce83fde40 10 int RGWRados::Bucket::List::list_objects_ordered(const DoutPrefixProvider*, int64_t, std::vector*, std::map, bool>*, bool*, optional_yield): end of outer loop, truncated=0, count=0, attempt=1 2023-10-05T19:41:34.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.852+0000 7f9ce83fde40 10 int RGWRados::Bucket::List::list_objects_ordered(const DoutPrefixProvider*, int64_t, std::vector*, std::map, bool>*, bool*, optional_yield): exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2023-10-05T19:41:34.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.852+0000 7f9ce83fde40 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":myfoo7[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8])"; uploads.size()=0; is_truncated=0 2023-10-05T19:41:34.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.852+0000 7f9ce83fde40 10 lifecycle: RGWRados::convert_old_bucket_info(): bucket=:myfoo7[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8]) 2023-10-05T19:41:34.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.852+0000 7f9ce83fde40 20 lifecycle: get_system_obj_state: rctx=0x7ffc751eacd8 obj=default.rgw.meta:root:myfoo7 state=0x55f447b7f0b0 s->prefetch_data=0 2023-10-05T19:41:34.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.852+0000 7f9ce83fde40 10 lifecycle: cache get: name=default.rgw.meta+root+myfoo7 : miss 2023-10-05T19:41:34.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.852+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:135 8.9 8:938c738d:root::myfoo7:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d8f440 con 0x55f447cdbd60 2023-10-05T19:41:34.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.852+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 <== osd.2 v2:172.21.15.12:6800/29934 25 ==== osd_op_reply(135 myfoo7 [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv44 ondisk = 0) v8 ==== 234+0+139 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447cdbd60 2023-10-05T19:41:34.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.852+0000 7f9ce83fde40 10 lifecycle: cache put: name=default.rgw.meta+root+myfoo7 info.flags=0x16 2023-10-05T19:41:34.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.852+0000 7f9ce83fde40 10 lifecycle: adding default.rgw.meta+root+myfoo7 to cache LRU end 2023-10-05T19:41:34.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.852+0000 7f9ce83fde40 10 lifecycle: updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:41:34.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.852+0000 7f9ce83fde40 20 lifecycle: get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:41:34.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.852+0000 7f9ce83fde40 20 lifecycle: Read xattr: user.rgw.idtag 2023-10-05T19:41:34.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.852+0000 7f9ce83fde40 10 lifecycle: cache get: name=default.rgw.meta+root+myfoo7 : type miss (requested=0x13, cached=0x16) 2023-10-05T19:41:34.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.852+0000 7f9ce83fde40 20 lifecycle: rados->read ofs=0 len=0 2023-10-05T19:41:34.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.852+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:136 8.9 8:938c738d:root::myfoo7:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d97790 con 0x55f447cdbd60 2023-10-05T19:41:34.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.856+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 <== osd.2 v2:172.21.15.12:6800/29934 26 ==== osd_op_reply(136 myfoo7 [call,call out=48b,read 0~166 out=166b,getxattrs out=75b] v0'0 uv44 ondisk = 0) v8 ==== 276+0+289 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447cdbd60 2023-10-05T19:41:34.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.856+0000 7f9ce83fde40 20 lifecycle: rados_obj.operate() r=0 bl.length=166 2023-10-05T19:41:34.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.856+0000 7f9ce83fde40 10 lifecycle: cache put: name=default.rgw.meta+root+myfoo7 info.flags=0x13 2023-10-05T19:41:34.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.856+0000 7f9ce83fde40 10 lifecycle: moving default.rgw.meta+root+myfoo7 to cache LRU end 2023-10-05T19:41:34.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.856+0000 7f9ce83fde40 10 lifecycle: updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:41:34.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.856+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:137 8.1b 8:db019024:root::.bucket.meta.myfoo7%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~353 in=353b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d97790 con 0x55f447cdbd60 2023-10-05T19:41:34.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.856+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 <== osd.2 v2:172.21.15.12:6800/29934 27 ==== osd_op_reply(137 .bucket.meta.myfoo7:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8 [delete,create,call,call,writefull 0~353,setxattr (123)] v30'2 uv2 ondisk = 0) v8 ==== 417+0+0 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447cdbd60 2023-10-05T19:41:34.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.856+0000 7f9ce83fde40 10 lifecycle: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo7:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8 info.flags=0x17 2023-10-05T19:41:34.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.856+0000 7f9ce83fde40 10 lifecycle: moving default.rgw.meta+root+.bucket.meta.myfoo7:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8 to cache LRU end 2023-10-05T19:41:34.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.856+0000 7f9ce83fde40 10 lifecycle: updating xattr: name=user.rgw.acl bl.length()=123 2023-10-05T19:41:34.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.856+0000 7f9ce83fde40 10 lifecycle: distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo7:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8, ofs0, ns] 2023-10-05T19:41:34.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.856+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:138 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94507665815440 in=739b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d8ee90 con 0x7f9ce0010070 2023-10-05T19:41:34.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.856+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 32 ==== watch-notify(notify (1) cookie 94507665100384 notify 128849018922 ret 0) v3 ==== 769+0+0 (crc 0 0 0) 0x7f9cd8052040 con 0x7f9ce0010070 2023-10-05T19:41:34.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.856+0000 7f9cd77fe700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018922 cookie 94507665100384 notifier 4735 bl.length()=727 2023-10-05T19:41:34.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.856+0000 7f9cd77fe700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo7:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8 info.flags=0x17 2023-10-05T19:41:34.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.856+0000 7f9cd77fe700 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo7:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8 to cache LRU end 2023-10-05T19:41:34.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.856+0000 7f9cd77fe700 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2023-10-05T19:41:34.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.856+0000 7f9cd77fe700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:139 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f9c340034e0 con 0x7f9ce0010070 2023-10-05T19:41:34.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.860+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 33 ==== osd_op_reply(138 notify.6 [notify cookie 94507665815440 out=8b] v0'0 uv255 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f9cd8014040 con 0x7f9ce0010070 2023-10-05T19:41:34.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.860+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 34 ==== watch-notify(notify_complete (2) cookie 94507665815440 notify 128849018922 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f9cd805b020 con 0x7f9ce0010070 2023-10-05T19:41:34.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.860+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 35 ==== osd_op_reply(139 notify.6 [notify-ack] v0'0 uv255 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9cd805d040 con 0x7f9ce0010070 2023-10-05T19:41:34.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.860+0000 7f9ce83fde40 20 lifecycle: handle_bi_update(): bucket=:myfoo7[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8]): orig_sources= new_sources= 2023-10-05T19:41:34.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.860+0000 7f9ce83fde40 20 lifecycle: handle_bi_update(): bucket=:myfoo7[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8]): potential sources added=[] removed=[] 2023-10-05T19:41:34.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.860+0000 7f9ce83fde40 20 lifecycle: handle_bi_update(): bucket=:myfoo7[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8]): orig_dests= new_dests= 2023-10-05T19:41:34.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.860+0000 7f9ce83fde40 20 lifecycle: handle_bi_update(): bucket=:myfoo7[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8]): potential dests added=[] removed=[] 2023-10-05T19:41:34.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.860+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:33 6.0 6:05588ce8:lc::lc.6:head [call lock.lock in=65b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d8ee30 con 0x55f447d0d930 2023-10-05T19:41:34.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.864+0000 7f9ce5731700 1 -- 172.21.15.12:0/2605150000 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(33 lc.6 [call] v30'58 uv58 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447d0d930 2023-10-05T19:41:34.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.864+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:34 6.0 6:05588ce8:lc::lc.6:head [call rgw.lc_rm_entry in=93b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d8ee30 con 0x55f447d0d930 2023-10-05T19:41:34.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.864+0000 7f9ce5731700 1 -- 172.21.15.12:0/2605150000 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(34 lc.6 [call] v30'59 uv59 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447d0d930 2023-10-05T19:41:34.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.864+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 6.0 6:05588ce8:lc::lc.6:head [call lock.unlock in=49b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d8ee30 con 0x55f447d0d930 2023-10-05T19:41:34.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.868+0000 7f9ce5731700 1 -- 172.21.15.12:0/2605150000 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(35 lc.6 [call] v30'60 uv60 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447d0d930 2023-10-05T19:41:34.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.868+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:140 4.35 4:ae976f7d:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d8ee30 con 0x55f447c1a570 2023-10-05T19:41:34.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.868+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:141 4.16 4:6979bfc1:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d7b500 con 0x55f447ce0500 2023-10-05T19:41:34.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.868+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:142 4.24 4:24316731:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d8a4b0 con 0x7f9ce0010070 2023-10-05T19:41:34.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.868+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:143 4.22 4:44b64489:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d8ee30 con 0x55f447c1a570 2023-10-05T19:41:34.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.868+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:144 4.11 4:8845c530:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d8a4b0 con 0x7f9ce0010070 2023-10-05T19:41:34.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.868+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:145 4.25 4:a776f692:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d7b500 con 0x55f447ce0500 2023-10-05T19:41:34.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.868+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:146 4.2 4:41f5dc17:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d91f40 con 0x55f447c1a570 2023-10-05T19:41:34.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.868+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:147 4.3a 4:5dc1d38a:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d8a4b0 con 0x55f447ce0500 2023-10-05T19:41:34.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.868+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:148 4.1d 4:b96c601a:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d9f9c0 con 0x55f447ce0500 2023-10-05T19:41:34.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.868+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:149 4.0 4:03d7646b:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d91f40 con 0x55f447cdbd60 2023-10-05T19:41:34.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.868+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:150 4.39 4:9e5b4203:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d85ab0 con 0x55f447ce0500 2023-10-05T19:41:34.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.868+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 42 ==== osd_op_reply(140 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.0 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f9cd805d040 con 0x55f447c1a570 2023-10-05T19:41:34.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.868+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 36 ==== osd_op_reply(142 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.2 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f9cd805d040 con 0x7f9ce0010070 2023-10-05T19:41:34.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.868+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 39 ==== osd_op_reply(141 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.1 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447ce0500 2023-10-05T19:41:34.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.868+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 43 ==== osd_op_reply(143 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.3 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f9cd805d040 con 0x55f447c1a570 2023-10-05T19:41:34.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.868+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 44 ==== osd_op_reply(146 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.6 [call out=85b] v0'0 uv8 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f9cd805d040 con 0x55f447c1a570 2023-10-05T19:41:34.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.868+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 <== osd.2 v2:172.21.15.12:6800/29934 28 ==== osd_op_reply(149 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.9 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447cdbd60 2023-10-05T19:41:34.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.868+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 37 ==== osd_op_reply(144 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f9cd805d040 con 0x7f9ce0010070 2023-10-05T19:41:34.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.868+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 40 ==== osd_op_reply(145 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.5 [call out=124b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447ce0500 2023-10-05T19:41:34.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.868+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 41 ==== osd_op_reply(148 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.8 [call out=85b] v0'0 uv4 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447ce0500 2023-10-05T19:41:34.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.868+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 42 ==== osd_op_reply(147 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.7 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447ce0500 2023-10-05T19:41:34.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.868+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 43 ==== osd_op_reply(150 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.10 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447ce0500 2023-10-05T19:41:34.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.868+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:151 8.1 8:810902b1:users.uid::fud.buckets:head [call user.set_buckets_info in=208b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d8a4b0 con 0x7f9ce0010070 2023-10-05T19:41:34.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.872+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 38 ==== osd_op_reply(151 fud.buckets [call] v30'6 uv6 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x7f9cd805d040 con 0x7f9ce0010070 2023-10-05T19:41:34.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.872+0000 7f9ce83fde40 20 get_system_obj_state: rctx=0x7ffc751ea758 obj=default.rgw.meta:root:myfoo7 state=0x55f447b7f0b0 s->prefetch_data=0 2023-10-05T19:41:34.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.872+0000 7f9ce83fde40 10 cache get: name=default.rgw.meta+root+myfoo7 : type miss (requested=0x16, cached=0x13) 2023-10-05T19:41:34.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.872+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:152 8.9 8:938c738d:root::myfoo7:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d7b500 con 0x55f447cdbd60 2023-10-05T19:41:34.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.872+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 <== osd.2 v2:172.21.15.12:6800/29934 29 ==== osd_op_reply(152 myfoo7 [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv44 ondisk = 0) v8 ==== 234+0+139 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447cdbd60 2023-10-05T19:41:34.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.872+0000 7f9ce83fde40 10 cache put: name=default.rgw.meta+root+myfoo7 info.flags=0x16 2023-10-05T19:41:34.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.872+0000 7f9ce83fde40 10 moving default.rgw.meta+root+myfoo7 to cache LRU end 2023-10-05T19:41:34.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.872+0000 7f9ce83fde40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:41:34.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.872+0000 7f9ce83fde40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:41:34.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.872+0000 7f9ce83fde40 10 cache get: name=default.rgw.meta+root+myfoo7 : hit (requested=0x11, cached=0x17) 2023-10-05T19:41:34.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.872+0000 7f9ce83fde40 10 removing default.rgw.meta+root+myfoo7 from cache 2023-10-05T19:41:34.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.872+0000 7f9ce83fde40 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:myfoo7, ofs0, ns] 2023-10-05T19:41:34.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.872+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:153 7.e 7:7759931f:::notify.3:head [notify cookie 94507665698592 in=162b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d8a4b0 con 0x55f447cdbd60 2023-10-05T19:41:34.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.872+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 <== osd.2 v2:172.21.15.12:6800/29934 30 ==== watch-notify(notify (1) cookie 94507665102960 notify 128849018912 ret 0) v3 ==== 192+0+0 (crc 0 0 0) 0x7f9ce0065040 con 0x55f447cdbd60 2023-10-05T19:41:34.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.872+0000 7f9cd6ffd700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018912 cookie 94507665102960 notifier 4735 bl.length()=150 2023-10-05T19:41:34.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.872+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 <== osd.2 v2:172.21.15.12:6800/29934 31 ==== osd_op_reply(153 notify.3 [notify cookie 94507665698592 out=8b] v0'0 uv255 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447cdbd60 2023-10-05T19:41:34.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.872+0000 7f9cd6ffd700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:154 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f9cc4003c50 con 0x55f447cdbd60 2023-10-05T19:41:34.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.872+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 <== osd.2 v2:172.21.15.12:6800/29934 32 ==== osd_op_reply(154 notify.3 [notify-ack] v0'0 uv255 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447cdbd60 2023-10-05T19:41:34.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.876+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 <== osd.2 v2:172.21.15.12:6800/29934 33 ==== watch-notify(notify_complete (2) cookie 94507665698592 notify 128849018912 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55f447cce730 con 0x55f447cdbd60 2023-10-05T19:41:34.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.876+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:155 8.9 8:938c738d:root::myfoo7:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d7b500 con 0x55f447cdbd60 2023-10-05T19:41:34.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.876+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 <== osd.2 v2:172.21.15.12:6800/29934 34 ==== osd_op_reply(155 myfoo7 [call,call,delete] v30'47 uv47 ondisk = 0) v8 ==== 234+0+0 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447cdbd60 2023-10-05T19:41:34.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.876+0000 7f9ce83fde40 10 removing default.rgw.meta+root+.bucket.meta.myfoo7:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8 from cache 2023-10-05T19:41:34.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.876+0000 7f9ce83fde40 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.myfoo7:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8, ofs0, ns] 2023-10-05T19:41:34.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.876+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:156 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94507665698592 in=219b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d8a4b0 con 0x7f9ce0010070 2023-10-05T19:41:34.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.876+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 39 ==== watch-notify(notify (1) cookie 94507665100384 notify 128849018923 ret 0) v3 ==== 249+0+0 (crc 0 0 0) 0x7f9cd805b020 con 0x7f9ce0010070 2023-10-05T19:41:34.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.876+0000 7f9cd6ffd700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018923 cookie 94507665100384 notifier 4735 bl.length()=207 2023-10-05T19:41:34.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.876+0000 7f9cd6ffd700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:157 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f9cc4003c50 con 0x7f9ce0010070 2023-10-05T19:41:34.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.876+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 40 ==== osd_op_reply(156 notify.6 [notify cookie 94507665698592 out=8b] v0'0 uv255 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f9cd805d040 con 0x7f9ce0010070 2023-10-05T19:41:34.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.880+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 41 ==== watch-notify(notify_complete (2) cookie 94507665698592 notify 128849018923 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f9cd8026810 con 0x7f9ce0010070 2023-10-05T19:41:34.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.880+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 42 ==== osd_op_reply(157 notify.6 [notify-ack] v0'0 uv255 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9cd805d040 con 0x7f9ce0010070 2023-10-05T19:41:34.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.880+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:158 8.1b 8:db019024:root::.bucket.meta.myfoo7%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8:head [call version.check_conds in=122b,call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d8a4b0 con 0x55f447cdbd60 2023-10-05T19:41:34.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.880+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 <== osd.2 v2:172.21.15.12:6800/29934 35 ==== osd_op_reply(158 .bucket.meta.myfoo7:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8 [call,call,delete] v30'3 uv3 ondisk = 0) v8 ==== 291+0+0 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447cdbd60 2023-10-05T19:41:34.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.880+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:159 4.35 4:ae976f7d:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d8a4b0 con 0x55f447c1a570 2023-10-05T19:41:34.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.880+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:160 4.16 4:6979bfc1:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d7b940 con 0x55f447ce0500 2023-10-05T19:41:34.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.880+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:161 4.24 4:24316731:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d79720 con 0x7f9ce0010070 2023-10-05T19:41:34.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.880+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:162 4.22 4:44b64489:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d7a000 con 0x55f447c1a570 2023-10-05T19:41:34.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.880+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:163 4.11 4:8845c530:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.4:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d9b170 con 0x7f9ce0010070 2023-10-05T19:41:34.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.880+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:164 4.25 4:a776f692:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d9bae0 con 0x55f447ce0500 2023-10-05T19:41:34.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.880+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:165 4.2 4:41f5dc17:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d8a4b0 con 0x55f447c1a570 2023-10-05T19:41:34.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.880+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:166 4.3a 4:5dc1d38a:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d9c3c0 con 0x55f447ce0500 2023-10-05T19:41:34.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.880+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:167 4.1d 4:b96c601a:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d9bae0 con 0x55f447ce0500 2023-10-05T19:41:34.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.880+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:168 4.0 4:03d7646b:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d7a000 con 0x55f447cdbd60 2023-10-05T19:41:34.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.880+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:169 4.39 4:9e5b4203:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.10:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d8a4b0 con 0x55f447ce0500 2023-10-05T19:41:34.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.884+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 44 ==== osd_op_reply(164 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.5 [delete] v30'6 uv6 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447ce0500 2023-10-05T19:41:34.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.884+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 45 ==== osd_op_reply(162 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.3 [delete] v30'7 uv7 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f9cd8067040 con 0x55f447c1a570 2023-10-05T19:41:34.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.884+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 45 ==== osd_op_reply(166 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.7 [delete] v30'2 uv2 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447ce0500 2023-10-05T19:41:34.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.884+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 <== osd.2 v2:172.21.15.12:6800/29934 36 ==== osd_op_reply(168 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.9 [delete] v30'2 uv2 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447cdbd60 2023-10-05T19:41:34.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.884+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 46 ==== osd_op_reply(160 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.1 [delete] v30'4 uv4 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447ce0500 2023-10-05T19:41:34.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.884+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 46 ==== osd_op_reply(159 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.0 [delete] v30'4 uv4 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f9cd8067040 con 0x55f447c1a570 2023-10-05T19:41:34.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.884+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 43 ==== osd_op_reply(161 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.2 [delete] v30'2 uv2 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f9cd8067040 con 0x7f9ce0010070 2023-10-05T19:41:34.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.884+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 47 ==== osd_op_reply(167 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.8 [delete] v30'6 uv6 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447ce0500 2023-10-05T19:41:34.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.888+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 47 ==== osd_op_reply(165 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.6 [delete] v30'12 uv12 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f9cd8067040 con 0x55f447c1a570 2023-10-05T19:41:34.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.888+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 44 ==== osd_op_reply(163 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.4 [delete] v30'2 uv2 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f9cd8067040 con 0x7f9ce0010070 2023-10-05T19:41:34.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.888+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 48 ==== osd_op_reply(169 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8.10 [delete] v30'4 uv4 ondisk = 0) v8 ==== 195+0+0 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447ce0500 2023-10-05T19:41:34.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.888+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:170 6.17 6:ef2c869a:::pubsub..bucket.myfoo7%2f93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d9bae0 con 0x55f447c1a570 2023-10-05T19:41:34.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.888+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 48 ==== osd_op_reply(170 pubsub..bucket.myfoo7/93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8 [getxattrs,stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 251+0+0 (crc 0 0 0) 0x7f9cd8067040 con 0x55f447c1a570 2023-10-05T19:41:34.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.888+0000 7f9ce83fde40 10 removing default.rgw.log++pubsub..bucket.myfoo7/93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8 from cache 2023-10-05T19:41:34.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.888+0000 7f9ce83fde40 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.log:pubsub..bucket.myfoo7/93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8, ofs0, ns] 2023-10-05T19:41:34.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.888+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:171 7.3 7:c609908c:::notify.5:head [notify cookie 94507665802976 in=216b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d9c3c0 con 0x7f9ce0010070 2023-10-05T19:41:34.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.888+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 45 ==== watch-notify(notify (1) cookie 94507665107376 notify 128849018924 ret 0) v3 ==== 246+0+0 (crc 0 0 0) 0x7f9cd8026810 con 0x7f9ce0010070 2023-10-05T19:41:34.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.888+0000 7f9cd6ffd700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018924 cookie 94507665107376 notifier 4735 bl.length()=204 2023-10-05T19:41:34.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.888+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 46 ==== osd_op_reply(171 notify.5 [notify cookie 94507665802976 out=8b] v0'0 uv255 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f9cd8067040 con 0x7f9ce0010070 2023-10-05T19:41:34.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.888+0000 7f9cd6ffd700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:172 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f9cc4003c50 con 0x7f9ce0010070 2023-10-05T19:41:34.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.888+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 47 ==== osd_op_reply(172 notify.5 [notify-ack] v0'0 uv255 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9cd8067040 con 0x7f9ce0010070 2023-10-05T19:41:34.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.888+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 48 ==== watch-notify(notify_complete (2) cookie 94507665802976 notify 128849018924 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f9cd8026a40 con 0x7f9ce0010070 2023-10-05T19:41:34.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.888+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:173 6.17 6:ef2c869a:::pubsub..bucket.myfoo7%2f93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d946e0 con 0x55f447c1a570 2023-10-05T19:41:34.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.892+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 49 ==== osd_op_reply(173 pubsub..bucket.myfoo7/93ac49f4-ff49-4f99-811b-6453b28decd5.4221.8 [delete] v30'7 uv6 ondisk = -2 ((2) No such file or directory)) v8 ==== 209+0+0 (crc 0 0 0) 0x7f9cd8067040 con 0x55f447c1a570 2023-10-05T19:41:34.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.892+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:174 8.1 8:810902b1:users.uid::fud.buckets:head [call user.remove_bucket in=59b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d946e0 con 0x7f9ce0010070 2023-10-05T19:41:34.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.892+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 49 ==== osd_op_reply(174 fud.buckets [call] v30'7 uv7 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x7f9cd8067040 con 0x7f9ce0010070 2023-10-05T19:41:34.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.892+0000 7f9ce83fde40 10 removing key index: p5YnriCv1nAtykxBrupQ 2023-10-05T19:41:34.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.892+0000 7f9ce83fde40 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2023-10-05T19:41:34.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.892+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:175 7.15 7:a93a5511:::notify.2:head [notify cookie 94507665780272 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d7af20 con 0x55f447ce0500 2023-10-05T19:41:34.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.896+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 49 ==== watch-notify(notify (1) cookie 94507665098032 notify 128849018891 ret 0) v3 ==== 212+0+0 (crc 0 0 0) 0x7f9cdc067000 con 0x55f447ce0500 2023-10-05T19:41:34.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.896+0000 7f9cd77fe700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018891 cookie 94507665098032 notifier 4735 bl.length()=170 2023-10-05T19:41:34.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.896+0000 7f9cd77fe700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:176 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f9c340034c0 con 0x55f447ce0500 2023-10-05T19:41:34.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.896+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 50 ==== osd_op_reply(175 notify.2 [notify cookie 94507665780272 out=8b] v0'0 uv255 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447ce0500 2023-10-05T19:41:34.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.896+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 51 ==== osd_op_reply(176 notify.2 [notify-ack] v0'0 uv255 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447ce0500 2023-10-05T19:41:34.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.896+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 52 ==== watch-notify(notify_complete (2) cookie 94507665780272 notify 128849018891 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55f447d00080 con 0x55f447ce0500 2023-10-05T19:41:34.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.896+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:177 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d8ee30 con 0x7f9ce0010070 2023-10-05T19:41:34.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.896+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 50 ==== osd_op_reply(177 p5YnriCv1nAtykxBrupQ [delete] v30'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9cd8067040 con 0x7f9ce0010070 2023-10-05T19:41:34.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.896+0000 7f9ce83fde40 10 removing email index: 2023-10-05T19:41:34.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.896+0000 7f9ce83fde40 10 removing user buckets index 2023-10-05T19:41:34.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.896+0000 7f9ce83fde40 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:users.uid:fud.buckets, ofs0, ns] 2023-10-05T19:41:34.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.896+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:178 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94507665802976 in=172b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d7af20 con 0x7f9ce0010070 2023-10-05T19:41:34.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.900+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 51 ==== watch-notify(notify (1) cookie 94507665100384 notify 128849018925 ret 0) v3 ==== 202+0+0 (crc 0 0 0) 0x7f9cd8026a40 con 0x7f9ce0010070 2023-10-05T19:41:34.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.900+0000 7f9cd6ffd700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018925 cookie 94507665100384 notifier 4735 bl.length()=160 2023-10-05T19:41:34.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.900+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 52 ==== osd_op_reply(178 notify.6 [notify cookie 94507665802976 out=8b] v0'0 uv255 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f9cd8067040 con 0x7f9ce0010070 2023-10-05T19:41:34.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.900+0000 7f9cd6ffd700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:179 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f9cc4003c50 con 0x7f9ce0010070 2023-10-05T19:41:34.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.900+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 53 ==== osd_op_reply(179 notify.6 [notify-ack] v0'0 uv255 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9cd8067040 con 0x7f9ce0010070 2023-10-05T19:41:34.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.900+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 54 ==== watch-notify(notify_complete (2) cookie 94507665802976 notify 128849018925 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f9cd8026c70 con 0x7f9ce0010070 2023-10-05T19:41:34.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.900+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:180 8.1 8:810902b1:users.uid::fud.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d946e0 con 0x7f9ce0010070 2023-10-05T19:41:34.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.904+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 55 ==== osd_op_reply(180 fud.buckets [delete] v30'8 uv8 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x7f9cd8067040 con 0x7f9ce0010070 2023-10-05T19:41:34.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.904+0000 7f9ce83fde40 10 removing user index: fud 2023-10-05T19:41:34.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.904+0000 7f9ce83fde40 10 removing default.rgw.meta+users.uid+fud from cache 2023-10-05T19:41:34.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.904+0000 7f9ce83fde40 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.uid:fud, ofs0, ns] 2023-10-05T19:41:34.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.904+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:181 7.3 7:c609908c:::notify.5:head [notify cookie 94507665780272 in=164b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d7af20 con 0x7f9ce0010070 2023-10-05T19:41:34.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.904+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 56 ==== watch-notify(notify (1) cookie 94507665107376 notify 128849018926 ret 0) v3 ==== 194+0+0 (crc 0 0 0) 0x7f9cd8026c70 con 0x7f9ce0010070 2023-10-05T19:41:34.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.904+0000 7f9cd77fe700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018926 cookie 94507665107376 notifier 4735 bl.length()=152 2023-10-05T19:41:34.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.904+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 57 ==== osd_op_reply(181 notify.5 [notify cookie 94507665780272 out=8b] v0'0 uv255 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f9cd8067040 con 0x7f9ce0010070 2023-10-05T19:41:34.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.904+0000 7f9cd77fe700 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:182 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f9c340034c0 con 0x7f9ce0010070 2023-10-05T19:41:34.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.904+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 58 ==== osd_op_reply(182 notify.5 [notify-ack] v0'0 uv255 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9cd8067040 con 0x7f9ce0010070 2023-10-05T19:41:34.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.904+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 59 ==== watch-notify(notify_complete (2) cookie 94507665780272 notify 128849018926 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f9cd8061450 con 0x7f9ce0010070 2023-10-05T19:41:34.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.904+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:183 8.e 8:75f48285:users.uid::fud:head [call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d946e0 con 0x7f9ce0010070 2023-10-05T19:41:34.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.908+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 60 ==== osd_op_reply(183 fud [call,delete] v30'16 uv16 ondisk = 0) v8 ==== 189+0+0 (crc 0 0 0) 0x7f9cd8067040 con 0x7f9ce0010070 2023-10-05T19:41:34.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.908+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:184 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94507665082672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447cbb840 con 0x55f447cdbd60 2023-10-05T19:41:34.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.912+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 <== osd.2 v2:172.21.15.12:6800/29934 37 ==== osd_op_reply(184 notify.0 [watch unwatch cookie 94507665082672] v30'257 uv255 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447cdbd60 2023-10-05T19:41:34.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.912+0000 7f9ce83fde40 20 remove_watcher() i=0 2023-10-05T19:41:34.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.912+0000 7f9ce83fde40 2 removed watcher, disabling cache 2023-10-05T19:41:34.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.912+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:185 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94507665095200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447cbb840 con 0x55f447c1a570 2023-10-05T19:41:34.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.916+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 50 ==== osd_op_reply(185 notify.1 [watch unwatch cookie 94507665095200] v30'257 uv255 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9cd8067040 con 0x55f447c1a570 2023-10-05T19:41:34.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.916+0000 7f9ce83fde40 20 remove_watcher() i=1 2023-10-05T19:41:34.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.916+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:186 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94507665098032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447cbb840 con 0x55f447ce0500 2023-10-05T19:41:34.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.916+0000 7f9ce5731700 1 -- 172.21.15.12:0/819885356 <== osd.3 v2:172.21.15.12:6816/29951 53 ==== osd_op_reply(186 notify.2 [watch unwatch cookie 94507665098032] v30'257 uv255 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9cdc027080 con 0x55f447ce0500 2023-10-05T19:41:34.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.916+0000 7f9ce83fde40 20 remove_watcher() i=2 2023-10-05T19:41:34.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.916+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:187 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94507665102960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447cbb840 con 0x55f447cdbd60 2023-10-05T19:41:34.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.920+0000 7f9ce5f32700 1 -- 172.21.15.12:0/819885356 <== osd.2 v2:172.21.15.12:6800/29934 38 ==== osd_op_reply(187 notify.3 [watch unwatch cookie 94507665102960] v30'257 uv255 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9ce002b070 con 0x55f447cdbd60 2023-10-05T19:41:34.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.920+0000 7f9ce83fde40 20 remove_watcher() i=3 2023-10-05T19:41:34.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.920+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:188 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94507665104064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447cbb840 con 0x55f447c1a570 2023-10-05T19:41:34.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.924+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 51 ==== osd_op_reply(188 notify.4 [watch unwatch cookie 94507665104064] v30'257 uv255 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9cd8067040 con 0x55f447c1a570 2023-10-05T19:41:34.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.924+0000 7f9ce83fde40 20 remove_watcher() i=4 2023-10-05T19:41:34.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.924+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:189 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94507665107376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447cbb840 con 0x7f9ce0010070 2023-10-05T19:41:34.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.928+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 61 ==== osd_op_reply(189 notify.5 [watch unwatch cookie 94507665107376] v30'257 uv255 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9cd8067040 con 0x7f9ce0010070 2023-10-05T19:41:34.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.928+0000 7f9ce83fde40 20 remove_watcher() i=5 2023-10-05T19:41:34.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.928+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:190 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94507665100384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447cbb840 con 0x7f9ce0010070 2023-10-05T19:41:34.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.928+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 62 ==== osd_op_reply(190 notify.6 [watch unwatch cookie 94507665100384] v30'257 uv255 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9cd8067040 con 0x7f9ce0010070 2023-10-05T19:41:34.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.928+0000 7f9ce83fde40 20 remove_watcher() i=6 2023-10-05T19:41:34.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.928+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:191 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94507665111056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447cbb840 con 0x7f9ce0010070 2023-10-05T19:41:34.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.932+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.0 v2:172.21.15.8:6800/27563 63 ==== osd_op_reply(191 notify.7 [watch unwatch cookie 94507665111056] v30'257 uv255 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9cd8067040 con 0x7f9ce0010070 2023-10-05T19:41:34.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.932+0000 7f9ce83fde40 20 remove_watcher() i=7 2023-10-05T19:41:34.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.936+0000 7f9cb6fe5700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:41:34.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.936+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:192 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94507665113632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f447d39540 con 0x55f447c1a570 2023-10-05T19:41:34.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.936+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 <== osd.1 v2:172.21.15.12:6808/29933 52 ==== osd_op_reply(192 data_loggenerations_metadata [watch unwatch cookie 94507665113632] v30'248 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f9cd8067040 con 0x55f447c1a570 2023-10-05T19:41:34.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.936+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f9ce0010070 msgr2=0x7f9ce001c6e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:34.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.936+0000 7f9ce83fde40 1 --2- 172.21.15.12:0/819885356 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f9ce0010070 0x7f9ce001c6e0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:34.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.936+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55f447c1a570 msgr2=0x55f447cd97b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:34.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.936+0000 7f9ce83fde40 1 --2- 172.21.15.12:0/819885356 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55f447c1a570 0x55f447cd97b0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:34.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.936+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55f447cdbd60 msgr2=0x55f447cde140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:34.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.936+0000 7f9ce83fde40 1 --2- 172.21.15.12:0/819885356 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55f447cdbd60 0x55f447cde140 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:34.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.936+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55f447ce0500 msgr2=0x55f447ce2910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:34.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.936+0000 7f9ce83fde40 1 --2- 172.21.15.12:0/819885356 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55f447ce0500 0x55f447ce2910 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:34.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.936+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f9cc8030ca0 msgr2=0x7f9cc8033150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:34.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.936+0000 7f9ce83fde40 1 --2- 172.21.15.12:0/819885356 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f9cc8030ca0 0x7f9cc8033150 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc001e70 tx=0x7f9cdc001390 comp rx=0 tx=0).stop 2023-10-05T19:41:34.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.940+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f447c19c60 msgr2=0x55f447ccddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:34.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.940+0000 7f9ce83fde40 1 --2- 172.21.15.12:0/819885356 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f447c19c60 0x55f447ccddb0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0005860 tx=0x7f9ce001c000 comp rx=0 tx=0).stop 2023-10-05T19:41:34.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.940+0000 7f9ce6733700 1 -- 172.21.15.12:0/819885356 reap_dead start 2023-10-05T19:41:34.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.940+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 shutdown_connections 2023-10-05T19:41:34.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.940+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 >> 172.21.15.12:0/819885356 conn(0x55f447ca0730 msgr2=0x55f447cbf5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:34.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.940+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 shutdown_connections 2023-10-05T19:41:34.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.940+0000 7f9ce83fde40 1 -- 172.21.15.12:0/819885356 wait complete. 2023-10-05T19:41:34.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.940+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55f447d0d930 msgr2=0x55f447d0fd10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:34.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.940+0000 7f9ce83fde40 1 --2- 172.21.15.12:0/2605150000 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x55f447d0d930 0x55f447d0fd10 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:34.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.940+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55f447d0a200 msgr2=0x55f447d0c610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:34.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.940+0000 7f9ce83fde40 1 --2- 172.21.15.12:0/2605150000 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55f447d0a200 0x55f447d0c610 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:34.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.940+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55f447d14f10 msgr2=0x55f447d172f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:34.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.940+0000 7f9ce83fde40 1 --2- 172.21.15.12:0/2605150000 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55f447d14f10 0x55f447d172f0 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:34.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.940+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55f447d11110 msgr2=0x55f447d134f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:34.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.940+0000 7f9ce83fde40 1 --2- 172.21.15.12:0/2605150000 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55f447d11110 0x55f447d134f0 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:34.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.940+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f9c2c030c50 msgr2=0x7f9c2c033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:34.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.940+0000 7f9ce83fde40 1 --2- 172.21.15.12:0/2605150000 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f9c2c030c50 0x7f9c2c033100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x55f447cec060 tx=0x7f9cd80325c0 comp rx=0 tx=0).stop 2023-10-05T19:41:34.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.940+0000 7f9ce5f32700 1 -- 172.21.15.12:0/2605150000 reap_dead start 2023-10-05T19:41:34.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.940+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f447cf9a90 msgr2=0x55f447cff5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:41:34.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.940+0000 7f9ce83fde40 1 --2- 172.21.15.12:0/2605150000 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f447cf9a90 0x55f447cff5e0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f9ce00461b0 tx=0x7f9cdc03c000 comp rx=0 tx=0).stop 2023-10-05T19:41:34.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.940+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 shutdown_connections 2023-10-05T19:41:34.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.940+0000 7f9ce83fde40 1 --2- 172.21.15.12:0/2605150000 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f447cf9a90 0x55f447cff5e0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:41:34.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.940+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 >> 172.21.15.12:0/2605150000 conn(0x55f447cf4e50 msgr2=0x55f447cf2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:41:34.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.940+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 shutdown_connections 2023-10-05T19:41:34.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:41:34.940+0000 7f9ce83fde40 1 -- 172.21.15.12:0/2605150000 wait complete. 2023-10-05T19:42:14.954 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'trim', '--uid', 'foo'] 2023-10-05T19:42:14.954 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'trim', '--uid', 'foo'] 2023-10-05T19:42:14.955 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage trim --uid foo 2023-10-05T19:42:14.997 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:42:14.997 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:42:14.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:14.992+0000 7fe18c4ebe40 1 Processor -- start 2023-10-05T19:42:14.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:14.992+0000 7fe18c4ebe40 1 -- start start 2023-10-05T19:42:14.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:14.992+0000 7fe18c4ebe40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55eda6171e30 0x55eda6172200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:14.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:14.992+0000 7fe18c4ebe40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55eda60bebd0 con 0x55eda6172740 2023-10-05T19:42:14.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:14.992+0000 7fe18c4ebe40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x55eda607bee0 con 0x55eda6171e30 2023-10-05T19:42:14.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:14.992+0000 7fe18a020700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55eda6171e30 0x55eda6172200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:14.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:14.992+0000 7fe18981f700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x55eda6172740 0x55eda61731e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:47328/0 (socket says 172.21.15.12:47328) 2023-10-05T19:42:14.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:14.992+0000 7fe18981f700 1 -- 172.21.15.12:0/3685476369 learned_addr learned my addr 172.21.15.12:0/3685476369 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:42:15.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:14.992+0000 7fe18a020700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55eda6171e30 0x55eda6172200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:54180/0 (socket says 172.21.15.12:54180) 2023-10-05T19:42:15.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:14.992+0000 7fe18901e700 1 -- 172.21.15.12:0/3685476369 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2130443229 0 0) 0x55eda60bebd0 con 0x55eda6172740 2023-10-05T19:42:15.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:14.996+0000 7fe18901e700 1 -- 172.21.15.12:0/3685476369 --> v1:172.21.15.8:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe178001e10 con 0x55eda6172740 2023-10-05T19:42:15.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:14.996+0000 7fe18a020700 1 -- 172.21.15.12:0/3685476369 >> v1:172.21.15.8:6789/0 conn(0x55eda6172740 legacy=0x55eda61731e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:15.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:14.996+0000 7fe18a020700 1 -- 172.21.15.12:0/3685476369 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eda61738f0 con 0x55eda6171e30 2023-10-05T19:42:15.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:14.996+0000 7fe18a020700 1 --2- 172.21.15.12:0/3685476369 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eda6171e30 0x55eda6172200 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fe17c00a5d0 tx=0x7fe17c005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b9240ffcf8a27ef server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:15.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:14.996+0000 7fe18901e700 1 -- 172.21.15.12:0/3685476369 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe17c007610 con 0x55eda6171e30 2023-10-05T19:42:15.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:14.996+0000 7fe18901e700 1 -- 172.21.15.12:0/3685476369 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe17c007770 con 0x55eda6171e30 2023-10-05T19:42:15.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:14.996+0000 7fe18901e700 1 -- 172.21.15.12:0/3685476369 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe17c007a40 con 0x55eda6171e30 2023-10-05T19:42:15.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:14.996+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/3685476369 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eda6171e30 msgr2=0x55eda6172200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:15.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:14.996+0000 7fe18c4ebe40 1 --2- 172.21.15.12:0/3685476369 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eda6171e30 0x55eda6172200 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fe17c00a5d0 tx=0x7fe17c005a50 comp rx=0 tx=0).stop 2023-10-05T19:42:15.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:14.996+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/3685476369 shutdown_connections 2023-10-05T19:42:15.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:14.996+0000 7fe18c4ebe40 1 --2- 172.21.15.12:0/3685476369 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eda6171e30 0x55eda6172200 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:15.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:14.996+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/3685476369 >> 172.21.15.12:0/3685476369 conn(0x55eda616d850 msgr2=0x55eda616fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:15.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:14.996+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/3685476369 shutdown_connections 2023-10-05T19:42:15.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:14.996+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/3685476369 wait complete. 2023-10-05T19:42:15.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:14.996+0000 7fe18c4ebe40 1 Processor -- start 2023-10-05T19:42:15.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:14.996+0000 7fe18c4ebe40 1 -- start start 2023-10-05T19:42:15.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:14.996+0000 7fe18c4ebe40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eda6171e30 0x55eda60e78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:15.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:14.996+0000 7fe18c4ebe40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eda603fe30 con 0x55eda6171e30 2023-10-05T19:42:15.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:14.996+0000 7fe18a020700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eda6171e30 0x55eda60e78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:15.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:14.996+0000 7fe18a020700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eda6171e30 0x55eda60e78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:54188/0 (socket says 172.21.15.12:54188) 2023-10-05T19:42:15.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:14.996+0000 7fe18a020700 1 -- 172.21.15.12:0/2042662646 learned_addr learned my addr 172.21.15.12:0/2042662646 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:42:15.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.000+0000 7fe18a020700 1 -- 172.21.15.12:0/2042662646 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe17c005cf0 con 0x55eda6171e30 2023-10-05T19:42:15.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.000+0000 7fe18a020700 1 --2- 172.21.15.12:0/2042662646 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eda6171e30 0x55eda60e78e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fe17c000c00 tx=0x7fe17c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e28f9c737a960e6b server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:15.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.000+0000 7fe183fff700 1 -- 172.21.15.12:0/2042662646 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe17c0078e0 con 0x55eda6171e30 2023-10-05T19:42:15.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.000+0000 7fe183fff700 1 -- 172.21.15.12:0/2042662646 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe17c007a40 con 0x55eda6171e30 2023-10-05T19:42:15.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.000+0000 7fe183fff700 1 -- 172.21.15.12:0/2042662646 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe17c007bd0 con 0x55eda6171e30 2023-10-05T19:42:15.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.000+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/2042662646 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eda6171e30 msgr2=0x55eda60e78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:15.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.000+0000 7fe18c4ebe40 1 --2- 172.21.15.12:0/2042662646 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eda6171e30 0x55eda60e78e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fe17c000c00 tx=0x7fe17c014000 comp rx=0 tx=0).stop 2023-10-05T19:42:15.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.000+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/2042662646 shutdown_connections 2023-10-05T19:42:15.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.000+0000 7fe18c4ebe40 1 --2- 172.21.15.12:0/2042662646 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eda6171e30 0x55eda60e78e0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:15.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.000+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/2042662646 >> 172.21.15.12:0/2042662646 conn(0x55eda616d850 msgr2=0x55eda618c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:15.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.000+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/2042662646 shutdown_connections 2023-10-05T19:42:15.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.000+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/2042662646 wait complete. 2023-10-05T19:42:15.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.000+0000 7fe18c4ebe40 1 Processor -- start 2023-10-05T19:42:15.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.000+0000 7fe18c4ebe40 1 -- start start 2023-10-05T19:42:15.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.000+0000 7fe18c4ebe40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eda6171e30 0x55eda6199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:15.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.000+0000 7fe18c4ebe40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eda6012ed0 con 0x55eda6171e30 2023-10-05T19:42:15.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.000+0000 7fe18a020700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eda6171e30 0x55eda6199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:15.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.000+0000 7fe18a020700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eda6171e30 0x55eda6199b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:54200/0 (socket says 172.21.15.12:54200) 2023-10-05T19:42:15.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.000+0000 7fe18a020700 1 -- 172.21.15.12:0/976955935 learned_addr learned my addr 172.21.15.12:0/976955935 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:42:15.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.000+0000 7fe18a020700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eda61738f0 con 0x55eda6171e30 2023-10-05T19:42:15.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.004+0000 7fe18a020700 1 --2- 172.21.15.12:0/976955935 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eda6171e30 0x55eda6199b90 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fe17c01f040 tx=0x7fe17c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:15.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.004+0000 7fe1827fc700 1 -- 172.21.15.12:0/976955935 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe17c007e50 con 0x55eda6171e30 2023-10-05T19:42:15.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.004+0000 7fe1827fc700 1 -- 172.21.15.12:0/976955935 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe17c029070 con 0x55eda6171e30 2023-10-05T19:42:15.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.004+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55eda619a0d0 con 0x55eda6171e30 2023-10-05T19:42:15.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.004+0000 7fe1827fc700 1 -- 172.21.15.12:0/976955935 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe17c030480 con 0x55eda6171e30 2023-10-05T19:42:15.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.004+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eda619a510 con 0x55eda6171e30 2023-10-05T19:42:15.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.004+0000 7fe1827fc700 1 -- 172.21.15.12:0/976955935 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fe17c012400 con 0x55eda6171e30 2023-10-05T19:42:15.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.004+0000 7fe18c4ebe40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.004+0000 7fe1827fc700 1 --2- 172.21.15.12:0/976955935 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe16c030c40 0x7fe16c0330f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:15.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.004+0000 7fe1827fc700 1 -- 172.21.15.12:0/976955935 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7fe17c02b070 con 0x55eda6171e30 2023-10-05T19:42:15.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.004+0000 7fe18981f700 1 --2- 172.21.15.12:0/976955935 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe16c030c40 0x7fe16c0330f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:15.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.004+0000 7fe18c4ebe40 1 --2- 172.21.15.12:0/976955935 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55eda6172740 0x55eda61a5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:15.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.004+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61a5cc0 con 0x55eda6172740 2023-10-05T19:42:15.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.004+0000 7fe18a821700 1 --2- 172.21.15.12:0/976955935 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55eda6172740 0x55eda61a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:15.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.008+0000 7fe18981f700 1 --2- 172.21.15.12:0/976955935 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe16c030c40 0x7fe16c0330f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe1740044f0 tx=0x7fe174006690 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:15.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.008+0000 7fe18a821700 1 --2- 172.21.15.12:0/976955935 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55eda6172740 0x55eda61a5650 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:15.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.008+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe18400a500 con 0x55eda6172740 2023-10-05T19:42:15.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.008+0000 7fe18c4ebe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:15.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.008+0000 7fe18c4ebe40 20 realm 2023-10-05T19:42:15.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.008+0000 7fe18c4ebe40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.008+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61a2cb0 con 0x55eda6172740 2023-10-05T19:42:15.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.008+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe18400a500 con 0x55eda6172740 2023-10-05T19:42:15.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.008+0000 7fe18c4ebe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:15.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.008+0000 7fe18c4ebe40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:42:15.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.008+0000 7fe18c4ebe40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.008+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61a67a0 con 0x55eda6172740 2023-10-05T19:42:15.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.008+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe18400a500 con 0x55eda6172740 2023-10-05T19:42:15.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.008+0000 7fe18c4ebe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:15.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.008+0000 7fe18c4ebe40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.008+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61a63b0 con 0x55eda6172740 2023-10-05T19:42:15.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.008+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fe18400a500 con 0x55eda6172740 2023-10-05T19:42:15.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.008+0000 7fe18c4ebe40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:42:15.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.008+0000 7fe18c4ebe40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.008+0000 7fe18c4ebe40 1 --2- 172.21.15.12:0/976955935 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55eda61a7b40 0x55eda61a9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:15.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.008+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61aa460 con 0x55eda61a7b40 2023-10-05T19:42:15.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.008+0000 7fe18a020700 1 --2- 172.21.15.12:0/976955935 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55eda61a7b40 0x55eda61a9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:15.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.012+0000 7fe18a020700 1 --2- 172.21.15.12:0/976955935 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55eda61a7b40 0x55eda61a9f20 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:15.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.012+0000 7fe18a020700 1 -- 172.21.15.12:0/976955935 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fe17c01c7f0 con 0x55eda61a7b40 2023-10-05T19:42:15.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.012+0000 7fe18c4ebe40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:42:15.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.012+0000 7fe18c4ebe40 20 searching for the correct realm 2023-10-05T19:42:15.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.012+0000 7fe18c4ebe40 1 --2- 172.21.15.12:0/976955935 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55eda61ac250 0x55eda61ae630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:15.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.012+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61aece0 con 0x55eda61ac250 2023-10-05T19:42:15.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.012+0000 7fe18981f700 1 --2- 172.21.15.12:0/976955935 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55eda61ac250 0x55eda61ae630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:15.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.012+0000 7fe18981f700 1 --2- 172.21.15.12:0/976955935 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55eda61ac250 0x55eda61ae630 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:15.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.016+0000 7fe18981f700 1 -- 172.21.15.12:0/976955935 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fe174004520 con 0x55eda61ac250 2023-10-05T19:42:15.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.016+0000 7fe18981f700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61aece0 con 0x55eda61a7b40 2023-10-05T19:42:15.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.016+0000 7fe18a020700 1 -- 172.21.15.12:0/976955935 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe17c01c7f0 con 0x55eda61a7b40 2023-10-05T19:42:15.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.016+0000 7fe18a020700 1 --2- 172.21.15.12:0/976955935 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fe17c007030 0x7fe17c00c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:15.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.016+0000 7fe18a020700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61aece0 con 0x7fe17c007030 2023-10-05T19:42:15.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.016+0000 7fe18a821700 1 --2- 172.21.15.12:0/976955935 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fe17c007030 0x7fe17c00c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:15.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.016+0000 7fe18a821700 1 --2- 172.21.15.12:0/976955935 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fe17c007030 0x7fe17c00c9c0 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:15.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.016+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe184014040 con 0x7fe17c007030 2023-10-05T19:42:15.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.016+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61aece0 con 0x55eda6172740 2023-10-05T19:42:15.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.020+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe184014040 con 0x55eda6172740 2023-10-05T19:42:15.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.020+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61aece0 con 0x55eda6172740 2023-10-05T19:42:15.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.020+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe184014040 con 0x55eda6172740 2023-10-05T19:42:15.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.020+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61aece0 con 0x55eda6172740 2023-10-05T19:42:15.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.020+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(11 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe184014040 con 0x55eda6172740 2023-10-05T19:42:15.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.020+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61aece0 con 0x7fe17c007030 2023-10-05T19:42:15.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.020+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe184014040 con 0x7fe17c007030 2023-10-05T19:42:15.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.020+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61aece0 con 0x55eda61a7b40 2023-10-05T19:42:15.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.020+0000 7fe18a020700 1 -- 172.21.15.12:0/976955935 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe17c01c7f0 con 0x55eda61a7b40 2023-10-05T19:42:15.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.020+0000 7fe18a020700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61aece0 con 0x55eda61a7b40 2023-10-05T19:42:15.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.020+0000 7fe18a020700 1 -- 172.21.15.12:0/976955935 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe17c01c7f0 con 0x55eda61a7b40 2023-10-05T19:42:15.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.020+0000 7fe18a020700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61aece0 con 0x55eda6172740 2023-10-05T19:42:15.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.020+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe184014040 con 0x55eda6172740 2023-10-05T19:42:15.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.024+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61aece0 con 0x7fe17c007030 2023-10-05T19:42:15.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.024+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe184014040 con 0x7fe17c007030 2023-10-05T19:42:15.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.024+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61aece0 con 0x55eda6172740 2023-10-05T19:42:15.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.024+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe184014040 con 0x55eda6172740 2023-10-05T19:42:15.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.024+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61aece0 con 0x55eda61a7b40 2023-10-05T19:42:15.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.024+0000 7fe18a020700 1 -- 172.21.15.12:0/976955935 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(18 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe17c01c7f0 con 0x55eda61a7b40 2023-10-05T19:42:15.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.024+0000 7fe18a020700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61aece0 con 0x55eda61ac250 2023-10-05T19:42:15.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.024+0000 7fe18981f700 1 -- 172.21.15.12:0/976955935 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe174004520 con 0x55eda61ac250 2023-10-05T19:42:15.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.024+0000 7fe18981f700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61aece0 con 0x55eda61ac250 2023-10-05T19:42:15.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.028+0000 7fe18981f700 1 -- 172.21.15.12:0/976955935 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe174004520 con 0x55eda61ac250 2023-10-05T19:42:15.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.028+0000 7fe18981f700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61aece0 con 0x7fe17c007030 2023-10-05T19:42:15.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.028+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe184014040 con 0x7fe17c007030 2023-10-05T19:42:15.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.028+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61aece0 con 0x55eda61ac250 2023-10-05T19:42:15.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.028+0000 7fe18981f700 1 -- 172.21.15.12:0/976955935 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe17401d040 con 0x55eda61ac250 2023-10-05T19:42:15.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.028+0000 7fe18981f700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61aece0 con 0x55eda61ac250 2023-10-05T19:42:15.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.028+0000 7fe18981f700 1 -- 172.21.15.12:0/976955935 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe17401d040 con 0x55eda61ac250 2023-10-05T19:42:15.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.028+0000 7fe18981f700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61aece0 con 0x55eda6172740 2023-10-05T19:42:15.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.028+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe184014040 con 0x55eda6172740 2023-10-05T19:42:15.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.028+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61aece0 con 0x55eda6172740 2023-10-05T19:42:15.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.032+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe184014040 con 0x55eda6172740 2023-10-05T19:42:15.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.032+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61aece0 con 0x7fe17c007030 2023-10-05T19:42:15.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.032+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe184014040 con 0x7fe17c007030 2023-10-05T19:42:15.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.032+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61aece0 con 0x55eda61ac250 2023-10-05T19:42:15.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.032+0000 7fe18981f700 1 -- 172.21.15.12:0/976955935 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe17401d040 con 0x55eda61ac250 2023-10-05T19:42:15.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.032+0000 7fe18981f700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61aece0 con 0x55eda61a7b40 2023-10-05T19:42:15.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.032+0000 7fe18a020700 1 -- 172.21.15.12:0/976955935 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe17c01c7f0 con 0x55eda61a7b40 2023-10-05T19:42:15.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.032+0000 7fe18a020700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61aece0 con 0x55eda6172740 2023-10-05T19:42:15.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.036+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe184014040 con 0x55eda6172740 2023-10-05T19:42:15.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.036+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61aece0 con 0x7fe17c007030 2023-10-05T19:42:15.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.036+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe184014040 con 0x7fe17c007030 2023-10-05T19:42:15.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.036+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61aece0 con 0x55eda61ac250 2023-10-05T19:42:15.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.036+0000 7fe18981f700 1 -- 172.21.15.12:0/976955935 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe17401d040 con 0x55eda61ac250 2023-10-05T19:42:15.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.036+0000 7fe18981f700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61aece0 con 0x7fe17c007030 2023-10-05T19:42:15.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.036+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe184014040 con 0x7fe17c007030 2023-10-05T19:42:15.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.036+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61aece0 con 0x55eda61a7b40 2023-10-05T19:42:15.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.040+0000 7fe18a020700 1 -- 172.21.15.12:0/976955935 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe17c01c7f0 con 0x55eda61a7b40 2023-10-05T19:42:15.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.040+0000 7fe18a020700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61aece0 con 0x55eda61ac250 2023-10-05T19:42:15.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.040+0000 7fe18981f700 1 -- 172.21.15.12:0/976955935 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe17401d040 con 0x55eda61ac250 2023-10-05T19:42:15.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.040+0000 7fe18981f700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61aece0 con 0x7fe17c007030 2023-10-05T19:42:15.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.040+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe184014040 con 0x7fe17c007030 2023-10-05T19:42:15.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.040+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61aece0 con 0x55eda6172740 2023-10-05T19:42:15.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.040+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fe184014040 con 0x55eda6172740 2023-10-05T19:42:15.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.040+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61aece0 con 0x55eda61a7b40 2023-10-05T19:42:15.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.044+0000 7fe18a020700 1 -- 172.21.15.12:0/976955935 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(37 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fe17c01c7f0 con 0x55eda61a7b40 2023-10-05T19:42:15.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.044+0000 7fe18c4ebe40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:42:15.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.044+0000 7fe18c4ebe40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:42:15.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.044+0000 7fe18c4ebe40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:42:15.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.044+0000 7fe18c4ebe40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:42:15.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.044+0000 7fe18c4ebe40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.044+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61aafd0 con 0x55eda6172740 2023-10-05T19:42:15.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.044+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe184014040 con 0x55eda6172740 2023-10-05T19:42:15.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.044+0000 7fe18c4ebe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:15.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.044+0000 7fe18c4ebe40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.044+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61aac00 con 0x55eda6172740 2023-10-05T19:42:15.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.044+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fe184014040 con 0x55eda6172740 2023-10-05T19:42:15.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.044+0000 7fe18c4ebe40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:42:15.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.044+0000 7fe18c4ebe40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.044+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61af150 con 0x55eda61ac250 2023-10-05T19:42:15.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.044+0000 7fe18981f700 1 -- 172.21.15.12:0/976955935 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fe17401d040 con 0x55eda61ac250 2023-10-05T19:42:15.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.044+0000 7fe18c4ebe40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:42:15.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.044+0000 7fe18c4ebe40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.044+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61af500 con 0x55eda61ac250 2023-10-05T19:42:15.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.044+0000 7fe18981f700 1 -- 172.21.15.12:0/976955935 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7fe17401d040 con 0x55eda61ac250 2023-10-05T19:42:15.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.044+0000 7fe18c4ebe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:15.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.044+0000 7fe18c4ebe40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.044+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61af8b0 con 0x55eda6172740 2023-10-05T19:42:15.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.044+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe184014040 con 0x55eda6172740 2023-10-05T19:42:15.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.044+0000 7fe18c4ebe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:15.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.044+0000 7fe18c4ebe40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.044+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61ab460 con 0x55eda6172740 2023-10-05T19:42:15.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.048+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fe184014040 con 0x55eda6172740 2023-10-05T19:42:15.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.048+0000 7fe18c4ebe40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:42:15.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.048+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61ab430 con 0x55eda61ac250 2023-10-05T19:42:15.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.048+0000 7fe18981f700 1 -- 172.21.15.12:0/976955935 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(44 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fe17401d040 con 0x55eda61ac250 2023-10-05T19:42:15.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.048+0000 7fe18981f700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61ab430 con 0x55eda61a7b40 2023-10-05T19:42:15.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.052+0000 7fe18a020700 1 -- 172.21.15.12:0/976955935 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(45 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe17c01c7f0 con 0x55eda61a7b40 2023-10-05T19:42:15.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.052+0000 7fe18a020700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61ab430 con 0x7fe17c007030 2023-10-05T19:42:15.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.052+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(46 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe184014040 con 0x7fe17c007030 2023-10-05T19:42:15.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.052+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61af8b0 con 0x55eda6172740 2023-10-05T19:42:15.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.052+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(47 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe184014040 con 0x55eda6172740 2023-10-05T19:42:15.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.052+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61af8b0 con 0x55eda6172740 2023-10-05T19:42:15.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.052+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(48 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe184014040 con 0x55eda6172740 2023-10-05T19:42:15.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.052+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61af8b0 con 0x55eda6172740 2023-10-05T19:42:15.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.052+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(49 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe184014040 con 0x55eda6172740 2023-10-05T19:42:15.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.052+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61af8b0 con 0x7fe17c007030 2023-10-05T19:42:15.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.056+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(50 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe184014040 con 0x7fe17c007030 2023-10-05T19:42:15.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.056+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61af8b0 con 0x55eda61a7b40 2023-10-05T19:42:15.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.056+0000 7fe18a020700 1 -- 172.21.15.12:0/976955935 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(51 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe17c01c7f0 con 0x55eda61a7b40 2023-10-05T19:42:15.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.056+0000 7fe18a020700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61af8b0 con 0x55eda61a7b40 2023-10-05T19:42:15.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.056+0000 7fe18a020700 1 -- 172.21.15.12:0/976955935 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(52 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe17c01c7f0 con 0x55eda61a7b40 2023-10-05T19:42:15.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.056+0000 7fe18a020700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61af8b0 con 0x55eda6172740 2023-10-05T19:42:15.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.056+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(53 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe184014040 con 0x55eda6172740 2023-10-05T19:42:15.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.056+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61af8b0 con 0x7fe17c007030 2023-10-05T19:42:15.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.060+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(54 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe184014040 con 0x7fe17c007030 2023-10-05T19:42:15.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.060+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61af8b0 con 0x55eda6172740 2023-10-05T19:42:15.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.060+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(55 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe184014040 con 0x55eda6172740 2023-10-05T19:42:15.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.060+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61af8b0 con 0x55eda61a7b40 2023-10-05T19:42:15.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.060+0000 7fe18a020700 1 -- 172.21.15.12:0/976955935 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(56 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe17c01c7f0 con 0x55eda61a7b40 2023-10-05T19:42:15.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.060+0000 7fe18a020700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61af8b0 con 0x55eda61ac250 2023-10-05T19:42:15.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.060+0000 7fe18981f700 1 -- 172.21.15.12:0/976955935 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(57 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe17401d040 con 0x55eda61ac250 2023-10-05T19:42:15.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.060+0000 7fe18981f700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61af8b0 con 0x55eda61ac250 2023-10-05T19:42:15.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.060+0000 7fe18981f700 1 -- 172.21.15.12:0/976955935 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(58 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe17402d040 con 0x55eda61ac250 2023-10-05T19:42:15.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.060+0000 7fe18981f700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61af8b0 con 0x7fe17c007030 2023-10-05T19:42:15.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.064+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(59 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe184014040 con 0x7fe17c007030 2023-10-05T19:42:15.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.064+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61af8b0 con 0x55eda61ac250 2023-10-05T19:42:15.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.064+0000 7fe18981f700 1 -- 172.21.15.12:0/976955935 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(60 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe17402d040 con 0x55eda61ac250 2023-10-05T19:42:15.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.064+0000 7fe18981f700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61af8b0 con 0x55eda61ac250 2023-10-05T19:42:15.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.064+0000 7fe18981f700 1 -- 172.21.15.12:0/976955935 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(61 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe17402d040 con 0x55eda61ac250 2023-10-05T19:42:15.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.064+0000 7fe18981f700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61af8b0 con 0x55eda6172740 2023-10-05T19:42:15.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.064+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(62 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe184014040 con 0x55eda6172740 2023-10-05T19:42:15.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.064+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61af8b0 con 0x55eda6172740 2023-10-05T19:42:15.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.064+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(63 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe184014040 con 0x55eda6172740 2023-10-05T19:42:15.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.064+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61af8b0 con 0x7fe17c007030 2023-10-05T19:42:15.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.068+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(64 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe184014040 con 0x7fe17c007030 2023-10-05T19:42:15.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.068+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61af8b0 con 0x55eda61ac250 2023-10-05T19:42:15.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.068+0000 7fe18981f700 1 -- 172.21.15.12:0/976955935 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(65 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe17402d040 con 0x55eda61ac250 2023-10-05T19:42:15.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.068+0000 7fe18981f700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61af8b0 con 0x55eda61a7b40 2023-10-05T19:42:15.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.068+0000 7fe18a020700 1 -- 172.21.15.12:0/976955935 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(66 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe17c01c7f0 con 0x55eda61a7b40 2023-10-05T19:42:15.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.068+0000 7fe18a020700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61af8b0 con 0x55eda6172740 2023-10-05T19:42:15.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.068+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(67 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe184014040 con 0x55eda6172740 2023-10-05T19:42:15.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.068+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61af8b0 con 0x7fe17c007030 2023-10-05T19:42:15.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.072+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(68 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe184014040 con 0x7fe17c007030 2023-10-05T19:42:15.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.072+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61af8b0 con 0x55eda61ac250 2023-10-05T19:42:15.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.072+0000 7fe18981f700 1 -- 172.21.15.12:0/976955935 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(69 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe17402d040 con 0x55eda61ac250 2023-10-05T19:42:15.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.072+0000 7fe18981f700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61af8b0 con 0x7fe17c007030 2023-10-05T19:42:15.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.072+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(70 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe184014040 con 0x7fe17c007030 2023-10-05T19:42:15.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.072+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61af8b0 con 0x55eda61a7b40 2023-10-05T19:42:15.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.072+0000 7fe18a020700 1 -- 172.21.15.12:0/976955935 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(71 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe17c01c7f0 con 0x55eda61a7b40 2023-10-05T19:42:15.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.072+0000 7fe18a020700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61af8b0 con 0x55eda61ac250 2023-10-05T19:42:15.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.076+0000 7fe18981f700 1 -- 172.21.15.12:0/976955935 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(72 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe17402d040 con 0x55eda61ac250 2023-10-05T19:42:15.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.076+0000 7fe18981f700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61af8b0 con 0x7fe17c007030 2023-10-05T19:42:15.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.076+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(73 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe184014040 con 0x7fe17c007030 2023-10-05T19:42:15.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.076+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61af8b0 con 0x55eda6172740 2023-10-05T19:42:15.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.076+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(74 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fe184014040 con 0x55eda6172740 2023-10-05T19:42:15.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.076+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61af8b0 con 0x55eda61a7b40 2023-10-05T19:42:15.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.076+0000 7fe18a020700 1 -- 172.21.15.12:0/976955935 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(75 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fe17c01c7f0 con 0x55eda61a7b40 2023-10-05T19:42:15.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.076+0000 7fe18c4ebe40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:42:15.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.076+0000 7fe18c4ebe40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:42:15.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.076+0000 7fe18c4ebe40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:42:15.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.076+0000 7fe18c4ebe40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:42:15.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.076+0000 7fe18c4ebe40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.076+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61ab430 con 0x55eda6172740 2023-10-05T19:42:15.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.080+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fe184014040 con 0x55eda6172740 2023-10-05T19:42:15.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.080+0000 7fe18c4ebe40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:42:15.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.080+0000 7fe18c4ebe40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.080+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61b01a0 con 0x55eda61a7b40 2023-10-05T19:42:15.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.080+0000 7fe18a020700 1 -- 172.21.15.12:0/976955935 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fe17c01c7f0 con 0x55eda61a7b40 2023-10-05T19:42:15.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.080+0000 7fe18c4ebe40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:42:15.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.080+0000 7fe18c4ebe40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.080+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61b0a90 con 0x55eda6172740 2023-10-05T19:42:15.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.080+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fe184014040 con 0x55eda6172740 2023-10-05T19:42:15.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.080+0000 7fe18c4ebe40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:42:15.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.080+0000 7fe18c4ebe40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.080+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61b0a90 con 0x55eda61ac250 2023-10-05T19:42:15.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.080+0000 7fe18981f700 1 -- 172.21.15.12:0/976955935 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7fe17402d040 con 0x55eda61ac250 2023-10-05T19:42:15.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.080+0000 7fe18c4ebe40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:42:15.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.080+0000 7fe18c4ebe40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.080+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61b1200 con 0x7fe17c007030 2023-10-05T19:42:15.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.080+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7fe184014040 con 0x7fe17c007030 2023-10-05T19:42:15.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.080+0000 7fe18c4ebe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:15.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.080+0000 7fe18c4ebe40 20 zone default found 2023-10-05T19:42:15.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.080+0000 7fe18c4ebe40 4 Realm: () 2023-10-05T19:42:15.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.080+0000 7fe18c4ebe40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:42:15.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.080+0000 7fe18c4ebe40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:42:15.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.080+0000 7fe18c4ebe40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:42:15.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.080+0000 7fe18c4ebe40 20 zonegroup default 2023-10-05T19:42:15.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.080+0000 7fe18c4ebe40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.080+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61b1200 con 0x7fe17c007030 2023-10-05T19:42:15.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.084+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fe184014040 con 0x7fe17c007030 2023-10-05T19:42:15.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.084+0000 7fe18c4ebe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:15.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.084+0000 7fe18c4ebe40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.084+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61b2b80 con 0x55eda61ac250 2023-10-05T19:42:15.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.084+0000 7fe18981f700 1 -- 172.21.15.12:0/976955935 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fe17402d040 con 0x55eda61ac250 2023-10-05T19:42:15.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.084+0000 7fe18c4ebe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:15.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.084+0000 7fe18c4ebe40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.084+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61b2f30 con 0x55eda61ac250 2023-10-05T19:42:15.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.084+0000 7fe18981f700 1 -- 172.21.15.12:0/976955935 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fe17402d040 con 0x55eda61ac250 2023-10-05T19:42:15.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.084+0000 7fe18c4ebe40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:15.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.084+0000 7fe18c4ebe40 20 started sync module instance, tier type = 2023-10-05T19:42:15.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.084+0000 7fe18c4ebe40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:42:15.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.084+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61b2240 con 0x55eda61a7b40 2023-10-05T19:42:15.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.088+0000 7fe18a020700 1 -- 172.21.15.12:0/976955935 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(84 notify.0 [create] v30'258 uv258 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe17c01c7f0 con 0x55eda61a7b40 2023-10-05T19:42:15.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.088+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94479182398880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61b32e0 con 0x55eda61a7b40 2023-10-05T19:42:15.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.088+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61b3b80 con 0x55eda6172740 2023-10-05T19:42:15.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.092+0000 7fe18a020700 1 -- 172.21.15.12:0/976955935 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(85 notify.0 [watch watch cookie 94479182398880] v30'259 uv258 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe17c01c7f0 con 0x55eda61a7b40 2023-10-05T19:42:15.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.092+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(86 notify.1 [create] v30'258 uv258 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe184014040 con 0x55eda6172740 2023-10-05T19:42:15.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.092+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94479182411408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61b27a0 con 0x55eda6172740 2023-10-05T19:42:15.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.092+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61b4700 con 0x55eda61ac250 2023-10-05T19:42:15.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.096+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(87 notify.1 [watch watch cookie 94479182411408] v30'259 uv258 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe184014040 con 0x55eda6172740 2023-10-05T19:42:15.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.096+0000 7fe18981f700 1 -- 172.21.15.12:0/976955935 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(88 notify.2 [create] v30'258 uv258 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe17402d040 con 0x55eda61ac250 2023-10-05T19:42:15.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.096+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94479182412512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61b4b00 con 0x55eda61ac250 2023-10-05T19:42:15.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.096+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61b53e0 con 0x55eda61a7b40 2023-10-05T19:42:15.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.096+0000 7fe18a020700 1 -- 172.21.15.12:0/976955935 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(90 notify.3 [create] v30'258 uv258 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe17c01c7f0 con 0x55eda61a7b40 2023-10-05T19:42:15.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.096+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94479182417584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61b5cd0 con 0x55eda61a7b40 2023-10-05T19:42:15.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.096+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61b66e0 con 0x55eda6172740 2023-10-05T19:42:15.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.100+0000 7fe18981f700 1 -- 172.21.15.12:0/976955935 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(89 notify.2 [watch watch cookie 94479182412512] v30'259 uv258 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe17402d040 con 0x55eda61ac250 2023-10-05T19:42:15.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.100+0000 7fe18a020700 1 -- 172.21.15.12:0/976955935 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(91 notify.3 [watch watch cookie 94479182417584] v30'259 uv258 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe17c01c7f0 con 0x55eda61a7b40 2023-10-05T19:42:15.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.100+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(92 notify.4 [create] v30'258 uv258 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe184014040 con 0x55eda6172740 2023-10-05T19:42:15.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.100+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94479182419936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61b5c30 con 0x55eda6172740 2023-10-05T19:42:15.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.100+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61b6bd0 con 0x7fe17c007030 2023-10-05T19:42:15.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.104+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(93 notify.4 [watch watch cookie 94479182419936] v30'259 uv258 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe184014040 con 0x55eda6172740 2023-10-05T19:42:15.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.104+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(94 notify.5 [create] v30'258 uv258 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe184014040 con 0x7fe17c007030 2023-10-05T19:42:15.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.104+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94479182422512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61b6b30 con 0x7fe17c007030 2023-10-05T19:42:15.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.104+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61b73d0 con 0x7fe17c007030 2023-10-05T19:42:15.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.108+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(96 notify.6 [create] v30'258 uv258 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe184014040 con 0x7fe17c007030 2023-10-05T19:42:15.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.108+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94479182425440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61b77d0 con 0x7fe17c007030 2023-10-05T19:42:15.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.108+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61b8330 con 0x7fe17c007030 2023-10-05T19:42:15.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.108+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(95 notify.5 [watch watch cookie 94479182422512] v30'259 uv258 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe184014040 con 0x7fe17c007030 2023-10-05T19:42:15.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.108+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(97 notify.6 [watch watch cookie 94479182425440] v30'259 uv258 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe184014040 con 0x7fe17c007030 2023-10-05T19:42:15.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.112+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(98 notify.7 [create] v30'258 uv258 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe184014040 con 0x7fe17c007030 2023-10-05T19:42:15.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.112+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94479182423616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61b7e00 con 0x7fe17c007030 2023-10-05T19:42:15.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.112+0000 7fe18c4ebe40 20 add_watcher() i=0 2023-10-05T19:42:15.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.112+0000 7fe18c4ebe40 20 add_watcher() i=1 2023-10-05T19:42:15.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.112+0000 7fe18c4ebe40 20 add_watcher() i=2 2023-10-05T19:42:15.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.112+0000 7fe18c4ebe40 20 add_watcher() i=3 2023-10-05T19:42:15.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.112+0000 7fe18c4ebe40 20 add_watcher() i=4 2023-10-05T19:42:15.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.112+0000 7fe18c4ebe40 20 add_watcher() i=5 2023-10-05T19:42:15.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.112+0000 7fe18c4ebe40 20 add_watcher() i=6 2023-10-05T19:42:15.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.112+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(99 notify.7 [watch watch cookie 94479182423616] v30'259 uv258 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe184014040 con 0x7fe17c007030 2023-10-05T19:42:15.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.112+0000 7fe18c4ebe40 20 add_watcher() i=7 2023-10-05T19:42:15.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.112+0000 7fe18c4ebe40 2 all 8 watchers are set, enabling cache 2023-10-05T19:42:15.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.112+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61bb320 con 0x55eda6172740 2023-10-05T19:42:15.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.112+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fe184014040 con 0x55eda6172740 2023-10-05T19:42:15.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.112+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94479182429696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61b8740 con 0x55eda6172740 2023-10-05T19:42:15.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.116+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94479182429696] v30'249 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fe184014040 con 0x55eda6172740 2023-10-05T19:42:15.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.116+0000 7fe15bfdf700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:42:15.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.116+0000 7fe18c4ebe40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:42:15.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.116+0000 7fe18c4ebe40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:42:15.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.116+0000 7fe18c4ebe40 1 Processor -- start 2023-10-05T19:42:15.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.116+0000 7fe18c4ebe40 1 -- start start 2023-10-05T19:42:15.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.116+0000 7fe18c4ebe40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eda61c57b0 0x55eda61c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:15.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.116+0000 7fe18c4ebe40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eda61c60c0 con 0x55eda61c57b0 2023-10-05T19:42:15.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.116+0000 7fe18981f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eda61c57b0 0x55eda61c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:15.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.120+0000 7fe18981f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eda61c57b0 0x55eda61c5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:54214/0 (socket says 172.21.15.12:54214) 2023-10-05T19:42:15.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.120+0000 7fe18981f700 1 -- 172.21.15.12:0/3226364418 learned_addr learned my addr 172.21.15.12:0/3226364418 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:42:15.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.120+0000 7fe18981f700 1 -- 172.21.15.12:0/3226364418 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eda61c6200 con 0x55eda61c57b0 2023-10-05T19:42:15.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.120+0000 7fe18981f700 1 --2- 172.21.15.12:0/3226364418 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eda61c57b0 0x55eda61c5b80 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x55eda61a7580 tx=0x7fe174027180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16912fa451594bda server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:15.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.120+0000 7fe15b7de700 1 -- 172.21.15.12:0/3226364418 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe174039040 con 0x55eda61c57b0 2023-10-05T19:42:15.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.120+0000 7fe15b7de700 1 -- 172.21.15.12:0/3226364418 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe174046070 con 0x55eda61c57b0 2023-10-05T19:42:15.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.120+0000 7fe15b7de700 1 -- 172.21.15.12:0/3226364418 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe17402a040 con 0x55eda61c57b0 2023-10-05T19:42:15.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.120+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/3226364418 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eda61c57b0 msgr2=0x55eda61c5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:15.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.120+0000 7fe18c4ebe40 1 --2- 172.21.15.12:0/3226364418 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eda61c57b0 0x55eda61c5b80 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x55eda61a7580 tx=0x7fe174027180 comp rx=0 tx=0).stop 2023-10-05T19:42:15.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.120+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/3226364418 shutdown_connections 2023-10-05T19:42:15.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.120+0000 7fe18c4ebe40 1 --2- 172.21.15.12:0/3226364418 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eda61c57b0 0x55eda61c5b80 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:15.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.120+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/3226364418 >> 172.21.15.12:0/3226364418 conn(0x55eda61c0b70 msgr2=0x55eda61becc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:15.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.120+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/3226364418 shutdown_connections 2023-10-05T19:42:15.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.120+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/3226364418 wait complete. 2023-10-05T19:42:15.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.120+0000 7fe18c4ebe40 1 Processor -- start 2023-10-05T19:42:15.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.120+0000 7fe18c4ebe40 1 -- start start 2023-10-05T19:42:15.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.120+0000 7fe18c4ebe40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eda61c57b0 0x55eda61cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:15.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.120+0000 7fe18c4ebe40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eda61cc3c0 con 0x55eda61c57b0 2023-10-05T19:42:15.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.124+0000 7fe18981f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eda61c57b0 0x55eda61cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:15.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.124+0000 7fe18981f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eda61c57b0 0x55eda61cb3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:54224/0 (socket says 172.21.15.12:54224) 2023-10-05T19:42:15.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.124+0000 7fe18981f700 1 -- 172.21.15.12:0/2775072013 learned_addr learned my addr 172.21.15.12:0/2775072013 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:42:15.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.124+0000 7fe18981f700 1 -- 172.21.15.12:0/2775072013 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe178001e10 con 0x55eda61c57b0 2023-10-05T19:42:15.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.124+0000 7fe18981f700 1 --2- 172.21.15.12:0/2775072013 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eda61c57b0 0x55eda61cb3c0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x55eda61af090 tx=0x7fe174044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:15.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.124+0000 7fe159fdb700 1 -- 172.21.15.12:0/2775072013 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe174039070 con 0x55eda61c57b0 2023-10-05T19:42:15.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.124+0000 7fe159fdb700 1 -- 172.21.15.12:0/2775072013 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe17401b040 con 0x55eda61c57b0 2023-10-05T19:42:15.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.124+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/2775072013 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55eda61cb9c0 con 0x55eda61c57b0 2023-10-05T19:42:15.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.124+0000 7fe159fdb700 1 -- 172.21.15.12:0/2775072013 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe17402a030 con 0x55eda61c57b0 2023-10-05T19:42:15.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.124+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/2775072013 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eda61cbe00 con 0x55eda61c57b0 2023-10-05T19:42:15.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.124+0000 7fe159fdb700 1 -- 172.21.15.12:0/2775072013 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fe17403c030 con 0x55eda61c57b0 2023-10-05T19:42:15.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.124+0000 7fe159fdb700 1 --2- 172.21.15.12:0/2775072013 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe0d0030bb0 0x7fe0d0033060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:15.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.124+0000 7fe159fdb700 1 -- 172.21.15.12:0/2775072013 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7fe174057c20 con 0x55eda61c57b0 2023-10-05T19:42:15.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.124+0000 7fe18c4ebe40 5 note: GC not initialized 2023-10-05T19:42:15.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.124+0000 7fe18a821700 1 --2- 172.21.15.12:0/2775072013 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe0d0030bb0 0x7fe0d0033060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:15.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.128+0000 7fe18a821700 1 --2- 172.21.15.12:0/2775072013 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe0d0030bb0 0x7fe0d0033060 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x55eda61b73b0 tx=0x7fe1840335f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:15.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.128+0000 7fe0caffd700 20 reqs_thread_entry: start 2023-10-05T19:42:15.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.128+0000 7fe0c8ff9700 20 reqs_thread_entry: start 2023-10-05T19:42:15.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.128+0000 7fe18c4ebe40 20 init_complete bucket index max shards: 11 2023-10-05T19:42:15.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.128+0000 7fe18c4ebe40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:42:15.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.128+0000 7fe0baffd700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:42:45 2023 2023-10-05T19:42:15.136 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:42:15.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.128+0000 7fe18c4ebe40 20 get_system_obj_state: rctx=0x7ffe5d042998 obj=default.rgw.meta:users.uid:foo state=0x55eda604b0b0 s->prefetch_data=0 2023-10-05T19:42:15.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.128+0000 7fe18c4ebe40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2023-10-05T19:42:15.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.128+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda6233c70 con 0x55eda6172740 2023-10-05T19:42:15.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.128+0000 7fe0ba7fc700 20 reqs_thread_entry: start 2023-10-05T19:42:15.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.132+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(102 foo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv15 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7fe184014040 con 0x55eda6172740 2023-10-05T19:42:15.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.132+0000 7fe18c4ebe40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x16 2023-10-05T19:42:15.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.132+0000 7fe18c4ebe40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:42:15.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.132+0000 7fe18c4ebe40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:42:15.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.132+0000 7fe18c4ebe40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:42:15.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.132+0000 7fe18c4ebe40 20 Read xattr: user.rgw.idtag 2023-10-05T19:42:15.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.132+0000 7fe18c4ebe40 10 cache get: name=default.rgw.meta+users.uid+foo : type miss (requested=0x13, cached=0x16) 2023-10-05T19:42:15.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.132+0000 7fe18c4ebe40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.132+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda6236700 con 0x55eda6172740 2023-10-05T19:42:15.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.132+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(103 foo [call,call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv15 ondisk = 0) v8 ==== 273+0+480 (crc 0 0 0) 0x7fe184014040 con 0x55eda6172740 2023-10-05T19:42:15.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.132+0000 7fe18c4ebe40 20 rados_obj.operate() r=0 bl.length=357 2023-10-05T19:42:15.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.132+0000 7fe18c4ebe40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2023-10-05T19:42:15.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.132+0000 7fe18c4ebe40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:42:15.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.132+0000 7fe18c4ebe40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:42:15.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.132+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:104 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_trim in=55b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda6236700 con 0x7fe17c007030 2023-10-05T19:42:15.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.136+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(104 usage.10 [call] v30'35 uv35 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe184014040 con 0x7fe17c007030 2023-10-05T19:42:15.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.136+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:105 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_trim in=55b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda6236700 con 0x7fe17c007030 2023-10-05T19:42:15.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.140+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(105 usage.10 [call] v30'36 uv35 ondisk = -61 ((61) No data available)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe184014040 con 0x7fe17c007030 2023-10-05T19:42:15.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.140+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:106 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94479182398880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61f2060 con 0x55eda61a7b40 2023-10-05T19:42:15.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.144+0000 7fe18a020700 1 -- 172.21.15.12:0/976955935 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(106 notify.0 [watch unwatch cookie 94479182398880] v30'260 uv258 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe17c01c7f0 con 0x55eda61a7b40 2023-10-05T19:42:15.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.144+0000 7fe18c4ebe40 20 remove_watcher() i=0 2023-10-05T19:42:15.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.144+0000 7fe18c4ebe40 2 removed watcher, disabling cache 2023-10-05T19:42:15.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.144+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:107 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94479182411408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda6187580 con 0x55eda6172740 2023-10-05T19:42:15.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.144+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(107 notify.1 [watch unwatch cookie 94479182411408] v30'260 uv258 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe184014040 con 0x55eda6172740 2023-10-05T19:42:15.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.144+0000 7fe18c4ebe40 20 remove_watcher() i=1 2023-10-05T19:42:15.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.144+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:108 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94479182412512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda6187580 con 0x55eda61ac250 2023-10-05T19:42:15.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.148+0000 7fe18981f700 1 -- 172.21.15.12:0/976955935 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(108 notify.2 [watch unwatch cookie 94479182412512] v30'260 uv258 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe17402d040 con 0x55eda61ac250 2023-10-05T19:42:15.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.148+0000 7fe18c4ebe40 20 remove_watcher() i=2 2023-10-05T19:42:15.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.148+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:109 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94479182417584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda61b3b80 con 0x55eda61a7b40 2023-10-05T19:42:15.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.152+0000 7fe18a020700 1 -- 172.21.15.12:0/976955935 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(109 notify.3 [watch unwatch cookie 94479182417584] v30'260 uv258 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe17c01c7f0 con 0x55eda61a7b40 2023-10-05T19:42:15.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.152+0000 7fe18c4ebe40 20 remove_watcher() i=3 2023-10-05T19:42:15.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.152+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:110 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94479182419936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda6187580 con 0x55eda6172740 2023-10-05T19:42:15.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.156+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== osd_op_reply(110 notify.4 [watch unwatch cookie 94479182419936] v30'260 uv258 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe184014040 con 0x55eda6172740 2023-10-05T19:42:15.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.156+0000 7fe18c4ebe40 20 remove_watcher() i=4 2023-10-05T19:42:15.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.156+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:111 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94479182422512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda6187580 con 0x7fe17c007030 2023-10-05T19:42:15.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.156+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(111 notify.5 [watch unwatch cookie 94479182422512] v30'260 uv258 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe184014040 con 0x7fe17c007030 2023-10-05T19:42:15.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.156+0000 7fe18c4ebe40 20 remove_watcher() i=5 2023-10-05T19:42:15.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.156+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:112 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94479182425440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda6187580 con 0x7fe17c007030 2023-10-05T19:42:15.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.160+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(112 notify.6 [watch unwatch cookie 94479182425440] v30'260 uv258 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe184014040 con 0x7fe17c007030 2023-10-05T19:42:15.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.160+0000 7fe18c4ebe40 20 remove_watcher() i=6 2023-10-05T19:42:15.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.160+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:113 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94479182423616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda6187580 con 0x7fe17c007030 2023-10-05T19:42:15.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.164+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(113 notify.7 [watch unwatch cookie 94479182423616] v30'260 uv258 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe184014040 con 0x7fe17c007030 2023-10-05T19:42:15.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.164+0000 7fe18c4ebe40 20 remove_watcher() i=7 2023-10-05T19:42:15.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.168+0000 7fe15bfdf700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:42:15.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.168+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:114 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94479182429696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55eda6187580 con 0x55eda6172740 2023-10-05T19:42:15.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.168+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 <== osd.1 v2:172.21.15.12:6808/29933 39 ==== osd_op_reply(114 data_loggenerations_metadata [watch unwatch cookie 94479182429696] v30'250 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fe184014040 con 0x55eda6172740 2023-10-05T19:42:15.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.168+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fe17c007030 msgr2=0x7fe17c00c9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:15.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.168+0000 7fe18c4ebe40 1 --2- 172.21.15.12:0/976955935 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fe17c007030 0x7fe17c00c9c0 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:15.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.168+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55eda6172740 msgr2=0x55eda61a5650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:15.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.168+0000 7fe18c4ebe40 1 --2- 172.21.15.12:0/976955935 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55eda6172740 0x55eda61a5650 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:15.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.168+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55eda61a7b40 msgr2=0x55eda61a9f20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:15.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.168+0000 7fe18c4ebe40 1 --2- 172.21.15.12:0/976955935 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55eda61a7b40 0x55eda61a9f20 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:15.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.168+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55eda61ac250 msgr2=0x55eda61ae630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:15.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.168+0000 7fe18c4ebe40 1 --2- 172.21.15.12:0/976955935 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55eda61ac250 0x55eda61ae630 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:15.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.168+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe16c030c40 msgr2=0x7fe16c0330f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:15.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.168+0000 7fe18c4ebe40 1 --2- 172.21.15.12:0/976955935 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe16c030c40 0x7fe16c0330f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe1740044f0 tx=0x7fe174006690 comp rx=0 tx=0).stop 2023-10-05T19:42:15.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.168+0000 7fe18a821700 1 -- 172.21.15.12:0/976955935 reap_dead start 2023-10-05T19:42:15.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.168+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eda6171e30 msgr2=0x55eda6199b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:15.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.168+0000 7fe18c4ebe40 1 --2- 172.21.15.12:0/976955935 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eda6171e30 0x55eda6199b90 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fe17c01f040 tx=0x7fe17c01c000 comp rx=0 tx=0).stop 2023-10-05T19:42:15.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.172+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 shutdown_connections 2023-10-05T19:42:15.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.172+0000 7fe18c4ebe40 1 --2- 172.21.15.12:0/976955935 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eda6171e30 0x55eda6199b90 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:15.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.172+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 >> 172.21.15.12:0/976955935 conn(0x55eda616d850 msgr2=0x55eda618b320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:15.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.172+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 shutdown_connections 2023-10-05T19:42:15.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.172+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/976955935 wait complete. 2023-10-05T19:42:15.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.172+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/2775072013 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe0d0030bb0 msgr2=0x7fe0d0033060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:15.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.172+0000 7fe18c4ebe40 1 --2- 172.21.15.12:0/2775072013 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe0d0030bb0 0x7fe0d0033060 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x55eda61b73b0 tx=0x7fe1840335f0 comp rx=0 tx=0).stop 2023-10-05T19:42:15.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.172+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/2775072013 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eda61c57b0 msgr2=0x55eda61cb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:15.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.172+0000 7fe18c4ebe40 1 --2- 172.21.15.12:0/2775072013 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eda61c57b0 0x55eda61cb3c0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x55eda61af090 tx=0x7fe174044000 comp rx=0 tx=0).stop 2023-10-05T19:42:15.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.172+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/2775072013 shutdown_connections 2023-10-05T19:42:15.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.172+0000 7fe18c4ebe40 1 --2- 172.21.15.12:0/2775072013 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fe0d0030bb0 0x7fe0d0033060 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:15.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.172+0000 7fe18c4ebe40 1 --2- 172.21.15.12:0/2775072013 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eda61c57b0 0x55eda61cb3c0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:15.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.172+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/2775072013 >> 172.21.15.12:0/2775072013 conn(0x55eda61c0b70 msgr2=0x55eda61bec10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:15.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.172+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/2775072013 shutdown_connections 2023-10-05T19:42:15.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.172+0000 7fe18c4ebe40 1 -- 172.21.15.12:0/2775072013 wait complete. 2023-10-05T19:42:15.183 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo'] 2023-10-05T19:42:15.183 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo'] 2023-10-05T19:42:15.184 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo 2023-10-05T19:42:15.266 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:42:15.266 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:42:15.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.260+0000 7f6619d06e40 1 Processor -- start 2023-10-05T19:42:15.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.260+0000 7f6619d06e40 1 -- start start 2023-10-05T19:42:15.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.260+0000 7f6619d06e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55c4aeab8410 0x55c4aeab6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:15.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.260+0000 7f6619d06e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55c4aea04bd0 con 0x55c4aeab87e0 2023-10-05T19:42:15.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.260+0000 7f6619d06e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x55c4ae9c1ee0 con 0x55c4aeab8410 2023-10-05T19:42:15.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.264+0000 7f661783b700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55c4aeab8410 0x55c4aeab6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:15.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.264+0000 7f661703a700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x55c4aeab87e0 0x55c4aeab7500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:47336/0 (socket says 172.21.15.12:47336) 2023-10-05T19:42:15.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.264+0000 7f661703a700 1 -- 172.21.15.12:0/881805298 learned_addr learned my addr 172.21.15.12:0/881805298 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:42:15.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.264+0000 7f661783b700 1 -- 172.21.15.12:0/881805298 >> v1:172.21.15.8:6789/0 conn(0x55c4aeab87e0 legacy=0x55c4aeab7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:15.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.264+0000 7f6616839700 1 -- 172.21.15.12:0/881805298 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1422446396 0 0) 0x55c4aea04bd0 con 0x55c4aeab87e0 2023-10-05T19:42:15.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.264+0000 7f661783b700 1 -- 172.21.15.12:0/881805298 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4aeab7cd0 con 0x55c4aeab8410 2023-10-05T19:42:15.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.264+0000 7f661783b700 1 --2- 172.21.15.12:0/881805298 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4aeab8410 0x55c4aeab6fc0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f661000a5d0 tx=0x7f6610005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9d36d8e68316916 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:15.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.264+0000 7f6616839700 1 -- 172.21.15.12:0/881805298 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f66100086a0 con 0x55c4aeab8410 2023-10-05T19:42:15.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.264+0000 7f6616839700 1 -- 172.21.15.12:0/881805298 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6610008800 con 0x55c4aeab8410 2023-10-05T19:42:15.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.264+0000 7f6616839700 1 -- 172.21.15.12:0/881805298 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6610008ad0 con 0x55c4aeab8410 2023-10-05T19:42:15.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.264+0000 7f6619d06e40 1 -- 172.21.15.12:0/881805298 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4aeab8410 msgr2=0x55c4aeab6fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:15.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.264+0000 7f6619d06e40 1 --2- 172.21.15.12:0/881805298 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4aeab8410 0x55c4aeab6fc0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f661000a5d0 tx=0x7f6610005a50 comp rx=0 tx=0).stop 2023-10-05T19:42:15.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.264+0000 7f6619d06e40 1 -- 172.21.15.12:0/881805298 shutdown_connections 2023-10-05T19:42:15.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.264+0000 7f6619d06e40 1 --2- 172.21.15.12:0/881805298 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4aeab8410 0x55c4aeab6fc0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:15.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.264+0000 7f6619d06e40 1 -- 172.21.15.12:0/881805298 >> 172.21.15.12:0/881805298 conn(0x55c4aeab2720 msgr2=0x55c4aeab4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:15.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.264+0000 7f6619d06e40 1 -- 172.21.15.12:0/881805298 shutdown_connections 2023-10-05T19:42:15.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.264+0000 7f6619d06e40 1 -- 172.21.15.12:0/881805298 wait complete. 2023-10-05T19:42:15.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.264+0000 7f6619d06e40 1 Processor -- start 2023-10-05T19:42:15.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.264+0000 7f6619d06e40 1 -- start start 2023-10-05T19:42:15.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.264+0000 7f6619d06e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4aeab8410 0x55c4aead8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:15.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.264+0000 7f6619d06e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4ae985e30 con 0x55c4aeab8410 2023-10-05T19:42:15.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.268+0000 7f661783b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4aeab8410 0x55c4aead8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:15.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.268+0000 7f661783b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4aeab8410 0x55c4aead8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:54250/0 (socket says 172.21.15.12:54250) 2023-10-05T19:42:15.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.268+0000 7f661783b700 1 -- 172.21.15.12:0/1664182485 learned_addr learned my addr 172.21.15.12:0/1664182485 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:42:15.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.268+0000 7f661783b700 1 -- 172.21.15.12:0/1664182485 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6610005cf0 con 0x55c4aeab8410 2023-10-05T19:42:15.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.268+0000 7f661783b700 1 --2- 172.21.15.12:0/1664182485 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4aeab8410 0x55c4aead8fb0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f6610005a90 tx=0x7f6610015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70a944812efd8c18 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:15.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.268+0000 7f6615837700 1 -- 172.21.15.12:0/1664182485 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6610008970 con 0x55c4aeab8410 2023-10-05T19:42:15.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.268+0000 7f6615837700 1 -- 172.21.15.12:0/1664182485 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6610008af0 con 0x55c4aeab8410 2023-10-05T19:42:15.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.268+0000 7f6615837700 1 -- 172.21.15.12:0/1664182485 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6610008de0 con 0x55c4aeab8410 2023-10-05T19:42:15.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.268+0000 7f6619d06e40 1 -- 172.21.15.12:0/1664182485 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4aeab8410 msgr2=0x55c4aead8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:15.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.268+0000 7f6619d06e40 1 --2- 172.21.15.12:0/1664182485 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4aeab8410 0x55c4aead8fb0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f6610005a90 tx=0x7f6610015000 comp rx=0 tx=0).stop 2023-10-05T19:42:15.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.268+0000 7f6619d06e40 1 -- 172.21.15.12:0/1664182485 shutdown_connections 2023-10-05T19:42:15.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.268+0000 7f6619d06e40 1 --2- 172.21.15.12:0/1664182485 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4aeab8410 0x55c4aead8fb0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:15.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.268+0000 7f6619d06e40 1 -- 172.21.15.12:0/1664182485 >> 172.21.15.12:0/1664182485 conn(0x55c4aeab2720 msgr2=0x55c4aead4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:15.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.268+0000 7f6619d06e40 1 -- 172.21.15.12:0/1664182485 shutdown_connections 2023-10-05T19:42:15.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.268+0000 7f6619d06e40 1 -- 172.21.15.12:0/1664182485 wait complete. 2023-10-05T19:42:15.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.272+0000 7f6619d06e40 1 Processor -- start 2023-10-05T19:42:15.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.272+0000 7f6619d06e40 1 -- start start 2023-10-05T19:42:15.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.272+0000 7f6619d06e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4aeab8410 0x55c4aeadfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:15.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.272+0000 7f6619d06e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4ae958ed0 con 0x55c4aeab8410 2023-10-05T19:42:15.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.272+0000 7f661783b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4aeab8410 0x55c4aeadfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:15.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.272+0000 7f661783b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4aeab8410 0x55c4aeadfbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:54256/0 (socket says 172.21.15.12:54256) 2023-10-05T19:42:15.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.272+0000 7f661783b700 1 -- 172.21.15.12:0/507072695 learned_addr learned my addr 172.21.15.12:0/507072695 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:42:15.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.272+0000 7f661783b700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4aeab7cd0 con 0x55c4aeab8410 2023-10-05T19:42:15.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.272+0000 7f661783b700 1 --2- 172.21.15.12:0/507072695 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4aeab8410 0x55c4aeadfbc0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f661001e040 tx=0x7f6610003050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:15.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.272+0000 7f660ffff700 1 -- 172.21.15.12:0/507072695 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f661001c410 con 0x55c4aeab8410 2023-10-05T19:42:15.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.272+0000 7f660ffff700 1 -- 172.21.15.12:0/507072695 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f661001c570 con 0x55c4aeab8410 2023-10-05T19:42:15.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.272+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c4aeae0100 con 0x55c4aeab8410 2023-10-05T19:42:15.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.272+0000 7f660ffff700 1 -- 172.21.15.12:0/507072695 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f661001c840 con 0x55c4aeab8410 2023-10-05T19:42:15.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.272+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c4aeae04e0 con 0x55c4aeab8410 2023-10-05T19:42:15.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.276+0000 7f660ffff700 1 -- 172.21.15.12:0/507072695 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f6610019070 con 0x55c4aeab8410 2023-10-05T19:42:15.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.276+0000 7f660ffff700 1 --2- 172.21.15.12:0/507072695 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f65f8030c50 0x7f65f8033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:15.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.276+0000 7f660ffff700 1 -- 172.21.15.12:0/507072695 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7f66100316b0 con 0x55c4aeab8410 2023-10-05T19:42:15.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.276+0000 7f661703a700 1 --2- 172.21.15.12:0/507072695 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f65f8030c50 0x7f65f8033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:15.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.276+0000 7f6619d06e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.276+0000 7f6619d06e40 1 --2- 172.21.15.12:0/507072695 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55c4aeab87e0 0x55c4aeaeb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:15.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.276+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaebd10 con 0x55c4aeab87e0 2023-10-05T19:42:15.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.276+0000 7f661803c700 1 --2- 172.21.15.12:0/507072695 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55c4aeab87e0 0x55c4aeaeb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:15.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.276+0000 7f661703a700 1 --2- 172.21.15.12:0/507072695 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f65f8030c50 0x7f65f8033100 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f66000011e0 tx=0x7f66000013a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:15.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.276+0000 7f661803c700 1 --2- 172.21.15.12:0/507072695 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55c4aeab87e0 0x55c4aeaeb5e0 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:15.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.276+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f660800a5a0 con 0x55c4aeab87e0 2023-10-05T19:42:15.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.276+0000 7f6619d06e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:15.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.276+0000 7f6619d06e40 20 realm 2023-10-05T19:42:15.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.276+0000 7f6619d06e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.276+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeae8c10 con 0x55c4aeab87e0 2023-10-05T19:42:15.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.280+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f660800a5a0 con 0x55c4aeab87e0 2023-10-05T19:42:15.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.280+0000 7f6619d06e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:15.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.280+0000 7f6619d06e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:42:15.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.280+0000 7f6619d06e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.280+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaec840 con 0x55c4aeab87e0 2023-10-05T19:42:15.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.280+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f660800a5a0 con 0x55c4aeab87e0 2023-10-05T19:42:15.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.280+0000 7f6619d06e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:15.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.280+0000 7f6619d06e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.280+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaec420 con 0x55c4aeab87e0 2023-10-05T19:42:15.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.280+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f660800a5a0 con 0x55c4aeab87e0 2023-10-05T19:42:15.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.280+0000 7f6619d06e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:42:15.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.280+0000 7f6619d06e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.280+0000 7f6619d06e40 1 --2- 172.21.15.12:0/507072695 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55c4aeaed7d0 0x55c4aeaefbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:15.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.280+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf0320 con 0x55c4aeaed7d0 2023-10-05T19:42:15.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.280+0000 7f661783b700 1 --2- 172.21.15.12:0/507072695 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55c4aeaed7d0 0x55c4aeaefbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:15.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.280+0000 7f661783b700 1 --2- 172.21.15.12:0/507072695 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55c4aeaed7d0 0x55c4aeaefbb0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:15.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.280+0000 7f661783b700 1 -- 172.21.15.12:0/507072695 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f6610036070 con 0x55c4aeaed7d0 2023-10-05T19:42:15.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.280+0000 7f6619d06e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:42:15.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.280+0000 7f6619d06e40 20 searching for the correct realm 2023-10-05T19:42:15.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.280+0000 7f6619d06e40 1 --2- 172.21.15.12:0/507072695 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55c4aeaf2150 0x55c4aeaf4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:15.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.280+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf4a70 con 0x55c4aeaf2150 2023-10-05T19:42:15.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.280+0000 7f661703a700 1 --2- 172.21.15.12:0/507072695 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55c4aeaf2150 0x55c4aeaf4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:15.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.280+0000 7f661703a700 1 --2- 172.21.15.12:0/507072695 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55c4aeaf2150 0x55c4aeaf4530 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:15.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.284+0000 7f661703a700 1 -- 172.21.15.12:0/507072695 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f6600006070 con 0x55c4aeaf2150 2023-10-05T19:42:15.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.284+0000 7f661703a700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf4a70 con 0x55c4aeaed7d0 2023-10-05T19:42:15.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.284+0000 7f661783b700 1 -- 172.21.15.12:0/507072695 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6610036070 con 0x55c4aeaed7d0 2023-10-05T19:42:15.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.284+0000 7f661783b700 1 --2- 172.21.15.12:0/507072695 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f6610039030 0x7f6610011a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:15.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.284+0000 7f661783b700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf4a70 con 0x7f6610039030 2023-10-05T19:42:15.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.284+0000 7f661803c700 1 --2- 172.21.15.12:0/507072695 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f6610039030 0x7f6610011a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:15.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.288+0000 7f661803c700 1 --2- 172.21.15.12:0/507072695 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f6610039030 0x7f6610011a30 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:15.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.288+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f660800a5a0 con 0x7f6610039030 2023-10-05T19:42:15.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.288+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf4a70 con 0x55c4aeab87e0 2023-10-05T19:42:15.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.288+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f660800a5a0 con 0x55c4aeab87e0 2023-10-05T19:42:15.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.288+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf4a70 con 0x55c4aeab87e0 2023-10-05T19:42:15.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.288+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f660800a5a0 con 0x55c4aeab87e0 2023-10-05T19:42:15.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.288+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf4a70 con 0x55c4aeab87e0 2023-10-05T19:42:15.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.288+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(11 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f660800a5a0 con 0x55c4aeab87e0 2023-10-05T19:42:15.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.288+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf4a70 con 0x7f6610039030 2023-10-05T19:42:15.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.288+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f660800a5a0 con 0x7f6610039030 2023-10-05T19:42:15.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.288+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf4a70 con 0x55c4aeaed7d0 2023-10-05T19:42:15.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.292+0000 7f661783b700 1 -- 172.21.15.12:0/507072695 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6610036070 con 0x55c4aeaed7d0 2023-10-05T19:42:15.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.292+0000 7f661783b700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf4a70 con 0x55c4aeaed7d0 2023-10-05T19:42:15.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.292+0000 7f661783b700 1 -- 172.21.15.12:0/507072695 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6610036070 con 0x55c4aeaed7d0 2023-10-05T19:42:15.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.292+0000 7f661783b700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf4a70 con 0x55c4aeab87e0 2023-10-05T19:42:15.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.292+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f660800a5a0 con 0x55c4aeab87e0 2023-10-05T19:42:15.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.292+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf4a70 con 0x7f6610039030 2023-10-05T19:42:15.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.292+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f660800a5a0 con 0x7f6610039030 2023-10-05T19:42:15.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.292+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf4a70 con 0x55c4aeab87e0 2023-10-05T19:42:15.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.292+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f660800a5a0 con 0x55c4aeab87e0 2023-10-05T19:42:15.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.292+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf4a70 con 0x55c4aeaed7d0 2023-10-05T19:42:15.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.296+0000 7f661783b700 1 -- 172.21.15.12:0/507072695 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(18 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6610036070 con 0x55c4aeaed7d0 2023-10-05T19:42:15.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.296+0000 7f661783b700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf4a70 con 0x55c4aeaf2150 2023-10-05T19:42:15.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.296+0000 7f661703a700 1 -- 172.21.15.12:0/507072695 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6600006070 con 0x55c4aeaf2150 2023-10-05T19:42:15.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.296+0000 7f661703a700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf4a70 con 0x55c4aeaf2150 2023-10-05T19:42:15.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.296+0000 7f661703a700 1 -- 172.21.15.12:0/507072695 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6600006070 con 0x55c4aeaf2150 2023-10-05T19:42:15.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.296+0000 7f661703a700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf4a70 con 0x7f6610039030 2023-10-05T19:42:15.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.296+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f660800a5a0 con 0x7f6610039030 2023-10-05T19:42:15.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.296+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf4a70 con 0x55c4aeaf2150 2023-10-05T19:42:15.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.296+0000 7f661703a700 1 -- 172.21.15.12:0/507072695 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6600006070 con 0x55c4aeaf2150 2023-10-05T19:42:15.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.296+0000 7f661703a700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf4a70 con 0x55c4aeaf2150 2023-10-05T19:42:15.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.300+0000 7f661703a700 1 -- 172.21.15.12:0/507072695 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6600006070 con 0x55c4aeaf2150 2023-10-05T19:42:15.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.300+0000 7f661703a700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf4a70 con 0x55c4aeab87e0 2023-10-05T19:42:15.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.300+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f660800a5a0 con 0x55c4aeab87e0 2023-10-05T19:42:15.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.300+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf4a70 con 0x55c4aeab87e0 2023-10-05T19:42:15.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.300+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f660800a5a0 con 0x55c4aeab87e0 2023-10-05T19:42:15.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.300+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf4a70 con 0x7f6610039030 2023-10-05T19:42:15.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.300+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f660800a5a0 con 0x7f6610039030 2023-10-05T19:42:15.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.300+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf4a70 con 0x55c4aeaf2150 2023-10-05T19:42:15.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.300+0000 7f661703a700 1 -- 172.21.15.12:0/507072695 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6600006070 con 0x55c4aeaf2150 2023-10-05T19:42:15.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.300+0000 7f661703a700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf4a70 con 0x55c4aeaed7d0 2023-10-05T19:42:15.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.304+0000 7f661783b700 1 -- 172.21.15.12:0/507072695 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6610036070 con 0x55c4aeaed7d0 2023-10-05T19:42:15.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.304+0000 7f661783b700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf4a70 con 0x55c4aeab87e0 2023-10-05T19:42:15.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.304+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f660800a5a0 con 0x55c4aeab87e0 2023-10-05T19:42:15.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.304+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf4a70 con 0x7f6610039030 2023-10-05T19:42:15.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.304+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f660800a5a0 con 0x7f6610039030 2023-10-05T19:42:15.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.304+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf4a70 con 0x55c4aeaf2150 2023-10-05T19:42:15.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.304+0000 7f661703a700 1 -- 172.21.15.12:0/507072695 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6600006070 con 0x55c4aeaf2150 2023-10-05T19:42:15.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.304+0000 7f661703a700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf4a70 con 0x7f6610039030 2023-10-05T19:42:15.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.308+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f660800a5a0 con 0x7f6610039030 2023-10-05T19:42:15.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.308+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf4a70 con 0x55c4aeaed7d0 2023-10-05T19:42:15.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.308+0000 7f661783b700 1 -- 172.21.15.12:0/507072695 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6610036070 con 0x55c4aeaed7d0 2023-10-05T19:42:15.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.308+0000 7f661783b700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf4a70 con 0x55c4aeaf2150 2023-10-05T19:42:15.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.308+0000 7f661703a700 1 -- 172.21.15.12:0/507072695 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6600006070 con 0x55c4aeaf2150 2023-10-05T19:42:15.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.308+0000 7f661703a700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf4a70 con 0x7f6610039030 2023-10-05T19:42:15.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.308+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f660800a5a0 con 0x7f6610039030 2023-10-05T19:42:15.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.308+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf4a70 con 0x55c4aeab87e0 2023-10-05T19:42:15.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.308+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f660800a5a0 con 0x55c4aeab87e0 2023-10-05T19:42:15.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.308+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf4a70 con 0x55c4aeaed7d0 2023-10-05T19:42:15.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.312+0000 7f661783b700 1 -- 172.21.15.12:0/507072695 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(37 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f6610036070 con 0x55c4aeaed7d0 2023-10-05T19:42:15.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.312+0000 7f6619d06e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:42:15.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.312+0000 7f6619d06e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:42:15.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.312+0000 7f6619d06e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:42:15.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.312+0000 7f6619d06e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:42:15.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.312+0000 7f6619d06e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.312+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf0f00 con 0x55c4aeab87e0 2023-10-05T19:42:15.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.312+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f660800a5a0 con 0x55c4aeab87e0 2023-10-05T19:42:15.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.312+0000 7f6619d06e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:15.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.312+0000 7f6619d06e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.312+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf1850 con 0x55c4aeab87e0 2023-10-05T19:42:15.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.312+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f660800a5a0 con 0x55c4aeab87e0 2023-10-05T19:42:15.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.312+0000 7f6619d06e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:42:15.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.312+0000 7f6619d06e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.312+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf0ac0 con 0x55c4aeaf2150 2023-10-05T19:42:15.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.312+0000 7f661703a700 1 -- 172.21.15.12:0/507072695 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f6600006070 con 0x55c4aeaf2150 2023-10-05T19:42:15.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.312+0000 7f6619d06e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:42:15.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.312+0000 7f6619d06e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.312+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf53a0 con 0x55c4aeaf2150 2023-10-05T19:42:15.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.312+0000 7f661703a700 1 -- 172.21.15.12:0/507072695 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f6600006070 con 0x55c4aeaf2150 2023-10-05T19:42:15.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.312+0000 7f6619d06e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:15.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.312+0000 7f6619d06e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.312+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf5750 con 0x55c4aeab87e0 2023-10-05T19:42:15.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.316+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f660800a5a0 con 0x55c4aeab87e0 2023-10-05T19:42:15.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.316+0000 7f6619d06e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:15.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.316+0000 7f6619d06e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.316+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf5b00 con 0x55c4aeab87e0 2023-10-05T19:42:15.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.316+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f660800a5a0 con 0x55c4aeab87e0 2023-10-05T19:42:15.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.316+0000 7f6619d06e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:42:15.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.316+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf6b10 con 0x55c4aeaf2150 2023-10-05T19:42:15.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.320+0000 7f661703a700 1 -- 172.21.15.12:0/507072695 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(44 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f6600006070 con 0x55c4aeaf2150 2023-10-05T19:42:15.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.320+0000 7f661703a700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf6b10 con 0x55c4aeaed7d0 2023-10-05T19:42:15.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.320+0000 7f661783b700 1 -- 172.21.15.12:0/507072695 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(45 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6610036070 con 0x55c4aeaed7d0 2023-10-05T19:42:15.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.320+0000 7f661783b700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf6b10 con 0x7f6610039030 2023-10-05T19:42:15.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.320+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(46 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f660800a5a0 con 0x7f6610039030 2023-10-05T19:42:15.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.320+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf5750 con 0x55c4aeab87e0 2023-10-05T19:42:15.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.320+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(47 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f660800a5a0 con 0x55c4aeab87e0 2023-10-05T19:42:15.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.320+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf5750 con 0x55c4aeab87e0 2023-10-05T19:42:15.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.320+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(48 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f660800a5a0 con 0x55c4aeab87e0 2023-10-05T19:42:15.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.320+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf5750 con 0x55c4aeab87e0 2023-10-05T19:42:15.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.324+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(49 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f660800a5a0 con 0x55c4aeab87e0 2023-10-05T19:42:15.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.324+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf5750 con 0x7f6610039030 2023-10-05T19:42:15.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.324+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(50 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f660800a5a0 con 0x7f6610039030 2023-10-05T19:42:15.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.324+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf5750 con 0x55c4aeaed7d0 2023-10-05T19:42:15.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.324+0000 7f661783b700 1 -- 172.21.15.12:0/507072695 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(51 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6610036070 con 0x55c4aeaed7d0 2023-10-05T19:42:15.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.324+0000 7f661783b700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf5750 con 0x55c4aeaed7d0 2023-10-05T19:42:15.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.324+0000 7f661783b700 1 -- 172.21.15.12:0/507072695 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(52 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6610036070 con 0x55c4aeaed7d0 2023-10-05T19:42:15.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.324+0000 7f661783b700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf5750 con 0x55c4aeab87e0 2023-10-05T19:42:15.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.324+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(53 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f660800a5a0 con 0x55c4aeab87e0 2023-10-05T19:42:15.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.324+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf5750 con 0x7f6610039030 2023-10-05T19:42:15.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.328+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(54 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f660800a5a0 con 0x7f6610039030 2023-10-05T19:42:15.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.328+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf5750 con 0x55c4aeab87e0 2023-10-05T19:42:15.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.328+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(55 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f660800a5a0 con 0x55c4aeab87e0 2023-10-05T19:42:15.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.328+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf5750 con 0x55c4aeaed7d0 2023-10-05T19:42:15.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.328+0000 7f661783b700 1 -- 172.21.15.12:0/507072695 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(56 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6610036070 con 0x55c4aeaed7d0 2023-10-05T19:42:15.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.328+0000 7f661783b700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf5750 con 0x55c4aeaf2150 2023-10-05T19:42:15.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.328+0000 7f661703a700 1 -- 172.21.15.12:0/507072695 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(57 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6600006070 con 0x55c4aeaf2150 2023-10-05T19:42:15.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.328+0000 7f661703a700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf5750 con 0x55c4aeaf2150 2023-10-05T19:42:15.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.328+0000 7f661703a700 1 -- 172.21.15.12:0/507072695 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(58 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6600006070 con 0x55c4aeaf2150 2023-10-05T19:42:15.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.328+0000 7f661703a700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf5750 con 0x7f6610039030 2023-10-05T19:42:15.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.332+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(59 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f660800a5a0 con 0x7f6610039030 2023-10-05T19:42:15.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.332+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf5750 con 0x55c4aeaf2150 2023-10-05T19:42:15.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.332+0000 7f661703a700 1 -- 172.21.15.12:0/507072695 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(60 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6600006070 con 0x55c4aeaf2150 2023-10-05T19:42:15.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.332+0000 7f661703a700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf5750 con 0x55c4aeaf2150 2023-10-05T19:42:15.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.332+0000 7f661703a700 1 -- 172.21.15.12:0/507072695 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(61 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6600006070 con 0x55c4aeaf2150 2023-10-05T19:42:15.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.332+0000 7f661703a700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf5750 con 0x55c4aeab87e0 2023-10-05T19:42:15.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.332+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(62 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f660800a5a0 con 0x55c4aeab87e0 2023-10-05T19:42:15.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.332+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf5750 con 0x55c4aeab87e0 2023-10-05T19:42:15.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.336+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(63 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f660800a5a0 con 0x55c4aeab87e0 2023-10-05T19:42:15.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.336+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf5750 con 0x7f6610039030 2023-10-05T19:42:15.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.336+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(64 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f660800a5a0 con 0x7f6610039030 2023-10-05T19:42:15.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.336+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf5750 con 0x55c4aeaf2150 2023-10-05T19:42:15.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.336+0000 7f661703a700 1 -- 172.21.15.12:0/507072695 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(65 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6600006070 con 0x55c4aeaf2150 2023-10-05T19:42:15.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.336+0000 7f661703a700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf5750 con 0x55c4aeaed7d0 2023-10-05T19:42:15.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.336+0000 7f661783b700 1 -- 172.21.15.12:0/507072695 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(66 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6610036070 con 0x55c4aeaed7d0 2023-10-05T19:42:15.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.336+0000 7f661783b700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf5750 con 0x55c4aeab87e0 2023-10-05T19:42:15.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.336+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(67 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f660800a5a0 con 0x55c4aeab87e0 2023-10-05T19:42:15.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.336+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf5750 con 0x7f6610039030 2023-10-05T19:42:15.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.340+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(68 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f660800a5a0 con 0x7f6610039030 2023-10-05T19:42:15.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.340+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf5750 con 0x55c4aeaf2150 2023-10-05T19:42:15.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.340+0000 7f661703a700 1 -- 172.21.15.12:0/507072695 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(69 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6600006070 con 0x55c4aeaf2150 2023-10-05T19:42:15.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.340+0000 7f661703a700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf5750 con 0x7f6610039030 2023-10-05T19:42:15.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.340+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(70 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f660800a5a0 con 0x7f6610039030 2023-10-05T19:42:15.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.340+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf5750 con 0x55c4aeaed7d0 2023-10-05T19:42:15.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.340+0000 7f661783b700 1 -- 172.21.15.12:0/507072695 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(71 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6610036070 con 0x55c4aeaed7d0 2023-10-05T19:42:15.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.340+0000 7f661783b700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf5750 con 0x55c4aeaf2150 2023-10-05T19:42:15.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.344+0000 7f661703a700 1 -- 172.21.15.12:0/507072695 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(72 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6600006070 con 0x55c4aeaf2150 2023-10-05T19:42:15.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.344+0000 7f661703a700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf5750 con 0x7f6610039030 2023-10-05T19:42:15.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.344+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(73 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f660800a5a0 con 0x7f6610039030 2023-10-05T19:42:15.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.344+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf5750 con 0x55c4aeab87e0 2023-10-05T19:42:15.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.344+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(74 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f660800a5a0 con 0x55c4aeab87e0 2023-10-05T19:42:15.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.344+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf5750 con 0x55c4aeaed7d0 2023-10-05T19:42:15.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.344+0000 7f661783b700 1 -- 172.21.15.12:0/507072695 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(75 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f6610036070 con 0x55c4aeaed7d0 2023-10-05T19:42:15.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.344+0000 7f6619d06e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:42:15.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.344+0000 7f6619d06e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:42:15.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.344+0000 7f6619d06e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:42:15.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.344+0000 7f6619d06e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:42:15.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.344+0000 7f6619d06e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.344+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf5b00 con 0x55c4aeab87e0 2023-10-05T19:42:15.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.348+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f660800a5a0 con 0x55c4aeab87e0 2023-10-05T19:42:15.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.348+0000 7f6619d06e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:42:15.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.348+0000 7f6619d06e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.348+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf5eb0 con 0x55c4aeaed7d0 2023-10-05T19:42:15.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.348+0000 7f661783b700 1 -- 172.21.15.12:0/507072695 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f6610036070 con 0x55c4aeaed7d0 2023-10-05T19:42:15.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.348+0000 7f6619d06e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:42:15.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.348+0000 7f6619d06e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.348+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf6750 con 0x55c4aeab87e0 2023-10-05T19:42:15.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.348+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f660800a5a0 con 0x55c4aeab87e0 2023-10-05T19:42:15.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.348+0000 7f6619d06e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:42:15.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.348+0000 7f6619d06e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.348+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf6750 con 0x55c4aeaf2150 2023-10-05T19:42:15.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.348+0000 7f661703a700 1 -- 172.21.15.12:0/507072695 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f6600006070 con 0x55c4aeaf2150 2023-10-05T19:42:15.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.348+0000 7f6619d06e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:42:15.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.348+0000 7f6619d06e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.348+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf6ff0 con 0x7f6610039030 2023-10-05T19:42:15.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.348+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f660800a5a0 con 0x7f6610039030 2023-10-05T19:42:15.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.348+0000 7f6619d06e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:15.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.348+0000 7f6619d06e40 20 zone default found 2023-10-05T19:42:15.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.348+0000 7f6619d06e40 4 Realm: () 2023-10-05T19:42:15.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.348+0000 7f6619d06e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:42:15.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.348+0000 7f6619d06e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:42:15.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.348+0000 7f6619d06e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:42:15.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.348+0000 7f6619d06e40 20 zonegroup default 2023-10-05T19:42:15.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.348+0000 7f6619d06e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.352+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf6ff0 con 0x7f6610039030 2023-10-05T19:42:15.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.352+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f660800a5a0 con 0x7f6610039030 2023-10-05T19:42:15.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.352+0000 7f6619d06e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:15.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.352+0000 7f6619d06e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.352+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf8970 con 0x55c4aeaf2150 2023-10-05T19:42:15.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.352+0000 7f661703a700 1 -- 172.21.15.12:0/507072695 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f6600006070 con 0x55c4aeaf2150 2023-10-05T19:42:15.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.352+0000 7f6619d06e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:15.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.352+0000 7f6619d06e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.352+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf8d20 con 0x55c4aeaf2150 2023-10-05T19:42:15.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.352+0000 7f661703a700 1 -- 172.21.15.12:0/507072695 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f6600006070 con 0x55c4aeaf2150 2023-10-05T19:42:15.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.352+0000 7f6619d06e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:15.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.352+0000 7f6619d06e40 20 started sync module instance, tier type = 2023-10-05T19:42:15.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.352+0000 7f6619d06e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:42:15.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.352+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf8030 con 0x55c4aeaed7d0 2023-10-05T19:42:15.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.356+0000 7f661783b700 1 -- 172.21.15.12:0/507072695 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(84 notify.0 [create] v30'261 uv261 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6610036070 con 0x55c4aeaed7d0 2023-10-05T19:42:15.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.356+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94303232686912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf90d0 con 0x55c4aeaed7d0 2023-10-05T19:42:15.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.356+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf9970 con 0x55c4aeab87e0 2023-10-05T19:42:15.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.360+0000 7f661783b700 1 -- 172.21.15.12:0/507072695 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(85 notify.0 [watch watch cookie 94303232686912] v30'262 uv261 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6610036070 con 0x55c4aeaed7d0 2023-10-05T19:42:15.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.360+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(86 notify.1 [create] v30'261 uv261 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f660800a5a0 con 0x55c4aeab87e0 2023-10-05T19:42:15.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.360+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94303232689200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf9970 con 0x55c4aeab87e0 2023-10-05T19:42:15.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.360+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeafa3e0 con 0x55c4aeaf2150 2023-10-05T19:42:15.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.360+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(87 notify.1 [watch watch cookie 94303232689200] v30'262 uv261 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f660800a5a0 con 0x55c4aeab87e0 2023-10-05T19:42:15.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.364+0000 7f661703a700 1 -- 172.21.15.12:0/507072695 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(88 notify.2 [create] v30'261 uv261 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6600006070 con 0x55c4aeaf2150 2023-10-05T19:42:15.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.364+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94303232693376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeaf98d0 con 0x55c4aeaf2150 2023-10-05T19:42:15.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.364+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeafb1c0 con 0x55c4aeaed7d0 2023-10-05T19:42:15.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.368+0000 7f661703a700 1 -- 172.21.15.12:0/507072695 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(89 notify.2 [watch watch cookie 94303232693376] v30'262 uv261 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6600006070 con 0x55c4aeaf2150 2023-10-05T19:42:15.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.368+0000 7f661783b700 1 -- 172.21.15.12:0/507072695 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(90 notify.3 [create] v30'261 uv261 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6610036070 con 0x55c4aeaed7d0 2023-10-05T19:42:15.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.368+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94303232698624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeafb240 con 0x55c4aeaed7d0 2023-10-05T19:42:15.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.368+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeafbd10 con 0x55c4aeab87e0 2023-10-05T19:42:15.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.368+0000 7f661783b700 1 -- 172.21.15.12:0/507072695 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(91 notify.3 [watch watch cookie 94303232698624] v30'262 uv261 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6610036070 con 0x55c4aeaed7d0 2023-10-05T19:42:15.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.368+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(92 notify.4 [create] v30'261 uv261 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f660800a5a0 con 0x55c4aeab87e0 2023-10-05T19:42:15.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.368+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94303232700288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeafbd10 con 0x55c4aeab87e0 2023-10-05T19:42:15.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.368+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeafc720 con 0x7f6610039030 2023-10-05T19:42:15.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.372+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(94 notify.5 [create] v30'261 uv261 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f660800a5a0 con 0x7f6610039030 2023-10-05T19:42:15.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.372+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94303232705072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeafcb70 con 0x7f6610039030 2023-10-05T19:42:15.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.372+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeafd650 con 0x7f6610039030 2023-10-05T19:42:15.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.372+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(93 notify.4 [watch watch cookie 94303232700288] v30'262 uv261 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f660800a5a0 con 0x55c4aeab87e0 2023-10-05T19:42:15.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.376+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(95 notify.5 [watch watch cookie 94303232705072] v30'262 uv261 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f660800a5a0 con 0x7f6610039030 2023-10-05T19:42:15.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.376+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(96 notify.6 [create] v30'261 uv261 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f660800a5a0 con 0x7f6610039030 2023-10-05T19:42:15.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.376+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94303232702496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeafc680 con 0x7f6610039030 2023-10-05T19:42:15.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.376+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeafdd70 con 0x7f6610039030 2023-10-05T19:42:15.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.376+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(98 notify.7 [create] v30'261 uv261 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f660800a5a0 con 0x7f6610039030 2023-10-05T19:42:15.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.376+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94303232710784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeafc680 con 0x7f6610039030 2023-10-05T19:42:15.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.376+0000 7f6619d06e40 20 add_watcher() i=0 2023-10-05T19:42:15.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.376+0000 7f6619d06e40 20 add_watcher() i=1 2023-10-05T19:42:15.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.376+0000 7f6619d06e40 20 add_watcher() i=2 2023-10-05T19:42:15.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.376+0000 7f6619d06e40 20 add_watcher() i=3 2023-10-05T19:42:15.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.376+0000 7f6619d06e40 20 add_watcher() i=4 2023-10-05T19:42:15.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.376+0000 7f6619d06e40 20 add_watcher() i=5 2023-10-05T19:42:15.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.376+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(97 notify.6 [watch watch cookie 94303232702496] v30'262 uv261 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f660800a5a0 con 0x7f6610039030 2023-10-05T19:42:15.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.376+0000 7f6619d06e40 20 add_watcher() i=6 2023-10-05T19:42:15.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.380+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(99 notify.7 [watch watch cookie 94303232710784] v30'262 uv261 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f660800a5a0 con 0x7f6610039030 2023-10-05T19:42:15.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.380+0000 7f6619d06e40 20 add_watcher() i=7 2023-10-05T19:42:15.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.380+0000 7f6619d06e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:42:15.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.380+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeafd120 con 0x55c4aeab87e0 2023-10-05T19:42:15.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.380+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f660800a5a0 con 0x55c4aeab87e0 2023-10-05T19:42:15.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.380+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94303232711888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeafe360 con 0x55c4aeab87e0 2023-10-05T19:42:15.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.384+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94303232711888] v30'251 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f660800a5a0 con 0x55c4aeab87e0 2023-10-05T19:42:15.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.384+0000 7f65e97e2700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:42:15.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.384+0000 7f6619d06e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:42:15.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.384+0000 7f6619d06e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:42:15.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.384+0000 7f6619d06e40 1 Processor -- start 2023-10-05T19:42:15.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.384+0000 7f6619d06e40 1 -- start start 2023-10-05T19:42:15.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.384+0000 7f6619d06e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4aeb0b1c0 0x55c4aeb0b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:15.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.384+0000 7f6619d06e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4aeb0bad0 con 0x55c4aeb0b1c0 2023-10-05T19:42:15.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.384+0000 7f661703a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4aeb0b1c0 0x55c4aeb0b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:15.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.384+0000 7f661703a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4aeb0b1c0 0x55c4aeb0b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:54258/0 (socket says 172.21.15.12:54258) 2023-10-05T19:42:15.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.384+0000 7f661703a700 1 -- 172.21.15.12:0/1080837767 learned_addr learned my addr 172.21.15.12:0/1080837767 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:42:15.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.384+0000 7f661703a700 1 -- 172.21.15.12:0/1080837767 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4aeb0bc10 con 0x55c4aeb0b1c0 2023-10-05T19:42:15.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.388+0000 7f661703a700 1 --2- 172.21.15.12:0/1080837767 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4aeb0b1c0 0x55c4aeb0b590 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x55c4aeaf09e0 tx=0x7f660001e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c3528f2fb52dfe3 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:15.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.388+0000 7f65e8fe1700 1 -- 172.21.15.12:0/1080837767 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6600030040 con 0x55c4aeb0b1c0 2023-10-05T19:42:15.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.388+0000 7f65e8fe1700 1 -- 172.21.15.12:0/1080837767 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f660000edd0 con 0x55c4aeb0b1c0 2023-10-05T19:42:15.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.388+0000 7f65e8fe1700 1 -- 172.21.15.12:0/1080837767 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6600021040 con 0x55c4aeb0b1c0 2023-10-05T19:42:15.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.388+0000 7f6619d06e40 1 -- 172.21.15.12:0/1080837767 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4aeb0b1c0 msgr2=0x55c4aeb0b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:15.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.388+0000 7f6619d06e40 1 --2- 172.21.15.12:0/1080837767 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4aeb0b1c0 0x55c4aeb0b590 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x55c4aeaf09e0 tx=0x7f660001e040 comp rx=0 tx=0).stop 2023-10-05T19:42:15.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.388+0000 7f6619d06e40 1 -- 172.21.15.12:0/1080837767 shutdown_connections 2023-10-05T19:42:15.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.388+0000 7f6619d06e40 1 --2- 172.21.15.12:0/1080837767 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4aeb0b1c0 0x55c4aeb0b590 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:15.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.388+0000 7f6619d06e40 1 -- 172.21.15.12:0/1080837767 >> 172.21.15.12:0/1080837767 conn(0x55c4aeb06580 msgr2=0x55c4aeb04730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:15.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.388+0000 7f6619d06e40 1 -- 172.21.15.12:0/1080837767 shutdown_connections 2023-10-05T19:42:15.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.388+0000 7f6619d06e40 1 -- 172.21.15.12:0/1080837767 wait complete. 2023-10-05T19:42:15.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.388+0000 7f6619d06e40 1 Processor -- start 2023-10-05T19:42:15.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.388+0000 7f6619d06e40 1 -- start start 2023-10-05T19:42:15.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.388+0000 7f6619d06e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4aeb0b1c0 0x55c4aeb10d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:15.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.388+0000 7f6619d06e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4aeb11d20 con 0x55c4aeb0b1c0 2023-10-05T19:42:15.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.388+0000 7f661703a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4aeb0b1c0 0x55c4aeb10d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:15.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.388+0000 7f661703a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4aeb0b1c0 0x55c4aeb10d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:54262/0 (socket says 172.21.15.12:54262) 2023-10-05T19:42:15.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.388+0000 7f661703a700 1 -- 172.21.15.12:0/2187694027 learned_addr learned my addr 172.21.15.12:0/2187694027 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:42:15.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.392+0000 7f661703a700 1 -- 172.21.15.12:0/2187694027 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6600033040 con 0x55c4aeb0b1c0 2023-10-05T19:42:15.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.392+0000 7f661703a700 1 --2- 172.21.15.12:0/2187694027 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4aeb0b1c0 0x55c4aeb10d20 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x55c4aeb052a0 tx=0x7f660003c030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:15.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.392+0000 7f6562ffd700 1 -- 172.21.15.12:0/2187694027 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6600030070 con 0x55c4aeb0b1c0 2023-10-05T19:42:15.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.392+0000 7f6562ffd700 1 -- 172.21.15.12:0/2187694027 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f660002b730 con 0x55c4aeb0b1c0 2023-10-05T19:42:15.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.392+0000 7f6619d06e40 1 -- 172.21.15.12:0/2187694027 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c4aeb11320 con 0x55c4aeb0b1c0 2023-10-05T19:42:15.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.392+0000 7f6562ffd700 1 -- 172.21.15.12:0/2187694027 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6600021040 con 0x55c4aeb0b1c0 2023-10-05T19:42:15.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.392+0000 7f6619d06e40 1 -- 172.21.15.12:0/2187694027 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c4aeb11700 con 0x55c4aeb0b1c0 2023-10-05T19:42:15.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.392+0000 7f6562ffd700 1 -- 172.21.15.12:0/2187694027 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f660002b8e0 con 0x55c4aeb0b1c0 2023-10-05T19:42:15.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.392+0000 7f6562ffd700 1 --2- 172.21.15.12:0/2187694027 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f6558030c50 0x7f6558033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:15.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.392+0000 7f6562ffd700 1 -- 172.21.15.12:0/2187694027 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7f6600058bd0 con 0x55c4aeb0b1c0 2023-10-05T19:42:15.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.392+0000 7f661803c700 1 --2- 172.21.15.12:0/2187694027 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f6558030c50 0x7f6558033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:15.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.392+0000 7f6619d06e40 5 note: GC not initialized 2023-10-05T19:42:15.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.396+0000 7f661803c700 1 --2- 172.21.15.12:0/2187694027 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f6558030c50 0x7f6558033100 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x55c4aeafc9c0 tx=0x7f660800aee0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:15.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.396+0000 7f6557fff700 20 reqs_thread_entry: start 2023-10-05T19:42:15.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.396+0000 7f6555ffb700 20 reqs_thread_entry: start 2023-10-05T19:42:15.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.396+0000 7f6619d06e40 20 init_complete bucket index max shards: 11 2023-10-05T19:42:15.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.396+0000 7f6619d06e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:42:15.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.396+0000 7f6619d06e40 20 get_system_obj_state: rctx=0x7ffd52bc1df8 obj=default.rgw.meta:users.uid:foo state=0x55c4ae9910b0 s->prefetch_data=0 2023-10-05T19:42:15.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.396+0000 7f6543fff700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:42:45 2023 2023-10-05T19:42:15.404 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:42:15.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.396+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeb79550 con 0x55c4aeab87e0 2023-10-05T19:42:15.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.396+0000 7f65437fe700 20 reqs_thread_entry: start 2023-10-05T19:42:15.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.396+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(102 foo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv15 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7f660800a5a0 con 0x55c4aeab87e0 2023-10-05T19:42:15.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.396+0000 7f6619d06e40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:42:15.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.396+0000 7f6619d06e40 20 Read xattr: user.rgw.idtag 2023-10-05T19:42:15.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.396+0000 7f6619d06e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.396+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeb794f0 con 0x55c4aeab87e0 2023-10-05T19:42:15.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.400+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(103 foo [call,call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv15 ondisk = 0) v8 ==== 273+0+480 (crc 0 0 0) 0x7f660800a5a0 con 0x55c4aeab87e0 2023-10-05T19:42:15.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.400+0000 7f6619d06e40 20 rados_obj.operate() r=0 bl.length=357 2023-10-05T19:42:15.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.400+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:104 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeb794f0 con 0x7f6610039030 2023-10-05T19:42:15.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.400+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(104 usage.10 [call out=15b] v0'0 uv35 ondisk = 0) v8 ==== 152+0+15 (crc 0 0 0) 0x7f660800a5a0 con 0x7f6610039030 2023-10-05T19:42:15.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.400+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:105 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94303232686912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeb794f0 con 0x55c4aeaed7d0 2023-10-05T19:42:15.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.404+0000 7f661783b700 1 -- 172.21.15.12:0/507072695 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(105 notify.0 [watch unwatch cookie 94303232686912] v30'263 uv261 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6610036070 con 0x55c4aeaed7d0 2023-10-05T19:42:15.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.404+0000 7f6619d06e40 20 remove_watcher() i=0 2023-10-05T19:42:15.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.404+0000 7f6619d06e40 2 removed watcher, disabling cache 2023-10-05T19:42:15.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.404+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:106 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94303232689200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeacf890 con 0x55c4aeab87e0 2023-10-05T19:42:15.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.404+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(106 notify.1 [watch unwatch cookie 94303232689200] v30'263 uv261 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f660800a5a0 con 0x55c4aeab87e0 2023-10-05T19:42:15.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.404+0000 7f6619d06e40 20 remove_watcher() i=1 2023-10-05T19:42:15.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.404+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:107 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94303232693376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeb794f0 con 0x55c4aeaf2150 2023-10-05T19:42:15.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.408+0000 7f661703a700 1 -- 172.21.15.12:0/507072695 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(107 notify.2 [watch unwatch cookie 94303232693376] v30'263 uv261 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6600006070 con 0x55c4aeaf2150 2023-10-05T19:42:15.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.408+0000 7f6619d06e40 20 remove_watcher() i=2 2023-10-05T19:42:15.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.408+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:108 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94303232698624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeb794f0 con 0x55c4aeaed7d0 2023-10-05T19:42:15.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.412+0000 7f661783b700 1 -- 172.21.15.12:0/507072695 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(108 notify.3 [watch unwatch cookie 94303232698624] v30'263 uv261 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6610036070 con 0x55c4aeaed7d0 2023-10-05T19:42:15.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.412+0000 7f6619d06e40 20 remove_watcher() i=3 2023-10-05T19:42:15.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.412+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:109 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94303232700288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeb794f0 con 0x55c4aeab87e0 2023-10-05T19:42:15.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.412+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== osd_op_reply(109 notify.4 [watch unwatch cookie 94303232700288] v30'263 uv261 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f660800a5a0 con 0x55c4aeab87e0 2023-10-05T19:42:15.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.412+0000 7f6619d06e40 20 remove_watcher() i=4 2023-10-05T19:42:15.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.412+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:110 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94303232705072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeb794f0 con 0x7f6610039030 2023-10-05T19:42:15.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.416+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(110 notify.5 [watch unwatch cookie 94303232705072] v30'263 uv261 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f660800a5a0 con 0x7f6610039030 2023-10-05T19:42:15.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.416+0000 7f6619d06e40 20 remove_watcher() i=5 2023-10-05T19:42:15.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.416+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:111 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94303232702496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeb794f0 con 0x7f6610039030 2023-10-05T19:42:15.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.420+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(111 notify.6 [watch unwatch cookie 94303232702496] v30'263 uv261 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f660800a5a0 con 0x7f6610039030 2023-10-05T19:42:15.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.420+0000 7f6619d06e40 20 remove_watcher() i=6 2023-10-05T19:42:15.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.420+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:112 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94303232710784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeb794f0 con 0x7f6610039030 2023-10-05T19:42:15.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.424+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(112 notify.7 [watch unwatch cookie 94303232710784] v30'263 uv261 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f660800a5a0 con 0x7f6610039030 2023-10-05T19:42:15.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.424+0000 7f6619d06e40 20 remove_watcher() i=7 2023-10-05T19:42:15.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.424+0000 7f65e97e2700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:42:15.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.424+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:113 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94303232711888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55c4aeb794f0 con 0x55c4aeab87e0 2023-10-05T19:42:15.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.428+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 <== osd.1 v2:172.21.15.12:6808/29933 39 ==== osd_op_reply(113 data_loggenerations_metadata [watch unwatch cookie 94303232711888] v30'252 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f660800a5a0 con 0x55c4aeab87e0 2023-10-05T19:42:15.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.428+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f6610039030 msgr2=0x7f6610011a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:15.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.428+0000 7f6619d06e40 1 --2- 172.21.15.12:0/507072695 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f6610039030 0x7f6610011a30 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:15.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.428+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55c4aeab87e0 msgr2=0x55c4aeaeb5e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:15.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.428+0000 7f6619d06e40 1 --2- 172.21.15.12:0/507072695 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55c4aeab87e0 0x55c4aeaeb5e0 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:15.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.428+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55c4aeaed7d0 msgr2=0x55c4aeaefbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:15.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.428+0000 7f6619d06e40 1 --2- 172.21.15.12:0/507072695 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55c4aeaed7d0 0x55c4aeaefbb0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:15.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.428+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55c4aeaf2150 msgr2=0x55c4aeaf4530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:15.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.428+0000 7f6619d06e40 1 --2- 172.21.15.12:0/507072695 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55c4aeaf2150 0x55c4aeaf4530 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:15.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.428+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f65f8030c50 msgr2=0x7f65f8033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:15.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.428+0000 7f6619d06e40 1 --2- 172.21.15.12:0/507072695 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f65f8030c50 0x7f65f8033100 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f66000011e0 tx=0x7f66000013a0 comp rx=0 tx=0).stop 2023-10-05T19:42:15.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.428+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4aeab8410 msgr2=0x55c4aeadfbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:15.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.428+0000 7f6619d06e40 1 --2- 172.21.15.12:0/507072695 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4aeab8410 0x55c4aeadfbc0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f661001e040 tx=0x7f6610003050 comp rx=0 tx=0).stop 2023-10-05T19:42:15.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.428+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 reap_dead start 2023-10-05T19:42:15.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.428+0000 7f661803c700 1 -- 172.21.15.12:0/507072695 reap_dead start 2023-10-05T19:42:15.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.428+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 shutdown_connections 2023-10-05T19:42:15.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.428+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 >> 172.21.15.12:0/507072695 conn(0x55c4aeab2720 msgr2=0x55c4aead35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:15.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.428+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 shutdown_connections 2023-10-05T19:42:15.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.428+0000 7f6619d06e40 1 -- 172.21.15.12:0/507072695 wait complete. 2023-10-05T19:42:15.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.428+0000 7f6619d06e40 1 -- 172.21.15.12:0/2187694027 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f6558030c50 msgr2=0x7f6558033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:15.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.428+0000 7f6619d06e40 1 --2- 172.21.15.12:0/2187694027 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f6558030c50 0x7f6558033100 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x55c4aeafc9c0 tx=0x7f660800aee0 comp rx=0 tx=0).stop 2023-10-05T19:42:15.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.428+0000 7f6619d06e40 1 -- 172.21.15.12:0/2187694027 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4aeb0b1c0 msgr2=0x55c4aeb10d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:15.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.428+0000 7f6619d06e40 1 --2- 172.21.15.12:0/2187694027 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4aeb0b1c0 0x55c4aeb10d20 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x55c4aeb052a0 tx=0x7f660003c030 comp rx=0 tx=0).stop 2023-10-05T19:42:15.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.428+0000 7f6619d06e40 1 -- 172.21.15.12:0/2187694027 shutdown_connections 2023-10-05T19:42:15.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.428+0000 7f6619d06e40 1 --2- 172.21.15.12:0/2187694027 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f6558030c50 0x7f6558033100 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:15.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.428+0000 7f6619d06e40 1 --2- 172.21.15.12:0/2187694027 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c4aeb0b1c0 0x55c4aeb10d20 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:15.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.428+0000 7f6619d06e40 1 -- 172.21.15.12:0/2187694027 >> 172.21.15.12:0/2187694027 conn(0x55c4aeb06580 msgr2=0x55c4aeb04680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:15.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.432+0000 7f6619d06e40 1 -- 172.21.15.12:0/2187694027 shutdown_connections 2023-10-05T19:42:15.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.432+0000 7f6619d06e40 1 -- 172.21.15.12:0/2187694027 wait complete. 2023-10-05T19:42:15.438 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:42:15.438 INFO:teuthology.orchestra.run.smithi012.stdout: "entries": [], 2023-10-05T19:42:15.438 INFO:teuthology.orchestra.run.smithi012.stdout: "summary": [] 2023-10-05T19:42:15.438 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:42:15.442 DEBUG:tasks.util.rgw: json result: {'entries': [], 'summary': []} 2023-10-05T19:42:15.442 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'foo', '--purge-data'] 2023-10-05T19:42:15.443 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'foo', '--purge-data'] 2023-10-05T19:42:15.443 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid foo --purge-data 2023-10-05T19:42:15.485 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:42:15.485 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:42:15.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.480+0000 7f93afc5ee40 1 Processor -- start 2023-10-05T19:42:15.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.480+0000 7f93afc5ee40 1 -- start start 2023-10-05T19:42:15.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.480+0000 7f93afc5ee40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x56462babf2f0 0x56462bac16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:15.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.480+0000 7f93afc5ee40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x56462ba08bd0 con 0x56462babad10 2023-10-05T19:42:15.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.480+0000 7f93afc5ee40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x56462b9c5ee0 con 0x56462babf2f0 2023-10-05T19:42:15.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.480+0000 7f93acf92700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x56462babf2f0 0x56462bac16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:15.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.480+0000 7f93acf92700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x56462babf2f0 0x56462bac16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:54268/0 (socket says 172.21.15.12:54268) 2023-10-05T19:42:15.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.480+0000 7f93ad793700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x56462babad10 0x56462babb0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:47340/0 (socket says 172.21.15.12:47340) 2023-10-05T19:42:15.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.480+0000 7f93acf92700 1 -- 172.21.15.12:0/1040418432 learned_addr learned my addr 172.21.15.12:0/1040418432 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:42:15.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.484+0000 7f93acf92700 1 -- 172.21.15.12:0/1040418432 >> v1:172.21.15.8:6789/0 conn(0x56462babad10 legacy=0x56462babb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:15.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.484+0000 7f93a7fff700 1 -- 172.21.15.12:0/1040418432 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1312788028 0 0) 0x56462ba08bd0 con 0x56462babad10 2023-10-05T19:42:15.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.484+0000 7f93acf92700 1 -- 172.21.15.12:0/1040418432 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56462babb920 con 0x56462babf2f0 2023-10-05T19:42:15.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.484+0000 7f93acf92700 1 --2- 172.21.15.12:0/1040418432 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56462babf2f0 0x56462bac16d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f939800a650 tx=0x7f9398005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab7893e2448a2267 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:15.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.484+0000 7f93a7fff700 1 -- 172.21.15.12:0/1040418432 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9398008760 con 0x56462babf2f0 2023-10-05T19:42:15.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.484+0000 7f93a7fff700 1 -- 172.21.15.12:0/1040418432 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93980088c0 con 0x56462babf2f0 2023-10-05T19:42:15.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.484+0000 7f93a7fff700 1 -- 172.21.15.12:0/1040418432 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9398008bb0 con 0x56462babf2f0 2023-10-05T19:42:15.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.484+0000 7f93afc5ee40 1 -- 172.21.15.12:0/1040418432 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56462babf2f0 msgr2=0x56462bac16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:15.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.484+0000 7f93afc5ee40 1 --2- 172.21.15.12:0/1040418432 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56462babf2f0 0x56462bac16d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f939800a650 tx=0x7f9398005b10 comp rx=0 tx=0).stop 2023-10-05T19:42:15.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.484+0000 7f93afc5ee40 1 -- 172.21.15.12:0/1040418432 shutdown_connections 2023-10-05T19:42:15.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.484+0000 7f93afc5ee40 1 --2- 172.21.15.12:0/1040418432 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56462babf2f0 0x56462bac16d0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:15.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.484+0000 7f93afc5ee40 1 -- 172.21.15.12:0/1040418432 >> 172.21.15.12:0/1040418432 conn(0x56462bab6730 msgr2=0x56462bab8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:15.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.484+0000 7f93afc5ee40 1 -- 172.21.15.12:0/1040418432 shutdown_connections 2023-10-05T19:42:15.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.484+0000 7f93afc5ee40 1 -- 172.21.15.12:0/1040418432 wait complete. 2023-10-05T19:42:15.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.484+0000 7f93afc5ee40 1 Processor -- start 2023-10-05T19:42:15.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.484+0000 7f93afc5ee40 1 -- start start 2023-10-05T19:42:15.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.488+0000 7f93afc5ee40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56462babad10 0x56462bac43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:15.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.488+0000 7f93afc5ee40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56462b989e30 con 0x56462babad10 2023-10-05T19:42:15.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.488+0000 7f93ad793700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56462babad10 0x56462bac43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:15.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.488+0000 7f93ad793700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56462babad10 0x56462bac43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:54270/0 (socket says 172.21.15.12:54270) 2023-10-05T19:42:15.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.488+0000 7f93ad793700 1 -- 172.21.15.12:0/4236895153 learned_addr learned my addr 172.21.15.12:0/4236895153 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:42:15.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.488+0000 7f93ad793700 1 -- 172.21.15.12:0/4236895153 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9398005db0 con 0x56462babad10 2023-10-05T19:42:15.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.488+0000 7f93ad793700 1 --2- 172.21.15.12:0/4236895153 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56462babad10 0x56462bac43e0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f93a0001630 tx=0x7f93a00069b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c652843d8d890e32 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:15.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.488+0000 7f93a6ffd700 1 -- 172.21.15.12:0/4236895153 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f93a0006320 con 0x56462babad10 2023-10-05T19:42:15.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.488+0000 7f93a6ffd700 1 -- 172.21.15.12:0/4236895153 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a0013420 con 0x56462babad10 2023-10-05T19:42:15.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.488+0000 7f93a6ffd700 1 -- 172.21.15.12:0/4236895153 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f93a00135d0 con 0x56462babad10 2023-10-05T19:42:15.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.488+0000 7f93afc5ee40 1 -- 172.21.15.12:0/4236895153 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56462babad10 msgr2=0x56462bac43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:15.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.488+0000 7f93afc5ee40 1 --2- 172.21.15.12:0/4236895153 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56462babad10 0x56462bac43e0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f93a0001630 tx=0x7f93a00069b0 comp rx=0 tx=0).stop 2023-10-05T19:42:15.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.488+0000 7f93afc5ee40 1 -- 172.21.15.12:0/4236895153 shutdown_connections 2023-10-05T19:42:15.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.488+0000 7f93afc5ee40 1 --2- 172.21.15.12:0/4236895153 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56462babad10 0x56462bac43e0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:15.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.488+0000 7f93afc5ee40 1 -- 172.21.15.12:0/4236895153 >> 172.21.15.12:0/4236895153 conn(0x56462bab6730 msgr2=0x56462bad69b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:15.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.488+0000 7f93afc5ee40 1 -- 172.21.15.12:0/4236895153 shutdown_connections 2023-10-05T19:42:15.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.488+0000 7f93afc5ee40 1 -- 172.21.15.12:0/4236895153 wait complete. 2023-10-05T19:42:15.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.492+0000 7f93afc5ee40 1 Processor -- start 2023-10-05T19:42:15.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.492+0000 7f93afc5ee40 1 -- start start 2023-10-05T19:42:15.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.492+0000 7f93afc5ee40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56462babad10 0x56462bae3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:15.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.492+0000 7f93afc5ee40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56462b95ced0 con 0x56462babad10 2023-10-05T19:42:15.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.492+0000 7f93ad793700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56462babad10 0x56462bae3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:15.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.492+0000 7f93ad793700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56462babad10 0x56462bae3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:54272/0 (socket says 172.21.15.12:54272) 2023-10-05T19:42:15.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.492+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 learned_addr learned my addr 172.21.15.12:0/2409209808 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:42:15.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.492+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a0006c90 con 0x56462babad10 2023-10-05T19:42:15.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.492+0000 7f93ad793700 1 --2- 172.21.15.12:0/2409209808 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56462babad10 0x56462bae3d90 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f93a0000f80 tx=0x7f93a0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:15.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.492+0000 7f93a57fa700 1 -- 172.21.15.12:0/2409209808 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f93a0006320 con 0x56462babad10 2023-10-05T19:42:15.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.492+0000 7f93a57fa700 1 -- 172.21.15.12:0/2409209808 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a0013590 con 0x56462babad10 2023-10-05T19:42:15.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.492+0000 7f93a57fa700 1 -- 172.21.15.12:0/2409209808 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f93a0013e00 con 0x56462babad10 2023-10-05T19:42:15.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.492+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56462bae42d0 con 0x56462babad10 2023-10-05T19:42:15.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.492+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56462bae4710 con 0x56462babad10 2023-10-05T19:42:15.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.496+0000 7f93a57fa700 1 -- 172.21.15.12:0/2409209808 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f93a0028460 con 0x56462babad10 2023-10-05T19:42:15.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.496+0000 7f93a57fa700 1 --2- 172.21.15.12:0/2409209808 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f9394030c00 0x7f93940330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:15.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.496+0000 7f93afc5ee40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.496+0000 7f93a57fa700 1 -- 172.21.15.12:0/2409209808 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7f93a0045370 con 0x56462babad10 2023-10-05T19:42:15.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.496+0000 7f93acf92700 1 --2- 172.21.15.12:0/2409209808 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f9394030c00 0x7f93940330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:15.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.496+0000 7f93afc5ee40 1 --2- 172.21.15.12:0/2409209808 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x56462babf2f0 0x56462baef850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:15.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.496+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baeff00 con 0x56462babf2f0 2023-10-05T19:42:15.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.496+0000 7f93adf94700 1 --2- 172.21.15.12:0/2409209808 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x56462babf2f0 0x56462baef850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:15.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.496+0000 7f93acf92700 1 --2- 172.21.15.12:0/2409209808 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f9394030c00 0x7f93940330b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f93980030a0 tx=0x7f9398015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:15.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.496+0000 7f93adf94700 1 --2- 172.21.15.12:0/2409209808 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x56462babf2f0 0x56462baef850 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:15.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.496+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f93a800a500 con 0x56462babf2f0 2023-10-05T19:42:15.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.500+0000 7f93afc5ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:15.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.500+0000 7f93afc5ee40 20 realm 2023-10-05T19:42:15.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.500+0000 7f93afc5ee40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.500+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baece70 con 0x56462babf2f0 2023-10-05T19:42:15.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.500+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f93a800a500 con 0x56462babf2f0 2023-10-05T19:42:15.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.500+0000 7f93afc5ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:15.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.500+0000 7f93afc5ee40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:42:15.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.500+0000 7f93afc5ee40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.500+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf09e0 con 0x56462babf2f0 2023-10-05T19:42:15.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.500+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f93a800a500 con 0x56462babf2f0 2023-10-05T19:42:15.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.500+0000 7f93afc5ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:15.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.500+0000 7f93afc5ee40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.500+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf05c0 con 0x56462babf2f0 2023-10-05T19:42:15.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.500+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f93a800a500 con 0x56462babf2f0 2023-10-05T19:42:15.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.500+0000 7f93afc5ee40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:42:15.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.500+0000 7f93afc5ee40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.500+0000 7f93afc5ee40 1 --2- 172.21.15.12:0/2409209808 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x56462baf1aa0 0x56462baf3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:15.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.500+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf4520 con 0x56462baf1aa0 2023-10-05T19:42:15.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.500+0000 7f93ad793700 1 --2- 172.21.15.12:0/2409209808 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x56462baf1aa0 0x56462baf3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:15.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.500+0000 7f93ad793700 1 --2- 172.21.15.12:0/2409209808 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x56462baf1aa0 0x56462baf3e80 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:15.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.500+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f93a0024020 con 0x56462baf1aa0 2023-10-05T19:42:15.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.500+0000 7f93afc5ee40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:42:15.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.500+0000 7f93afc5ee40 20 searching for the correct realm 2023-10-05T19:42:15.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.500+0000 7f93afc5ee40 1 --2- 172.21.15.12:0/2409209808 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x56462baf6460 0x56462baf8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:15.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.500+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf8f20 con 0x56462baf6460 2023-10-05T19:42:15.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.504+0000 7f93acf92700 1 --2- 172.21.15.12:0/2409209808 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x56462baf6460 0x56462baf8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:15.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.504+0000 7f93acf92700 1 --2- 172.21.15.12:0/2409209808 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x56462baf6460 0x56462baf8870 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:15.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.504+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f9398012450 con 0x56462baf6460 2023-10-05T19:42:15.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.504+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf8f20 con 0x56462baf1aa0 2023-10-05T19:42:15.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.504+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f93a0024020 con 0x56462baf1aa0 2023-10-05T19:42:15.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.504+0000 7f93ad793700 1 --2- 172.21.15.12:0/2409209808 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f93a0043b90 0x7f93a0042380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:15.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.504+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf8f20 con 0x7f93a0043b90 2023-10-05T19:42:15.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.508+0000 7f93adf94700 1 --2- 172.21.15.12:0/2409209808 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f93a0043b90 0x7f93a0042380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:15.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.508+0000 7f93adf94700 1 --2- 172.21.15.12:0/2409209808 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f93a0043b90 0x7f93a0042380 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:15.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.508+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.508+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf8f20 con 0x56462babf2f0 2023-10-05T19:42:15.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.508+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.508+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf8f20 con 0x56462babf2f0 2023-10-05T19:42:15.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.508+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.508+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf8f20 con 0x56462babf2f0 2023-10-05T19:42:15.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.508+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(11 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.508+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf8f20 con 0x7f93a0043b90 2023-10-05T19:42:15.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.512+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.512+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf8f20 con 0x56462baf1aa0 2023-10-05T19:42:15.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.512+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f93a0024020 con 0x56462baf1aa0 2023-10-05T19:42:15.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.512+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf8f20 con 0x56462baf1aa0 2023-10-05T19:42:15.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.512+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f93a0024020 con 0x56462baf1aa0 2023-10-05T19:42:15.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.512+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf8f20 con 0x56462babf2f0 2023-10-05T19:42:15.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.512+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.512+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf8f20 con 0x7f93a0043b90 2023-10-05T19:42:15.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.512+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.512+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf8f20 con 0x56462babf2f0 2023-10-05T19:42:15.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.516+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.516+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf8f20 con 0x56462baf1aa0 2023-10-05T19:42:15.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.516+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(18 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f93a0024020 con 0x56462baf1aa0 2023-10-05T19:42:15.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.516+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf8f20 con 0x56462baf6460 2023-10-05T19:42:15.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.516+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9398012450 con 0x56462baf6460 2023-10-05T19:42:15.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.516+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf8f20 con 0x56462baf6460 2023-10-05T19:42:15.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.516+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9398012450 con 0x56462baf6460 2023-10-05T19:42:15.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.516+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf8f20 con 0x7f93a0043b90 2023-10-05T19:42:15.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.516+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.516+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf8f20 con 0x56462baf6460 2023-10-05T19:42:15.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.520+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9398012450 con 0x56462baf6460 2023-10-05T19:42:15.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.520+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf8f20 con 0x56462baf6460 2023-10-05T19:42:15.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.520+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9398012450 con 0x56462baf6460 2023-10-05T19:42:15.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.520+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf8f20 con 0x56462babf2f0 2023-10-05T19:42:15.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.520+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.520+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf8f20 con 0x56462babf2f0 2023-10-05T19:42:15.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.520+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.520+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf8f20 con 0x7f93a0043b90 2023-10-05T19:42:15.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.520+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.520+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf8f20 con 0x56462baf6460 2023-10-05T19:42:15.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.524+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9398012450 con 0x56462baf6460 2023-10-05T19:42:15.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.524+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf8f20 con 0x56462baf1aa0 2023-10-05T19:42:15.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.524+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f93a0024020 con 0x56462baf1aa0 2023-10-05T19:42:15.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.524+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf8f20 con 0x56462babf2f0 2023-10-05T19:42:15.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.524+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.524+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf8f20 con 0x7f93a0043b90 2023-10-05T19:42:15.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.524+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.524+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf8f20 con 0x56462baf6460 2023-10-05T19:42:15.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.524+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9398012450 con 0x56462baf6460 2023-10-05T19:42:15.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.528+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf8f20 con 0x7f93a0043b90 2023-10-05T19:42:15.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.528+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.528+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf8f20 con 0x56462baf1aa0 2023-10-05T19:42:15.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.528+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f93a0024020 con 0x56462baf1aa0 2023-10-05T19:42:15.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.528+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf8f20 con 0x56462baf6460 2023-10-05T19:42:15.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.528+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9398012450 con 0x56462baf6460 2023-10-05T19:42:15.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.528+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf8f20 con 0x7f93a0043b90 2023-10-05T19:42:15.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.528+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.528+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf8f20 con 0x56462babf2f0 2023-10-05T19:42:15.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.532+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.532+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf8f20 con 0x56462baf1aa0 2023-10-05T19:42:15.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.532+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(37 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f93a0024020 con 0x56462baf1aa0 2023-10-05T19:42:15.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.532+0000 7f93afc5ee40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:42:15.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.532+0000 7f93afc5ee40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:42:15.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.532+0000 7f93afc5ee40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:42:15.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.532+0000 7f93afc5ee40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:42:15.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.532+0000 7f93afc5ee40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.532+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf51e0 con 0x56462babf2f0 2023-10-05T19:42:15.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.532+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.532+0000 7f93afc5ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:15.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.532+0000 7f93afc5ee40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.532+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf4e10 con 0x56462babf2f0 2023-10-05T19:42:15.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.532+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.532+0000 7f93afc5ee40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:42:15.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.532+0000 7f93afc5ee40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.532+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf9390 con 0x56462baf6460 2023-10-05T19:42:15.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.536+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f9398012450 con 0x56462baf6460 2023-10-05T19:42:15.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.536+0000 7f93afc5ee40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:42:15.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.536+0000 7f93afc5ee40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.536+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf9740 con 0x56462baf6460 2023-10-05T19:42:15.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.536+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f9398012450 con 0x56462baf6460 2023-10-05T19:42:15.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.536+0000 7f93afc5ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:15.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.536+0000 7f93afc5ee40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.536+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf9af0 con 0x56462babf2f0 2023-10-05T19:42:15.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.536+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.536+0000 7f93afc5ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:15.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.536+0000 7f93afc5ee40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.536+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf9f00 con 0x56462babf2f0 2023-10-05T19:42:15.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.536+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.536+0000 7f93afc5ee40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:42:15.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.536+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf9ed0 con 0x56462baf6460 2023-10-05T19:42:15.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.540+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(44 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f9398012450 con 0x56462baf6460 2023-10-05T19:42:15.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.540+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf9ed0 con 0x56462baf1aa0 2023-10-05T19:42:15.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.540+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(45 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f93a0024020 con 0x56462baf1aa0 2023-10-05T19:42:15.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.540+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf9ed0 con 0x7f93a0043b90 2023-10-05T19:42:15.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.540+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(46 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.540+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf9af0 con 0x56462babf2f0 2023-10-05T19:42:15.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.544+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(47 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.544+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf9af0 con 0x56462babf2f0 2023-10-05T19:42:15.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.544+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(48 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.544+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf9af0 con 0x56462babf2f0 2023-10-05T19:42:15.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.544+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(49 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.544+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf9af0 con 0x7f93a0043b90 2023-10-05T19:42:15.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.544+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(50 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.544+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf9af0 con 0x56462baf1aa0 2023-10-05T19:42:15.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.544+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(51 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f93a0024020 con 0x56462baf1aa0 2023-10-05T19:42:15.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.544+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf9af0 con 0x56462baf1aa0 2023-10-05T19:42:15.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.548+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(52 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f93a0024020 con 0x56462baf1aa0 2023-10-05T19:42:15.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.548+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf9af0 con 0x56462babf2f0 2023-10-05T19:42:15.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.548+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(53 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.548+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf9af0 con 0x7f93a0043b90 2023-10-05T19:42:15.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.548+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(54 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.548+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf9af0 con 0x56462babf2f0 2023-10-05T19:42:15.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.548+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(55 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.548+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf9af0 con 0x56462baf1aa0 2023-10-05T19:42:15.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.552+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(56 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f93a0024020 con 0x56462baf1aa0 2023-10-05T19:42:15.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.552+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf9af0 con 0x56462baf6460 2023-10-05T19:42:15.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.552+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(57 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9398012450 con 0x56462baf6460 2023-10-05T19:42:15.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.552+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf9af0 con 0x56462baf6460 2023-10-05T19:42:15.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.552+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(58 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9398012450 con 0x56462baf6460 2023-10-05T19:42:15.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.552+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf9af0 con 0x7f93a0043b90 2023-10-05T19:42:15.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.552+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(59 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.552+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf9af0 con 0x56462baf6460 2023-10-05T19:42:15.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.552+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(60 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9398012450 con 0x56462baf6460 2023-10-05T19:42:15.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.556+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf9af0 con 0x56462baf6460 2023-10-05T19:42:15.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.556+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(61 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9398012450 con 0x56462baf6460 2023-10-05T19:42:15.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.556+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf9af0 con 0x56462babf2f0 2023-10-05T19:42:15.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.556+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(62 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.556+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf9af0 con 0x56462babf2f0 2023-10-05T19:42:15.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.556+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(63 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.556+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf9af0 con 0x7f93a0043b90 2023-10-05T19:42:15.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.556+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(64 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.556+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf9af0 con 0x56462baf6460 2023-10-05T19:42:15.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.560+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(65 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9398012450 con 0x56462baf6460 2023-10-05T19:42:15.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.560+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf9af0 con 0x56462baf1aa0 2023-10-05T19:42:15.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.560+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(66 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f93a0024020 con 0x56462baf1aa0 2023-10-05T19:42:15.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.560+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf9af0 con 0x56462babf2f0 2023-10-05T19:42:15.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.560+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(67 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.560+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf9af0 con 0x7f93a0043b90 2023-10-05T19:42:15.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.560+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(68 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.560+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf9af0 con 0x56462baf6460 2023-10-05T19:42:15.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.560+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(69 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9398012450 con 0x56462baf6460 2023-10-05T19:42:15.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.560+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf9af0 con 0x7f93a0043b90 2023-10-05T19:42:15.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.564+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(70 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.564+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf9af0 con 0x56462baf1aa0 2023-10-05T19:42:15.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.564+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(71 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f93a0024020 con 0x56462baf1aa0 2023-10-05T19:42:15.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.564+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf9af0 con 0x56462baf6460 2023-10-05T19:42:15.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.564+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(72 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9398012450 con 0x56462baf6460 2023-10-05T19:42:15.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.564+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf9af0 con 0x7f93a0043b90 2023-10-05T19:42:15.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.564+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(73 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.564+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf9af0 con 0x56462babf2f0 2023-10-05T19:42:15.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.568+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(74 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.568+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf9af0 con 0x56462baf1aa0 2023-10-05T19:42:15.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.568+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(75 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f93a0024020 con 0x56462baf1aa0 2023-10-05T19:42:15.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.568+0000 7f93afc5ee40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:42:15.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.568+0000 7f93afc5ee40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:42:15.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.568+0000 7f93afc5ee40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:42:15.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.568+0000 7f93afc5ee40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:42:15.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.568+0000 7f93afc5ee40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.568+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baf9ed0 con 0x56462babf2f0 2023-10-05T19:42:15.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.568+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.568+0000 7f93afc5ee40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:42:15.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.568+0000 7f93afc5ee40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.568+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bafa330 con 0x56462baf1aa0 2023-10-05T19:42:15.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.568+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f93a0024020 con 0x56462baf1aa0 2023-10-05T19:42:15.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.568+0000 7f93afc5ee40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:42:15.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.568+0000 7f93afc5ee40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.568+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bafabd0 con 0x56462babf2f0 2023-10-05T19:42:15.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.572+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.572+0000 7f93afc5ee40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:42:15.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.572+0000 7f93afc5ee40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.572+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bafabd0 con 0x56462baf6460 2023-10-05T19:42:15.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.572+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f9398012450 con 0x56462baf6460 2023-10-05T19:42:15.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.572+0000 7f93afc5ee40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:42:15.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.572+0000 7f93afc5ee40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.572+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bafb470 con 0x7f93a0043b90 2023-10-05T19:42:15.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.572+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.572+0000 7f93afc5ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:15.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.572+0000 7f93afc5ee40 20 zone default found 2023-10-05T19:42:15.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.572+0000 7f93afc5ee40 4 Realm: () 2023-10-05T19:42:15.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.572+0000 7f93afc5ee40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:42:15.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.572+0000 7f93afc5ee40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:42:15.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.572+0000 7f93afc5ee40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:42:15.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.572+0000 7f93afc5ee40 20 zonegroup default 2023-10-05T19:42:15.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.572+0000 7f93afc5ee40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.572+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bafb470 con 0x7f93a0043b90 2023-10-05T19:42:15.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.572+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.572+0000 7f93afc5ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:15.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.572+0000 7f93afc5ee40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.572+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bafcdf0 con 0x56462baf6460 2023-10-05T19:42:15.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.576+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f9398012450 con 0x56462baf6460 2023-10-05T19:42:15.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.576+0000 7f93afc5ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:15.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.576+0000 7f93afc5ee40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.576+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bafd1a0 con 0x56462baf6460 2023-10-05T19:42:15.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.576+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f9398012450 con 0x56462baf6460 2023-10-05T19:42:15.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.576+0000 7f93afc5ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:15.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.576+0000 7f93afc5ee40 20 started sync module instance, tier type = 2023-10-05T19:42:15.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.576+0000 7f93afc5ee40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:42:15.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.576+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bafc4b0 con 0x56462baf1aa0 2023-10-05T19:42:15.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.580+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(84 notify.0 [create] v30'264 uv264 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f93a0024020 con 0x56462baf1aa0 2023-10-05T19:42:15.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.580+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94859380637632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bafd550 con 0x56462baf1aa0 2023-10-05T19:42:15.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.580+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bafddf0 con 0x56462babf2f0 2023-10-05T19:42:15.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.580+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(85 notify.0 [watch watch cookie 94859380637632] v30'265 uv264 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f93a0024020 con 0x56462baf1aa0 2023-10-05T19:42:15.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.580+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(86 notify.1 [create] v30'264 uv264 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.580+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94859380639920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bafddf0 con 0x56462babf2f0 2023-10-05T19:42:15.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.580+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bafe870 con 0x56462baf6460 2023-10-05T19:42:15.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.584+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(87 notify.1 [watch watch cookie 94859380639920] v30'265 uv264 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.584+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(88 notify.2 [create] v30'264 uv264 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9398012450 con 0x56462baf6460 2023-10-05T19:42:15.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.584+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94859380644096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bafdd50 con 0x56462baf6460 2023-10-05T19:42:15.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.584+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462baff600 con 0x56462baf1aa0 2023-10-05T19:42:15.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.588+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(90 notify.3 [create] v30'264 uv264 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f93a0024020 con 0x56462baf1aa0 2023-10-05T19:42:15.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.588+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94859380652464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb002f0 con 0x56462baf1aa0 2023-10-05T19:42:15.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.588+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb00b90 con 0x56462babf2f0 2023-10-05T19:42:15.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.588+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(89 notify.2 [watch watch cookie 94859380644096] v30'265 uv264 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9398012450 con 0x56462baf6460 2023-10-05T19:42:15.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.592+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(91 notify.3 [watch watch cookie 94859380652464] v30'265 uv264 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f93a0024020 con 0x56462baf1aa0 2023-10-05T19:42:15.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.592+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(92 notify.4 [create] v30'264 uv264 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.592+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94859380653568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb00740 con 0x56462babf2f0 2023-10-05T19:42:15.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.592+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb00fe0 con 0x7f93a0043b90 2023-10-05T19:42:15.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.596+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(94 notify.5 [create] v30'264 uv264 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.596+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94859380656208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb012a0 con 0x7f93a0043b90 2023-10-05T19:42:15.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.596+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb01e00 con 0x7f93a0043b90 2023-10-05T19:42:15.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.596+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(93 notify.4 [watch watch cookie 94859380653568] v30'265 uv264 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.596+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(96 notify.6 [create] v30'264 uv264 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.596+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94859380654672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb01dc0 con 0x7f93a0043b90 2023-10-05T19:42:15.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.596+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb02860 con 0x7f93a0043b90 2023-10-05T19:42:15.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.596+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(95 notify.5 [watch watch cookie 94859380656208] v30'265 uv264 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.600+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(97 notify.6 [watch watch cookie 94859380654672] v30'265 uv264 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.600+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(98 notify.7 [create] v30'264 uv264 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.600+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94859380660432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb01d20 con 0x7f93a0043b90 2023-10-05T19:42:15.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.600+0000 7f93afc5ee40 20 add_watcher() i=0 2023-10-05T19:42:15.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.600+0000 7f93afc5ee40 20 add_watcher() i=1 2023-10-05T19:42:15.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.600+0000 7f93afc5ee40 20 add_watcher() i=2 2023-10-05T19:42:15.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.600+0000 7f93afc5ee40 20 add_watcher() i=3 2023-10-05T19:42:15.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.600+0000 7f93afc5ee40 20 add_watcher() i=4 2023-10-05T19:42:15.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.600+0000 7f93afc5ee40 20 add_watcher() i=5 2023-10-05T19:42:15.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.600+0000 7f93afc5ee40 20 add_watcher() i=6 2023-10-05T19:42:15.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.600+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(99 notify.7 [watch watch cookie 94859380660432] v30'265 uv264 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.600+0000 7f93afc5ee40 20 add_watcher() i=7 2023-10-05T19:42:15.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.600+0000 7f93afc5ee40 2 all 8 watchers are set, enabling cache 2023-10-05T19:42:15.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.600+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb00e50 con 0x56462babf2f0 2023-10-05T19:42:15.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.604+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.604+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94859380657744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb02a00 con 0x56462babf2f0 2023-10-05T19:42:15.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.604+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94859380657744] v30'253 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.604+0000 7f93afc5ee40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:42:15.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.604+0000 7f93afc5ee40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:42:15.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.604+0000 7f93afc5ee40 1 Processor -- start 2023-10-05T19:42:15.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.604+0000 7f937dfeb700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:42:15.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.604+0000 7f93afc5ee40 1 -- start start 2023-10-05T19:42:15.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.604+0000 7f93afc5ee40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56462bb0fc60 0x56462bb10030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:15.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.604+0000 7f93afc5ee40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56462bb10570 con 0x56462bb0fc60 2023-10-05T19:42:15.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.608+0000 7f93acf92700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56462bb0fc60 0x56462bb10030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:15.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.608+0000 7f93acf92700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56462bb0fc60 0x56462bb10030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:54286/0 (socket says 172.21.15.12:54286) 2023-10-05T19:42:15.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.608+0000 7f93acf92700 1 -- 172.21.15.12:0/3502115551 learned_addr learned my addr 172.21.15.12:0/3502115551 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:42:15.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.608+0000 7f93acf92700 1 -- 172.21.15.12:0/3502115551 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56462bb106b0 con 0x56462bb0fc60 2023-10-05T19:42:15.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.608+0000 7f93acf92700 1 --2- 172.21.15.12:0/3502115551 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56462bb0fc60 0x56462bb10030 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x56462babbd90 tx=0x7f9398012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2914f6c59a571307 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:15.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.608+0000 7f937d7ea700 1 -- 172.21.15.12:0/3502115551 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9398012d50 con 0x56462bb0fc60 2023-10-05T19:42:15.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.608+0000 7f937d7ea700 1 -- 172.21.15.12:0/3502115551 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9398030c10 con 0x56462bb0fc60 2023-10-05T19:42:15.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.608+0000 7f937d7ea700 1 -- 172.21.15.12:0/3502115551 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9398030dc0 con 0x56462bb0fc60 2023-10-05T19:42:15.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.608+0000 7f93afc5ee40 1 -- 172.21.15.12:0/3502115551 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56462bb0fc60 msgr2=0x56462bb10030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:15.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.608+0000 7f93afc5ee40 1 --2- 172.21.15.12:0/3502115551 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56462bb0fc60 0x56462bb10030 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x56462babbd90 tx=0x7f9398012000 comp rx=0 tx=0).stop 2023-10-05T19:42:15.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.608+0000 7f93afc5ee40 1 -- 172.21.15.12:0/3502115551 shutdown_connections 2023-10-05T19:42:15.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.608+0000 7f93afc5ee40 1 --2- 172.21.15.12:0/3502115551 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56462bb0fc60 0x56462bb10030 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:15.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.608+0000 7f93afc5ee40 1 -- 172.21.15.12:0/3502115551 >> 172.21.15.12:0/3502115551 conn(0x56462bb0b020 msgr2=0x56462bb09170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:15.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.608+0000 7f93afc5ee40 1 -- 172.21.15.12:0/3502115551 shutdown_connections 2023-10-05T19:42:15.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.608+0000 7f93afc5ee40 1 -- 172.21.15.12:0/3502115551 wait complete. 2023-10-05T19:42:15.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.608+0000 7f93afc5ee40 1 Processor -- start 2023-10-05T19:42:15.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.608+0000 7f93afc5ee40 1 -- start start 2023-10-05T19:42:15.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.608+0000 7f93afc5ee40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56462bb0fc60 0x56462bb15790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:15.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.608+0000 7f93afc5ee40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56462bb16790 con 0x56462bb0fc60 2023-10-05T19:42:15.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.612+0000 7f93acf92700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56462bb0fc60 0x56462bb15790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:15.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.612+0000 7f93acf92700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56462bb0fc60 0x56462bb15790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:54290/0 (socket says 172.21.15.12:54290) 2023-10-05T19:42:15.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.612+0000 7f93acf92700 1 -- 172.21.15.12:0/2551009071 learned_addr learned my addr 172.21.15.12:0/2551009071 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:42:15.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.612+0000 7f93acf92700 1 -- 172.21.15.12:0/2551009071 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56462babb920 con 0x56462bb0fc60 2023-10-05T19:42:15.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.612+0000 7f93acf92700 1 --2- 172.21.15.12:0/2551009071 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56462bb0fc60 0x56462bb15790 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x56462bb09ce0 tx=0x7f9398034000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:15.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.612+0000 7f92f77fe700 1 -- 172.21.15.12:0/2551009071 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9398042410 con 0x56462bb0fc60 2023-10-05T19:42:15.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.612+0000 7f92f77fe700 1 -- 172.21.15.12:0/2551009071 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9398042590 con 0x56462bb0fc60 2023-10-05T19:42:15.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.612+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56462bb15d90 con 0x56462bb0fc60 2023-10-05T19:42:15.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.612+0000 7f92f77fe700 1 -- 172.21.15.12:0/2551009071 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9398042740 con 0x56462bb0fc60 2023-10-05T19:42:15.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.612+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56462bb161d0 con 0x56462bb0fc60 2023-10-05T19:42:15.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.612+0000 7f92f77fe700 1 -- 172.21.15.12:0/2551009071 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f93980428a0 con 0x56462bb0fc60 2023-10-05T19:42:15.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.612+0000 7f92f77fe700 1 --2- 172.21.15.12:0/2551009071 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f92ec030c50 0x7f92ec033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:15.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.612+0000 7f92f77fe700 1 -- 172.21.15.12:0/2551009071 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7f9398056bd0 con 0x56462bb0fc60 2023-10-05T19:42:15.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.612+0000 7f93adf94700 1 --2- 172.21.15.12:0/2551009071 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f92ec030c50 0x7f92ec033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:15.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.612+0000 7f93afc5ee40 1 --2- 172.21.15.12:0/2551009071 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x56462bb20380 0x56462bb22760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:15.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.612+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb22d80 con 0x56462bb20380 2023-10-05T19:42:15.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.616+0000 7f93ad793700 1 --2- 172.21.15.12:0/2551009071 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x56462bb20380 0x56462bb22760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:15.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.616+0000 7f93adf94700 1 --2- 172.21.15.12:0/2551009071 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f92ec030c50 0x7f92ec033100 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x56462bb02960 tx=0x7f93a800ab60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:15.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.616+0000 7f93ad793700 1 --2- 172.21.15.12:0/2551009071 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x56462bb20380 0x56462bb22760 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:15.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.616+0000 7f93ad793700 1 -- 172.21.15.12:0/2551009071 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v30'37 uv30 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f93a0024020 con 0x56462bb20380 2023-10-05T19:42:15.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.616+0000 7f93afc5ee40 1 --2- 172.21.15.12:0/2551009071 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x56462bb23a50 0x56462bb25e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:15.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.616+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb26590 con 0x56462bb23a50 2023-10-05T19:42:15.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.616+0000 7f93acf92700 1 --2- 172.21.15.12:0/2551009071 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x56462bb23a50 0x56462bb25e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:15.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.620+0000 7f93acf92700 1 --2- 172.21.15.12:0/2551009071 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x56462bb23a50 0x56462bb25e60 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:15.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.620+0000 7f93acf92700 1 -- 172.21.15.12:0/2551009071 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v30'28 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f9398012450 con 0x56462bb23a50 2023-10-05T19:42:15.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.620+0000 7f93afc5ee40 1 --2- 172.21.15.12:0/2551009071 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x56462bb27260 0x56462bb29640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:15.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.620+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb29d70 con 0x56462bb27260 2023-10-05T19:42:15.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.620+0000 7f93adf94700 1 --2- 172.21.15.12:0/2551009071 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x56462bb27260 0x56462bb29640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:15.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.624+0000 7f93adf94700 1 --2- 172.21.15.12:0/2551009071 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x56462bb27260 0x56462bb29640 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:15.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.624+0000 7f93adf94700 1 -- 172.21.15.12:0/2551009071 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v30'28 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x56462bb27260 2023-10-05T19:42:15.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.624+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb2a3c0 con 0x56462bb20380 2023-10-05T19:42:15.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.628+0000 7f93ad793700 1 -- 172.21.15.12:0/2551009071 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v30'254 uv185 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f93a0024020 con 0x56462bb20380 2023-10-05T19:42:15.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.628+0000 7f93afc5ee40 1 --2- 172.21.15.12:0/2551009071 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x56462bb2b060 0x56462bb2d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:15.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.628+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb2daf0 con 0x56462bb2b060 2023-10-05T19:42:15.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.628+0000 7f93ad793700 1 --2- 172.21.15.12:0/2551009071 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x56462bb2b060 0x56462bb2d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:15.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.628+0000 7f93ad793700 1 --2- 172.21.15.12:0/2551009071 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x56462bb2b060 0x56462bb2d440 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:15.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.632+0000 7f93ad793700 1 -- 172.21.15.12:0/2551009071 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v30'74 uv51 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f93a0024020 con 0x56462bb2b060 2023-10-05T19:42:15.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.632+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb1bab0 con 0x56462bb27260 2023-10-05T19:42:15.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.636+0000 7f93adf94700 1 -- 172.21.15.12:0/2551009071 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v30'52 uv39 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x56462bb27260 2023-10-05T19:42:15.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.636+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb1bab0 con 0x56462bb20380 2023-10-05T19:42:15.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.636+0000 7f93ad793700 1 -- 172.21.15.12:0/2551009071 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v30'30 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f93a0024020 con 0x56462bb20380 2023-10-05T19:42:15.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.636+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb1bab0 con 0x56462bb23a50 2023-10-05T19:42:15.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.640+0000 7f93acf92700 1 -- 172.21.15.12:0/2551009071 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v30'49 uv36 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f9398012450 con 0x56462bb23a50 2023-10-05T19:42:15.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.640+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb2ed70 con 0x56462bb2b060 2023-10-05T19:42:15.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.644+0000 7f93ad793700 1 -- 172.21.15.12:0/2551009071 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v30'28 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f93a0024020 con 0x56462bb2b060 2023-10-05T19:42:15.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.644+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb2ed70 con 0x56462bb27260 2023-10-05T19:42:15.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.644+0000 7f93adf94700 1 -- 172.21.15.12:0/2551009071 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v30'34 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x56462bb27260 2023-10-05T19:42:15.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.644+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb2f390 con 0x56462bb2b060 2023-10-05T19:42:15.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.648+0000 7f93ad793700 1 -- 172.21.15.12:0/2551009071 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v30'75 uv53 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f93a0024020 con 0x56462bb2b060 2023-10-05T19:42:15.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.648+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb1b8c0 con 0x56462bb23a50 2023-10-05T19:42:15.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.652+0000 7f93acf92700 1 -- 172.21.15.12:0/2551009071 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v30'37 uv24 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9398012450 con 0x56462bb23a50 2023-10-05T19:42:15.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.652+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb2faf0 con 0x56462bb2b060 2023-10-05T19:42:15.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.656+0000 7f93ad793700 1 -- 172.21.15.12:0/2551009071 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v30'43 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f93a0024020 con 0x56462bb2b060 2023-10-05T19:42:15.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.656+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb2faf0 con 0x56462bb23a50 2023-10-05T19:42:15.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.656+0000 7f93acf92700 1 -- 172.21.15.12:0/2551009071 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v30'61 uv45 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9398012450 con 0x56462bb23a50 2023-10-05T19:42:15.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.660+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb2ffe0 con 0x56462bb20380 2023-10-05T19:42:15.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.660+0000 7f93ad793700 1 -- 172.21.15.12:0/2551009071 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v30'255 uv187 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f93a0024020 con 0x56462bb20380 2023-10-05T19:42:15.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.660+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb2ffe0 con 0x56462bb27260 2023-10-05T19:42:15.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.664+0000 7f93adf94700 1 -- 172.21.15.12:0/2551009071 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v30'74 uv49 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x56462bb27260 2023-10-05T19:42:15.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.664+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb1eed0 con 0x56462bb2b060 2023-10-05T19:42:15.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.668+0000 7f93ad793700 1 -- 172.21.15.12:0/2551009071 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v30'76 uv55 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f93a0024020 con 0x56462bb2b060 2023-10-05T19:42:15.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.668+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb1eed0 con 0x56462bb27260 2023-10-05T19:42:15.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.668+0000 7f93adf94700 1 -- 172.21.15.12:0/2551009071 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v30'28 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x56462bb27260 2023-10-05T19:42:15.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.668+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb1eed0 con 0x56462bb20380 2023-10-05T19:42:15.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.672+0000 7f93ad793700 1 -- 172.21.15.12:0/2551009071 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v30'52 uv41 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f93a0024020 con 0x56462bb20380 2023-10-05T19:42:15.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.672+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb1eed0 con 0x56462bb20380 2023-10-05T19:42:15.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.676+0000 7f93ad793700 1 -- 172.21.15.12:0/2551009071 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v30'256 uv183 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f93a0024020 con 0x56462bb20380 2023-10-05T19:42:15.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.676+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb1eed0 con 0x56462bb27260 2023-10-05T19:42:15.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.680+0000 7f93adf94700 1 -- 172.21.15.12:0/2551009071 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v30'75 uv45 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x56462bb27260 2023-10-05T19:42:15.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.680+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb1eed0 con 0x56462bb2b060 2023-10-05T19:42:15.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.680+0000 7f93ad793700 1 -- 172.21.15.12:0/2551009071 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v30'38 uv31 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f93a0024020 con 0x56462bb2b060 2023-10-05T19:42:15.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.684+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb1eed0 con 0x56462bb27260 2023-10-05T19:42:15.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.684+0000 7f93adf94700 1 -- 172.21.15.12:0/2551009071 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v30'32 uv25 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x56462bb27260 2023-10-05T19:42:15.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.684+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb1eed0 con 0x56462bb27260 2023-10-05T19:42:15.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.688+0000 7f93adf94700 1 -- 172.21.15.12:0/2551009071 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v30'37 uv30 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x56462bb27260 2023-10-05T19:42:15.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.688+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb1eed0 con 0x56462bb27260 2023-10-05T19:42:15.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.688+0000 7f93adf94700 1 -- 172.21.15.12:0/2551009071 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v30'32 uv18 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x56462bb27260 2023-10-05T19:42:15.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.688+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb1eed0 con 0x56462bb27260 2023-10-05T19:42:15.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.692+0000 7f93adf94700 1 -- 172.21.15.12:0/2551009071 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v30'53 uv37 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x56462bb27260 2023-10-05T19:42:15.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.692+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb1eed0 con 0x56462bb23a50 2023-10-05T19:42:15.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.696+0000 7f93acf92700 1 -- 172.21.15.12:0/2551009071 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v30'34 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9398012450 con 0x56462bb23a50 2023-10-05T19:42:15.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.696+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb1eed0 con 0x56462bb27260 2023-10-05T19:42:15.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.696+0000 7f93adf94700 1 -- 172.21.15.12:0/2551009071 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v30'29 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x56462bb27260 2023-10-05T19:42:15.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.696+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb1eed0 con 0x56462bb23a50 2023-10-05T19:42:15.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.700+0000 7f93acf92700 1 -- 172.21.15.12:0/2551009071 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v30'50 uv34 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9398012450 con 0x56462bb23a50 2023-10-05T19:42:15.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.700+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb1eed0 con 0x56462bb27260 2023-10-05T19:42:15.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.700+0000 7f93adf94700 1 -- 172.21.15.12:0/2551009071 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v30'76 uv47 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x56462bb27260 2023-10-05T19:42:15.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.700+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb1eed0 con 0x56462bb23a50 2023-10-05T19:42:15.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.704+0000 7f93acf92700 1 -- 172.21.15.12:0/2551009071 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v30'62 uv43 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9398012450 con 0x56462bb23a50 2023-10-05T19:42:15.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.704+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb1eed0 con 0x56462bb2b060 2023-10-05T19:42:15.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.708+0000 7f93ad793700 1 -- 172.21.15.12:0/2551009071 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v30'34 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f93a0024020 con 0x56462bb2b060 2023-10-05T19:42:15.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.712+0000 7f92f4ff9700 20 reqs_thread_entry: start 2023-10-05T19:42:15.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.712+0000 7f92ea7fc700 20 reqs_thread_entry: start 2023-10-05T19:42:15.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.712+0000 7f93afc5ee40 20 init_complete bucket index max shards: 11 2023-10-05T19:42:15.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.712+0000 7f93afc5ee40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:42:15.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.712+0000 7f92e8ff9700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:42:46 2023 2023-10-05T19:42:15.719 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:42:15.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.712+0000 7f93afc5ee40 20 get_system_obj_state: rctx=0x7ffc7c886be8 obj=default.rgw.meta:users.uid:foo state=0x56462b9950b0 s->prefetch_data=0 2023-10-05T19:42:15.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.712+0000 7f93afc5ee40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2023-10-05T19:42:15.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.712+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb8ebd0 con 0x56462babf2f0 2023-10-05T19:42:15.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.712+0000 7f92cffff700 20 reqs_thread_entry: start 2023-10-05T19:42:15.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.712+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(102 foo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv15 ondisk = 0) v8 ==== 231+0+139 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.712+0000 7f93afc5ee40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x16 2023-10-05T19:42:15.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.712+0000 7f93afc5ee40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:42:15.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.712+0000 7f93afc5ee40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:42:15.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.712+0000 7f93afc5ee40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:42:15.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.712+0000 7f93afc5ee40 20 Read xattr: user.rgw.idtag 2023-10-05T19:42:15.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.712+0000 7f93afc5ee40 10 cache get: name=default.rgw.meta+users.uid+foo : type miss (requested=0x13, cached=0x16) 2023-10-05T19:42:15.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.712+0000 7f93afc5ee40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.712+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb91660 con 0x56462babf2f0 2023-10-05T19:42:15.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.712+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(103 foo [call,call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv15 ondisk = 0) v8 ==== 273+0+480 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.712+0000 7f93afc5ee40 20 rados_obj.operate() r=0 bl.length=357 2023-10-05T19:42:15.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.712+0000 7f93afc5ee40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2023-10-05T19:42:15.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.712+0000 7f93afc5ee40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2023-10-05T19:42:15.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.712+0000 7f93afc5ee40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:42:15.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.712+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:104 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb91660 con 0x56462baf1aa0 2023-10-05T19:42:15.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.716+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(104 foo.buckets [call out=183b] v0'0 uv41 ondisk = 0) v8 ==== 155+0+183 (crc 0 0 0) 0x7f93a0024020 con 0x56462baf1aa0 2023-10-05T19:42:15.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.716+0000 7f93afc5ee40 20 get_system_obj_state: rctx=0x7ffc7c886890 obj=default.rgw.meta:root:.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6 state=0x56462b9950b0 s->prefetch_data=0 2023-10-05T19:42:15.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.716+0000 7f93afc5ee40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6 : miss 2023-10-05T19:42:15.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.716+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:105 8.1d 8:ba06c85e:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb90e70 con 0x7f93a0043b90 2023-10-05T19:42:15.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.716+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(105 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6 [call out=48b,getxattrs out=218b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 290+0+282 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.716+0000 7f93afc5ee40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6 info.flags=0x16 2023-10-05T19:42:15.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.716+0000 7f93afc5ee40 10 adding default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6 to cache LRU end 2023-10-05T19:42:15.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.716+0000 7f93afc5ee40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:42:15.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.716+0000 7f93afc5ee40 10 updating xattr: name=user.rgw.acl bl.length()=123 2023-10-05T19:42:15.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.716+0000 7f93afc5ee40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:42:15.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.716+0000 7f93afc5ee40 20 Read xattr: user.rgw.acl 2023-10-05T19:42:15.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.716+0000 7f93afc5ee40 20 Read xattr: user.rgw.idtag 2023-10-05T19:42:15.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.716+0000 7f93afc5ee40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6 : type miss (requested=0x13, cached=0x16) 2023-10-05T19:42:15.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.716+0000 7f93afc5ee40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.716+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:106 8.1d 8:ba06c85e:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb93210 con 0x7f93a0043b90 2023-10-05T19:42:15.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.716+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(106 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6 [call,call out=48b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 332+0+618 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.716+0000 7f93afc5ee40 20 rados_obj.operate() r=0 bl.length=352 2023-10-05T19:42:15.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.716+0000 7f93afc5ee40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6 info.flags=0x13 2023-10-05T19:42:15.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.716+0000 7f93afc5ee40 10 moving default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6 to cache LRU end 2023-10-05T19:42:15.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.716+0000 7f93afc5ee40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:42:15.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.716+0000 7f93afc5ee40 10 updating xattr: name=user.rgw.acl bl.length()=123 2023-10-05T19:42:15.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.716+0000 7f93afc5ee40 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6 2023-10-05T19:42:15.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.716+0000 7f93afc5ee40 10 int RGWRados::cls_bucket_list_unordered(const DoutPrefixProvider*, RGWBucketInfo&, int, const rgw_obj_index_key&, const string&, uint32_t, bool, std::vector&, bool*, rgw_obj_index_key*, optional_yield, RGWBucketListNameFilter) :myfoo[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6]) start_after="", prefix="", shard_id=-1", num_entries=1100, list_versions=1, force_check_filter is unset 2023-10-05T19:42:15.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.716+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:107 4.10 4:09eebd90:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bc0e5a0 con 0x56462baf6460 2023-10-05T19:42:15.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.720+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(107 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.0 [call out=85b] v0'0 uv9 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f9398012450 con 0x56462baf6460 2023-10-05T19:42:15.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.720+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:108 4.26 4:650d4ce8:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bc0e5a0 con 0x7f93a0043b90 2023-10-05T19:42:15.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.720+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(108 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.1 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.720+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:109 4.14 4:29362031:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bc0e5a0 con 0x56462baf6460 2023-10-05T19:42:15.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.720+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(109 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.2 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f9398012450 con 0x56462baf6460 2023-10-05T19:42:15.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.720+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:110 4.2f 4:f6580d38:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bc0e5a0 con 0x56462babf2f0 2023-10-05T19:42:15.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.724+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(110 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.3 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.724+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:111 4.7 4:e11072b4:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bc0e5a0 con 0x7f93a0043b90 2023-10-05T19:42:15.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.724+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(111 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.4 [call out=425b] v0'0 uv10 ondisk = 0) v8 ==== 194+0+425 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.724+0000 7f93afc5ee40 10 cls_bucket_list_unordered: got 15f6f8dc036519d7fe15b39338f6e5db 2023-10-05T19:42:15.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.724+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:112 4.26 4:65a16e0c:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bc0e5a0 con 0x7f93a0043b90 2023-10-05T19:42:15.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.724+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(112 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.5 [call out=85b] v0'0 uv4 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.724+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:113 4.22 4:44463ee5:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bc0e5a0 con 0x56462babf2f0 2023-10-05T19:42:15.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.724+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== osd_op_reply(113 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.6 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.724+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:114 4.14 4:2bdf07f8:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bc0e5a0 con 0x56462baf6460 2023-10-05T19:42:15.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.728+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(114 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.7 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f9398012450 con 0x56462baf6460 2023-10-05T19:42:15.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.728+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:115 4.f 4:f36617a4:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bc0e5a0 con 0x56462babf2f0 2023-10-05T19:42:15.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.728+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 39 ==== osd_op_reply(115 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.8 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.728+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:116 4.1f 4:f979c869:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bc0e5a0 con 0x56462baf6460 2023-10-05T19:42:15.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.728+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 <== osd.3 v2:172.21.15.12:6816/29951 27 ==== osd_op_reply(116 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.9 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f9398012450 con 0x56462baf6460 2023-10-05T19:42:15.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.728+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:117 4.7 4:e3f1771f:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bc0e5a0 con 0x7f93a0043b90 2023-10-05T19:42:15.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.732+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 30 ==== osd_op_reply(117 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.10 [call out=85b] v0'0 uv8 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.732+0000 7f93afc5ee40 20 get_obj_state: rctx=0x7ffc7c8868b0 obj=myfoo:15f6f8dc036519d7fe15b39338f6e5db state=0x56462bb90b68 s->prefetch_data=0 2023-10-05T19:42:15.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.732+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:118 3.c 3:30ab9517:::93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6_15f6f8dc036519d7fe15b39338f6e5db:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb91b60 con 0x56462baf6460 2023-10-05T19:42:15.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.732+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 <== osd.3 v2:172.21.15.12:6816/29951 28 ==== osd_op_reply(118 93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6_15f6f8dc036519d7fe15b39338f6e5db [getxattrs out=877b,stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 262+0+893 (crc 0 0 0) 0x7f9398012450 con 0x56462baf6460 2023-10-05T19:42:15.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.732+0000 7f93afc5ee40 10 manifest: total_size = 6 2023-10-05T19:42:15.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.732+0000 7f93afc5ee40 20 get_obj_state: setting s->obj_tag to 93ac49f4-ff49-4f99-811b-6453b28decd5.4219.1140115835386700545 2023-10-05T19:42:15.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.732+0000 7f93afc5ee40 20 get_obj_state: rctx=0x7ffc7c8868b0 obj=myfoo:15f6f8dc036519d7fe15b39338f6e5db state=0x56462bb90b68 s->prefetch_data=0 2023-10-05T19:42:15.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.732+0000 7f93afc5ee40 20 prepare_atomic_modification: state is not atomic. state=0x56462bb90b68 2023-10-05T19:42:15.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.732+0000 7f93afc5ee40 20 bucket index object: default.rgw.buckets.index:.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.4 2023-10-05T19:42:15.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.732+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:119 4.7 4:e11072b4:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.4:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb97980 con 0x7f93a0043b90 2023-10-05T19:42:15.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.736+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 31 ==== osd_op_reply(119 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.4 [stat,call,call] v30'11 uv11 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.736+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:120 3.c 3:30ab9517:::93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6_15f6f8dc036519d7fe15b39338f6e5db:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb95960 con 0x56462baf6460 2023-10-05T19:42:15.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.736+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 <== osd.3 v2:172.21.15.12:6816/29951 29 ==== osd_op_reply(120 93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6_15f6f8dc036519d7fe15b39338f6e5db [call] v30'4 uv4 ondisk = 0) v8 ==== 220+0+0 (crc 0 0 0) 0x7f9398012450 con 0x56462baf6460 2023-10-05T19:42:15.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.740+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:121 4.7 4:e11072b4:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.4:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb97fe0 con 0x7f93a0043b90 2023-10-05T19:42:15.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.740+0000 7f93afc5ee40 10 cache get: name=default.rgw.log++bucket.sync-source-hints.myfoo : miss 2023-10-05T19:42:15.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.740+0000 7f93afc5ee40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.740+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:122 6.18 6:1df25bda:::bucket.sync-source-hints.myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb97fe0 con 0x7f93a0043b90 2023-10-05T19:42:15.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.740+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 32 ==== osd_op_reply(122 bucket.sync-source-hints.myfoo [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 216+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.740+0000 7f93afc5ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:15.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.740+0000 7f93afc5ee40 10 cache put: name=default.rgw.log++bucket.sync-source-hints.myfoo info.flags=0x0 2023-10-05T19:42:15.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.740+0000 7f93afc5ee40 10 adding default.rgw.log++bucket.sync-source-hints.myfoo to cache LRU end 2023-10-05T19:42:15.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.740+0000 7f93afc5ee40 10 cache get: name=default.rgw.log++bucket.sync-target-hints.myfoo : miss 2023-10-05T19:42:15.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.740+0000 7f93afc5ee40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.740+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:123 6.4 6:26e4c4bc:::bucket.sync-target-hints.myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb9a4a0 con 0x56462babf2f0 2023-10-05T19:42:15.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.740+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 40 ==== osd_op_reply(123 bucket.sync-target-hints.myfoo [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 216+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.740+0000 7f93afc5ee40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:15.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.740+0000 7f93afc5ee40 10 cache put: name=default.rgw.log++bucket.sync-target-hints.myfoo info.flags=0x0 2023-10-05T19:42:15.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.740+0000 7f93afc5ee40 10 adding default.rgw.log++bucket.sync-target-hints.myfoo to cache LRU end 2023-10-05T19:42:15.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.740+0000 7f93afc5ee40 10 chain_cache_entry: cache_locator= 2023-10-05T19:42:15.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.740+0000 7f93afc5ee40 20 chain_cache_entry: couldn't find cache locator 2023-10-05T19:42:15.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.740+0000 7f93afc5ee40 20 couldn't put bucket_sync_policy cache entry, might have raced with data changes 2023-10-05T19:42:15.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.740+0000 7f93afc5ee40 20 int RGWRados::Bucket::List::list_objects_ordered(const DoutPrefixProvider*, int64_t, std::vector*, std::map, bool>*, bool*, optional_yield): starting attempt 1 2023-10-05T19:42:15.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.740+0000 7f93afc5ee40 10 int RGWRados::cls_bucket_list_ordered(const DoutPrefixProvider*, RGWBucketInfo&, int, const rgw_obj_index_key&, const string&, const string&, uint32_t, bool, uint16_t, RGWRados::ent_map_t&, bool*, bool*, rgw_obj_index_key*, optional_yield, RGWBucketListNameFilter): :myfoo[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6]) start_after="_multipart_", prefix="_multipart_, delimiter="", shard_id=-1", num_entries=1001, list_versions=0, expansion_factor=1, force_check_filter is unset 2023-10-05T19:42:15.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.740+0000 7f93afc5ee40 10 int RGWRados::cls_bucket_list_ordered(const DoutPrefixProvider*, RGWBucketInfo&, int, const rgw_obj_index_key&, const string&, const string&, uint32_t, bool, uint16_t, RGWRados::ent_map_t&, bool*, bool*, rgw_obj_index_key*, optional_yield, RGWBucketListNameFilter): request from each of 11 shard(s) for 112 entries to get 1001 total entries 2023-10-05T19:42:15.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.740+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:124 4.10 4:09eebd90:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.0:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb981b0 con 0x56462baf6460 2023-10-05T19:42:15.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.740+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:125 4.26 4:650d4ce8:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.1:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb99e90 con 0x7f93a0043b90 2023-10-05T19:42:15.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.740+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:126 4.14 4:29362031:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.2:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb981b0 con 0x56462baf6460 2023-10-05T19:42:15.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.740+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:127 4.2f 4:f6580d38:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.3:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bba1fb0 con 0x56462babf2f0 2023-10-05T19:42:15.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.740+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:128 4.7 4:e11072b4:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.4:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb981b0 con 0x7f93a0043b90 2023-10-05T19:42:15.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.740+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:129 4.26 4:65a16e0c:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.5:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bba7140 con 0x7f93a0043b90 2023-10-05T19:42:15.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.740+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:130 4.22 4:44463ee5:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.6:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb981b0 con 0x56462babf2f0 2023-10-05T19:42:15.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.740+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:131 4.14 4:2bdf07f8:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.7:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bbac220 con 0x56462baf6460 2023-10-05T19:42:15.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.740+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:132 4.f 4:f36617a4:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.8:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bbaeca0 con 0x56462babf2f0 2023-10-05T19:42:15.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.740+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:133 4.1f 4:f979c869:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.9:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bbb1ba0 con 0x56462baf6460 2023-10-05T19:42:15.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.740+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:134 4.7 4:e3f1771f:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.10:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bbb4a80 con 0x7f93a0043b90 2023-10-05T19:42:15.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.740+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 33 ==== osd_op_reply(125 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.1 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.740+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 41 ==== osd_op_reply(130 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.6 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.740+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 42 ==== osd_op_reply(127 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.3 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.740+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 <== osd.3 v2:172.21.15.12:6816/29951 30 ==== osd_op_reply(124 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.0 [call out=85b] v0'0 uv9 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f9398012450 con 0x56462baf6460 2023-10-05T19:42:15.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.740+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 34 ==== osd_op_reply(134 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.10 [call out=85b] v0'0 uv8 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.740+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 <== osd.3 v2:172.21.15.12:6816/29951 31 ==== osd_op_reply(126 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.2 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f9398012450 con 0x56462baf6460 2023-10-05T19:42:15.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.744+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 35 ==== osd_op_reply(121 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.4 [stat,call,call] v30'12 uv12 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.744+0000 7f93a67fc700 20 handle_completion(): completion ok for obj=15f6f8dc036519d7fe15b39338f6e5db 2023-10-05T19:42:15.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.744+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 36 ==== osd_op_reply(129 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.5 [call out=85b] v0'0 uv4 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.744+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 43 ==== osd_op_reply(132 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.8 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.744+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 <== osd.3 v2:172.21.15.12:6816/29951 32 ==== osd_op_reply(131 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.7 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f9398012450 con 0x56462baf6460 2023-10-05T19:42:15.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.744+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 <== osd.3 v2:172.21.15.12:6816/29951 33 ==== osd_op_reply(133 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.9 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f9398012450 con 0x56462baf6460 2023-10-05T19:42:15.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.744+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 37 ==== osd_op_reply(128 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.4 [call out=124b] v0'0 uv12 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.744+0000 7f93afc5ee40 20 int RGWRados::cls_bucket_list_ordered(const DoutPrefixProvider*, RGWBucketInfo&, int, const rgw_obj_index_key&, const string&, const string&, uint32_t, bool, uint16_t, RGWRados::ent_map_t&, bool*, bool*, rgw_obj_index_key*, optional_yield, RGWBucketListNameFilter): returning, count=0, is_truncated=0 2023-10-05T19:42:15.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.744+0000 7f93afc5ee40 20 int RGWRados::cls_bucket_list_ordered(const DoutPrefixProvider*, RGWBucketInfo&, int, const rgw_obj_index_key&, const string&, const string&, uint32_t, bool, uint16_t, RGWRados::ent_map_t&, bool*, bool*, rgw_obj_index_key*, optional_yield, RGWBucketListNameFilter): returning, last_entry NOT SET 2023-10-05T19:42:15.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.744+0000 7f93afc5ee40 10 int RGWRados::Bucket::List::list_objects_ordered(const DoutPrefixProvider*, int64_t, std::vector*, std::map, bool>*, bool*, optional_yield): end of outer loop, truncated=0, count=0, attempt=1 2023-10-05T19:42:15.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.744+0000 7f93afc5ee40 10 int RGWRados::Bucket::List::list_objects_ordered(const DoutPrefixProvider*, int64_t, std::vector*, std::map, bool>*, bool*, optional_yield): exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2023-10-05T19:42:15.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.744+0000 7f93afc5ee40 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":myfoo[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6])"; uploads.size()=0; is_truncated=0 2023-10-05T19:42:15.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.744+0000 7f93afc5ee40 10 lifecycle: RGWRados::convert_old_bucket_info(): bucket=:myfoo[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6]) 2023-10-05T19:42:15.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.744+0000 7f93afc5ee40 20 lifecycle: get_system_obj_state: rctx=0x7ffc7c886348 obj=default.rgw.meta:root:myfoo state=0x56462b9950b0 s->prefetch_data=0 2023-10-05T19:42:15.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.744+0000 7f93afc5ee40 10 lifecycle: cache get: name=default.rgw.meta+root+myfoo : miss 2023-10-05T19:42:15.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.744+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:135 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bbb4a80 con 0x56462baf1aa0 2023-10-05T19:42:15.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.744+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(135 myfoo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv39 ondisk = 0) v8 ==== 233+0+139 (crc 0 0 0) 0x7f93a0024020 con 0x56462baf1aa0 2023-10-05T19:42:15.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.744+0000 7f93afc5ee40 10 lifecycle: cache put: name=default.rgw.meta+root+myfoo info.flags=0x16 2023-10-05T19:42:15.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.744+0000 7f93afc5ee40 10 lifecycle: adding default.rgw.meta+root+myfoo to cache LRU end 2023-10-05T19:42:15.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.744+0000 7f93afc5ee40 10 lifecycle: updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:42:15.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.744+0000 7f93afc5ee40 20 lifecycle: get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:42:15.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.744+0000 7f93afc5ee40 20 lifecycle: Read xattr: user.rgw.idtag 2023-10-05T19:42:15.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.744+0000 7f93afc5ee40 10 lifecycle: cache get: name=default.rgw.meta+root+myfoo : type miss (requested=0x13, cached=0x16) 2023-10-05T19:42:15.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.744+0000 7f93afc5ee40 20 lifecycle: rados->read ofs=0 len=0 2023-10-05T19:42:15.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.744+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:136 8.9 8:91b71841:root::myfoo:head [call version.check_conds in=122b,call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bbb4a80 con 0x56462baf1aa0 2023-10-05T19:42:15.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.744+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 <== osd.2 v2:172.21.15.12:6800/29934 23 ==== osd_op_reply(136 myfoo [call,call out=48b,read 0~165 out=165b,getxattrs out=75b] v0'0 uv39 ondisk = 0) v8 ==== 275+0+288 (crc 0 0 0) 0x7f93a0024020 con 0x56462baf1aa0 2023-10-05T19:42:15.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.744+0000 7f93afc5ee40 20 lifecycle: rados_obj.operate() r=0 bl.length=165 2023-10-05T19:42:15.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.744+0000 7f93afc5ee40 10 lifecycle: cache put: name=default.rgw.meta+root+myfoo info.flags=0x13 2023-10-05T19:42:15.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.744+0000 7f93afc5ee40 10 lifecycle: moving default.rgw.meta+root+myfoo to cache LRU end 2023-10-05T19:42:15.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.744+0000 7f93afc5ee40 10 lifecycle: updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:42:15.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.744+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:137 8.1d 8:ba06c85e:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~352 in=352b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bba6bb0 con 0x7f93a0043b90 2023-10-05T19:42:15.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.748+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 38 ==== osd_op_reply(137 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6 [delete,create,call,call,writefull 0~352,setxattr (123)] v30'2 uv2 ondisk = 0) v8 ==== 416+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.748+0000 7f93afc5ee40 10 lifecycle: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6 info.flags=0x17 2023-10-05T19:42:15.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.748+0000 7f93afc5ee40 10 lifecycle: moving default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6 to cache LRU end 2023-10-05T19:42:15.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.748+0000 7f93afc5ee40 10 lifecycle: updating xattr: name=user.rgw.acl bl.length()=123 2023-10-05T19:42:15.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.748+0000 7f93afc5ee40 10 lifecycle: distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6, ofs0, ns] 2023-10-05T19:42:15.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.748+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:138 7.e 7:7759931f:::notify.3:head [notify cookie 94859381283184 in=737b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb93130 con 0x56462baf1aa0 2023-10-05T19:42:15.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.748+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 <== osd.2 v2:172.21.15.12:6800/29934 24 ==== watch-notify(notify (1) cookie 94859380652464 notify 128849018913 ret 0) v3 ==== 767+0+0 (crc 0 0 0) 0x7f93a0049040 con 0x56462baf1aa0 2023-10-05T19:42:15.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.748+0000 7f93a67fc700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018913 cookie 94859380652464 notifier 4753 bl.length()=725 2023-10-05T19:42:15.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.748+0000 7f93a67fc700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6 info.flags=0x17 2023-10-05T19:42:15.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.748+0000 7f93a67fc700 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6 to cache LRU end 2023-10-05T19:42:15.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.748+0000 7f93a67fc700 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2023-10-05T19:42:15.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.748+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 <== osd.2 v2:172.21.15.12:6800/29934 25 ==== osd_op_reply(138 notify.3 [notify cookie 94859381283184 out=8b] v0'0 uv264 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f93a0024020 con 0x56462baf1aa0 2023-10-05T19:42:15.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.748+0000 7f93a67fc700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:139 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f92fc0034e0 con 0x56462baf1aa0 2023-10-05T19:42:15.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.752+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 <== osd.2 v2:172.21.15.12:6800/29934 26 ==== osd_op_reply(139 notify.3 [notify-ack] v0'0 uv264 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f93a0024020 con 0x56462baf1aa0 2023-10-05T19:42:15.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.752+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 <== osd.2 v2:172.21.15.12:6800/29934 27 ==== watch-notify(notify_complete (2) cookie 94859381283184 notify 128849018913 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x56462bae4710 con 0x56462baf1aa0 2023-10-05T19:42:15.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.752+0000 7f93afc5ee40 20 lifecycle: handle_bi_update(): bucket=:myfoo[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6]): orig_sources= new_sources= 2023-10-05T19:42:15.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.752+0000 7f93afc5ee40 20 lifecycle: handle_bi_update(): bucket=:myfoo[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6]): potential sources added=[] removed=[] 2023-10-05T19:42:15.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.752+0000 7f93afc5ee40 20 lifecycle: handle_bi_update(): bucket=:myfoo[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6]): orig_dests= new_dests= 2023-10-05T19:42:15.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.752+0000 7f93afc5ee40 20 lifecycle: handle_bi_update(): bucket=:myfoo[93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6]): potential dests added=[] removed=[] 2023-10-05T19:42:15.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.752+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:33 6.11 6:8f661e56:lc::lc.12:head [call lock.lock in=65b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb91720 con 0x56462bb27260 2023-10-05T19:42:15.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.752+0000 7f93adf94700 1 -- 172.21.15.12:0/2551009071 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(33 lc.12 [call] v30'33 uv33 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x56462bb27260 2023-10-05T19:42:15.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.752+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 6.11 6:8f661e56:lc::lc.12:head [call rgw.lc_rm_entry in=92b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb91720 con 0x56462bb27260 2023-10-05T19:42:15.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.756+0000 7f93adf94700 1 -- 172.21.15.12:0/2551009071 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(34 lc.12 [call] v30'34 uv34 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x56462bb27260 2023-10-05T19:42:15.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.756+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:35 6.11 6:8f661e56:lc::lc.12:head [call lock.unlock in=49b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb91700 con 0x56462bb27260 2023-10-05T19:42:15.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.756+0000 7f93adf94700 1 -- 172.21.15.12:0/2551009071 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(35 lc.12 [call] v30'35 uv35 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x56462bb27260 2023-10-05T19:42:15.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.756+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:140 4.10 4:09eebd90:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bba6bb0 con 0x56462baf6460 2023-10-05T19:42:15.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.760+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:141 4.26 4:650d4ce8:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bba1a40 con 0x7f93a0043b90 2023-10-05T19:42:15.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.760+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:142 4.14 4:29362031:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb90980 con 0x56462baf6460 2023-10-05T19:42:15.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.760+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:143 4.2f 4:f6580d38:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb90d30 con 0x56462babf2f0 2023-10-05T19:42:15.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.760+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:144 4.7 4:e11072b4:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bba1a40 con 0x7f93a0043b90 2023-10-05T19:42:15.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.760+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:145 4.26 4:65a16e0c:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb90980 con 0x7f93a0043b90 2023-10-05T19:42:15.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.760+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:146 4.22 4:44463ee5:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb90d30 con 0x56462babf2f0 2023-10-05T19:42:15.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.760+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:147 4.14 4:2bdf07f8:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bba1a40 con 0x56462baf6460 2023-10-05T19:42:15.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.760+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:148 4.f 4:f36617a4:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb90980 con 0x56462babf2f0 2023-10-05T19:42:15.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.760+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:149 4.1f 4:f979c869:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb90d30 con 0x56462baf6460 2023-10-05T19:42:15.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.760+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:150 4.7 4:e3f1771f:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb9d770 con 0x7f93a0043b90 2023-10-05T19:42:15.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.760+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 <== osd.3 v2:172.21.15.12:6816/29951 34 ==== osd_op_reply(140 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.0 [call out=85b] v0'0 uv9 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f9398012450 con 0x56462baf6460 2023-10-05T19:42:15.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.760+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 <== osd.3 v2:172.21.15.12:6816/29951 35 ==== osd_op_reply(142 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.2 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f9398012450 con 0x56462baf6460 2023-10-05T19:42:15.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.760+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 39 ==== osd_op_reply(144 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.4 [call out=124b] v0'0 uv12 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.760+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 40 ==== osd_op_reply(141 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.1 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.760+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 44 ==== osd_op_reply(143 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.3 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.760+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 45 ==== osd_op_reply(146 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.6 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.760+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 46 ==== osd_op_reply(148 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.8 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.760+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 <== osd.3 v2:172.21.15.12:6816/29951 36 ==== osd_op_reply(149 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.9 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f9398012450 con 0x56462baf6460 2023-10-05T19:42:15.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.760+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 <== osd.3 v2:172.21.15.12:6816/29951 37 ==== osd_op_reply(147 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.7 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f9398012450 con 0x56462baf6460 2023-10-05T19:42:15.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.760+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 41 ==== osd_op_reply(145 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.5 [call out=85b] v0'0 uv4 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.760+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 42 ==== osd_op_reply(150 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.10 [call out=85b] v0'0 uv8 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.760+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:151 8.9 8:96d98480:users.uid::foo.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb93130 con 0x56462baf1aa0 2023-10-05T19:42:15.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.764+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 <== osd.2 v2:172.21.15.12:6800/29934 28 ==== osd_op_reply(151 foo.buckets [call] v30'48 uv48 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x7f93a0024020 con 0x56462baf1aa0 2023-10-05T19:42:15.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.764+0000 7f93afc5ee40 20 get_system_obj_state: rctx=0x7ffc7c885dc8 obj=default.rgw.meta:root:myfoo state=0x56462b9950b0 s->prefetch_data=0 2023-10-05T19:42:15.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.764+0000 7f93afc5ee40 10 cache get: name=default.rgw.meta+root+myfoo : type miss (requested=0x16, cached=0x13) 2023-10-05T19:42:15.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.764+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:152 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb93130 con 0x56462baf1aa0 2023-10-05T19:42:15.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.764+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 <== osd.2 v2:172.21.15.12:6800/29934 29 ==== osd_op_reply(152 myfoo [call out=48b,getxattrs out=75b,stat out=16b] v0'0 uv39 ondisk = 0) v8 ==== 233+0+139 (crc 0 0 0) 0x7f93a0024020 con 0x56462baf1aa0 2023-10-05T19:42:15.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.764+0000 7f93afc5ee40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x16 2023-10-05T19:42:15.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.764+0000 7f93afc5ee40 10 moving default.rgw.meta+root+myfoo to cache LRU end 2023-10-05T19:42:15.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.764+0000 7f93afc5ee40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2023-10-05T19:42:15.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.764+0000 7f93afc5ee40 20 get_system_obj_state: s->obj_tag was set empty 2023-10-05T19:42:15.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.764+0000 7f93afc5ee40 10 cache get: name=default.rgw.meta+root+myfoo : hit (requested=0x11, cached=0x17) 2023-10-05T19:42:15.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.764+0000 7f93afc5ee40 10 removing default.rgw.meta+root+myfoo from cache 2023-10-05T19:42:15.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.764+0000 7f93afc5ee40 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2023-10-05T19:42:15.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.764+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:153 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94859381256496 in=161b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb99940 con 0x7f93a0043b90 2023-10-05T19:42:15.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.764+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 43 ==== watch-notify(notify (1) cookie 94859380654672 notify 128849018927 ret 0) v3 ==== 191+0+0 (crc 0 0 0) 0x7f93a805a040 con 0x7f93a0043b90 2023-10-05T19:42:15.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.764+0000 7f93a6ffd700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018927 cookie 94859380654672 notifier 4753 bl.length()=149 2023-10-05T19:42:15.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.764+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 44 ==== osd_op_reply(153 notify.6 [notify cookie 94859381256496 out=8b] v0'0 uv264 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.764+0000 7f93a6ffd700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:154 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f9388003c50 con 0x7f93a0043b90 2023-10-05T19:42:15.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.768+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 45 ==== watch-notify(notify_complete (2) cookie 94859381256496 notify 128849018927 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f93a80269e0 con 0x7f93a0043b90 2023-10-05T19:42:15.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.768+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 46 ==== osd_op_reply(154 notify.6 [notify-ack] v0'0 uv264 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.768+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:155 8.9 8:91b71841:root::myfoo:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb99940 con 0x56462baf1aa0 2023-10-05T19:42:15.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.768+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 <== osd.2 v2:172.21.15.12:6800/29934 30 ==== osd_op_reply(155 myfoo [call,call,delete] v30'49 uv49 ondisk = 0) v8 ==== 233+0+0 (crc 0 0 0) 0x7f93a0024020 con 0x56462baf1aa0 2023-10-05T19:42:15.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.768+0000 7f93afc5ee40 10 removing default.rgw.meta+root+.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6 from cache 2023-10-05T19:42:15.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.768+0000 7f93afc5ee40 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6, ofs0, ns] 2023-10-05T19:42:15.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.768+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:156 7.e 7:7759931f:::notify.3:head [notify cookie 94859381256496 in=218b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb99940 con 0x56462baf1aa0 2023-10-05T19:42:15.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.768+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 <== osd.2 v2:172.21.15.12:6800/29934 31 ==== watch-notify(notify (1) cookie 94859380652464 notify 128849018914 ret 0) v3 ==== 248+0+0 (crc 0 0 0) 0x56462bae4710 con 0x56462baf1aa0 2023-10-05T19:42:15.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.768+0000 7f93a67fc700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018914 cookie 94859380652464 notifier 4753 bl.length()=206 2023-10-05T19:42:15.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.768+0000 7f93a67fc700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:157 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f92fc0034c0 con 0x56462baf1aa0 2023-10-05T19:42:15.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.768+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 <== osd.2 v2:172.21.15.12:6800/29934 32 ==== osd_op_reply(156 notify.3 [notify cookie 94859381256496 out=8b] v0'0 uv264 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f93a0024020 con 0x56462baf1aa0 2023-10-05T19:42:15.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.768+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 <== osd.2 v2:172.21.15.12:6800/29934 33 ==== watch-notify(notify_complete (2) cookie 94859381256496 notify 128849018914 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f93a002d070 con 0x56462baf1aa0 2023-10-05T19:42:15.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.772+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 <== osd.2 v2:172.21.15.12:6800/29934 34 ==== osd_op_reply(157 notify.3 [notify-ack] v0'0 uv264 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f93a0024020 con 0x56462baf1aa0 2023-10-05T19:42:15.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.772+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:158 8.1d 8:ba06c85e:root::.bucket.meta.myfoo%3a93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6:head [call version.check_conds in=122b,call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb99940 con 0x7f93a0043b90 2023-10-05T19:42:15.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.772+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 47 ==== osd_op_reply(158 .bucket.meta.myfoo:93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6 [call,call,delete] v30'3 uv3 ondisk = 0) v8 ==== 290+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.772+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:159 4.10 4:09eebd90:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb93130 con 0x56462baf6460 2023-10-05T19:42:15.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.772+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:160 4.26 4:650d4ce8:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb98e20 con 0x7f93a0043b90 2023-10-05T19:42:15.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.772+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:161 4.14 4:29362031:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb99740 con 0x56462baf6460 2023-10-05T19:42:15.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.772+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:162 4.2f 4:f6580d38:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb8f970 con 0x56462babf2f0 2023-10-05T19:42:15.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.772+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:163 4.7 4:e11072b4:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.4:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb99740 con 0x7f93a0043b90 2023-10-05T19:42:15.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.772+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:164 4.26 4:65a16e0c:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb98e20 con 0x7f93a0043b90 2023-10-05T19:42:15.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.772+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:165 4.22 4:44463ee5:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb90c80 con 0x56462babf2f0 2023-10-05T19:42:15.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.772+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:166 4.14 4:2bdf07f8:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb8f970 con 0x56462baf6460 2023-10-05T19:42:15.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.772+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:167 4.f 4:f36617a4:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bbb2bf0 con 0x56462babf2f0 2023-10-05T19:42:15.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.772+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:168 4.1f 4:f979c869:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb90c80 con 0x56462baf6460 2023-10-05T19:42:15.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.772+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:169 4.7 4:e3f1771f:::.dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.10:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bbb3a10 con 0x7f93a0043b90 2023-10-05T19:42:15.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.776+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 <== osd.3 v2:172.21.15.12:6816/29951 38 ==== osd_op_reply(159 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.0 [delete] v30'10 uv10 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f9398012450 con 0x56462baf6460 2023-10-05T19:42:15.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.776+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 48 ==== osd_op_reply(163 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.4 [delete] v30'13 uv13 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.776+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 47 ==== osd_op_reply(162 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.3 [delete] v30'2 uv2 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.776+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 49 ==== osd_op_reply(160 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.1 [delete] v30'5 uv5 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.776+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 48 ==== osd_op_reply(167 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.8 [delete] v30'2 uv2 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.776+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 50 ==== osd_op_reply(169 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.10 [delete] v30'14 uv14 ondisk = 0) v8 ==== 195+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.780+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 <== osd.3 v2:172.21.15.12:6816/29951 39 ==== osd_op_reply(161 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.2 [delete] v30'3 uv3 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f9398012450 con 0x56462baf6460 2023-10-05T19:42:15.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.780+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 51 ==== osd_op_reply(164 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.5 [delete] v30'6 uv6 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.780+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 <== osd.3 v2:172.21.15.12:6816/29951 40 ==== osd_op_reply(168 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.9 [delete] v30'2 uv2 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f9398012450 con 0x56462baf6460 2023-10-05T19:42:15.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.780+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 49 ==== osd_op_reply(165 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.6 [delete] v30'8 uv8 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.780+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 <== osd.3 v2:172.21.15.12:6816/29951 41 ==== osd_op_reply(166 .dir.93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6.7 [delete] v30'4 uv4 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f9398012450 con 0x56462baf6460 2023-10-05T19:42:15.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.780+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:170 6.1f 6:fe358a86:::pubsub..bucket.myfoo%2f93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb8f970 con 0x7f93a0043b90 2023-10-05T19:42:15.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.780+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 52 ==== osd_op_reply(170 pubsub..bucket.myfoo/93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6 [getxattrs,stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 250+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.780+0000 7f93afc5ee40 10 removing default.rgw.log++pubsub..bucket.myfoo/93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6 from cache 2023-10-05T19:42:15.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.780+0000 7f93afc5ee40 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.log:pubsub..bucket.myfoo/93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6, ofs0, ns] 2023-10-05T19:42:15.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.780+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:171 7.15 7:a93a5511:::notify.2:head [notify cookie 94859381390880 in=215b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb99940 con 0x56462baf6460 2023-10-05T19:42:15.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.780+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 <== osd.3 v2:172.21.15.12:6816/29951 42 ==== watch-notify(notify (1) cookie 94859380644096 notify 128849018892 ret 0) v3 ==== 245+0+0 (crc 0 0 0) 0x7f9398068040 con 0x56462baf6460 2023-10-05T19:42:15.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.780+0000 7f93a67fc700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018892 cookie 94859380644096 notifier 4753 bl.length()=203 2023-10-05T19:42:15.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.780+0000 7f93a67fc700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:172 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f92fc0034c0 con 0x56462baf6460 2023-10-05T19:42:15.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.780+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 <== osd.3 v2:172.21.15.12:6816/29951 43 ==== osd_op_reply(171 notify.2 [notify cookie 94859381390880 out=8b] v0'0 uv264 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f9398012450 con 0x56462baf6460 2023-10-05T19:42:15.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.780+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 <== osd.3 v2:172.21.15.12:6816/29951 44 ==== watch-notify(notify_complete (2) cookie 94859381390880 notify 128849018892 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x56462bb161d0 con 0x56462baf6460 2023-10-05T19:42:15.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.780+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 <== osd.3 v2:172.21.15.12:6816/29951 45 ==== osd_op_reply(172 notify.2 [notify-ack] v0'0 uv264 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9398012450 con 0x56462baf6460 2023-10-05T19:42:15.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.780+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:173 6.1f 6:fe358a86:::pubsub..bucket.myfoo%2f93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bbb3e20 con 0x7f93a0043b90 2023-10-05T19:42:15.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.784+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 53 ==== osd_op_reply(173 pubsub..bucket.myfoo/93ac49f4-ff49-4f99-811b-6453b28decd5.4221.6 [delete] v30'35 uv27 ondisk = -2 ((2) No such file or directory)) v8 ==== 208+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.784+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:174 8.9 8:96d98480:users.uid::foo.buckets:head [call user.remove_bucket in=58b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb99940 con 0x56462baf1aa0 2023-10-05T19:42:15.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.788+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 <== osd.2 v2:172.21.15.12:6800/29934 35 ==== osd_op_reply(174 foo.buckets [call] v30'50 uv50 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x7f93a0024020 con 0x56462baf1aa0 2023-10-05T19:42:15.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.788+0000 7f93afc5ee40 10 removing key index: 9te6NH5mcdcq0Tc5i8i1 2023-10-05T19:42:15.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.788+0000 7f93afc5ee40 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:users.keys:9te6NH5mcdcq0Tc5i8i1, ofs0, ns] 2023-10-05T19:42:15.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.788+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:175 7.e 7:7759931f:::notify.3:head [notify cookie 94859381390880 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb99940 con 0x56462baf1aa0 2023-10-05T19:42:15.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.788+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 <== osd.2 v2:172.21.15.12:6800/29934 36 ==== watch-notify(notify (1) cookie 94859380652464 notify 128849018915 ret 0) v3 ==== 212+0+0 (crc 0 0 0) 0x7f93a002d070 con 0x56462baf1aa0 2023-10-05T19:42:15.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.788+0000 7f93a6ffd700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018915 cookie 94859380652464 notifier 4753 bl.length()=170 2023-10-05T19:42:15.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.788+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 <== osd.2 v2:172.21.15.12:6800/29934 37 ==== osd_op_reply(175 notify.3 [notify cookie 94859381390880 out=8b] v0'0 uv264 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f93a0024020 con 0x56462baf1aa0 2023-10-05T19:42:15.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.788+0000 7f93a6ffd700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:176 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f9388003c50 con 0x56462baf1aa0 2023-10-05T19:42:15.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.788+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 <== osd.2 v2:172.21.15.12:6800/29934 38 ==== watch-notify(notify_complete (2) cookie 94859381390880 notify 128849018915 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x56462bae42d0 con 0x56462baf1aa0 2023-10-05T19:42:15.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.788+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 <== osd.2 v2:172.21.15.12:6800/29934 39 ==== osd_op_reply(176 notify.3 [notify-ack] v0'0 uv264 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f93a0024020 con 0x56462baf1aa0 2023-10-05T19:42:15.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.788+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:177 8.7 8:e4e0e270:users.keys::9te6NH5mcdcq0Tc5i8i1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bbb3e20 con 0x7f93a0043b90 2023-10-05T19:42:15.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.792+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 54 ==== osd_op_reply(177 9te6NH5mcdcq0Tc5i8i1 [delete] v30'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.792+0000 7f93afc5ee40 10 removing email index: foo@foo.com 2023-10-05T19:42:15.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.792+0000 7f93afc5ee40 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:users.email:foo@foo.com, ofs0, ns] 2023-10-05T19:42:15.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.792+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:178 7.e 7:7759931f:::notify.3:head [notify cookie 94859381390880 in=174b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb99940 con 0x56462baf1aa0 2023-10-05T19:42:15.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.792+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 <== osd.2 v2:172.21.15.12:6800/29934 40 ==== watch-notify(notify (1) cookie 94859380652464 notify 128849018916 ret 0) v3 ==== 204+0+0 (crc 0 0 0) 0x56462bae42d0 con 0x56462baf1aa0 2023-10-05T19:42:15.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.792+0000 7f93a6ffd700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018916 cookie 94859380652464 notifier 4753 bl.length()=162 2023-10-05T19:42:15.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.792+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 <== osd.2 v2:172.21.15.12:6800/29934 41 ==== osd_op_reply(178 notify.3 [notify cookie 94859381390880 out=8b] v0'0 uv264 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f93a0024020 con 0x56462baf1aa0 2023-10-05T19:42:15.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.792+0000 7f93a6ffd700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:179 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f9388003c50 con 0x56462baf1aa0 2023-10-05T19:42:15.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.792+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 <== osd.2 v2:172.21.15.12:6800/29934 42 ==== watch-notify(notify_complete (2) cookie 94859381390880 notify 128849018916 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f93a0006c90 con 0x56462baf1aa0 2023-10-05T19:42:15.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.792+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 <== osd.2 v2:172.21.15.12:6800/29934 43 ==== osd_op_reply(179 notify.3 [notify-ack] v0'0 uv264 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f93a0024020 con 0x56462baf1aa0 2023-10-05T19:42:15.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.792+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:180 8.3 8:c162d5d1:users.email::foo@foo.com:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bbb3e20 con 0x56462babf2f0 2023-10-05T19:42:15.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.796+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 50 ==== osd_op_reply(180 foo@foo.com [delete] v30'2 uv2 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.796+0000 7f93afc5ee40 10 removing user buckets index 2023-10-05T19:42:15.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.796+0000 7f93afc5ee40 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.uid:foo.buckets, ofs0, ns] 2023-10-05T19:42:15.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.796+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:181 7.3 7:c609908c:::notify.5:head [notify cookie 94859381390880 in=172b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb99940 con 0x7f93a0043b90 2023-10-05T19:42:15.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.796+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 55 ==== watch-notify(notify (1) cookie 94859380656208 notify 128849018928 ret 0) v3 ==== 202+0+0 (crc 0 0 0) 0x7f93a8063040 con 0x7f93a0043b90 2023-10-05T19:42:15.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.796+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 56 ==== osd_op_reply(181 notify.5 [notify cookie 94859381390880 out=8b] v0'0 uv264 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.796+0000 7f93a67fc700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018928 cookie 94859380656208 notifier 4753 bl.length()=160 2023-10-05T19:42:15.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.796+0000 7f93a67fc700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:182 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f92fc0034c0 con 0x7f93a0043b90 2023-10-05T19:42:15.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.796+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 57 ==== osd_op_reply(182 notify.5 [notify-ack] v0'0 uv264 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.796+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 58 ==== watch-notify(notify_complete (2) cookie 94859381390880 notify 128849018928 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f93a80269e0 con 0x7f93a0043b90 2023-10-05T19:42:15.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.796+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:183 8.9 8:96d98480:users.uid::foo.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bbb3e20 con 0x56462baf1aa0 2023-10-05T19:42:15.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.800+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 <== osd.2 v2:172.21.15.12:6800/29934 44 ==== osd_op_reply(183 foo.buckets [delete] v30'51 uv51 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x7f93a0024020 con 0x56462baf1aa0 2023-10-05T19:42:15.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.800+0000 7f93afc5ee40 10 removing user index: foo 2023-10-05T19:42:15.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.800+0000 7f93afc5ee40 10 removing default.rgw.meta+users.uid+foo from cache 2023-10-05T19:42:15.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.800+0000 7f93afc5ee40 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2023-10-05T19:42:15.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.800+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:184 7.d 7:b4812045:::notify.4:head [notify cookie 94859381390880 in=164b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb99940 con 0x56462babf2f0 2023-10-05T19:42:15.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.800+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 51 ==== watch-notify(notify (1) cookie 94859380653568 notify 128849018898 ret 0) v3 ==== 194+0+0 (crc 0 0 0) 0x7f93a80269e0 con 0x56462babf2f0 2023-10-05T19:42:15.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.800+0000 7f93a67fc700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018898 cookie 94859380653568 notifier 4753 bl.length()=152 2023-10-05T19:42:15.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.800+0000 7f93a67fc700 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:185 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x7f92fc0034c0 con 0x56462babf2f0 2023-10-05T19:42:15.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.800+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 52 ==== osd_op_reply(184 notify.4 [notify cookie 94859381390880 out=8b] v0'0 uv264 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.800+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 53 ==== watch-notify(notify_complete (2) cookie 94859381390880 notify 128849018898 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f93a8026de0 con 0x56462babf2f0 2023-10-05T19:42:15.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.800+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 54 ==== osd_op_reply(185 notify.4 [notify-ack] v0'0 uv264 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.804+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:186 8.18 8:1e6ca5ed:users.uid::foo:head [call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bb99940 con 0x56462babf2f0 2023-10-05T19:42:15.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.804+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 55 ==== osd_op_reply(186 foo [call,delete] v30'20 uv20 ondisk = 0) v8 ==== 189+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.804+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:187 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94859380637632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bad1690 con 0x56462baf1aa0 2023-10-05T19:42:15.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.808+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 <== osd.2 v2:172.21.15.12:6800/29934 45 ==== osd_op_reply(187 notify.0 [watch unwatch cookie 94859380637632] v30'266 uv264 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f93a0024020 con 0x56462baf1aa0 2023-10-05T19:42:15.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.808+0000 7f93afc5ee40 20 remove_watcher() i=0 2023-10-05T19:42:15.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.808+0000 7f93afc5ee40 2 removed watcher, disabling cache 2023-10-05T19:42:15.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.808+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:188 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94859380639920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bad1690 con 0x56462babf2f0 2023-10-05T19:42:15.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.812+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 56 ==== osd_op_reply(188 notify.1 [watch unwatch cookie 94859380639920] v30'266 uv264 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.812+0000 7f93afc5ee40 20 remove_watcher() i=1 2023-10-05T19:42:15.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.812+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:189 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94859380644096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bad1690 con 0x56462baf6460 2023-10-05T19:42:15.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.812+0000 7f93acf92700 1 -- 172.21.15.12:0/2409209808 <== osd.3 v2:172.21.15.12:6816/29951 46 ==== osd_op_reply(189 notify.2 [watch unwatch cookie 94859380644096] v30'266 uv264 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9398012450 con 0x56462baf6460 2023-10-05T19:42:15.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.812+0000 7f93afc5ee40 20 remove_watcher() i=2 2023-10-05T19:42:15.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.812+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:190 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94859380652464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bad1690 con 0x56462baf1aa0 2023-10-05T19:42:15.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.816+0000 7f93ad793700 1 -- 172.21.15.12:0/2409209808 <== osd.2 v2:172.21.15.12:6800/29934 46 ==== osd_op_reply(190 notify.3 [watch unwatch cookie 94859380652464] v30'266 uv264 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f93a0024020 con 0x56462baf1aa0 2023-10-05T19:42:15.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.816+0000 7f93afc5ee40 20 remove_watcher() i=3 2023-10-05T19:42:15.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.816+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:191 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94859380653568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bad1690 con 0x56462babf2f0 2023-10-05T19:42:15.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.820+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 57 ==== osd_op_reply(191 notify.4 [watch unwatch cookie 94859380653568] v30'266 uv264 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.820+0000 7f93afc5ee40 20 remove_watcher() i=4 2023-10-05T19:42:15.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.820+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:192 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94859380656208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bad1690 con 0x7f93a0043b90 2023-10-05T19:42:15.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.820+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 59 ==== osd_op_reply(192 notify.5 [watch unwatch cookie 94859380656208] v30'266 uv264 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.824+0000 7f93afc5ee40 20 remove_watcher() i=5 2023-10-05T19:42:15.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.824+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:193 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94859380654672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bad1690 con 0x7f93a0043b90 2023-10-05T19:42:15.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.824+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 60 ==== osd_op_reply(193 notify.6 [watch unwatch cookie 94859380654672] v30'266 uv264 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.824+0000 7f93afc5ee40 20 remove_watcher() i=6 2023-10-05T19:42:15.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.824+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:194 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94859380660432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bad1690 con 0x7f93a0043b90 2023-10-05T19:42:15.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.828+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.0 v2:172.21.15.8:6800/27563 61 ==== osd_op_reply(194 notify.7 [watch unwatch cookie 94859380660432] v30'266 uv264 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x7f93a0043b90 2023-10-05T19:42:15.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.828+0000 7f93afc5ee40 20 remove_watcher() i=7 2023-10-05T19:42:15.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.832+0000 7f937dfeb700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:42:15.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.832+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:195 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94859380657744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x56462bad1690 con 0x56462babf2f0 2023-10-05T19:42:15.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.832+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 <== osd.1 v2:172.21.15.12:6808/29933 58 ==== osd_op_reply(195 data_loggenerations_metadata [watch unwatch cookie 94859380657744] v30'257 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f93a8014040 con 0x56462babf2f0 2023-10-05T19:42:15.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.832+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f93a0043b90 msgr2=0x7f93a0042380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:15.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.832+0000 7f93afc5ee40 1 --2- 172.21.15.12:0/2409209808 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f93a0043b90 0x7f93a0042380 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:15.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.832+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x56462babf2f0 msgr2=0x56462baef850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:15.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.832+0000 7f93afc5ee40 1 --2- 172.21.15.12:0/2409209808 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x56462babf2f0 0x56462baef850 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:15.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.832+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x56462baf1aa0 msgr2=0x56462baf3e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:15.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.832+0000 7f93afc5ee40 1 --2- 172.21.15.12:0/2409209808 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x56462baf1aa0 0x56462baf3e80 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:15.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.832+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x56462baf6460 msgr2=0x56462baf8870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:15.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.832+0000 7f93afc5ee40 1 --2- 172.21.15.12:0/2409209808 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x56462baf6460 0x56462baf8870 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:15.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.832+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f9394030c00 msgr2=0x7f93940330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:15.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.832+0000 7f93afc5ee40 1 --2- 172.21.15.12:0/2409209808 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f9394030c00 0x7f93940330b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f93980030a0 tx=0x7f9398015000 comp rx=0 tx=0).stop 2023-10-05T19:42:15.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.832+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56462babad10 msgr2=0x56462bae3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:15.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.832+0000 7f93afc5ee40 1 --2- 172.21.15.12:0/2409209808 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56462babad10 0x56462bae3d90 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f93a0000f80 tx=0x7f93a0014000 comp rx=0 tx=0).stop 2023-10-05T19:42:15.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.832+0000 7f93adf94700 1 -- 172.21.15.12:0/2409209808 reap_dead start 2023-10-05T19:42:15.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.832+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 shutdown_connections 2023-10-05T19:42:15.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.832+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 >> 172.21.15.12:0/2409209808 conn(0x56462bab6730 msgr2=0x56462bad5430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:15.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.836+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 shutdown_connections 2023-10-05T19:42:15.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.836+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2409209808 wait complete. 2023-10-05T19:42:15.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.836+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x56462bb23a50 msgr2=0x56462bb25e60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:15.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.836+0000 7f93afc5ee40 1 --2- 172.21.15.12:0/2551009071 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x56462bb23a50 0x56462bb25e60 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:15.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.836+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x56462bb20380 msgr2=0x56462bb22760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:15.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.836+0000 7f93afc5ee40 1 --2- 172.21.15.12:0/2551009071 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x56462bb20380 0x56462bb22760 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:15.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.836+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x56462bb2b060 msgr2=0x56462bb2d440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:15.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.836+0000 7f93afc5ee40 1 --2- 172.21.15.12:0/2551009071 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x56462bb2b060 0x56462bb2d440 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:15.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.836+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x56462bb27260 msgr2=0x56462bb29640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:15.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.836+0000 7f93afc5ee40 1 --2- 172.21.15.12:0/2551009071 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x56462bb27260 0x56462bb29640 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:15.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.836+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f92ec030c50 msgr2=0x7f92ec033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:15.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.836+0000 7f93afc5ee40 1 --2- 172.21.15.12:0/2551009071 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f92ec030c50 0x7f92ec033100 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x56462bb02960 tx=0x7f93a800ab60 comp rx=0 tx=0).stop 2023-10-05T19:42:15.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.836+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56462bb0fc60 msgr2=0x56462bb15790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:15.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.836+0000 7f93afc5ee40 1 --2- 172.21.15.12:0/2551009071 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56462bb0fc60 0x56462bb15790 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x56462bb09ce0 tx=0x7f9398034000 comp rx=0 tx=0).stop 2023-10-05T19:42:15.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.836+0000 7f93ad793700 1 -- 172.21.15.12:0/2551009071 reap_dead start 2023-10-05T19:42:15.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.836+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 shutdown_connections 2023-10-05T19:42:15.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.836+0000 7f93afc5ee40 1 --2- 172.21.15.12:0/2551009071 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56462bb0fc60 0x56462bb15790 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:15.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.836+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 >> 172.21.15.12:0/2551009071 conn(0x56462bb0b020 msgr2=0x56462bb090c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:15.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.836+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 shutdown_connections 2023-10-05T19:42:15.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.836+0000 7f93afc5ee40 1 -- 172.21.15.12:0/2551009071 wait complete. 2023-10-05T19:42:15.847 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2023-10-05T19:42:15.847 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2023-10-05T19:42:15.847 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2023-10-05T19:42:15.883 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:42:15.883 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:42:15.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.880+0000 7f4dc3409e40 1 Processor -- start 2023-10-05T19:42:15.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.880+0000 7f4dc3409e40 1 -- start start 2023-10-05T19:42:15.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.880+0000 7f4dc3409e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x5566b21be420 0x5566b21c0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:15.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.880+0000 7f4dc3409e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5566b2106bd0 con 0x5566b212d3c0 2023-10-05T19:42:15.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.880+0000 7f4dc3409e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x5566b20c3ee0 con 0x5566b21be420 2023-10-05T19:42:15.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.880+0000 7f4dc073d700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x5566b21be420 0x5566b21c0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:15.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.880+0000 7f4dc073d700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x5566b21be420 0x5566b21c0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:54294/0 (socket says 172.21.15.12:54294) 2023-10-05T19:42:15.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.880+0000 7f4dc0f3e700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x5566b212d3c0 0x5566b212d790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:47352/0 (socket says 172.21.15.12:47352) 2023-10-05T19:42:15.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.880+0000 7f4dc073d700 1 -- 172.21.15.12:0/3557251373 learned_addr learned my addr 172.21.15.12:0/3557251373 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:42:15.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.880+0000 7f4dc073d700 1 -- 172.21.15.12:0/3557251373 >> v1:172.21.15.8:6789/0 conn(0x5566b212d3c0 legacy=0x5566b212d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:15.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.880+0000 7f4dbff3c700 1 -- 172.21.15.12:0/3557251373 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 341386086 0 0) 0x5566b2106bd0 con 0x5566b212d3c0 2023-10-05T19:42:15.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.880+0000 7f4dc073d700 1 -- 172.21.15.12:0/3557251373 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566b212df00 con 0x5566b21be420 2023-10-05T19:42:15.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.880+0000 7f4dc073d700 1 --2- 172.21.15.12:0/3557251373 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5566b21be420 0x5566b21c0800 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4da800a650 tx=0x7f4da8005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62d4ec90069bbf3f server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:15.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.880+0000 7f4dbff3c700 1 -- 172.21.15.12:0/3557251373 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4da8008760 con 0x5566b21be420 2023-10-05T19:42:15.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.880+0000 7f4dbff3c700 1 -- 172.21.15.12:0/3557251373 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4da80088c0 con 0x5566b21be420 2023-10-05T19:42:15.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.880+0000 7f4dbff3c700 1 -- 172.21.15.12:0/3557251373 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4da8008bb0 con 0x5566b21be420 2023-10-05T19:42:15.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.884+0000 7f4dc3409e40 1 -- 172.21.15.12:0/3557251373 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5566b21be420 msgr2=0x5566b21c0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:15.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.884+0000 7f4dc3409e40 1 --2- 172.21.15.12:0/3557251373 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5566b21be420 0x5566b21c0800 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4da800a650 tx=0x7f4da8005b10 comp rx=0 tx=0).stop 2023-10-05T19:42:15.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.884+0000 7f4dc3409e40 1 -- 172.21.15.12:0/3557251373 shutdown_connections 2023-10-05T19:42:15.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.884+0000 7f4dc3409e40 1 --2- 172.21.15.12:0/3557251373 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5566b21be420 0x5566b21c0800 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:15.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.884+0000 7f4dc3409e40 1 -- 172.21.15.12:0/3557251373 >> 172.21.15.12:0/3557251373 conn(0x5566b21b4740 msgr2=0x5566b21b6b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:15.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.884+0000 7f4dc3409e40 1 -- 172.21.15.12:0/3557251373 shutdown_connections 2023-10-05T19:42:15.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.884+0000 7f4dc3409e40 1 -- 172.21.15.12:0/3557251373 wait complete. 2023-10-05T19:42:15.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.884+0000 7f4dc3409e40 1 Processor -- start 2023-10-05T19:42:15.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.884+0000 7f4dc3409e40 1 -- start start 2023-10-05T19:42:15.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.884+0000 7f4dc3409e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5566b212d3c0 0x5566b21ba1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:15.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.884+0000 7f4dc3409e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566b2087e30 con 0x5566b212d3c0 2023-10-05T19:42:15.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.884+0000 7f4dc0f3e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5566b212d3c0 0x5566b21ba1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:15.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.884+0000 7f4dc0f3e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5566b212d3c0 0x5566b21ba1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:54310/0 (socket says 172.21.15.12:54310) 2023-10-05T19:42:15.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.884+0000 7f4dc0f3e700 1 -- 172.21.15.12:0/2524483454 learned_addr learned my addr 172.21.15.12:0/2524483454 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:42:15.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.884+0000 7f4dc0f3e700 1 -- 172.21.15.12:0/2524483454 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da8005db0 con 0x5566b212d3c0 2023-10-05T19:42:15.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.884+0000 7f4dc0f3e700 1 --2- 172.21.15.12:0/2524483454 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5566b212d3c0 0x5566b21ba1d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f4db8001630 tx=0x7f4db80069b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9d5565622dca870 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:15.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.884+0000 7f4dbef3a700 1 -- 172.21.15.12:0/2524483454 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4db8006320 con 0x5566b212d3c0 2023-10-05T19:42:15.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.884+0000 7f4dbef3a700 1 -- 172.21.15.12:0/2524483454 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4db8013420 con 0x5566b212d3c0 2023-10-05T19:42:15.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.884+0000 7f4dbef3a700 1 -- 172.21.15.12:0/2524483454 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4db80135d0 con 0x5566b212d3c0 2023-10-05T19:42:15.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.884+0000 7f4dc3409e40 1 -- 172.21.15.12:0/2524483454 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5566b212d3c0 msgr2=0x5566b21ba1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:15.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.884+0000 7f4dc3409e40 1 --2- 172.21.15.12:0/2524483454 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5566b212d3c0 0x5566b21ba1d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f4db8001630 tx=0x7f4db80069b0 comp rx=0 tx=0).stop 2023-10-05T19:42:15.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.888+0000 7f4dc3409e40 1 -- 172.21.15.12:0/2524483454 shutdown_connections 2023-10-05T19:42:15.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.888+0000 7f4dc3409e40 1 --2- 172.21.15.12:0/2524483454 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5566b212d3c0 0x5566b21ba1d0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:15.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.888+0000 7f4dc3409e40 1 -- 172.21.15.12:0/2524483454 >> 172.21.15.12:0/2524483454 conn(0x5566b21b4740 msgr2=0x5566b21d48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:15.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.888+0000 7f4dc3409e40 1 -- 172.21.15.12:0/2524483454 shutdown_connections 2023-10-05T19:42:15.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.888+0000 7f4dc3409e40 1 -- 172.21.15.12:0/2524483454 wait complete. 2023-10-05T19:42:15.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.888+0000 7f4dc3409e40 1 Processor -- start 2023-10-05T19:42:15.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.888+0000 7f4dc3409e40 1 -- start start 2023-10-05T19:42:15.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.888+0000 7f4dc3409e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5566b212d3c0 0x5566b21dc260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:15.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.888+0000 7f4dc3409e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566b205aed0 con 0x5566b212d3c0 2023-10-05T19:42:15.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.888+0000 7f4dc0f3e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5566b212d3c0 0x5566b21dc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:15.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.888+0000 7f4dc0f3e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5566b212d3c0 0x5566b21dc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:54314/0 (socket says 172.21.15.12:54314) 2023-10-05T19:42:15.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.888+0000 7f4dc0f3e700 1 -- 172.21.15.12:0/267339602 learned_addr learned my addr 172.21.15.12:0/267339602 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:42:15.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.888+0000 7f4dc0f3e700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db8006c90 con 0x5566b212d3c0 2023-10-05T19:42:15.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.888+0000 7f4dc0f3e700 1 --2- 172.21.15.12:0/267339602 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5566b212d3c0 0x5566b21dc260 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4db8001330 tx=0x7f4db8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:15.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.888+0000 7f4dbd737700 1 -- 172.21.15.12:0/267339602 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4db8013590 con 0x5566b212d3c0 2023-10-05T19:42:15.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.888+0000 7f4dbd737700 1 -- 172.21.15.12:0/267339602 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4db8006320 con 0x5566b212d3c0 2023-10-05T19:42:15.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.888+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5566b21dc7a0 con 0x5566b212d3c0 2023-10-05T19:42:15.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.888+0000 7f4dbd737700 1 -- 172.21.15.12:0/267339602 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4db8004d20 con 0x5566b212d3c0 2023-10-05T19:42:15.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.888+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5566b21dcbe0 con 0x5566b212d3c0 2023-10-05T19:42:15.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.892+0000 7f4dbd737700 1 -- 172.21.15.12:0/267339602 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f4db8028460 con 0x5566b212d3c0 2023-10-05T19:42:15.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.892+0000 7f4dbd737700 1 --2- 172.21.15.12:0/267339602 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f4da0030c00 0x7f4da00330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:15.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.892+0000 7f4dc3409e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.892+0000 7f4dbd737700 1 -- 172.21.15.12:0/267339602 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7f4db80453d0 con 0x5566b212d3c0 2023-10-05T19:42:15.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.892+0000 7f4dc073d700 1 --2- 172.21.15.12:0/267339602 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f4da0030c00 0x7f4da00330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:15.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.892+0000 7f4dc3409e40 1 --2- 172.21.15.12:0/267339602 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5566b21be420 0x5566b21f1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:15.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.892+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21f2260 con 0x5566b21be420 2023-10-05T19:42:15.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.892+0000 7f4dc173f700 1 --2- 172.21.15.12:0/267339602 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5566b21be420 0x5566b21f1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:15.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.892+0000 7f4dc073d700 1 --2- 172.21.15.12:0/267339602 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f4da0030c00 0x7f4da00330b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4da80030a0 tx=0x7f4da8015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:15.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.892+0000 7f4dc173f700 1 --2- 172.21.15.12:0/267339602 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5566b21be420 0x5566b21f1bb0 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:15.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.892+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4db000a500 con 0x5566b21be420 2023-10-05T19:42:15.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.892+0000 7f4dc3409e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:15.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.892+0000 7f4dc3409e40 20 realm 2023-10-05T19:42:15.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.892+0000 7f4dc3409e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.892+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21ef1d0 con 0x5566b21be420 2023-10-05T19:42:15.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.892+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4db000a500 con 0x5566b21be420 2023-10-05T19:42:15.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.892+0000 7f4dc3409e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:15.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.892+0000 7f4dc3409e40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:42:15.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.892+0000 7f4dc3409e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.892+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21f2d40 con 0x5566b21be420 2023-10-05T19:42:15.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.896+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4db000a500 con 0x5566b21be420 2023-10-05T19:42:15.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.896+0000 7f4dc3409e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:15.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.896+0000 7f4dc3409e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.896+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21f28f0 con 0x5566b21be420 2023-10-05T19:42:15.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.896+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f4db000a500 con 0x5566b21be420 2023-10-05T19:42:15.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.896+0000 7f4dc3409e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:42:15.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.896+0000 7f4dc3409e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.896+0000 7f4dc3409e40 1 --2- 172.21.15.12:0/267339602 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5566b21f3c90 0x5566b21f6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:15.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.896+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21f67b0 con 0x5566b21f3c90 2023-10-05T19:42:15.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.896+0000 7f4dc0f3e700 1 --2- 172.21.15.12:0/267339602 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5566b21f3c90 0x5566b21f6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:15.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.896+0000 7f4dc0f3e700 1 --2- 172.21.15.12:0/267339602 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5566b21f3c90 0x5566b21f6070 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:15.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.896+0000 7f4dc0f3e700 1 -- 172.21.15.12:0/267339602 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f4db8024020 con 0x5566b21f3c90 2023-10-05T19:42:15.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.896+0000 7f4dc3409e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:42:15.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.896+0000 7f4dc3409e40 20 searching for the correct realm 2023-10-05T19:42:15.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.896+0000 7f4dc3409e40 1 --2- 172.21.15.12:0/267339602 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5566b21f85e0 0x5566b21fa9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:15.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.896+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21faf00 con 0x5566b21f85e0 2023-10-05T19:42:15.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.896+0000 7f4dc073d700 1 --2- 172.21.15.12:0/267339602 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5566b21f85e0 0x5566b21fa9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:15.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.896+0000 7f4dc073d700 1 --2- 172.21.15.12:0/267339602 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5566b21f85e0 0x5566b21fa9c0 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:15.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.900+0000 7f4dc073d700 1 -- 172.21.15.12:0/267339602 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f4da8012450 con 0x5566b21f85e0 2023-10-05T19:42:15.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.900+0000 7f4dc073d700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21faf00 con 0x5566b21f3c90 2023-10-05T19:42:15.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.900+0000 7f4dc0f3e700 1 -- 172.21.15.12:0/267339602 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4db8024020 con 0x5566b21f3c90 2023-10-05T19:42:15.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.900+0000 7f4dc0f3e700 1 --2- 172.21.15.12:0/267339602 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f4db80123c0 0x7f4db8042440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:15.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.900+0000 7f4dc0f3e700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21faf00 con 0x7f4db80123c0 2023-10-05T19:42:15.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.900+0000 7f4dc173f700 1 --2- 172.21.15.12:0/267339602 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f4db80123c0 0x7f4db8042440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:15.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.900+0000 7f4dc173f700 1 --2- 172.21.15.12:0/267339602 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f4db80123c0 0x7f4db8042440 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:15.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.900+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4db0014040 con 0x7f4db80123c0 2023-10-05T19:42:15.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.900+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21faf00 con 0x5566b21be420 2023-10-05T19:42:15.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.904+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4db0014040 con 0x5566b21be420 2023-10-05T19:42:15.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.904+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21faf00 con 0x5566b21be420 2023-10-05T19:42:15.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.904+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4db0014040 con 0x5566b21be420 2023-10-05T19:42:15.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.904+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21faf00 con 0x5566b21be420 2023-10-05T19:42:15.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.904+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(11 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4db0014040 con 0x5566b21be420 2023-10-05T19:42:15.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.904+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21faf00 con 0x7f4db80123c0 2023-10-05T19:42:15.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.904+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4db0014040 con 0x7f4db80123c0 2023-10-05T19:42:15.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.904+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21faf00 con 0x5566b21f3c90 2023-10-05T19:42:15.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.904+0000 7f4dc0f3e700 1 -- 172.21.15.12:0/267339602 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4db8024020 con 0x5566b21f3c90 2023-10-05T19:42:15.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.904+0000 7f4dc0f3e700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21faf00 con 0x5566b21f3c90 2023-10-05T19:42:15.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.904+0000 7f4dc0f3e700 1 -- 172.21.15.12:0/267339602 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4db8024020 con 0x5566b21f3c90 2023-10-05T19:42:15.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.904+0000 7f4dc0f3e700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21faf00 con 0x5566b21be420 2023-10-05T19:42:15.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.908+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4db0014040 con 0x5566b21be420 2023-10-05T19:42:15.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.908+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21faf00 con 0x7f4db80123c0 2023-10-05T19:42:15.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.908+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4db0014040 con 0x7f4db80123c0 2023-10-05T19:42:15.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.908+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21faf00 con 0x5566b21be420 2023-10-05T19:42:15.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.908+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4db0014040 con 0x5566b21be420 2023-10-05T19:42:15.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.908+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21faf00 con 0x5566b21f3c90 2023-10-05T19:42:15.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.908+0000 7f4dc0f3e700 1 -- 172.21.15.12:0/267339602 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(18 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4db8024020 con 0x5566b21f3c90 2023-10-05T19:42:15.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.908+0000 7f4dc0f3e700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21faf00 con 0x5566b21f85e0 2023-10-05T19:42:15.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.908+0000 7f4dc073d700 1 -- 172.21.15.12:0/267339602 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4da8012450 con 0x5566b21f85e0 2023-10-05T19:42:15.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.908+0000 7f4dc073d700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21faf00 con 0x5566b21f85e0 2023-10-05T19:42:15.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.912+0000 7f4dc073d700 1 -- 172.21.15.12:0/267339602 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4da8012450 con 0x5566b21f85e0 2023-10-05T19:42:15.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.912+0000 7f4dc073d700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21faf00 con 0x7f4db80123c0 2023-10-05T19:42:15.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.912+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4db0014040 con 0x7f4db80123c0 2023-10-05T19:42:15.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.912+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21faf00 con 0x5566b21f85e0 2023-10-05T19:42:15.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.912+0000 7f4dc073d700 1 -- 172.21.15.12:0/267339602 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4da8012450 con 0x5566b21f85e0 2023-10-05T19:42:15.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.912+0000 7f4dc073d700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21faf00 con 0x5566b21f85e0 2023-10-05T19:42:15.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.912+0000 7f4dc073d700 1 -- 172.21.15.12:0/267339602 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4da8012450 con 0x5566b21f85e0 2023-10-05T19:42:15.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.912+0000 7f4dc073d700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21faf00 con 0x5566b21be420 2023-10-05T19:42:15.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.912+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4db0014040 con 0x5566b21be420 2023-10-05T19:42:15.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.912+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21faf00 con 0x5566b21be420 2023-10-05T19:42:15.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.916+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4db0014040 con 0x5566b21be420 2023-10-05T19:42:15.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.916+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21faf00 con 0x7f4db80123c0 2023-10-05T19:42:15.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.916+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4db0014040 con 0x7f4db80123c0 2023-10-05T19:42:15.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.916+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21faf00 con 0x5566b21f85e0 2023-10-05T19:42:15.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.916+0000 7f4dc073d700 1 -- 172.21.15.12:0/267339602 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4da8012450 con 0x5566b21f85e0 2023-10-05T19:42:15.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.916+0000 7f4dc073d700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21faf00 con 0x5566b21f3c90 2023-10-05T19:42:15.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.916+0000 7f4dc0f3e700 1 -- 172.21.15.12:0/267339602 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4db8024020 con 0x5566b21f3c90 2023-10-05T19:42:15.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.916+0000 7f4dc0f3e700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21faf00 con 0x5566b21be420 2023-10-05T19:42:15.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.916+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4db0014040 con 0x5566b21be420 2023-10-05T19:42:15.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.916+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21faf00 con 0x7f4db80123c0 2023-10-05T19:42:15.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.920+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4db0014040 con 0x7f4db80123c0 2023-10-05T19:42:15.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.920+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21faf00 con 0x5566b21f85e0 2023-10-05T19:42:15.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.920+0000 7f4dc073d700 1 -- 172.21.15.12:0/267339602 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4da8012450 con 0x5566b21f85e0 2023-10-05T19:42:15.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.920+0000 7f4dc073d700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21faf00 con 0x7f4db80123c0 2023-10-05T19:42:15.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.920+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4db0014040 con 0x7f4db80123c0 2023-10-05T19:42:15.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.920+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21faf00 con 0x5566b21f3c90 2023-10-05T19:42:15.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.920+0000 7f4dc0f3e700 1 -- 172.21.15.12:0/267339602 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4db8024020 con 0x5566b21f3c90 2023-10-05T19:42:15.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.920+0000 7f4dc0f3e700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21faf00 con 0x5566b21f85e0 2023-10-05T19:42:15.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.920+0000 7f4dc073d700 1 -- 172.21.15.12:0/267339602 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4da8012450 con 0x5566b21f85e0 2023-10-05T19:42:15.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.920+0000 7f4dc073d700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21faf00 con 0x7f4db80123c0 2023-10-05T19:42:15.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.924+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4db0014040 con 0x7f4db80123c0 2023-10-05T19:42:15.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.924+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21faf00 con 0x5566b21be420 2023-10-05T19:42:15.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.924+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f4db0014040 con 0x5566b21be420 2023-10-05T19:42:15.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.924+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21faf00 con 0x5566b21f3c90 2023-10-05T19:42:15.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.924+0000 7f4dc0f3e700 1 -- 172.21.15.12:0/267339602 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(37 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f4db8024020 con 0x5566b21f3c90 2023-10-05T19:42:15.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.924+0000 7f4dc3409e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:42:15.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.924+0000 7f4dc3409e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:42:15.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.924+0000 7f4dc3409e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:42:15.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.924+0000 7f4dc3409e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:42:15.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.924+0000 7f4dc3409e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.924+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21f73e0 con 0x5566b21be420 2023-10-05T19:42:15.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.924+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4db0014040 con 0x5566b21be420 2023-10-05T19:42:15.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.924+0000 7f4dc3409e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:15.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.924+0000 7f4dc3409e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.924+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21f6f50 con 0x5566b21be420 2023-10-05T19:42:15.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.928+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f4db0014040 con 0x5566b21be420 2023-10-05T19:42:15.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.928+0000 7f4dc3409e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:42:15.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.928+0000 7f4dc3409e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.928+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21f7d00 con 0x5566b21f85e0 2023-10-05T19:42:15.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.928+0000 7f4dc073d700 1 -- 172.21.15.12:0/267339602 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f4da8012450 con 0x5566b21f85e0 2023-10-05T19:42:15.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.928+0000 7f4dc3409e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:42:15.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.928+0000 7f4dc3409e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.928+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21fb830 con 0x5566b21f85e0 2023-10-05T19:42:15.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.928+0000 7f4dc073d700 1 -- 172.21.15.12:0/267339602 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7f4da8012450 con 0x5566b21f85e0 2023-10-05T19:42:15.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.928+0000 7f4dc3409e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:15.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.928+0000 7f4dc3409e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.928+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21fbbe0 con 0x5566b21be420 2023-10-05T19:42:15.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.928+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4db0014040 con 0x5566b21be420 2023-10-05T19:42:15.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.928+0000 7f4dc3409e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:15.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.928+0000 7f4dc3409e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.928+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21fbf90 con 0x5566b21be420 2023-10-05T19:42:15.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.928+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f4db0014040 con 0x5566b21be420 2023-10-05T19:42:15.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.928+0000 7f4dc3409e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:42:15.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.928+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21fcfa0 con 0x5566b21f85e0 2023-10-05T19:42:15.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.932+0000 7f4dc073d700 1 -- 172.21.15.12:0/267339602 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(44 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f4da8012450 con 0x5566b21f85e0 2023-10-05T19:42:15.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.932+0000 7f4dc073d700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21fcfa0 con 0x5566b21f3c90 2023-10-05T19:42:15.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.932+0000 7f4dc0f3e700 1 -- 172.21.15.12:0/267339602 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(45 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4db8024020 con 0x5566b21f3c90 2023-10-05T19:42:15.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.932+0000 7f4dc0f3e700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21fcfa0 con 0x7f4db80123c0 2023-10-05T19:42:15.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.932+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(46 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4db0014040 con 0x7f4db80123c0 2023-10-05T19:42:15.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.932+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21fbbe0 con 0x5566b21be420 2023-10-05T19:42:15.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.936+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(47 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4db0014040 con 0x5566b21be420 2023-10-05T19:42:15.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.936+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21fbbe0 con 0x5566b21be420 2023-10-05T19:42:15.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.936+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(48 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4db0014040 con 0x5566b21be420 2023-10-05T19:42:15.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.936+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21fbbe0 con 0x5566b21be420 2023-10-05T19:42:15.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.936+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(49 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4db0014040 con 0x5566b21be420 2023-10-05T19:42:15.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.936+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21fbbe0 con 0x7f4db80123c0 2023-10-05T19:42:15.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.936+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(50 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4db0014040 con 0x7f4db80123c0 2023-10-05T19:42:15.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.936+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21fbbe0 con 0x5566b21f3c90 2023-10-05T19:42:15.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.940+0000 7f4dc0f3e700 1 -- 172.21.15.12:0/267339602 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(51 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4db8024020 con 0x5566b21f3c90 2023-10-05T19:42:15.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.940+0000 7f4dc0f3e700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21fbbe0 con 0x5566b21f3c90 2023-10-05T19:42:15.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.940+0000 7f4dc0f3e700 1 -- 172.21.15.12:0/267339602 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(52 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4db8024020 con 0x5566b21f3c90 2023-10-05T19:42:15.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.940+0000 7f4dc0f3e700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21fbbe0 con 0x5566b21be420 2023-10-05T19:42:15.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.940+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(53 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4db0014040 con 0x5566b21be420 2023-10-05T19:42:15.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.940+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21fbbe0 con 0x7f4db80123c0 2023-10-05T19:42:15.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.940+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(54 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4db0014040 con 0x7f4db80123c0 2023-10-05T19:42:15.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.940+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21fbbe0 con 0x5566b21be420 2023-10-05T19:42:15.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.944+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(55 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4db0014040 con 0x5566b21be420 2023-10-05T19:42:15.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.944+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21fbbe0 con 0x5566b21f3c90 2023-10-05T19:42:15.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.944+0000 7f4dc0f3e700 1 -- 172.21.15.12:0/267339602 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(56 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4db8024020 con 0x5566b21f3c90 2023-10-05T19:42:15.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.944+0000 7f4dc0f3e700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21fbbe0 con 0x5566b21f85e0 2023-10-05T19:42:15.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.944+0000 7f4dc073d700 1 -- 172.21.15.12:0/267339602 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(57 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4da8012450 con 0x5566b21f85e0 2023-10-05T19:42:15.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.944+0000 7f4dc073d700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21fbbe0 con 0x5566b21f85e0 2023-10-05T19:42:15.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.944+0000 7f4dc073d700 1 -- 172.21.15.12:0/267339602 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(58 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4da8012450 con 0x5566b21f85e0 2023-10-05T19:42:15.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.944+0000 7f4dc073d700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21fbbe0 con 0x7f4db80123c0 2023-10-05T19:42:15.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.948+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(59 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4db0014040 con 0x7f4db80123c0 2023-10-05T19:42:15.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.948+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21fbbe0 con 0x5566b21f85e0 2023-10-05T19:42:15.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.948+0000 7f4dc073d700 1 -- 172.21.15.12:0/267339602 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(60 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4da8012450 con 0x5566b21f85e0 2023-10-05T19:42:15.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.948+0000 7f4dc073d700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21fbbe0 con 0x5566b21f85e0 2023-10-05T19:42:15.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.948+0000 7f4dc073d700 1 -- 172.21.15.12:0/267339602 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(61 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4da8012450 con 0x5566b21f85e0 2023-10-05T19:42:15.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.948+0000 7f4dc073d700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21fbbe0 con 0x5566b21be420 2023-10-05T19:42:15.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.948+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(62 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4db0014040 con 0x5566b21be420 2023-10-05T19:42:15.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.948+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21fbbe0 con 0x5566b21be420 2023-10-05T19:42:15.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.948+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(63 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4db0014040 con 0x5566b21be420 2023-10-05T19:42:15.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.948+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21fbbe0 con 0x7f4db80123c0 2023-10-05T19:42:15.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.952+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(64 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4db0014040 con 0x7f4db80123c0 2023-10-05T19:42:15.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.952+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21fbbe0 con 0x5566b21f85e0 2023-10-05T19:42:15.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.952+0000 7f4dc073d700 1 -- 172.21.15.12:0/267339602 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(65 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4da8012450 con 0x5566b21f85e0 2023-10-05T19:42:15.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.952+0000 7f4dc073d700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21fbbe0 con 0x5566b21f3c90 2023-10-05T19:42:15.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.952+0000 7f4dc0f3e700 1 -- 172.21.15.12:0/267339602 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(66 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4db8024020 con 0x5566b21f3c90 2023-10-05T19:42:15.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.952+0000 7f4dc0f3e700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21fbbe0 con 0x5566b21be420 2023-10-05T19:42:15.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.952+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(67 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4db0014040 con 0x5566b21be420 2023-10-05T19:42:15.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.952+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21fbbe0 con 0x7f4db80123c0 2023-10-05T19:42:15.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.956+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(68 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4db0014040 con 0x7f4db80123c0 2023-10-05T19:42:15.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.956+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21fbbe0 con 0x5566b21f85e0 2023-10-05T19:42:15.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.956+0000 7f4dc073d700 1 -- 172.21.15.12:0/267339602 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(69 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4da8012450 con 0x5566b21f85e0 2023-10-05T19:42:15.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.956+0000 7f4dc073d700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21fbbe0 con 0x7f4db80123c0 2023-10-05T19:42:15.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.956+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(70 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4db0014040 con 0x7f4db80123c0 2023-10-05T19:42:15.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.956+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21fbbe0 con 0x5566b21f3c90 2023-10-05T19:42:15.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.956+0000 7f4dc0f3e700 1 -- 172.21.15.12:0/267339602 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(71 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4db8024020 con 0x5566b21f3c90 2023-10-05T19:42:15.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.956+0000 7f4dc0f3e700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21fbbe0 con 0x5566b21f85e0 2023-10-05T19:42:15.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.960+0000 7f4dc073d700 1 -- 172.21.15.12:0/267339602 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(72 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4da8012450 con 0x5566b21f85e0 2023-10-05T19:42:15.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.960+0000 7f4dc073d700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21fbbe0 con 0x7f4db80123c0 2023-10-05T19:42:15.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.960+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(73 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4db0014040 con 0x7f4db80123c0 2023-10-05T19:42:15.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.960+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21fbbe0 con 0x5566b21be420 2023-10-05T19:42:15.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.960+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(74 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f4db0014040 con 0x5566b21be420 2023-10-05T19:42:15.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.960+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21fbbe0 con 0x5566b21f3c90 2023-10-05T19:42:15.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.960+0000 7f4dc0f3e700 1 -- 172.21.15.12:0/267339602 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(75 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f4db8024020 con 0x5566b21f3c90 2023-10-05T19:42:15.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.960+0000 7f4dc3409e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:42:15.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.960+0000 7f4dc3409e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:42:15.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.960+0000 7f4dc3409e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:42:15.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.960+0000 7f4dc3409e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:42:15.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.960+0000 7f4dc3409e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.960+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21fbf90 con 0x5566b21be420 2023-10-05T19:42:15.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.960+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f4db0014040 con 0x5566b21be420 2023-10-05T19:42:15.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.964+0000 7f4dc3409e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:42:15.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.964+0000 7f4dc3409e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.964+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21fc340 con 0x5566b21f3c90 2023-10-05T19:42:15.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.964+0000 7f4dc0f3e700 1 -- 172.21.15.12:0/267339602 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f4db8024020 con 0x5566b21f3c90 2023-10-05T19:42:15.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.964+0000 7f4dc3409e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:42:15.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.964+0000 7f4dc3409e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.964+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21fcbe0 con 0x5566b21be420 2023-10-05T19:42:15.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.964+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f4db0014040 con 0x5566b21be420 2023-10-05T19:42:15.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.964+0000 7f4dc3409e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:42:15.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.964+0000 7f4dc3409e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.964+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21fcbe0 con 0x5566b21f85e0 2023-10-05T19:42:15.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.964+0000 7f4dc073d700 1 -- 172.21.15.12:0/267339602 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f4da8012450 con 0x5566b21f85e0 2023-10-05T19:42:15.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.964+0000 7f4dc3409e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:42:15.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.964+0000 7f4dc3409e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.964+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21fd480 con 0x7f4db80123c0 2023-10-05T19:42:15.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.964+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7f4db0014040 con 0x7f4db80123c0 2023-10-05T19:42:15.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.964+0000 7f4dc3409e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:15.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.964+0000 7f4dc3409e40 20 zone default found 2023-10-05T19:42:15.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.964+0000 7f4dc3409e40 4 Realm: () 2023-10-05T19:42:15.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.964+0000 7f4dc3409e40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:42:15.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.964+0000 7f4dc3409e40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:42:15.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.964+0000 7f4dc3409e40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:42:15.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.964+0000 7f4dc3409e40 20 zonegroup default 2023-10-05T19:42:15.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.964+0000 7f4dc3409e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.964+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21fd480 con 0x7f4db80123c0 2023-10-05T19:42:15.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.968+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f4db0014040 con 0x7f4db80123c0 2023-10-05T19:42:15.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.968+0000 7f4dc3409e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:15.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.968+0000 7f4dc3409e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.968+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21fee00 con 0x5566b21f85e0 2023-10-05T19:42:15.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.968+0000 7f4dc073d700 1 -- 172.21.15.12:0/267339602 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f4da8012450 con 0x5566b21f85e0 2023-10-05T19:42:15.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.968+0000 7f4dc3409e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:15.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.968+0000 7f4dc3409e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:15.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.968+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21ff1b0 con 0x5566b21f85e0 2023-10-05T19:42:15.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.968+0000 7f4dc073d700 1 -- 172.21.15.12:0/267339602 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f4da8012450 con 0x5566b21f85e0 2023-10-05T19:42:15.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.968+0000 7f4dc3409e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:15.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.968+0000 7f4dc3409e40 20 started sync module instance, tier type = 2023-10-05T19:42:15.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.968+0000 7f4dc3409e40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:42:15.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.968+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21fe4c0 con 0x5566b21f3c90 2023-10-05T19:42:15.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.972+0000 7f4dc0f3e700 1 -- 172.21.15.12:0/267339602 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(84 notify.0 [create] v30'267 uv267 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4db8024020 con 0x5566b21f3c90 2023-10-05T19:42:15.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.972+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93899563458512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21ff560 con 0x5566b21f3c90 2023-10-05T19:42:15.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.972+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21ffe00 con 0x5566b21be420 2023-10-05T19:42:15.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.976+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(86 notify.1 [create] v30'267 uv267 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4db0014040 con 0x5566b21be420 2023-10-05T19:42:15.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.976+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93899563464976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b2200310 con 0x5566b21be420 2023-10-05T19:42:15.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.976+0000 7f4dc0f3e700 1 -- 172.21.15.12:0/267339602 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(85 notify.0 [watch watch cookie 93899563458512] v30'268 uv267 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4db8024020 con 0x5566b21f3c90 2023-10-05T19:42:15.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.976+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b21fe8c0 con 0x5566b21f85e0 2023-10-05T19:42:15.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.980+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(87 notify.1 [watch watch cookie 93899563464976] v30'268 uv267 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4db0014040 con 0x5566b21be420 2023-10-05T19:42:15.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.980+0000 7f4dc073d700 1 -- 172.21.15.12:0/267339602 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(88 notify.2 [create] v30'267 uv267 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4da8012450 con 0x5566b21f85e0 2023-10-05T19:42:15.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.980+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93899563466144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b2200230 con 0x5566b21f85e0 2023-10-05T19:42:15.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.980+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b2201590 con 0x5566b21f3c90 2023-10-05T19:42:15.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.984+0000 7f4dc073d700 1 -- 172.21.15.12:0/267339602 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(89 notify.2 [watch watch cookie 93899563466144] v30'268 uv267 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4da8012450 con 0x5566b21f85e0 2023-10-05T19:42:15.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.984+0000 7f4dc0f3e700 1 -- 172.21.15.12:0/267339602 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(90 notify.3 [create] v30'267 uv267 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4db8024020 con 0x5566b21f3c90 2023-10-05T19:42:15.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.984+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 93899563469168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b2201180 con 0x5566b21f3c90 2023-10-05T19:42:15.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.984+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b2202550 con 0x5566b21be420 2023-10-05T19:42:15.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.984+0000 7f4dc0f3e700 1 -- 172.21.15.12:0/267339602 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(91 notify.3 [watch watch cookie 93899563469168] v30'268 uv267 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4db8024020 con 0x5566b21f3c90 2023-10-05T19:42:15.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.984+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(92 notify.4 [create] v30'267 uv267 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4db0014040 con 0x5566b21be420 2023-10-05T19:42:15.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.984+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 93899563473296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b2202510 con 0x5566b21be420 2023-10-05T19:42:15.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.984+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b2202fb0 con 0x7f4db80123c0 2023-10-05T19:42:15.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.988+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(93 notify.4 [watch watch cookie 93899563473296] v30'268 uv267 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4db0014040 con 0x5566b21be420 2023-10-05T19:42:15.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.988+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(94 notify.5 [create] v30'267 uv267 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4db0014040 con 0x7f4db80123c0 2023-10-05T19:42:15.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.988+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 93899563475536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b2202fb0 con 0x7f4db80123c0 2023-10-05T19:42:15.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.988+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b2202fb0 con 0x7f4db80123c0 2023-10-05T19:42:15.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.992+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(95 notify.5 [watch watch cookie 93899563475536] v30'268 uv267 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4db0014040 con 0x7f4db80123c0 2023-10-05T19:42:15.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.992+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(96 notify.6 [create] v30'267 uv267 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4db0014040 con 0x7f4db80123c0 2023-10-05T19:42:15.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.992+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93899563478256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b22036b0 con 0x7f4db80123c0 2023-10-05T19:42:15.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.992+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b22041a0 con 0x7f4db80123c0 2023-10-05T19:42:15.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.992+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(97 notify.6 [watch watch cookie 93899563478256] v30'268 uv267 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4db0014040 con 0x7f4db80123c0 2023-10-05T19:42:15.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.996+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(98 notify.7 [create] v30'267 uv267 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4db0014040 con 0x7f4db80123c0 2023-10-05T19:42:15.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.996+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93899563479920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b22041a0 con 0x7f4db80123c0 2023-10-05T19:42:15.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.996+0000 7f4dc3409e40 20 add_watcher() i=0 2023-10-05T19:42:15.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.996+0000 7f4dc3409e40 20 add_watcher() i=1 2023-10-05T19:42:16.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.996+0000 7f4dc3409e40 20 add_watcher() i=2 2023-10-05T19:42:16.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.996+0000 7f4dc3409e40 20 add_watcher() i=3 2023-10-05T19:42:16.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.996+0000 7f4dc3409e40 20 add_watcher() i=4 2023-10-05T19:42:16.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.996+0000 7f4dc3409e40 20 add_watcher() i=5 2023-10-05T19:42:16.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.996+0000 7f4dc3409e40 20 add_watcher() i=6 2023-10-05T19:42:16.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.996+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(99 notify.7 [watch watch cookie 93899563479920] v30'268 uv267 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4db0014040 con 0x7f4db80123c0 2023-10-05T19:42:16.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.996+0000 7f4dc3409e40 20 add_watcher() i=7 2023-10-05T19:42:16.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.996+0000 7f4dc3409e40 2 all 8 watchers are set, enabling cache 2023-10-05T19:42:16.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.996+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b22073e0 con 0x5566b21be420 2023-10-05T19:42:16.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.996+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f4db0014040 con 0x5566b21be420 2023-10-05T19:42:16.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:15.996+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93899563482656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b2204760 con 0x5566b21be420 2023-10-05T19:42:16.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.000+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 93899563482656] v30'258 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f4db0014040 con 0x5566b21be420 2023-10-05T19:42:16.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.000+0000 7f4d927e4700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:42:16.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.000+0000 7f4dc3409e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:42:16.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.000+0000 7f4dc3409e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:42:16.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.000+0000 7f4dc3409e40 1 Processor -- start 2023-10-05T19:42:16.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.000+0000 7f4dc3409e40 1 -- start start 2023-10-05T19:42:16.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.000+0000 7f4dc3409e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5566b2211840 0x5566b2211c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.000+0000 7f4dc3409e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566b2212150 con 0x5566b2211840 2023-10-05T19:42:16.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.000+0000 7f4dc073d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5566b2211840 0x5566b2211c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.000+0000 7f4dc073d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5566b2211840 0x5566b2211c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:54320/0 (socket says 172.21.15.12:54320) 2023-10-05T19:42:16.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.000+0000 7f4dc073d700 1 -- 172.21.15.12:0/4038850785 learned_addr learned my addr 172.21.15.12:0/4038850785 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:42:16.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.004+0000 7f4dc073d700 1 -- 172.21.15.12:0/4038850785 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566b2212290 con 0x5566b2211840 2023-10-05T19:42:16.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.004+0000 7f4dc073d700 1 --2- 172.21.15.12:0/4038850785 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5566b2211840 0x5566b2211c10 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x5566b212ded0 tx=0x7f4da8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64b8477169d52938 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.004+0000 7f4d91fe3700 1 -- 172.21.15.12:0/4038850785 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4da8030930 con 0x5566b2211840 2023-10-05T19:42:16.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.004+0000 7f4d91fe3700 1 -- 172.21.15.12:0/4038850785 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4da8030a90 con 0x5566b2211840 2023-10-05T19:42:16.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.004+0000 7f4d91fe3700 1 -- 172.21.15.12:0/4038850785 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4da8030c40 con 0x5566b2211840 2023-10-05T19:42:16.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.004+0000 7f4dc3409e40 1 -- 172.21.15.12:0/4038850785 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5566b2211840 msgr2=0x5566b2211c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.004+0000 7f4dc3409e40 1 --2- 172.21.15.12:0/4038850785 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5566b2211840 0x5566b2211c10 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x5566b212ded0 tx=0x7f4da8012000 comp rx=0 tx=0).stop 2023-10-05T19:42:16.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.004+0000 7f4dc3409e40 1 -- 172.21.15.12:0/4038850785 shutdown_connections 2023-10-05T19:42:16.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.004+0000 7f4dc3409e40 1 --2- 172.21.15.12:0/4038850785 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5566b2211840 0x5566b2211c10 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.004+0000 7f4dc3409e40 1 -- 172.21.15.12:0/4038850785 >> 172.21.15.12:0/4038850785 conn(0x5566b220cc00 msgr2=0x5566b220adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:16.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.004+0000 7f4dc3409e40 1 -- 172.21.15.12:0/4038850785 shutdown_connections 2023-10-05T19:42:16.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.004+0000 7f4dc3409e40 1 -- 172.21.15.12:0/4038850785 wait complete. 2023-10-05T19:42:16.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.004+0000 7f4dc3409e40 1 Processor -- start 2023-10-05T19:42:16.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.004+0000 7f4dc3409e40 1 -- start start 2023-10-05T19:42:16.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.004+0000 7f4dc3409e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5566b2211840 0x5566b22172f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.004+0000 7f4dc3409e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566b22182f0 con 0x5566b2211840 2023-10-05T19:42:16.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.004+0000 7f4dc073d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5566b2211840 0x5566b22172f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.004+0000 7f4dc073d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5566b2211840 0x5566b22172f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:54330/0 (socket says 172.21.15.12:54330) 2023-10-05T19:42:16.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.004+0000 7f4dc073d700 1 -- 172.21.15.12:0/925075629 learned_addr learned my addr 172.21.15.12:0/925075629 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:42:16.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.008+0000 7f4dc073d700 1 -- 172.21.15.12:0/925075629 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566b212df00 con 0x5566b2211840 2023-10-05T19:42:16.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.008+0000 7f4dc073d700 1 --2- 172.21.15.12:0/925075629 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5566b2211840 0x5566b22172f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x5566b220b920 tx=0x7f4da8032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.008+0000 7f4d03fff700 1 -- 172.21.15.12:0/925075629 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4da8028c80 con 0x5566b2211840 2023-10-05T19:42:16.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.008+0000 7f4d03fff700 1 -- 172.21.15.12:0/925075629 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4da8044070 con 0x5566b2211840 2023-10-05T19:42:16.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.008+0000 7f4dc3409e40 1 -- 172.21.15.12:0/925075629 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5566b22178f0 con 0x5566b2211840 2023-10-05T19:42:16.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.008+0000 7f4d03fff700 1 -- 172.21.15.12:0/925075629 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4da8028e40 con 0x5566b2211840 2023-10-05T19:42:16.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.008+0000 7f4dc3409e40 1 -- 172.21.15.12:0/925075629 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5566b2217cd0 con 0x5566b2211840 2023-10-05T19:42:16.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.008+0000 7f4d03fff700 1 -- 172.21.15.12:0/925075629 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f4da8036040 con 0x5566b2211840 2023-10-05T19:42:16.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.008+0000 7f4d03fff700 1 --2- 172.21.15.12:0/925075629 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f4d08030c00 0x7f4d080330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.008+0000 7f4d03fff700 1 -- 172.21.15.12:0/925075629 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7f4da8046070 con 0x5566b2211840 2023-10-05T19:42:16.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.008+0000 7f4dc3409e40 5 note: GC not initialized 2023-10-05T19:42:16.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.008+0000 7f4dc173f700 1 --2- 172.21.15.12:0/925075629 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f4d08030c00 0x7f4d080330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.012+0000 7f4dc173f700 1 --2- 172.21.15.12:0/925075629 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f4d08030c00 0x7f4d080330b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x5566b2203340 tx=0x7f4db00335d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.012+0000 7f4d017fa700 20 reqs_thread_entry: start 2023-10-05T19:42:16.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.012+0000 7f4cfaffd700 20 reqs_thread_entry: start 2023-10-05T19:42:16.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.012+0000 7f4dc3409e40 20 init_complete bucket index max shards: 11 2023-10-05T19:42:16.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.012+0000 7f4dc3409e40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:42:16.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.012+0000 7f4cf97fa700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:42:46 2023 2023-10-05T19:42:16.018 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:42:16.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.012+0000 7f4cf8ff9700 20 reqs_thread_entry: start 2023-10-05T19:42:16.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.012+0000 7f4dc3409e40 20 get_system_obj_state: rctx=0x7ffd4e80af38 obj=default.rgw.meta:users.uid:foo state=0x5566b20930b0 s->prefetch_data=0 2023-10-05T19:42:16.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.012+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:102 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b227fbf0 con 0x5566b21be420 2023-10-05T19:42:16.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.012+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(102 foo [call,getxattrs,stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x7f4db0014040 con 0x5566b21be420 2023-10-05T19:42:16.020 INFO:teuthology.orchestra.run.smithi012.stderr:could not fetch user info: no user info saved 2023-10-05T19:42:16.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.016+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:103 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93899563458512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b227fbf0 con 0x5566b21f3c90 2023-10-05T19:42:16.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.016+0000 7f4dc0f3e700 1 -- 172.21.15.12:0/267339602 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(103 notify.0 [watch unwatch cookie 93899563458512] v30'269 uv267 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4db8024020 con 0x5566b21f3c90 2023-10-05T19:42:16.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.016+0000 7f4dc3409e40 20 remove_watcher() i=0 2023-10-05T19:42:16.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.016+0000 7f4dc3409e40 2 removed watcher, disabling cache 2023-10-05T19:42:16.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.016+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:104 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93899563464976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b227fbf0 con 0x5566b21be420 2023-10-05T19:42:16.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.020+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== osd_op_reply(104 notify.1 [watch unwatch cookie 93899563464976] v30'269 uv267 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4db0014040 con 0x5566b21be420 2023-10-05T19:42:16.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.020+0000 7f4dc3409e40 20 remove_watcher() i=1 2023-10-05T19:42:16.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.020+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:105 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93899563466144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b227fbf0 con 0x5566b21f85e0 2023-10-05T19:42:16.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.024+0000 7f4dc073d700 1 -- 172.21.15.12:0/267339602 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== osd_op_reply(105 notify.2 [watch unwatch cookie 93899563466144] v30'269 uv267 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4da8012450 con 0x5566b21f85e0 2023-10-05T19:42:16.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.024+0000 7f4dc3409e40 20 remove_watcher() i=2 2023-10-05T19:42:16.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.024+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:106 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93899563469168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b227fbf0 con 0x5566b21f3c90 2023-10-05T19:42:16.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.028+0000 7f4dc0f3e700 1 -- 172.21.15.12:0/267339602 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(106 notify.3 [watch unwatch cookie 93899563469168] v30'269 uv267 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4db8024020 con 0x5566b21f3c90 2023-10-05T19:42:16.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.028+0000 7f4dc3409e40 20 remove_watcher() i=3 2023-10-05T19:42:16.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.028+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:107 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93899563473296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b227fbf0 con 0x5566b21be420 2023-10-05T19:42:16.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.028+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(107 notify.4 [watch unwatch cookie 93899563473296] v30'269 uv267 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4db0014040 con 0x5566b21be420 2023-10-05T19:42:16.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.028+0000 7f4dc3409e40 20 remove_watcher() i=4 2023-10-05T19:42:16.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.028+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:108 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93899563475536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b227fbf0 con 0x7f4db80123c0 2023-10-05T19:42:16.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.032+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(108 notify.5 [watch unwatch cookie 93899563475536] v30'269 uv267 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4db0014040 con 0x7f4db80123c0 2023-10-05T19:42:16.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.032+0000 7f4dc3409e40 20 remove_watcher() i=5 2023-10-05T19:42:16.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.032+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:109 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93899563478256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b227fbf0 con 0x7f4db80123c0 2023-10-05T19:42:16.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.036+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(109 notify.6 [watch unwatch cookie 93899563478256] v30'269 uv267 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4db0014040 con 0x7f4db80123c0 2023-10-05T19:42:16.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.036+0000 7f4dc3409e40 20 remove_watcher() i=6 2023-10-05T19:42:16.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.036+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:110 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93899563479920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b227fbf0 con 0x7f4db80123c0 2023-10-05T19:42:16.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.040+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(110 notify.7 [watch unwatch cookie 93899563479920] v30'269 uv267 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4db0014040 con 0x7f4db80123c0 2023-10-05T19:42:16.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.040+0000 7f4dc3409e40 20 remove_watcher() i=7 2023-10-05T19:42:16.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.040+0000 7f4d927e4700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:42:16.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.040+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:111 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93899563482656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x5566b223ef70 con 0x5566b21be420 2023-10-05T19:42:16.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.044+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== osd_op_reply(111 data_loggenerations_metadata [watch unwatch cookie 93899563482656] v30'259 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f4db0014040 con 0x5566b21be420 2023-10-05T19:42:16.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.044+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f4db80123c0 msgr2=0x7f4db8042440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.044+0000 7f4dc3409e40 1 --2- 172.21.15.12:0/267339602 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f4db80123c0 0x7f4db8042440 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.044+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5566b21be420 msgr2=0x5566b21f1bb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.044+0000 7f4dc3409e40 1 --2- 172.21.15.12:0/267339602 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x5566b21be420 0x5566b21f1bb0 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.044+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5566b21f3c90 msgr2=0x5566b21f6070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.044+0000 7f4dc3409e40 1 --2- 172.21.15.12:0/267339602 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x5566b21f3c90 0x5566b21f6070 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.044+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5566b21f85e0 msgr2=0x5566b21fa9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.044+0000 7f4dc3409e40 1 --2- 172.21.15.12:0/267339602 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x5566b21f85e0 0x5566b21fa9c0 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.044+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f4da0030c00 msgr2=0x7f4da00330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.044+0000 7f4dc3409e40 1 --2- 172.21.15.12:0/267339602 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f4da0030c00 0x7f4da00330b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4da80030a0 tx=0x7f4da8015000 comp rx=0 tx=0).stop 2023-10-05T19:42:16.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.044+0000 7f4dc173f700 1 -- 172.21.15.12:0/267339602 reap_dead start 2023-10-05T19:42:16.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.044+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5566b212d3c0 msgr2=0x5566b21dc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.044+0000 7f4dc3409e40 1 --2- 172.21.15.12:0/267339602 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5566b212d3c0 0x5566b21dc260 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4db8001330 tx=0x7f4db8014000 comp rx=0 tx=0).stop 2023-10-05T19:42:16.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.044+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 shutdown_connections 2023-10-05T19:42:16.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.044+0000 7f4dc3409e40 1 --2- 172.21.15.12:0/267339602 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5566b212d3c0 0x5566b21dc260 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.044+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 >> 172.21.15.12:0/267339602 conn(0x5566b21b4740 msgr2=0x5566b21d3330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:16.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.044+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 shutdown_connections 2023-10-05T19:42:16.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.044+0000 7f4dc3409e40 1 -- 172.21.15.12:0/267339602 wait complete. 2023-10-05T19:42:16.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.044+0000 7f4dc3409e40 1 -- 172.21.15.12:0/925075629 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f4d08030c00 msgr2=0x7f4d080330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.044+0000 7f4dc3409e40 1 --2- 172.21.15.12:0/925075629 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f4d08030c00 0x7f4d080330b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x5566b2203340 tx=0x7f4db00335d0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.044+0000 7f4dc3409e40 1 -- 172.21.15.12:0/925075629 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5566b2211840 msgr2=0x5566b22172f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.044+0000 7f4dc3409e40 1 --2- 172.21.15.12:0/925075629 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5566b2211840 0x5566b22172f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x5566b220b920 tx=0x7f4da8032000 comp rx=0 tx=0).stop 2023-10-05T19:42:16.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.044+0000 7f4dc3409e40 1 -- 172.21.15.12:0/925075629 shutdown_connections 2023-10-05T19:42:16.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.044+0000 7f4dc3409e40 1 --2- 172.21.15.12:0/925075629 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f4d08030c00 0x7f4d080330b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.044+0000 7f4dc3409e40 1 --2- 172.21.15.12:0/925075629 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5566b2211840 0x5566b22172f0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.044+0000 7f4dc3409e40 1 -- 172.21.15.12:0/925075629 >> 172.21.15.12:0/925075629 conn(0x5566b220cc00 msgr2=0x5566b220ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:16.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.044+0000 7f4dc3409e40 1 -- 172.21.15.12:0/925075629 shutdown_connections 2023-10-05T19:42:16.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.044+0000 7f4dc3409e40 1 -- 172.21.15.12:0/925075629 wait complete. 2023-10-05T19:42:16.056 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-10-05T19:42:16.056 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'get', '--rgw-zone', 'default'] 2023-10-05T19:42:16.056 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'get', '--rgw-zone', 'default'] 2023-10-05T19:42:16.057 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone get --rgw-zone default 2023-10-05T19:42:16.138 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:42:16.139 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:42:16.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.136+0000 7f5120087e40 1 Processor -- start 2023-10-05T19:42:16.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.136+0000 7f5120087e40 1 -- start start 2023-10-05T19:42:16.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.136+0000 7f5120087e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x559beb9eaea0 0x559beb9eb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.136+0000 7f5120087e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x559beb937bd0 con 0x559beb9eb7b0 2023-10-05T19:42:16.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.136+0000 7f5120087e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x559beb8f4ee0 con 0x559beb9eaea0 2023-10-05T19:42:16.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.136+0000 7f511dbbc700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x559beb9eaea0 0x559beb9eb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.136+0000 7f511d3bb700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x559beb9eb7b0 0x559beb9ec250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:47366/0 (socket says 172.21.15.12:47366) 2023-10-05T19:42:16.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.136+0000 7f511d3bb700 1 -- 172.21.15.12:0/1325655065 learned_addr learned my addr 172.21.15.12:0/1325655065 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:42:16.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.136+0000 7f511dbbc700 1 -- 172.21.15.12:0/1325655065 >> v1:172.21.15.8:6789/0 conn(0x559beb9eb7b0 legacy=0x559beb9ec250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.136+0000 7f511dbbc700 1 -- 172.21.15.12:0/1325655065 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559beb9ec960 con 0x559beb9eaea0 2023-10-05T19:42:16.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.136+0000 7f511cbba700 1 -- 172.21.15.12:0/1325655065 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2445977160 0 0) 0x559beb937bd0 con 0x559beb9eb7b0 2023-10-05T19:42:16.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.136+0000 7f511dbbc700 1 --2- 172.21.15.12:0/1325655065 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559beb9eaea0 0x559beb9eb270 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f511000a5d0 tx=0x7f5110005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4e4614727ce3a43 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.136+0000 7f511cbba700 1 -- 172.21.15.12:0/1325655065 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f51100086a0 con 0x559beb9eaea0 2023-10-05T19:42:16.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.136+0000 7f511cbba700 1 -- 172.21.15.12:0/1325655065 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5110008800 con 0x559beb9eaea0 2023-10-05T19:42:16.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.136+0000 7f511cbba700 1 -- 172.21.15.12:0/1325655065 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5110008ad0 con 0x559beb9eaea0 2023-10-05T19:42:16.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.136+0000 7f5120087e40 1 -- 172.21.15.12:0/1325655065 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559beb9eaea0 msgr2=0x559beb9eb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.136+0000 7f5120087e40 1 --2- 172.21.15.12:0/1325655065 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559beb9eaea0 0x559beb9eb270 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f511000a5d0 tx=0x7f5110005a50 comp rx=0 tx=0).stop 2023-10-05T19:42:16.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.136+0000 7f5120087e40 1 -- 172.21.15.12:0/1325655065 shutdown_connections 2023-10-05T19:42:16.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.136+0000 7f5120087e40 1 --2- 172.21.15.12:0/1325655065 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559beb9eaea0 0x559beb9eb270 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.136+0000 7f5120087e40 1 -- 172.21.15.12:0/1325655065 >> 172.21.15.12:0/1325655065 conn(0x559beb9e68c0 msgr2=0x559beb9e8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:16.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.136+0000 7f5120087e40 1 -- 172.21.15.12:0/1325655065 shutdown_connections 2023-10-05T19:42:16.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.136+0000 7f5120087e40 1 -- 172.21.15.12:0/1325655065 wait complete. 2023-10-05T19:42:16.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.140+0000 7f5120087e40 1 Processor -- start 2023-10-05T19:42:16.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.140+0000 7f5120087e40 1 -- start start 2023-10-05T19:42:16.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.140+0000 7f5120087e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559beb9eaea0 0x559beba09e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.140+0000 7f5120087e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559beb8b8e30 con 0x559beb9eaea0 2023-10-05T19:42:16.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.140+0000 7f511dbbc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559beb9eaea0 0x559beba09e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.140+0000 7f511dbbc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559beb9eaea0 0x559beba09e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:54344/0 (socket says 172.21.15.12:54344) 2023-10-05T19:42:16.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.140+0000 7f511dbbc700 1 -- 172.21.15.12:0/3620514685 learned_addr learned my addr 172.21.15.12:0/3620514685 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:42:16.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.140+0000 7f511dbbc700 1 -- 172.21.15.12:0/3620514685 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5110005cf0 con 0x559beb9eaea0 2023-10-05T19:42:16.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.140+0000 7f511dbbc700 1 --2- 172.21.15.12:0/3620514685 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559beb9eaea0 0x559beba09e00 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f5110005cc0 tx=0x7f5110015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30a385c3bcb86a33 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.140+0000 7f510f7fe700 1 -- 172.21.15.12:0/3620514685 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5110008970 con 0x559beb9eaea0 2023-10-05T19:42:16.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.140+0000 7f510f7fe700 1 -- 172.21.15.12:0/3620514685 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5110008ad0 con 0x559beb9eaea0 2023-10-05T19:42:16.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.140+0000 7f510f7fe700 1 -- 172.21.15.12:0/3620514685 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5110008dc0 con 0x559beb9eaea0 2023-10-05T19:42:16.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.140+0000 7f5120087e40 1 -- 172.21.15.12:0/3620514685 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559beb9eaea0 msgr2=0x559beba09e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.140+0000 7f5120087e40 1 --2- 172.21.15.12:0/3620514685 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559beb9eaea0 0x559beba09e00 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f5110005cc0 tx=0x7f5110015000 comp rx=0 tx=0).stop 2023-10-05T19:42:16.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.140+0000 7f5120087e40 1 -- 172.21.15.12:0/3620514685 shutdown_connections 2023-10-05T19:42:16.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.140+0000 7f5120087e40 1 --2- 172.21.15.12:0/3620514685 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559beb9eaea0 0x559beba09e00 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.140+0000 7f5120087e40 1 -- 172.21.15.12:0/3620514685 >> 172.21.15.12:0/3620514685 conn(0x559beb9e68c0 msgr2=0x559beba059c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:16.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.140+0000 7f5120087e40 1 -- 172.21.15.12:0/3620514685 shutdown_connections 2023-10-05T19:42:16.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.140+0000 7f5120087e40 1 -- 172.21.15.12:0/3620514685 wait complete. 2023-10-05T19:42:16.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.144+0000 7f5120087e40 1 Processor -- start 2023-10-05T19:42:16.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.144+0000 7f5120087e40 1 -- start start 2023-10-05T19:42:16.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.144+0000 7f5120087e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559beb9eaea0 0x559beba10a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.144+0000 7f5120087e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559beb88bed0 con 0x559beb9eaea0 2023-10-05T19:42:16.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.144+0000 7f511dbbc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559beb9eaea0 0x559beba10a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.144+0000 7f511dbbc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559beb9eaea0 0x559beba10a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:54350/0 (socket says 172.21.15.12:54350) 2023-10-05T19:42:16.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.144+0000 7f511dbbc700 1 -- 172.21.15.12:0/3302061696 learned_addr learned my addr 172.21.15.12:0/3302061696 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:42:16.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.144+0000 7f511dbbc700 1 -- 172.21.15.12:0/3302061696 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559beb9ec960 con 0x559beb9eaea0 2023-10-05T19:42:16.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.144+0000 7f511dbbc700 1 --2- 172.21.15.12:0/3302061696 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559beb9eaea0 0x559beba10a10 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f511001e040 tx=0x7f511001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.144+0000 7f510dffb700 1 -- 172.21.15.12:0/3302061696 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f51100086a0 con 0x559beb9eaea0 2023-10-05T19:42:16.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.144+0000 7f5120087e40 1 -- 172.21.15.12:0/3302061696 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559beba10f50 con 0x559beb9eaea0 2023-10-05T19:42:16.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.144+0000 7f510dffb700 1 -- 172.21.15.12:0/3302061696 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f511002a070 con 0x559beb9eaea0 2023-10-05T19:42:16.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.144+0000 7f510dffb700 1 -- 172.21.15.12:0/3302061696 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5110012cd0 con 0x559beb9eaea0 2023-10-05T19:42:16.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.144+0000 7f5120087e40 1 -- 172.21.15.12:0/3302061696 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559beba113f0 con 0x559beb9eaea0 2023-10-05T19:42:16.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.148+0000 7f510dffb700 1 -- 172.21.15.12:0/3302061696 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f5110019070 con 0x559beb9eaea0 2023-10-05T19:42:16.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.148+0000 7f510dffb700 1 --2- 172.21.15.12:0/3302061696 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5100030c00 0x7f51000330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.148+0000 7f5120087e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:42:16.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.148+0000 7f5120087e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:42:16.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.148+0000 7f510dffb700 1 -- 172.21.15.12:0/3302061696 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7f511002c070 con 0x559beb9eaea0 2023-10-05T19:42:16.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.148+0000 7f511d3bb700 1 --2- 172.21.15.12:0/3302061696 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5100030c00 0x7f51000330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.148+0000 7f5120087e40 1 Processor -- start 2023-10-05T19:42:16.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.148+0000 7f5120087e40 1 -- start start 2023-10-05T19:42:16.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.148+0000 7f5120087e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559beba22070 0x559beba22440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.148+0000 7f5120087e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559beba22980 con 0x559beba22070 2023-10-05T19:42:16.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.148+0000 7f511d3bb700 1 --2- 172.21.15.12:0/3302061696 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5100030c00 0x7f51000330b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f51140011e0 tx=0x7f51140013a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.148+0000 7f511dbbc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559beba22070 0x559beba22440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.148+0000 7f511dbbc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559beba22070 0x559beba22440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:54362/0 (socket says 172.21.15.12:54362) 2023-10-05T19:42:16.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.148+0000 7f511dbbc700 1 -- 172.21.15.12:0/1024194381 learned_addr learned my addr 172.21.15.12:0/1024194381 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:42:16.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.148+0000 7f511dbbc700 1 -- 172.21.15.12:0/1024194381 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559beba22ac0 con 0x559beba22070 2023-10-05T19:42:16.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.148+0000 7f511dbbc700 1 --2- 172.21.15.12:0/1024194381 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559beba22070 0x559beba22440 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f5110011fd0 tx=0x7f511001dfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36a24116d23650fd server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.148+0000 7f50eefe5700 1 -- 172.21.15.12:0/1024194381 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5110049070 con 0x559beba22070 2023-10-05T19:42:16.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.148+0000 7f50eefe5700 1 -- 172.21.15.12:0/1024194381 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5110040050 con 0x559beba22070 2023-10-05T19:42:16.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.148+0000 7f50eefe5700 1 -- 172.21.15.12:0/1024194381 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f511003a3f0 con 0x559beba22070 2023-10-05T19:42:16.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.152+0000 7f5120087e40 1 -- 172.21.15.12:0/1024194381 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559beba22070 msgr2=0x559beba22440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.152+0000 7f5120087e40 1 --2- 172.21.15.12:0/1024194381 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559beba22070 0x559beba22440 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f5110011fd0 tx=0x7f511001dfd0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.152+0000 7f5120087e40 1 -- 172.21.15.12:0/1024194381 shutdown_connections 2023-10-05T19:42:16.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.152+0000 7f5120087e40 1 --2- 172.21.15.12:0/1024194381 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559beba22070 0x559beba22440 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.152+0000 7f5120087e40 1 -- 172.21.15.12:0/1024194381 >> 172.21.15.12:0/1024194381 conn(0x559beb9eb7b0 msgr2=0x559beba1edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:16.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.152+0000 7f5120087e40 1 -- 172.21.15.12:0/1024194381 shutdown_connections 2023-10-05T19:42:16.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.152+0000 7f5120087e40 1 -- 172.21.15.12:0/1024194381 wait complete. 2023-10-05T19:42:16.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.152+0000 7f5120087e40 1 Processor -- start 2023-10-05T19:42:16.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.152+0000 7f5120087e40 1 -- start start 2023-10-05T19:42:16.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.152+0000 7f5120087e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559beba22070 0x559beba292f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.152+0000 7f5120087e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559beba2a2f0 con 0x559beba22070 2023-10-05T19:42:16.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.152+0000 7f511dbbc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559beba22070 0x559beba292f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.152+0000 7f511dbbc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559beba22070 0x559beba292f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:54366/0 (socket says 172.21.15.12:54366) 2023-10-05T19:42:16.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.152+0000 7f511dbbc700 1 -- 172.21.15.12:0/4249938110 learned_addr learned my addr 172.21.15.12:0/4249938110 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:42:16.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.152+0000 7f511dbbc700 1 -- 172.21.15.12:0/4249938110 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559beba113f0 con 0x559beba22070 2023-10-05T19:42:16.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.152+0000 7f511dbbc700 1 --2- 172.21.15.12:0/4249938110 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559beba22070 0x559beba292f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f5110011fd0 tx=0x7f511004b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.152+0000 7f50ed7e2700 1 -- 172.21.15.12:0/4249938110 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5110024890 con 0x559beba22070 2023-10-05T19:42:16.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.156+0000 7f50ed7e2700 1 -- 172.21.15.12:0/4249938110 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51100249f0 con 0x559beba22070 2023-10-05T19:42:16.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.156+0000 7f50ed7e2700 1 -- 172.21.15.12:0/4249938110 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f511000c0b0 con 0x559beba22070 2023-10-05T19:42:16.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.156+0000 7f5120087e40 1 -- 172.21.15.12:0/4249938110 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559beba298f0 con 0x559beba22070 2023-10-05T19:42:16.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.156+0000 7f5120087e40 1 -- 172.21.15.12:0/4249938110 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559beba29e10 con 0x559beba22070 2023-10-05T19:42:16.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.156+0000 7f50ed7e2700 1 -- 172.21.15.12:0/4249938110 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f511000c210 con 0x559beba22070 2023-10-05T19:42:16.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.156+0000 7f5120087e40 20 try_to_resolve_local_entities(): before: realm_id= realm_name= zonegroup_id= zonegroup_name= zone_id= zone_name=default 2023-10-05T19:42:16.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.156+0000 7f5066ffd700 20 reqs_thread_entry: start 2023-10-05T19:42:16.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.156+0000 7f5120087e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:16.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.156+0000 7f50ed7e2700 1 --2- 172.21.15.12:0/4249938110 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f506c030bb0 0x7f506c033060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.156+0000 7f50ed7e2700 1 -- 172.21.15.12:0/4249938110 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7f511005abf0 con 0x559beba22070 2023-10-05T19:42:16.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.156+0000 7f5120087e40 1 --2- 172.21.15.12:0/3302061696 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x559beba47170 0x559beba47610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.156+0000 7f5120087e40 1 -- 172.21.15.12:0/3302061696 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x559beba5a6b0 con 0x559beba47170 2023-10-05T19:42:16.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.156+0000 7f511d3bb700 1 --2- 172.21.15.12:0/4249938110 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f506c030bb0 0x7f506c033060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.156+0000 7f511e3bd700 1 --2- 172.21.15.12:0/3302061696 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x559beba47170 0x559beba47610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.156+0000 7f511d3bb700 1 --2- 172.21.15.12:0/4249938110 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f506c030bb0 0x7f506c033060 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5114000c00 tx=0x7f5114010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.156+0000 7f511e3bd700 1 --2- 172.21.15.12:0/3302061696 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x559beba47170 0x559beba47610 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.160+0000 7f511e3bd700 1 -- 172.21.15.12:0/3302061696 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f511800a520 con 0x559beba47170 2023-10-05T19:42:16.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.160+0000 7f5120087e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:42:16.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.160+0000 7f5120087e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:16.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.160+0000 7f5120087e40 1 --2- 172.21.15.12:0/3302061696 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x559beba5af70 0x559beba5d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.160+0000 7f5120087e40 1 -- 172.21.15.12:0/3302061696 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:2 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x559beba5d890 con 0x559beba5af70 2023-10-05T19:42:16.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.160+0000 7f511dbbc700 1 --2- 172.21.15.12:0/3302061696 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x559beba5af70 0x559beba5d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.160+0000 7f511dbbc700 1 --2- 172.21.15.12:0/3302061696 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x559beba5af70 0x559beba5d350 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.160+0000 7f511dbbc700 1 -- 172.21.15.12:0/3302061696 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(2 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f5110056070 con 0x559beba5af70 2023-10-05T19:42:16.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.160+0000 7f5120087e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:42:16.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.160+0000 7f5120087e40 1 --2- 172.21.15.12:0/3302061696 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x559beba49910 0x559beba61170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.160+0000 7f5120087e40 1 -- 172.21.15.12:0/3302061696 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:3 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559beba616b0 con 0x559beba49910 2023-10-05T19:42:16.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.160+0000 7f511d3bb700 1 --2- 172.21.15.12:0/3302061696 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x559beba49910 0x559beba61170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.160+0000 7f511d3bb700 1 --2- 172.21.15.12:0/3302061696 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x559beba49910 0x559beba61170 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.164+0000 7f511d3bb700 1 -- 172.21.15.12:0/3302061696 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(3 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f5114006070 con 0x559beba49910 2023-10-05T19:42:16.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.164+0000 7f511d3bb700 1 -- 172.21.15.12:0/3302061696 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:4 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559beba616b0 con 0x559beba5af70 2023-10-05T19:42:16.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.164+0000 7f511dbbc700 1 -- 172.21.15.12:0/3302061696 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(4 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5110056070 con 0x559beba5af70 2023-10-05T19:42:16.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.164+0000 7f511dbbc700 1 --2- 172.21.15.12:0/3302061696 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f5110062020 0x7f51100321b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.164+0000 7f511dbbc700 1 -- 172.21.15.12:0/3302061696 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:5 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559beba616b0 con 0x7f5110062020 2023-10-05T19:42:16.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.164+0000 7f511e3bd700 1 --2- 172.21.15.12:0/3302061696 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f5110062020 0x7f51100321b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.164+0000 7f511e3bd700 1 --2- 172.21.15.12:0/3302061696 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f5110062020 0x7f51100321b0 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.168+0000 7f511e3bd700 1 -- 172.21.15.12:0/3302061696 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(5 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f511800a520 con 0x7f5110062020 2023-10-05T19:42:16.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.168+0000 7f511e3bd700 1 -- 172.21.15.12:0/3302061696 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:6 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559beba616b0 con 0x559beba47170 2023-10-05T19:42:16.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.168+0000 7f511e3bd700 1 -- 172.21.15.12:0/3302061696 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f511800a520 con 0x559beba47170 2023-10-05T19:42:16.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.168+0000 7f511e3bd700 1 -- 172.21.15.12:0/3302061696 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:7 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559beba616b0 con 0x559beba47170 2023-10-05T19:42:16.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.168+0000 7f511e3bd700 1 -- 172.21.15.12:0/3302061696 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f511800a520 con 0x559beba47170 2023-10-05T19:42:16.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.168+0000 7f511e3bd700 1 -- 172.21.15.12:0/3302061696 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:8 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559beba616b0 con 0x559beba47170 2023-10-05T19:42:16.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.168+0000 7f511e3bd700 1 -- 172.21.15.12:0/3302061696 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f511800a520 con 0x559beba47170 2023-10-05T19:42:16.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.168+0000 7f511e3bd700 1 -- 172.21.15.12:0/3302061696 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:9 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559beba616b0 con 0x7f5110062020 2023-10-05T19:42:16.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.168+0000 7f511e3bd700 1 -- 172.21.15.12:0/3302061696 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f511800a520 con 0x7f5110062020 2023-10-05T19:42:16.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.168+0000 7f511e3bd700 1 -- 172.21.15.12:0/3302061696 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:10 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559beba616b0 con 0x559beba5af70 2023-10-05T19:42:16.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.172+0000 7f511dbbc700 1 -- 172.21.15.12:0/3302061696 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5110056070 con 0x559beba5af70 2023-10-05T19:42:16.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.172+0000 7f511dbbc700 1 -- 172.21.15.12:0/3302061696 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:11 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559beba616b0 con 0x559beba5af70 2023-10-05T19:42:16.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.172+0000 7f511dbbc700 1 -- 172.21.15.12:0/3302061696 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(11 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5110056070 con 0x559beba5af70 2023-10-05T19:42:16.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.172+0000 7f511dbbc700 1 -- 172.21.15.12:0/3302061696 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:12 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559beba616b0 con 0x559beba47170 2023-10-05T19:42:16.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.172+0000 7f511e3bd700 1 -- 172.21.15.12:0/3302061696 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f511800a520 con 0x559beba47170 2023-10-05T19:42:16.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.172+0000 7f511e3bd700 1 -- 172.21.15.12:0/3302061696 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:13 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559beba616b0 con 0x7f5110062020 2023-10-05T19:42:16.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.172+0000 7f511e3bd700 1 -- 172.21.15.12:0/3302061696 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f511800a520 con 0x7f5110062020 2023-10-05T19:42:16.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.172+0000 7f511e3bd700 1 -- 172.21.15.12:0/3302061696 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:14 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559beba616b0 con 0x559beba47170 2023-10-05T19:42:16.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.176+0000 7f511e3bd700 1 -- 172.21.15.12:0/3302061696 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f511800a520 con 0x559beba47170 2023-10-05T19:42:16.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.176+0000 7f511e3bd700 1 -- 172.21.15.12:0/3302061696 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:15 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559beba616b0 con 0x559beba5af70 2023-10-05T19:42:16.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.176+0000 7f511dbbc700 1 -- 172.21.15.12:0/3302061696 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5110056070 con 0x559beba5af70 2023-10-05T19:42:16.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.176+0000 7f511dbbc700 1 -- 172.21.15.12:0/3302061696 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:16 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559beba616b0 con 0x559beba49910 2023-10-05T19:42:16.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.176+0000 7f511d3bb700 1 -- 172.21.15.12:0/3302061696 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5114006070 con 0x559beba49910 2023-10-05T19:42:16.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.176+0000 7f511d3bb700 1 -- 172.21.15.12:0/3302061696 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:17 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559beba616b0 con 0x559beba49910 2023-10-05T19:42:16.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.176+0000 7f511d3bb700 1 -- 172.21.15.12:0/3302061696 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5114006070 con 0x559beba49910 2023-10-05T19:42:16.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.176+0000 7f511d3bb700 1 -- 172.21.15.12:0/3302061696 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:18 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559beba616b0 con 0x7f5110062020 2023-10-05T19:42:16.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.176+0000 7f511e3bd700 1 -- 172.21.15.12:0/3302061696 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(18 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f511800a520 con 0x7f5110062020 2023-10-05T19:42:16.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.176+0000 7f511e3bd700 1 -- 172.21.15.12:0/3302061696 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559beba616b0 con 0x559beba49910 2023-10-05T19:42:16.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.180+0000 7f511d3bb700 1 -- 172.21.15.12:0/3302061696 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5114006070 con 0x559beba49910 2023-10-05T19:42:16.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.180+0000 7f511d3bb700 1 -- 172.21.15.12:0/3302061696 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559beba616b0 con 0x559beba49910 2023-10-05T19:42:16.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.180+0000 7f511d3bb700 1 -- 172.21.15.12:0/3302061696 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5114006070 con 0x559beba49910 2023-10-05T19:42:16.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.180+0000 7f511d3bb700 1 -- 172.21.15.12:0/3302061696 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:21 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559beba616b0 con 0x559beba47170 2023-10-05T19:42:16.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.180+0000 7f511e3bd700 1 -- 172.21.15.12:0/3302061696 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f511800a520 con 0x559beba47170 2023-10-05T19:42:16.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.180+0000 7f511e3bd700 1 -- 172.21.15.12:0/3302061696 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:22 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559beba616b0 con 0x559beba47170 2023-10-05T19:42:16.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.180+0000 7f511e3bd700 1 -- 172.21.15.12:0/3302061696 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f511800a520 con 0x559beba47170 2023-10-05T19:42:16.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.180+0000 7f511e3bd700 1 -- 172.21.15.12:0/3302061696 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:23 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559beba616b0 con 0x7f5110062020 2023-10-05T19:42:16.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.184+0000 7f511e3bd700 1 -- 172.21.15.12:0/3302061696 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f511800a520 con 0x7f5110062020 2023-10-05T19:42:16.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.184+0000 7f511e3bd700 1 -- 172.21.15.12:0/3302061696 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:24 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559beba616b0 con 0x559beba49910 2023-10-05T19:42:16.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.184+0000 7f511d3bb700 1 -- 172.21.15.12:0/3302061696 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5114006070 con 0x559beba49910 2023-10-05T19:42:16.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.184+0000 7f511d3bb700 1 -- 172.21.15.12:0/3302061696 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:25 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559beba616b0 con 0x559beba5af70 2023-10-05T19:42:16.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.184+0000 7f511dbbc700 1 -- 172.21.15.12:0/3302061696 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(25 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5110056070 con 0x559beba5af70 2023-10-05T19:42:16.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.184+0000 7f511dbbc700 1 -- 172.21.15.12:0/3302061696 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:26 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559beba616b0 con 0x559beba47170 2023-10-05T19:42:16.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.184+0000 7f511e3bd700 1 -- 172.21.15.12:0/3302061696 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f511800a520 con 0x559beba47170 2023-10-05T19:42:16.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.184+0000 7f511e3bd700 1 -- 172.21.15.12:0/3302061696 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:27 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559beba616b0 con 0x7f5110062020 2023-10-05T19:42:16.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.184+0000 7f511e3bd700 1 -- 172.21.15.12:0/3302061696 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f511800a520 con 0x7f5110062020 2023-10-05T19:42:16.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.184+0000 7f511e3bd700 1 -- 172.21.15.12:0/3302061696 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:28 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559beba616b0 con 0x559beba49910 2023-10-05T19:42:16.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.188+0000 7f511d3bb700 1 -- 172.21.15.12:0/3302061696 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5114006070 con 0x559beba49910 2023-10-05T19:42:16.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.188+0000 7f511d3bb700 1 -- 172.21.15.12:0/3302061696 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:29 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559beba616b0 con 0x7f5110062020 2023-10-05T19:42:16.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.188+0000 7f511e3bd700 1 -- 172.21.15.12:0/3302061696 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(29 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f511800a520 con 0x7f5110062020 2023-10-05T19:42:16.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.188+0000 7f511e3bd700 1 -- 172.21.15.12:0/3302061696 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:30 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559beba616b0 con 0x559beba5af70 2023-10-05T19:42:16.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.188+0000 7f511dbbc700 1 -- 172.21.15.12:0/3302061696 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5110056070 con 0x559beba5af70 2023-10-05T19:42:16.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.188+0000 7f511dbbc700 1 -- 172.21.15.12:0/3302061696 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559beba616b0 con 0x559beba49910 2023-10-05T19:42:16.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.188+0000 7f511d3bb700 1 -- 172.21.15.12:0/3302061696 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5114006070 con 0x559beba49910 2023-10-05T19:42:16.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.188+0000 7f511d3bb700 1 -- 172.21.15.12:0/3302061696 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559beba616b0 con 0x7f5110062020 2023-10-05T19:42:16.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.192+0000 7f511e3bd700 1 -- 172.21.15.12:0/3302061696 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f511800a520 con 0x7f5110062020 2023-10-05T19:42:16.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.192+0000 7f511e3bd700 1 -- 172.21.15.12:0/3302061696 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:33 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559beba616b0 con 0x559beba47170 2023-10-05T19:42:16.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.192+0000 7f511e3bd700 1 -- 172.21.15.12:0/3302061696 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(33 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f511800a520 con 0x559beba47170 2023-10-05T19:42:16.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.192+0000 7f511e3bd700 1 -- 172.21.15.12:0/3302061696 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:34 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x559beba616b0 con 0x559beba5af70 2023-10-05T19:42:16.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.192+0000 7f511dbbc700 1 -- 172.21.15.12:0/3302061696 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(34 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f5110056070 con 0x559beba5af70 2023-10-05T19:42:16.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.192+0000 7f5120087e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:42:16.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.192+0000 7f5120087e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:42:16.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.192+0000 7f5120087e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:42:16.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.192+0000 7f5120087e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:42:16.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.192+0000 7f5120087e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:16.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.192+0000 7f5120087e40 1 -- 172.21.15.12:0/3302061696 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:35 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x559beba5dc40 con 0x559beba5af70 2023-10-05T19:42:16.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.192+0000 7f511dbbc700 1 -- 172.21.15.12:0/3302061696 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(35 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f5110056070 con 0x559beba5af70 2023-10-05T19:42:16.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.192+0000 7f5120087e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:42:16.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.196+0000 7f5120087e40 1 -- 172.21.15.12:0/3302061696 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f5110062020 msgr2=0x7f51100321b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.196+0000 7f5120087e40 1 --2- 172.21.15.12:0/3302061696 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f5110062020 0x7f51100321b0 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.196+0000 7f5120087e40 1 -- 172.21.15.12:0/3302061696 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x559beba47170 msgr2=0x559beba47610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.196+0000 7f5120087e40 1 --2- 172.21.15.12:0/3302061696 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x559beba47170 0x559beba47610 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.196+0000 7f5120087e40 1 -- 172.21.15.12:0/3302061696 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x559beba5af70 msgr2=0x559beba5d350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.196+0000 7f5120087e40 1 --2- 172.21.15.12:0/3302061696 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x559beba5af70 0x559beba5d350 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.196+0000 7f5120087e40 1 -- 172.21.15.12:0/3302061696 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x559beba49910 msgr2=0x559beba61170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.196+0000 7f5120087e40 1 --2- 172.21.15.12:0/3302061696 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x559beba49910 0x559beba61170 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.196+0000 7f5120087e40 1 -- 172.21.15.12:0/3302061696 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5100030c00 msgr2=0x7f51000330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.196+0000 7f5120087e40 1 --2- 172.21.15.12:0/3302061696 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f5100030c00 0x7f51000330b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f51140011e0 tx=0x7f51140013a0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.196+0000 7f5120087e40 1 -- 172.21.15.12:0/3302061696 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559beb9eaea0 msgr2=0x559beba10a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.196+0000 7f511e3bd700 1 -- 172.21.15.12:0/3302061696 reap_dead start 2023-10-05T19:42:16.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.196+0000 7f5120087e40 1 --2- 172.21.15.12:0/3302061696 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559beb9eaea0 0x559beba10a10 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f511001e040 tx=0x7f511001c000 comp rx=0 tx=0).stop 2023-10-05T19:42:16.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.196+0000 7f5120087e40 1 -- 172.21.15.12:0/3302061696 shutdown_connections 2023-10-05T19:42:16.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.196+0000 7f5120087e40 1 --2- 172.21.15.12:0/3302061696 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559beb9eaea0 0x559beba10a10 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.196+0000 7f5120087e40 1 -- 172.21.15.12:0/3302061696 >> 172.21.15.12:0/3302061696 conn(0x559beb9e68c0 msgr2=0x559beba04440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:16.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.196+0000 7f5120087e40 1 -- 172.21.15.12:0/3302061696 shutdown_connections 2023-10-05T19:42:16.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.196+0000 7f5120087e40 1 -- 172.21.15.12:0/3302061696 wait complete. 2023-10-05T19:42:16.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.196+0000 7f5120087e40 1 -- 172.21.15.12:0/4249938110 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f506c030bb0 msgr2=0x7f506c033060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.196+0000 7f5120087e40 1 --2- 172.21.15.12:0/4249938110 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f506c030bb0 0x7f506c033060 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5114000c00 tx=0x7f5114010000 comp rx=0 tx=0).stop 2023-10-05T19:42:16.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.196+0000 7f5120087e40 1 -- 172.21.15.12:0/4249938110 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559beba22070 msgr2=0x559beba292f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.196+0000 7f5120087e40 1 --2- 172.21.15.12:0/4249938110 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559beba22070 0x559beba292f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f5110011fd0 tx=0x7f511004b000 comp rx=0 tx=0).stop 2023-10-05T19:42:16.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.196+0000 7f5120087e40 1 -- 172.21.15.12:0/4249938110 shutdown_connections 2023-10-05T19:42:16.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.196+0000 7f5120087e40 1 --2- 172.21.15.12:0/4249938110 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f506c030bb0 0x7f506c033060 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.196+0000 7f5120087e40 1 --2- 172.21.15.12:0/4249938110 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559beba22070 0x559beba292f0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.196+0000 7f5120087e40 1 -- 172.21.15.12:0/4249938110 >> 172.21.15.12:0/4249938110 conn(0x559beb9eb7b0 msgr2=0x559beba1ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:16.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.196+0000 7f5120087e40 1 -- 172.21.15.12:0/4249938110 shutdown_connections 2023-10-05T19:42:16.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.196+0000 7f5120087e40 1 -- 172.21.15.12:0/4249938110 wait complete. 2023-10-05T19:42:16.205 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:42:16.205 INFO:teuthology.orchestra.run.smithi012.stdout: "id": "93ac49f4-ff49-4f99-811b-6453b28decd5", 2023-10-05T19:42:16.205 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default", 2023-10-05T19:42:16.205 INFO:teuthology.orchestra.run.smithi012.stdout: "domain_root": "default.rgw.meta:root", 2023-10-05T19:42:16.205 INFO:teuthology.orchestra.run.smithi012.stdout: "control_pool": "default.rgw.control", 2023-10-05T19:42:16.205 INFO:teuthology.orchestra.run.smithi012.stdout: "gc_pool": "default.rgw.log:gc", 2023-10-05T19:42:16.206 INFO:teuthology.orchestra.run.smithi012.stdout: "lc_pool": "default.rgw.log:lc", 2023-10-05T19:42:16.206 INFO:teuthology.orchestra.run.smithi012.stdout: "log_pool": "default.rgw.log", 2023-10-05T19:42:16.206 INFO:teuthology.orchestra.run.smithi012.stdout: "intent_log_pool": "default.rgw.log:intent", 2023-10-05T19:42:16.206 INFO:teuthology.orchestra.run.smithi012.stdout: "usage_log_pool": "default.rgw.log:usage", 2023-10-05T19:42:16.206 INFO:teuthology.orchestra.run.smithi012.stdout: "roles_pool": "default.rgw.meta:roles", 2023-10-05T19:42:16.206 INFO:teuthology.orchestra.run.smithi012.stdout: "reshard_pool": "default.rgw.log:reshard", 2023-10-05T19:42:16.206 INFO:teuthology.orchestra.run.smithi012.stdout: "user_keys_pool": "default.rgw.meta:users.keys", 2023-10-05T19:42:16.206 INFO:teuthology.orchestra.run.smithi012.stdout: "user_email_pool": "default.rgw.meta:users.email", 2023-10-05T19:42:16.206 INFO:teuthology.orchestra.run.smithi012.stdout: "user_swift_pool": "default.rgw.meta:users.swift", 2023-10-05T19:42:16.206 INFO:teuthology.orchestra.run.smithi012.stdout: "user_uid_pool": "default.rgw.meta:users.uid", 2023-10-05T19:42:16.206 INFO:teuthology.orchestra.run.smithi012.stdout: "otp_pool": "default.rgw.otp", 2023-10-05T19:42:16.207 INFO:teuthology.orchestra.run.smithi012.stdout: "system_key": { 2023-10-05T19:42:16.207 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key": "", 2023-10-05T19:42:16.207 INFO:teuthology.orchestra.run.smithi012.stdout: "secret_key": "" 2023-10-05T19:42:16.207 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:42:16.207 INFO:teuthology.orchestra.run.smithi012.stdout: "placement_pools": [ 2023-10-05T19:42:16.207 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:42:16.207 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "default-placement", 2023-10-05T19:42:16.207 INFO:teuthology.orchestra.run.smithi012.stdout: "val": { 2023-10-05T19:42:16.207 INFO:teuthology.orchestra.run.smithi012.stdout: "index_pool": "default.rgw.buckets.index", 2023-10-05T19:42:16.207 INFO:teuthology.orchestra.run.smithi012.stdout: "storage_classes": { 2023-10-05T19:42:16.207 INFO:teuthology.orchestra.run.smithi012.stdout: "STANDARD": { 2023-10-05T19:42:16.208 INFO:teuthology.orchestra.run.smithi012.stdout: "data_pool": "default.rgw.buckets.data" 2023-10-05T19:42:16.208 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:42:16.208 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:42:16.208 INFO:teuthology.orchestra.run.smithi012.stdout: "data_extra_pool": "default.rgw.buckets.non-ec", 2023-10-05T19:42:16.208 INFO:teuthology.orchestra.run.smithi012.stdout: "index_type": 0, 2023-10-05T19:42:16.208 INFO:teuthology.orchestra.run.smithi012.stdout: "inline_data": "true" 2023-10-05T19:42:16.208 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:42:16.208 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:42:16.208 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:42:16.208 INFO:teuthology.orchestra.run.smithi012.stdout: "realm_id": "", 2023-10-05T19:42:16.208 INFO:teuthology.orchestra.run.smithi012.stdout: "notif_pool": "default.rgw.log:notif" 2023-10-05T19:42:16.208 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:42:16.209 DEBUG:tasks.util.rgw: json result: {'id': '93ac49f4-ff49-4f99-811b-6453b28decd5', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': 'true'}}], 'realm_id': '', 'notif_pool': 'default.rgw.log:notif'} 2023-10-05T19:42:16.210 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'set'] 2023-10-05T19:42:16.210 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'set'] 2023-10-05T19:42:16.210 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone set 2023-10-05T19:42:16.247 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:42:16.247 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:42:16.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.244+0000 7ff00527ae40 1 Processor -- start 2023-10-05T19:42:16.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.244+0000 7ff00527ae40 1 -- start start 2023-10-05T19:42:16.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.244+0000 7ff00527ae40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x563de42f3fe0 0x563de42f43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.244+0000 7ff00527ae40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x563de423ec00 con 0x563de42f48f0 2023-10-05T19:42:16.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.244+0000 7ff00527ae40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x563de41fbee0 con 0x563de42f3fe0 2023-10-05T19:42:16.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.244+0000 7ff002daf700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x563de42f3fe0 0x563de42f43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.244+0000 7ff0025ae700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x563de42f48f0 0x563de42f5390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:47370/0 (socket says 172.21.15.12:47370) 2023-10-05T19:42:16.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.244+0000 7ff0025ae700 1 -- 172.21.15.12:0/2445489737 learned_addr learned my addr 172.21.15.12:0/2445489737 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:42:16.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.244+0000 7ff002daf700 1 -- 172.21.15.12:0/2445489737 >> v1:172.21.15.8:6789/0 conn(0x563de42f48f0 legacy=0x563de42f5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.244+0000 7ff001dad700 1 -- 172.21.15.12:0/2445489737 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 583185401 0 0) 0x563de423ec00 con 0x563de42f48f0 2023-10-05T19:42:16.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.244+0000 7ff002daf700 1 -- 172.21.15.12:0/2445489737 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563de42f5aa0 con 0x563de42f3fe0 2023-10-05T19:42:16.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.244+0000 7ff002daf700 1 --2- 172.21.15.12:0/2445489737 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563de42f3fe0 0x563de42f43b0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7feff400a5d0 tx=0x7feff4005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3286ae87867ac43 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.244+0000 7ff001dad700 1 -- 172.21.15.12:0/2445489737 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feff40086a0 con 0x563de42f3fe0 2023-10-05T19:42:16.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.244+0000 7ff001dad700 1 -- 172.21.15.12:0/2445489737 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff4008800 con 0x563de42f3fe0 2023-10-05T19:42:16.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.244+0000 7ff001dad700 1 -- 172.21.15.12:0/2445489737 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feff4008ad0 con 0x563de42f3fe0 2023-10-05T19:42:16.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.248+0000 7ff00527ae40 1 -- 172.21.15.12:0/2445489737 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563de42f3fe0 msgr2=0x563de42f43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.248+0000 7ff00527ae40 1 --2- 172.21.15.12:0/2445489737 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563de42f3fe0 0x563de42f43b0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7feff400a5d0 tx=0x7feff4005a50 comp rx=0 tx=0).stop 2023-10-05T19:42:16.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.248+0000 7ff00527ae40 1 -- 172.21.15.12:0/2445489737 shutdown_connections 2023-10-05T19:42:16.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.248+0000 7ff00527ae40 1 --2- 172.21.15.12:0/2445489737 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563de42f3fe0 0x563de42f43b0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.248+0000 7ff00527ae40 1 -- 172.21.15.12:0/2445489737 >> 172.21.15.12:0/2445489737 conn(0x563de4267830 msgr2=0x563de4267c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:16.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.248+0000 7ff00527ae40 1 -- 172.21.15.12:0/2445489737 shutdown_connections 2023-10-05T19:42:16.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.248+0000 7ff00527ae40 1 -- 172.21.15.12:0/2445489737 wait complete. 2023-10-05T19:42:16.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.248+0000 7ff00527ae40 1 Processor -- start 2023-10-05T19:42:16.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.248+0000 7ff00527ae40 1 -- start start 2023-10-05T19:42:16.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.248+0000 7ff00527ae40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563de42f3fe0 0x563de4312f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.248+0000 7ff00527ae40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x563de41bfe30 con 0x563de42f3fe0 2023-10-05T19:42:16.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.248+0000 7ff002daf700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563de42f3fe0 0x563de4312f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.248+0000 7ff002daf700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563de42f3fe0 0x563de4312f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:54374/0 (socket says 172.21.15.12:54374) 2023-10-05T19:42:16.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.248+0000 7ff002daf700 1 -- 172.21.15.12:0/4238337933 learned_addr learned my addr 172.21.15.12:0/4238337933 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:42:16.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.248+0000 7ff002daf700 1 -- 172.21.15.12:0/4238337933 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff4005cf0 con 0x563de42f3fe0 2023-10-05T19:42:16.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.248+0000 7ff002daf700 1 --2- 172.21.15.12:0/4238337933 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563de42f3fe0 0x563de4312f10 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7feff4005cc0 tx=0x7feff4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca9308ef4c16425d server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.248+0000 7ff000dab700 1 -- 172.21.15.12:0/4238337933 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feff4008970 con 0x563de42f3fe0 2023-10-05T19:42:16.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.248+0000 7ff000dab700 1 -- 172.21.15.12:0/4238337933 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff4008ad0 con 0x563de42f3fe0 2023-10-05T19:42:16.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.248+0000 7ff000dab700 1 -- 172.21.15.12:0/4238337933 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feff4008dc0 con 0x563de42f3fe0 2023-10-05T19:42:16.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.248+0000 7ff00527ae40 1 -- 172.21.15.12:0/4238337933 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563de42f3fe0 msgr2=0x563de4312f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.248+0000 7ff00527ae40 1 --2- 172.21.15.12:0/4238337933 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563de42f3fe0 0x563de4312f10 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7feff4005cc0 tx=0x7feff4015000 comp rx=0 tx=0).stop 2023-10-05T19:42:16.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.252+0000 7ff00527ae40 1 -- 172.21.15.12:0/4238337933 shutdown_connections 2023-10-05T19:42:16.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.252+0000 7ff00527ae40 1 --2- 172.21.15.12:0/4238337933 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563de42f3fe0 0x563de4312f10 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.252+0000 7ff00527ae40 1 -- 172.21.15.12:0/4238337933 >> 172.21.15.12:0/4238337933 conn(0x563de4267830 msgr2=0x563de430ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:16.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.252+0000 7ff00527ae40 1 -- 172.21.15.12:0/4238337933 shutdown_connections 2023-10-05T19:42:16.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.252+0000 7ff00527ae40 1 -- 172.21.15.12:0/4238337933 wait complete. 2023-10-05T19:42:16.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.252+0000 7ff00527ae40 1 Processor -- start 2023-10-05T19:42:16.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.252+0000 7ff00527ae40 1 -- start start 2023-10-05T19:42:16.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.252+0000 7ff00527ae40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563de42f3fe0 0x563de4319b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.252+0000 7ff00527ae40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x563de4192ed0 con 0x563de42f3fe0 2023-10-05T19:42:16.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.252+0000 7ff002daf700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563de42f3fe0 0x563de4319b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.252+0000 7ff002daf700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563de42f3fe0 0x563de4319b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:54380/0 (socket says 172.21.15.12:54380) 2023-10-05T19:42:16.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.252+0000 7ff002daf700 1 -- 172.21.15.12:0/2557768836 learned_addr learned my addr 172.21.15.12:0/2557768836 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:42:16.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.252+0000 7ff002daf700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563de42f5aa0 con 0x563de42f3fe0 2023-10-05T19:42:16.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.252+0000 7ff002daf700 1 --2- 172.21.15.12:0/2557768836 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563de42f3fe0 0x563de4319b20 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7feff401e040 tx=0x7feff401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.252+0000 7feffaffd700 1 -- 172.21.15.12:0/2557768836 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feff40086a0 con 0x563de42f3fe0 2023-10-05T19:42:16.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.252+0000 7feffaffd700 1 -- 172.21.15.12:0/2557768836 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff402a070 con 0x563de42f3fe0 2023-10-05T19:42:16.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.252+0000 7ff00527ae40 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563de431a060 con 0x563de42f3fe0 2023-10-05T19:42:16.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.252+0000 7feffaffd700 1 -- 172.21.15.12:0/2557768836 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feff4012cd0 con 0x563de42f3fe0 2023-10-05T19:42:16.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.252+0000 7ff00527ae40 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563de431a500 con 0x563de42f3fe0 2023-10-05T19:42:16.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.256+0000 7feffaffd700 1 -- 172.21.15.12:0/2557768836 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7feff4019070 con 0x563de42f3fe0 2023-10-05T19:42:16.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.256+0000 7ff00527ae40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:42:16.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.256+0000 7ff00527ae40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:42:16.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.256+0000 7feffaffd700 1 --2- 172.21.15.12:0/2557768836 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fefe8030c50 0x7fefe8033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.256+0000 7feffaffd700 1 -- 172.21.15.12:0/2557768836 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7feff402c070 con 0x563de42f3fe0 2023-10-05T19:42:16.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.256+0000 7ff00527ae40 1 Processor -- start 2023-10-05T19:42:16.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.256+0000 7ff0025ae700 1 --2- 172.21.15.12:0/2557768836 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fefe8030c50 0x7fefe8033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.256+0000 7ff00527ae40 1 -- start start 2023-10-05T19:42:16.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.256+0000 7ff00527ae40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563de432b0f0 0x563de432b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.256+0000 7ff00527ae40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x563de432ba00 con 0x563de432b0f0 2023-10-05T19:42:16.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.256+0000 7ff002daf700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563de432b0f0 0x563de432b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.256+0000 7ff0025ae700 1 --2- 172.21.15.12:0/2557768836 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fefe8030c50 0x7fefe8033100 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fefec001560 tx=0x7fefec006650 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.256+0000 7ff002daf700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563de432b0f0 0x563de432b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:54384/0 (socket says 172.21.15.12:54384) 2023-10-05T19:42:16.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.256+0000 7ff002daf700 1 -- 172.21.15.12:0/3657489264 learned_addr learned my addr 172.21.15.12:0/3657489264 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:42:16.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.256+0000 7ff002daf700 1 -- 172.21.15.12:0/3657489264 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563de432bb40 con 0x563de432b0f0 2023-10-05T19:42:16.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.256+0000 7ff002daf700 1 --2- 172.21.15.12:0/3657489264 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563de432b0f0 0x563de432b4c0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7feff403afb0 tx=0x7feff401dfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c457cbc9aeac5ea2 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.256+0000 7fefd37ee700 1 -- 172.21.15.12:0/3657489264 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feff4026e30 con 0x563de432b0f0 2023-10-05T19:42:16.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.256+0000 7fefd37ee700 1 -- 172.21.15.12:0/3657489264 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff4040040 con 0x563de432b0f0 2023-10-05T19:42:16.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.256+0000 7fefd37ee700 1 -- 172.21.15.12:0/3657489264 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feff4025dc0 con 0x563de432b0f0 2023-10-05T19:42:16.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.260+0000 7ff00527ae40 1 -- 172.21.15.12:0/3657489264 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563de432b0f0 msgr2=0x563de432b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.260+0000 7ff00527ae40 1 --2- 172.21.15.12:0/3657489264 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563de432b0f0 0x563de432b4c0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7feff403afb0 tx=0x7feff401dfd0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.260+0000 7ff00527ae40 1 -- 172.21.15.12:0/3657489264 shutdown_connections 2023-10-05T19:42:16.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.260+0000 7ff00527ae40 1 --2- 172.21.15.12:0/3657489264 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563de432b0f0 0x563de432b4c0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.260+0000 7ff00527ae40 1 -- 172.21.15.12:0/3657489264 >> 172.21.15.12:0/3657489264 conn(0x563de42f48f0 msgr2=0x563de4327e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:16.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.260+0000 7ff00527ae40 1 -- 172.21.15.12:0/3657489264 shutdown_connections 2023-10-05T19:42:16.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.260+0000 7ff00527ae40 1 -- 172.21.15.12:0/3657489264 wait complete. 2023-10-05T19:42:16.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.260+0000 7ff00527ae40 1 Processor -- start 2023-10-05T19:42:16.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.260+0000 7ff00527ae40 1 -- start start 2023-10-05T19:42:16.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.260+0000 7ff00527ae40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563de432b0f0 0x563de43321c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.260+0000 7ff00527ae40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x563de43331c0 con 0x563de432b0f0 2023-10-05T19:42:16.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.260+0000 7ff002daf700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563de432b0f0 0x563de43321c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.260+0000 7ff002daf700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563de432b0f0 0x563de43321c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:54398/0 (socket says 172.21.15.12:54398) 2023-10-05T19:42:16.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.260+0000 7ff002daf700 1 -- 172.21.15.12:0/1036759958 learned_addr learned my addr 172.21.15.12:0/1036759958 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:42:16.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.260+0000 7ff002daf700 1 -- 172.21.15.12:0/1036759958 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563de431a500 con 0x563de432b0f0 2023-10-05T19:42:16.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.260+0000 7ff002daf700 1 --2- 172.21.15.12:0/1036759958 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563de432b0f0 0x563de43321c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7feff402a040 tx=0x7feff404a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.260+0000 7fefd1feb700 1 -- 172.21.15.12:0/1036759958 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feff403a3f0 con 0x563de432b0f0 2023-10-05T19:42:16.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.264+0000 7fefd1feb700 1 -- 172.21.15.12:0/1036759958 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff4026610 con 0x563de432b0f0 2023-10-05T19:42:16.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.264+0000 7ff00527ae40 1 -- 172.21.15.12:0/1036759958 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563de43327c0 con 0x563de432b0f0 2023-10-05T19:42:16.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.264+0000 7fefd1feb700 1 -- 172.21.15.12:0/1036759958 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feff40267c0 con 0x563de432b0f0 2023-10-05T19:42:16.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.264+0000 7ff00527ae40 1 -- 172.21.15.12:0/1036759958 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563de4332c00 con 0x563de432b0f0 2023-10-05T19:42:16.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.264+0000 7fefd1feb700 1 -- 172.21.15.12:0/1036759958 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7feff40245b0 con 0x563de432b0f0 2023-10-05T19:42:16.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.264+0000 7fefd1feb700 1 --2- 172.21.15.12:0/1036759958 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fef50030c50 0x7fef50033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.264+0000 7fefd1feb700 1 -- 172.21.15.12:0/1036759958 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7feff405bbe0 con 0x563de432b0f0 2023-10-05T19:42:16.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.264+0000 7ff0025ae700 1 --2- 172.21.15.12:0/1036759958 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fef50030c50 0x7fef50033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.264+0000 7ff0025ae700 1 --2- 172.21.15.12:0/1036759958 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fef50030c50 0x7fef50033100 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fefec000f80 tx=0x7fefec010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.264+0000 7fef4b7fe700 20 reqs_thread_entry: start 2023-10-05T19:42:16.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.264+0000 7ff00527ae40 20 try_to_resolve_local_entities(): before: realm_id= realm_name= zonegroup_id= zonegroup_name= zone_id= zone_name= 2023-10-05T19:42:16.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.264+0000 7ff00527ae40 20 rados->read ofs=0 len=0 2023-10-05T19:42:16.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.268+0000 7ff00527ae40 1 --2- 172.21.15.12:0/2557768836 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x563de434fd10 0x563de4350180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.268+0000 7ff00527ae40 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de43506c0 con 0x563de434fd10 2023-10-05T19:42:16.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.268+0000 7ff0035b0700 1 --2- 172.21.15.12:0/2557768836 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x563de434fd10 0x563de4350180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.268+0000 7ff0035b0700 1 --2- 172.21.15.12:0/2557768836 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x563de434fd10 0x563de4350180 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.268+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7feffc00a480 con 0x563de434fd10 2023-10-05T19:42:16.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.268+0000 7ff00527ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:16.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.268+0000 7ff00527ae40 20 rados->read ofs=0 len=0 2023-10-05T19:42:16.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.268+0000 7ff00527ae40 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de4363250 con 0x563de434fd10 2023-10-05T19:42:16.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.268+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7feffc00a480 con 0x563de434fd10 2023-10-05T19:42:16.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.268+0000 7ff00527ae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:42:16.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.268+0000 7ff00527ae40 20 rados->read ofs=0 len=0 2023-10-05T19:42:16.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.268+0000 7ff00527ae40 1 --2- 172.21.15.12:0/2557768836 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x563de43524f0 0x563de4365b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.268+0000 7ff00527ae40 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:3 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de4366060 con 0x563de43524f0 2023-10-05T19:42:16.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.268+0000 7ff002daf700 1 --2- 172.21.15.12:0/2557768836 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x563de43524f0 0x563de4365b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.268+0000 7ff002daf700 1 --2- 172.21.15.12:0/2557768836 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x563de43524f0 0x563de4365b20 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.268+0000 7ff002daf700 1 -- 172.21.15.12:0/2557768836 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(3 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7feff4022020 con 0x563de43524f0 2023-10-05T19:42:16.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.268+0000 7ff00527ae40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:42:16.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.268+0000 7ff00527ae40 1 --2- 172.21.15.12:0/2557768836 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x563de4367930 0x563de4369d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.268+0000 7ff00527ae40 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:4 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de436a250 con 0x563de4367930 2023-10-05T19:42:16.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.272+0000 7ff0025ae700 1 --2- 172.21.15.12:0/2557768836 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x563de4367930 0x563de4369d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.272+0000 7ff0025ae700 1 --2- 172.21.15.12:0/2557768836 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x563de4367930 0x563de4369d10 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.272+0000 7ff0025ae700 1 -- 172.21.15.12:0/2557768836 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(4 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fefec0234d0 con 0x563de4367930 2023-10-05T19:42:16.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.272+0000 7ff0025ae700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de436a250 con 0x563de43524f0 2023-10-05T19:42:16.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.272+0000 7ff002daf700 1 -- 172.21.15.12:0/2557768836 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(5 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feff4022020 con 0x563de43524f0 2023-10-05T19:42:16.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.272+0000 7ff002daf700 1 --2- 172.21.15.12:0/2557768836 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7feff4063020 0x7feff4035920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.272+0000 7ff002daf700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:6 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de436a250 con 0x7feff4063020 2023-10-05T19:42:16.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.272+0000 7ff0035b0700 1 --2- 172.21.15.12:0/2557768836 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7feff4063020 0x7feff4035920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.276+0000 7ff0035b0700 1 --2- 172.21.15.12:0/2557768836 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7feff4063020 0x7feff4035920 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.276+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feffc00a480 con 0x7feff4063020 2023-10-05T19:42:16.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.276+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:7 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de436a250 con 0x563de434fd10 2023-10-05T19:42:16.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.276+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feffc00a480 con 0x563de434fd10 2023-10-05T19:42:16.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.276+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:8 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de436a250 con 0x563de434fd10 2023-10-05T19:42:16.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.276+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feffc00a480 con 0x563de434fd10 2023-10-05T19:42:16.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.276+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de436a250 con 0x563de434fd10 2023-10-05T19:42:16.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.276+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feffc00a480 con 0x563de434fd10 2023-10-05T19:42:16.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.276+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:10 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de436a250 con 0x7feff4063020 2023-10-05T19:42:16.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.280+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feffc00a480 con 0x7feff4063020 2023-10-05T19:42:16.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.280+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:11 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de436a250 con 0x563de43524f0 2023-10-05T19:42:16.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.280+0000 7ff002daf700 1 -- 172.21.15.12:0/2557768836 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(11 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feff4022020 con 0x563de43524f0 2023-10-05T19:42:16.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.280+0000 7ff002daf700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:12 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de436a250 con 0x563de43524f0 2023-10-05T19:42:16.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.280+0000 7ff002daf700 1 -- 172.21.15.12:0/2557768836 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feff4022020 con 0x563de43524f0 2023-10-05T19:42:16.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.280+0000 7ff002daf700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:13 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de436a250 con 0x563de434fd10 2023-10-05T19:42:16.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.280+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feffc00a480 con 0x563de434fd10 2023-10-05T19:42:16.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.280+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:14 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de436a250 con 0x7feff4063020 2023-10-05T19:42:16.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.280+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feffc00a480 con 0x7feff4063020 2023-10-05T19:42:16.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.280+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de436a250 con 0x563de434fd10 2023-10-05T19:42:16.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.284+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feffc00a480 con 0x563de434fd10 2023-10-05T19:42:16.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.284+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:16 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de436a250 con 0x563de43524f0 2023-10-05T19:42:16.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.284+0000 7ff002daf700 1 -- 172.21.15.12:0/2557768836 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feff4022020 con 0x563de43524f0 2023-10-05T19:42:16.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.284+0000 7ff002daf700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:17 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de436a250 con 0x563de4367930 2023-10-05T19:42:16.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.284+0000 7ff0025ae700 1 -- 172.21.15.12:0/2557768836 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fefec0234d0 con 0x563de4367930 2023-10-05T19:42:16.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.284+0000 7ff0025ae700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:18 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x7fefec01e040 con 0x563de4367930 2023-10-05T19:42:16.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.284+0000 7ff0025ae700 1 -- 172.21.15.12:0/2557768836 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fefec0234d0 con 0x563de4367930 2023-10-05T19:42:16.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.284+0000 7ff0025ae700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:19 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x7fefec01e040 con 0x7feff4063020 2023-10-05T19:42:16.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.284+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feffc00a480 con 0x7feff4063020 2023-10-05T19:42:16.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.284+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x7fefec01e040 con 0x563de4367930 2023-10-05T19:42:16.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.288+0000 7ff0025ae700 1 -- 172.21.15.12:0/2557768836 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fefec0234d0 con 0x563de4367930 2023-10-05T19:42:16.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.288+0000 7ff0025ae700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:21 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x7fefec01e040 con 0x563de4367930 2023-10-05T19:42:16.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.288+0000 7ff0025ae700 1 -- 172.21.15.12:0/2557768836 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fefec0234d0 con 0x563de4367930 2023-10-05T19:42:16.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.288+0000 7ff0025ae700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:22 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x7fefec01e040 con 0x563de434fd10 2023-10-05T19:42:16.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.288+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feffc00a480 con 0x563de434fd10 2023-10-05T19:42:16.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.288+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:23 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x7fefec01e040 con 0x563de434fd10 2023-10-05T19:42:16.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.288+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feffc00a480 con 0x563de434fd10 2023-10-05T19:42:16.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.288+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:24 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x7fefec01e040 con 0x7feff4063020 2023-10-05T19:42:16.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.288+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feffc00a480 con 0x7feff4063020 2023-10-05T19:42:16.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.288+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:25 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x7fefec01e040 con 0x563de4367930 2023-10-05T19:42:16.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.292+0000 7ff0025ae700 1 -- 172.21.15.12:0/2557768836 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(25 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fefec0234d0 con 0x563de4367930 2023-10-05T19:42:16.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.292+0000 7ff0025ae700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:26 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x7fefec01e040 con 0x563de43524f0 2023-10-05T19:42:16.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.292+0000 7ff002daf700 1 -- 172.21.15.12:0/2557768836 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feff4022020 con 0x563de43524f0 2023-10-05T19:42:16.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.292+0000 7ff002daf700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:27 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x7fefec01e040 con 0x563de434fd10 2023-10-05T19:42:16.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.292+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feffc00a480 con 0x563de434fd10 2023-10-05T19:42:16.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.292+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:28 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x7fefec01e040 con 0x7feff4063020 2023-10-05T19:42:16.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.292+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feffc00a480 con 0x7feff4063020 2023-10-05T19:42:16.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.292+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:29 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x7fefec01e040 con 0x563de4367930 2023-10-05T19:42:16.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.292+0000 7ff0025ae700 1 -- 172.21.15.12:0/2557768836 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(29 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fefec0234d0 con 0x563de4367930 2023-10-05T19:42:16.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.292+0000 7ff0025ae700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x7fefec01e040 con 0x7feff4063020 2023-10-05T19:42:16.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.296+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feffc00a480 con 0x7feff4063020 2023-10-05T19:42:16.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.296+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:31 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x7fefec01e040 con 0x563de43524f0 2023-10-05T19:42:16.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.296+0000 7ff002daf700 1 -- 172.21.15.12:0/2557768836 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feff4022020 con 0x563de43524f0 2023-10-05T19:42:16.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.296+0000 7ff002daf700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:32 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x7fefec01e040 con 0x563de4367930 2023-10-05T19:42:16.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.296+0000 7ff0025ae700 1 -- 172.21.15.12:0/2557768836 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fefec0234d0 con 0x563de4367930 2023-10-05T19:42:16.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.296+0000 7ff0025ae700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:33 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x7fefec01e040 con 0x7feff4063020 2023-10-05T19:42:16.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.296+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feffc00a480 con 0x7feff4063020 2023-10-05T19:42:16.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.296+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:34 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x7fefec01e040 con 0x563de434fd10 2023-10-05T19:42:16.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.300+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(34 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7feffc00a480 con 0x563de434fd10 2023-10-05T19:42:16.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.300+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:35 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x7fefec01e040 con 0x563de43524f0 2023-10-05T19:42:16.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.300+0000 7ff002daf700 1 -- 172.21.15.12:0/2557768836 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(35 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7feff4022020 con 0x563de43524f0 2023-10-05T19:42:16.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.300+0000 7ff00527ae40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:42:16.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.300+0000 7ff00527ae40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:42:16.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.300+0000 7ff00527ae40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:42:16.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.300+0000 7ff00527ae40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:42:16.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.300+0000 7ff00527ae40 20 rados->read ofs=0 len=0 2023-10-05T19:42:16.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.300+0000 7ff00527ae40 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de4366410 con 0x563de434fd10 2023-10-05T19:42:16.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.300+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7feffc00a480 con 0x563de434fd10 2023-10-05T19:42:16.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.300+0000 7ff00527ae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:42:16.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.300+0000 7ff00527ae40 20 rados->read ofs=0 len=0 2023-10-05T19:42:16.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.300+0000 7ff00527ae40 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de43667c0 con 0x563de43524f0 2023-10-05T19:42:16.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.300+0000 7ff002daf700 1 -- 172.21.15.12:0/2557768836 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(37 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7feff4022020 con 0x563de43524f0 2023-10-05T19:42:16.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.300+0000 7ff00527ae40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:42:16.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.300+0000 7ff00527ae40 20 rados->read ofs=0 len=0 2023-10-05T19:42:16.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.300+0000 7ff00527ae40 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de4354380 con 0x563de434fd10 2023-10-05T19:42:16.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.300+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7feffc00a480 con 0x563de434fd10 2023-10-05T19:42:16.307 INFO:teuthology.orchestra.run.smithi012.stderr:zone id 2023-10-05T19:42:16.300+0000 7ff00527ae40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:16.307 INFO:teuthology.orchestra.run.smithi012.stderr:93ac49f4-ff49-4f99-811b-6453b28decd52023-10-05T19:42:16.300+0000 7ff00527ae40 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:39 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de436eb50 con 0x563de4367930 2023-10-05T19:42:16.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.304+0000 7ff0025ae700 1 -- 172.21.15.12:0/2557768836 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(39 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fefec0234d0 con 0x563de4367930 2023-10-05T19:42:16.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.304+0000 7ff0025ae700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:40 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de436eb50 con 0x563de43524f0 2023-10-05T19:42:16.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.304+0000 7ff002daf700 1 -- 172.21.15.12:0/2557768836 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(40 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feff4022020 con 0x563de43524f0 2023-10-05T19:42:16.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.304+0000 7ff002daf700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:41 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de436eb50 con 0x7feff4063020 2023-10-05T19:42:16.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.308+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(41 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feffc00a480 con 0x7feff4063020 2023-10-05T19:42:16.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.308+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de436eb50 con 0x563de434fd10 2023-10-05T19:42:16.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.308+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(42 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feffc00a480 con 0x563de434fd10 2023-10-05T19:42:16.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.308+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de436eb50 con 0x563de434fd10 2023-10-05T19:42:16.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.308+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(43 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feffc00a480 con 0x563de434fd10 2023-10-05T19:42:16.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.308+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:44 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de436eb50 con 0x563de434fd10 2023-10-05T19:42:16.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.308+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(44 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feffc00a480 con 0x563de434fd10 2023-10-05T19:42:16.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.308+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:45 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de436eb50 con 0x7feff4063020 2023-10-05T19:42:16.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.308+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(45 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feffc00a480 con 0x7feff4063020 2023-10-05T19:42:16.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.308+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:46 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de436eb50 con 0x563de43524f0 2023-10-05T19:42:16.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.312+0000 7ff002daf700 1 -- 172.21.15.12:0/2557768836 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(46 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feff4022020 con 0x563de43524f0 2023-10-05T19:42:16.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.312+0000 7ff002daf700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:47 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de436eb50 con 0x563de43524f0 2023-10-05T19:42:16.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.312+0000 7ff002daf700 1 -- 172.21.15.12:0/2557768836 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(47 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feff4022020 con 0x563de43524f0 2023-10-05T19:42:16.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.312+0000 7ff002daf700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de436eb50 con 0x563de434fd10 2023-10-05T19:42:16.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.312+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(48 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feffc00a480 con 0x563de434fd10 2023-10-05T19:42:16.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.312+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:49 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de436eb50 con 0x7feff4063020 2023-10-05T19:42:16.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.312+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(49 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feffc00a480 con 0x7feff4063020 2023-10-05T19:42:16.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.312+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:50 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de436eb50 con 0x563de434fd10 2023-10-05T19:42:16.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.312+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(50 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feffc00a480 con 0x563de434fd10 2023-10-05T19:42:16.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.312+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de436eb50 con 0x563de43524f0 2023-10-05T19:42:16.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.316+0000 7ff002daf700 1 -- 172.21.15.12:0/2557768836 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(51 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feff4022020 con 0x563de43524f0 2023-10-05T19:42:16.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.316+0000 7ff002daf700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:52 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de436eb50 con 0x563de4367930 2023-10-05T19:42:16.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.316+0000 7ff0025ae700 1 -- 172.21.15.12:0/2557768836 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(52 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fefec0234d0 con 0x563de4367930 2023-10-05T19:42:16.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.316+0000 7ff0025ae700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:53 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de436eb50 con 0x563de4367930 2023-10-05T19:42:16.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.316+0000 7ff0025ae700 1 -- 172.21.15.12:0/2557768836 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(53 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fefec0234d0 con 0x563de4367930 2023-10-05T19:42:16.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.316+0000 7ff0025ae700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de436eb50 con 0x7feff4063020 2023-10-05T19:42:16.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.316+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(54 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feffc00a480 con 0x7feff4063020 2023-10-05T19:42:16.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.316+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:55 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de436eb50 con 0x563de4367930 2023-10-05T19:42:16.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.320+0000 7ff0025ae700 1 -- 172.21.15.12:0/2557768836 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(55 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fefec0234d0 con 0x563de4367930 2023-10-05T19:42:16.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.320+0000 7ff0025ae700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:56 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de436eb50 con 0x563de4367930 2023-10-05T19:42:16.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.320+0000 7ff0025ae700 1 -- 172.21.15.12:0/2557768836 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(56 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fefec0234d0 con 0x563de4367930 2023-10-05T19:42:16.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.320+0000 7ff0025ae700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:57 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de436eb50 con 0x563de434fd10 2023-10-05T19:42:16.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.320+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(57 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feffc00a480 con 0x563de434fd10 2023-10-05T19:42:16.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.320+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:58 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de436eb50 con 0x563de434fd10 2023-10-05T19:42:16.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.320+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(58 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feffc00a480 con 0x563de434fd10 2023-10-05T19:42:16.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.320+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de436eb50 con 0x7feff4063020 2023-10-05T19:42:16.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.320+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(59 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feffc00a480 con 0x7feff4063020 2023-10-05T19:42:16.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.320+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de436eb50 con 0x563de4367930 2023-10-05T19:42:16.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.324+0000 7ff0025ae700 1 -- 172.21.15.12:0/2557768836 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(60 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fefec0234d0 con 0x563de4367930 2023-10-05T19:42:16.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.324+0000 7ff0025ae700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:61 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de436eb50 con 0x563de43524f0 2023-10-05T19:42:16.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.324+0000 7ff002daf700 1 -- 172.21.15.12:0/2557768836 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(61 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feff4022020 con 0x563de43524f0 2023-10-05T19:42:16.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.324+0000 7ff002daf700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de436eb50 con 0x563de434fd10 2023-10-05T19:42:16.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.324+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(62 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feffc00a480 con 0x563de434fd10 2023-10-05T19:42:16.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.324+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:63 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de436eb50 con 0x7feff4063020 2023-10-05T19:42:16.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.324+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(63 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feffc00a480 con 0x7feff4063020 2023-10-05T19:42:16.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.324+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:64 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de436eb50 con 0x563de4367930 2023-10-05T19:42:16.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.328+0000 7ff0025ae700 1 -- 172.21.15.12:0/2557768836 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(64 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fefec0234d0 con 0x563de4367930 2023-10-05T19:42:16.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.328+0000 7ff0025ae700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:65 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de436eb50 con 0x7feff4063020 2023-10-05T19:42:16.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.328+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(65 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feffc00a480 con 0x7feff4063020 2023-10-05T19:42:16.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.328+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de436eb50 con 0x563de43524f0 2023-10-05T19:42:16.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.328+0000 7ff002daf700 1 -- 172.21.15.12:0/2557768836 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(66 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feff4022020 con 0x563de43524f0 2023-10-05T19:42:16.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.328+0000 7ff002daf700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:67 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de436eb50 con 0x563de4367930 2023-10-05T19:42:16.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.328+0000 7ff0025ae700 1 -- 172.21.15.12:0/2557768836 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(67 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fefec0234d0 con 0x563de4367930 2023-10-05T19:42:16.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.328+0000 7ff0025ae700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de436eb50 con 0x7feff4063020 2023-10-05T19:42:16.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.328+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(68 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feffc00a480 con 0x7feff4063020 2023-10-05T19:42:16.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.328+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:69 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de436eb50 con 0x563de434fd10 2023-10-05T19:42:16.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.332+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(69 [pgnls start_epoch 30 out=115b] v21'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7feffc00a480 con 0x563de434fd10 2023-10-05T19:42:16.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.332+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:70 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de436eb50 con 0x563de43524f0 2023-10-05T19:42:16.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.332+0000 7ff002daf700 1 -- 172.21.15.12:0/2557768836 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(70 [pgnls start_epoch 30 out=107b] v28'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7feff4022020 con 0x563de43524f0 2023-10-05T19:42:16.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.332+0000 7ff00527ae40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:42:16.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.332+0000 7ff00527ae40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:42:16.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.332+0000 7ff00527ae40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:42:16.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.332+0000 7ff00527ae40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:42:16.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.332+0000 7ff00527ae40 20 rados->read ofs=0 len=0 2023-10-05T19:42:16.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.332+0000 7ff00527ae40 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:71 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de4351980 con 0x563de434fd10 2023-10-05T19:42:16.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.332+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(71 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7feffc00a480 con 0x563de434fd10 2023-10-05T19:42:16.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.332+0000 7ff00527ae40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:42:16.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.332+0000 7ff00527ae40 20 rados->read ofs=0 len=0 2023-10-05T19:42:16.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.332+0000 7ff00527ae40 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:72 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de4373490 con 0x563de43524f0 2023-10-05T19:42:16.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.332+0000 7ff002daf700 1 -- 172.21.15.12:0/2557768836 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(72 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7feff4022020 con 0x563de43524f0 2023-10-05T19:42:16.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.332+0000 7ff00527ae40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:42:16.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.332+0000 7ff00527ae40 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:73 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [delete,create,writefull 0~1036 in=1036b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de4370a60 con 0x563de43524f0 2023-10-05T19:42:16.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.336+0000 7ff002daf700 1 -- 172.21.15.12:0/2557768836 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(73 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [delete,create,writefull 0~1036] v30'3 uv3 ondisk = 0) v8 ==== 274+0+0 (crc 0 0 0) 0x7feff4022020 con 0x563de43524f0 2023-10-05T19:42:16.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.336+0000 7ff00527ae40 1 -- 172.21.15.12:0/2557768836 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f43b6ece:::zone_names.default:head [delete,create,writefull 0~46 in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x563de4370a60 con 0x563de434fd10 2023-10-05T19:42:16.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.340+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(74 zone_names.default [delete,create,writefull 0~46] v30'3 uv3 ondisk = 0) v8 ==== 246+0+0 (crc 0 0 0) 0x7feffc00a480 con 0x563de434fd10 2023-10-05T19:42:16.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.344+0000 7ff00527ae40 1 -- 172.21.15.12:0/2557768836 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7feff4063020 msgr2=0x7feff4035920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.344+0000 7ff00527ae40 1 --2- 172.21.15.12:0/2557768836 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7feff4063020 0x7feff4035920 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.344+0000 7ff00527ae40 1 -- 172.21.15.12:0/2557768836 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x563de434fd10 msgr2=0x563de4350180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.344+0000 7ff00527ae40 1 --2- 172.21.15.12:0/2557768836 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x563de434fd10 0x563de4350180 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.344+0000 7ff00527ae40 1 -- 172.21.15.12:0/2557768836 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x563de43524f0 msgr2=0x563de4365b20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.344+0000 7ff00527ae40 1 --2- 172.21.15.12:0/2557768836 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x563de43524f0 0x563de4365b20 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.344+0000 7ff00527ae40 1 -- 172.21.15.12:0/2557768836 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x563de4367930 msgr2=0x563de4369d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.344+0000 7ff00527ae40 1 --2- 172.21.15.12:0/2557768836 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x563de4367930 0x563de4369d10 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.344+0000 7ff00527ae40 1 -- 172.21.15.12:0/2557768836 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fefe8030c50 msgr2=0x7fefe8033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.344+0000 7ff00527ae40 1 --2- 172.21.15.12:0/2557768836 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fefe8030c50 0x7fefe8033100 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fefec001560 tx=0x7fefec006650 comp rx=0 tx=0).stop 2023-10-05T19:42:16.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.344+0000 7ff0035b0700 1 -- 172.21.15.12:0/2557768836 reap_dead start 2023-10-05T19:42:16.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.344+0000 7ff00527ae40 1 -- 172.21.15.12:0/2557768836 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563de42f3fe0 msgr2=0x563de4319b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.344+0000 7ff00527ae40 1 --2- 172.21.15.12:0/2557768836 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563de42f3fe0 0x563de4319b20 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7feff401e040 tx=0x7feff401c000 comp rx=0 tx=0).stop 2023-10-05T19:42:16.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.344+0000 7ff00527ae40 1 -- 172.21.15.12:0/2557768836 shutdown_connections 2023-10-05T19:42:16.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.344+0000 7ff00527ae40 1 --2- 172.21.15.12:0/2557768836 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563de42f3fe0 0x563de4319b20 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.344+0000 7ff00527ae40 1 -- 172.21.15.12:0/2557768836 >> 172.21.15.12:0/2557768836 conn(0x563de4267830 msgr2=0x563de430d550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:16.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.344+0000 7ff00527ae40 1 -- 172.21.15.12:0/2557768836 shutdown_connections 2023-10-05T19:42:16.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.344+0000 7ff00527ae40 1 -- 172.21.15.12:0/2557768836 wait complete. 2023-10-05T19:42:16.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.344+0000 7ff00527ae40 1 -- 172.21.15.12:0/1036759958 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fef50030c50 msgr2=0x7fef50033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.344+0000 7ff00527ae40 1 --2- 172.21.15.12:0/1036759958 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fef50030c50 0x7fef50033100 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fefec000f80 tx=0x7fefec010000 comp rx=0 tx=0).stop 2023-10-05T19:42:16.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.344+0000 7ff00527ae40 1 -- 172.21.15.12:0/1036759958 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563de432b0f0 msgr2=0x563de43321c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.344+0000 7ff00527ae40 1 --2- 172.21.15.12:0/1036759958 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563de432b0f0 0x563de43321c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7feff402a040 tx=0x7feff404a000 comp rx=0 tx=0).stop 2023-10-05T19:42:16.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.344+0000 7ff00527ae40 1 -- 172.21.15.12:0/1036759958 shutdown_connections 2023-10-05T19:42:16.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.344+0000 7ff00527ae40 1 --2- 172.21.15.12:0/1036759958 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fef50030c50 0x7fef50033100 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.344+0000 7ff00527ae40 1 --2- 172.21.15.12:0/1036759958 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563de432b0f0 0x563de43321c0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.344+0000 7ff00527ae40 1 -- 172.21.15.12:0/1036759958 >> 172.21.15.12:0/1036759958 conn(0x563de42f48f0 msgr2=0x563de4327ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:16.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.344+0000 7ff00527ae40 1 -- 172.21.15.12:0/1036759958 shutdown_connections 2023-10-05T19:42:16.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.344+0000 7ff00527ae40 1 -- 172.21.15.12:0/1036759958 wait complete. 2023-10-05T19:42:16.352 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:42:16.352 INFO:teuthology.orchestra.run.smithi012.stdout: "id": "93ac49f4-ff49-4f99-811b-6453b28decd5", 2023-10-05T19:42:16.352 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default", 2023-10-05T19:42:16.352 INFO:teuthology.orchestra.run.smithi012.stdout: "domain_root": "default.rgw.meta:root", 2023-10-05T19:42:16.352 INFO:teuthology.orchestra.run.smithi012.stdout: "control_pool": "default.rgw.control", 2023-10-05T19:42:16.352 INFO:teuthology.orchestra.run.smithi012.stdout: "gc_pool": "default.rgw.log:gc", 2023-10-05T19:42:16.352 INFO:teuthology.orchestra.run.smithi012.stdout: "lc_pool": "default.rgw.log:lc", 2023-10-05T19:42:16.353 INFO:teuthology.orchestra.run.smithi012.stdout: "log_pool": "default.rgw.log", 2023-10-05T19:42:16.353 INFO:teuthology.orchestra.run.smithi012.stdout: "intent_log_pool": "default.rgw.log:intent", 2023-10-05T19:42:16.353 INFO:teuthology.orchestra.run.smithi012.stdout: "usage_log_pool": "default.rgw.log:usage", 2023-10-05T19:42:16.353 INFO:teuthology.orchestra.run.smithi012.stdout: "roles_pool": "default.rgw.meta:roles", 2023-10-05T19:42:16.353 INFO:teuthology.orchestra.run.smithi012.stdout: "reshard_pool": "default.rgw.log:reshard", 2023-10-05T19:42:16.353 INFO:teuthology.orchestra.run.smithi012.stdout: "user_keys_pool": "default.rgw.meta:users.keys", 2023-10-05T19:42:16.353 INFO:teuthology.orchestra.run.smithi012.stdout: "user_email_pool": "default.rgw.meta:users.email", 2023-10-05T19:42:16.353 INFO:teuthology.orchestra.run.smithi012.stdout: "user_swift_pool": "default.rgw.meta:users.swift", 2023-10-05T19:42:16.353 INFO:teuthology.orchestra.run.smithi012.stdout: "user_uid_pool": "default.rgw.meta:users.uid", 2023-10-05T19:42:16.353 INFO:teuthology.orchestra.run.smithi012.stdout: "otp_pool": "default.rgw.otp", 2023-10-05T19:42:16.353 INFO:teuthology.orchestra.run.smithi012.stdout: "system_key": { 2023-10-05T19:42:16.354 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key": "", 2023-10-05T19:42:16.354 INFO:teuthology.orchestra.run.smithi012.stdout: "secret_key": "" 2023-10-05T19:42:16.354 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:42:16.354 INFO:teuthology.orchestra.run.smithi012.stdout: "placement_pools": [ 2023-10-05T19:42:16.354 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:42:16.354 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "default-placement", 2023-10-05T19:42:16.354 INFO:teuthology.orchestra.run.smithi012.stdout: "val": { 2023-10-05T19:42:16.354 INFO:teuthology.orchestra.run.smithi012.stdout: "index_pool": "default.rgw.buckets.index", 2023-10-05T19:42:16.354 INFO:teuthology.orchestra.run.smithi012.stdout: "storage_classes": { 2023-10-05T19:42:16.354 INFO:teuthology.orchestra.run.smithi012.stdout: "STANDARD": { 2023-10-05T19:42:16.354 INFO:teuthology.orchestra.run.smithi012.stdout: "data_pool": "default.rgw.buckets.data" 2023-10-05T19:42:16.355 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:42:16.355 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:42:16.355 INFO:teuthology.orchestra.run.smithi012.stdout: "data_extra_pool": "default.rgw.buckets.non-ec", 2023-10-05T19:42:16.355 INFO:teuthology.orchestra.run.smithi012.stdout: "index_type": 0, 2023-10-05T19:42:16.355 INFO:teuthology.orchestra.run.smithi012.stdout: "inline_data": "true" 2023-10-05T19:42:16.355 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:42:16.355 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:42:16.355 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:42:16.355 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "new-placement", 2023-10-05T19:42:16.355 INFO:teuthology.orchestra.run.smithi012.stdout: "val": { 2023-10-05T19:42:16.355 INFO:teuthology.orchestra.run.smithi012.stdout: "index_pool": ".rgw.buckets.index.2", 2023-10-05T19:42:16.355 INFO:teuthology.orchestra.run.smithi012.stdout: "storage_classes": { 2023-10-05T19:42:16.356 INFO:teuthology.orchestra.run.smithi012.stdout: "STANDARD": { 2023-10-05T19:42:16.356 INFO:teuthology.orchestra.run.smithi012.stdout: "data_pool": ".rgw.buckets.2" 2023-10-05T19:42:16.356 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:42:16.356 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:42:16.356 INFO:teuthology.orchestra.run.smithi012.stdout: "data_extra_pool": "default.rgw.buckets.non-ec", 2023-10-05T19:42:16.356 INFO:teuthology.orchestra.run.smithi012.stdout: "index_type": 0, 2023-10-05T19:42:16.356 INFO:teuthology.orchestra.run.smithi012.stdout: "inline_data": "false" 2023-10-05T19:42:16.356 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:42:16.356 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:42:16.356 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:42:16.357 INFO:teuthology.orchestra.run.smithi012.stdout: "realm_id": "", 2023-10-05T19:42:16.357 INFO:teuthology.orchestra.run.smithi012.stdout: "notif_pool": "default.rgw.log:notif" 2023-10-05T19:42:16.357 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:42:16.358 DEBUG:tasks.util.rgw: json result: {'id': '93ac49f4-ff49-4f99-811b-6453b28decd5', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': 'true'}}, {'key': 'new-placement', 'val': {'index_pool': '.rgw.buckets.index.2', 'storage_classes': {'STANDARD': {'data_pool': '.rgw.buckets.2'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': 'false'}}], 'realm_id': '', 'notif_pool': 'default.rgw.log:notif'} 2023-10-05T19:42:16.359 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'get'] 2023-10-05T19:42:16.359 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'get'] 2023-10-05T19:42:16.359 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone get 2023-10-05T19:42:16.396 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:42:16.396 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:42:16.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.392+0000 7fceba430e40 1 Processor -- start 2023-10-05T19:42:16.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.392+0000 7fceba430e40 1 -- start start 2023-10-05T19:42:16.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.392+0000 7fceba430e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55cd195d05c0 0x55cd195d29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.392+0000 7fceba430e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55cd19516c00 con 0x55cd195cbfe0 2023-10-05T19:42:16.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.392+0000 7fceba430e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x55cd194d3ee0 con 0x55cd195d05c0 2023-10-05T19:42:16.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.392+0000 7fceb7764700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55cd195d05c0 0x55cd195d29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.392+0000 7fceb7764700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55cd195d05c0 0x55cd195d29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:54402/0 (socket says 172.21.15.12:54402) 2023-10-05T19:42:16.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.392+0000 7fceb7764700 1 -- 172.21.15.12:0/3745818487 learned_addr learned my addr 172.21.15.12:0/3745818487 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:42:16.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.392+0000 7fceb7764700 1 -- 172.21.15.12:0/3745818487 >> v1:172.21.15.8:6789/0 conn(0x55cd195cbfe0 legacy=0x55cd195cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.392+0000 7fceb7764700 1 -- 172.21.15.12:0/3745818487 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd195ccbf0 con 0x55cd195d05c0 2023-10-05T19:42:16.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.392+0000 7fceb7764700 1 --2- 172.21.15.12:0/3745818487 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55cd195d05c0 0x55cd195d29a0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fceac00a650 tx=0x7fceac005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b0b90d43c770582 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.392+0000 7fceb6f63700 1 -- 172.21.15.12:0/3745818487 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fceac008760 con 0x55cd195d05c0 2023-10-05T19:42:16.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.392+0000 7fceb6f63700 1 -- 172.21.15.12:0/3745818487 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceac0088c0 con 0x55cd195d05c0 2023-10-05T19:42:16.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.392+0000 7fceb6f63700 1 -- 172.21.15.12:0/3745818487 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fceac008bb0 con 0x55cd195d05c0 2023-10-05T19:42:16.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.396+0000 7fceba430e40 1 -- 172.21.15.12:0/3745818487 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55cd195d05c0 msgr2=0x55cd195d29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.396+0000 7fceba430e40 1 --2- 172.21.15.12:0/3745818487 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55cd195d05c0 0x55cd195d29a0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fceac00a650 tx=0x7fceac005b10 comp rx=0 tx=0).stop 2023-10-05T19:42:16.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.396+0000 7fceba430e40 1 -- 172.21.15.12:0/3745818487 shutdown_connections 2023-10-05T19:42:16.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.396+0000 7fceba430e40 1 --2- 172.21.15.12:0/3745818487 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55cd195d05c0 0x55cd195d29a0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.396+0000 7fceba430e40 1 -- 172.21.15.12:0/3745818487 >> 172.21.15.12:0/3745818487 conn(0x55cd1953f830 msgr2=0x55cd1953fc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:16.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.396+0000 7fceba430e40 1 -- 172.21.15.12:0/3745818487 shutdown_connections 2023-10-05T19:42:16.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.396+0000 7fceba430e40 1 -- 172.21.15.12:0/3745818487 wait complete. 2023-10-05T19:42:16.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.396+0000 7fceba430e40 1 Processor -- start 2023-10-05T19:42:16.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.396+0000 7fceba430e40 1 -- start start 2023-10-05T19:42:16.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.396+0000 7fceba430e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55cd195cbfe0 0x55cd195eb160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.396+0000 7fceba430e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd19497e30 con 0x55cd195cbfe0 2023-10-05T19:42:16.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.396+0000 7fceb7f65700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55cd195cbfe0 0x55cd195eb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.396+0000 7fceb7f65700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55cd195cbfe0 0x55cd195eb160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:54408/0 (socket says 172.21.15.12:54408) 2023-10-05T19:42:16.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.396+0000 7fceb7f65700 1 -- 172.21.15.12:0/1045676180 learned_addr learned my addr 172.21.15.12:0/1045676180 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:42:16.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.396+0000 7fceb7f65700 1 -- 172.21.15.12:0/1045676180 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceac005db0 con 0x55cd195cbfe0 2023-10-05T19:42:16.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.396+0000 7fceb7f65700 1 --2- 172.21.15.12:0/1045676180 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55cd195cbfe0 0x55cd195eb160 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fcea8006fd0 tx=0x7fcea8006ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f20f88c9ef0699 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.396+0000 7fceb5f61700 1 -- 172.21.15.12:0/1045676180 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcea8004130 con 0x55cd195cbfe0 2023-10-05T19:42:16.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.396+0000 7fceb5f61700 1 -- 172.21.15.12:0/1045676180 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcea800e470 con 0x55cd195cbfe0 2023-10-05T19:42:16.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.396+0000 7fceb5f61700 1 -- 172.21.15.12:0/1045676180 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcea800e760 con 0x55cd195cbfe0 2023-10-05T19:42:16.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.400+0000 7fceba430e40 1 -- 172.21.15.12:0/1045676180 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55cd195cbfe0 msgr2=0x55cd195eb160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.400+0000 7fceba430e40 1 --2- 172.21.15.12:0/1045676180 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55cd195cbfe0 0x55cd195eb160 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fcea8006fd0 tx=0x7fcea8006ba0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.400+0000 7fceba430e40 1 -- 172.21.15.12:0/1045676180 shutdown_connections 2023-10-05T19:42:16.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.400+0000 7fceba430e40 1 --2- 172.21.15.12:0/1045676180 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55cd195cbfe0 0x55cd195eb160 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.400+0000 7fceba430e40 1 -- 172.21.15.12:0/1045676180 >> 172.21.15.12:0/1045676180 conn(0x55cd1953f830 msgr2=0x55cd195e6d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:16.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.400+0000 7fceba430e40 1 -- 172.21.15.12:0/1045676180 shutdown_connections 2023-10-05T19:42:16.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.400+0000 7fceba430e40 1 -- 172.21.15.12:0/1045676180 wait complete. 2023-10-05T19:42:16.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.400+0000 7fceba430e40 1 Processor -- start 2023-10-05T19:42:16.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.400+0000 7fceba430e40 1 -- start start 2023-10-05T19:42:16.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.400+0000 7fceba430e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55cd195cbfe0 0x55cd195f1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.400+0000 7fceba430e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd1946aed0 con 0x55cd195cbfe0 2023-10-05T19:42:16.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.400+0000 7fceb7f65700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55cd195cbfe0 0x55cd195f1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.400+0000 7fceb7f65700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55cd195cbfe0 0x55cd195f1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:54424/0 (socket says 172.21.15.12:54424) 2023-10-05T19:42:16.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.400+0000 7fceb7f65700 1 -- 172.21.15.12:0/3394690014 learned_addr learned my addr 172.21.15.12:0/3394690014 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:42:16.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.400+0000 7fceb7f65700 1 -- 172.21.15.12:0/3394690014 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd19516c00 con 0x55cd195cbfe0 2023-10-05T19:42:16.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.400+0000 7fceb7f65700 1 --2- 172.21.15.12:0/3394690014 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55cd195cbfe0 0x55cd195f1f30 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x55cd195e7840 tx=0x7fcea8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.400+0000 7fce9ffff700 1 -- 172.21.15.12:0/3394690014 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcea8004130 con 0x55cd195cbfe0 2023-10-05T19:42:16.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.400+0000 7fce9ffff700 1 -- 172.21.15.12:0/3394690014 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcea8018070 con 0x55cd195cbfe0 2023-10-05T19:42:16.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.400+0000 7fceba430e40 1 -- 172.21.15.12:0/3394690014 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55cd195f2470 con 0x55cd195cbfe0 2023-10-05T19:42:16.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.400+0000 7fce9ffff700 1 -- 172.21.15.12:0/3394690014 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcea8006260 con 0x55cd195cbfe0 2023-10-05T19:42:16.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.400+0000 7fceba430e40 1 -- 172.21.15.12:0/3394690014 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd195f2850 con 0x55cd195cbfe0 2023-10-05T19:42:16.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.404+0000 7fceba430e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:42:16.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.404+0000 7fceba430e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:42:16.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.404+0000 7fceba430e40 1 Processor -- start 2023-10-05T19:42:16.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.404+0000 7fceba430e40 1 -- start start 2023-10-05T19:42:16.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.404+0000 7fce9ffff700 1 -- 172.21.15.12:0/3394690014 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fcea8013990 con 0x55cd195cbfe0 2023-10-05T19:42:16.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.404+0000 7fceba430e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55cd19603660 0x55cd19603a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.404+0000 7fceba430e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd19603f70 con 0x55cd19603660 2023-10-05T19:42:16.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.404+0000 7fce9ffff700 1 --2- 172.21.15.12:0/3394690014 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fcea4030c50 0x7fcea4033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.404+0000 7fceb8766700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55cd19603660 0x55cd19603a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.404+0000 7fceb8766700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55cd19603660 0x55cd19603a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:54428/0 (socket says 172.21.15.12:54428) 2023-10-05T19:42:16.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.404+0000 7fce9ffff700 1 -- 172.21.15.12:0/3394690014 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7fcea80447f0 con 0x55cd195cbfe0 2023-10-05T19:42:16.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.404+0000 7fceb8766700 1 -- 172.21.15.12:0/3251109628 learned_addr learned my addr 172.21.15.12:0/3251109628 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:42:16.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.404+0000 7fceb7f65700 1 --2- 172.21.15.12:0/3394690014 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fcea4030c50 0x7fcea4033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.404+0000 7fceb8766700 1 -- 172.21.15.12:0/3251109628 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd196040b0 con 0x55cd19603660 2023-10-05T19:42:16.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.404+0000 7fceb8766700 1 --2- 172.21.15.12:0/3251109628 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55cd19603660 0x55cd19603a30 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fceb000a320 tx=0x7fceb00059d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=244e49c18e5ffef9 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.404+0000 7fceb7f65700 1 --2- 172.21.15.12:0/3394690014 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fcea4030c50 0x7fcea4033100 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcea8010560 tx=0x7fcea8042bf0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.404+0000 7fce897e2700 1 -- 172.21.15.12:0/3251109628 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fceb000ae30 con 0x55cd19603660 2023-10-05T19:42:16.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.404+0000 7fce897e2700 1 -- 172.21.15.12:0/3251109628 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceb000c610 con 0x55cd19603660 2023-10-05T19:42:16.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.404+0000 7fce897e2700 1 -- 172.21.15.12:0/3251109628 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fceb000c7c0 con 0x55cd19603660 2023-10-05T19:42:16.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.404+0000 7fceba430e40 1 -- 172.21.15.12:0/3251109628 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55cd19603660 msgr2=0x55cd19603a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.404+0000 7fceba430e40 1 --2- 172.21.15.12:0/3251109628 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55cd19603660 0x55cd19603a30 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fceb000a320 tx=0x7fceb00059d0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.404+0000 7fceba430e40 1 -- 172.21.15.12:0/3251109628 shutdown_connections 2023-10-05T19:42:16.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.404+0000 7fceba430e40 1 --2- 172.21.15.12:0/3251109628 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55cd19603660 0x55cd19603a30 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.404+0000 7fceba430e40 1 -- 172.21.15.12:0/3251109628 >> 172.21.15.12:0/3251109628 conn(0x55cd195d05c0 msgr2=0x55cd196003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:16.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.404+0000 7fceba430e40 1 -- 172.21.15.12:0/3251109628 shutdown_connections 2023-10-05T19:42:16.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.404+0000 7fceba430e40 1 -- 172.21.15.12:0/3251109628 wait complete. 2023-10-05T19:42:16.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.408+0000 7fceba430e40 1 Processor -- start 2023-10-05T19:42:16.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.408+0000 7fceba430e40 1 -- start start 2023-10-05T19:42:16.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.408+0000 7fceba430e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55cd19603660 0x55cd1960a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.408+0000 7fceba430e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd1960b840 con 0x55cd19603660 2023-10-05T19:42:16.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.408+0000 7fceb8766700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55cd19603660 0x55cd1960a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.408+0000 7fceb8766700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55cd19603660 0x55cd1960a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:54432/0 (socket says 172.21.15.12:54432) 2023-10-05T19:42:16.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.408+0000 7fceb8766700 1 -- 172.21.15.12:0/1384388392 learned_addr learned my addr 172.21.15.12:0/1384388392 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:42:16.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.408+0000 7fceb8766700 1 -- 172.21.15.12:0/1384388392 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceb0005c90 con 0x55cd19603660 2023-10-05T19:42:16.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.408+0000 7fceb8766700 1 --2- 172.21.15.12:0/1384388392 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55cd19603660 0x55cd1960a840 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x55cd19600b00 tx=0x7fceb0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.408+0000 7fce137fe700 1 -- 172.21.15.12:0/1384388392 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fceb0015660 con 0x55cd19603660 2023-10-05T19:42:16.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.408+0000 7fceba430e40 1 -- 172.21.15.12:0/1384388392 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55cd1960ae40 con 0x55cd19603660 2023-10-05T19:42:16.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.408+0000 7fce137fe700 1 -- 172.21.15.12:0/1384388392 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceb00157c0 con 0x55cd19603660 2023-10-05T19:42:16.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.408+0000 7fce137fe700 1 -- 172.21.15.12:0/1384388392 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fceb0015ab0 con 0x55cd19603660 2023-10-05T19:42:16.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.408+0000 7fceba430e40 1 -- 172.21.15.12:0/1384388392 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd1960b300 con 0x55cd19603660 2023-10-05T19:42:16.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.412+0000 7fce137fe700 1 -- 172.21.15.12:0/1384388392 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fceb0007050 con 0x55cd19603660 2023-10-05T19:42:16.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.412+0000 7fce117fa700 20 reqs_thread_entry: start 2023-10-05T19:42:16.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.412+0000 7fceba430e40 20 try_to_resolve_local_entities(): before: realm_id= realm_name= zonegroup_id= zonegroup_name= zone_id= zone_name= 2023-10-05T19:42:16.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.412+0000 7fceba430e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:16.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.412+0000 7fceba430e40 1 --2- 172.21.15.12:0/3394690014 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55cd19628560 0x55cd196289e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.412+0000 7fceba430e40 1 -- 172.21.15.12:0/3394690014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55cd19628f20 con 0x55cd19628560 2023-10-05T19:42:16.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.412+0000 7fce137fe700 1 --2- 172.21.15.12:0/1384388392 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fcdfc030c00 0x7fcdfc0330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.412+0000 7fce137fe700 1 -- 172.21.15.12:0/1384388392 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7fceb0044140 con 0x55cd19603660 2023-10-05T19:42:16.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.412+0000 7fceb7764700 1 --2- 172.21.15.12:0/3394690014 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55cd19628560 0x55cd196289e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.412+0000 7fceb8766700 1 --2- 172.21.15.12:0/1384388392 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fcdfc030c00 0x7fcdfc0330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.412+0000 7fceb7764700 1 --2- 172.21.15.12:0/3394690014 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55cd19628560 0x55cd196289e0 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.412+0000 7fceb8766700 1 --2- 172.21.15.12:0/1384388392 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fcdfc030c00 0x7fcdfc0330b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fceb0005600 tx=0x7fceb00424a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.412+0000 7fceb7764700 1 -- 172.21.15.12:0/3394690014 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fceac008c30 con 0x55cd19628560 2023-10-05T19:42:16.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.412+0000 7fceba430e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:16.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.412+0000 7fceba430e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:16.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.412+0000 7fceba430e40 1 -- 172.21.15.12:0/3394690014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55cd1963baa0 con 0x55cd19628560 2023-10-05T19:42:16.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.412+0000 7fceb7764700 1 -- 172.21.15.12:0/3394690014 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fceac008c30 con 0x55cd19628560 2023-10-05T19:42:16.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.412+0000 7fceba430e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:42:16.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.412+0000 7fceba430e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:16.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.412+0000 7fceba430e40 1 --2- 172.21.15.12:0/3394690014 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55cd1962ad40 0x55cd1963e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.412+0000 7fceba430e40 1 -- 172.21.15.12:0/3394690014 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:3 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55cd1963e8b0 con 0x55cd1962ad40 2023-10-05T19:42:16.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.412+0000 7fceb7f65700 1 --2- 172.21.15.12:0/3394690014 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55cd1962ad40 0x55cd1963e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.412+0000 7fceb7f65700 1 --2- 172.21.15.12:0/3394690014 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55cd1962ad40 0x55cd1963e370 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.416+0000 7fceb7f65700 1 -- 172.21.15.12:0/3394690014 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(3 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~1036 out=1036b] v0'0 uv3 ondisk = 0) v8 ==== 190+0+1036 (crc 0 0 0) 0x7fcea801d070 con 0x55cd1962ad40 2023-10-05T19:42:16.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.416+0000 7fceba430e40 20 rados_obj.operate() r=0 bl.length=1036 2023-10-05T19:42:16.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.416+0000 7fceba430e40 1 --2- 172.21.15.12:0/3394690014 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55cd19640180 0x55cd19642560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.416+0000 7fceba430e40 1 -- 172.21.15.12:0/3394690014 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:4 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55cd19642aa0 con 0x55cd19640180 2023-10-05T19:42:16.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.416+0000 7fceb7764700 1 --2- 172.21.15.12:0/3394690014 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55cd19640180 0x55cd19642560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.416+0000 7fceb7764700 1 --2- 172.21.15.12:0/3394690014 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55cd19640180 0x55cd19642560 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.420+0000 7fceb7764700 1 -- 172.21.15.12:0/3394690014 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(4 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fceac008c30 con 0x55cd19640180 2023-10-05T19:42:16.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.420+0000 7fceb7764700 1 -- 172.21.15.12:0/3394690014 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55cd19642aa0 con 0x55cd1962ad40 2023-10-05T19:42:16.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.420+0000 7fceb7f65700 1 -- 172.21.15.12:0/3394690014 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(5 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcea801d070 con 0x55cd1962ad40 2023-10-05T19:42:16.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.420+0000 7fceb7f65700 1 --2- 172.21.15.12:0/3394690014 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fcea801f740 0x7fcea80423a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.420+0000 7fceb7f65700 1 -- 172.21.15.12:0/3394690014 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:6 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55cd19642aa0 con 0x7fcea801f740 2023-10-05T19:42:16.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.420+0000 7fceb8766700 1 --2- 172.21.15.12:0/3394690014 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fcea801f740 0x7fcea80423a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.420+0000 7fceb8766700 1 --2- 172.21.15.12:0/3394690014 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fcea801f740 0x7fcea80423a0 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.420+0000 7fceb8766700 1 -- 172.21.15.12:0/3394690014 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fceb0006d80 con 0x7fcea801f740 2023-10-05T19:42:16.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.420+0000 7fceb8766700 1 -- 172.21.15.12:0/3394690014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:7 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55cd19642aa0 con 0x55cd19628560 2023-10-05T19:42:16.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.424+0000 7fceb7764700 1 -- 172.21.15.12:0/3394690014 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fceac008c30 con 0x55cd19628560 2023-10-05T19:42:16.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.424+0000 7fceb7764700 1 -- 172.21.15.12:0/3394690014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:8 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55cd19642aa0 con 0x55cd19628560 2023-10-05T19:42:16.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.424+0000 7fceb7764700 1 -- 172.21.15.12:0/3394690014 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fceac008c30 con 0x55cd19628560 2023-10-05T19:42:16.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.424+0000 7fceb7764700 1 -- 172.21.15.12:0/3394690014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55cd19642aa0 con 0x55cd19628560 2023-10-05T19:42:16.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.424+0000 7fceb7764700 1 -- 172.21.15.12:0/3394690014 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fceac008c30 con 0x55cd19628560 2023-10-05T19:42:16.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.424+0000 7fceb7764700 1 -- 172.21.15.12:0/3394690014 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:10 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55cd19642aa0 con 0x7fcea801f740 2023-10-05T19:42:16.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.424+0000 7fceb8766700 1 -- 172.21.15.12:0/3394690014 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fceb0006d80 con 0x7fcea801f740 2023-10-05T19:42:16.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.424+0000 7fceb8766700 1 -- 172.21.15.12:0/3394690014 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:11 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55cd19642aa0 con 0x55cd1962ad40 2023-10-05T19:42:16.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.424+0000 7fceb7f65700 1 -- 172.21.15.12:0/3394690014 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(11 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcea801d070 con 0x55cd1962ad40 2023-10-05T19:42:16.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.424+0000 7fceb7f65700 1 -- 172.21.15.12:0/3394690014 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:12 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55cd19642aa0 con 0x55cd1962ad40 2023-10-05T19:42:16.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.428+0000 7fceb7f65700 1 -- 172.21.15.12:0/3394690014 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcea801d070 con 0x55cd1962ad40 2023-10-05T19:42:16.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.428+0000 7fceb7f65700 1 -- 172.21.15.12:0/3394690014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:13 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55cd19642aa0 con 0x55cd19628560 2023-10-05T19:42:16.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.428+0000 7fceb7764700 1 -- 172.21.15.12:0/3394690014 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fceac008c30 con 0x55cd19628560 2023-10-05T19:42:16.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.428+0000 7fceb7764700 1 -- 172.21.15.12:0/3394690014 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:14 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55cd19642aa0 con 0x7fcea801f740 2023-10-05T19:42:16.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.428+0000 7fceb8766700 1 -- 172.21.15.12:0/3394690014 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fceb0006d80 con 0x7fcea801f740 2023-10-05T19:42:16.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.428+0000 7fceb8766700 1 -- 172.21.15.12:0/3394690014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55cd19642aa0 con 0x55cd19628560 2023-10-05T19:42:16.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.428+0000 7fceb7764700 1 -- 172.21.15.12:0/3394690014 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fceac008c30 con 0x55cd19628560 2023-10-05T19:42:16.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.428+0000 7fceb7764700 1 -- 172.21.15.12:0/3394690014 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:16 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55cd19642aa0 con 0x55cd1962ad40 2023-10-05T19:42:16.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.428+0000 7fceb7f65700 1 -- 172.21.15.12:0/3394690014 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcea801d070 con 0x55cd1962ad40 2023-10-05T19:42:16.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.428+0000 7fceb7f65700 1 -- 172.21.15.12:0/3394690014 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:17 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55cd19642aa0 con 0x55cd19640180 2023-10-05T19:42:16.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.432+0000 7fceb7764700 1 -- 172.21.15.12:0/3394690014 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fceac008c30 con 0x55cd19640180 2023-10-05T19:42:16.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.432+0000 7fceb7764700 1 -- 172.21.15.12:0/3394690014 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:18 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55cd19642aa0 con 0x55cd19640180 2023-10-05T19:42:16.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.432+0000 7fceb7764700 1 -- 172.21.15.12:0/3394690014 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fceac008c30 con 0x55cd19640180 2023-10-05T19:42:16.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.432+0000 7fceb7764700 1 -- 172.21.15.12:0/3394690014 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:19 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55cd19642aa0 con 0x7fcea801f740 2023-10-05T19:42:16.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.432+0000 7fceb8766700 1 -- 172.21.15.12:0/3394690014 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fceb0006d80 con 0x7fcea801f740 2023-10-05T19:42:16.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.432+0000 7fceb8766700 1 -- 172.21.15.12:0/3394690014 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55cd19642aa0 con 0x55cd19640180 2023-10-05T19:42:16.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.432+0000 7fceb7764700 1 -- 172.21.15.12:0/3394690014 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fceac008c30 con 0x55cd19640180 2023-10-05T19:42:16.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.432+0000 7fceb7764700 1 -- 172.21.15.12:0/3394690014 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:21 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55cd19642aa0 con 0x55cd19640180 2023-10-05T19:42:16.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.432+0000 7fceb7764700 1 -- 172.21.15.12:0/3394690014 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fceac008c30 con 0x55cd19640180 2023-10-05T19:42:16.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.432+0000 7fceb7764700 1 -- 172.21.15.12:0/3394690014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:22 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55cd19642aa0 con 0x55cd19628560 2023-10-05T19:42:16.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.436+0000 7fceb7764700 1 -- 172.21.15.12:0/3394690014 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fceac008c30 con 0x55cd19628560 2023-10-05T19:42:16.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.436+0000 7fceb7764700 1 -- 172.21.15.12:0/3394690014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:23 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55cd19642aa0 con 0x55cd19628560 2023-10-05T19:42:16.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.436+0000 7fceb7764700 1 -- 172.21.15.12:0/3394690014 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fceac008c30 con 0x55cd19628560 2023-10-05T19:42:16.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.436+0000 7fceb7764700 1 -- 172.21.15.12:0/3394690014 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:24 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55cd19642aa0 con 0x7fcea801f740 2023-10-05T19:42:16.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.436+0000 7fceb8766700 1 -- 172.21.15.12:0/3394690014 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fceb0006d80 con 0x7fcea801f740 2023-10-05T19:42:16.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.436+0000 7fceb8766700 1 -- 172.21.15.12:0/3394690014 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:25 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55cd19642aa0 con 0x55cd19640180 2023-10-05T19:42:16.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.436+0000 7fceb7764700 1 -- 172.21.15.12:0/3394690014 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(25 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fceac008c30 con 0x55cd19640180 2023-10-05T19:42:16.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.436+0000 7fceb7764700 1 -- 172.21.15.12:0/3394690014 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:26 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55cd19642aa0 con 0x55cd1962ad40 2023-10-05T19:42:16.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.436+0000 7fceb7f65700 1 -- 172.21.15.12:0/3394690014 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcea801d070 con 0x55cd1962ad40 2023-10-05T19:42:16.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.436+0000 7fceb7f65700 1 -- 172.21.15.12:0/3394690014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:27 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55cd19642aa0 con 0x55cd19628560 2023-10-05T19:42:16.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.440+0000 7fceb7764700 1 -- 172.21.15.12:0/3394690014 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fceac008c30 con 0x55cd19628560 2023-10-05T19:42:16.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.440+0000 7fceb7764700 1 -- 172.21.15.12:0/3394690014 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:28 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55cd19642aa0 con 0x7fcea801f740 2023-10-05T19:42:16.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.440+0000 7fceb8766700 1 -- 172.21.15.12:0/3394690014 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fceb0006d80 con 0x7fcea801f740 2023-10-05T19:42:16.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.440+0000 7fceb8766700 1 -- 172.21.15.12:0/3394690014 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:29 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55cd19642aa0 con 0x55cd19640180 2023-10-05T19:42:16.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.440+0000 7fceb7764700 1 -- 172.21.15.12:0/3394690014 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(29 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fceac008c30 con 0x55cd19640180 2023-10-05T19:42:16.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.440+0000 7fceb7764700 1 -- 172.21.15.12:0/3394690014 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55cd19642aa0 con 0x7fcea801f740 2023-10-05T19:42:16.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.440+0000 7fceb8766700 1 -- 172.21.15.12:0/3394690014 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fceb0006d80 con 0x7fcea801f740 2023-10-05T19:42:16.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.440+0000 7fceb8766700 1 -- 172.21.15.12:0/3394690014 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:31 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55cd19642aa0 con 0x55cd1962ad40 2023-10-05T19:42:16.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.440+0000 7fceb7f65700 1 -- 172.21.15.12:0/3394690014 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcea801d070 con 0x55cd1962ad40 2023-10-05T19:42:16.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.444+0000 7fceb7f65700 1 -- 172.21.15.12:0/3394690014 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:32 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55cd19642aa0 con 0x55cd19640180 2023-10-05T19:42:16.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.444+0000 7fceb7764700 1 -- 172.21.15.12:0/3394690014 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fceac008c30 con 0x55cd19640180 2023-10-05T19:42:16.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.444+0000 7fceb7764700 1 -- 172.21.15.12:0/3394690014 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:33 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55cd19642aa0 con 0x7fcea801f740 2023-10-05T19:42:16.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.444+0000 7fceb8766700 1 -- 172.21.15.12:0/3394690014 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fceb0006d80 con 0x7fcea801f740 2023-10-05T19:42:16.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.444+0000 7fceb8766700 1 -- 172.21.15.12:0/3394690014 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:34 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55cd19642aa0 con 0x55cd19628560 2023-10-05T19:42:16.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.444+0000 7fceb7764700 1 -- 172.21.15.12:0/3394690014 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(34 [pgnls start_epoch 30 out=115b] v30'3 uv3 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fceac008c30 con 0x55cd19628560 2023-10-05T19:42:16.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.444+0000 7fceb7764700 1 -- 172.21.15.12:0/3394690014 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:35 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55cd19642aa0 con 0x55cd1962ad40 2023-10-05T19:42:16.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.444+0000 7fceb7f65700 1 -- 172.21.15.12:0/3394690014 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(35 [pgnls start_epoch 30 out=107b] v30'3 uv3 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fcea801d070 con 0x55cd1962ad40 2023-10-05T19:42:16.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.444+0000 7fceba430e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:42:16.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.444+0000 7fceba430e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:42:16.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.444+0000 7fceba430e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:42:16.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.444+0000 7fceba430e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:42:16.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.444+0000 7fceba430e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:16.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.444+0000 7fceba430e40 1 -- 172.21.15.12:0/3394690014 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:36 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55cd1963ec60 con 0x55cd1962ad40 2023-10-05T19:42:16.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.448+0000 7fceb7f65700 1 -- 172.21.15.12:0/3394690014 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(36 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~1036 out=1036b] v0'0 uv3 ondisk = 0) v8 ==== 190+0+1036 (crc 0 0 0) 0x7fcea801d070 con 0x55cd1962ad40 2023-10-05T19:42:16.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.448+0000 7fceba430e40 20 rados_obj.operate() r=0 bl.length=1036 2023-10-05T19:42:16.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.448+0000 7fceba430e40 1 -- 172.21.15.12:0/3394690014 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fcea801f740 msgr2=0x7fcea80423a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.448+0000 7fceba430e40 1 --2- 172.21.15.12:0/3394690014 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7fcea801f740 0x7fcea80423a0 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.448+0000 7fceba430e40 1 -- 172.21.15.12:0/3394690014 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55cd19628560 msgr2=0x55cd196289e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.448+0000 7fceba430e40 1 --2- 172.21.15.12:0/3394690014 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55cd19628560 0x55cd196289e0 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.448+0000 7fceba430e40 1 -- 172.21.15.12:0/3394690014 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55cd1962ad40 msgr2=0x55cd1963e370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.448+0000 7fceba430e40 1 --2- 172.21.15.12:0/3394690014 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55cd1962ad40 0x55cd1963e370 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.448+0000 7fceba430e40 1 -- 172.21.15.12:0/3394690014 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55cd19640180 msgr2=0x55cd19642560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.448+0000 7fceba430e40 1 --2- 172.21.15.12:0/3394690014 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55cd19640180 0x55cd19642560 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.448+0000 7fceba430e40 1 -- 172.21.15.12:0/3394690014 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fcea4030c50 msgr2=0x7fcea4033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.448+0000 7fceba430e40 1 --2- 172.21.15.12:0/3394690014 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fcea4030c50 0x7fcea4033100 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcea8010560 tx=0x7fcea8042bf0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.448+0000 7fceba430e40 1 -- 172.21.15.12:0/3394690014 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55cd195cbfe0 msgr2=0x55cd195f1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.448+0000 7fceb8766700 1 -- 172.21.15.12:0/3394690014 reap_dead start 2023-10-05T19:42:16.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.448+0000 7fceba430e40 1 --2- 172.21.15.12:0/3394690014 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55cd195cbfe0 0x55cd195f1f30 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x55cd195e7840 tx=0x7fcea8013000 comp rx=0 tx=0).stop 2023-10-05T19:42:16.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.448+0000 7fceba430e40 1 -- 172.21.15.12:0/3394690014 shutdown_connections 2023-10-05T19:42:16.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.448+0000 7fceba430e40 1 --2- 172.21.15.12:0/3394690014 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55cd195cbfe0 0x55cd195f1f30 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.448+0000 7fceba430e40 1 -- 172.21.15.12:0/3394690014 >> 172.21.15.12:0/3394690014 conn(0x55cd1953f830 msgr2=0x55cd195e57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:16.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.448+0000 7fceba430e40 1 -- 172.21.15.12:0/3394690014 shutdown_connections 2023-10-05T19:42:16.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.448+0000 7fceba430e40 1 -- 172.21.15.12:0/3394690014 wait complete. 2023-10-05T19:42:16.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.448+0000 7fceba430e40 1 -- 172.21.15.12:0/1384388392 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fcdfc030c00 msgr2=0x7fcdfc0330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.448+0000 7fceba430e40 1 --2- 172.21.15.12:0/1384388392 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fcdfc030c00 0x7fcdfc0330b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fceb0005600 tx=0x7fceb00424a0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.448+0000 7fceba430e40 1 -- 172.21.15.12:0/1384388392 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55cd19603660 msgr2=0x55cd1960a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.448+0000 7fceba430e40 1 --2- 172.21.15.12:0/1384388392 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55cd19603660 0x55cd1960a840 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x55cd19600b00 tx=0x7fceb0013000 comp rx=0 tx=0).stop 2023-10-05T19:42:16.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.448+0000 7fceba430e40 1 -- 172.21.15.12:0/1384388392 shutdown_connections 2023-10-05T19:42:16.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.448+0000 7fceba430e40 1 --2- 172.21.15.12:0/1384388392 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fcdfc030c00 0x7fcdfc0330b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.448+0000 7fceba430e40 1 --2- 172.21.15.12:0/1384388392 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55cd19603660 0x55cd1960a840 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.448+0000 7fceba430e40 1 -- 172.21.15.12:0/1384388392 >> 172.21.15.12:0/1384388392 conn(0x55cd195d05c0 msgr2=0x55cd195fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:16.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.448+0000 7fceba430e40 1 -- 172.21.15.12:0/1384388392 shutdown_connections 2023-10-05T19:42:16.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.448+0000 7fceba430e40 1 -- 172.21.15.12:0/1384388392 wait complete. 2023-10-05T19:42:16.459 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:42:16.460 INFO:teuthology.orchestra.run.smithi012.stdout: "id": "93ac49f4-ff49-4f99-811b-6453b28decd5", 2023-10-05T19:42:16.460 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default", 2023-10-05T19:42:16.460 INFO:teuthology.orchestra.run.smithi012.stdout: "domain_root": "default.rgw.meta:root", 2023-10-05T19:42:16.460 INFO:teuthology.orchestra.run.smithi012.stdout: "control_pool": "default.rgw.control", 2023-10-05T19:42:16.460 INFO:teuthology.orchestra.run.smithi012.stdout: "gc_pool": "default.rgw.log:gc", 2023-10-05T19:42:16.460 INFO:teuthology.orchestra.run.smithi012.stdout: "lc_pool": "default.rgw.log:lc", 2023-10-05T19:42:16.460 INFO:teuthology.orchestra.run.smithi012.stdout: "log_pool": "default.rgw.log", 2023-10-05T19:42:16.460 INFO:teuthology.orchestra.run.smithi012.stdout: "intent_log_pool": "default.rgw.log:intent", 2023-10-05T19:42:16.460 INFO:teuthology.orchestra.run.smithi012.stdout: "usage_log_pool": "default.rgw.log:usage", 2023-10-05T19:42:16.460 INFO:teuthology.orchestra.run.smithi012.stdout: "roles_pool": "default.rgw.meta:roles", 2023-10-05T19:42:16.460 INFO:teuthology.orchestra.run.smithi012.stdout: "reshard_pool": "default.rgw.log:reshard", 2023-10-05T19:42:16.460 INFO:teuthology.orchestra.run.smithi012.stdout: "user_keys_pool": "default.rgw.meta:users.keys", 2023-10-05T19:42:16.461 INFO:teuthology.orchestra.run.smithi012.stdout: "user_email_pool": "default.rgw.meta:users.email", 2023-10-05T19:42:16.461 INFO:teuthology.orchestra.run.smithi012.stdout: "user_swift_pool": "default.rgw.meta:users.swift", 2023-10-05T19:42:16.461 INFO:teuthology.orchestra.run.smithi012.stdout: "user_uid_pool": "default.rgw.meta:users.uid", 2023-10-05T19:42:16.461 INFO:teuthology.orchestra.run.smithi012.stdout: "otp_pool": "default.rgw.otp", 2023-10-05T19:42:16.461 INFO:teuthology.orchestra.run.smithi012.stdout: "system_key": { 2023-10-05T19:42:16.461 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key": "", 2023-10-05T19:42:16.461 INFO:teuthology.orchestra.run.smithi012.stdout: "secret_key": "" 2023-10-05T19:42:16.461 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:42:16.461 INFO:teuthology.orchestra.run.smithi012.stdout: "placement_pools": [ 2023-10-05T19:42:16.461 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:42:16.461 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "default-placement", 2023-10-05T19:42:16.461 INFO:teuthology.orchestra.run.smithi012.stdout: "val": { 2023-10-05T19:42:16.462 INFO:teuthology.orchestra.run.smithi012.stdout: "index_pool": "default.rgw.buckets.index", 2023-10-05T19:42:16.462 INFO:teuthology.orchestra.run.smithi012.stdout: "storage_classes": { 2023-10-05T19:42:16.462 INFO:teuthology.orchestra.run.smithi012.stdout: "STANDARD": { 2023-10-05T19:42:16.462 INFO:teuthology.orchestra.run.smithi012.stdout: "data_pool": "default.rgw.buckets.data" 2023-10-05T19:42:16.462 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:42:16.462 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:42:16.462 INFO:teuthology.orchestra.run.smithi012.stdout: "data_extra_pool": "default.rgw.buckets.non-ec", 2023-10-05T19:42:16.462 INFO:teuthology.orchestra.run.smithi012.stdout: "index_type": 0, 2023-10-05T19:42:16.462 INFO:teuthology.orchestra.run.smithi012.stdout: "inline_data": "true" 2023-10-05T19:42:16.462 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:42:16.462 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:42:16.463 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:42:16.463 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "new-placement", 2023-10-05T19:42:16.463 INFO:teuthology.orchestra.run.smithi012.stdout: "val": { 2023-10-05T19:42:16.463 INFO:teuthology.orchestra.run.smithi012.stdout: "index_pool": ".rgw.buckets.index.2", 2023-10-05T19:42:16.463 INFO:teuthology.orchestra.run.smithi012.stdout: "storage_classes": { 2023-10-05T19:42:16.463 INFO:teuthology.orchestra.run.smithi012.stdout: "STANDARD": { 2023-10-05T19:42:16.463 INFO:teuthology.orchestra.run.smithi012.stdout: "data_pool": ".rgw.buckets.2" 2023-10-05T19:42:16.463 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:42:16.463 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:42:16.463 INFO:teuthology.orchestra.run.smithi012.stdout: "data_extra_pool": "default.rgw.buckets.non-ec", 2023-10-05T19:42:16.463 INFO:teuthology.orchestra.run.smithi012.stdout: "index_type": 0, 2023-10-05T19:42:16.463 INFO:teuthology.orchestra.run.smithi012.stdout: "inline_data": "false" 2023-10-05T19:42:16.463 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:42:16.464 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:42:16.464 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:42:16.464 INFO:teuthology.orchestra.run.smithi012.stdout: "realm_id": "", 2023-10-05T19:42:16.464 INFO:teuthology.orchestra.run.smithi012.stdout: "notif_pool": "default.rgw.log:notif" 2023-10-05T19:42:16.464 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:42:16.465 DEBUG:tasks.util.rgw: json result: {'id': '93ac49f4-ff49-4f99-811b-6453b28decd5', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': 'true'}}, {'key': 'new-placement', 'val': {'index_pool': '.rgw.buckets.index.2', 'storage_classes': {'STANDARD': {'data_pool': '.rgw.buckets.2'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': 'false'}}], 'realm_id': '', 'notif_pool': 'default.rgw.log:notif'} 2023-10-05T19:42:16.465 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'rm', '--rgw-zone', 'default', '--placement-id', 'new-placement'] 2023-10-05T19:42:16.465 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'rm', '--rgw-zone', 'default', '--placement-id', 'new-placement'] 2023-10-05T19:42:16.466 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement rm --rgw-zone default --placement-id new-placement 2023-10-05T19:42:16.551 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:42:16.551 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:42:16.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.548+0000 7f8871c0ee40 1 Processor -- start 2023-10-05T19:42:16.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.548+0000 7f8871c0ee40 1 -- start start 2023-10-05T19:42:16.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.548+0000 7f8871c0ee40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55bd17d68320 0x55bd17d6a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.548+0000 7f8871c0ee40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55bd17cb0bd0 con 0x55bd17cd87c0 2023-10-05T19:42:16.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.548+0000 7f8871c0ee40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x55bd17c6dee0 con 0x55bd17d68320 2023-10-05T19:42:16.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.548+0000 7f886ef42700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55bd17d68320 0x55bd17d6a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.548+0000 7f886ef42700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55bd17d68320 0x55bd17d6a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:54440/0 (socket says 172.21.15.12:54440) 2023-10-05T19:42:16.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.548+0000 7f886ef42700 1 -- 172.21.15.12:0/1802221319 learned_addr learned my addr 172.21.15.12:0/1802221319 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:42:16.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.548+0000 7f886f743700 1 --1- 172.21.15.12:0/1802221319 >> v1:172.21.15.8:6789/0 conn(0x55bd17cd87c0 0x55bd17d66c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:47396/0 (socket says 172.21.15.12:47396) 2023-10-05T19:42:16.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.548+0000 7f886ef42700 1 -- 172.21.15.12:0/1802221319 >> v1:172.21.15.8:6789/0 conn(0x55bd17cd87c0 legacy=0x55bd17d66c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.548+0000 7f886ef42700 1 -- 172.21.15.12:0/1802221319 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd17d6bc50 con 0x55bd17d68320 2023-10-05T19:42:16.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.548+0000 7f886ef42700 1 --2- 172.21.15.12:0/1802221319 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd17d68320 0x55bd17d6a700 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f885800a5c0 tx=0x7f8858005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8c3fa7f31d87212 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.548+0000 7f886e741700 1 -- 172.21.15.12:0/1802221319 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f885800e510 con 0x55bd17d68320 2023-10-05T19:42:16.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.548+0000 7f886e741700 1 -- 172.21.15.12:0/1802221319 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f885800e670 con 0x55bd17d68320 2023-10-05T19:42:16.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.548+0000 7f886e741700 1 -- 172.21.15.12:0/1802221319 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f885800e920 con 0x55bd17d68320 2023-10-05T19:42:16.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.548+0000 7f8871c0ee40 1 -- 172.21.15.12:0/1802221319 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd17d68320 msgr2=0x55bd17d6a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.548+0000 7f8871c0ee40 1 --2- 172.21.15.12:0/1802221319 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd17d68320 0x55bd17d6a700 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f885800a5c0 tx=0x7f8858005e00 comp rx=0 tx=0).stop 2023-10-05T19:42:16.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.548+0000 7f8871c0ee40 1 -- 172.21.15.12:0/1802221319 shutdown_connections 2023-10-05T19:42:16.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.548+0000 7f8871c0ee40 1 --2- 172.21.15.12:0/1802221319 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd17d68320 0x55bd17d6a700 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.548+0000 7f8871c0ee40 1 -- 172.21.15.12:0/1802221319 >> 172.21.15.12:0/1802221319 conn(0x55bd17d5e750 msgr2=0x55bd17d60b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:16.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.548+0000 7f8871c0ee40 1 -- 172.21.15.12:0/1802221319 shutdown_connections 2023-10-05T19:42:16.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.548+0000 7f8871c0ee40 1 -- 172.21.15.12:0/1802221319 wait complete. 2023-10-05T19:42:16.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.552+0000 7f8871c0ee40 1 Processor -- start 2023-10-05T19:42:16.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.552+0000 7f8871c0ee40 1 -- start start 2023-10-05T19:42:16.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.552+0000 7f8871c0ee40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd17cd87c0 0x55bd17d82fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.552+0000 7f8871c0ee40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd17c31e30 con 0x55bd17cd87c0 2023-10-05T19:42:16.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.552+0000 7f886f743700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd17cd87c0 0x55bd17d82fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.552+0000 7f886f743700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd17cd87c0 0x55bd17d82fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:54446/0 (socket says 172.21.15.12:54446) 2023-10-05T19:42:16.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.552+0000 7f886f743700 1 -- 172.21.15.12:0/3980789452 learned_addr learned my addr 172.21.15.12:0/3980789452 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:42:16.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.552+0000 7f886f743700 1 -- 172.21.15.12:0/3980789452 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f885800a180 con 0x55bd17cd87c0 2023-10-05T19:42:16.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.552+0000 7f886f743700 1 --2- 172.21.15.12:0/3980789452 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd17cd87c0 0x55bd17d82fd0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f8860001cf0 tx=0x7f88600018a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac6ec21b855aec83 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.552+0000 7f886d73f700 1 -- 172.21.15.12:0/3980789452 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8860014070 con 0x55bd17cd87c0 2023-10-05T19:42:16.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.552+0000 7f886d73f700 1 -- 172.21.15.12:0/3980789452 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f886000be10 con 0x55bd17cd87c0 2023-10-05T19:42:16.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.552+0000 7f886d73f700 1 -- 172.21.15.12:0/3980789452 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f886000f590 con 0x55bd17cd87c0 2023-10-05T19:42:16.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.552+0000 7f8871c0ee40 1 -- 172.21.15.12:0/3980789452 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd17cd87c0 msgr2=0x55bd17d82fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.552+0000 7f8871c0ee40 1 --2- 172.21.15.12:0/3980789452 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd17cd87c0 0x55bd17d82fd0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f8860001cf0 tx=0x7f88600018a0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.552+0000 7f8871c0ee40 1 -- 172.21.15.12:0/3980789452 shutdown_connections 2023-10-05T19:42:16.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.552+0000 7f8871c0ee40 1 --2- 172.21.15.12:0/3980789452 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd17cd87c0 0x55bd17d82fd0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.552+0000 7f8871c0ee40 1 -- 172.21.15.12:0/3980789452 >> 172.21.15.12:0/3980789452 conn(0x55bd17d5e750 msgr2=0x55bd17d7eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:16.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.552+0000 7f8871c0ee40 1 -- 172.21.15.12:0/3980789452 shutdown_connections 2023-10-05T19:42:16.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.552+0000 7f8871c0ee40 1 -- 172.21.15.12:0/3980789452 wait complete. 2023-10-05T19:42:16.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.552+0000 7f8871c0ee40 1 Processor -- start 2023-10-05T19:42:16.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.552+0000 7f8871c0ee40 1 -- start start 2023-10-05T19:42:16.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.552+0000 7f8871c0ee40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd17cd87c0 0x55bd17d89da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.552+0000 7f8871c0ee40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd17c04ed0 con 0x55bd17cd87c0 2023-10-05T19:42:16.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.556+0000 7f886f743700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd17cd87c0 0x55bd17d89da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.556+0000 7f886f743700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd17cd87c0 0x55bd17d89da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:54456/0 (socket says 172.21.15.12:54456) 2023-10-05T19:42:16.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.556+0000 7f886f743700 1 -- 172.21.15.12:0/862828640 learned_addr learned my addr 172.21.15.12:0/862828640 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:42:16.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.556+0000 7f886f743700 1 -- 172.21.15.12:0/862828640 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd17cb0bd0 con 0x55bd17cd87c0 2023-10-05T19:42:16.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.556+0000 7f886f743700 1 --2- 172.21.15.12:0/862828640 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd17cd87c0 0x55bd17d89da0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f88600018e0 tx=0x7f8860016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.556+0000 7f88677fe700 1 -- 172.21.15.12:0/862828640 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f886001c670 con 0x55bd17cd87c0 2023-10-05T19:42:16.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.556+0000 7f88677fe700 1 -- 172.21.15.12:0/862828640 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f886001c7f0 con 0x55bd17cd87c0 2023-10-05T19:42:16.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.556+0000 7f8871c0ee40 1 -- 172.21.15.12:0/862828640 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bd17d8a2e0 con 0x55bd17cd87c0 2023-10-05T19:42:16.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.556+0000 7f88677fe700 1 -- 172.21.15.12:0/862828640 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f886001cae0 con 0x55bd17cd87c0 2023-10-05T19:42:16.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.556+0000 7f8871c0ee40 1 -- 172.21.15.12:0/862828640 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bd17d8a720 con 0x55bd17cd87c0 2023-10-05T19:42:16.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.556+0000 7f8871c0ee40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:42:16.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.556+0000 7f8871c0ee40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:42:16.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.556+0000 7f8871c0ee40 1 Processor -- start 2023-10-05T19:42:16.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.556+0000 7f8871c0ee40 1 -- start start 2023-10-05T19:42:16.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.556+0000 7f88677fe700 1 -- 172.21.15.12:0/862828640 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f8860023020 con 0x55bd17cd87c0 2023-10-05T19:42:16.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.560+0000 7f8871c0ee40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd17d9b520 0x55bd17d9b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.560+0000 7f8871c0ee40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd17d9be30 con 0x55bd17d9b520 2023-10-05T19:42:16.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.560+0000 7f88677fe700 1 --2- 172.21.15.12:0/862828640 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f8854030c50 0x7f8854033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.560+0000 7f88677fe700 1 -- 172.21.15.12:0/862828640 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7f8860008030 con 0x55bd17cd87c0 2023-10-05T19:42:16.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.560+0000 7f886f743700 1 --2- 172.21.15.12:0/862828640 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f8854030c50 0x7f8854033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.560+0000 7f886ff44700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd17d9b520 0x55bd17d9b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.560+0000 7f886ff44700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd17d9b520 0x55bd17d9b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:54458/0 (socket says 172.21.15.12:54458) 2023-10-05T19:42:16.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.560+0000 7f886ff44700 1 -- 172.21.15.12:0/3706244179 learned_addr learned my addr 172.21.15.12:0/3706244179 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:42:16.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.560+0000 7f886ff44700 1 -- 172.21.15.12:0/3706244179 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd17d9bf70 con 0x55bd17d9b520 2023-10-05T19:42:16.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.560+0000 7f886ff44700 1 --2- 172.21.15.12:0/3706244179 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd17d9b520 0x55bd17d9b8f0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f886800a320 tx=0x7f88680059d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=961f54c76d72fdb3 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.560+0000 7f886f743700 1 --2- 172.21.15.12:0/862828640 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f8854030c50 0x7f8854033100 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f886002ef70 tx=0x7f88600435a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.560+0000 7f883ffef700 1 -- 172.21.15.12:0/3706244179 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f886800ae30 con 0x55bd17d9b520 2023-10-05T19:42:16.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.560+0000 7f883ffef700 1 -- 172.21.15.12:0/3706244179 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f886800c610 con 0x55bd17d9b520 2023-10-05T19:42:16.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.560+0000 7f883ffef700 1 -- 172.21.15.12:0/3706244179 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f886800c7c0 con 0x55bd17d9b520 2023-10-05T19:42:16.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.560+0000 7f8871c0ee40 1 -- 172.21.15.12:0/3706244179 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd17d9b520 msgr2=0x55bd17d9b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.560+0000 7f8871c0ee40 1 --2- 172.21.15.12:0/3706244179 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd17d9b520 0x55bd17d9b8f0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f886800a320 tx=0x7f88680059d0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.560+0000 7f8871c0ee40 1 -- 172.21.15.12:0/3706244179 shutdown_connections 2023-10-05T19:42:16.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.560+0000 7f8871c0ee40 1 --2- 172.21.15.12:0/3706244179 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd17d9b520 0x55bd17d9b8f0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.560+0000 7f8871c0ee40 1 -- 172.21.15.12:0/3706244179 >> 172.21.15.12:0/3706244179 conn(0x55bd17d68320 msgr2=0x55bd17d98250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:16.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.560+0000 7f8871c0ee40 1 -- 172.21.15.12:0/3706244179 shutdown_connections 2023-10-05T19:42:16.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.560+0000 7f8871c0ee40 1 -- 172.21.15.12:0/3706244179 wait complete. 2023-10-05T19:42:16.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.560+0000 7f8871c0ee40 1 Processor -- start 2023-10-05T19:42:16.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.564+0000 7f8871c0ee40 1 -- start start 2023-10-05T19:42:16.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.564+0000 7f8871c0ee40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd17d9b520 0x55bd17da2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.564+0000 7f8871c0ee40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd17da3740 con 0x55bd17d9b520 2023-10-05T19:42:16.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.564+0000 7f886ff44700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd17d9b520 0x55bd17da2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.564+0000 7f886ff44700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd17d9b520 0x55bd17da2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:54466/0 (socket says 172.21.15.12:54466) 2023-10-05T19:42:16.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.564+0000 7f886ff44700 1 -- 172.21.15.12:0/2113854309 learned_addr learned my addr 172.21.15.12:0/2113854309 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:42:16.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.564+0000 7f886ff44700 1 -- 172.21.15.12:0/2113854309 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8868005c90 con 0x55bd17d9b520 2023-10-05T19:42:16.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.564+0000 7f886ff44700 1 --2- 172.21.15.12:0/2113854309 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd17d9b520 0x55bd17da2740 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x55bd17d989e0 tx=0x7f8868013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.564+0000 7f883e7ec700 1 -- 172.21.15.12:0/2113854309 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8868015660 con 0x55bd17d9b520 2023-10-05T19:42:16.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.564+0000 7f883e7ec700 1 -- 172.21.15.12:0/2113854309 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88680157c0 con 0x55bd17d9b520 2023-10-05T19:42:16.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.564+0000 7f8871c0ee40 1 -- 172.21.15.12:0/2113854309 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bd17da2d40 con 0x55bd17d9b520 2023-10-05T19:42:16.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.564+0000 7f883e7ec700 1 -- 172.21.15.12:0/2113854309 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8868015ab0 con 0x55bd17d9b520 2023-10-05T19:42:16.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.564+0000 7f8871c0ee40 1 -- 172.21.15.12:0/2113854309 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bd17da3200 con 0x55bd17d9b520 2023-10-05T19:42:16.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.564+0000 7f883e7ec700 1 -- 172.21.15.12:0/2113854309 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f8868007050 con 0x55bd17d9b520 2023-10-05T19:42:16.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.568+0000 7f8871c0ee40 20 try_to_resolve_local_entities(): before: realm_id= realm_name= zonegroup_id= zonegroup_name= zone_id= zone_name=default 2023-10-05T19:42:16.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.568+0000 7f8871c0ee40 20 rados->read ofs=0 len=0 2023-10-05T19:42:16.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.568+0000 7f87b7fff700 20 reqs_thread_entry: start 2023-10-05T19:42:16.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.568+0000 7f8871c0ee40 1 --2- 172.21.15.12:0/862828640 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55bd17dc04a0 0x55bd17dc0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.568+0000 7f883e7ec700 1 --2- 172.21.15.12:0/2113854309 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f87bc030c50 0x7f87bc033100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.568+0000 7f8871c0ee40 1 -- 172.21.15.12:0/862828640 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd17dd39e0 con 0x55bd17dc04a0 2023-10-05T19:42:16.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.568+0000 7f883e7ec700 1 -- 172.21.15.12:0/2113854309 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7f88680440e0 con 0x55bd17d9b520 2023-10-05T19:42:16.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.568+0000 7f886ef42700 1 --2- 172.21.15.12:0/862828640 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55bd17dc04a0 0x55bd17dc0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.568+0000 7f886ff44700 1 --2- 172.21.15.12:0/2113854309 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f87bc030c50 0x7f87bc033100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.568+0000 7f886ef42700 1 --2- 172.21.15.12:0/862828640 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55bd17dc04a0 0x55bd17dc0940 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.568+0000 7f886ff44700 1 --2- 172.21.15.12:0/2113854309 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f87bc030c50 0x7f87bc033100 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f886800bb40 tx=0x7f8868042420 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.568+0000 7f886ef42700 1 -- 172.21.15.12:0/862828640 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f8858017040 con 0x55bd17dc04a0 2023-10-05T19:42:16.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.568+0000 7f8871c0ee40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:42:16.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.568+0000 7f8871c0ee40 20 rados->read ofs=0 len=0 2023-10-05T19:42:16.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.568+0000 7f8871c0ee40 1 --2- 172.21.15.12:0/862828640 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55bd17dd42a0 0x55bd17dd6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.568+0000 7f8871c0ee40 1 -- 172.21.15.12:0/862828640 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:2 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd17dd6bc0 con 0x55bd17dd42a0 2023-10-05T19:42:16.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.568+0000 7f886f743700 1 --2- 172.21.15.12:0/862828640 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55bd17dd42a0 0x55bd17dd6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.568+0000 7f886f743700 1 --2- 172.21.15.12:0/862828640 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55bd17dd42a0 0x55bd17dd6680 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.568+0000 7f886f743700 1 -- 172.21.15.12:0/862828640 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(2 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~1036 out=1036b] v0'0 uv3 ondisk = 0) v8 ==== 190+0+1036 (crc 0 0 0) 0x7f886003a040 con 0x55bd17dd42a0 2023-10-05T19:42:16.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.568+0000 7f8871c0ee40 20 rados_obj.operate() r=0 bl.length=1036 2023-10-05T19:42:16.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.568+0000 7f8871c0ee40 1 --2- 172.21.15.12:0/862828640 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55bd17dd8490 0x55bd17dda870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.568+0000 7f8871c0ee40 1 -- 172.21.15.12:0/862828640 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:3 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd17ddadb0 con 0x55bd17dd8490 2023-10-05T19:42:16.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.568+0000 7f886ef42700 1 --2- 172.21.15.12:0/862828640 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55bd17dd8490 0x55bd17dda870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.572+0000 7f886ef42700 1 --2- 172.21.15.12:0/862828640 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55bd17dd8490 0x55bd17dda870 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.572+0000 7f886ef42700 1 -- 172.21.15.12:0/862828640 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(3 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f8858017040 con 0x55bd17dd8490 2023-10-05T19:42:16.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.572+0000 7f886ef42700 1 -- 172.21.15.12:0/862828640 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:4 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd17ddadb0 con 0x55bd17dd42a0 2023-10-05T19:42:16.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.572+0000 7f886f743700 1 -- 172.21.15.12:0/862828640 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(4 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f886003a040 con 0x55bd17dd42a0 2023-10-05T19:42:16.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.572+0000 7f886f743700 1 --2- 172.21.15.12:0/862828640 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f886001a070 0x7f886003e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.572+0000 7f886f743700 1 -- 172.21.15.12:0/862828640 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:5 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd17ddadb0 con 0x7f886001a070 2023-10-05T19:42:16.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.572+0000 7f886ff44700 1 --2- 172.21.15.12:0/862828640 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f886001a070 0x7f886003e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.576+0000 7f886ff44700 1 --2- 172.21.15.12:0/862828640 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f886001a070 0x7f886003e830 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.576+0000 7f886ff44700 1 -- 172.21.15.12:0/862828640 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(5 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f886800bbb0 con 0x7f886001a070 2023-10-05T19:42:16.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.576+0000 7f886ff44700 1 -- 172.21.15.12:0/862828640 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:6 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd17ddadb0 con 0x55bd17dc04a0 2023-10-05T19:42:16.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.576+0000 7f886ef42700 1 -- 172.21.15.12:0/862828640 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8858017040 con 0x55bd17dc04a0 2023-10-05T19:42:16.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.576+0000 7f886ef42700 1 -- 172.21.15.12:0/862828640 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:7 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd17ddadb0 con 0x55bd17dc04a0 2023-10-05T19:42:16.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.576+0000 7f886ef42700 1 -- 172.21.15.12:0/862828640 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8858017040 con 0x55bd17dc04a0 2023-10-05T19:42:16.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.576+0000 7f886ef42700 1 -- 172.21.15.12:0/862828640 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:8 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd17ddadb0 con 0x55bd17dc04a0 2023-10-05T19:42:16.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.576+0000 7f886ef42700 1 -- 172.21.15.12:0/862828640 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8858017040 con 0x55bd17dc04a0 2023-10-05T19:42:16.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.576+0000 7f886ef42700 1 -- 172.21.15.12:0/862828640 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:9 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd17ddadb0 con 0x7f886001a070 2023-10-05T19:42:16.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.580+0000 7f886ff44700 1 -- 172.21.15.12:0/862828640 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f886800bbb0 con 0x7f886001a070 2023-10-05T19:42:16.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.580+0000 7f886ff44700 1 -- 172.21.15.12:0/862828640 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:10 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd17ddadb0 con 0x55bd17dd42a0 2023-10-05T19:42:16.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.580+0000 7f886f743700 1 -- 172.21.15.12:0/862828640 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f886003a040 con 0x55bd17dd42a0 2023-10-05T19:42:16.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.580+0000 7f886f743700 1 -- 172.21.15.12:0/862828640 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:11 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd17ddadb0 con 0x55bd17dd42a0 2023-10-05T19:42:16.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.580+0000 7f886f743700 1 -- 172.21.15.12:0/862828640 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(11 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f886003a040 con 0x55bd17dd42a0 2023-10-05T19:42:16.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.580+0000 7f886f743700 1 -- 172.21.15.12:0/862828640 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:12 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd17ddadb0 con 0x55bd17dc04a0 2023-10-05T19:42:16.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.580+0000 7f886ef42700 1 -- 172.21.15.12:0/862828640 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8858017040 con 0x55bd17dc04a0 2023-10-05T19:42:16.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.580+0000 7f886ef42700 1 -- 172.21.15.12:0/862828640 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:13 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd17ddadb0 con 0x7f886001a070 2023-10-05T19:42:16.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.580+0000 7f886ff44700 1 -- 172.21.15.12:0/862828640 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f886800bbb0 con 0x7f886001a070 2023-10-05T19:42:16.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.580+0000 7f886ff44700 1 -- 172.21.15.12:0/862828640 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:14 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd17ddadb0 con 0x55bd17dc04a0 2023-10-05T19:42:16.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.584+0000 7f886ef42700 1 -- 172.21.15.12:0/862828640 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8858017040 con 0x55bd17dc04a0 2023-10-05T19:42:16.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.584+0000 7f886ef42700 1 -- 172.21.15.12:0/862828640 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:15 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd17ddadb0 con 0x55bd17dd42a0 2023-10-05T19:42:16.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.584+0000 7f886f743700 1 -- 172.21.15.12:0/862828640 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f886003a040 con 0x55bd17dd42a0 2023-10-05T19:42:16.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.584+0000 7f886f743700 1 -- 172.21.15.12:0/862828640 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:16 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd17ddadb0 con 0x55bd17dd8490 2023-10-05T19:42:16.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.584+0000 7f886ef42700 1 -- 172.21.15.12:0/862828640 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8858017040 con 0x55bd17dd8490 2023-10-05T19:42:16.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.584+0000 7f886ef42700 1 -- 172.21.15.12:0/862828640 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:17 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd17ddadb0 con 0x55bd17dd8490 2023-10-05T19:42:16.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.584+0000 7f886ef42700 1 -- 172.21.15.12:0/862828640 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8858017040 con 0x55bd17dd8490 2023-10-05T19:42:16.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.584+0000 7f886ef42700 1 -- 172.21.15.12:0/862828640 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:18 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd17ddadb0 con 0x7f886001a070 2023-10-05T19:42:16.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.584+0000 7f886ff44700 1 -- 172.21.15.12:0/862828640 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(18 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f886800bbb0 con 0x7f886001a070 2023-10-05T19:42:16.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.588+0000 7f886ff44700 1 -- 172.21.15.12:0/862828640 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd17ddadb0 con 0x55bd17dd8490 2023-10-05T19:42:16.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.588+0000 7f886ef42700 1 -- 172.21.15.12:0/862828640 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8858017040 con 0x55bd17dd8490 2023-10-05T19:42:16.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.588+0000 7f886ef42700 1 -- 172.21.15.12:0/862828640 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd17ddadb0 con 0x55bd17dd8490 2023-10-05T19:42:16.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.588+0000 7f886ef42700 1 -- 172.21.15.12:0/862828640 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8858017040 con 0x55bd17dd8490 2023-10-05T19:42:16.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.588+0000 7f886ef42700 1 -- 172.21.15.12:0/862828640 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:21 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd17ddadb0 con 0x55bd17dc04a0 2023-10-05T19:42:16.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.588+0000 7f886ef42700 1 -- 172.21.15.12:0/862828640 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8858017040 con 0x55bd17dc04a0 2023-10-05T19:42:16.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.588+0000 7f886ef42700 1 -- 172.21.15.12:0/862828640 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:22 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd17ddadb0 con 0x55bd17dc04a0 2023-10-05T19:42:16.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.588+0000 7f886ef42700 1 -- 172.21.15.12:0/862828640 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8858017040 con 0x55bd17dc04a0 2023-10-05T19:42:16.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.588+0000 7f886ef42700 1 -- 172.21.15.12:0/862828640 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:23 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd17ddadb0 con 0x7f886001a070 2023-10-05T19:42:16.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.592+0000 7f886ff44700 1 -- 172.21.15.12:0/862828640 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f886800bbb0 con 0x7f886001a070 2023-10-05T19:42:16.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.592+0000 7f886ff44700 1 -- 172.21.15.12:0/862828640 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:24 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd17ddadb0 con 0x55bd17dd8490 2023-10-05T19:42:16.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.592+0000 7f886ef42700 1 -- 172.21.15.12:0/862828640 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8858017040 con 0x55bd17dd8490 2023-10-05T19:42:16.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.592+0000 7f886ef42700 1 -- 172.21.15.12:0/862828640 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:25 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd17ddadb0 con 0x55bd17dd42a0 2023-10-05T19:42:16.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.592+0000 7f886f743700 1 -- 172.21.15.12:0/862828640 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(25 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f886003a040 con 0x55bd17dd42a0 2023-10-05T19:42:16.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.592+0000 7f886f743700 1 -- 172.21.15.12:0/862828640 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:26 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd17ddadb0 con 0x55bd17dc04a0 2023-10-05T19:42:16.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.592+0000 7f886ef42700 1 -- 172.21.15.12:0/862828640 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8858017040 con 0x55bd17dc04a0 2023-10-05T19:42:16.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.592+0000 7f886ef42700 1 -- 172.21.15.12:0/862828640 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:27 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd17ddadb0 con 0x7f886001a070 2023-10-05T19:42:16.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.592+0000 7f886ff44700 1 -- 172.21.15.12:0/862828640 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f886800bbb0 con 0x7f886001a070 2023-10-05T19:42:16.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.592+0000 7f886ff44700 1 -- 172.21.15.12:0/862828640 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:28 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd17ddadb0 con 0x55bd17dd8490 2023-10-05T19:42:16.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.596+0000 7f886ef42700 1 -- 172.21.15.12:0/862828640 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8858017040 con 0x55bd17dd8490 2023-10-05T19:42:16.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.596+0000 7f886ef42700 1 -- 172.21.15.12:0/862828640 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:29 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd17ddadb0 con 0x7f886001a070 2023-10-05T19:42:16.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.596+0000 7f886ff44700 1 -- 172.21.15.12:0/862828640 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(29 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f886800bbb0 con 0x7f886001a070 2023-10-05T19:42:16.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.596+0000 7f886ff44700 1 -- 172.21.15.12:0/862828640 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:30 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd17ddadb0 con 0x55bd17dd42a0 2023-10-05T19:42:16.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.596+0000 7f886f743700 1 -- 172.21.15.12:0/862828640 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f886003a040 con 0x55bd17dd42a0 2023-10-05T19:42:16.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.596+0000 7f886f743700 1 -- 172.21.15.12:0/862828640 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd17ddadb0 con 0x55bd17dd8490 2023-10-05T19:42:16.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.596+0000 7f886ef42700 1 -- 172.21.15.12:0/862828640 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8858017040 con 0x55bd17dd8490 2023-10-05T19:42:16.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.596+0000 7f886ef42700 1 -- 172.21.15.12:0/862828640 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd17ddadb0 con 0x7f886001a070 2023-10-05T19:42:16.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.600+0000 7f886ff44700 1 -- 172.21.15.12:0/862828640 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f886800bbb0 con 0x7f886001a070 2023-10-05T19:42:16.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.600+0000 7f886ff44700 1 -- 172.21.15.12:0/862828640 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:33 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd17ddadb0 con 0x55bd17dc04a0 2023-10-05T19:42:16.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.600+0000 7f886ef42700 1 -- 172.21.15.12:0/862828640 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(33 [pgnls start_epoch 30 out=115b] v30'3 uv3 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f8858017040 con 0x55bd17dc04a0 2023-10-05T19:42:16.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.600+0000 7f886ef42700 1 -- 172.21.15.12:0/862828640 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:34 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd17ddadb0 con 0x55bd17dd42a0 2023-10-05T19:42:16.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.600+0000 7f886f743700 1 -- 172.21.15.12:0/862828640 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(34 [pgnls start_epoch 30 out=107b] v30'3 uv3 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f886003a040 con 0x55bd17dd42a0 2023-10-05T19:42:16.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.600+0000 7f8871c0ee40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:42:16.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.600+0000 7f8871c0ee40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:42:16.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.600+0000 7f8871c0ee40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:42:16.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.600+0000 7f8871c0ee40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:42:16.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.600+0000 7f8871c0ee40 20 rados->read ofs=0 len=0 2023-10-05T19:42:16.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.600+0000 7f8871c0ee40 1 -- 172.21.15.12:0/862828640 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:35 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd17dc13e0 con 0x55bd17dd42a0 2023-10-05T19:42:16.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.600+0000 7f886f743700 1 -- 172.21.15.12:0/862828640 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(35 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~1036 out=1036b] v0'0 uv3 ondisk = 0) v8 ==== 190+0+1036 (crc 0 0 0) 0x7f886003a040 con 0x55bd17dd42a0 2023-10-05T19:42:16.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.600+0000 7f8871c0ee40 20 rados_obj.operate() r=0 bl.length=1036 2023-10-05T19:42:16.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.600+0000 7f8871c0ee40 1 -- 172.21.15.12:0/862828640 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:36 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [delete,create,writefull 0~874 in=874b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55bd17ddb160 con 0x55bd17dd42a0 2023-10-05T19:42:16.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.604+0000 7f886f743700 1 -- 172.21.15.12:0/862828640 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(36 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [delete,create,writefull 0~874] v30'4 uv4 ondisk = 0) v8 ==== 274+0+0 (crc 0 0 0) 0x7f886003a040 con 0x55bd17dd42a0 2023-10-05T19:42:16.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.608+0000 7f8871c0ee40 1 -- 172.21.15.12:0/862828640 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f886001a070 msgr2=0x7f886003e830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.608+0000 7f8871c0ee40 1 --2- 172.21.15.12:0/862828640 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f886001a070 0x7f886003e830 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.608+0000 7f8871c0ee40 1 -- 172.21.15.12:0/862828640 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55bd17dc04a0 msgr2=0x55bd17dc0940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.608+0000 7f8871c0ee40 1 --2- 172.21.15.12:0/862828640 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55bd17dc04a0 0x55bd17dc0940 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.608+0000 7f8871c0ee40 1 -- 172.21.15.12:0/862828640 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55bd17dd42a0 msgr2=0x55bd17dd6680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.608+0000 7f8871c0ee40 1 --2- 172.21.15.12:0/862828640 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55bd17dd42a0 0x55bd17dd6680 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.608+0000 7f8871c0ee40 1 -- 172.21.15.12:0/862828640 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55bd17dd8490 msgr2=0x55bd17dda870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.608+0000 7f8871c0ee40 1 --2- 172.21.15.12:0/862828640 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55bd17dd8490 0x55bd17dda870 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.608+0000 7f8871c0ee40 1 -- 172.21.15.12:0/862828640 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f8854030c50 msgr2=0x7f8854033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.608+0000 7f8871c0ee40 1 --2- 172.21.15.12:0/862828640 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f8854030c50 0x7f8854033100 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f886002ef70 tx=0x7f88600435a0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.608+0000 7f8871c0ee40 1 -- 172.21.15.12:0/862828640 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd17cd87c0 msgr2=0x55bd17d89da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.608+0000 7f8871c0ee40 1 --2- 172.21.15.12:0/862828640 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd17cd87c0 0x55bd17d89da0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f88600018e0 tx=0x7f8860016000 comp rx=0 tx=0).stop 2023-10-05T19:42:16.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.608+0000 7f886ff44700 1 -- 172.21.15.12:0/862828640 reap_dead start 2023-10-05T19:42:16.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.608+0000 7f8871c0ee40 1 -- 172.21.15.12:0/862828640 shutdown_connections 2023-10-05T19:42:16.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.608+0000 7f8871c0ee40 1 -- 172.21.15.12:0/862828640 >> 172.21.15.12:0/862828640 conn(0x55bd17d5e750 msgr2=0x55bd17d7d590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:16.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.608+0000 7f8871c0ee40 1 -- 172.21.15.12:0/862828640 shutdown_connections 2023-10-05T19:42:16.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.608+0000 7f8871c0ee40 1 -- 172.21.15.12:0/862828640 wait complete. 2023-10-05T19:42:16.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.608+0000 7f8871c0ee40 1 -- 172.21.15.12:0/2113854309 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f87bc030c50 msgr2=0x7f87bc033100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.608+0000 7f8871c0ee40 1 --2- 172.21.15.12:0/2113854309 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f87bc030c50 0x7f87bc033100 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f886800bb40 tx=0x7f8868042420 comp rx=0 tx=0).stop 2023-10-05T19:42:16.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.608+0000 7f8871c0ee40 1 -- 172.21.15.12:0/2113854309 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd17d9b520 msgr2=0x55bd17da2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.608+0000 7f8871c0ee40 1 --2- 172.21.15.12:0/2113854309 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd17d9b520 0x55bd17da2740 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x55bd17d989e0 tx=0x7f8868013000 comp rx=0 tx=0).stop 2023-10-05T19:42:16.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.608+0000 7f8871c0ee40 1 -- 172.21.15.12:0/2113854309 shutdown_connections 2023-10-05T19:42:16.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.608+0000 7f8871c0ee40 1 --2- 172.21.15.12:0/2113854309 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f87bc030c50 0x7f87bc033100 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.608+0000 7f8871c0ee40 1 --2- 172.21.15.12:0/2113854309 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55bd17d9b520 0x55bd17da2740 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.608+0000 7f8871c0ee40 1 -- 172.21.15.12:0/2113854309 >> 172.21.15.12:0/2113854309 conn(0x55bd17d68320 msgr2=0x55bd17d97e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:16.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.608+0000 7f8871c0ee40 1 -- 172.21.15.12:0/2113854309 shutdown_connections 2023-10-05T19:42:16.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.608+0000 7f8871c0ee40 1 -- 172.21.15.12:0/2113854309 wait complete. 2023-10-05T19:42:16.616 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:42:16.616 INFO:teuthology.orchestra.run.smithi012.stdout: "id": "93ac49f4-ff49-4f99-811b-6453b28decd5", 2023-10-05T19:42:16.617 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default", 2023-10-05T19:42:16.617 INFO:teuthology.orchestra.run.smithi012.stdout: "domain_root": "default.rgw.meta:root", 2023-10-05T19:42:16.617 INFO:teuthology.orchestra.run.smithi012.stdout: "control_pool": "default.rgw.control", 2023-10-05T19:42:16.617 INFO:teuthology.orchestra.run.smithi012.stdout: "gc_pool": "default.rgw.log:gc", 2023-10-05T19:42:16.617 INFO:teuthology.orchestra.run.smithi012.stdout: "lc_pool": "default.rgw.log:lc", 2023-10-05T19:42:16.617 INFO:teuthology.orchestra.run.smithi012.stdout: "log_pool": "default.rgw.log", 2023-10-05T19:42:16.617 INFO:teuthology.orchestra.run.smithi012.stdout: "intent_log_pool": "default.rgw.log:intent", 2023-10-05T19:42:16.617 INFO:teuthology.orchestra.run.smithi012.stdout: "usage_log_pool": "default.rgw.log:usage", 2023-10-05T19:42:16.617 INFO:teuthology.orchestra.run.smithi012.stdout: "roles_pool": "default.rgw.meta:roles", 2023-10-05T19:42:16.617 INFO:teuthology.orchestra.run.smithi012.stdout: "reshard_pool": "default.rgw.log:reshard", 2023-10-05T19:42:16.618 INFO:teuthology.orchestra.run.smithi012.stdout: "user_keys_pool": "default.rgw.meta:users.keys", 2023-10-05T19:42:16.618 INFO:teuthology.orchestra.run.smithi012.stdout: "user_email_pool": "default.rgw.meta:users.email", 2023-10-05T19:42:16.618 INFO:teuthology.orchestra.run.smithi012.stdout: "user_swift_pool": "default.rgw.meta:users.swift", 2023-10-05T19:42:16.618 INFO:teuthology.orchestra.run.smithi012.stdout: "user_uid_pool": "default.rgw.meta:users.uid", 2023-10-05T19:42:16.618 INFO:teuthology.orchestra.run.smithi012.stdout: "otp_pool": "default.rgw.otp", 2023-10-05T19:42:16.618 INFO:teuthology.orchestra.run.smithi012.stdout: "system_key": { 2023-10-05T19:42:16.618 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key": "", 2023-10-05T19:42:16.618 INFO:teuthology.orchestra.run.smithi012.stdout: "secret_key": "" 2023-10-05T19:42:16.618 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:42:16.618 INFO:teuthology.orchestra.run.smithi012.stdout: "placement_pools": [ 2023-10-05T19:42:16.618 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:42:16.619 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "default-placement", 2023-10-05T19:42:16.619 INFO:teuthology.orchestra.run.smithi012.stdout: "val": { 2023-10-05T19:42:16.619 INFO:teuthology.orchestra.run.smithi012.stdout: "index_pool": "default.rgw.buckets.index", 2023-10-05T19:42:16.619 INFO:teuthology.orchestra.run.smithi012.stdout: "storage_classes": { 2023-10-05T19:42:16.619 INFO:teuthology.orchestra.run.smithi012.stdout: "STANDARD": { 2023-10-05T19:42:16.619 INFO:teuthology.orchestra.run.smithi012.stdout: "data_pool": "default.rgw.buckets.data" 2023-10-05T19:42:16.619 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:42:16.619 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:42:16.619 INFO:teuthology.orchestra.run.smithi012.stdout: "data_extra_pool": "default.rgw.buckets.non-ec", 2023-10-05T19:42:16.619 INFO:teuthology.orchestra.run.smithi012.stdout: "index_type": 0, 2023-10-05T19:42:16.619 INFO:teuthology.orchestra.run.smithi012.stdout: "inline_data": "true" 2023-10-05T19:42:16.619 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:42:16.619 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:42:16.620 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:42:16.620 INFO:teuthology.orchestra.run.smithi012.stdout: "realm_id": "", 2023-10-05T19:42:16.620 INFO:teuthology.orchestra.run.smithi012.stdout: "notif_pool": "default.rgw.log:notif" 2023-10-05T19:42:16.620 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:42:16.621 DEBUG:tasks.util.rgw: json result: {'id': '93ac49f4-ff49-4f99-811b-6453b28decd5', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': 'true'}}], 'realm_id': '', 'notif_pool': 'default.rgw.log:notif'} 2023-10-05T19:42:16.621 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zonegroup', 'get'] 2023-10-05T19:42:16.621 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zonegroup', 'get'] 2023-10-05T19:42:16.621 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zonegroup get 2023-10-05T19:42:16.663 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:42:16.663 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:42:16.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.660+0000 7f3a77c90e40 1 Processor -- start 2023-10-05T19:42:16.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.660+0000 7f3a77c90e40 1 -- start start 2023-10-05T19:42:16.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.660+0000 7f3a77c90e40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55f1df1c42e0 0x55f1df1c46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.660+0000 7f3a77c90e40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55f1df112c00 con 0x55f1df1c4bf0 2023-10-05T19:42:16.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.660+0000 7f3a77c90e40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x55f1df0cfee0 con 0x55f1df1c42e0 2023-10-05T19:42:16.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.660+0000 7f3a757c5700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55f1df1c42e0 0x55f1df1c46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.660+0000 7f3a757c5700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55f1df1c42e0 0x55f1df1c46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:54482/0 (socket says 172.21.15.12:54482) 2023-10-05T19:42:16.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.660+0000 7f3a757c5700 1 -- 172.21.15.12:0/2702364154 learned_addr learned my addr 172.21.15.12:0/2702364154 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:42:16.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.660+0000 7f3a74fc4700 1 --1- 172.21.15.12:0/2702364154 >> v1:172.21.15.8:6789/0 conn(0x55f1df1c4bf0 0x55f1df1cb430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:47412/0 (socket says 172.21.15.12:47412) 2023-10-05T19:42:16.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.660+0000 7f3a757c5700 1 -- 172.21.15.12:0/2702364154 >> v1:172.21.15.8:6789/0 conn(0x55f1df1c4bf0 legacy=0x55f1df1cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.660+0000 7f3a757c5700 1 -- 172.21.15.12:0/2702364154 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1df1cdb60 con 0x55f1df1c42e0 2023-10-05T19:42:16.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.660+0000 7f3a757c5700 1 --2- 172.21.15.12:0/2702364154 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1df1c42e0 0x55f1df1c46b0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f3a6800a5c0 tx=0x7f3a68005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=498a8b19273b5242 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.660+0000 7f3a6ffff700 1 -- 172.21.15.12:0/2702364154 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a6800e510 con 0x55f1df1c42e0 2023-10-05T19:42:16.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.660+0000 7f3a6ffff700 1 -- 172.21.15.12:0/2702364154 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a6800e670 con 0x55f1df1c42e0 2023-10-05T19:42:16.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.660+0000 7f3a6ffff700 1 -- 172.21.15.12:0/2702364154 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a6800e920 con 0x55f1df1c42e0 2023-10-05T19:42:16.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.660+0000 7f3a77c90e40 1 -- 172.21.15.12:0/2702364154 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1df1c42e0 msgr2=0x55f1df1c46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.660+0000 7f3a77c90e40 1 --2- 172.21.15.12:0/2702364154 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1df1c42e0 0x55f1df1c46b0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f3a6800a5c0 tx=0x7f3a68005e00 comp rx=0 tx=0).stop 2023-10-05T19:42:16.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.660+0000 7f3a77c90e40 1 -- 172.21.15.12:0/2702364154 shutdown_connections 2023-10-05T19:42:16.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.660+0000 7f3a77c90e40 1 --2- 172.21.15.12:0/2702364154 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1df1c42e0 0x55f1df1c46b0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.660+0000 7f3a77c90e40 1 -- 172.21.15.12:0/2702364154 >> 172.21.15.12:0/2702364154 conn(0x55f1df1bfe60 msgr2=0x55f1df1c2270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:16.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.660+0000 7f3a77c90e40 1 -- 172.21.15.12:0/2702364154 shutdown_connections 2023-10-05T19:42:16.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.660+0000 7f3a77c90e40 1 -- 172.21.15.12:0/2702364154 wait complete. 2023-10-05T19:42:16.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.664+0000 7f3a77c90e40 1 Processor -- start 2023-10-05T19:42:16.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.664+0000 7f3a77c90e40 1 -- start start 2023-10-05T19:42:16.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.664+0000 7f3a77c90e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1df1c42e0 0x55f1df1e4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.664+0000 7f3a77c90e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1df093e30 con 0x55f1df1c42e0 2023-10-05T19:42:16.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.664+0000 7f3a757c5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1df1c42e0 0x55f1df1e4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.664+0000 7f3a757c5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1df1c42e0 0x55f1df1e4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:54486/0 (socket says 172.21.15.12:54486) 2023-10-05T19:42:16.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.664+0000 7f3a757c5700 1 -- 172.21.15.12:0/1488214501 learned_addr learned my addr 172.21.15.12:0/1488214501 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:42:16.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.664+0000 7f3a757c5700 1 -- 172.21.15.12:0/1488214501 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a6800a180 con 0x55f1df1c42e0 2023-10-05T19:42:16.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.664+0000 7f3a757c5700 1 --2- 172.21.15.12:0/1488214501 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1df1c42e0 0x55f1df1e4ee0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f3a68005e40 tx=0x7f3a68013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=756de9b3b05e6abb server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.664+0000 7f3a6effd700 1 -- 172.21.15.12:0/1488214501 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a6800e7e0 con 0x55f1df1c42e0 2023-10-05T19:42:16.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.664+0000 7f3a6effd700 1 -- 172.21.15.12:0/1488214501 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a6800e940 con 0x55f1df1c42e0 2023-10-05T19:42:16.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.664+0000 7f3a6effd700 1 -- 172.21.15.12:0/1488214501 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a6800ec30 con 0x55f1df1c42e0 2023-10-05T19:42:16.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.664+0000 7f3a77c90e40 1 -- 172.21.15.12:0/1488214501 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1df1c42e0 msgr2=0x55f1df1e4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.664+0000 7f3a77c90e40 1 --2- 172.21.15.12:0/1488214501 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1df1c42e0 0x55f1df1e4ee0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f3a68005e40 tx=0x7f3a68013000 comp rx=0 tx=0).stop 2023-10-05T19:42:16.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.664+0000 7f3a77c90e40 1 -- 172.21.15.12:0/1488214501 shutdown_connections 2023-10-05T19:42:16.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.664+0000 7f3a77c90e40 1 --2- 172.21.15.12:0/1488214501 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1df1c42e0 0x55f1df1e4ee0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.664+0000 7f3a77c90e40 1 -- 172.21.15.12:0/1488214501 >> 172.21.15.12:0/1488214501 conn(0x55f1df1bfe60 msgr2=0x55f1df1e0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:16.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.664+0000 7f3a77c90e40 1 -- 172.21.15.12:0/1488214501 shutdown_connections 2023-10-05T19:42:16.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.664+0000 7f3a77c90e40 1 -- 172.21.15.12:0/1488214501 wait complete. 2023-10-05T19:42:16.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.668+0000 7f3a77c90e40 1 Processor -- start 2023-10-05T19:42:16.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.668+0000 7f3a77c90e40 1 -- start start 2023-10-05T19:42:16.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.668+0000 7f3a77c90e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1df1c42e0 0x55f1df1ebb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.668+0000 7f3a77c90e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1df066ed0 con 0x55f1df1c42e0 2023-10-05T19:42:16.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.668+0000 7f3a757c5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1df1c42e0 0x55f1df1ebb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.668+0000 7f3a757c5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1df1c42e0 0x55f1df1ebb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:54502/0 (socket says 172.21.15.12:54502) 2023-10-05T19:42:16.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.668+0000 7f3a757c5700 1 -- 172.21.15.12:0/2698548643 learned_addr learned my addr 172.21.15.12:0/2698548643 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:42:16.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.668+0000 7f3a757c5700 1 -- 172.21.15.12:0/2698548643 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1df1cdb60 con 0x55f1df1c42e0 2023-10-05T19:42:16.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.668+0000 7f3a757c5700 1 --2- 172.21.15.12:0/2698548643 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1df1c42e0 0x55f1df1ebb40 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f3a68005b90 tx=0x7f3a6801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.668+0000 7f3a6d7fa700 1 -- 172.21.15.12:0/2698548643 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a6800e7e0 con 0x55f1df1c42e0 2023-10-05T19:42:16.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.668+0000 7f3a77c90e40 1 -- 172.21.15.12:0/2698548643 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f1df1ec080 con 0x55f1df1c42e0 2023-10-05T19:42:16.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.668+0000 7f3a6d7fa700 1 -- 172.21.15.12:0/2698548643 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a6800ee30 con 0x55f1df1c42e0 2023-10-05T19:42:16.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.668+0000 7f3a6d7fa700 1 -- 172.21.15.12:0/2698548643 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a6801ccb0 con 0x55f1df1c42e0 2023-10-05T19:42:16.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.668+0000 7f3a77c90e40 1 -- 172.21.15.12:0/2698548643 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f1df1ec4c0 con 0x55f1df1c42e0 2023-10-05T19:42:16.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.668+0000 7f3a6d7fa700 1 -- 172.21.15.12:0/2698548643 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f3a68010440 con 0x55f1df1c42e0 2023-10-05T19:42:16.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.668+0000 7f3a77c90e40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:42:16.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.668+0000 7f3a77c90e40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:42:16.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.672+0000 7f3a6d7fa700 1 --2- 172.21.15.12:0/2698548643 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3a58030c00 0x7f3a580330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.672+0000 7f3a77c90e40 1 Processor -- start 2023-10-05T19:42:16.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.672+0000 7f3a6d7fa700 1 -- 172.21.15.12:0/2698548643 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7f3a680462e0 con 0x55f1df1c42e0 2023-10-05T19:42:16.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.672+0000 7f3a77c90e40 1 -- start start 2023-10-05T19:42:16.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.672+0000 7f3a74fc4700 1 --2- 172.21.15.12:0/2698548643 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3a58030c00 0x7f3a580330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.672+0000 7f3a77c90e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1df1fd240 0x55f1df1fd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.672+0000 7f3a77c90e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1df1fdb50 con 0x55f1df1fd240 2023-10-05T19:42:16.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.672+0000 7f3a757c5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1df1fd240 0x55f1df1fd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.672+0000 7f3a74fc4700 1 --2- 172.21.15.12:0/2698548643 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3a58030c00 0x7f3a580330b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3a60001e70 tx=0x7f3a60001390 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.672+0000 7f3a757c5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1df1fd240 0x55f1df1fd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:54516/0 (socket says 172.21.15.12:54516) 2023-10-05T19:42:16.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.672+0000 7f3a757c5700 1 -- 172.21.15.12:0/3358122072 learned_addr learned my addr 172.21.15.12:0/3358122072 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:42:16.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.672+0000 7f3a757c5700 1 -- 172.21.15.12:0/3358122072 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1df1fdc90 con 0x55f1df1fd240 2023-10-05T19:42:16.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.672+0000 7f3a757c5700 1 --2- 172.21.15.12:0/3358122072 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1df1fd240 0x55f1df1fd610 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f3a680374b0 tx=0x7f3a68044750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ad1f4d09111eab server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.672+0000 7f3a467e4700 1 -- 172.21.15.12:0/3358122072 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a6803c040 con 0x55f1df1fd240 2023-10-05T19:42:16.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.672+0000 7f3a467e4700 1 -- 172.21.15.12:0/3358122072 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a680460f0 con 0x55f1df1fd240 2023-10-05T19:42:16.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.672+0000 7f3a467e4700 1 -- 172.21.15.12:0/3358122072 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a6802e570 con 0x55f1df1fd240 2023-10-05T19:42:16.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.672+0000 7f3a77c90e40 1 -- 172.21.15.12:0/3358122072 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1df1fd240 msgr2=0x55f1df1fd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.672+0000 7f3a77c90e40 1 --2- 172.21.15.12:0/3358122072 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1df1fd240 0x55f1df1fd610 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f3a680374b0 tx=0x7f3a68044750 comp rx=0 tx=0).stop 2023-10-05T19:42:16.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.672+0000 7f3a77c90e40 1 -- 172.21.15.12:0/3358122072 shutdown_connections 2023-10-05T19:42:16.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.672+0000 7f3a77c90e40 1 --2- 172.21.15.12:0/3358122072 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1df1fd240 0x55f1df1fd610 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.672+0000 7f3a77c90e40 1 -- 172.21.15.12:0/3358122072 >> 172.21.15.12:0/3358122072 conn(0x55f1df1c4bf0 msgr2=0x55f1df1f9fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:16.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.672+0000 7f3a77c90e40 1 -- 172.21.15.12:0/3358122072 shutdown_connections 2023-10-05T19:42:16.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.672+0000 7f3a77c90e40 1 -- 172.21.15.12:0/3358122072 wait complete. 2023-10-05T19:42:16.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.672+0000 7f3a77c90e40 1 Processor -- start 2023-10-05T19:42:16.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.676+0000 7f3a77c90e40 1 -- start start 2023-10-05T19:42:16.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.676+0000 7f3a77c90e40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1df1fd240 0x55f1df204480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.676+0000 7f3a77c90e40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1df205480 con 0x55f1df1fd240 2023-10-05T19:42:16.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.676+0000 7f3a757c5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1df1fd240 0x55f1df204480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.676+0000 7f3a757c5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1df1fd240 0x55f1df204480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:54520/0 (socket says 172.21.15.12:54520) 2023-10-05T19:42:16.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.676+0000 7f3a757c5700 1 -- 172.21.15.12:0/1299953594 learned_addr learned my addr 172.21.15.12:0/1299953594 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:42:16.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.676+0000 7f3a757c5700 1 -- 172.21.15.12:0/1299953594 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1df1ec4c0 con 0x55f1df1fd240 2023-10-05T19:42:16.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.676+0000 7f3a757c5700 1 --2- 172.21.15.12:0/1299953594 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1df1fd240 0x55f1df204480 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f3a68037a60 tx=0x7f3a68043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.676+0000 7f3a44fe1700 1 -- 172.21.15.12:0/1299953594 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a6800fea0 con 0x55f1df1fd240 2023-10-05T19:42:16.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.676+0000 7f3a44fe1700 1 -- 172.21.15.12:0/1299953594 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a6804a070 con 0x55f1df1fd240 2023-10-05T19:42:16.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.676+0000 7f3a77c90e40 1 -- 172.21.15.12:0/1299953594 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f1df204a80 con 0x55f1df1fd240 2023-10-05T19:42:16.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.676+0000 7f3a44fe1700 1 -- 172.21.15.12:0/1299953594 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a68044c60 con 0x55f1df1fd240 2023-10-05T19:42:16.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.676+0000 7f3a77c90e40 1 -- 172.21.15.12:0/1299953594 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f1df204ee0 con 0x55f1df1fd240 2023-10-05T19:42:16.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.676+0000 7f3a44fe1700 1 -- 172.21.15.12:0/1299953594 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f3a68037ad0 con 0x55f1df1fd240 2023-10-05T19:42:16.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.676+0000 7f3a44fe1700 1 --2- 172.21.15.12:0/1299953594 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f39c4030c00 0x7f39c40330b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.676+0000 7f3a44fe1700 1 -- 172.21.15.12:0/1299953594 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7f3a6805abe0 con 0x55f1df1fd240 2023-10-05T19:42:16.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.676+0000 7f3a74fc4700 1 --2- 172.21.15.12:0/1299953594 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f39c4030c00 0x7f39c40330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.680+0000 7f3a74fc4700 1 --2- 172.21.15.12:0/1299953594 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f39c4030c00 0x7f39c40330b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3a60000c00 tx=0x7f3a60010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.680+0000 7f39be7fc700 20 reqs_thread_entry: start 2023-10-05T19:42:16.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.680+0000 7f3a77c90e40 20 try_to_resolve_local_entities(): before: realm_id= realm_name= zonegroup_id= zonegroup_name= zone_id= zone_name= 2023-10-05T19:42:16.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.680+0000 7f3a77c90e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:16.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.680+0000 7f3a77c90e40 1 --2- 172.21.15.12:0/2698548643 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55f1df2220e0 0x55f1df222560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.680+0000 7f3a77c90e40 1 -- 172.21.15.12:0/2698548643 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f1df222aa0 con 0x55f1df2220e0 2023-10-05T19:42:16.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.680+0000 7f3a75fc6700 1 --2- 172.21.15.12:0/2698548643 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55f1df2220e0 0x55f1df222560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.680+0000 7f3a75fc6700 1 --2- 172.21.15.12:0/2698548643 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55f1df2220e0 0x55f1df222560 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.680+0000 7f3a75fc6700 1 -- 172.21.15.12:0/2698548643 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3a7000a480 con 0x55f1df2220e0 2023-10-05T19:42:16.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.680+0000 7f3a77c90e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:16.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.680+0000 7f3a77c90e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:16.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.680+0000 7f3a77c90e40 1 -- 172.21.15.12:0/2698548643 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f1df235620 con 0x55f1df2220e0 2023-10-05T19:42:16.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.680+0000 7f3a75fc6700 1 -- 172.21.15.12:0/2698548643 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f3a7000a480 con 0x55f1df2220e0 2023-10-05T19:42:16.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.680+0000 7f3a77c90e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:42:16.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.680+0000 7f3a77c90e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:16.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.680+0000 7f3a77c90e40 1 --2- 172.21.15.12:0/2698548643 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55f1df2248c0 0x55f1df237ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.680+0000 7f3a77c90e40 1 -- 172.21.15.12:0/2698548643 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:3 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f1df238430 con 0x55f1df2248c0 2023-10-05T19:42:16.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.684+0000 7f3a757c5700 1 --2- 172.21.15.12:0/2698548643 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55f1df2248c0 0x55f1df237ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.684+0000 7f3a757c5700 1 --2- 172.21.15.12:0/2698548643 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55f1df2248c0 0x55f1df237ef0 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.684+0000 7f3a757c5700 1 -- 172.21.15.12:0/2698548643 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(3 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv4 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f3a68056070 con 0x55f1df2248c0 2023-10-05T19:42:16.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.684+0000 7f3a77c90e40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:42:16.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.684+0000 7f3a77c90e40 1 --2- 172.21.15.12:0/2698548643 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55f1df239d00 0x55f1df23c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.684+0000 7f3a77c90e40 1 -- 172.21.15.12:0/2698548643 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:4 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f1df23c620 con 0x55f1df239d00 2023-10-05T19:42:16.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.684+0000 7f3a74fc4700 1 --2- 172.21.15.12:0/2698548643 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55f1df239d00 0x55f1df23c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.684+0000 7f3a74fc4700 1 --2- 172.21.15.12:0/2698548643 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55f1df239d00 0x55f1df23c0e0 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.688+0000 7f3a74fc4700 1 -- 172.21.15.12:0/2698548643 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(4 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f3a6001c460 con 0x55f1df239d00 2023-10-05T19:42:16.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.688+0000 7f3a74fc4700 1 -- 172.21.15.12:0/2698548643 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f1df23c620 con 0x55f1df2248c0 2023-10-05T19:42:16.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.688+0000 7f3a757c5700 1 -- 172.21.15.12:0/2698548643 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(5 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a68056070 con 0x55f1df2248c0 2023-10-05T19:42:16.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.688+0000 7f3a757c5700 1 --2- 172.21.15.12:0/2698548643 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f3a68059030 0x7f3a68015900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.688+0000 7f3a757c5700 1 -- 172.21.15.12:0/2698548643 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:6 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f1df23c620 con 0x7f3a68059030 2023-10-05T19:42:16.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.688+0000 7f3a75fc6700 1 --2- 172.21.15.12:0/2698548643 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f3a68059030 0x7f3a68015900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.688+0000 7f3a75fc6700 1 --2- 172.21.15.12:0/2698548643 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f3a68059030 0x7f3a68015900 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.688+0000 7f3a75fc6700 1 -- 172.21.15.12:0/2698548643 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a7000a480 con 0x7f3a68059030 2023-10-05T19:42:16.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.688+0000 7f3a75fc6700 1 -- 172.21.15.12:0/2698548643 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:7 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f1df23c620 con 0x55f1df2220e0 2023-10-05T19:42:16.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.688+0000 7f3a75fc6700 1 -- 172.21.15.12:0/2698548643 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a7000a480 con 0x55f1df2220e0 2023-10-05T19:42:16.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.688+0000 7f3a75fc6700 1 -- 172.21.15.12:0/2698548643 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:8 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f1df23c620 con 0x55f1df2220e0 2023-10-05T19:42:16.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.692+0000 7f3a75fc6700 1 -- 172.21.15.12:0/2698548643 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a7000a480 con 0x55f1df2220e0 2023-10-05T19:42:16.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.692+0000 7f3a75fc6700 1 -- 172.21.15.12:0/2698548643 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f1df23c620 con 0x55f1df2220e0 2023-10-05T19:42:16.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.692+0000 7f3a75fc6700 1 -- 172.21.15.12:0/2698548643 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a7000a480 con 0x55f1df2220e0 2023-10-05T19:42:16.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.692+0000 7f3a75fc6700 1 -- 172.21.15.12:0/2698548643 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:10 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f1df23c620 con 0x7f3a68059030 2023-10-05T19:42:16.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.692+0000 7f3a75fc6700 1 -- 172.21.15.12:0/2698548643 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a7000a480 con 0x7f3a68059030 2023-10-05T19:42:16.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.692+0000 7f3a75fc6700 1 -- 172.21.15.12:0/2698548643 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:11 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f1df23c620 con 0x55f1df2248c0 2023-10-05T19:42:16.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.692+0000 7f3a757c5700 1 -- 172.21.15.12:0/2698548643 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(11 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a68056070 con 0x55f1df2248c0 2023-10-05T19:42:16.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.692+0000 7f3a757c5700 1 -- 172.21.15.12:0/2698548643 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:12 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f1df23c620 con 0x55f1df2248c0 2023-10-05T19:42:16.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.692+0000 7f3a757c5700 1 -- 172.21.15.12:0/2698548643 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a68056070 con 0x55f1df2248c0 2023-10-05T19:42:16.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.692+0000 7f3a757c5700 1 -- 172.21.15.12:0/2698548643 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:13 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f1df23c620 con 0x55f1df2220e0 2023-10-05T19:42:16.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.692+0000 7f3a75fc6700 1 -- 172.21.15.12:0/2698548643 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a7000a480 con 0x55f1df2220e0 2023-10-05T19:42:16.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.692+0000 7f3a75fc6700 1 -- 172.21.15.12:0/2698548643 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:14 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f1df23c620 con 0x7f3a68059030 2023-10-05T19:42:16.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.696+0000 7f3a75fc6700 1 -- 172.21.15.12:0/2698548643 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a7000a480 con 0x7f3a68059030 2023-10-05T19:42:16.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.696+0000 7f3a75fc6700 1 -- 172.21.15.12:0/2698548643 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f1df23c620 con 0x55f1df2220e0 2023-10-05T19:42:16.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.696+0000 7f3a75fc6700 1 -- 172.21.15.12:0/2698548643 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a7000a480 con 0x55f1df2220e0 2023-10-05T19:42:16.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.696+0000 7f3a75fc6700 1 -- 172.21.15.12:0/2698548643 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:16 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f1df23c620 con 0x55f1df2248c0 2023-10-05T19:42:16.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.696+0000 7f3a757c5700 1 -- 172.21.15.12:0/2698548643 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a68056070 con 0x55f1df2248c0 2023-10-05T19:42:16.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.696+0000 7f3a757c5700 1 -- 172.21.15.12:0/2698548643 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:17 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f1df23c620 con 0x55f1df239d00 2023-10-05T19:42:16.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.696+0000 7f3a74fc4700 1 -- 172.21.15.12:0/2698548643 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a6001c460 con 0x55f1df239d00 2023-10-05T19:42:16.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.696+0000 7f3a74fc4700 1 -- 172.21.15.12:0/2698548643 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:18 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f1df23c620 con 0x55f1df239d00 2023-10-05T19:42:16.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.696+0000 7f3a74fc4700 1 -- 172.21.15.12:0/2698548643 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(18 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a6001c460 con 0x55f1df239d00 2023-10-05T19:42:16.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.696+0000 7f3a74fc4700 1 -- 172.21.15.12:0/2698548643 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:19 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f1df23c620 con 0x7f3a68059030 2023-10-05T19:42:16.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.696+0000 7f3a75fc6700 1 -- 172.21.15.12:0/2698548643 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a7000a480 con 0x7f3a68059030 2023-10-05T19:42:16.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.696+0000 7f3a75fc6700 1 -- 172.21.15.12:0/2698548643 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f1df23c620 con 0x55f1df239d00 2023-10-05T19:42:16.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.700+0000 7f3a74fc4700 1 -- 172.21.15.12:0/2698548643 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a6001c460 con 0x55f1df239d00 2023-10-05T19:42:16.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.700+0000 7f3a74fc4700 1 -- 172.21.15.12:0/2698548643 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:21 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f1df23c620 con 0x55f1df239d00 2023-10-05T19:42:16.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.700+0000 7f3a74fc4700 1 -- 172.21.15.12:0/2698548643 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a6001c460 con 0x55f1df239d00 2023-10-05T19:42:16.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.700+0000 7f3a74fc4700 1 -- 172.21.15.12:0/2698548643 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:22 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f1df23c620 con 0x55f1df2220e0 2023-10-05T19:42:16.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.700+0000 7f3a75fc6700 1 -- 172.21.15.12:0/2698548643 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a7000a480 con 0x55f1df2220e0 2023-10-05T19:42:16.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.700+0000 7f3a75fc6700 1 -- 172.21.15.12:0/2698548643 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:23 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f1df23c620 con 0x55f1df2220e0 2023-10-05T19:42:16.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.700+0000 7f3a75fc6700 1 -- 172.21.15.12:0/2698548643 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a7000a480 con 0x55f1df2220e0 2023-10-05T19:42:16.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.700+0000 7f3a75fc6700 1 -- 172.21.15.12:0/2698548643 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:24 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f1df23c620 con 0x7f3a68059030 2023-10-05T19:42:16.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.704+0000 7f3a75fc6700 1 -- 172.21.15.12:0/2698548643 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a7000a480 con 0x7f3a68059030 2023-10-05T19:42:16.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.704+0000 7f3a75fc6700 1 -- 172.21.15.12:0/2698548643 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:25 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f1df23c620 con 0x55f1df239d00 2023-10-05T19:42:16.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.704+0000 7f3a74fc4700 1 -- 172.21.15.12:0/2698548643 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(25 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a6001c460 con 0x55f1df239d00 2023-10-05T19:42:16.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.704+0000 7f3a74fc4700 1 -- 172.21.15.12:0/2698548643 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:26 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f1df23c620 con 0x55f1df2248c0 2023-10-05T19:42:16.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.704+0000 7f3a757c5700 1 -- 172.21.15.12:0/2698548643 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a68056070 con 0x55f1df2248c0 2023-10-05T19:42:16.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.704+0000 7f3a757c5700 1 -- 172.21.15.12:0/2698548643 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:27 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f1df23c620 con 0x55f1df2220e0 2023-10-05T19:42:16.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.704+0000 7f3a75fc6700 1 -- 172.21.15.12:0/2698548643 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a7000a480 con 0x55f1df2220e0 2023-10-05T19:42:16.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.704+0000 7f3a75fc6700 1 -- 172.21.15.12:0/2698548643 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:28 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f1df23c620 con 0x7f3a68059030 2023-10-05T19:42:16.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.704+0000 7f3a75fc6700 1 -- 172.21.15.12:0/2698548643 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a7000a480 con 0x7f3a68059030 2023-10-05T19:42:16.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.704+0000 7f3a75fc6700 1 -- 172.21.15.12:0/2698548643 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:29 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f1df23c620 con 0x55f1df239d00 2023-10-05T19:42:16.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.708+0000 7f3a74fc4700 1 -- 172.21.15.12:0/2698548643 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(29 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a6001c460 con 0x55f1df239d00 2023-10-05T19:42:16.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.708+0000 7f3a74fc4700 1 -- 172.21.15.12:0/2698548643 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f1df23c620 con 0x7f3a68059030 2023-10-05T19:42:16.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.708+0000 7f3a75fc6700 1 -- 172.21.15.12:0/2698548643 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a7000a480 con 0x7f3a68059030 2023-10-05T19:42:16.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.708+0000 7f3a75fc6700 1 -- 172.21.15.12:0/2698548643 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:31 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f1df23c620 con 0x55f1df2248c0 2023-10-05T19:42:16.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.708+0000 7f3a757c5700 1 -- 172.21.15.12:0/2698548643 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a68056070 con 0x55f1df2248c0 2023-10-05T19:42:16.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.708+0000 7f3a757c5700 1 -- 172.21.15.12:0/2698548643 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:32 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f1df23c620 con 0x55f1df239d00 2023-10-05T19:42:16.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.708+0000 7f3a74fc4700 1 -- 172.21.15.12:0/2698548643 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a6001c460 con 0x55f1df239d00 2023-10-05T19:42:16.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.708+0000 7f3a74fc4700 1 -- 172.21.15.12:0/2698548643 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:33 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f1df23c620 con 0x7f3a68059030 2023-10-05T19:42:16.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.712+0000 7f3a75fc6700 1 -- 172.21.15.12:0/2698548643 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3a7000a480 con 0x7f3a68059030 2023-10-05T19:42:16.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.712+0000 7f3a75fc6700 1 -- 172.21.15.12:0/2698548643 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:34 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f1df23c620 con 0x55f1df2220e0 2023-10-05T19:42:16.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.712+0000 7f3a75fc6700 1 -- 172.21.15.12:0/2698548643 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(34 [pgnls start_epoch 30 out=115b] v30'3 uv3 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f3a7000a480 con 0x55f1df2220e0 2023-10-05T19:42:16.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.712+0000 7f3a75fc6700 1 -- 172.21.15.12:0/2698548643 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:35 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f1df23c620 con 0x55f1df2248c0 2023-10-05T19:42:16.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.712+0000 7f3a757c5700 1 -- 172.21.15.12:0/2698548643 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(35 [pgnls start_epoch 30 out=107b] v30'4 uv4 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f3a68056070 con 0x55f1df2248c0 2023-10-05T19:42:16.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.712+0000 7f3a77c90e40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:42:16.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.712+0000 7f3a77c90e40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:42:16.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.712+0000 7f3a77c90e40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:42:16.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.712+0000 7f3a77c90e40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:42:16.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.712+0000 7f3a77c90e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:16.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.712+0000 7f3a77c90e40 1 -- 172.21.15.12:0/2698548643 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f1df2387e0 con 0x55f1df2220e0 2023-10-05T19:42:16.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.712+0000 7f3a75fc6700 1 -- 172.21.15.12:0/2698548643 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(36 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3a7000a480 con 0x55f1df2220e0 2023-10-05T19:42:16.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.712+0000 7f3a77c90e40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:16.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.712+0000 7f3a77c90e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:16.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.712+0000 7f3a77c90e40 1 -- 172.21.15.12:0/2698548643 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:37 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f1df238b90 con 0x55f1df2220e0 2023-10-05T19:42:16.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.716+0000 7f3a75fc6700 1 -- 172.21.15.12:0/2698548643 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(37 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f3a7000a480 con 0x55f1df2220e0 2023-10-05T19:42:16.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.716+0000 7f3a77c90e40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:42:16.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.716+0000 7f3a77c90e40 20 rados->read ofs=0 len=0 2023-10-05T19:42:16.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.716+0000 7f3a77c90e40 1 -- 172.21.15.12:0/2698548643 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:38 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55f1df223020 con 0x55f1df239d00 2023-10-05T19:42:16.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.716+0000 7f3a74fc4700 1 -- 172.21.15.12:0/2698548643 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(38 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7f3a6001c460 con 0x55f1df239d00 2023-10-05T19:42:16.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.716+0000 7f3a77c90e40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:42:16.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.716+0000 7f3a77c90e40 1 -- 172.21.15.12:0/2698548643 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f3a68059030 msgr2=0x7f3a68015900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.716+0000 7f3a77c90e40 1 --2- 172.21.15.12:0/2698548643 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7f3a68059030 0x7f3a68015900 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.716+0000 7f3a77c90e40 1 -- 172.21.15.12:0/2698548643 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55f1df2220e0 msgr2=0x55f1df222560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.716+0000 7f3a77c90e40 1 --2- 172.21.15.12:0/2698548643 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55f1df2220e0 0x55f1df222560 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.716+0000 7f3a77c90e40 1 -- 172.21.15.12:0/2698548643 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55f1df2248c0 msgr2=0x55f1df237ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.716+0000 7f3a77c90e40 1 --2- 172.21.15.12:0/2698548643 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55f1df2248c0 0x55f1df237ef0 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.716+0000 7f3a77c90e40 1 -- 172.21.15.12:0/2698548643 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55f1df239d00 msgr2=0x55f1df23c0e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.716+0000 7f3a77c90e40 1 --2- 172.21.15.12:0/2698548643 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55f1df239d00 0x55f1df23c0e0 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.716+0000 7f3a77c90e40 1 -- 172.21.15.12:0/2698548643 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3a58030c00 msgr2=0x7f3a580330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.716+0000 7f3a77c90e40 1 --2- 172.21.15.12:0/2698548643 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f3a58030c00 0x7f3a580330b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3a60001e70 tx=0x7f3a60001390 comp rx=0 tx=0).stop 2023-10-05T19:42:16.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.716+0000 7f3a77c90e40 1 -- 172.21.15.12:0/2698548643 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1df1c42e0 msgr2=0x55f1df1ebb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.716+0000 7f3a77c90e40 1 --2- 172.21.15.12:0/2698548643 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1df1c42e0 0x55f1df1ebb40 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f3a68005b90 tx=0x7f3a6801c000 comp rx=0 tx=0).stop 2023-10-05T19:42:16.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.716+0000 7f3a75fc6700 1 -- 172.21.15.12:0/2698548643 reap_dead start 2023-10-05T19:42:16.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.716+0000 7f3a75fc6700 1 -- 172.21.15.12:0/2698548643 reap_dead start 2023-10-05T19:42:16.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.716+0000 7f3a77c90e40 1 -- 172.21.15.12:0/2698548643 shutdown_connections 2023-10-05T19:42:16.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.716+0000 7f3a77c90e40 1 -- 172.21.15.12:0/2698548643 >> 172.21.15.12:0/2698548643 conn(0x55f1df1bfe60 msgr2=0x55f1df1df520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:16.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.716+0000 7f3a77c90e40 1 -- 172.21.15.12:0/2698548643 shutdown_connections 2023-10-05T19:42:16.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.716+0000 7f3a77c90e40 1 -- 172.21.15.12:0/2698548643 wait complete. 2023-10-05T19:42:16.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.720+0000 7f3a77c90e40 1 -- 172.21.15.12:0/1299953594 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f39c4030c00 msgr2=0x7f39c40330b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.720+0000 7f3a77c90e40 1 --2- 172.21.15.12:0/1299953594 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f39c4030c00 0x7f39c40330b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3a60000c00 tx=0x7f3a60010000 comp rx=0 tx=0).stop 2023-10-05T19:42:16.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.720+0000 7f3a77c90e40 1 -- 172.21.15.12:0/1299953594 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1df1fd240 msgr2=0x55f1df204480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.720+0000 7f3a77c90e40 1 --2- 172.21.15.12:0/1299953594 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1df1fd240 0x55f1df204480 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f3a68037a60 tx=0x7f3a68043000 comp rx=0 tx=0).stop 2023-10-05T19:42:16.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.720+0000 7f3a77c90e40 1 -- 172.21.15.12:0/1299953594 shutdown_connections 2023-10-05T19:42:16.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.720+0000 7f3a77c90e40 1 --2- 172.21.15.12:0/1299953594 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f39c4030c00 0x7f39c40330b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.720+0000 7f3a77c90e40 1 --2- 172.21.15.12:0/1299953594 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f1df1fd240 0x55f1df204480 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.720+0000 7f3a77c90e40 1 -- 172.21.15.12:0/1299953594 >> 172.21.15.12:0/1299953594 conn(0x55f1df1c4bf0 msgr2=0x55f1df1f9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:16.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.720+0000 7f3a77c90e40 1 -- 172.21.15.12:0/1299953594 shutdown_connections 2023-10-05T19:42:16.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.720+0000 7f3a77c90e40 1 -- 172.21.15.12:0/1299953594 wait complete. 2023-10-05T19:42:16.726 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:42:16.726 INFO:teuthology.orchestra.run.smithi012.stdout: "id": "1602ab00-ae0f-4921-ad63-b9cfe7bc77db", 2023-10-05T19:42:16.726 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default", 2023-10-05T19:42:16.726 INFO:teuthology.orchestra.run.smithi012.stdout: "api_name": "default", 2023-10-05T19:42:16.727 INFO:teuthology.orchestra.run.smithi012.stdout: "is_master": "true", 2023-10-05T19:42:16.727 INFO:teuthology.orchestra.run.smithi012.stdout: "endpoints": [], 2023-10-05T19:42:16.727 INFO:teuthology.orchestra.run.smithi012.stdout: "hostnames": [], 2023-10-05T19:42:16.727 INFO:teuthology.orchestra.run.smithi012.stdout: "hostnames_s3website": [], 2023-10-05T19:42:16.727 INFO:teuthology.orchestra.run.smithi012.stdout: "master_zone": "93ac49f4-ff49-4f99-811b-6453b28decd5", 2023-10-05T19:42:16.727 INFO:teuthology.orchestra.run.smithi012.stdout: "zones": [ 2023-10-05T19:42:16.727 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:42:16.727 INFO:teuthology.orchestra.run.smithi012.stdout: "id": "93ac49f4-ff49-4f99-811b-6453b28decd5", 2023-10-05T19:42:16.727 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default", 2023-10-05T19:42:16.727 INFO:teuthology.orchestra.run.smithi012.stdout: "endpoints": [], 2023-10-05T19:42:16.727 INFO:teuthology.orchestra.run.smithi012.stdout: "log_meta": "false", 2023-10-05T19:42:16.728 INFO:teuthology.orchestra.run.smithi012.stdout: "log_data": "false", 2023-10-05T19:42:16.728 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket_index_max_shards": 11, 2023-10-05T19:42:16.728 INFO:teuthology.orchestra.run.smithi012.stdout: "read_only": "false", 2023-10-05T19:42:16.728 INFO:teuthology.orchestra.run.smithi012.stdout: "tier_type": "", 2023-10-05T19:42:16.728 INFO:teuthology.orchestra.run.smithi012.stdout: "sync_from_all": "true", 2023-10-05T19:42:16.728 INFO:teuthology.orchestra.run.smithi012.stdout: "sync_from": [], 2023-10-05T19:42:16.728 INFO:teuthology.orchestra.run.smithi012.stdout: "redirect_zone": "" 2023-10-05T19:42:16.728 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:42:16.728 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:42:16.728 INFO:teuthology.orchestra.run.smithi012.stdout: "placement_targets": [ 2023-10-05T19:42:16.728 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:42:16.728 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default-placement", 2023-10-05T19:42:16.729 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2023-10-05T19:42:16.729 INFO:teuthology.orchestra.run.smithi012.stdout: "storage_classes": [ 2023-10-05T19:42:16.729 INFO:teuthology.orchestra.run.smithi012.stdout: "STANDARD" 2023-10-05T19:42:16.729 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2023-10-05T19:42:16.729 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:42:16.729 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:42:16.729 INFO:teuthology.orchestra.run.smithi012.stdout: "default_placement": "default-placement", 2023-10-05T19:42:16.729 INFO:teuthology.orchestra.run.smithi012.stdout: "realm_id": "", 2023-10-05T19:42:16.729 INFO:teuthology.orchestra.run.smithi012.stdout: "sync_policy": { 2023-10-05T19:42:16.729 INFO:teuthology.orchestra.run.smithi012.stdout: "groups": [] 2023-10-05T19:42:16.729 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:42:16.729 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:42:16.730 DEBUG:tasks.util.rgw: json result: {'id': '1602ab00-ae0f-4921-ad63-b9cfe7bc77db', 'name': 'default', 'api_name': 'default', 'is_master': 'true', 'endpoints': [], 'hostnames': [], 'hostnames_s3website': [], 'master_zone': '93ac49f4-ff49-4f99-811b-6453b28decd5', 'zones': [{'id': '93ac49f4-ff49-4f99-811b-6453b28decd5', 'name': 'default', 'endpoints': [], 'log_meta': 'false', 'log_data': 'false', 'bucket_index_max_shards': 11, 'read_only': 'false', 'tier_type': '', 'sync_from_all': 'true', 'sync_from': [], 'redirect_zone': ''}], 'placement_targets': [{'name': 'default-placement', 'tags': [], 'storage_classes': ['STANDARD']}], 'default_placement': 'default-placement', 'realm_id': '', 'sync_policy': {'groups': []}} 2023-10-05T19:42:16.733 INFO:teuthology.run_tasks:Running task radosgw-admin-rest... 2023-10-05T19:42:16.752 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'ada', '--display-name', 'Ms. Admin User', '--access-key', 'MH1WC2XQ1S8UISFDZC8W', '--secret', 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', '--max-buckets', '0', '--caps', 'users=read, write; usage=read, write; buckets=read, write; zone=read, write; info=read;ratelimit=read, write'] 2023-10-05T19:42:16.752 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'ada', '--display-name', 'Ms. Admin User', '--access-key', 'MH1WC2XQ1S8UISFDZC8W', '--secret', 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', '--max-buckets', '0', '--caps', 'users=read, write; usage=read, write; buckets=read, write; zone=read, write; info=read;ratelimit=read, write'] 2023-10-05T19:42:16.753 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid ada --display-name 'Ms. Admin User' --access-key MH1WC2XQ1S8UISFDZC8W --secret dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG --max-buckets 0 --caps 'users=read, write; usage=read, write; buckets=read, write; zone=read, write; info=read;ratelimit=read, write' 2023-10-05T19:42:16.805 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2023-10-05T19:42:16.805 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2023-10-05T19:42:16.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.800+0000 7efc4759be40 1 Processor -- start 2023-10-05T19:42:16.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.800+0000 7efc4759be40 1 -- start start 2023-10-05T19:42:16.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.800+0000 7efc4759be40 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55b52dadcdd0 0x55b52dadd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.800+0000 7efc4759be40 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55b52da29bd0 con 0x55b52dadd6e0 2023-10-05T19:42:16.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.800+0000 7efc4759be40 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x55b52d9e6ee0 con 0x55b52dadcdd0 2023-10-05T19:42:16.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.800+0000 7efc450d0700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x55b52dadcdd0 0x55b52dadd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.800+0000 7efc448cf700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x55b52dadd6e0 0x55b52dade180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.12:47422/0 (socket says 172.21.15.12:47422) 2023-10-05T19:42:16.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.800+0000 7efc448cf700 1 -- 172.21.15.12:0/2750840302 learned_addr learned my addr 172.21.15.12:0/2750840302 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-10-05T19:42:16.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.800+0000 7efc450d0700 1 -- 172.21.15.12:0/2750840302 >> v1:172.21.15.8:6789/0 conn(0x55b52dadd6e0 legacy=0x55b52dade180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.804+0000 7efc3ffff700 1 -- 172.21.15.12:0/2750840302 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2308148981 0 0) 0x55b52da29bd0 con 0x55b52dadd6e0 2023-10-05T19:42:16.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.804+0000 7efc450d0700 1 -- 172.21.15.12:0/2750840302 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b52dade890 con 0x55b52dadcdd0 2023-10-05T19:42:16.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.804+0000 7efc450d0700 1 --2- 172.21.15.12:0/2750840302 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b52dadcdd0 0x55b52dadd1a0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7efc3800a520 tx=0x7efc38005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e578b26f184853b server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.804+0000 7efc3ffff700 1 -- 172.21.15.12:0/2750840302 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc3800f590 con 0x55b52dadcdd0 2023-10-05T19:42:16.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.804+0000 7efc3ffff700 1 -- 172.21.15.12:0/2750840302 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc3800f6f0 con 0x55b52dadcdd0 2023-10-05T19:42:16.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.804+0000 7efc3ffff700 1 -- 172.21.15.12:0/2750840302 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc3800f880 con 0x55b52dadcdd0 2023-10-05T19:42:16.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.804+0000 7efc4759be40 1 -- 172.21.15.12:0/2750840302 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b52dadcdd0 msgr2=0x55b52dadd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.804+0000 7efc4759be40 1 --2- 172.21.15.12:0/2750840302 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b52dadcdd0 0x55b52dadd1a0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7efc3800a520 tx=0x7efc38005d40 comp rx=0 tx=0).stop 2023-10-05T19:42:16.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.804+0000 7efc4759be40 1 -- 172.21.15.12:0/2750840302 shutdown_connections 2023-10-05T19:42:16.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.804+0000 7efc4759be40 1 --2- 172.21.15.12:0/2750840302 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b52dadcdd0 0x55b52dadd1a0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.804+0000 7efc4759be40 1 -- 172.21.15.12:0/2750840302 >> 172.21.15.12:0/2750840302 conn(0x55b52dad87f0 msgr2=0x55b52dadac00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:16.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.804+0000 7efc4759be40 1 -- 172.21.15.12:0/2750840302 shutdown_connections 2023-10-05T19:42:16.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.804+0000 7efc4759be40 1 -- 172.21.15.12:0/2750840302 wait complete. 2023-10-05T19:42:16.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.804+0000 7efc4759be40 1 Processor -- start 2023-10-05T19:42:16.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.804+0000 7efc4759be40 1 -- start start 2023-10-05T19:42:16.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.804+0000 7efc4759be40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b52dadcdd0 0x55b52da52880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.804+0000 7efc4759be40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b52d9aae30 con 0x55b52dadcdd0 2023-10-05T19:42:16.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.804+0000 7efc450d0700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b52dadcdd0 0x55b52da52880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.804+0000 7efc450d0700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b52dadcdd0 0x55b52da52880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:54536/0 (socket says 172.21.15.12:54536) 2023-10-05T19:42:16.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.804+0000 7efc450d0700 1 -- 172.21.15.12:0/3457679212 learned_addr learned my addr 172.21.15.12:0/3457679212 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:42:16.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.808+0000 7efc450d0700 1 -- 172.21.15.12:0/3457679212 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc3800a0e0 con 0x55b52dadcdd0 2023-10-05T19:42:16.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.808+0000 7efc450d0700 1 --2- 172.21.15.12:0/3457679212 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b52dadcdd0 0x55b52da52880 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7efc38005b20 tx=0x7efc38014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5102fa7871096a07 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.808+0000 7efc3effd700 1 -- 172.21.15.12:0/3457679212 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc3800f860 con 0x55b52dadcdd0 2023-10-05T19:42:16.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.808+0000 7efc3effd700 1 -- 172.21.15.12:0/3457679212 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc3800f9c0 con 0x55b52dadcdd0 2023-10-05T19:42:16.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.808+0000 7efc3effd700 1 -- 172.21.15.12:0/3457679212 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc3800fb50 con 0x55b52dadcdd0 2023-10-05T19:42:16.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.808+0000 7efc4759be40 1 -- 172.21.15.12:0/3457679212 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b52dadcdd0 msgr2=0x55b52da52880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.808+0000 7efc4759be40 1 --2- 172.21.15.12:0/3457679212 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b52dadcdd0 0x55b52da52880 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7efc38005b20 tx=0x7efc38014000 comp rx=0 tx=0).stop 2023-10-05T19:42:16.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.808+0000 7efc4759be40 1 -- 172.21.15.12:0/3457679212 shutdown_connections 2023-10-05T19:42:16.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.808+0000 7efc4759be40 1 --2- 172.21.15.12:0/3457679212 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b52dadcdd0 0x55b52da52880 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.808+0000 7efc4759be40 1 -- 172.21.15.12:0/3457679212 >> 172.21.15.12:0/3457679212 conn(0x55b52dad87f0 msgr2=0x55b52daf78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:16.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.808+0000 7efc4759be40 1 -- 172.21.15.12:0/3457679212 shutdown_connections 2023-10-05T19:42:16.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.808+0000 7efc4759be40 1 -- 172.21.15.12:0/3457679212 wait complete. 2023-10-05T19:42:16.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.808+0000 7efc4759be40 1 Processor -- start 2023-10-05T19:42:16.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.808+0000 7efc4759be40 1 -- start start 2023-10-05T19:42:16.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.808+0000 7efc4759be40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b52dadcdd0 0x55b52db04c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.808+0000 7efc4759be40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b52d97ded0 con 0x55b52dadcdd0 2023-10-05T19:42:16.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.808+0000 7efc450d0700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b52dadcdd0 0x55b52db04c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.808+0000 7efc450d0700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b52dadcdd0 0x55b52db04c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:54552/0 (socket says 172.21.15.12:54552) 2023-10-05T19:42:16.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.808+0000 7efc450d0700 1 -- 172.21.15.12:0/1591481545 learned_addr learned my addr 172.21.15.12:0/1591481545 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:42:16.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.808+0000 7efc450d0700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b52dade890 con 0x55b52dadcdd0 2023-10-05T19:42:16.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.808+0000 7efc450d0700 1 --2- 172.21.15.12:0/1591481545 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b52dadcdd0 0x55b52db04c10 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7efc38005b20 tx=0x7efc3801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.812+0000 7efc3d7fa700 1 -- 172.21.15.12:0/1591481545 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc38010420 con 0x55b52dadcdd0 2023-10-05T19:42:16.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.812+0000 7efc3d7fa700 1 -- 172.21.15.12:0/1591481545 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc38010580 con 0x55b52dadcdd0 2023-10-05T19:42:16.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.812+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b52db05150 con 0x55b52dadcdd0 2023-10-05T19:42:16.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.812+0000 7efc3d7fa700 1 -- 172.21.15.12:0/1591481545 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc38010710 con 0x55b52dadcdd0 2023-10-05T19:42:16.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.812+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b52db05590 con 0x55b52dadcdd0 2023-10-05T19:42:16.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.812+0000 7efc3d7fa700 1 -- 172.21.15.12:0/1591481545 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7efc38010980 con 0x55b52dadcdd0 2023-10-05T19:42:16.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.812+0000 7efc4759be40 20 rados->read ofs=0 len=0 2023-10-05T19:42:16.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.812+0000 7efc3d7fa700 1 --2- 172.21.15.12:0/1591481545 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7efc28030b70 0x7efc28033020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.812+0000 7efc3d7fa700 1 -- 172.21.15.12:0/1591481545 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7efc3802b070 con 0x55b52dadcdd0 2023-10-05T19:42:16.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.812+0000 7efc4759be40 1 --2- 172.21.15.12:0/1591481545 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55b52dadd6e0 0x55b52db106a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.812+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db10d50 con 0x55b52dadd6e0 2023-10-05T19:42:16.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.812+0000 7efc448cf700 1 --2- 172.21.15.12:0/1591481545 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7efc28030b70 0x7efc28033020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.812+0000 7efc458d1700 1 --2- 172.21.15.12:0/1591481545 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55b52dadd6e0 0x55b52db106a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.812+0000 7efc448cf700 1 --2- 172.21.15.12:0/1591481545 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7efc28030b70 0x7efc28033020 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efc300011e0 tx=0x7efc300013a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.816+0000 7efc458d1700 1 --2- 172.21.15.12:0/1591481545 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55b52dadd6e0 0x55b52db106a0 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.816+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.1 v2:172.21.15.12:6808/29933 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7efc4000a500 con 0x55b52dadd6e0 2023-10-05T19:42:16.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.816+0000 7efc4759be40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:16.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.816+0000 7efc4759be40 20 realm 2023-10-05T19:42:16.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.816+0000 7efc4759be40 20 rados->read ofs=0 len=0 2023-10-05T19:42:16.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.816+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db0dd00 con 0x55b52dadd6e0 2023-10-05T19:42:16.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.816+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.1 v2:172.21.15.12:6808/29933 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7efc4000a500 con 0x55b52dadd6e0 2023-10-05T19:42:16.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.816+0000 7efc4759be40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:16.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.816+0000 7efc4759be40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-10-05T19:42:16.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.816+0000 7efc4759be40 20 rados->read ofs=0 len=0 2023-10-05T19:42:16.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.816+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db11830 con 0x55b52dadd6e0 2023-10-05T19:42:16.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.816+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.1 v2:172.21.15.12:6808/29933 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7efc4000a500 con 0x55b52dadd6e0 2023-10-05T19:42:16.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.816+0000 7efc4759be40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:16.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.816+0000 7efc4759be40 20 rados->read ofs=0 len=0 2023-10-05T19:42:16.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.816+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db11440 con 0x55b52dadd6e0 2023-10-05T19:42:16.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.816+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.1 v2:172.21.15.12:6808/29933 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7efc4000a500 con 0x55b52dadd6e0 2023-10-05T19:42:16.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.816+0000 7efc4759be40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:42:16.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.816+0000 7efc4759be40 20 rados->read ofs=0 len=0 2023-10-05T19:42:16.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.816+0000 7efc4759be40 1 --2- 172.21.15.12:0/1591481545 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55b52db12bd0 0x55b52db14fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.816+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:5 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db154f0 con 0x55b52db12bd0 2023-10-05T19:42:16.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.816+0000 7efc450d0700 1 --2- 172.21.15.12:0/1591481545 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55b52db12bd0 0x55b52db14fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.816+0000 7efc450d0700 1 --2- 172.21.15.12:0/1591481545 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55b52db12bd0 0x55b52db14fb0 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.816+0000 7efc450d0700 1 -- 172.21.15.12:0/1591481545 <== osd.2 v2:172.21.15.12:6800/29934 1 ==== osd_op_reply(5 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv4 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7efc38018c60 con 0x55b52db12bd0 2023-10-05T19:42:16.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.816+0000 7efc4759be40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:42:16.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.816+0000 7efc4759be40 20 searching for the correct realm 2023-10-05T19:42:16.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.816+0000 7efc4759be40 1 --2- 172.21.15.12:0/1591481545 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55b52db17370 0x55b52db19780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.816+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db19e30 con 0x55b52db17370 2023-10-05T19:42:16.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.820+0000 7efc448cf700 1 --2- 172.21.15.12:0/1591481545 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55b52db17370 0x55b52db19780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.820+0000 7efc448cf700 1 --2- 172.21.15.12:0/1591481545 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55b52db17370 0x55b52db19780 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.820+0000 7efc448cf700 1 -- 172.21.15.12:0/1591481545 <== osd.3 v2:172.21.15.12:6816/29951 1 ==== osd_op_reply(6 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7efc30006070 con 0x55b52db17370 2023-10-05T19:42:16.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.820+0000 7efc448cf700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db19e30 con 0x55b52db12bd0 2023-10-05T19:42:16.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.820+0000 7efc450d0700 1 -- 172.21.15.12:0/1591481545 <== osd.2 v2:172.21.15.12:6800/29934 2 ==== osd_op_reply(7 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc38018c60 con 0x55b52db12bd0 2023-10-05T19:42:16.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.820+0000 7efc450d0700 1 --2- 172.21.15.12:0/1591481545 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7efc3801a740 0x7efc38039880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.820+0000 7efc450d0700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db19e30 con 0x7efc3801a740 2023-10-05T19:42:16.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.820+0000 7efc458d1700 1 --2- 172.21.15.12:0/1591481545 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7efc3801a740 0x7efc38039880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.824+0000 7efc458d1700 1 --2- 172.21.15.12:0/1591481545 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7efc3801a740 0x7efc38039880 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.824+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.0 v2:172.21.15.8:6800/27563 1 ==== osd_op_reply(8 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc40014040 con 0x7efc3801a740 2023-10-05T19:42:16.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.824+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db19e30 con 0x55b52dadd6e0 2023-10-05T19:42:16.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.824+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.1 v2:172.21.15.12:6808/29933 5 ==== osd_op_reply(9 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc40014040 con 0x55b52dadd6e0 2023-10-05T19:42:16.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.824+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db19e30 con 0x55b52dadd6e0 2023-10-05T19:42:16.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.824+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.1 v2:172.21.15.12:6808/29933 6 ==== osd_op_reply(10 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc40014040 con 0x55b52dadd6e0 2023-10-05T19:42:16.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.824+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db19e30 con 0x55b52dadd6e0 2023-10-05T19:42:16.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.824+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.1 v2:172.21.15.12:6808/29933 7 ==== osd_op_reply(11 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc40014040 con 0x55b52dadd6e0 2023-10-05T19:42:16.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.824+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db19e30 con 0x7efc3801a740 2023-10-05T19:42:16.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.824+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.0 v2:172.21.15.8:6800/27563 2 ==== osd_op_reply(12 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc40014040 con 0x7efc3801a740 2023-10-05T19:42:16.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.828+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db19e30 con 0x55b52db12bd0 2023-10-05T19:42:16.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.828+0000 7efc450d0700 1 -- 172.21.15.12:0/1591481545 <== osd.2 v2:172.21.15.12:6800/29934 3 ==== osd_op_reply(13 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc38018c60 con 0x55b52db12bd0 2023-10-05T19:42:16.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.828+0000 7efc450d0700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db19e30 con 0x55b52db12bd0 2023-10-05T19:42:16.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.828+0000 7efc450d0700 1 -- 172.21.15.12:0/1591481545 <== osd.2 v2:172.21.15.12:6800/29934 4 ==== osd_op_reply(14 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc38018c60 con 0x55b52db12bd0 2023-10-05T19:42:16.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.828+0000 7efc450d0700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db19e30 con 0x55b52dadd6e0 2023-10-05T19:42:16.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.828+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.1 v2:172.21.15.12:6808/29933 8 ==== osd_op_reply(15 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc40014040 con 0x55b52dadd6e0 2023-10-05T19:42:16.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.828+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db19e30 con 0x7efc3801a740 2023-10-05T19:42:16.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.828+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.0 v2:172.21.15.8:6800/27563 3 ==== osd_op_reply(16 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc40014040 con 0x7efc3801a740 2023-10-05T19:42:16.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.828+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db19e30 con 0x55b52dadd6e0 2023-10-05T19:42:16.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.832+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.1 v2:172.21.15.12:6808/29933 9 ==== osd_op_reply(17 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc40014040 con 0x55b52dadd6e0 2023-10-05T19:42:16.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.832+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db19e30 con 0x55b52db12bd0 2023-10-05T19:42:16.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.832+0000 7efc450d0700 1 -- 172.21.15.12:0/1591481545 <== osd.2 v2:172.21.15.12:6800/29934 5 ==== osd_op_reply(18 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc38018c60 con 0x55b52db12bd0 2023-10-05T19:42:16.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.832+0000 7efc450d0700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db19e30 con 0x55b52db17370 2023-10-05T19:42:16.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.832+0000 7efc448cf700 1 -- 172.21.15.12:0/1591481545 <== osd.3 v2:172.21.15.12:6816/29951 2 ==== osd_op_reply(19 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc30006070 con 0x55b52db17370 2023-10-05T19:42:16.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.832+0000 7efc448cf700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db19e30 con 0x55b52db17370 2023-10-05T19:42:16.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.832+0000 7efc448cf700 1 -- 172.21.15.12:0/1591481545 <== osd.3 v2:172.21.15.12:6816/29951 3 ==== osd_op_reply(20 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc30006070 con 0x55b52db17370 2023-10-05T19:42:16.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.832+0000 7efc448cf700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db19e30 con 0x7efc3801a740 2023-10-05T19:42:16.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.832+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.0 v2:172.21.15.8:6800/27563 4 ==== osd_op_reply(21 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc40014040 con 0x7efc3801a740 2023-10-05T19:42:16.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.832+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db19e30 con 0x55b52db17370 2023-10-05T19:42:16.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.836+0000 7efc448cf700 1 -- 172.21.15.12:0/1591481545 <== osd.3 v2:172.21.15.12:6816/29951 4 ==== osd_op_reply(22 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc30006070 con 0x55b52db17370 2023-10-05T19:42:16.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.836+0000 7efc448cf700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db19e30 con 0x55b52db17370 2023-10-05T19:42:16.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.836+0000 7efc448cf700 1 -- 172.21.15.12:0/1591481545 <== osd.3 v2:172.21.15.12:6816/29951 5 ==== osd_op_reply(23 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc30006070 con 0x55b52db17370 2023-10-05T19:42:16.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.836+0000 7efc448cf700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db19e30 con 0x55b52dadd6e0 2023-10-05T19:42:16.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.836+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.1 v2:172.21.15.12:6808/29933 10 ==== osd_op_reply(24 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc40014040 con 0x55b52dadd6e0 2023-10-05T19:42:16.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.836+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db19e30 con 0x55b52dadd6e0 2023-10-05T19:42:16.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.836+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.1 v2:172.21.15.12:6808/29933 11 ==== osd_op_reply(25 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc40014040 con 0x55b52dadd6e0 2023-10-05T19:42:16.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.836+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db19e30 con 0x7efc3801a740 2023-10-05T19:42:16.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.836+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.0 v2:172.21.15.8:6800/27563 5 ==== osd_op_reply(26 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc40014040 con 0x7efc3801a740 2023-10-05T19:42:16.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.836+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db19e30 con 0x55b52db17370 2023-10-05T19:42:16.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.840+0000 7efc448cf700 1 -- 172.21.15.12:0/1591481545 <== osd.3 v2:172.21.15.12:6816/29951 6 ==== osd_op_reply(27 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc30006070 con 0x55b52db17370 2023-10-05T19:42:16.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.840+0000 7efc448cf700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db19e30 con 0x55b52db12bd0 2023-10-05T19:42:16.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.840+0000 7efc450d0700 1 -- 172.21.15.12:0/1591481545 <== osd.2 v2:172.21.15.12:6800/29934 6 ==== osd_op_reply(28 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc38018c60 con 0x55b52db12bd0 2023-10-05T19:42:16.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.840+0000 7efc450d0700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db19e30 con 0x55b52dadd6e0 2023-10-05T19:42:16.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.840+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.1 v2:172.21.15.12:6808/29933 12 ==== osd_op_reply(29 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc40014040 con 0x55b52dadd6e0 2023-10-05T19:42:16.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.840+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db19e30 con 0x7efc3801a740 2023-10-05T19:42:16.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.840+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.0 v2:172.21.15.8:6800/27563 6 ==== osd_op_reply(30 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc40014040 con 0x7efc3801a740 2023-10-05T19:42:16.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.840+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db19e30 con 0x55b52db17370 2023-10-05T19:42:16.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.844+0000 7efc448cf700 1 -- 172.21.15.12:0/1591481545 <== osd.3 v2:172.21.15.12:6816/29951 7 ==== osd_op_reply(31 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc30006070 con 0x55b52db17370 2023-10-05T19:42:16.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.844+0000 7efc448cf700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db19e30 con 0x7efc3801a740 2023-10-05T19:42:16.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.844+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.0 v2:172.21.15.8:6800/27563 7 ==== osd_op_reply(32 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc40014040 con 0x7efc3801a740 2023-10-05T19:42:16.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.844+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db19e30 con 0x55b52db12bd0 2023-10-05T19:42:16.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.844+0000 7efc450d0700 1 -- 172.21.15.12:0/1591481545 <== osd.2 v2:172.21.15.12:6800/29934 7 ==== osd_op_reply(33 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc38018c60 con 0x55b52db12bd0 2023-10-05T19:42:16.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.844+0000 7efc450d0700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db19e30 con 0x55b52db17370 2023-10-05T19:42:16.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.844+0000 7efc448cf700 1 -- 172.21.15.12:0/1591481545 <== osd.3 v2:172.21.15.12:6816/29951 8 ==== osd_op_reply(34 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc30006070 con 0x55b52db17370 2023-10-05T19:42:16.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.844+0000 7efc448cf700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db19e30 con 0x7efc3801a740 2023-10-05T19:42:16.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.844+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.0 v2:172.21.15.8:6800/27563 8 ==== osd_op_reply(35 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc40014040 con 0x7efc3801a740 2023-10-05T19:42:16.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.844+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db19e30 con 0x55b52dadd6e0 2023-10-05T19:42:16.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.848+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.1 v2:172.21.15.12:6808/29933 13 ==== osd_op_reply(36 [pgnls start_epoch 30 out=115b] v30'3 uv3 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7efc40014040 con 0x55b52dadd6e0 2023-10-05T19:42:16.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.848+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db19e30 con 0x55b52db12bd0 2023-10-05T19:42:16.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.848+0000 7efc450d0700 1 -- 172.21.15.12:0/1591481545 <== osd.2 v2:172.21.15.12:6800/29934 8 ==== osd_op_reply(37 [pgnls start_epoch 30 out=107b] v30'4 uv4 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7efc38018c60 con 0x55b52db12bd0 2023-10-05T19:42:16.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.848+0000 7efc4759be40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:42:16.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.848+0000 7efc4759be40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:42:16.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.848+0000 7efc4759be40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:42:16.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.848+0000 7efc4759be40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:42:16.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.848+0000 7efc4759be40 20 rados->read ofs=0 len=0 2023-10-05T19:42:16.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.848+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db160f0 con 0x55b52dadd6e0 2023-10-05T19:42:16.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.848+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.1 v2:172.21.15.12:6808/29933 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7efc40014040 con 0x55b52dadd6e0 2023-10-05T19:42:16.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.848+0000 7efc4759be40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:16.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.848+0000 7efc4759be40 20 rados->read ofs=0 len=0 2023-10-05T19:42:16.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.848+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db15d20 con 0x55b52dadd6e0 2023-10-05T19:42:16.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.848+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.1 v2:172.21.15.12:6808/29933 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7efc40014040 con 0x55b52dadd6e0 2023-10-05T19:42:16.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.848+0000 7efc4759be40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:42:16.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.848+0000 7efc4759be40 20 rados->read ofs=0 len=0 2023-10-05T19:42:16.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.848+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:40 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1a2a0 con 0x55b52db17370 2023-10-05T19:42:16.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.852+0000 7efc448cf700 1 -- 172.21.15.12:0/1591481545 <== osd.3 v2:172.21.15.12:6816/29951 9 ==== osd_op_reply(40 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7efc30006070 con 0x55b52db17370 2023-10-05T19:42:16.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.852+0000 7efc4759be40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:42:16.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.852+0000 7efc4759be40 20 rados->read ofs=0 len=0 2023-10-05T19:42:16.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.852+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:41 5.13 5:cbbf6c60:::converted:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1a650 con 0x55b52db17370 2023-10-05T19:42:16.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.852+0000 7efc448cf700 1 -- 172.21.15.12:0/1591481545 <== osd.3 v2:172.21.15.12:6816/29951 10 ==== osd_op_reply(41 converted [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 153+0+0 (crc 0 0 0) 0x7efc30006070 con 0x55b52db17370 2023-10-05T19:42:16.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.852+0000 7efc4759be40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:16.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.852+0000 7efc4759be40 20 rados->read ofs=0 len=0 2023-10-05T19:42:16.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.852+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:42 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1aa00 con 0x55b52dadd6e0 2023-10-05T19:42:16.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.852+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.1 v2:172.21.15.12:6808/29933 16 ==== osd_op_reply(42 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7efc40014040 con 0x55b52dadd6e0 2023-10-05T19:42:16.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.852+0000 7efc4759be40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:16.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.852+0000 7efc4759be40 20 rados->read ofs=0 len=0 2023-10-05T19:42:16.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.852+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:43 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db16610 con 0x55b52dadd6e0 2023-10-05T19:42:16.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.852+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.1 v2:172.21.15.12:6808/29933 17 ==== osd_op_reply(43 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7efc40014040 con 0x55b52dadd6e0 2023-10-05T19:42:16.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.852+0000 7efc4759be40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:42:16.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.852+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:44 5.0 5:00000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db16610 con 0x55b52db17370 2023-10-05T19:42:16.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.856+0000 7efc448cf700 1 -- 172.21.15.12:0/1591481545 <== osd.3 v2:172.21.15.12:6816/29951 11 ==== osd_op_reply(44 [pgnls start_epoch 30 out=112b] v21'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7efc30006070 con 0x55b52db17370 2023-10-05T19:42:16.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.856+0000 7efc448cf700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:45 5.10 5:08000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db16610 con 0x55b52db12bd0 2023-10-05T19:42:16.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.856+0000 7efc450d0700 1 -- 172.21.15.12:0/1591481545 <== osd.2 v2:172.21.15.12:6800/29934 9 ==== osd_op_reply(45 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc3804f040 con 0x55b52db12bd0 2023-10-05T19:42:16.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.856+0000 7efc450d0700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:46 5.8 5:10000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db16610 con 0x7efc3801a740 2023-10-05T19:42:16.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.856+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.0 v2:172.21.15.8:6800/27563 9 ==== osd_op_reply(46 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc40014040 con 0x7efc3801a740 2023-10-05T19:42:16.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.856+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:47 5.18 5:18000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1aa00 con 0x55b52dadd6e0 2023-10-05T19:42:16.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.856+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.1 v2:172.21.15.12:6808/29933 18 ==== osd_op_reply(47 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc40014040 con 0x55b52dadd6e0 2023-10-05T19:42:16.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.856+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:48 5.4 5:20000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1aa00 con 0x55b52dadd6e0 2023-10-05T19:42:16.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.860+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.1 v2:172.21.15.12:6808/29933 19 ==== osd_op_reply(48 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc40014040 con 0x55b52dadd6e0 2023-10-05T19:42:16.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.860+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:49 5.14 5:28000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1aa00 con 0x55b52dadd6e0 2023-10-05T19:42:16.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.860+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.1 v2:172.21.15.12:6808/29933 20 ==== osd_op_reply(49 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc40014040 con 0x55b52dadd6e0 2023-10-05T19:42:16.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.860+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:50 5.c 5:30000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1aa00 con 0x7efc3801a740 2023-10-05T19:42:16.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.860+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.0 v2:172.21.15.8:6800/27563 10 ==== osd_op_reply(50 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc40014040 con 0x7efc3801a740 2023-10-05T19:42:16.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.860+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:51 5.1c 5:38000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1aa00 con 0x55b52db12bd0 2023-10-05T19:42:16.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.860+0000 7efc450d0700 1 -- 172.21.15.12:0/1591481545 <== osd.2 v2:172.21.15.12:6800/29934 10 ==== osd_op_reply(51 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc3804f040 con 0x55b52db12bd0 2023-10-05T19:42:16.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.860+0000 7efc450d0700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:52 5.2 5:40000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1aa00 con 0x55b52db12bd0 2023-10-05T19:42:16.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.864+0000 7efc450d0700 1 -- 172.21.15.12:0/1591481545 <== osd.2 v2:172.21.15.12:6800/29934 11 ==== osd_op_reply(52 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc3804f040 con 0x55b52db12bd0 2023-10-05T19:42:16.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.864+0000 7efc450d0700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:53 5.12 5:48000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1aa00 con 0x55b52dadd6e0 2023-10-05T19:42:16.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.864+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.1 v2:172.21.15.12:6808/29933 21 ==== osd_op_reply(53 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc40014040 con 0x55b52dadd6e0 2023-10-05T19:42:16.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.864+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:54 5.a 5:50000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1aa00 con 0x7efc3801a740 2023-10-05T19:42:16.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.864+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.0 v2:172.21.15.8:6800/27563 11 ==== osd_op_reply(54 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc40014040 con 0x7efc3801a740 2023-10-05T19:42:16.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.864+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:55 5.1a 5:58000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1aa00 con 0x55b52dadd6e0 2023-10-05T19:42:16.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.864+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.1 v2:172.21.15.12:6808/29933 22 ==== osd_op_reply(55 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc40014040 con 0x55b52dadd6e0 2023-10-05T19:42:16.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.864+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:56 5.6 5:60000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1aa00 con 0x55b52db12bd0 2023-10-05T19:42:16.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.868+0000 7efc450d0700 1 -- 172.21.15.12:0/1591481545 <== osd.2 v2:172.21.15.12:6800/29934 12 ==== osd_op_reply(56 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc3804f040 con 0x55b52db12bd0 2023-10-05T19:42:16.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.868+0000 7efc450d0700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:57 5.16 5:68000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1aa00 con 0x55b52db17370 2023-10-05T19:42:16.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.868+0000 7efc448cf700 1 -- 172.21.15.12:0/1591481545 <== osd.3 v2:172.21.15.12:6816/29951 12 ==== osd_op_reply(57 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc30006070 con 0x55b52db17370 2023-10-05T19:42:16.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.868+0000 7efc448cf700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:58 5.e 5:70000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1aa00 con 0x55b52db17370 2023-10-05T19:42:16.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.868+0000 7efc448cf700 1 -- 172.21.15.12:0/1591481545 <== osd.3 v2:172.21.15.12:6816/29951 13 ==== osd_op_reply(58 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc30006070 con 0x55b52db17370 2023-10-05T19:42:16.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.868+0000 7efc448cf700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:59 5.1e 5:78000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1aa00 con 0x7efc3801a740 2023-10-05T19:42:16.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.868+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.0 v2:172.21.15.8:6800/27563 12 ==== osd_op_reply(59 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc40014040 con 0x7efc3801a740 2023-10-05T19:42:16.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.868+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:60 5.1 5:80000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1aa00 con 0x55b52db17370 2023-10-05T19:42:16.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.868+0000 7efc448cf700 1 -- 172.21.15.12:0/1591481545 <== osd.3 v2:172.21.15.12:6816/29951 14 ==== osd_op_reply(60 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc30006070 con 0x55b52db17370 2023-10-05T19:42:16.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.868+0000 7efc448cf700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:61 5.11 5:88000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1aa00 con 0x55b52db17370 2023-10-05T19:42:16.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.872+0000 7efc448cf700 1 -- 172.21.15.12:0/1591481545 <== osd.3 v2:172.21.15.12:6816/29951 15 ==== osd_op_reply(61 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc30006070 con 0x55b52db17370 2023-10-05T19:42:16.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.872+0000 7efc448cf700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:62 5.9 5:90000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1aa00 con 0x55b52dadd6e0 2023-10-05T19:42:16.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.872+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.1 v2:172.21.15.12:6808/29933 23 ==== osd_op_reply(62 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc40014040 con 0x55b52dadd6e0 2023-10-05T19:42:16.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.872+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:63 5.19 5:98000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1aa00 con 0x55b52dadd6e0 2023-10-05T19:42:16.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.872+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.1 v2:172.21.15.12:6808/29933 24 ==== osd_op_reply(63 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc40014040 con 0x55b52dadd6e0 2023-10-05T19:42:16.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.872+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:64 5.5 5:a0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1aa00 con 0x7efc3801a740 2023-10-05T19:42:16.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.872+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.0 v2:172.21.15.8:6800/27563 13 ==== osd_op_reply(64 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc40014040 con 0x7efc3801a740 2023-10-05T19:42:16.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.872+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:65 5.15 5:a8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1aa00 con 0x55b52db17370 2023-10-05T19:42:16.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.872+0000 7efc448cf700 1 -- 172.21.15.12:0/1591481545 <== osd.3 v2:172.21.15.12:6816/29951 16 ==== osd_op_reply(65 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc30006070 con 0x55b52db17370 2023-10-05T19:42:16.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.876+0000 7efc448cf700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:66 5.d 5:b0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1aa00 con 0x55b52db12bd0 2023-10-05T19:42:16.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.876+0000 7efc450d0700 1 -- 172.21.15.12:0/1591481545 <== osd.2 v2:172.21.15.12:6800/29934 13 ==== osd_op_reply(66 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc3804f040 con 0x55b52db12bd0 2023-10-05T19:42:16.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.876+0000 7efc450d0700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:67 5.1d 5:b8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1aa00 con 0x55b52dadd6e0 2023-10-05T19:42:16.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.876+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.1 v2:172.21.15.12:6808/29933 25 ==== osd_op_reply(67 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc40014040 con 0x55b52dadd6e0 2023-10-05T19:42:16.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.876+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:68 5.3 5:c0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1aa00 con 0x7efc3801a740 2023-10-05T19:42:16.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.876+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.0 v2:172.21.15.8:6800/27563 14 ==== osd_op_reply(68 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc40014040 con 0x7efc3801a740 2023-10-05T19:42:16.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.876+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:69 5.13 5:c8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1aa00 con 0x55b52db17370 2023-10-05T19:42:16.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.876+0000 7efc448cf700 1 -- 172.21.15.12:0/1591481545 <== osd.3 v2:172.21.15.12:6816/29951 17 ==== osd_op_reply(69 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc30006070 con 0x55b52db17370 2023-10-05T19:42:16.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.876+0000 7efc448cf700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:70 5.b 5:d0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1aa00 con 0x7efc3801a740 2023-10-05T19:42:16.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.880+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.0 v2:172.21.15.8:6800/27563 15 ==== osd_op_reply(70 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc40014040 con 0x7efc3801a740 2023-10-05T19:42:16.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.880+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:71 5.1b 5:d8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1aa00 con 0x55b52db12bd0 2023-10-05T19:42:16.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.880+0000 7efc450d0700 1 -- 172.21.15.12:0/1591481545 <== osd.2 v2:172.21.15.12:6800/29934 14 ==== osd_op_reply(71 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc3804f040 con 0x55b52db12bd0 2023-10-05T19:42:16.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.880+0000 7efc450d0700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:72 5.7 5:e0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1aa00 con 0x55b52db17370 2023-10-05T19:42:16.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.880+0000 7efc448cf700 1 -- 172.21.15.12:0/1591481545 <== osd.3 v2:172.21.15.12:6816/29951 18 ==== osd_op_reply(72 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc30006070 con 0x55b52db17370 2023-10-05T19:42:16.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.880+0000 7efc448cf700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:73 5.17 5:e8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1aa00 con 0x7efc3801a740 2023-10-05T19:42:16.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.880+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.0 v2:172.21.15.8:6800/27563 16 ==== osd_op_reply(73 [pgnls start_epoch 30 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efc40014040 con 0x7efc3801a740 2023-10-05T19:42:16.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.880+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:74 5.f 5:f0000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1aa00 con 0x55b52dadd6e0 2023-10-05T19:42:16.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.880+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.1 v2:172.21.15.12:6808/29933 26 ==== osd_op_reply(74 [pgnls start_epoch 30 out=115b] v30'3 uv3 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7efc40014040 con 0x55b52dadd6e0 2023-10-05T19:42:16.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.880+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:75 5.1f 5:f8000000::::head [pgnls start_epoch 30 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1aa00 con 0x55b52db12bd0 2023-10-05T19:42:16.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.884+0000 7efc450d0700 1 -- 172.21.15.12:0/1591481545 <== osd.2 v2:172.21.15.12:6800/29934 15 ==== osd_op_reply(75 [pgnls start_epoch 30 out=107b] v30'4 uv4 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7efc3804f040 con 0x55b52db12bd0 2023-10-05T19:42:16.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.884+0000 7efc4759be40 20 RGWRados::pool_iterate: got zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db 2023-10-05T19:42:16.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.884+0000 7efc4759be40 20 RGWRados::pool_iterate: got zone_names.default 2023-10-05T19:42:16.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.884+0000 7efc4759be40 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-10-05T19:42:16.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.884+0000 7efc4759be40 20 RGWRados::pool_iterate: got zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 2023-10-05T19:42:16.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.884+0000 7efc4759be40 20 rados->read ofs=0 len=0 2023-10-05T19:42:16.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.884+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:76 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db16610 con 0x55b52dadd6e0 2023-10-05T19:42:16.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.884+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.1 v2:172.21.15.12:6808/29933 27 ==== osd_op_reply(76 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7efc40014040 con 0x55b52dadd6e0 2023-10-05T19:42:16.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.884+0000 7efc4759be40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:42:16.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.884+0000 7efc4759be40 20 rados->read ofs=0 len=0 2023-10-05T19:42:16.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.884+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:77 5.1f 5:fee23f0b:::zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1b350 con 0x55b52db12bd0 2023-10-05T19:42:16.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.884+0000 7efc450d0700 1 -- 172.21.15.12:0/1591481545 <== osd.2 v2:172.21.15.12:6800/29934 16 ==== osd_op_reply(77 zone_info.93ac49f4-ff49-4f99-811b-6453b28decd5 [read 0~874 out=874b] v0'0 uv4 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7efc3804f040 con 0x55b52db12bd0 2023-10-05T19:42:16.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.884+0000 7efc4759be40 20 rados_obj.operate() r=0 bl.length=874 2023-10-05T19:42:16.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.884+0000 7efc4759be40 20 rados->read ofs=0 len=0 2023-10-05T19:42:16.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.884+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:78 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1bc40 con 0x55b52dadd6e0 2023-10-05T19:42:16.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.884+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.1 v2:172.21.15.12:6808/29933 28 ==== osd_op_reply(78 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7efc40014040 con 0x55b52dadd6e0 2023-10-05T19:42:16.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.884+0000 7efc4759be40 20 rados_obj.operate() r=0 bl.length=46 2023-10-05T19:42:16.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.884+0000 7efc4759be40 20 rados->read ofs=0 len=0 2023-10-05T19:42:16.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.884+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:79 5.0 5:06a351a7:::zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1bc40 con 0x55b52db17370 2023-10-05T19:42:16.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.888+0000 7efc448cf700 1 -- 172.21.15.12:0/1591481545 <== osd.3 v2:172.21.15.12:6816/29951 19 ==== osd_op_reply(79 zonegroup_info.1602ab00-ae0f-4921-ad63-b9cfe7bc77db [read 0~362 out=362b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+362 (crc 0 0 0) 0x7efc30006070 con 0x55b52db17370 2023-10-05T19:42:16.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.888+0000 7efc4759be40 20 rados_obj.operate() r=0 bl.length=362 2023-10-05T19:42:16.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.888+0000 7efc4759be40 20 rados->read ofs=0 len=0 2023-10-05T19:42:16.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.888+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:80 5.3 5:c0ad8cbd:::region_map:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1c3b0 con 0x7efc3801a740 2023-10-05T19:42:16.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.888+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.0 v2:172.21.15.8:6800/27563 17 ==== osd_op_reply(80 region_map [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x7efc40014040 con 0x7efc3801a740 2023-10-05T19:42:16.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.888+0000 7efc4759be40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:16.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.888+0000 7efc4759be40 20 zone default found 2023-10-05T19:42:16.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.888+0000 7efc4759be40 4 Realm: () 2023-10-05T19:42:16.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.888+0000 7efc4759be40 4 ZoneGroup: default (1602ab00-ae0f-4921-ad63-b9cfe7bc77db) 2023-10-05T19:42:16.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.888+0000 7efc4759be40 4 Zone: default (93ac49f4-ff49-4f99-811b-6453b28decd5) 2023-10-05T19:42:16.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.888+0000 7efc4759be40 10 cannot find current period zonegroup using local zonegroup configuration 2023-10-05T19:42:16.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.888+0000 7efc4759be40 20 zonegroup default 2023-10-05T19:42:16.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.888+0000 7efc4759be40 20 rados->read ofs=0 len=0 2023-10-05T19:42:16.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.888+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:81 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1c3b0 con 0x7efc3801a740 2023-10-05T19:42:16.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.888+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.0 v2:172.21.15.8:6800/27563 18 ==== osd_op_reply(81 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7efc40014040 con 0x7efc3801a740 2023-10-05T19:42:16.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.888+0000 7efc4759be40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:16.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.888+0000 7efc4759be40 20 rados->read ofs=0 len=0 2023-10-05T19:42:16.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.888+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:82 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1dd30 con 0x55b52db17370 2023-10-05T19:42:16.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.888+0000 7efc448cf700 1 -- 172.21.15.12:0/1591481545 <== osd.3 v2:172.21.15.12:6816/29951 20 ==== osd_op_reply(82 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7efc30006070 con 0x55b52db17370 2023-10-05T19:42:16.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.888+0000 7efc4759be40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:16.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.888+0000 7efc4759be40 20 rados->read ofs=0 len=0 2023-10-05T19:42:16.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.888+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:83 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1e0e0 con 0x55b52db17370 2023-10-05T19:42:16.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.892+0000 7efc448cf700 1 -- 172.21.15.12:0/1591481545 <== osd.3 v2:172.21.15.12:6816/29951 21 ==== osd_op_reply(83 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7efc30006070 con 0x55b52db17370 2023-10-05T19:42:16.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.892+0000 7efc4759be40 20 rados_obj.operate() r=-2 bl.length=0 2023-10-05T19:42:16.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.892+0000 7efc4759be40 20 started sync module instance, tier type = 2023-10-05T19:42:16.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.892+0000 7efc4759be40 20 started zone id=93ac49f4-ff49-4f99-811b-6453b28decd5 (name=default) with tier type = 2023-10-05T19:42:16.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.892+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:84 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1d3f0 con 0x55b52db12bd0 2023-10-05T19:42:16.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.892+0000 7efc450d0700 1 -- 172.21.15.12:0/1591481545 <== osd.2 v2:172.21.15.12:6800/29934 17 ==== osd_op_reply(84 notify.0 [create] v30'270 uv270 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc3804f040 con 0x55b52db12bd0 2023-10-05T19:42:16.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.892+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:85 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94236644063056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1e490 con 0x55b52db12bd0 2023-10-05T19:42:16.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.892+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:86 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1ed70 con 0x55b52dadd6e0 2023-10-05T19:42:16.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.896+0000 7efc450d0700 1 -- 172.21.15.12:0/1591481545 <== osd.2 v2:172.21.15.12:6800/29934 18 ==== osd_op_reply(85 notify.0 [watch watch cookie 94236644063056] v30'271 uv270 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc3804f040 con 0x55b52db12bd0 2023-10-05T19:42:16.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.896+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.1 v2:172.21.15.12:6808/29933 29 ==== osd_op_reply(86 notify.1 [create] v30'270 uv270 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc40014040 con 0x55b52dadd6e0 2023-10-05T19:42:16.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.896+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:87 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94236644071408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1e980 con 0x55b52dadd6e0 2023-10-05T19:42:16.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.896+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:88 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1f660 con 0x55b52db17370 2023-10-05T19:42:16.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.900+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.1 v2:172.21.15.12:6808/29933 30 ==== osd_op_reply(87 notify.1 [watch watch cookie 94236644071408] v30'271 uv270 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc40014040 con 0x55b52dadd6e0 2023-10-05T19:42:16.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.900+0000 7efc448cf700 1 -- 172.21.15.12:0/1591481545 <== osd.3 v2:172.21.15.12:6816/29951 22 ==== osd_op_reply(88 notify.2 [create] v30'270 uv270 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc30006070 con 0x55b52db17370 2023-10-05T19:42:16.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.900+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:89 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94236644074640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1e8e0 con 0x55b52db17370 2023-10-05T19:42:16.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.900+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:90 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db20440 con 0x55b52db12bd0 2023-10-05T19:42:16.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.904+0000 7efc450d0700 1 -- 172.21.15.12:0/1591481545 <== osd.2 v2:172.21.15.12:6800/29934 19 ==== osd_op_reply(90 notify.3 [create] v30'270 uv270 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc3804f040 con 0x55b52db12bd0 2023-10-05T19:42:16.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.904+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:91 7.e 7:7759931f:::notify.3:head [watch watch cookie 94236644081488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db20950 con 0x55b52db12bd0 2023-10-05T19:42:16.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.904+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:92 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db21460 con 0x55b52dadd6e0 2023-10-05T19:42:16.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.904+0000 7efc448cf700 1 -- 172.21.15.12:0/1591481545 <== osd.3 v2:172.21.15.12:6816/29951 23 ==== osd_op_reply(89 notify.2 [watch watch cookie 94236644074640] v30'271 uv270 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc30006070 con 0x55b52db17370 2023-10-05T19:42:16.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.908+0000 7efc450d0700 1 -- 172.21.15.12:0/1591481545 <== osd.2 v2:172.21.15.12:6800/29934 20 ==== osd_op_reply(91 notify.3 [watch watch cookie 94236644081488] v30'271 uv270 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc3804f040 con 0x55b52db12bd0 2023-10-05T19:42:16.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.908+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.1 v2:172.21.15.12:6808/29933 31 ==== osd_op_reply(92 notify.4 [create] v30'270 uv270 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc40014040 con 0x55b52dadd6e0 2023-10-05T19:42:16.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.908+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:93 7.d 7:b4812045:::notify.4:head [watch watch cookie 94236644080224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db20f30 con 0x55b52dadd6e0 2023-10-05T19:42:16.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.908+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db21940 con 0x7efc3801a740 2023-10-05T19:42:16.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.912+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.1 v2:172.21.15.12:6808/29933 32 ==== osd_op_reply(93 notify.4 [watch watch cookie 94236644080224] v30'271 uv270 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc40014040 con 0x55b52dadd6e0 2023-10-05T19:42:16.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.912+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.0 v2:172.21.15.8:6800/27563 19 ==== osd_op_reply(94 notify.5 [create] v30'270 uv270 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc40014040 con 0x7efc3801a740 2023-10-05T19:42:16.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.912+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [watch watch cookie 94236644082656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db21940 con 0x7efc3801a740 2023-10-05T19:42:16.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.912+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:96 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db221e0 con 0x7efc3801a740 2023-10-05T19:42:16.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.912+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.0 v2:172.21.15.8:6800/27563 20 ==== osd_op_reply(95 notify.5 [watch watch cookie 94236644082656] v30'271 uv270 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc40014040 con 0x7efc3801a740 2023-10-05T19:42:16.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.912+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.0 v2:172.21.15.8:6800/27563 21 ==== osd_op_reply(96 notify.6 [create] v30'270 uv270 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc40014040 con 0x7efc3801a740 2023-10-05T19:42:16.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.912+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:97 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94236644087424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db21fc0 con 0x7efc3801a740 2023-10-05T19:42:16.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.912+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:98 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db22b50 con 0x7efc3801a740 2023-10-05T19:42:16.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.916+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.0 v2:172.21.15.8:6800/27563 22 ==== osd_op_reply(97 notify.6 [watch watch cookie 94236644087424] v30'271 uv270 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc40014040 con 0x7efc3801a740 2023-10-05T19:42:16.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.916+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.0 v2:172.21.15.8:6800/27563 23 ==== osd_op_reply(98 notify.7 [create] v30'270 uv270 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc40014040 con 0x7efc3801a740 2023-10-05T19:42:16.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.916+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:99 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94236644088528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db22d40 con 0x7efc3801a740 2023-10-05T19:42:16.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.916+0000 7efc4759be40 20 add_watcher() i=0 2023-10-05T19:42:16.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.916+0000 7efc4759be40 20 add_watcher() i=1 2023-10-05T19:42:16.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.916+0000 7efc4759be40 20 add_watcher() i=2 2023-10-05T19:42:16.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.916+0000 7efc4759be40 20 add_watcher() i=3 2023-10-05T19:42:16.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.916+0000 7efc4759be40 20 add_watcher() i=4 2023-10-05T19:42:16.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.916+0000 7efc4759be40 20 add_watcher() i=5 2023-10-05T19:42:16.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.916+0000 7efc4759be40 20 add_watcher() i=6 2023-10-05T19:42:16.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.920+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.0 v2:172.21.15.8:6800/27563 24 ==== osd_op_reply(99 notify.7 [watch watch cookie 94236644088528] v30'271 uv270 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc40014040 con 0x7efc3801a740 2023-10-05T19:42:16.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.920+0000 7efc4759be40 20 add_watcher() i=7 2023-10-05T19:42:16.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.920+0000 7efc4759be40 2 all 8 watchers are set, enabling cache 2023-10-05T19:42:16.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.920+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db25da0 con 0x55b52dadd6e0 2023-10-05T19:42:16.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.920+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.1 v2:172.21.15.12:6808/29933 33 ==== osd_op_reply(100 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7efc40014040 con 0x55b52dadd6e0 2023-10-05T19:42:16.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.920+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:101 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94236644091712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db23080 con 0x55b52dadd6e0 2023-10-05T19:42:16.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.920+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.1 v2:172.21.15.12:6808/29933 34 ==== osd_op_reply(101 data_loggenerations_metadata [watch watch cookie 94236644091712] v30'260 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7efc40014040 con 0x55b52dadd6e0 2023-10-05T19:42:16.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.924+0000 7efc167e4700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:42:16.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.924+0000 7efc4759be40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-10-05T19:42:16.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.924+0000 7efc4759be40 20 check_secure_mon_conn(): mode 1 is insecure 2023-10-05T19:42:16.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.924+0000 7efc4759be40 1 Processor -- start 2023-10-05T19:42:16.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.924+0000 7efc4759be40 1 -- start start 2023-10-05T19:42:16.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.924+0000 7efc4759be40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b52db30620 0x55b52db309f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.924+0000 7efc4759be40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b52db30f30 con 0x55b52db30620 2023-10-05T19:42:16.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.924+0000 7efc448cf700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b52db30620 0x55b52db309f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.924+0000 7efc448cf700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b52db30620 0x55b52db309f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:54562/0 (socket says 172.21.15.12:54562) 2023-10-05T19:42:16.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.924+0000 7efc448cf700 1 -- 172.21.15.12:0/666489391 learned_addr learned my addr 172.21.15.12:0/666489391 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:42:16.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.924+0000 7efc448cf700 1 -- 172.21.15.12:0/666489391 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b52db31070 con 0x55b52db30620 2023-10-05T19:42:16.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.924+0000 7efc448cf700 1 --2- 172.21.15.12:0/666489391 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b52db30620 0x55b52db309f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x55b52db0e130 tx=0x7efc3001e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0ea272415ba12c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.924+0000 7efc15fe3700 1 -- 172.21.15.12:0/666489391 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc30030040 con 0x55b52db30620 2023-10-05T19:42:16.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.924+0000 7efc15fe3700 1 -- 172.21.15.12:0/666489391 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc3000edb0 con 0x55b52db30620 2023-10-05T19:42:16.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.924+0000 7efc15fe3700 1 -- 172.21.15.12:0/666489391 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc30021040 con 0x55b52db30620 2023-10-05T19:42:16.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.928+0000 7efc4759be40 1 -- 172.21.15.12:0/666489391 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b52db30620 msgr2=0x55b52db309f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.928+0000 7efc4759be40 1 --2- 172.21.15.12:0/666489391 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b52db30620 0x55b52db309f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x55b52db0e130 tx=0x7efc3001e040 comp rx=0 tx=0).stop 2023-10-05T19:42:16.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.928+0000 7efc4759be40 1 -- 172.21.15.12:0/666489391 shutdown_connections 2023-10-05T19:42:16.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.928+0000 7efc4759be40 1 --2- 172.21.15.12:0/666489391 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b52db30620 0x55b52db309f0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.928+0000 7efc4759be40 1 -- 172.21.15.12:0/666489391 >> 172.21.15.12:0/666489391 conn(0x55b52db2b9e0 msgr2=0x55b52db29b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:16.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.928+0000 7efc4759be40 1 -- 172.21.15.12:0/666489391 shutdown_connections 2023-10-05T19:42:16.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.928+0000 7efc4759be40 1 -- 172.21.15.12:0/666489391 wait complete. 2023-10-05T19:42:16.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.928+0000 7efc4759be40 1 Processor -- start 2023-10-05T19:42:16.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.928+0000 7efc4759be40 1 -- start start 2023-10-05T19:42:16.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.928+0000 7efc4759be40 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b52db30620 0x55b52db361e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.928+0000 7efc4759be40 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b52db371e0 con 0x55b52db30620 2023-10-05T19:42:16.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.928+0000 7efc448cf700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b52db30620 0x55b52db361e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.928+0000 7efc448cf700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b52db30620 0x55b52db361e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.12:54578/0 (socket says 172.21.15.12:54578) 2023-10-05T19:42:16.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.928+0000 7efc448cf700 1 -- 172.21.15.12:0/1057623985 learned_addr learned my addr 172.21.15.12:0/1057623985 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-10-05T19:42:16.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.928+0000 7efc448cf700 1 -- 172.21.15.12:0/1057623985 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc30033040 con 0x55b52db30620 2023-10-05T19:42:16.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.928+0000 7efc448cf700 1 --2- 172.21.15.12:0/1057623985 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b52db30620 0x55b52db361e0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7efc30000fb0 tx=0x7efc3003c030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.928+0000 7efb87fff700 1 -- 172.21.15.12:0/1057623985 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc30021030 con 0x55b52db30620 2023-10-05T19:42:16.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.928+0000 7efb87fff700 1 -- 172.21.15.12:0/1057623985 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc3002b860 con 0x55b52db30620 2023-10-05T19:42:16.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.928+0000 7efb87fff700 1 -- 172.21.15.12:0/1057623985 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc30030040 con 0x55b52db30620 2023-10-05T19:42:16.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.928+0000 7efc4759be40 1 -- 172.21.15.12:0/1057623985 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b52db367e0 con 0x55b52db30620 2023-10-05T19:42:16.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.928+0000 7efc4759be40 1 -- 172.21.15.12:0/1057623985 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b52db36bc0 con 0x55b52db30620 2023-10-05T19:42:16.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.932+0000 7efb87fff700 1 -- 172.21.15.12:0/1057623985 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7efc3002b9f0 con 0x55b52db30620 2023-10-05T19:42:16.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.932+0000 7efb87fff700 1 --2- 172.21.15.12:0/1057623985 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7efb8c030ca0 0x7efb8c033150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:42:16.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.932+0000 7efb87fff700 1 -- 172.21.15.12:0/1057623985 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7efc30058bd0 con 0x55b52db30620 2023-10-05T19:42:16.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.932+0000 7efc458d1700 1 --2- 172.21.15.12:0/1057623985 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7efb8c030ca0 0x7efb8c033150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:42:16.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.932+0000 7efc4759be40 5 note: GC not initialized 2023-10-05T19:42:16.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.932+0000 7efc458d1700 1 --2- 172.21.15.12:0/1057623985 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7efb8c030ca0 0x7efb8c033150 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55b52db25e70 tx=0x7efc400325e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:42:16.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.936+0000 7efb857fa700 20 reqs_thread_entry: start 2023-10-05T19:42:16.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.936+0000 7efb7effd700 20 reqs_thread_entry: start 2023-10-05T19:42:16.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.936+0000 7efc4759be40 20 init_complete bucket index max shards: 11 2023-10-05T19:42:16.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.936+0000 7efc4759be40 10 rgw notify: Started notification manager with: 1 workers 2023-10-05T19:42:16.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.936+0000 7efb7d7fa700 20 rgw notify: INFO: next queues processing will happen at: Thu Oct 5 19:42:47 2023 2023-10-05T19:42:16.945 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-10-05T19:42:16.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.936+0000 7efb7cff9700 20 reqs_thread_entry: start 2023-10-05T19:42:16.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.936+0000 7efc4759be40 20 get_system_obj_state: rctx=0x7ffe8e3f7528 obj=default.rgw.meta:users.uid:ada state=0x55b52d9b60b0 s->prefetch_data=0 2023-10-05T19:42:16.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.936+0000 7efc4759be40 10 cache get: name=default.rgw.meta+users.uid+ada : miss 2023-10-05T19:42:16.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.936+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:102 8.7 8:e1d59a8c:users.uid::ada:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db9eb70 con 0x7efc3801a740 2023-10-05T19:42:16.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.936+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.0 v2:172.21.15.8:6800/27563 25 ==== osd_op_reply(102 ada [call,getxattrs,stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x7efc40014040 con 0x7efc3801a740 2023-10-05T19:42:16.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.936+0000 7efc4759be40 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x0 2023-10-05T19:42:16.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.936+0000 7efc4759be40 10 adding default.rgw.meta+users.uid+ada to cache LRU end 2023-10-05T19:42:16.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.936+0000 7efc4759be40 20 get_system_obj_state: rctx=0x7ffe8e3f7218 obj=default.rgw.meta:users.keys:MH1WC2XQ1S8UISFDZC8W state=0x55b52d9b60b0 s->prefetch_data=0 2023-10-05T19:42:16.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.936+0000 7efc4759be40 10 cache get: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W : miss 2023-10-05T19:42:16.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.936+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:103 8.18 8:1f190392:users.keys::MH1WC2XQ1S8UISFDZC8W:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db9eb70 con 0x55b52dadd6e0 2023-10-05T19:42:16.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.936+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.1 v2:172.21.15.12:6808/29933 35 ==== osd_op_reply(103 MH1WC2XQ1S8UISFDZC8W [getxattrs,stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 206+0+0 (crc 0 0 0) 0x7efc40014040 con 0x55b52dadd6e0 2023-10-05T19:42:16.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.936+0000 7efc4759be40 10 cache put: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W info.flags=0x0 2023-10-05T19:42:16.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.936+0000 7efc4759be40 10 adding default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W to cache LRU end 2023-10-05T19:42:16.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.936+0000 7efc4759be40 20 get_system_obj_state: rctx=0x7ffe8e3f6ab8 obj=default.rgw.meta:users.keys:MH1WC2XQ1S8UISFDZC8W state=0x55b52d9b60b0 s->prefetch_data=0 2023-10-05T19:42:16.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.936+0000 7efc4759be40 10 cache get: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W : hit (negative entry) 2023-10-05T19:42:16.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.936+0000 7efc4759be40 20 get_system_obj_state: rctx=0x7ffe8e3f7158 obj=default.rgw.meta:users.keys:MH1WC2XQ1S8UISFDZC8W state=0x55b52d9b60b0 s->prefetch_data=0 2023-10-05T19:42:16.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.936+0000 7efc4759be40 10 cache get: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W : hit (negative entry) 2023-10-05T19:42:16.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.936+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:104 8.7 8:e1d59a8c:users.uid::ada:head [delete,create,call version.set in=58b,writefull 0~439 in=439b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52dba1600 con 0x7efc3801a740 2023-10-05T19:42:16.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.940+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.0 v2:172.21.15.8:6800/27563 26 ==== osd_op_reply(104 ada [delete,create,call,writefull 0~439] v30'3 uv3 ondisk = 0) v8 ==== 273+0+0 (crc 0 0 0) 0x7efc40014040 con 0x7efc3801a740 2023-10-05T19:42:16.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.940+0000 7efc4759be40 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2023-10-05T19:42:16.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.940+0000 7efc4759be40 10 moving default.rgw.meta+users.uid+ada to cache LRU end 2023-10-05T19:42:16.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.940+0000 7efc4759be40 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ada, ofs0, ns] 2023-10-05T19:42:16.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.940+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:105 7.d 7:b4812045:::notify.4:head [notify cookie 94236644606528 in=627b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52dba1f90 con 0x55b52dadd6e0 2023-10-05T19:42:16.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.940+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.1 v2:172.21.15.12:6808/29933 36 ==== watch-notify(notify (1) cookie 94236644080224 notify 128849018899 ret 0) v3 ==== 657+0+0 (crc 0 0 0) 0x7efc40026880 con 0x55b52dadd6e0 2023-10-05T19:42:16.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.944+0000 7efc3e7fc700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018899 cookie 94236644080224 notifier 4795 bl.length()=615 2023-10-05T19:42:16.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.944+0000 7efc3e7fc700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2023-10-05T19:42:16.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.944+0000 7efc3e7fc700 10 rgw watcher librados: moving default.rgw.meta+users.uid+ada to cache LRU end 2023-10-05T19:42:16.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.944+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.1 v2:172.21.15.12:6808/29933 37 ==== osd_op_reply(105 notify.4 [notify cookie 94236644606528 out=8b] v0'0 uv270 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7efc40014040 con 0x55b52dadd6e0 2023-10-05T19:42:16.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.944+0000 7efc3e7fc700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:106 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x7efc24003d90 con 0x55b52dadd6e0 2023-10-05T19:42:16.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.944+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.1 v2:172.21.15.12:6808/29933 38 ==== osd_op_reply(106 notify.4 [notify-ack] v0'0 uv270 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc40014040 con 0x55b52dadd6e0 2023-10-05T19:42:16.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.944+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.1 v2:172.21.15.12:6808/29933 39 ==== watch-notify(notify_complete (2) cookie 94236644606528 notify 128849018899 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7efc40026a10 con 0x55b52dadd6e0 2023-10-05T19:42:16.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.944+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:107 8.18 8:1f190392:users.keys::MH1WC2XQ1S8UISFDZC8W:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52dba1f10 con 0x55b52dadd6e0 2023-10-05T19:42:16.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.948+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.1 v2:172.21.15.12:6808/29933 40 ==== osd_op_reply(107 MH1WC2XQ1S8UISFDZC8W [delete,create,writefull 0~7] v30'21 uv21 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x7efc40014040 con 0x55b52dadd6e0 2023-10-05T19:42:16.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.948+0000 7efc4759be40 10 cache put: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W info.flags=0x7 2023-10-05T19:42:16.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.948+0000 7efc4759be40 10 moving default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W to cache LRU end 2023-10-05T19:42:16.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.948+0000 7efc4759be40 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:MH1WC2XQ1S8UISFDZC8W, ofs0, ns] 2023-10-05T19:42:16.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.948+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:108 7.15 7:a93a5511:::notify.2:head [notify cookie 94236644612640 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52dba1250 con 0x55b52db17370 2023-10-05T19:42:16.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.948+0000 7efc448cf700 1 -- 172.21.15.12:0/1591481545 <== osd.3 v2:172.21.15.12:6816/29951 24 ==== watch-notify(notify (1) cookie 94236644074640 notify 128849018893 ret 0) v3 ==== 219+0+0 (crc 0 0 0) 0x55b52db36bc0 con 0x55b52db17370 2023-10-05T19:42:16.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.948+0000 7efc3effd700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 128849018893 cookie 94236644074640 notifier 4795 bl.length()=177 2023-10-05T19:42:16.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.948+0000 7efc3effd700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W info.flags=0x7 2023-10-05T19:42:16.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.948+0000 7efc3effd700 10 rgw watcher librados: moving default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W to cache LRU end 2023-10-05T19:42:16.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.948+0000 7efc448cf700 1 -- 172.21.15.12:0/1591481545 <== osd.3 v2:172.21.15.12:6816/29951 25 ==== osd_op_reply(108 notify.2 [notify cookie 94236644612640 out=8b] v0'0 uv270 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7efc30006070 con 0x55b52db17370 2023-10-05T19:42:16.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.948+0000 7efc3effd700 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:109 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x7efb94003450 con 0x55b52db17370 2023-10-05T19:42:16.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.948+0000 7efc448cf700 1 -- 172.21.15.12:0/1591481545 <== osd.3 v2:172.21.15.12:6816/29951 26 ==== osd_op_reply(109 notify.2 [notify-ack] v0'0 uv270 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc30006070 con 0x55b52db17370 2023-10-05T19:42:16.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.948+0000 7efc448cf700 1 -- 172.21.15.12:0/1591481545 <== osd.3 v2:172.21.15.12:6816/29951 27 ==== watch-notify(notify_complete (2) cookie 94236644612640 notify 128849018893 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55b52db367e0 con 0x55b52db17370 2023-10-05T19:42:16.957 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-10-05T19:42:16.958 INFO:teuthology.orchestra.run.smithi012.stdout: "user_id": "ada", 2023-10-05T19:42:16.958 INFO:teuthology.orchestra.run.smithi012.stdout: "display_name": "Ms. Admin User", 2023-10-05T19:42:16.958 INFO:teuthology.orchestra.run.smithi012.stdout: "email": "", 2023-10-05T19:42:16.958 INFO:teuthology.orchestra.run.smithi012.stdout: "suspended": 0, 2023-10-05T19:42:16.958 INFO:teuthology.orchestra.run.smithi012.stdout: "max_buckets": 0, 2023-10-05T19:42:16.958 INFO:teuthology.orchestra.run.smithi012.stdout: "subusers": [], 2023-10-05T19:42:16.959 INFO:teuthology.orchestra.run.smithi012.stdout: "keys": [ 2023-10-05T19:42:16.959 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:42:16.959 INFO:teuthology.orchestra.run.smithi012.stdout: "user": "ada", 2023-10-05T19:42:16.959 INFO:teuthology.orchestra.run.smithi012.stdout: "access_key": "MH1WC2XQ1S8UISFDZC8W", 2023-10-05T19:42:16.959 INFO:teuthology.orchestra.run.smithi012.stdout: "secret_key": "dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG" 2023-10-05T19:42:16.959 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:42:16.960 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:42:16.960 INFO:teuthology.orchestra.run.smithi012.stdout: "swift_keys": [], 2023-10-05T19:42:16.960 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": [ 2023-10-05T19:42:16.960 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:42:16.960 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "buckets", 2023-10-05T19:42:16.960 INFO:teuthology.orchestra.run.smithi012.stdout: "perm": "*" 2023-10-05T19:42:16.961 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:42:16.961 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:42:16.961 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "info", 2023-10-05T19:42:16.961 INFO:teuthology.orchestra.run.smithi012.stdout: "perm": "read" 2023-10-05T19:42:16.961 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:42:16.961 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:42:16.962 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "ratelimit", 2023-10-05T19:42:16.962 INFO:teuthology.orchestra.run.smithi012.stdout: "perm": "*" 2023-10-05T19:42:16.962 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:42:16.962 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:42:16.962 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "usage", 2023-10-05T19:42:16.962 INFO:teuthology.orchestra.run.smithi012.stdout: "perm": "*" 2023-10-05T19:42:16.963 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:42:16.963 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:42:16.963 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "users", 2023-10-05T19:42:16.963 INFO:teuthology.orchestra.run.smithi012.stdout: "perm": "*" 2023-10-05T19:42:16.963 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:42:16.963 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-10-05T19:42:16.964 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "zone", 2023-10-05T19:42:16.964 INFO:teuthology.orchestra.run.smithi012.stdout: "perm": "*" 2023-10-05T19:42:16.964 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-10-05T19:42:16.964 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2023-10-05T19:42:16.964 INFO:teuthology.orchestra.run.smithi012.stdout: "op_mask": "read, write, delete", 2023-10-05T19:42:16.964 INFO:teuthology.orchestra.run.smithi012.stdout: "default_placement": "", 2023-10-05T19:42:16.964 INFO:teuthology.orchestra.run.smithi012.stdout: "default_storage_class": "", 2023-10-05T19:42:16.964 INFO:teuthology.orchestra.run.smithi012.stdout: "placement_tags": [], 2023-10-05T19:42:16.964 INFO:teuthology.orchestra.run.smithi012.stdout: "bucket_quota": { 2023-10-05T19:42:16.964 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:42:16.964 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:42:16.964 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:42:16.965 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:42:16.965 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:42:16.965 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:42:16.965 INFO:teuthology.orchestra.run.smithi012.stdout: "user_quota": { 2023-10-05T19:42:16.965 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": false, 2023-10-05T19:42:16.965 INFO:teuthology.orchestra.run.smithi012.stdout: "check_on_raw": false, 2023-10-05T19:42:16.965 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size": -1, 2023-10-05T19:42:16.965 INFO:teuthology.orchestra.run.smithi012.stdout: "max_size_kb": 0, 2023-10-05T19:42:16.965 INFO:teuthology.orchestra.run.smithi012.stdout: "max_objects": -1 2023-10-05T19:42:16.965 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-10-05T19:42:16.965 INFO:teuthology.orchestra.run.smithi012.stdout: "temp_url_keys": [], 2023-10-05T19:42:16.965 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "rgw", 2023-10-05T19:42:16.965 INFO:teuthology.orchestra.run.smithi012.stdout: "mfa_ids": [] 2023-10-05T19:42:16.966 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-10-05T19:42:16.966 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-10-05T19:42:16.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.948+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:110 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94236644063056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52daf2590 con 0x55b52db12bd0 2023-10-05T19:42:16.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.952+0000 7efc450d0700 1 -- 172.21.15.12:0/1591481545 <== osd.2 v2:172.21.15.12:6800/29934 21 ==== osd_op_reply(110 notify.0 [watch unwatch cookie 94236644063056] v30'272 uv270 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc3804f040 con 0x55b52db12bd0 2023-10-05T19:42:16.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.952+0000 7efc4759be40 20 remove_watcher() i=0 2023-10-05T19:42:16.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.952+0000 7efc4759be40 2 removed watcher, disabling cache 2023-10-05T19:42:16.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.952+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:111 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94236644071408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1b750 con 0x55b52dadd6e0 2023-10-05T19:42:16.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.956+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.1 v2:172.21.15.12:6808/29933 41 ==== osd_op_reply(111 notify.1 [watch unwatch cookie 94236644071408] v30'272 uv270 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc40014040 con 0x55b52dadd6e0 2023-10-05T19:42:16.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.956+0000 7efc4759be40 20 remove_watcher() i=1 2023-10-05T19:42:16.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.956+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] -- osd_op(unknown.0.0:112 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94236644074640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1b750 con 0x55b52db17370 2023-10-05T19:42:16.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.960+0000 7efc448cf700 1 -- 172.21.15.12:0/1591481545 <== osd.3 v2:172.21.15.12:6816/29951 28 ==== osd_op_reply(112 notify.2 [watch unwatch cookie 94236644074640] v30'272 uv270 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc30006070 con 0x55b52db17370 2023-10-05T19:42:16.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.960+0000 7efc4759be40 20 remove_watcher() i=2 2023-10-05T19:42:16.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.960+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] -- osd_op(unknown.0.0:113 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94236644081488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1e490 con 0x55b52db12bd0 2023-10-05T19:42:16.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.960+0000 7efc450d0700 1 -- 172.21.15.12:0/1591481545 <== osd.2 v2:172.21.15.12:6800/29934 22 ==== osd_op_reply(113 notify.3 [watch unwatch cookie 94236644081488] v30'272 uv270 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc3804f040 con 0x55b52db12bd0 2023-10-05T19:42:16.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.960+0000 7efc4759be40 20 remove_watcher() i=3 2023-10-05T19:42:16.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.960+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:114 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94236644080224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1ff50 con 0x55b52dadd6e0 2023-10-05T19:42:16.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.964+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.1 v2:172.21.15.12:6808/29933 42 ==== osd_op_reply(114 notify.4 [watch unwatch cookie 94236644080224] v30'272 uv270 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc40014040 con 0x55b52dadd6e0 2023-10-05T19:42:16.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.964+0000 7efc4759be40 20 remove_watcher() i=4 2023-10-05T19:42:16.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.964+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:115 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94236644082656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1e490 con 0x7efc3801a740 2023-10-05T19:42:16.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.968+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.0 v2:172.21.15.8:6800/27563 27 ==== osd_op_reply(115 notify.5 [watch unwatch cookie 94236644082656] v30'272 uv270 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc40014040 con 0x7efc3801a740 2023-10-05T19:42:16.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.968+0000 7efc4759be40 20 remove_watcher() i=5 2023-10-05T19:42:16.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.968+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:116 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94236644087424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1fa60 con 0x7efc3801a740 2023-10-05T19:42:16.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.972+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.0 v2:172.21.15.8:6800/27563 28 ==== osd_op_reply(116 notify.6 [watch unwatch cookie 94236644087424] v30'272 uv270 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc40014040 con 0x7efc3801a740 2023-10-05T19:42:16.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.972+0000 7efc4759be40 20 remove_watcher() i=6 2023-10-05T19:42:16.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.972+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] -- osd_op(unknown.0.0:117 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94236644088528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1e490 con 0x7efc3801a740 2023-10-05T19:42:16.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.972+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.0 v2:172.21.15.8:6800/27563 29 ==== osd_op_reply(117 notify.7 [watch unwatch cookie 94236644088528] v30'272 uv270 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efc40014040 con 0x7efc3801a740 2023-10-05T19:42:16.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.972+0000 7efc4759be40 20 remove_watcher() i=7 2023-10-05T19:42:16.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.976+0000 7efc167e4700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-10-05T19:42:16.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.976+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 --> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] -- osd_op(unknown.0.0:118 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94236644091712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30) v8 -- 0x55b52db1e490 con 0x55b52dadd6e0 2023-10-05T19:42:16.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.980+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 <== osd.1 v2:172.21.15.12:6808/29933 43 ==== osd_op_reply(118 data_loggenerations_metadata [watch unwatch cookie 94236644091712] v30'261 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7efc40014040 con 0x55b52dadd6e0 2023-10-05T19:42:16.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.980+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7efc3801a740 msgr2=0x7efc38039880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.980+0000 7efc4759be40 1 --2- 172.21.15.12:0/1591481545 >> [v2:172.21.15.8:6800/27563,v1:172.21.15.8:6801/27563] conn(0x7efc3801a740 0x7efc38039880 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.980+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55b52dadd6e0 msgr2=0x55b52db106a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.980+0000 7efc4759be40 1 --2- 172.21.15.12:0/1591481545 >> [v2:172.21.15.12:6808/29933,v1:172.21.15.12:6809/29933] conn(0x55b52dadd6e0 0x55b52db106a0 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.980+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55b52db12bd0 msgr2=0x55b52db14fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.980+0000 7efc4759be40 1 --2- 172.21.15.12:0/1591481545 >> [v2:172.21.15.12:6800/29934,v1:172.21.15.12:6801/29934] conn(0x55b52db12bd0 0x55b52db14fb0 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.980+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55b52db17370 msgr2=0x55b52db19780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.980+0000 7efc4759be40 1 --2- 172.21.15.12:0/1591481545 >> [v2:172.21.15.12:6816/29951,v1:172.21.15.12:6817/29951] conn(0x55b52db17370 0x55b52db19780 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.980+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7efc28030b70 msgr2=0x7efc28033020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.980+0000 7efc4759be40 1 --2- 172.21.15.12:0/1591481545 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7efc28030b70 0x7efc28033020 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efc300011e0 tx=0x7efc300013a0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.980+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b52dadcdd0 msgr2=0x55b52db04c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.980+0000 7efc458d1700 1 -- 172.21.15.12:0/1591481545 reap_dead start 2023-10-05T19:42:16.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.980+0000 7efc4759be40 1 --2- 172.21.15.12:0/1591481545 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b52dadcdd0 0x55b52db04c10 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7efc38005b20 tx=0x7efc3801d000 comp rx=0 tx=0).stop 2023-10-05T19:42:16.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.980+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 shutdown_connections 2023-10-05T19:42:16.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.980+0000 7efc4759be40 1 --2- 172.21.15.12:0/1591481545 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b52dadcdd0 0x55b52db04c10 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.980+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 >> 172.21.15.12:0/1591481545 conn(0x55b52dad87f0 msgr2=0x55b52daf6370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:16.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.980+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 shutdown_connections 2023-10-05T19:42:16.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.980+0000 7efc4759be40 1 -- 172.21.15.12:0/1591481545 wait complete. 2023-10-05T19:42:16.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.980+0000 7efc4759be40 1 -- 172.21.15.12:0/1057623985 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7efb8c030ca0 msgr2=0x7efb8c033150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.980+0000 7efc4759be40 1 --2- 172.21.15.12:0/1057623985 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7efb8c030ca0 0x7efb8c033150 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55b52db25e70 tx=0x7efc400325e0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.980+0000 7efc4759be40 1 -- 172.21.15.12:0/1057623985 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b52db30620 msgr2=0x55b52db361e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:42:16.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.980+0000 7efc4759be40 1 --2- 172.21.15.12:0/1057623985 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b52db30620 0x55b52db361e0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7efc30000fb0 tx=0x7efc3003c030 comp rx=0 tx=0).stop 2023-10-05T19:42:16.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.980+0000 7efc4759be40 1 -- 172.21.15.12:0/1057623985 shutdown_connections 2023-10-05T19:42:16.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.980+0000 7efc4759be40 1 --2- 172.21.15.12:0/1057623985 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7efb8c030ca0 0x7efb8c033150 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.980+0000 7efc4759be40 1 --2- 172.21.15.12:0/1057623985 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55b52db30620 0x55b52db361e0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:42:16.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.980+0000 7efc4759be40 1 -- 172.21.15.12:0/1057623985 >> 172.21.15.12:0/1057623985 conn(0x55b52db2b9e0 msgr2=0x55b52db29a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:42:16.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.980+0000 7efc4759be40 1 -- 172.21.15.12:0/1057623985 shutdown_connections 2023-10-05T19:42:16.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-10-05T19:42:16.980+0000 7efc4759be40 1 -- 172.21.15.12:0/1057623985 wait complete. 2023-10-05T19:42:16.991 DEBUG:tasks.util.rgw: json result: {'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'users', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:42:16.992 ERROR:root:{'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'users', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:42:16.992 ERROR:root:0 2023-10-05T19:42:16.992 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2023-10-05T19:42:16.999 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchUser', 'RequestId': 'tx000009a51b0f2692ba76e-00651f1198-107b-default', 'HostId': '107b-default-default'} 2023-10-05T19:42:16.999 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'create'] {'uid': 'foo', 'display-name': 'Foo', 'email': 'foo@foo.com', 'access-key': '9te6NH5mcdcq0Tc5i8i1', 'secret-key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'max-buckets': '4'} 2023-10-05T19:42:17.020 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': 'false', 'admin': 'false', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:42:17.020 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'list'] {'list': '', 'max-entries': 0} 2023-10-05T19:42:17.057 INFO:tasks.radosgw_admin_rest: json result: {'keys': [], 'truncated': True, 'count': 0, 'marker': '8:1e6ca5ed:users.uid::foo:head'} 2023-10-05T19:42:17.057 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'list'] {'list': '', 'max-entries': 1} 2023-10-05T19:42:17.094 INFO:tasks.radosgw_admin_rest: json result: {'keys': ['foo'], 'truncated': True, 'count': 1, 'marker': '8:a00000ee:users.uid::bar.buckets:head'} 2023-10-05T19:42:17.095 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'list'] {'list': '', 'max-entries': 1, 'marker': '8:a00000ee:users.uid::bar.buckets:head'} 2023-10-05T19:42:17.113 INFO:tasks.radosgw_admin_rest: json result: {'keys': ['ada'], 'truncated': False, 'count': 1} 2023-10-05T19:42:17.113 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2023-10-05T19:42:17.118 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': 'false', 'admin': 'false', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:42:17.118 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'access-key': '9te6NH5mcdcq0Tc5i8i1', 'uid': 'uid_not_exist'} 2023-10-05T19:42:17.124 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': 'false', 'admin': 'false', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:42:17.124 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'modify'] {'uid': 'foo', 'suspended': True} 2023-10-05T19:42:17.135 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': 'false', 'admin': 'false', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:42:17.135 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2023-10-05T19:42:17.139 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': 'false', 'admin': 'false', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:42:17.140 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'modify'] {'uid': 'foo', 'suspended': 'false'} 2023-10-05T19:42:17.151 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': 'false', 'admin': 'false', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:42:17.151 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2023-10-05T19:42:17.156 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': 'false', 'admin': 'false', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:42:17.156 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['key', 'create'] {'uid': 'foo', 'access-key': 'p5YnriCv1nAtykxBrupQ', 'secret-key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'} 2023-10-05T19:42:17.172 INFO:tasks.radosgw_admin_rest: json result: [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}, {'user': 'foo', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'}] 2023-10-05T19:42:17.173 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2023-10-05T19:42:17.177 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}, {'user': 'foo', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': 'false', 'admin': 'false', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:42:17.177 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['key', 'rm'] {'uid': 'foo', 'access-key': 'p5YnriCv1nAtykxBrupQ'} 2023-10-05T19:42:17.191 INFO:tasks.radosgw_admin_rest: empty result 2023-10-05T19:42:17.192 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2023-10-05T19:42:17.196 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': 'false', 'admin': 'false', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:42:17.197 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['subuser', 'create'] {'subuser': 'foo:foo1', 'secret-key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', 'key-type': 'swift'} 2023-10-05T19:42:17.212 INFO:tasks.radosgw_admin_rest: json result: [{'id': 'foo:foo1', 'permissions': ''}] 2023-10-05T19:42:17.212 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2023-10-05T19:42:17.216 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL'}], 'caps': [], 'op_mask': 'read, write, delete', 'system': 'false', 'admin': 'false', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:42:17.217 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['subuser', 'create'] {'subuser': 'foo:foo2', 'secret-key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', 'key-type': 'swift'} 2023-10-05T19:42:17.233 INFO:tasks.radosgw_admin_rest: json result: [{'id': 'foo:foo1', 'permissions': ''}, {'id': 'foo:foo2', 'permissions': ''}] 2023-10-05T19:42:17.233 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2023-10-05T19:42:17.238 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': ''}, {'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL'}, {'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy'}], 'caps': [], 'op_mask': 'read, write, delete', 'system': 'false', 'admin': 'false', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:42:17.238 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['key', 'rm'] {'subuser': 'foo:foo1', 'key-type': 'swift'} 2023-10-05T19:42:17.253 INFO:tasks.radosgw_admin_rest: empty result 2023-10-05T19:42:17.254 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2023-10-05T19:42:17.259 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': ''}, {'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy'}], 'caps': [], 'op_mask': 'read, write, delete', 'system': 'false', 'admin': 'false', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:42:17.260 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['subuser', 'rm'] {'subuser': 'foo:foo1'} 2023-10-05T19:42:17.271 INFO:tasks.radosgw_admin_rest: empty result 2023-10-05T19:42:17.271 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2023-10-05T19:42:17.279 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy'}], 'caps': [], 'op_mask': 'read, write, delete', 'system': 'false', 'admin': 'false', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:42:17.279 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['subuser', 'rm'] {'subuser': 'foo:foo2', 'key-type': 'swift', '{purge-keys': True} 2023-10-05T19:42:17.295 INFO:tasks.radosgw_admin_rest: empty result 2023-10-05T19:42:17.295 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2023-10-05T19:42:17.299 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': 'false', 'admin': 'false', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:42:17.300 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'uid': 'foo'} 2023-10-05T19:42:17.307 INFO:tasks.radosgw_admin_rest: json result: [] 2023-10-05T19:42:17.307 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'uid': 'foo', 'stats': True} 2023-10-05T19:42:17.313 INFO:tasks.radosgw_admin_rest: json result: [] 2023-10-05T19:42:17.337 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'uid': 'foo'} 2023-10-05T19:42:17.342 INFO:tasks.radosgw_admin_rest: json result: ['myfoo'] 2023-10-05T19:42:17.342 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'bucket': 'myfoo', 'stats': True} 2023-10-05T19:42:17.348 INFO:tasks.radosgw_admin_rest: json result: {'bucket': 'myfoo', 'num_shards': 11, 'tenant': '', 'zonegroup': '1602ab00-ae0f-4921-ad63-b9cfe7bc77db', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '93ac49f4-ff49-4f99-811b-6453b28decd5.4221.9', 'marker': '93ac49f4-ff49-4f99-811b-6453b28decd5.4221.9', 'index_type': 'Normal', 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2023-10-05T19:42:17.325565Z', 'creation_time': '2023-10-05T19:42:17.318079Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}} 2023-10-05T19:42:17.348 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'uid': 'foo', 'stats': True} 2023-10-05T19:42:17.354 INFO:tasks.radosgw_admin_rest: json result: [{'bucket': 'myfoo', 'num_shards': 11, 'tenant': '', 'zonegroup': '1602ab00-ae0f-4921-ad63-b9cfe7bc77db', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '93ac49f4-ff49-4f99-811b-6453b28decd5.4221.9', 'marker': '93ac49f4-ff49-4f99-811b-6453b28decd5.4221.9', 'index_type': 'Normal', 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2023-10-05T19:42:17.325565Z', 'creation_time': '2023-10-05T19:42:17.318079Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}}] 2023-10-05T19:42:17.365 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'bucket': 'myfoo', 'stats': True} 2023-10-05T19:42:17.372 INFO:tasks.radosgw_admin_rest: json result: {'bucket': 'myfoo', 'num_shards': 11, 'tenant': '', 'zonegroup': '1602ab00-ae0f-4921-ad63-b9cfe7bc77db', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '93ac49f4-ff49-4f99-811b-6453b28decd5.4221.9', 'marker': '93ac49f4-ff49-4f99-811b-6453b28decd5.4221.9', 'index_type': 'Normal', 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#2,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2023-10-05T19:42:17.325565Z', 'creation_time': '2023-10-05T19:42:17.318079Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {'rgw.main': {'size': 3, 'size_actual': 4096, 'size_utilized': 3, 'size_kb': 1, 'size_kb_actual': 4, 'size_kb_utilized': 1, 'num_objects': 1}}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}} 2023-10-05T19:42:17.372 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'bucket': 'doesnotexist'} 2023-10-05T19:42:17.377 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchBucket', 'RequestId': 'tx0000084512283c41f97c6-00651f1199-107b-default', 'HostId': '107b-default-default'} 2023-10-05T19:42:17.387 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'unlink'] {'uid': 'foo', 'bucket': 'myfoo'} 2023-10-05T19:42:17.400 INFO:tasks.radosgw_admin_rest: empty result 2023-10-05T19:42:17.400 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'create'] {'uid': 'fud', 'display-name': 'Fud', 'access-key': 'p5YnriCv1nAtykxBrupQ', 'secret-key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', 'max-buckets': '1'} 2023-10-05T19:42:17.417 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'fud', 'display_name': 'Fud', 'email': '', 'suspended': 0, 'max_buckets': 1, 'subusers': [], 'keys': [{'user': 'fud', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': 'false', 'admin': 'false', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:42:17.472 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'link'] {'uid': 'fud', 'bucket': 'myfoo', 'bucket-id': '93ac49f4-ff49-4f99-811b-6453b28decd5.4221.9'} 2023-10-05T19:42:17.494 INFO:tasks.radosgw_admin_rest: empty result 2023-10-05T19:42:17.497 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'link'] {'uid': 'foo', 'bucket': 'myfoo', 'bucket-id': '93ac49f4-ff49-4f99-811b-6453b28decd5.4221.9'} 2023-10-05T19:42:17.521 INFO:tasks.radosgw_admin_rest: empty result 2023-10-05T19:42:17.521 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'rm'] {'uid': 'fud'} 2023-10-05T19:42:17.543 INFO:tasks.radosgw_admin_rest: empty result 2023-10-05T19:42:17.556 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['object', 'rm'] {'bucket': 'myfoo', 'object': 'four'} 2023-10-05T19:42:17.568 INFO:tasks.radosgw_admin_rest: empty result 2023-10-05T19:42:17.569 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'bucket': 'myfoo', 'stats': True} 2023-10-05T19:42:17.576 INFO:tasks.radosgw_admin_rest: json result: {'bucket': 'myfoo', 'num_shards': 11, 'tenant': '', 'zonegroup': '1602ab00-ae0f-4921-ad63-b9cfe7bc77db', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '93ac49f4-ff49-4f99-811b-6453b28decd5.4221.9', 'marker': '93ac49f4-ff49-4f99-811b-6453b28decd5.4221.9', 'index_type': 'Normal', 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#3,6#3,7#1,8#3,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2023-10-05T19:42:17.506230Z', 'creation_time': '2023-10-05T19:42:17.318079Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {'rgw.main': {'size': 0, 'size_actual': 0, 'size_utilized': 0, 'size_kb': 0, 'size_kb_actual': 0, 'size_kb_utilized': 0, 'num_objects': 0}}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}} 2023-10-05T19:43:17.689 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'categories': 'delete_obj'} 2023-10-05T19:43:17.723 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'anonymous', 'buckets': [{'bucket': '', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'anonymous', 'categories': []}]}, {'user': 'bar', 'buckets': [{'bucket': 'myfoo6', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'bar', 'categories': []}]}, {'user': 'bud', 'buckets': [{'bucket': 'myfoo6', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'bud', 'categories': []}]}, {'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}]}, {'bucket': 'useless-bucket', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}, {'user': 'fud', 'buckets': [{'bucket': 'myfoo', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'fud', 'categories': []}, {'bucket': 'myfoo7', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'fud', 'categories': []}]}], 'summary': [{'user': 'anonymous', 'categories': [], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 0, 'successful_ops': 0}}, {'user': 'bar', 'categories': [], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 0, 'successful_ops': 0}}, {'user': 'bud', 'categories': [], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 0, 'successful_ops': 0}}, {'user': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3}}, {'user': 'fud', 'categories': [], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 0, 'successful_ops': 0}}]} 2023-10-05T19:43:17.723 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] None 2023-10-05T19:43:17.756 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'anonymous', 'buckets': [{'bucket': '', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'anonymous', 'categories': [{'category': 'list_buckets', 'bytes_sent': 231, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}, {'user': 'bar', 'buckets': [{'bucket': 'myfoo6', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'bar', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 8, 'ops': 1, 'successful_ops': 1}]}]}, {'user': 'bud', 'buckets': [{'bucket': 'myfoo6', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'bud', 'categories': [{'category': 'get_obj', 'bytes_sent': 16, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'stat_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}]}]}, {'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 10, 'ops': 3, 'successful_ops': 3}]}, {'bucket': 'useless-bucket', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 14, 'ops': 1, 'successful_ops': 1}]}]}, {'user': 'fud', 'buckets': [{'bucket': 'myfoo', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'fud', 'categories': [{'category': 'put_obj', 'bytes_sent': 215, 'bytes_received': 0, 'ops': 1, 'successful_ops': 0}]}, {'bucket': 'myfoo7', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'fud', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'get_obj', 'bytes_sent': 8, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 8, 'ops': 1, 'successful_ops': 1}, {'category': 'stat_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'anonymous', 'categories': [{'category': 'list_buckets', 'bytes_sent': 231, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 'total': {'bytes_sent': 231, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}}, {'user': 'bar', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 8, 'ops': 1, 'successful_ops': 1}], 'total': {'bytes_sent': 0, 'bytes_received': 8, 'ops': 2, 'successful_ops': 2}}, {'user': 'bud', 'categories': [{'category': 'get_obj', 'bytes_sent': 16, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'stat_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}], 'total': {'bytes_sent': 16, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}}, {'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 24, 'ops': 4, 'successful_ops': 4}], 'total': {'bytes_sent': 0, 'bytes_received': 24, 'ops': 10, 'successful_ops': 10}}, {'user': 'fud', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'get_obj', 'bytes_sent': 8, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 215, 'bytes_received': 8, 'ops': 2, 'successful_ops': 1}, {'category': 'stat_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 'total': {'bytes_sent': 223, 'bytes_received': 8, 'ops': 5, 'successful_ops': 4}}]} 2023-10-05T19:43:17.756 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo'} 2023-10-05T19:43:17.765 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 10, 'ops': 3, 'successful_ops': 3}]}, {'bucket': 'useless-bucket', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 14, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 24, 'ops': 4, 'successful_ops': 4}], 'total': {'bytes_sent': 0, 'bytes_received': 24, 'ops': 10, 'successful_ops': 10}}]} 2023-10-05T19:43:17.765 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo', 'categories': 'create_bucket'} 2023-10-05T19:43:17.771 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'useless-bucket', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}}]} 2023-10-05T19:43:17.771 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo', 'categories': 'put_obj'} 2023-10-05T19:43:17.777 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': [{'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 10, 'ops': 3, 'successful_ops': 3}]}, {'bucket': 'useless-bucket', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': [{'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 14, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 24, 'ops': 4, 'successful_ops': 4}], 'total': {'bytes_sent': 0, 'bytes_received': 24, 'ops': 4, 'successful_ops': 4}}]} 2023-10-05T19:43:17.777 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo', 'categories': 'delete_obj'} 2023-10-05T19:43:17.783 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}]}, {'bucket': 'useless-bucket', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3}}]} 2023-10-05T19:43:17.783 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo', 'categories': 'delete_bucket'} 2023-10-05T19:43:17.789 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': []}, {'bucket': 'useless-bucket', 'time': '2023-10-05T19:00:00.000000Z', 'epoch': 1696532400, 'owner': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}}]} 2023-10-05T19:43:17.789 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'trim'] {'uid': 'foo'} 2023-10-05T19:43:17.801 INFO:tasks.radosgw_admin_rest: empty result 2023-10-05T19:43:17.802 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo'} 2023-10-05T19:43:17.807 INFO:tasks.radosgw_admin_rest: json result: {'entries': [], 'summary': []} 2023-10-05T19:43:17.808 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'modify'] {'uid': 'foo', 'suspended': True} 2023-10-05T19:43:17.824 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': 'false', 'admin': 'false', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:43:17.828 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'modify'] {'uid': 'foo', 'suspended': 'false'} 2023-10-05T19:43:17.844 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': 'false', 'admin': 'false', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:43:18.144 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'rm'] {'uid': 'foo'} 2023-10-05T19:43:18.150 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'BucketAlreadyExists', 'RequestId': 'tx00000685f416c6a05222a-00651f11d6-107b-default', 'HostId': '107b-default-default'} 2023-10-05T19:43:18.345 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['policy', 'show'] {'bucket': 'myfoo', 'object': b'bb3aec0fdcdbc2974890f805c585d432'} 2023-10-05T19:43:18.351 INFO:tasks.radosgw_admin_rest: json result: {'acl': {'acl_user_map': [{'user': 'foo', 'acl': 15}], 'acl_group_map': [], 'grant_map': [{'id': 'foo', 'grant': {'type': {'type': 0}, 'id': 'foo', 'email': '', 'permission': {'flags': 15}, 'name': 'Foo', 'group': 0, 'url_spec': ''}}]}, 'owner': {'id': 'foo', 'display_name': 'Foo'}} 2023-10-05T19:43:18.360 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['policy', 'show'] {'bucket': 'myfoo', 'object': b'bb3aec0fdcdbc2974890f805c585d432'} 2023-10-05T19:43:18.365 INFO:tasks.radosgw_admin_rest: json result: {'acl': {'acl_user_map': [{'user': 'foo', 'acl': 15}], 'acl_group_map': [{'group': 1, 'acl': 1}], 'grant_map': [{'id': '', 'grant': {'type': {'type': 2}, 'id': '', 'email': '', 'permission': {'flags': 1}, 'name': '', 'group': 1, 'url_spec': ''}}, {'id': 'foo', 'grant': {'type': {'type': 0}, 'id': 'foo', 'email': '', 'permission': {'flags': 15}, 'name': 'Foo', 'group': 0, 'url_spec': ''}}]}, 'owner': {'id': 'foo', 'display_name': 'Foo'}} 2023-10-05T19:43:18.483 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'rm'] {'bucket': 'myfoo', 'purge-objects': True} 2023-10-05T19:43:18.582 INFO:tasks.radosgw_admin_rest: empty result 2023-10-05T19:43:18.583 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['caps', 'add'] {'uid': 'foo', 'user-caps': 'usage=read'} 2023-10-05T19:43:18.593 INFO:tasks.radosgw_admin_rest: json result: [{'type': 'usage', 'perm': 'read'}] 2023-10-05T19:43:18.594 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['caps', 'rm'] {'uid': 'foo', 'user-caps': 'usage=read'} 2023-10-05T19:43:18.604 INFO:tasks.radosgw_admin_rest: json result: [] 2023-10-05T19:43:18.627 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'rm'] {'uid': 'foo'} 2023-10-05T19:43:18.632 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'BucketAlreadyExists', 'RequestId': 'tx00000c3c29d5c2ccae9e1-00651f11d6-107b-default', 'HostId': '107b-default-default'} 2023-10-05T19:43:18.647 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'rm'] {'uid': 'foo', 'purge-data': True} 2023-10-05T19:43:18.717 INFO:tasks.radosgw_admin_rest: empty result 2023-10-05T19:43:18.718 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2023-10-05T19:43:18.724 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchUser', 'RequestId': 'tx000005401f03ec6810bfa-00651f11d6-107b-default', 'HostId': '107b-default-default'} 2023-10-05T19:43:18.724 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['info', ''] None 2023-10-05T19:43:18.728 INFO:tasks.radosgw_admin_rest: json result: {'info': {'storage_backends': [{'name': 'rados', 'cluster_id': 'dac6c186-0634-45d9-839d-04f1e484462d'}]}} 2023-10-05T19:43:18.728 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'create'] {'uid': 'ratelimit_user', 'display-name': 'Foo', 'email': 'foo@foo.com', 'access-key': '9te6NH5mcdcq0Tc5i8i1', 'secret-key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'max-buckets': '1000'} 2023-10-05T19:43:18.753 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'ratelimit_user', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 1000, 'subusers': [], 'keys': [{'user': 'ratelimit_user', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': 'false', 'admin': 'false', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2023-10-05T19:43:18.753 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'user', 'uid': 'ratelimit_user'} 2023-10-05T19:43:18.758 INFO:tasks.radosgw_admin_rest: json result: {'user_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 0, 'enabled': False}} 2023-10-05T19:43:18.758 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'user', 'uid': 'ratelimit_userstring'} 2023-10-05T19:43:18.764 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchKey', 'RequestId': 'tx000001157bbba686d06d9-00651f11d6-107b-default', 'HostId': '107b-default-default'} 2023-10-05T19:43:18.764 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'user'} 2023-10-05T19:43:18.768 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'InvalidArgument', 'RequestId': 'tx00000c56522d6d0133077-00651f11d6-107b-default', 'HostId': '107b-default-default'} 2023-10-05T19:43:18.792 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'bucket', 'bucket': 'ratelimitbucket'} 2023-10-05T19:43:18.799 INFO:tasks.radosgw_admin_rest: json result: {'bucket_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 0, 'enabled': False}} 2023-10-05T19:43:18.800 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'bucket', 'bucket': 'ratelimitbucketstring'} 2023-10-05T19:43:18.805 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchKey', 'RequestId': 'tx000004124cf6baac940fd-00651f11d6-107b-default', 'HostId': '107b-default-default'} 2023-10-05T19:43:18.806 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'bucket'} 2023-10-05T19:43:18.810 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'InvalidArgument', 'RequestId': 'tx000007be83acd2739ef3e-00651f11d6-107b-default', 'HostId': '107b-default-default'} 2023-10-05T19:43:18.810 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'global': 'true'} 2023-10-05T19:43:18.815 INFO:tasks.radosgw_admin_rest: json result: {'bucket_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 0, 'enabled': False}, 'user_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 0, 'enabled': False}, 'anonymous_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 0, 'enabled': False}} 2023-10-05T19:43:18.815 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'user', 'uid': 'ratelimit_userstring', 'enabled': 'true'} 2023-10-05T19:43:18.819 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchKey', 'RequestId': 'tx000000c4507b65ee6ec7d-00651f11d6-107b-default', 'HostId': '107b-default-default'} 2023-10-05T19:43:18.819 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'user'} 2023-10-05T19:43:18.823 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'InvalidArgument', 'RequestId': 'tx0000097ee17b340f61bc5-00651f11d6-107b-default', 'HostId': '107b-default-default'} 2023-10-05T19:43:18.823 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'bucket', 'bucket': 'ratelimitbucketstring', 'enabled': 'true'} 2023-10-05T19:43:18.827 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchKey', 'RequestId': 'tx000003902688d36335f2d-00651f11d6-107b-default', 'HostId': '107b-default-default'} 2023-10-05T19:43:18.827 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'bucket', 'enabled': 'true'} 2023-10-05T19:43:18.831 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'InvalidArgument', 'RequestId': 'tx0000050c4ed6e9e6d86e2-00651f11d6-107b-default', 'HostId': '107b-default-default'} 2023-10-05T19:43:18.831 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'user', 'uid': 'ratelimit_user', 'enabled': 'true', 'max-read-bytes': '2'} 2023-10-05T19:43:18.852 INFO:tasks.radosgw_admin_rest: empty result 2023-10-05T19:43:18.852 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'user', 'uid': 'ratelimit_user'} 2023-10-05T19:43:18.857 INFO:tasks.radosgw_admin_rest: json result: {'user_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 2, 'max_write_bytes': 0, 'enabled': True}} 2023-10-05T19:43:18.857 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'bucket', 'bucket': 'ratelimitbucket', 'enabled': 'true', 'max-write-bytes': '2'} 2023-10-05T19:43:18.868 INFO:tasks.radosgw_admin_rest: empty result 2023-10-05T19:43:18.868 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'bucket', 'bucket': 'ratelimitbucket'} 2023-10-05T19:43:18.873 INFO:tasks.radosgw_admin_rest: json result: {'bucket_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 2, 'enabled': True}} 2023-10-05T19:43:18.873 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'bucket', 'global': 'true', 'enabled': 'true'} 2023-10-05T19:43:18.883 INFO:tasks.radosgw_admin_rest: empty result 2023-10-05T19:43:18.887 DEBUG:teuthology.run_tasks:Unwinding manager rgw 2023-10-05T19:43:18.904 DEBUG:tasks.rgw.client.0:waiting for process to exit 2023-10-05T19:43:18.904 INFO:teuthology.orchestra.run:waiting for 300 2023-10-05T19:43:18.905 INFO:tasks.rgw.client.0.smithi012.stdout:2023-10-05T19:43:18.899+0000 7f2f0bfff700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term radosgw --rgw-frontends beast port=80 -n client.0 --cluster ceph -k /etc/ceph/ceph.client.0.keyring --log-file /var/log/ceph/rgw.ceph.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock --foreground (PID: 34038) UID: 0 2023-10-05T19:43:18.906 INFO:tasks.rgw.client.0.smithi012.stdout:2023-10-05T19:43:18.899+0000 7f2f1cbabe40 -1 shutting down 2023-10-05T19:43:24.277 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~0s 2023-10-05T19:43:25.006 INFO:tasks.rgw.client.0:Stopped 2023-10-05T19:43:25.006 DEBUG:teuthology.orchestra.run.smithi012:> rm -f /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock 2023-10-05T19:43:25.014 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f /etc/ceph/vault-root-token 2023-10-05T19:43:25.030 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-05T19:43:25.045 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2023-10-05T19:43:25.045 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-05T19:43:25.161 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.157+0000 7fd7faf2c700 1 Processor -- start 2023-10-05T19:43:25.162 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.157+0000 7fd7faf2c700 1 -- start start 2023-10-05T19:43:25.162 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.157+0000 7fd7faf2c700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fd7f406dc60 0x7fd7f4132af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:43:25.162 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.157+0000 7fd7faf2c700 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7f406b850 con 0x7fd7f412dda0 2023-10-05T19:43:25.162 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.157+0000 7fd7faf2c700 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7f406b100 con 0x7fd7f406dc60 2023-10-05T19:43:25.162 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.157+0000 7fd7f9729700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fd7f406dc60 0x7fd7f4132af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:43:25.162 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.157+0000 7fd7f9729700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7fd7f406dc60 0x7fd7f4132af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35928/0 (socket says 172.21.15.8:35928) 2023-10-05T19:43:25.162 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.157+0000 7fd7f9729700 1 -- 172.21.15.8:0/2860080706 learned_addr learned my addr 172.21.15.8:0/2860080706 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-05T19:43:25.162 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.157+0000 7fd7f9f2a700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7fd7f412dda0 0x7fd7f412e170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:47800/0 (socket says 172.21.15.8:47800) 2023-10-05T19:43:25.164 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.157+0000 7fd7f9729700 1 -- 172.21.15.8:0/2860080706 >> v1:172.21.15.8:6789/0 conn(0x7fd7f412dda0 legacy=0x7fd7f412e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:43:25.164 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.157+0000 7fd7f9729700 1 -- 172.21.15.8:0/2860080706 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f406b540 con 0x7fd7f406dc60 2023-10-05T19:43:25.164 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.157+0000 7fd7f9729700 1 --2- 172.21.15.8:0/2860080706 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd7f406dc60 0x7fd7f4132af0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0009930 tx=0x7fd7f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dc15ead4d59efe4 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:43:25.164 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.157+0000 7fd7f8f28700 1 -- 172.21.15.8:0/2860080706 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7f000b860 con 0x7fd7f406dc60 2023-10-05T19:43:25.164 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.157+0000 7fd7f8f28700 1 -- 172.21.15.8:0/2860080706 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7f000b9c0 con 0x7fd7f406dc60 2023-10-05T19:43:25.164 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.157+0000 7fd7f8f28700 1 -- 172.21.15.8:0/2860080706 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7f000bb50 con 0x7fd7f406dc60 2023-10-05T19:43:25.164 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.161+0000 7fd7faf2c700 1 -- 172.21.15.8:0/2860080706 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd7f406dc60 msgr2=0x7fd7f4132af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:43:25.165 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.161+0000 7fd7faf2c700 1 --2- 172.21.15.8:0/2860080706 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd7f406dc60 0x7fd7f4132af0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0009930 tx=0x7fd7f0009250 comp rx=0 tx=0).stop 2023-10-05T19:43:25.165 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.161+0000 7fd7faf2c700 1 -- 172.21.15.8:0/2860080706 shutdown_connections 2023-10-05T19:43:25.165 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.161+0000 7fd7faf2c700 1 --2- 172.21.15.8:0/2860080706 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd7f406dc60 0x7fd7f4132af0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:43:25.165 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.161+0000 7fd7faf2c700 1 -- 172.21.15.8:0/2860080706 >> 172.21.15.8:0/2860080706 conn(0x7fd7f409a530 msgr2=0x7fd7f409a930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:43:25.165 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.161+0000 7fd7faf2c700 1 -- 172.21.15.8:0/2860080706 shutdown_connections 2023-10-05T19:43:25.165 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.161+0000 7fd7faf2c700 1 -- 172.21.15.8:0/2860080706 wait complete. 2023-10-05T19:43:25.167 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.161+0000 7fd7faf2c700 1 Processor -- start 2023-10-05T19:43:25.167 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.161+0000 7fd7faf2c700 1 -- start start 2023-10-05T19:43:25.167 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.161+0000 7fd7faf2c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd7f406dc60 0x7fd7f411d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:43:25.167 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.161+0000 7fd7faf2c700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f411f430 con 0x7fd7f406dc60 2023-10-05T19:43:25.167 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.161+0000 7fd7f9f2a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd7f406dc60 0x7fd7f411d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:43:25.167 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.161+0000 7fd7f9f2a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd7f406dc60 0x7fd7f411d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35932/0 (socket says 172.21.15.8:35932) 2023-10-05T19:43:25.167 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.161+0000 7fd7f9f2a700 1 -- 172.21.15.8:0/2253289220 learned_addr learned my addr 172.21.15.8:0/2253289220 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-05T19:43:25.167 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.161+0000 7fd7f9f2a700 1 -- 172.21.15.8:0/2253289220 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f00094f0 con 0x7fd7f406dc60 2023-10-05T19:43:25.167 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.161+0000 7fd7f9f2a700 1 --2- 172.21.15.8:0/2253289220 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd7f406dc60 0x7fd7f411d7b0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fd7e800c6c0 tx=0x7fd7e8007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:43:25.167 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.161+0000 7fd7e5ffb700 1 -- 172.21.15.8:0/2253289220 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7e8005a70 con 0x7fd7f406dc60 2023-10-05T19:43:25.168 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.161+0000 7fd7e5ffb700 1 -- 172.21.15.8:0/2253289220 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7e8005bd0 con 0x7fd7f406dc60 2023-10-05T19:43:25.168 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.161+0000 7fd7faf2c700 1 -- 172.21.15.8:0/2253289220 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7f40996d0 con 0x7fd7f406dc60 2023-10-05T19:43:25.168 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.161+0000 7fd7e5ffb700 1 -- 172.21.15.8:0/2253289220 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7e8005e80 con 0x7fd7f406dc60 2023-10-05T19:43:25.168 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.165+0000 7fd7faf2c700 1 -- 172.21.15.8:0/2253289220 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7f41209e0 con 0x7fd7f406dc60 2023-10-05T19:43:25.168 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.165+0000 7fd7faf2c700 1 -- 172.21.15.8:0/2253289220 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7f405ecb0 con 0x7fd7f406dc60 2023-10-05T19:43:25.168 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.165+0000 7fd7e5ffb700 1 -- 172.21.15.8:0/2253289220 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7fd7e8007070 con 0x7fd7f406dc60 2023-10-05T19:43:25.169 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.165+0000 7fd7e5ffb700 1 --2- 172.21.15.8:0/2253289220 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd7d8032390 0x7fd7d8034840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:43:25.169 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.165+0000 7fd7e5ffb700 1 -- 172.21.15.8:0/2253289220 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7fd7e8043840 con 0x7fd7f406dc60 2023-10-05T19:43:25.169 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.165+0000 7fd7f9729700 1 --2- 172.21.15.8:0/2253289220 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd7d8032390 0x7fd7d8034840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:43:25.170 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.165+0000 7fd7f9729700 1 --2- 172.21.15.8:0/2253289220 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd7d8032390 0x7fd7d8034840 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0009930 tx=0x7fd7f0016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:43:25.180 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.177+0000 7fd7e5ffb700 1 -- 172.21.15.8:0/2253289220 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7fd7e800e660 con 0x7fd7f406dc60 2023-10-05T19:43:25.356 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.353+0000 7fd7faf2c700 1 -- 172.21.15.8:0/2253289220 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7f404b5c0 con 0x7fd7f406dc60 2023-10-05T19:43:25.357 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.353+0000 7fd7e5ffb700 1 -- 172.21.15.8:0/2253289220 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+18634 (secure 0 0 0) 0x7fd7e8017020 con 0x7fd7f406dc60 2023-10-05T19:43:25.358 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-10-05T19:43:25.358 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":30,"fsid":"dac6c186-0634-45d9-839d-04f1e484462d","created":"2023-10-05T19:39:35.575770+0000","modified":"2023-10-05T19:40:41.595659+0000","last_up_change":"2023-10-05T19:39:44.877343+0000","last_in_change":"2023-10-05T19:39:36.620941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-05T19:39:45.900259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-05T19:39:51.178391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"default.rgw.buckets.data","create_time":"2023-10-05T19:39:56.725103+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":4,"pool_name":"default.rgw.buckets.index","create_time":"2023-10-05T19:39:58.964525+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":".rgw.root","create_time":"2023-10-05T19:40:00.787540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.log","create_time":"2023-10-05T19:40:02.774978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":7,"pool_name":"default.rgw.control","create_time":"2023-10-05T19:40:04.720208+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":8,"pool_name":"default.rgw.meta","create_time":"2023-10-05T19:40:06.880327+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"c23c6d99-5a1e-499d-8ef8-e1a58dcd9e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6800","nonce":27563},{"type":"v1","addr":"172.21.15.8:6801","nonce":27563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":27563},{"type":"v1","addr":"172.21.15.8:6803","nonce":27563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":27563},{"type":"v1","addr":"172.21.15.8:6807","nonce":27563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":27563},{"type":"v1","addr":"172.21.15.8:6805","nonce":27563}]},"public_addr":"172.21.15.8:6801/27563","cluster_addr":"172.21.15.8:6803/27563","heartbeat_back_addr":"172.21.15.8:6807/27563","heartbeat_front_addr":"172.21.15.8:6805/27563","state":["exists","up"]},{"osd":1,"uuid":"e578b376-27a7-47e2-b262-9a2a703d2504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":29933},{"type":"v1","addr":"172.21.15.12:6809","nonce":29933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":29933},{"type":"v1","addr":"172.21.15.12:6811","nonce":29933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":29933},{"type":"v1","addr":"172.21.15.12:6815","nonce":29933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":29933},{"type":"v1","addr":"172.21.15.12:6813","nonce":29933}]},"public_addr":"172.21.15.12:6809/29933","cluster_addr":"172.21.15.12:6811/29933","heartbeat_back_addr":"172.21.15.12:6815/29933","heartbeat_front_addr":"172.21.15.12:6813/29933","state":["exists","up"]},{"osd":2,"uuid":"f687bf28-3a5f-4cd8-a242-9b95067cc77a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":29934},{"type":"v1","addr":"172.21.15.12:6801","nonce":29934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":29934},{"type":"v1","addr":"172.21.15.12:6803","nonce":29934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":29934},{"type":"v1","addr":"172.21.15.12:6807","nonce":29934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":29934},{"type":"v1","addr":"172.21.15.12:6805","nonce":29934}]},"public_addr":"172.21.15.12:6801/29934","cluster_addr":"172.21.15.12:6803/29934","heartbeat_back_addr":"172.21.15.12:6807/29934","heartbeat_front_addr":"172.21.15.12:6805/29934","state":["exists","up"]},{"osd":3,"uuid":"db25fbc8-9f15-4a87-8310-fcc69bb92fa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":29951},{"type":"v1","addr":"172.21.15.12:6817","nonce":29951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":29951},{"type":"v1","addr":"172.21.15.12:6819","nonce":29951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":29951},{"type":"v1","addr":"172.21.15.12:6823","nonce":29951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":29951},{"type":"v1","addr":"172.21.15.12:6821","nonce":29951}]},"public_addr":"172.21.15.12:6817/29951","cluster_addr":"172.21.15.12:6819/29951","heartbeat_back_addr":"172.21.15.12:6823/29951","heartbeat_front_addr":"172.21.15.12:6821/29951","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-05T19:39:41.740161+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-05T19:39:42.128798+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-05T19:39:42.140705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-05T19:39:41.961354+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":1}]},{"pgid":"3.2c","mappings":[{"from":3,"to":0}]},{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.1","mappings":[{"from":3,"to":0}]},{"pgid":"4.5","mappings":[{"from":3,"to":2}]},{"pgid":"4.36","mappings":[{"from":1,"to":2}]},{"pgid":"5.2","mappings":[{"from":3,"to":2}]},{"pgid":"5.14","mappings":[{"from":0,"to":1}]},{"pgid":"5.1d","mappings":[{"from":0,"to":2}]},{"pgid":"5.1f","mappings":[{"from":0,"to":1}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-05T19:43:25.360 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.357+0000 7fd7faf2c700 1 -- 172.21.15.8:0/2253289220 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd7d8032390 msgr2=0x7fd7d8034840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:43:25.360 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.357+0000 7fd7faf2c700 1 --2- 172.21.15.8:0/2253289220 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd7d8032390 0x7fd7d8034840 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0009930 tx=0x7fd7f0016000 comp rx=0 tx=0).stop 2023-10-05T19:43:25.360 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.357+0000 7fd7faf2c700 1 -- 172.21.15.8:0/2253289220 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd7f406dc60 msgr2=0x7fd7f411d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:43:25.360 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.357+0000 7fd7faf2c700 1 --2- 172.21.15.8:0/2253289220 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd7f406dc60 0x7fd7f411d7b0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fd7e800c6c0 tx=0x7fd7e8007f70 comp rx=0 tx=0).stop 2023-10-05T19:43:25.361 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.357+0000 7fd7faf2c700 1 -- 172.21.15.8:0/2253289220 shutdown_connections 2023-10-05T19:43:25.361 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.357+0000 7fd7faf2c700 1 --2- 172.21.15.8:0/2253289220 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7fd7d8032390 0x7fd7d8034840 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:43:25.361 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.357+0000 7fd7faf2c700 1 --2- 172.21.15.8:0/2253289220 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd7f406dc60 0x7fd7f411d7b0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:43:25.361 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.357+0000 7fd7faf2c700 1 -- 172.21.15.8:0/2253289220 >> 172.21.15.8:0/2253289220 conn(0x7fd7f409a530 msgr2=0x7fd7f412e720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:43:25.361 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.357+0000 7fd7faf2c700 1 -- 172.21.15.8:0/2253289220 shutdown_connections 2023-10-05T19:43:25.361 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.357+0000 7fd7faf2c700 1 -- 172.21.15.8:0/2253289220 wait complete. 2023-10-05T19:43:25.372 DEBUG:teuthology.orchestra.run.smithi008:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-10-05T19:43:25.479 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.477+0000 7f844717f700 1 Processor -- start 2023-10-05T19:43:25.479 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.477+0000 7f844717f700 1 -- start start 2023-10-05T19:43:25.481 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.477+0000 7f844717f700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f844006d6e0 0x7f844012e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:43:25.481 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.477+0000 7f844717f700 1 -- --> v1:172.21.15.8:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f844006ba10 con 0x7f844006cc00 2023-10-05T19:43:25.481 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.477+0000 7f844717f700 1 -- --> v2:172.21.15.8:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f844006b460 con 0x7f844006d6e0 2023-10-05T19:43:25.482 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.477+0000 7f844597c700 1 --2- >> v2:172.21.15.8:3300/0 conn(0x7f844006d6e0 0x7f844012e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:43:25.482 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.477+0000 7f844617d700 1 --1- >> v1:172.21.15.8:6789/0 conn(0x7f844006cc00 0x7f844006cfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.8:6789/0 says I am v1:172.21.15.8:47808/0 (socket says 172.21.15.8:47808) 2023-10-05T19:43:25.482 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.477+0000 7f844617d700 1 -- 172.21.15.8:0/3955340620 learned_addr learned my addr 172.21.15.8:0/3955340620 (peer_addr_for_me v1:172.21.15.8:0/0) 2023-10-05T19:43:25.482 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.477+0000 7f844597c700 1 -- 172.21.15.8:0/3955340620 >> v1:172.21.15.8:6789/0 conn(0x7f844006cc00 legacy=0x7f844006cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:43:25.482 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.477+0000 7f844517b700 1 -- 172.21.15.8:0/3955340620 <== mon.0 v1:172.21.15.8:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4034509470 0 0) 0x7f844006ba10 con 0x7f844006cc00 2023-10-05T19:43:25.482 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.477+0000 7f844597c700 1 -- 172.21.15.8:0/3955340620 --> v2:172.21.15.8:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f844006dae0 con 0x7f844006d6e0 2023-10-05T19:43:25.483 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.477+0000 7f844597c700 1 --2- 172.21.15.8:0/3955340620 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f844006d6e0 0x7f844012e560 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f843c009890 tx=0x7f843c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddeb50e4906e0552 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:43:25.483 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.477+0000 7f844517b700 1 -- 172.21.15.8:0/3955340620 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f843c00b670 con 0x7f844006d6e0 2023-10-05T19:43:25.483 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.477+0000 7f844517b700 1 -- 172.21.15.8:0/3955340620 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f843c00b7d0 con 0x7f844006d6e0 2023-10-05T19:43:25.483 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.477+0000 7f844517b700 1 -- 172.21.15.8:0/3955340620 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f843c00bac0 con 0x7f844006d6e0 2023-10-05T19:43:25.485 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.477+0000 7f844717f700 1 -- 172.21.15.8:0/3955340620 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f844006d6e0 msgr2=0x7f844012e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:43:25.486 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.477+0000 7f844717f700 1 --2- 172.21.15.8:0/3955340620 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f844006d6e0 0x7f844012e560 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f843c009890 tx=0x7f843c0091a0 comp rx=0 tx=0).stop 2023-10-05T19:43:25.486 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.477+0000 7f844717f700 1 -- 172.21.15.8:0/3955340620 shutdown_connections 2023-10-05T19:43:25.486 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.477+0000 7f844717f700 1 --2- 172.21.15.8:0/3955340620 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f844006d6e0 0x7f844012e560 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:43:25.486 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.477+0000 7f844717f700 1 -- 172.21.15.8:0/3955340620 >> 172.21.15.8:0/3955340620 conn(0x7f844009a5a0 msgr2=0x7f844009a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:43:25.486 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.477+0000 7f844717f700 1 -- 172.21.15.8:0/3955340620 shutdown_connections 2023-10-05T19:43:25.486 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.477+0000 7f844717f700 1 -- 172.21.15.8:0/3955340620 wait complete. 2023-10-05T19:43:25.486 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.481+0000 7f844717f700 1 Processor -- start 2023-10-05T19:43:25.486 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.481+0000 7f844717f700 1 -- start start 2023-10-05T19:43:25.486 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.481+0000 7f844717f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f844006cc00 0x7f8440138580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:43:25.486 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.481+0000 7f844717f700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f844011e310 con 0x7f844006cc00 2023-10-05T19:43:25.487 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.481+0000 7f844617d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f844006cc00 0x7f8440138580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:43:25.487 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.481+0000 7f844617d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f844006cc00 0x7f8440138580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35956/0 (socket says 172.21.15.8:35956) 2023-10-05T19:43:25.487 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.481+0000 7f844617d700 1 -- 172.21.15.8:0/2790411308 learned_addr learned my addr 172.21.15.8:0/2790411308 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-10-05T19:43:25.487 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.481+0000 7f844617d700 1 -- 172.21.15.8:0/2790411308 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f843c009420 con 0x7f844006cc00 2023-10-05T19:43:25.487 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.481+0000 7f844617d700 1 --2- 172.21.15.8:0/2790411308 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f844006cc00 0x7f8440138580 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f843000c460 tx=0x7f8430000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:43:25.487 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.481+0000 7f84367fc700 1 -- 172.21.15.8:0/2790411308 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8430005b30 con 0x7f844006cc00 2023-10-05T19:43:25.487 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.481+0000 7f84367fc700 1 -- 172.21.15.8:0/2790411308 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8430005c90 con 0x7f844006cc00 2023-10-05T19:43:25.487 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.481+0000 7f844717f700 1 -- 172.21.15.8:0/2790411308 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84400976f0 con 0x7f844006cc00 2023-10-05T19:43:25.487 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.481+0000 7f84367fc700 1 -- 172.21.15.8:0/2790411308 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84300070d0 con 0x7f844006cc00 2023-10-05T19:43:25.487 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.481+0000 7f844717f700 1 -- 172.21.15.8:0/2790411308 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f844011f330 con 0x7f844006cc00 2023-10-05T19:43:25.488 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.481+0000 7f84367fc700 1 -- 172.21.15.8:0/2790411308 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 40329+0+0 (secure 0 0 0) 0x7f8430007230 con 0x7f844006cc00 2023-10-05T19:43:25.488 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.481+0000 7f84367fc700 1 --2- 172.21.15.8:0/2790411308 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f84240323d0 0x7f8424034880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-05T19:43:25.489 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.481+0000 7f84367fc700 1 -- 172.21.15.8:0/2790411308 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5992+0+0 (secure 0 0 0) 0x7f8430043810 con 0x7f844006cc00 2023-10-05T19:43:25.489 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.485+0000 7f844717f700 1 -- 172.21.15.8:0/2790411308 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84280050b0 con 0x7f844006cc00 2023-10-05T19:43:25.489 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.485+0000 7f844597c700 1 --2- 172.21.15.8:0/2790411308 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f84240323d0 0x7f8424034880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-05T19:43:25.489 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.485+0000 7f844597c700 1 --2- 172.21.15.8:0/2790411308 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f84240323d0 0x7f8424034880 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f843c000c00 tx=0x7f843c016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-05T19:43:25.492 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.489+0000 7f84367fc700 1 -- 172.21.15.8:0/2790411308 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166807 (secure 0 0 0) 0x7f843000f6d0 con 0x7f844006cc00 2023-10-05T19:43:25.658 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.653+0000 7f844717f700 1 -- 172.21.15.8:0/2790411308 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f84280058e0 con 0x7f844006cc00 2023-10-05T19:43:25.661 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.657+0000 7f84367fc700 1 -- 172.21.15.8:0/2790411308 <== mon.0 v2:172.21.15.8:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f8430011e40 con 0x7f844006cc00 2023-10-05T19:43:25.661 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.657+0000 7f84367fc700 1 -- 172.21.15.8:0/2790411308 <== mon.0 v2:172.21.15.8:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v6)=0 v6) v1 ==== 123+0+0 (secure 0 0 0) 0x7f8430017020 con 0x7f844006cc00 2023-10-05T19:43:25.664 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.661+0000 7f844717f700 1 -- 172.21.15.8:0/2790411308 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f84240323d0 msgr2=0x7f8424034880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:43:25.664 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.661+0000 7f844717f700 1 --2- 172.21.15.8:0/2790411308 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f84240323d0 0x7f8424034880 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f843c000c00 tx=0x7f843c016000 comp rx=0 tx=0).stop 2023-10-05T19:43:25.664 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.661+0000 7f844717f700 1 -- 172.21.15.8:0/2790411308 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f844006cc00 msgr2=0x7f8440138580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-05T19:43:25.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.661+0000 7f844717f700 1 --2- 172.21.15.8:0/2790411308 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f844006cc00 0x7f8440138580 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f843000c460 tx=0x7f8430000fe0 comp rx=0 tx=0).stop 2023-10-05T19:43:25.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.661+0000 7f844717f700 1 -- 172.21.15.8:0/2790411308 shutdown_connections 2023-10-05T19:43:25.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.661+0000 7f844717f700 1 --2- 172.21.15.8:0/2790411308 >> [v2:172.21.15.12:6824/29788,v1:172.21.15.12:6825/29788] conn(0x7f84240323d0 0x7f8424034880 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:43:25.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.661+0000 7f844717f700 1 --2- 172.21.15.8:0/2790411308 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f844006cc00 0x7f8440138580 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-05T19:43:25.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.661+0000 7f844717f700 1 -- 172.21.15.8:0/2790411308 >> 172.21.15.8:0/2790411308 conn(0x7f844009a5a0 msgr2=0x7f8440115010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-05T19:43:25.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.661+0000 7f844717f700 1 -- 172.21.15.8:0/2790411308 shutdown_connections 2023-10-05T19:43:25.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-10-05T19:43:25.661+0000 7f844717f700 1 -- 172.21.15.8:0/2790411308 wait complete. 2023-10-05T19:43:25.676 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-05T19:43:25.676 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-05T19:43:25.676 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-05T19:43:25.676 INFO:teuthology.orchestra.run:waiting for 300 2023-10-05T19:43:25.753 INFO:tasks.ceph.osd.0:Stopped 2023-10-05T19:43:25.753 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-05T19:43:25.753 INFO:teuthology.orchestra.run:waiting for 300 2023-10-05T19:43:25.836 INFO:tasks.ceph.osd.1:Stopped 2023-10-05T19:43:25.837 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-05T19:43:25.837 INFO:teuthology.orchestra.run:waiting for 300 2023-10-05T19:43:25.919 INFO:tasks.ceph.osd.2:Stopped 2023-10-05T19:43:25.919 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-05T19:43:25.919 INFO:teuthology.orchestra.run:waiting for 300 2023-10-05T19:43:25.998 INFO:tasks.ceph.osd.3:Stopped 2023-10-05T19:43:25.999 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-05T19:43:25.999 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-05T19:43:25.999 INFO:teuthology.orchestra.run:waiting for 300 2023-10-05T19:43:26.025 INFO:tasks.ceph.mgr.x:Stopped 2023-10-05T19:43:26.025 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-05T19:43:26.025 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-05T19:43:26.026 INFO:teuthology.orchestra.run:waiting for 300 2023-10-05T19:43:26.045 INFO:tasks.ceph.mon.a:Stopped 2023-10-05T19:43:26.045 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-05T19:43:26.045 DEBUG:teuthology.orchestra.run.smithi008:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2023-10-05T19:43:26.064 INFO:teuthology.orchestra.run.smithi008.stdout:1696534792.5604463 mon.a (mon.0) 113 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-05T19:43:26.065 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-10-05T19:43:26.065 DEBUG:teuthology.orchestra.run.smithi008:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2023-10-05T19:43:26.081 DEBUG:teuthology.orchestra.run.smithi008:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2023-10-05T19:43:26.143 DEBUG:teuthology.orchestra.run.smithi008:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2023-10-05T19:43:26.161 INFO:teuthology.orchestra.run.smithi008.stdout:1696534792.5604463 mon.a (mon.0) 113 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-05T19:43:26.162 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi008.front.sepia.ceph.com 2023-10-05T19:43:26.162 DEBUG:teuthology.orchestra.run.smithi008:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-05T19:43:26.382 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi012.front.sepia.ceph.com 2023-10-05T19:43:26.382 DEBUG:teuthology.orchestra.run.smithi012:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-05T19:43:26.654 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi012.front.sepia.ceph.com 2023-10-05T19:43:26.654 DEBUG:teuthology.orchestra.run.smithi012:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-05T19:43:26.810 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi012.front.sepia.ceph.com 2023-10-05T19:43:26.810 DEBUG:teuthology.orchestra.run.smithi012:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-05T19:43:26.951 INFO:tasks.ceph:Archiving mon data... 2023-10-05T19:43:26.952 DEBUG:teuthology.misc:Transferring archived files from smithi008:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-10-05_14:30:58-rgw-wip-yuri2-testing-2023-10-04-1333-quincy-distro-default-smithi/7411759/data/mon.a.tgz 2023-10-05T19:43:26.952 DEBUG:teuthology.orchestra.run.smithi008:> mktemp 2023-10-05T19:43:26.959 INFO:teuthology.orchestra.run.smithi008.stdout:/tmp/tmp.8xCZ4rkVZD 2023-10-05T19:43:26.959 DEBUG:teuthology.orchestra.run.smithi008:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.8xCZ4rkVZD 2023-10-05T19:43:27.073 DEBUG:teuthology.orchestra.run.smithi008:> sudo chmod 0666 /tmp/tmp.8xCZ4rkVZD 2023-10-05T19:43:27.093 DEBUG:teuthology.orchestra.remote:smithi008:/tmp/tmp.8xCZ4rkVZD is 212KB 2023-10-05T19:43:27.153 DEBUG:teuthology.orchestra.run.smithi008:> rm -fr /tmp/tmp.8xCZ4rkVZD 2023-10-05T19:43:27.160 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-05T19:43:27.160 DEBUG:teuthology.orchestra.run.smithi008:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-05T19:43:27.162 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-05T19:43:27.179 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-05T19:43:27.179 DEBUG:teuthology.orchestra.run.smithi008:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-05T19:43:27.181 DEBUG:teuthology.orchestra.run.smithi012:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-05T19:43:27.207 INFO:teuthology.orchestra.run.smithi008.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-05T19:43:27.245 INFO:teuthology.orchestra.run.smithi012.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-05T19:43:27.251 INFO:tasks.ceph:Archiving crash dumps... 2023-10-05T19:43:27.252 DEBUG:teuthology.misc:Transferring archived files from smithi008:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-05_14:30:58-rgw-wip-yuri2-testing-2023-10-04-1333-quincy-distro-default-smithi/7411759/remote/smithi008/crash 2023-10-05T19:43:27.253 DEBUG:teuthology.orchestra.run.smithi008:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-05T19:43:27.270 DEBUG:teuthology.misc:Transferring archived files from smithi012:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-05_14:30:58-rgw-wip-yuri2-testing-2023-10-04-1333-quincy-distro-default-smithi/7411759/remote/smithi012/crash 2023-10-05T19:43:27.271 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-05T19:43:27.286 INFO:tasks.ceph:Compressing logs... 2023-10-05T19:43:27.286 DEBUG:teuthology.orchestra.run.smithi008:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-05T19:43:27.289 DEBUG:teuthology.orchestra.run.smithi012:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-05T19:43:29.278 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 has been restored 2023-10-05T19:43:29.975 INFO:tasks.ceph:Archiving logs... 2023-10-05T19:43:29.975 DEBUG:teuthology.misc:Transferring archived files from smithi008:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-05_14:30:58-rgw-wip-yuri2-testing-2023-10-04-1333-quincy-distro-default-smithi/7411759/remote/smithi008/log 2023-10-05T19:43:29.976 DEBUG:teuthology.orchestra.run.smithi008:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-05T19:43:30.153 DEBUG:teuthology.misc:Transferring archived files from smithi012:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-05_14:30:58-rgw-wip-yuri2-testing-2023-10-04-1333-quincy-distro-default-smithi/7411759/remote/smithi012/log 2023-10-05T19:43:30.153 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-05T19:43:30.536 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-05T19:43:30.547 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-05T19:43:30.547 DEBUG:teuthology.orchestra.run.smithi008:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-05T19:43:30.549 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-05T19:43:30.611 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-05T19:43:30.621 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-05T19:43:30.621 DEBUG:teuthology.orchestra.run.smithi008:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-05T19:43:30.624 DEBUG:teuthology.orchestra.run.smithi012:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-05T19:43:30.642 INFO:teuthology.orchestra.run.smithi008.stdout: remote refid st t when poll reach delay offset jitter 2023-10-05T19:43:30.643 INFO:teuthology.orchestra.run.smithi008.stdout:============================================================================== 2023-10-05T19:43:30.643 INFO:teuthology.orchestra.run.smithi008.stdout:*hv01.front.sepi 44.4.53.2 2 u 41 64 77 0.139 1.914 0.777 2023-10-05T19:43:30.643 INFO:teuthology.orchestra.run.smithi008.stdout:+hv02.front.sepi 63.231.80.2 3 u 40 64 77 0.122 1.268 0.705 2023-10-05T19:43:30.643 INFO:teuthology.orchestra.run.smithi008.stdout:+hv03.front.sepi 74.6.168.72 3 u 42 64 77 0.130 -0.286 0.802 2023-10-05T19:43:30.643 INFO:teuthology.orchestra.run.smithi008.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-05T19:43:30.646 INFO:teuthology.orchestra.run.smithi012.stdout: remote refid st t when poll reach delay offset jitter 2023-10-05T19:43:30.646 INFO:teuthology.orchestra.run.smithi012.stdout:============================================================================== 2023-10-05T19:43:30.646 INFO:teuthology.orchestra.run.smithi012.stdout:*hv01.front.sepi 44.4.53.2 2 u 43 64 77 0.113 0.952 1.700 2023-10-05T19:43:30.646 INFO:teuthology.orchestra.run.smithi012.stdout:+hv02.front.sepi 63.231.80.2 3 u 41 64 77 0.063 0.890 1.036 2023-10-05T19:43:30.646 INFO:teuthology.orchestra.run.smithi012.stdout:+hv03.front.sepi 74.6.168.72 3 u 41 64 77 0.074 -0.745 1.107 2023-10-05T19:43:30.646 INFO:teuthology.orchestra.run.smithi012.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-05T19:43:30.647 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-05T19:43:30.656 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-05T19:43:30.657 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-05T19:43:30.665 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-05T19:43:30.674 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-05T19:43:30.682 INFO:teuthology.task.internal:Duration was 726.928609 seconds 2023-10-05T19:43:30.683 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-05T19:43:30.691 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-05T19:43:30.691 DEBUG:teuthology.orchestra.run.smithi008:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-05T19:43:30.694 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-05T19:43:30.746 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-05T19:43:30.746 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi008.front.sepia.ceph.com 2023-10-05T19:43:30.747 DEBUG:teuthology.orchestra.run.smithi008:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-05T19:43:30.760 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi012.front.sepia.ceph.com 2023-10-05T19:43:30.760 DEBUG:teuthology.orchestra.run.smithi012:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-05T19:43:30.774 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-05T19:43:30.774 DEBUG:teuthology.orchestra.run.smithi008:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-05T19:43:30.776 DEBUG:teuthology.orchestra.run.smithi012:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-05T19:43:30.808 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-05T19:43:30.808 DEBUG:teuthology.orchestra.run.smithi008:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-05T19:43:30.853 DEBUG:teuthology.orchestra.run.smithi012:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-05T19:43:30.918 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-05T19:43:30.928 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-05T19:43:30.928 DEBUG:teuthology.orchestra.run.smithi008:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-05T19:43:30.945 DEBUG:teuthology.orchestra.run.smithi012:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-05T19:43:30.963 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-05T19:43:30.973 DEBUG:teuthology.orchestra.run.smithi008:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-05T19:43:30.988 DEBUG:teuthology.orchestra.run.smithi012:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-05T19:43:31.001 INFO:teuthology.orchestra.run.smithi008.stdout:kernel.core_pattern = core 2023-10-05T19:43:31.003 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern = core 2023-10-05T19:43:31.029 DEBUG:teuthology.orchestra.run.smithi008:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-05T19:43:31.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-05T19:43:31.035 DEBUG:teuthology.orchestra.run.smithi012:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-05T19:43:31.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-05T19:43:31.081 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-05T19:43:31.092 INFO:teuthology.task.internal:Transferring archived files... 2023-10-05T19:43:31.093 DEBUG:teuthology.misc:Transferring archived files from smithi008:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-05_14:30:58-rgw-wip-yuri2-testing-2023-10-04-1333-quincy-distro-default-smithi/7411759/remote/smithi008 2023-10-05T19:43:31.093 DEBUG:teuthology.orchestra.run.smithi008:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-05T19:43:31.116 DEBUG:teuthology.misc:Transferring archived files from smithi012:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-05_14:30:58-rgw-wip-yuri2-testing-2023-10-04-1333-quincy-distro-default-smithi/7411759/remote/smithi012 2023-10-05T19:43:31.116 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-05T19:43:31.137 INFO:teuthology.task.internal:Removing archive directory... 2023-10-05T19:43:31.137 DEBUG:teuthology.orchestra.run.smithi008:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-05T19:43:31.156 DEBUG:teuthology.orchestra.run.smithi012:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-05T19:43:31.163 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-05T19:43:31.248 INFO:teuthology.task.internal:Not uploading archives. 2023-10-05T19:43:31.249 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-05T19:43:31.258 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-05T19:43:31.258 DEBUG:teuthology.orchestra.run.smithi008:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-05T19:43:31.260 DEBUG:teuthology.orchestra.run.smithi012:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-05T19:43:31.266 INFO:teuthology.orchestra.run.smithi008.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 5 19:43 /home/ubuntu/cephtest 2023-10-05T19:43:31.269 INFO:teuthology.orchestra.run.smithi012.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 5 19:43 /home/ubuntu/cephtest 2023-10-05T19:43:31.271 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-05T19:43:31.281 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-05T19:43:31.357 INFO:teuthology.nuke:Checking targets against current locks 2023-10-05T19:43:31.384 DEBUG:teuthology.nuke:shortname: smithi008 2023-10-05T19:43:31.384 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-05T19:43:31.406 DEBUG:teuthology.nuke:shortname: smithi012 2023-10-05T19:43:31.406 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-05T19:43:31.411 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi008.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_14:30:58-rgw-wip-yuri2-testing-2023-10-04-1333-quincy-distro-default-smithi/7411759', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-05 19:22:29.289770', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='} 2023-10-05T19:43:31.428 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi012.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_14:30:58-rgw-wip-yuri2-testing-2023-10-04-1333-quincy-distro-default-smithi/7411759', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-05 19:22:29.291659', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='} 2023-10-05T19:43:31.454 INFO:teuthology.orchestra.console.smithi008:Power off 2023-10-05T19:43:31.454 DEBUG:teuthology.orchestra.console.smithi008:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-05T19:43:31.474 INFO:teuthology.orchestra.console.smithi012:Power off 2023-10-05T19:43:31.475 DEBUG:teuthology.orchestra.console.smithi012:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-05T19:43:31.484 DEBUG:teuthology.orchestra.console.smithi008:power off output: Chassis Power Control: Down/Off 2023-10-05T19:43:31.485 DEBUG:teuthology.orchestra.console.smithi008:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-05T19:43:31.499 DEBUG:teuthology.orchestra.console.smithi012:power off output: Chassis Power Control: Down/Off 2023-10-05T19:43:31.499 DEBUG:teuthology.orchestra.console.smithi012:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-05T19:43:31.512 DEBUG:teuthology.orchestra.console.smithi008:check power output: Chassis Power is on 2023-10-05T19:43:31.522 DEBUG:teuthology.orchestra.console.smithi012:check power output: Chassis Power is on 2023-10-05T19:43:35.514 DEBUG:teuthology.orchestra.console.smithi008:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-05T19:43:35.523 DEBUG:teuthology.orchestra.console.smithi012:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-05T19:43:35.625 DEBUG:teuthology.orchestra.console.smithi008:check power output: Chassis Power is on 2023-10-05T19:43:35.634 DEBUG:teuthology.orchestra.console.smithi012:check power output: Chassis Power is on 2023-10-05T19:43:39.626 DEBUG:teuthology.orchestra.console.smithi008:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-05T19:43:39.635 DEBUG:teuthology.orchestra.console.smithi012:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-05T19:43:39.735 DEBUG:teuthology.orchestra.console.smithi008:check power output: Chassis Power is off 2023-10-05T19:43:39.744 DEBUG:teuthology.orchestra.console.smithi012:check power output: Chassis Power is on 2023-10-05T19:43:39.837 INFO:teuthology.orchestra.console.smithi008:Power off completed 2023-10-05T19:43:43.747 DEBUG:teuthology.orchestra.console.smithi012:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-05T19:43:43.857 DEBUG:teuthology.orchestra.console.smithi012:check power output: Chassis Power is off 2023-10-05T19:43:43.958 INFO:teuthology.orchestra.console.smithi012:Power off completed 2023-10-05T19:43:44.062 INFO:teuthology.run:Summary data: description: rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated supported-random-distro$/{ubuntu_latest}} duration: 726.9286088943481 failure_reason: '"1696534792.5604463 mon.a (mon.0) 113 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log' flavor: default owner: scheduled_yuriw@teuthology success: false 2023-10-05T19:43:44.062 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-05T19:43:44.134 INFO:teuthology.run:FAIL